Air-ground collaborative inspection method based on heterogeneous platform

Abstract

The invention discloses an air-ground collaborative inspection method based on a heterogeneous platform, belongs to the technical field of intelligent inspection, and is suitable for multi-scene inspection of cities, electric power, maritime waters and the like. In order to solve the problems of view limitation, review deficiency, inaccurate positioning, lack of self-evolution ability and the like of traditional single platform inspection, the method constructs a full-link mechanism of discovery-induction-review-confirmation-learning optimization through functional complementation of air and ground equipment. The method comprises the core steps that a front end collects original data such as defect images and category labels, an industry term knowledge base is called, a structured document is formed through a template engine, an inspection report with a traceability identifier is output, and the credibility is guaranteed with the assistance of block chain evidence storage. Meanwhile, a heterogeneous node cooperation mechanism and a three-dimensional coordinate conversion model are established, cross-platform precise cooperation is achieved, closed-loop recheck and self-evolution optimization are combined, the inspection efficiency and precision are considered, false alarms are reduced, and inspection intelligence and credibility are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection technology, specifically to an air-ground collaborative inspection method based on a heterogeneous platform. Background Technology

[0002] With the development of unmanned systems technology, unmanned inspection has been widely applied in various scenarios. Traditional unmanned inspection systems typically rely on a single platform to perform tasks independently, but they have many shortcomings in practical applications: 1. Limitations of a single perspective: Some suspected targets have low confidence levels at long distances, in complex terrain, or under obstructed conditions, which can easily lead to false detections or missed detections, making it difficult to fully and accurately identify the true state of the inspected targets.

[0003] 2. Inability to verify across platforms: Data captured by a single platform lacks high-resolution compensation or verification from different source sensors, making it difficult to form a closed-loop evidence chain, resulting in insufficient credibility of the inspection results.

[0004] 3. Insufficient positioning accuracy: Single-node positioning relies heavily on its onboard camera, and the coordinates of the same target often have large errors, which cannot meet the needs of accurate inspection and subsequent processing.

[0005] 4. The system lacks learning ability: the inspection strategy is difficult to optimize dynamically, and there are problems such as repeated false alarms and inefficient path planning, resulting in low inspection efficiency and low level of intelligence.

[0006] Current air-to-ground inspection systems face challenges in coordinating heterogeneous nodes. Airborne and ground equipment operate independently, lacking standardized task triggering and information transmission mechanisms, making it difficult to achieve coordinated initial screening and verification. Furthermore, existing drones and ground robots lack unified data formats, spatiotemporal references, and task interfaces, hindering cross-platform collaborative detection. Traditional detection methods only output target pixel positions, failing to meet the verification needs of heterogeneous nodes. Moreover, the system lacks feedback mechanisms to optimize front-end detection thresholds and inspection paths, leading to repeated false alarms and further limiting the effectiveness of the inspection system. Summary of the Invention

[0007] To address the aforementioned problems, this invention proposes an air-ground collaborative inspection method based on a heterogeneous platform. This method leverages the complementary functional characteristics of airborne and ground-based equipment to establish a task relay mechanism triggered by low-cost wide-area perception for high-precision local verification. This mechanism enables efficient detection, accurate confirmation, and reliable reporting of suspected targets. Furthermore, it continuously improves inspection efficiency through system-level feedback optimization.

[0008] To solve the above-mentioned technical problems, the technical solution proposed by this invention is: an air-ground collaborative inspection method based on heterogeneous platforms, comprising the following steps: Step 1: Obtain raw inspection data through the front-end acquisition terminal. The raw inspection data includes at least the defect target image, defect category label, confidence level, and corresponding spatiotemporal metadata. The front-end acquisition terminal may include heterogeneous inspection equipment such as drones and ground robots. The spatiotemporal metadata includes key information such as inspection time and equipment location coordinates, providing basic data support for subsequent report generation and target traceability.

[0009] Step 2: Construct and maintain an industry terminology knowledge base. The knowledge base includes a defect category-industry terminology mapping table, defect level evaluation rules, causal relationship model, and standard report paragraph templates, and is stored in the form of Extensible Markup Language or graph database. The industry terminology knowledge base further includes a thesaurus and domain ontology to support the normalization of synonymous defect descriptions and hierarchical concept reasoning, ensuring that defects with different expressions can be uniformly identified and processed, and improving industry adaptability.

[0010] Step 3: Using the defect category label as input, the natural language processing engine performs semantic retrieval and alignment in the industry terminology knowledge base to generate standardized descriptive text that conforms to industry norms, and automatically determines the defect level according to the defect level evaluation rules. The natural language processing engine adopts a pre-trained language model based on Transformer and is fine-tuned through domain corpus to improve the accuracy of professional terminology generation and ensure that the generated descriptive text conforms to industry professional expression habits.

[0011] Step 4: Based on the pre-built report template engine, the standardized descriptive text, defect level, spatiotemporal metadata, and associated images are dynamically filled into the corresponding placeholders in the template to form a structured intermediate document. When multiple defect records exist for the same device, the template engine automatically merges similar defects and generates summary paragraphs to avoid repetitive descriptions and improve the conciseness and readability of the report. The report template engine supports user-defined templates and uses a version control mechanism to achieve template iteration and backtracking, meeting the report format requirements of different industries and scenarios.

[0012] Step 5: Output the structured intermediate documents in batches as Word and / or PDF inspection reports that meet the format requirements through the document conversion interface, and support the automatic attachment of digital signatures, QR codes and watermarks to complete the traceable release of the reports; digital signatures are used to ensure the authenticity of the signatory's identity, QR codes can be linked to the original inspection data and related details, and watermarks are used to identify the report's ownership and confidentiality level.

[0013] After step five, the generated inspection report hash value is further written into the blockchain for evidence storage. By utilizing the decentralized and tamper-proof characteristics of the blockchain, the integrity and non-repudiation of the report are achieved, ensuring the legal validity and credibility of the inspection results.

[0014] In addition, the present invention also includes the construction steps of a heterogeneous node collaboration mechanism: establishing a coordinate sharing protocol, a clock synchronization mechanism and a standardized task transmission interface between heterogeneous nodes to achieve efficient collaborative scheduling of multiple types of inspection nodes and solve the problem of independent operation and difficulty in collaboration between air and ground equipment.

[0015] Meanwhile, the present invention also includes a three-dimensional coordinate transformation step for pixel-level detection results: based on the real-time pose data of the inspection node and the sensor intrinsic and extrinsic parameters, a mapping model between pixel coordinates and three-dimensional spatial coordinates is established to realize the accurate inverse solution of pixel-level detection results to cross-platform shared three-dimensional target coordinates, meet the verification requirements of heterogeneous nodes, and improve positioning accuracy.

[0016] In addition, the present invention also includes a system self-evolution optimization step: the platform dynamically optimizes the front-end detection threshold and inspection path based on historical inspection results, reduces the recurrence of similar false alarms, and continuously improves the system's intelligence level and inspection efficiency.

[0017] The advantages of this invention compared to the prior art are: The complementary and synergistic effect of air and ground-based heterogeneous systems is outstanding, effectively breaking through the limitations of single-platform inspections. Aerial equipment, leveraging its wide-area coverage, quickly completes initial screening of suspected targets, significantly improving inspection efficiency over large areas; ground equipment focuses on close-range, precise verification, compensating for the lack of detail in long-distance aerial photography. The complementary and compatible functions of both systems balance inspection efficiency and defect identification accuracy, addressing the pain points of low efficiency or insufficient accuracy of traditional single-platform systems, and significantly improving overall inspection effectiveness.

[0018] A unified coordinate system and 3D transformation technology significantly enhance positioning reliability, providing solid support for multi-node collaboration. Cross-platform coordinate sharing achieves unified target 3D coordinates, avoiding the errors of traditional single-node positioning. A mapping model is constructed using real-time data to accurately resolve the target's 3D position. This ensures consistent target location perception between ground and air equipment, guaranteeing accurate ground-based verification positioning and smooth multi-node collaboration.

[0019] The dual empowerment of closed-loop verification and self-evolutionary optimization effectively reduces the false alarm rate and continuously improves the system's intelligence level. The closed-loop mechanism requires suspected targets to be confirmed by both ground and air, filtering false alarms and reducing the cost of invalid processing from the process perspective; the system accumulates historical inspection data, dynamically optimizes detection thresholds and inspection paths, forming a virtuous cycle of increasing accuracy with each inspection, and steadily improving intelligent operation capabilities.

[0020] The combination of standardized report generation and blockchain-based evidence storage comprehensively ensures the credibility and traceability of inspection results. It automatically generates standardized reports that conform to industry standards, avoiding the pitfalls of manual drafting, and supports multi-format output and the addition of traceability identifiers. The blockchain-stored report hash value, with its tamper-proof nature, ensures content integrity and meets the stringent requirements of industry regulators and auditors for the credibility and traceability of results. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the principle of an air-ground collaborative inspection method based on a heterogeneous platform according to the present invention. Detailed Implementation

[0022] The present invention will now be described in further detail with reference to the accompanying drawings.

[0023] Example 1 This embodiment is applied to a power line inspection scenario, using a drone as an aerial inspection node and a ground robot as a ground inspection node to perform power line inspection tasks. The specific steps are as follows: 1. Raw data collection during inspection: The drone carries a high-definition camera and positioning module to take wide-area pictures of the power transmission line according to the preset inspection path, collect image data of the power transmission line, detect suspected defects through image recognition algorithm, and generate defect target images, defect category labels, confidence scores and corresponding spatiotemporal metadata; the ground robot is on standby at the same time and receives the collaborative task trigger signal sent by the drone.

[0024] 2. Construction and maintenance of industry terminology knowledge base: Construct a power inspection industry terminology knowledge base, including a defect category-power industry terminology mapping table, defect level evaluation rules, causal relationship model and standard report paragraph templates, as well as a power industry thesaurus and domain ontology, stored in Extensible Markup Language format, and regularly updated according to power industry standards.

[0025] 3. Standardized Description Text Generation and Defect Level Determination: Taking the defect category labels detected by the UAV as input, the natural language processing engine adopts the BERT pre-trained model based on Transformer. Through fine-tuning of the corpus in the power inspection field, semantic retrieval and alignment are performed in the industry terminology knowledge base to generate standardized description text. Based on the defect level evaluation rules and combined with the specific parameters of the defect, the defect level is automatically determined.

[0026] 4. Structured intermediate document generation: Based on the pre-built power inspection report template engine, the standardized descriptive text, defect level, spatiotemporal metadata and corresponding defect target image are dynamically filled into the corresponding placeholders in the template; if multiple records of the same type of defect are detected in the transmission line section, the template engine automatically merges and generates summary paragraphs to form a structured intermediate document.

[0027] 5. Inspection Report Output and Traceability Identification: Through the document conversion interface, the structured intermediate documents are batch output as PDF power inspection reports, and the digital signatures of the inspectors, QR codes associated with the original data, and power inspection-specific watermarks are automatically added to complete the traceable release of the reports.

[0028] 6. Blockchain Evidence Storage: The generated PDF inspection report is hashed to obtain a unique hash value, which is then written into the consortium blockchain for evidence storage, ensuring that the report content cannot be tampered with and achieving non-repudiation of the inspection results.

[0029] 7. Heterogeneous Node Collaborative Scheduling: Establish a coordinate sharing protocol, clock synchronization mechanism, and standardized task transmission interface between UAVs and ground robots. The UAV will send the three-dimensional coordinates of suspected defects to the ground robot through the inverse solution of the pixel coordinate and three-dimensional space coordinate mapping model. The ground robot will adjust its travel path according to the coordinates and go to the target location for high-precision verification.

[0030] 8. 3D coordinate transformation and closed-loop tracking: During its movement, the ground robot acquires real-time pose data through its onboard LiDAR and positioning module. Combined with sensor intrinsic and extrinsic parameters, it dynamically updates the target's 3D coordinates to achieve closed-loop tracking of suspected defects, ensuring accurate arrival at the verification location. After confirming the existence of the defect through close-range photography and sensor detection, the verification results are fed back to the system.

[0031] 9. System self-evolution optimization: The system records the results of this inspection, combines them with historical inspection data, analyzes the high-incidence areas of defects and the suitability of detection thresholds, and dynamically optimizes the next inspection path of the drone and the front-end detection threshold to reduce the occurrence of similar false alarms.

[0032] Example 2 This embodiment is applied to an urban road inspection scenario, using a multi-rotor drone as an aerial inspection node and an intelligent inspection vehicle as a ground inspection node to perform inspection tasks on urban main roads and ancillary facilities: 1. Inspection raw data collection: Drones take aerial photos of the city's main roads to collect image data of suspected defects such as road surface cracks, damaged street lights, and crooked traffic signs, and generate defect category labels, confidence levels, and spatiotemporal metadata; the intelligent inspection vehicle is equipped with a high-definition camera, infrared sensor, and positioning equipment to receive collaborative task signals sent by the drone.

[0033] 2. Application of Industry Terminology Knowledge Base: The system utilizes the urban road inspection industry terminology knowledge base, with standardized terms corresponding to defect category tags. Based on the specific parameters of the defect, the system determines the defect level according to the defect level evaluation rules.

[0034] 3. Structured Document and Report Generation: Based on the urban road inspection report template engine, standardized descriptive text, defect level, spatiotemporal metadata and corresponding defect target images are dynamically filled into the template. If multiple records of the same type of defect are detected, the template engine automatically merges them to generate summary paragraphs, forming a structured intermediate document, and outputting an urban road inspection report in Word format, with an attached QR code and watermark from the urban management department.

[0035] 4. Collaborative Verification and System Optimization: The intelligent inspection vehicle travels to the target location for close-range verification based on the 3D coordinates sent by the drone. After confirming that the defect is true, the verification results are uploaded to the system. The system optimizes the drone inspection path based on historical data, increases the inspection frequency in high-incidence areas, and adjusts the defect detection threshold to improve the detection accuracy.

[0036] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for air-ground collaborative inspection based on heterogeneous platforms, characterized in that, Includes the following steps: Step 1: Obtain raw inspection data through the front-end acquisition terminal. The raw inspection data shall include at least the defect target image, defect category label, confidence level and corresponding spatiotemporal metadata. Step 2: Build and maintain an industry terminology knowledge base, which includes a defect category-industry terminology mapping table, defect level evaluation rules, causal relationship model and standard report paragraph templates, and stores them in the form of Extensible Markup Language or graph database; Step 3: Using the defect category label as input, the natural language processing engine performs semantic retrieval and alignment in the industry terminology knowledge base to generate standardized description text that conforms to industry standards, and automatically determines the defect level according to the defect level evaluation rules. Step 4: Based on the pre-built report template engine, dynamically fill the standardized description text, defect level, spatiotemporal metadata and associated images into the corresponding placeholders in the template to form a structured intermediate document; Step 5: Output the structured intermediate documents in batches as Word and / or PDF inspection reports that meet the format requirements through the document conversion interface, and support the automatic attachment of digital signatures, QR codes and watermarks to complete the traceable release of the reports.

2. The air-ground collaborative inspection method based on a heterogeneous platform according to claim 1, characterized in that, The industry terminology knowledge base further includes a thesaurus and a domain ontology to support the normalization of synonym defect descriptions and hierarchical concept reasoning.

3. A method for air-ground collaborative inspection based on a heterogeneous platform according to claim 1 or 2, characterized in that, The natural language processing engine employs a Transformer-based pre-trained language model and fine-tunes it using domain-specific corpora to improve the accuracy of terminology generation.

4. A method for air-ground collaborative inspection based on a heterogeneous platform according to any one of claims 1-3, characterized in that, In step four, when multiple defect records exist for the same device, the template engine automatically merges similar defects and generates summary paragraphs to avoid repetitive descriptions.

5. A method for air-ground collaborative inspection based on a heterogeneous platform according to any one of claims 1-4, characterized in that, The report template engine supports user-defined templates and uses a version control mechanism to enable template iteration and rollback.

6. A method for air-ground collaborative inspection based on a heterogeneous platform according to any one of claims 1-5, characterized in that, After step five, the generated inspection report hash value is further written into the blockchain for evidence storage, thereby ensuring the integrity and non-repudiation of the report.

7. A method for air-ground collaborative inspection based on a heterogeneous platform according to any one of claims 1-6, characterized in that, It also includes the steps for building a heterogeneous node collaboration mechanism: establishing a coordinate sharing protocol, clock synchronization mechanism and standardized task transmission interface between heterogeneous nodes to achieve efficient collaborative scheduling of multiple types of inspection nodes.

8. A method for air-ground collaborative inspection based on a heterogeneous platform according to any one of claims 1-7, characterized in that, It also includes a three-dimensional coordinate transformation step for pixel-level detection results: based on the real-time pose data of the inspection nodes and the intrinsic and extrinsic parameters of the sensors, a mapping model between pixel coordinates and three-dimensional spatial coordinates is established to achieve accurate inverse solution from pixel-level detection results to cross-platform shared three-dimensional target coordinates.

9. A method for air-ground collaborative inspection based on a heterogeneous platform according to any one of claims 1-8, characterized in that, It also includes system self-evolution optimization steps: the platform dynamically optimizes the front-end detection threshold and inspection path based on historical inspection results to reduce the recurrence of similar false alarms.

Images