Unmanned aerial vehicle inspection method and system

Abstract

The invention relates to the technical field of unmanned aerial vehicle inspection, in particular to an unmanned aerial vehicle inspection method and system. The method comprises the following steps: acquiring target area data, integrating a multi-source sensor to acquire multi-source sensing data, constructing a target area coordinate system, and determining a current coordinate point; acquiring real-time multi-source sensing data, dynamically generating an obstacle avoidance path and an inspection coordinate point, and triggering an inspection request; collecting defect data at the inspection coordinate points, associating historical inspection records, marking new defects to generate a trend analysis chart, and outputting the trend analysis chart; comprising a target area data acquisition module, an inspection path planning module and an inspection data reporting module, the obstacle avoidance points in the coordinate system of the target area are determined according to the real-time multi-source sensing data, the obstacle avoidance path and the inspection coordinate points are dynamically generated, inspection is carried out, the obstacle avoidance path is dynamically generated, and the inspection efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle inspection, and particularly relates to an unmanned aerial vehicle inspection method and system. BACKGROUND

[0002] Unmanned aerial vehicle inspection plays a key role in equipment monitoring and infrastructure maintenance in complex environments, and can replace manual work in high-risk areas such as power towers, wind turbine blades, and bridge cables, thereby avoiding the risk of personnel climbing.

[0003] In unmanned aerial vehicle inspection, path planning is prone to be rigid, and temporary obstacles such as construction equipment often need manual intervention to adjust the obstacle avoidance path, thereby reducing the inspection efficiency.

[0004] Therefore, it is necessary to provide an unmanned aerial vehicle inspection method and system for dynamically generating an obstacle avoidance path and improving the inspection efficiency. SUMMARY

[0005] The present application aims to provide an unmanned aerial vehicle inspection method and system, and aims to solve the technical problem in the prior art that in unmanned aerial vehicle inspection, path planning is prone to be rigid, and temporary obstacles often need manual intervention to adjust the obstacle avoidance path, thereby reducing the inspection efficiency.

[0006] To achieve the above-mentioned purpose, the present application adopts an unmanned aerial vehicle inspection method, which comprises the following steps: Obtain target area data, integrate multi-source sensors to obtain multi-source sensing data, construct a target area coordinate system, and determine a current coordinate point; Obtain real-time multi-source sensing data, dynamically generate an obstacle avoidance path and an inspection coordinate point, and trigger an inspection request; Collect defect data at the inspection coordinate point, associate historical inspection records, mark new defects to generate a trend analysis graph, and output.

[0007] In the step of obtaining target area data, integrating multi-source sensors to obtain multi-source sensing data, constructing a target area coordinate system, and determining a current coordinate point: Integrate multi-source sensors to obtain target area data; Determine a global coordinate system and a local coordinate system of the target area coordinate system respectively, align the local coordinate system with the global coordinate system, associate the map coordinate system with the global coordinate system, and dynamically update the coordinate system parameters.

[0008] After the step of determining a global coordinate system and a local coordinate system of the target area coordinate system respectively, aligning the local coordinate system with the global coordinate system, associating the map coordinate system with the global coordinate system, and dynamically updating the coordinate system parameters: Fuse multi-source sensor coordinate data and convert it to the global coordinate system to determine the current coordinate point.

[0009] Wherein, in the step of integrating multi-source sensors to acquire target area data: Deploy multiple sensors and trigger multiple sensor sampling with a unified clock source; Acquire multiple sensor sampling data and correlate and fuse the raw data to obtain target position and velocity; Extract intermediate features of the raw data, align and fuse in a unified space; Fuse multi-sensor recognition results to generate a final decision and output target area data.

[0010] Wherein, in the step of acquiring real-time multi-source sensor data, dynamically generating an obstacle avoidance path and inspection coordinate points, and triggering an inspection request: Set the scanning range, acquire obstacle data, and generate point cloud clusters; Generate an initial spiral flight path and mark inspection coordinate points, and determine the UAV obstacle avoidance action for obstacles.

[0011] Wherein, after the step of setting the scanning range, acquiring obstacle data, and generating point cloud clusters: Identify obstacles, mark obstacle bounding boxes and motion directions.

[0012] Wherein, after the step of generating an initial spiral flight path and marking inspection coordinate points, and determining the UAV obstacle avoidance action for obstacles: When the UAV reaches the inspection coordinate point, trigger the data collection task and record the trigger timestamp.

[0013] Wherein, in the step of collecting defect data at the inspection coordinate point, associating historical inspection records, marking new defects, generating trend analysis charts, and outputting: Acquire defect data at the inspection coordinate point and project the defect data into the coordinate system to generate a defect joint marking chart; Extract defect history of the same component, associate historical inspection records, and mark new defects.

[0014] Wherein, after the step of extracting defect history of the same component, associating historical inspection records, and marking new defects: Take time axis as abscissa and defect parameters as ordinate to draw trend curve and predict defect development, and output inspection report.

[0015] The present application also provides a UAV inspection system, comprising a target area data acquisition module, an inspection path planning module, and an inspection data reporting module; wherein: The target area data acquisition module is used to acquire target area data, integrate multi-source sensors to acquire multi-source sensor data, construct a target area coordinate system, and determine the current coordinate point; The inspection path planning module is used for acquiring real-time multi-source sensing data, dynamically generating an obstacle avoidance path and an inspection coordinate point, and triggering an inspection request. The inspection data reporting module is used for collecting defect data at the inspection coordinate point, associating historical inspection records, marking newly added defects to generate a trend analysis diagram, and outputting.

[0016] The unmanned aerial vehicle inspection method and system of the present application adopts the target area data acquisition module, the inspection path planning module, and the inspection data reporting module to perform the following steps: acquiring target area data, integrating multi-source sensors to acquire multi-source sensing data, constructing a target area coordinate system, and determining a current coordinate point; acquiring real-time multi-source sensing data, dynamically generating an obstacle avoidance path and an inspection coordinate point, and triggering an inspection request; collecting defect data at the inspection coordinate point, associating historical inspection records, marking newly added defects to generate a trend analysis diagram, and outputting; determining an obstacle avoidance point in the target area coordinate system according to real-time multi-source sensing data, dynamically generating an obstacle avoidance path and an inspection coordinate point, and performing inspection, which realizes dynamic generation of an obstacle avoidance path and improves inspection efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 is a step flow chart of the unmanned aerial vehicle inspection method of the present application.

[0019] Figure 2 is a step flow chart of S100 of the present application.

[0020] Figure 3 is a step flow chart of S200 of the present application.

[0021] Figure 4 is a step flow chart of S300 of the present application.

[0022] Figure 5 is a structure principle diagram of the unmanned aerial vehicle inspection system of the present application.

[0023] Figure 6 is a structure principle diagram of the electronic device of the present application.

[0024] 401-target area data acquisition module, 402-inspection path planning module, 403-inspection data reporting module. DETAILED DESCRIPTION

[0025] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, like reference numerals refer to like elements unless the context clearly dictates otherwise. The implementations described in the following exemplary embodiments are not meant to represent all implementations consistent with the present application.

[0026] The terminology used in this application, and by the inventors herein, is for the purpose of describing particular embodiments only and is not intended to be limiting, unless otherwise indicated. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0027] It is to be understood that the singular forms "a", "an", and "the" include plural referents unless the context clearly dictates otherwise. Pronouns in the masculine form include the feminine form, and vice versa, and the singular form also includes the plural form, unless the context clearly dictates otherwise. It will be further understood that the terms "comprises" and / or "comprising", when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0028] Referring now to the drawings Figures 1-4 The present application provides a UAV inspection method, comprising the following steps: S100: Obtain target area data, integrate multi-source sensors to obtain multi-source sensing data, construct a target area coordinate system, and determine a current coordinate point.

[0029] In this embodiment, the target area data is obtained, the multi-source sensors are integrated to obtain multi-source sensing data, the target area coordinate system is constructed, and the current coordinate point is determined. The specific process is as follows: S101: Integrate multi-source sensors to obtain target area data. S102: Determine a global coordinate system and a local coordinate system of the target area coordinate system respectively, align the local coordinate system with the global coordinate system, associate the map coordinate system with the global coordinate system, and dynamically update the coordinate system parameters. S103: Fuse multi-source sensor coordinate data and convert to the global coordinate system to determine the current coordinate point.

[0030] Further, in the step of integrating multi-source sensors to obtain target area data: Deploy multiple sensors and use a unified clock source to trigger multiple sensor sampling; Obtain multi-sensor sampling data and correlate and fuse the original data to obtain target position and velocity; Extract intermediate features of raw data, align and fuse in a unified space; Fuse multi-sensor recognition results, generate final decision, output target area data.

[0031] In the above process, hardware deployment: the unmanned aerial vehicle carries laser radar (LiDAR), visible light camera (RGB), infrared thermal imager (IR), inertial measurement unit (IMU) and GPS module, to ensure that the sensor field of view covers the target area.

[0032] Clock synchronization: use GPS time service or PTP protocol to synchronize all sensor clocks, trigger sampling time difference ≤1ms, avoid multi-source data time dislocation.

[0033] Data acquisition: Laser radar: scan the target area at a frequency of 10Hz, generate point cloud data (point density ≥100 points / m 2 ); Camera: synchronously shoot 4K resolution images (frame rate 5Hz), aligned with point cloud timestamp; IMU: record acceleration and angular velocity at a frequency of 100Hz, for motion compensation.

[0034] Data level fusion: Project laser point cloud to image plane, match point cloud and image feature points through ICP algorithm, correct point cloud distortion (such as wind speed 5m / s, distortion rate from 15% to 3%); Fuse infrared data and visible light image, mark temperature abnormal area through pseudo-color mapping (such as insulator self-explosion point temperature difference ≥5℃).

[0035] Feature level fusion: Extract geometric features (such as plane, cylinder) in point cloud and semantic features (such as crack, rust) in image, align and fuse in bird's eye view (BEV) space, generate target area 3D semantic map.

[0036] Decision level fusion: Combine laser radar detected obstacle distance, camera recognized obstacle category (such as trees, birds) and infrared detected living targets, generate final obstacle avoidance decision through D-S evidence theory fusion (such as "10m ahead of bird group, suggest detour").

[0037] Global coordinate system: use WGS84 geographic coordinate system, define target area latitude and longitude origin (such as tower bottom center point); Local coordinate system: take the unmanned aerial vehicle take-off point as the origin, the positive north direction as the positive direction of Y axis, construct the coordinate system, unit in meters.

[0038] Static calibration: Extract feature points of the target area (e.g. tower corners, grounding downline endpoints), measure their local coordinates with laser radar, combine with global coordinates from GPS positioning, use least squares method to solve rotation matrix R and translation vector T, realize alignment of local coordinate system and global coordinate system.

[0039] Dynamic calibration: During flight, estimate the UAV pose in real time through SLAM algorithm (e.g. LOAM), combine with GPS data to correct coordinate system parameters, compensate for wind disturbance or sensor drift (e.g. dynamic adjustment range of coordinate system offset ±0.5m).

[0040] Multi-sensor positioning fusion: GPS positioning: provides initial global coordinates (accuracy ±2m); Visual odometry (VO): estimates relative motion through feature matching of adjacent frames, combines with IMU data to optimize pose (accuracy ±0.1m); Laser odometry (LO): uses point cloud registration (e.g. NDT algorithm) to calculate displacement, combines with VO data through Kalman filtering, outputs final pose (comprehensive accuracy ±0.05m).

[0041] Coordinate conversion and output: converts the fused local coordinates to global coordinate system through calibration parameters R and T, generates current coordinate points), and labels timestamp (UTC time, accuracy 1ms).

[0042] S200: Acquire real-time multi-source sensor data, dynamically generate obstacle avoidance path and inspection coordinate points, and trigger inspection request.

[0043] In this embodiment, real-time multi-source sensor data is acquired, obstacle avoidance path and inspection coordinate points are dynamically generated, and inspection request is triggered. The specific process is as follows: S201: Set scan range, acquire obstacle data, and generate point cloud cluster; S202: Identify obstacles, mark obstacle bounding boxes and motion directions; S203: Generate initial spiral flight path, mark inspection coordinate points, and determine UAV obstacle avoidance action for obstacles; S204: When the UAV reaches the inspection coordinate point, trigger data acquisition task, and record trigger timestamp.

[0044] In the above process, the scan range is set to the current position of the UAV as the center, with a horizontal scan radius of 50m and a vertical range of -10m (ground) to +30m (air), covering the obstacles around the inspection path.

[0045] Point cloud cluster generation, voxel filtering (voxel size 0.1m) is performed on the raw point cloud of the laser radar, and the point cloud cluster is segmented by Euclidean clustering (distance threshold 1m), and the obstacle position is marked (such as "obstacle 1: center coordinates E116.3°N39.9°H15m, size 3m×2m×4m").

[0046] Obstacle classification, input the point cloud cluster into the PointNet++ network for semantic segmentation, identify the obstacle type (such as trees, power lines, birds), and the classification accuracy is ≥95%.

[0047] Motion direction marking, for dynamic obstacles (such as birds), calculate the velocity vector (such as "bird group speed 5m / s, direction northeast") by matching the point cloud of consecutive frames, and predict the trajectory in the next 3 seconds.

[0048] Initial route planning, adopt the spiral route algorithm, take the center of the target area as the starting point, increase the radius by 5m every circle, generate a patrol path covering all key components (such as tower insulators, fittings), and mark the patrol coordinate points.

[0049] Dynamic obstacle avoidance adjustment, when the obstacle intrudes into the route, trigger the RRT* algorithm to re-plan the path, detour the obstacle and keep the patrol coordinate points covered (such as detour radius ≥3m, path length increase ≤15%).

[0050] Task triggering, when the UAV reaches the patrol coordinate point, trigger the data collection task (such as camera shooting, infrared scanning) through hardware interruption, response time ≤100ms.

[0051] Timestamp recording, mark the UTC timestamp for each collected data, and synchronize with GPS time to ensure data time consistency (error ≤1ms).

[0052] S300: Collect defect data at the patrol coordinate point, associate historical patrol records, mark new defects and generate trend analysis chart, and output.

[0053] In this embodiment, the defect data at the patrol coordinate point is collected, the historical patrol records are associated, the new defects are marked, and the trend analysis chart is generated and output. The specific process is as follows: S301: Obtain the defect data at the patrol coordinate point, and project the defect data into the coordinate system to generate a defect joint marking chart; S302: Extract the defect history of the same component, associate the historical patrol records, and mark the new defects; S303: Take the time axis as the horizontal coordinate and the defect parameter as the vertical coordinate to draw the trend curve and predict the defect development, and output the patrol report.

[0054] In the above process, the collected defect data (such as crack length, temperature difference) is projected into the target area 3D semantic map to generate a defect joint label map (such as "insulator #3: crack length 0.2m, temperature difference 8℃").

[0055] Visual display, through WebGL rendering of defect label map, supports interactive viewing (such as rotation, zooming), and highlights serious defects (such as crack length ≥0.1m or temperature difference ≥5℃).

[0056] History extraction, call the historical inspection record of the same component (such as "insulator #3: crack length 0.1m, temperature difference 5℃ in March 2023").

[0057] New defect label, match the current defect with the historical defect feature through SIFT algorithm, label the new defect (such as "new crack 0.1m, temperature difference increased by 3℃ in March 2024"), and generate a change list.

[0058] Trend curve drawing, with time axis as horizontal coordinate and defect parameter (such as crack length, temperature difference) as vertical coordinate, draw trend curve (such as "insulator #3 crack length annual growth rate 200%").

[0059] Life prediction and report generation, based on linear regression model to predict remaining service life (such as "predicted temperature difference will reach 10℃ in 6 months, need to be replaced"), generate structured inspection report (including defect location, parameter, trend chart and maintenance suggestion), and push.

[0060] Corresponding to the foregoing embodiments of the unmanned aerial vehicle inspection method, the application also provides embodiments of an unmanned aerial vehicle inspection system.

[0061] Figure 5 is a block diagram of an unmanned aerial vehicle inspection system according to an exemplary embodiment. Referring to Figure 5 , the system can include: a target area data acquisition module 401, an inspection path planning module 402, and an inspection data reporting module 403; wherein: The target area data acquisition module 401 is configured to acquire target area data, integrate multi-source sensor to acquire multi-source sensor data, construct a target area coordinate system, and determine a current coordinate point. The inspection path planning module 402 is configured to acquire real-time multi-source sensor data, dynamically generate an obstacle avoidance path and an inspection coordinate point, and trigger an inspection request. The inspection data reporting module 403 is configured to collect defect data at the inspection coordinate point, associate historical inspection records, label new defects to generate a trend analysis chart, and output.

[0062] In the embodiment, the target area data acquisition module 401 acquires target area data, integrates multi-source sensor to acquire multi-source sensing data, constructs a target area coordinate system, and determines a current coordinate point; the inspection path planning module 402 acquires real-time multi-source sensing data, dynamically generates an obstacle avoidance path and an inspection coordinate point, and triggers an inspection request; the inspection data reporting module 403 acquires defect data at the inspection coordinate point, associates historical inspection records, marks a newly added defect to generate a trend analysis diagram, and outputs; by determining an obstacle avoidance point in the target area coordinate system according to real-time multi-source sensing data, dynamically generating an obstacle avoidance path and an inspection coordinate point, and performing inspection, the obstacle avoidance path is dynamically generated, and the inspection efficiency is improved.

[0063] As to the system in the above embodiments, the specific manner in which various modules perform operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

[0064] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts are described in the method embodiments. The above-described device embodiments are only schematic, and the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Some or all of the modules can be selected to achieve the purposes of the present application according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0065] Correspondingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the unmanned aerial vehicle inspection method as described above. As Figure 6 As shown in the figure, a hardware structure diagram of the unmanned aerial vehicle inspection system provided by the embodiment of the present application is in any device with data processing capability. In addition to the processor, memory and network interface shown in the figure, the device in the embodiment can also include other hardware according to the actual function of the device with data processing capability, which will not be described here. Figure 6 As shown in the figure, a hardware structure diagram of the unmanned aerial vehicle inspection system provided by the embodiment of the present application is in any device with data processing capability. In addition to the processor, memory and network interface shown in the figure, the device in the embodiment can also include other hardware according to the actual function of the device with data processing capability, which will not be described here.

[0066] Correspondingly, the application further provides a computer readable storage medium, which stores computer instructions, and the instructions are executed by a processor to implement the unmanned aerial vehicle inspection method. The computer readable storage medium can be an internal storage unit of any device with data processing capability, such as a hard disk or a memory. The computer readable storage medium can also be an external storage device, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. Further, the computer readable storage medium can include both the internal storage unit of any device with data processing capability and the external storage device. The computer readable storage medium is used to store the computer program and other programs and data required by the device with data processing capability, and can also be used to temporarily store data that has been output or will be output.

[0067] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the application embrace any and all variations of the present application that fall within the scope of the general inventive concept as defined by the appended claims and their equivalents. Accordingly, the application is not limited to only those embodiments that have been specifically described herein.

[0068] It is to be understood that the application is not limited to the precise construction described in the specification and shown in the drawings, and that various modifications and changes can be effected therein by those skilled in the art without departing from the scope of the application.

Claims

1. A drone inspection method, characterized in that: The steps include: Acquire target area data and integrate multi-source sensors to acquire multi-source sensing data, build the target area coordinate system, and determine the current coordinate point; Acquire real-time multi-source sensor data, dynamically generate obstacle avoidance paths and inspection coordinate points, and trigger inspection requests; Collect defect data at inspection coordinate points, associate historical inspection records, mark new defects, generate trend analysis charts, and output them.

2. The drone inspection method according to claim 1, wherein: In the steps of acquiring target area data, integrating multi-source sensors to acquire multi-source sensing data, constructing a target area coordinate system, and determining the current coordinate point: Integrate multi-source sensors to obtain target area data; Determine the global coordinate system and local coordinate system of the target area coordinate system respectively, align the local coordinate system with the global coordinate system, associate the map coordinate system with the global coordinate system, and dynamically update the coordinate system parameters.

3. The drone inspection method according to claim 2, wherein: After determining the global coordinate system and local coordinate system of the target area coordinate system, aligning the local coordinate system with the global coordinate system, associating the map coordinate system with the global coordinate system, and dynamically updating the coordinate system parameters: Fuse the multi-source sensor coordinate data and convert it into the global coordinate system to determine the current coordinate point.

4. The drone inspection method according to claim 2, wherein: In the steps of integrating multi-source sensors and acquiring target area data: Deploy multiple sensors and use a unified clock source to trigger multi-sensor sampling; Acquire multi-sensor sampling data, correlate and fuse the raw data, and obtain the target position and speed; Extract intermediate features of the original data, align and fuse them in a unified space; Fuse multi-sensor recognition results, generate final decisions, and output target area data.

5. The drone inspection method according to claim 1, wherein: In the steps of acquiring real-time multi-source sensor data, dynamically generating obstacle avoidance paths and inspection coordinate points, and triggering inspection requests: Set the scanning range, obtain obstacle data, and generate point cloud clusters; Generate an initial spiral route, mark inspection coordinate points, and determine the drone's obstacle avoidance actions based on obstacles.

6. The drone inspection method according to claim 5, wherein: After setting the scanning range, acquiring obstacle data, and generating point cloud clusters: Identify obstacles and mark the obstacle bounding box and movement direction.

7. The drone inspection method according to claim 6, wherein: After generating the initial spiral route, marking the inspection coordinate points, and determining the drone's obstacle avoidance actions: When the UAV reaches the inspection coordinate point, the data collection task is triggered and the trigger timestamp is recorded.

8. The drone inspection method according to claim 1, wherein: In the steps of collecting defect data at inspection coordinate points, correlating historical inspection records, marking new defects to generate trend analysis charts, and outputting them: Obtain defect data at the inspection coordinate points, project the defect data into the coordinate system, and generate a defect joint marking map; Extract the defect history of the same component, associate historical inspection records, and mark new defects.

9. The drone inspection method according to claim 8, wherein: After extracting the defect history of the same component, correlating historical inspection records, and marking new defects: With the time axis as the horizontal axis and the defect parameters as the vertical axis, draw a trend curve, predict the development of defects, and output an inspection report.

10. A drone inspection system, applied to the drone inspection method according to claim 1, characterized in that: It includes target area data collection module, inspection route planning module, and inspection data reporting module; among which: The target area data acquisition module is used to obtain target area data, integrate multi-source sensors to obtain multi-source sensing data, construct a target area coordinate system, and determine the current coordinate point; The inspection path planning module is used to obtain real-time multi-source sensor data, dynamically generate obstacle avoidance paths and inspection coordinate points, and trigger inspection requests; The inspection data reporting module is used to collect defect data at inspection coordinate points, associate historical inspection records, mark new defects, generate trend analysis charts, and output them.

Images