Unmanned aerial vehicle autonomous inspection and defect identification method and system for power transmission line

Abstract

This application provides a method and system for autonomous UAV inspection and defect identification of power transmission lines. Based on 3D point cloud data of the power transmission line, this application generates a target flight mission, simultaneously acquires high-resolution image sequences and 3D point cloud sequences, and simultaneously records the UAV's real-time positioning and attitude information. Each frame of the high-resolution image sequence is paired with a corresponding local point cloud in the 3D point cloud sequence to generate an inspection data unit. According to a preset prior geometric model, the component image regions contained in the images of the inspection data unit are determined. Based on the component image regions, the 3D point cloud in the inspection data unit and the normal state constraints defined by the prior geometric model are associated to generate defect identification results. The technical solution provided in this application achieves intelligent identification of line defects. This method effectively improves the automation level of inspection operations, the completeness of data acquisition, and the accuracy of defect identification.

Description

Technical Field

[0001] This application relates to the field of intelligent inspection technology for power transmission lines, and in particular to a method and system for autonomous inspection and defect identification of power transmission lines by unmanned aerial vehicles (UAVs). Background Technology

[0002] With the continuous expansion of power transmission networks and the increasing complexity of their distribution environments, traditional manual inspection methods face severe challenges in terms of efficiency, safety, and coverage. Developing automated and intelligent inspection technologies has become an urgent need to ensure the safe and stable operation of the power grid.

[0003] Currently, a representative technical solution is to use a drone equipped with a visible light camera to collect data along a preset route, and then transmit the captured images to a ground system. The system uses a deep learning-based image recognition model to automatically detect whether there are visible defects in the components. This solution achieves automation and preliminary intelligence in the inspection operation.

[0004] However, existing solutions mainly rely on two-dimensional image information, which is sensitive to shooting angle, lighting conditions and occlusion. They are difficult to accurately identify three-dimensional spatial defects such as insulator tilt and conductor sag. At the same time, they lack effective use of the precise three-dimensional shape and spatial connection relationship of components, which limits the accuracy and robustness of defect identification. Summary of the Invention

[0005] This application provides a method and system for autonomous inspection and defect identification of power transmission lines using unmanned aerial vehicles (UAVs), which solves the problems of insufficient accuracy and robustness in defect identification caused by relying solely on two-dimensional image recognition, which is sensitive to shooting angle and lighting conditions, making it difficult to effectively detect three-dimensional spatial morphological defects of components, and lacking joint analysis using the precise three-dimensional structure and spatial connection relationship of components.

[0006] Firstly, this application provides a method for autonomous unmanned aerial vehicle (UAV) inspection and defect identification of power transmission lines, including:

[0007] Based on the three-dimensional point cloud data of power transmission lines, a target flight mission is generated;

[0008] During the execution of the target flight mission, high-resolution image sequences and three-dimensional point cloud sequences are acquired simultaneously, and the real-time positioning and attitude determination information of the UAV is recorded simultaneously when the high-resolution image sequences and the three-dimensional point cloud sequences are acquired.

[0009] Based on the real-time positioning and the attitude information, each frame of the high-resolution image sequence is paired with the local point cloud of the corresponding spatial range in the three-dimensional point cloud sequence to generate an inspection data unit.

[0010] Based on the preset prior geometric model, determine the component image region contained in the image of the inspection data unit;

[0011] Based on the component image region, the defect identification result is generated by associating the 3D point cloud in the inspection data unit with the normal state constraints defined by the prior geometric model.

[0012] Optionally, based on the three-dimensional point cloud data of the transmission line, a target flight mission is generated, including:

[0013] From the 3D point cloud data, identify the set of prominent elevation points representing the tower body and the continuous linear point cloud representing the direction of the conductor;

[0014] Based on the spatial location of the set of elevation protrusions, determine the tower coordinates, and based on the extension direction of the continuous linear point cloud, determine the traverse path;

[0015] Connect the coordinates of adjacent towers and extend them along the conductor path to construct a line topology network;

[0016] With each tower node in the aforementioned line topology network as the center, plan the tower circling trajectory;

[0017] Based on each conductor connection edge in the aforementioned line topology network, a conductor tracking trajectory is planned;

[0018] All the tower-encircling flight paths and conductor-tracking flight paths are combined according to the connection sequence of the line topology network to form a space flight path;

[0019] Multi-angle shooting commands are configured for the tower circling track portion of the space flight track, and forward fixed-distance shooting commands are configured for the wire tracking track portion, together generating the target flight mission.

[0020] Optionally, during the execution of the target flight mission, high-resolution image sequences and three-dimensional point cloud sequences are acquired simultaneously, and the real-time positioning and attitude determination information of the UAV during the acquisition of the high-resolution image sequences and the three-dimensional point cloud sequences is recorded simultaneously, including:

[0021] During the flight mission of the UAV, the visible light sensor and the three-dimensional sensor on the UAV are triggered simultaneously to acquire high-resolution image sequences and three-dimensional point cloud sequences, respectively.

[0022] Each frame of the high-resolution image sequence is paired with a frame of point cloud in the three-dimensional point cloud sequence acquired at the same acquisition time to form an acquisition data pair;

[0023] While generating each data acquisition pair, the spatial coordinate information output by the UAV's global satellite navigation system at the acquisition time is recorded as the real-time positioning information at the acquisition time;

[0024] While generating each data acquisition pair, the attitude angle information output by the UAV inertial measurement unit at the acquisition time is recorded as the attitude information at the acquisition time.

[0025] Optionally, based on the real-time positioning and the pose information, each frame of the high-resolution image sequence is paired with a local point cloud of the corresponding spatial range in the three-dimensional point cloud sequence to generate an inspection data unit, including:

[0026] Based on the real-time positioning and attitude information recorded synchronously with each frame of the high-resolution image sequence, the imaging field of view of the image in three-dimensional space is determined;

[0027] From all the point cloud data contained in the three-dimensional point cloud sequence, extract all three-dimensional points whose spatial coordinates fall within the imaging field of view to form a local point cloud corresponding to the image.

[0028] The image is associated with the corresponding local point cloud to generate an inspection data unit.

[0029] Optionally, based on a preset prior geometric model, the component image region contained in the image of the inspection data unit is determined, including:

[0030] In the local point cloud contained in the inspection data unit, a three-dimensional point cloud structure that matches the standard three-dimensional shape of the preset prior geometric model is searched.

[0031] Based on the position and range of the three-dimensional point cloud structure in three-dimensional space, determine the pixel region corresponding to the three-dimensional point cloud structure on the image in the inspection data unit;

[0032] Based on the connection relationships between components in the prior geometric model, the spatial arrangement of the pixel region in the image is verified to determine the component image region.

[0033] Optionally, based on the connection relationships between components in the prior geometric model, the spatial arrangement of the pixel regions in the image is verified to determine the component image regions, including:

[0034] Establish spatial location indexes for all pixel regions;

[0035] Based on the spatial location index, find the adjacency relationship between each pixel region and other pixel regions;

[0036] The adjacency relationship is matched with the connection relationship between components in the prior geometric model to obtain the matching result;

[0037] Based on the matching results, the set of pixel regions that satisfy the connection relationship between the components is determined as the component image region.

[0038] Optionally, based on the component image region, and by associating the 3D point cloud in the inspection data unit with the normal state constraints defined by the prior geometric model, a defect identification result is generated, including:

[0039] For each component image region, the three-dimensional point set of the component surface corresponding to the component image region is extracted from the local point cloud contained in the inspection data unit.

[0040] The three-dimensional point set on the surface of the component is compared with the normal three-dimensional shape range in the prior geometric model to obtain the deviation information of the three-dimensional point set on the surface of the component relative to the normal three-dimensional shape range;

[0041] Based on the deviation information, it is determined whether there is a morphological abnormality in the component image region. When a morphological abnormality is determined to exist, a defect identification result is generated based on the location of the component image region where the morphological abnormality occurs and the corresponding component type.

[0042] Secondly, this application provides an unmanned aerial vehicle (UAV) autonomous inspection and defect identification system for power transmission lines, including:

[0043] The first generation module is used to generate target flight missions based on the three-dimensional point cloud data of power transmission lines.

[0044] The acquisition module is used to execute the target flight mission, simultaneously acquire high-resolution image sequences and three-dimensional point cloud sequences, and simultaneously record the real-time positioning and attitude determination information of the UAV when acquiring the high-resolution image sequences and the three-dimensional point cloud sequences;

[0045] The second generation module is used to pair each frame of the high-resolution image sequence with the local point cloud of the corresponding spatial range in the three-dimensional point cloud sequence according to the real-time positioning and the attitude information to generate an inspection data unit.

[0046] The determination module is used to determine the component image region contained in the image of the inspection data unit based on a preset prior geometric model.

[0047] The third generation module is used to generate defect identification results based on the component image region, associating the three-dimensional point cloud in the inspection data unit with the normal state constraints defined by the prior geometric model.

[0048] Thirdly, this application provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to realize the method for autonomous inspection and defect identification of power transmission lines by unmanned aerial vehicles as described in the first aspect above.

[0049] Fourthly, this application provides a computer storage medium storing a computer program, which, when executed by a computer, implements a method for autonomous inspection and defect identification of power transmission lines by unmanned aerial vehicles as described in the first aspect.

[0050] This application enables UAVs to autonomously fly and collect data on power transmission lines by autonomously planning inspection trajectories based on 3D point cloud data. By simultaneously acquiring and accurately registering images and 3D point cloud data, an inspection data unit containing multi-dimensional information is constructed. Then, by integrating prior geometric model knowledge, components are accurately located and their 3D morphology is analyzed, ultimately achieving intelligent identification of line defects. This method effectively improves the automation level of inspection operations, the completeness of data acquisition, and the accuracy of defect identification.

[0051] Furthermore, by extracting a set of three-dimensional points that precisely correspond to the component image region from the fused data and directly comparing it with the standard three-dimensional shape, the precise quantitative detection of three-dimensional deformation on the component surface is achieved. This overcomes the shortcomings of traditional two-dimensional image recognition in terms of sensitivity to angle and lighting, and can effectively identify spatial posture defects such as insulator tilt and vibration damper slippage, thereby significantly improving defect identification, especially the detection rate and reliability of three-dimensional morphological abnormality defects.

[0052] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 A flowchart of an unmanned aerial vehicle (UAV) autonomous inspection and defect identification method for power transmission lines provided in this application is shown.

[0055] Figure 2 This paper presents a schematic diagram of the structure of an unmanned aerial vehicle (UAV) autonomous inspection and defect identification system for power transmission lines provided in this application.

[0056] Figure 3 A schematic diagram of the structure of a computing device provided in this application is shown. Detailed Implementation

[0057] To enable those skilled in the art to better understand the present application, the technical solution of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0058] In some of the processes described in the specification, claims, and accompanying drawings of this application, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not themselves represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.

[0059] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0060] Figure 1 This application provides a flowchart of a method for autonomous UAV inspection and defect identification of power transmission lines, as shown in the flowchart. Figure 1 As shown, the method includes:

[0061] Step 101: Generate the target flight mission based on the three-dimensional point cloud data of the power transmission line.

[0062] Optionally, step 101 may specifically include the following steps:

[0063] Step 1011: Identify the set of prominent elevation points representing the tower body and the continuous linear point cloud representing the direction of the conductor from the three-dimensional point cloud data.

[0064] Step 1012: Determine the tower coordinates based on the spatial location of the set of elevation protrusions, and determine the traverse path based on the extension direction of the continuous linear point cloud;

[0065] Step 1013: Connect the coordinates of adjacent towers and extend them along the conductor path to construct a line topology network;

[0066] Step 1014: Using each tower node in the line topology network as the center, plan the tower circling trajectory;

[0067] Step 1015: Using each wire connection edge in the line topology network as a reference, plan the wire tracking trajectory;

[0068] Step 1016: Combine all the tower circling tracks and conductor tracking tracks according to the connection order of the line topology network to form a space flight track;

[0069] Step 1017: Configure multi-angle shooting instructions for the tower circling track portion of the space flight track, and configure forward fixed-distance shooting instructions for the wire tracking track portion, together generating the target flight mission.

[0070] In the above steps, the elevation prominence point set is a set of three-dimensional spatial points that are significantly higher in height than the surrounding terrain, automatically selected from the three-dimensional point cloud data describing the terrain of the transmission line corridor, and used to characterize the approximate position of the tower body.

[0071] Continuous linear point cloud is a collection of all three-dimensional spatial points that are identified from the same three-dimensional point cloud data and are presented as long strips that extend continuously. It is used to characterize the spatial orientation of a conductor or ground wire.

[0072] A line topology network is an abstract model consisting of tower nodes determined by a set of elevation protrusions and conductor paths determined by a continuous linear point cloud. It is used to express the spatial connection relationship between towers and conductors.

[0073] A tower-circling flight path is a three-dimensional circular flight route planned for drones, centered on a single tower node, to guide drones to observe towers from multiple angles.

[0074] A traverse tracking trajectory is a three-dimensional flight path planned for a UAV that is parallel to a single traverse path and maintains a fixed distance, used to guide the UAV to conduct continuous observations along the traverse path;

[0075] A space flight path is a complete and continuous three-dimensional flight route, which is composed of all the tower-encircling paths and wire-tracking paths combined in the connection sequence of the line topology network. It serves as the spatial path basis for UAV flight.

[0076] A target flight mission is a complete set of operational instructions that includes a space flight path and control commands for equipment attached to key points on the path, used to drive the UAV to autonomously complete data acquisition.

[0077] In this embodiment, the input 3D point cloud data is first processed. All 3D spatial data points significantly higher than the average terrain are automatically selected based on the height information of each 3D spatial data point constituting the 3D point cloud data. These selected 3D spatial data points are clustered to form a set of elevation prominence points. Simultaneously, based on the distribution pattern of the 3D spatial data points, all data point clusters with elongated, continuous distributions are identified, forming a continuous linear point cloud. Next, the spatial distribution center of each elevation prominence point set is calculated, and the coordinates of the spatial distribution center are determined as the tower coordinates. The main extension direction of each continuous linear point cloud is analyzed, and the centerline of the continuous linear point cloud is defined as the traverse path. Then, based on the principle of spatial adjacency, straight lines are used to connect the individual tower coordinates, ensuring that these connecting lines coincide spatially with the determined traverse path, thereby constructing a line topology network depicting the physical connection relationships of the lines.

[0078] Then, for each tower coordinate in the line topology network, a three-dimensional spatial loop path around the tower coordinate is calculated, namely the tower circling track. For each conductor path in the line topology network, a three-dimensional spatial path parallel to the conductor path and maintaining a safe distance is calculated, namely the conductor tracking track. Then, these scattered tower circling tracks and conductor tracking tracks are connected end to end and smoothly transitioned in strict accordance with the connection order described by the line topology network, and integrated into a continuous spatial flight track from the inspection start point to the end point. Finally, multiple different camera shooting angles are preset in the flight segments of the spatial flight track belonging to the tower circling track, and fixed shooting intervals and forward shooting angles are preset in the flight segments belonging to the conductor tracking track. After these spatial flight tracks and shooting commands are packaged, a target flight mission that can be directly read and executed by the UAV flight control system is generated.

[0079] For example, three-dimensional point cloud data of a transmission line corridor was acquired in area A. After processing the data, the set of prominent elevation points representing towers and the continuous linear point cloud representing conductors were identified. Based on this, the tower coordinates and conductor paths were calculated, the line topology network was constructed, and then the tower circling trajectory and conductor tracking trajectory were planned. These were combined into a complete space flight trajectory, and finally, an executable target flight mission was generated by adding shooting instructions.

[0080] This step can automatically parse the transmission line structure from 3D point cloud data and intelligently generate refined flight missions, achieving precise alignment of the inspection path with the line morphology, and laying the foundation for comprehensive and efficient data collection.

[0081] Step 102: Execute the target flight mission, simultaneously acquire high-resolution image sequences and three-dimensional point cloud sequences, and simultaneously record the real-time positioning and attitude determination information of the UAV when acquiring the high-resolution image sequences and the three-dimensional point cloud sequences.

[0082] Optionally, step 102 may specifically include the following steps:

[0083] Step 1021: During the process of the UAV performing the target flight mission, the visible light sensor and the three-dimensional sensor on the UAV are simultaneously triggered to acquire high-resolution image sequences and three-dimensional point cloud sequences, respectively.

[0084] Step 1022: Pair each frame of the high-resolution image sequence with a frame of the three-dimensional point cloud sequence acquired at the same acquisition time to form an acquisition data pair;

[0085] Step 1023: While generating each data acquisition pair, record the spatial coordinate information output by the UAV's global satellite navigation system at the acquisition time, as the real-time positioning information at the acquisition time;

[0086] Step 1024: While generating each data acquisition pair, record the attitude angle information output by the UAV inertial measurement unit at the acquisition time, as the attitude information at the acquisition time.

[0087] In the above steps, the high-resolution image sequence is a collection of multiple detailed two-dimensional photos continuously captured by the visible light sensor on the drone during flight, arranged in chronological order, and is used to record the visual appearance information of the power transmission line components.

[0088] A three-dimensional point cloud sequence is a collection of spatial point data representing the three-dimensional shape of an object's surface, obtained by a three-dimensional sensor carried by a UAV during flight, arranged in chronological order. It is used to record the three-dimensional structural information of power transmission line components.

[0089] A data pair is a data unit formed by combining a frame of image extracted from a high-resolution image sequence and a frame of point cloud extracted from a 3D point cloud sequence at the same precise moment. It is used to establish the correspondence between 2D visual information and 3D spatial information at the same moment.

[0090] Real-time positioning information is the specific position coordinate data of the UAV in three-dimensional space measured and output by the UAV's global satellite navigation system at the moment each data pair is generated. It is used to mark the spatial source of the data pair in the real world.

[0091] Attitude information refers to the pitch, roll, and yaw angles of the UAV in three-dimensional space, measured and output by the UAV's inertial measurement unit at the moment each data pair is generated. It is used to describe the spatial orientation of the sensor when the data pair is acquired.

[0092] In this embodiment, the flight control system first controls the UAV to fly strictly according to the space flight path and shooting instructions in the target flight mission. At the point on the flight path where data needs to be collected, a synchronous trigger signal is sent to the visible light sensor and the 3D sensor, so that the two sensors collect a high-resolution image and a frame of 3D point cloud at exactly the same trigger time, thereby obtaining a high-resolution image sequence and a 3D point cloud sequence that are perfectly aligned in time. Secondly, by reading the timestamp recorded by each sensor at the trigger time, each high-resolution image in the high-resolution image sequence and a frame of 3D point cloud with the same timestamp in the 3D point cloud sequence are automatically associated and bundled into an inseparable data acquisition pair.

[0093] Next, while generating each data pair, the system reads and saves the longitude, latitude, and altitude data output by the UAV's global satellite navigation system at the current acquisition time in real time. This longitude, latitude, and altitude data are used as real-time positioning information bound to the current data pair. Finally, while generating the same data pair, the system reads and saves the pitch, roll, and yaw angle data output by the UAV's inertial measurement unit at the current acquisition time in real time. This pitch, roll, and yaw angle data are used as attitude determination information bound to the current data pair, thereby completing the precise synchronization and recording of all sensor data and spatiotemporal reference information.

[0094] For example, following the implementation case from the previous step, during the inspection of power transmission lines in area A, the UAV begins executing the generated target flight mission file. When the UAV reaches the starting point of its orbital path around the first tower, the flight control system simultaneously triggers the visible light camera and lidar. The camera captures a high-resolution photograph, and the lidar scans to obtain a frame of 3D point cloud. These two data points are marked as being produced at the same time T1. The system immediately pairs the photograph and point cloud at time T1 to form the first data pair. At the same time, the system records the 3D coordinates output by the UAV's GNSS module at time T1 as the real-time positioning information of this data pair, and records the three-axis angles output by the UAV's IMU module at time T1 as the attitude information of this data pair. The UAV continues to fly along the path, repeating this synchronous acquisition and recording process at the next shooting point until the mission is completed.

[0095] This step ensures strict consistency between the 2D images and 3D point cloud data at the time of acquisition through precise hardware synchronization and timestamp management. By synchronously recording the spatial position and attitude angle of the UAV, each data pair is given a precise spatiotemporal label. This fundamentally solves the problem of aligning multi-source heterogeneous data in time and space, providing a reliable and complete data foundation for the accurate matching and fusion analysis of images and 3D point clouds in subsequent steps.

[0096] Step 103: Based on the real-time positioning and the attitude information, each frame of the high-resolution image sequence is paired with the local point cloud of the corresponding spatial range in the three-dimensional point cloud sequence to generate an inspection data unit.

[0097] Optionally, step 103 may specifically include the following steps:

[0098] Step 1031: Determine the imaging field of view of the image in three-dimensional space based on the real-time positioning and attitude information recorded synchronously with each frame of the high-resolution image sequence.

[0099] Step 1032: Extract all three-dimensional points whose spatial coordinates fall within the imaging field of view from all point cloud data contained in the three-dimensional point cloud sequence to form a local point cloud corresponding to the image.

[0100] Step 1033: Associate the image with the corresponding local point cloud to generate an inspection data unit.

[0101] In the above steps, the imaging field of view is a three-dimensional spatial region calculated by geometric projection relationship based on the real-time positioning and attitude information recorded synchronously with each frame of the high-resolution image sequence, combined with the internal parameters such as the lens focal length and photosensitive element size of the visible light sensor. It is used to describe the spatial range that the frame of the image can be observed in the real world.

[0102] Local point cloud is a set of three-dimensional spatial data points whose spatial coordinates fall within the imaging field of view, extracted from all point cloud data contained in a three-dimensional point cloud sequence. It is used to characterize the three-dimensional structure of transmission line components that fall within the field of view of a single image.

[0103] An inspection data unit is an independent and complete data package formed by binding a frame of an image in a high-resolution image sequence with the corresponding local point cloud through data association operations. It serves as the smallest processing unit for subsequent component identification and defect analysis.

[0104] In this embodiment, firstly, by reading the real-time positioning and attitude information bound to each frame of the high-resolution image sequence, and combining the camera's known focal length and sensor size parameters, the spatial range that each frame of the image can cover in three-dimensional space is calculated using the principle of perspective projection. This calculated spatial range is the imaging field of view of each frame of the image. Secondly, by using the imaging field of view as a spatial filter, all three-dimensional spatial data points contained in the three-dimensional point cloud sequence are traversed and judged, and data points whose spatial coordinates fall within the boundary of the imaging field of view are selected. These data points are then aggregated to form a local point cloud that strictly corresponds to the current image in space. Finally, by establishing a data index, the current image in the high-resolution image sequence is bound one-to-one with the local point cloud corresponding to the current image in space, forming a data combination containing two-dimensional visual information and corresponding three-dimensional structural information. This data combination is the inspection data unit.

[0105] For example, following the implementation case of the previous step, for the first high-resolution image acquired at time T1, the system reads the real-time positioning and attitude information bound to the image, and calculates a cone-shaped area seen by the camera lens in three-dimensional space at time T1 as the imaging field of view, in combination with the camera parameters; then the system traverses all the three-dimensional point cloud data acquired in step 102, extracts all the points located in this cone-shaped area, and forms a local point cloud within the field of view of the image at time T1; finally, the system packages the high-resolution image at time T1 with this newly extracted local point cloud to generate the first inspection data unit, and repeats this process for images acquired at each subsequent time.

[0106] This step utilizes precise spatiotemporal calibration information to accurately correlate two-dimensional images with three-dimensional point cloud data in the spatial dimension. Each inspection image is matched with point cloud data describing the three-dimensional shape of the same scene, thereby constructing a data unit containing multi-dimensional information. This greatly enhances the data's representational capabilities and provides direct and reliable data input for subsequent accurate identification of components and defect analysis that rely on three-dimensional spatial relationships.

[0107] Step 104: Determine the component image region contained in the image of the inspection data unit according to the preset prior geometric model.

[0108] Optionally, step 104 may specifically include the following steps:

[0109] Step 1041: In the local point cloud contained in the inspection data unit, search for a three-dimensional point cloud structure that matches the standard three-dimensional shape of the preset prior geometric model.

[0110] Step 1042: Based on the position and range of the three-dimensional point cloud structure in three-dimensional space, determine the pixel region corresponding to the three-dimensional point cloud structure on the image in the inspection data unit.

[0111] Step 1043: Based on the connection relationship between components in the prior geometric model, verify the spatial arrangement of the pixel region in the image to determine the component image region.

[0112] Step 1043 may specifically include the following steps:

[0113] A spatial location index is established for all pixel regions; based on the spatial location index, the adjacency relationship between each pixel region and other pixel regions is found; the adjacency relationship is matched with the component connection relationship in the prior geometric model to obtain the matching result; according to the matching result, the set of pixel regions that satisfy the component connection relationship is determined as the component image region.

[0114] In the above steps, the three-dimensional point cloud structure is a set of three-dimensional spatial data points that conform to the morphological characteristics of a specific component, which is identified from the local point cloud contained in the inspection data unit by comparing and matching the shape of the point set of the local point cloud with the three-dimensional shape of the standard component stored in the preset prior geometric model. This set is used to locate and delineate the outline of the suspected component in three-dimensional space.

[0115] A pixel region is a closed area formed by mapping the spatial contour of a three-dimensional structure onto a two-dimensional image plane in the inspection data unit, based on the position and range of the identified three-dimensional point cloud structure in three-dimensional space and combined with the imaging parameters of the camera. This area is composed of many pixels in the image and is used to mark the suspected position corresponding to the three-dimensional component on the two-dimensional image.

[0116] The component image region is an accurate image region that represents a real transmission line component. It integrates pixel region information obtained from the image and the spatial connection relationship between components defined in the prior geometric model. After logically verifying the layout and relative position of the pixel region in the whole image, it is finally confirmed.

[0117] In this embodiment, firstly, the shape of the local point cloud in the inspection data unit is compared with the standard component 3D shape template stored in the prior geometric model. All point sets in the local point cloud that are spatially similar to the standard component 3D shape template are identified; these point sets are the 3D point cloud structures that match the prior geometric model. Secondly, using the camera imaging model, the coordinates of the outer contour boundary points of each 3D point cloud structure in 3D space are converted into coordinate positions on the 2D image in the inspection data unit, determining the frame area occupied by each 3D point cloud structure on the 2D image; this frame area is the pixel region. Finally, by referring to the physical connection rules between components defined in the prior geometric model, the relative positional relationships between all pixel regions are checked, and those pixel regions that conform to the physical connection rules between components are determined as component image regions.

[0118] For example, following the implementation case of the previous step, for the first inspection data unit that has been generated, the system compares the local point cloud data contained therein with the standard three-dimensional shapes of insulator strings, vibration dampers, etc. in the model library one by one, and finds a three-dimensional point cloud structure that highly matches the insulator string model. Based on the spatial coordinates of this three-dimensional point cloud structure, the system defines a corresponding elongated pixel region on the high-resolution image of the inspection data unit. Then, according to the rule in the model that "there should be connecting hardware above the insulator string", the system finds and verifies another pixel region in the image above this pixel region that conforms to the shape of the hardware. Finally, the system determines the two pixel regions that conform to the model definition in terms of spatial relationship as a complete component image region of "insulator string component".

[0119] This step, by integrating shape matching of 3D point clouds with projection positioning of 2D images, accurately locates candidate regions of components from multi-dimensional data. Then, it introduces the spatial connection relationship between components as a verification basis, effectively eliminating image interference caused by trees, light and shadow, etc. Finally, it accurately and reliably determines the image regions that truly belong to the transmission line components on the inspection images with complex backgrounds, providing accurate target input for subsequent defect analysis.

[0120] Step 105: Based on the component image region, associate the 3D point cloud in the inspection data unit with the normal state constraints defined by the prior geometric model to generate defect identification results.

[0121] Optionally, step 105 may specifically include the following steps:

[0122] Step 1051: For each component image region, extract the three-dimensional point set of the component surface corresponding to the component image region from the local point cloud contained in the inspection data unit.

[0123] Step 1052: Compare the three-dimensional point set on the surface of the component with the normal three-dimensional shape range in the prior geometric model to obtain the deviation information of the three-dimensional point set on the surface of the component relative to the normal three-dimensional shape range;

[0124] Step 1053: Based on the deviation information, determine whether there is a morphological abnormality in the component image region. When it is determined that there is a morphological abnormality, generate a defect identification result based on the location of the component image region where the morphological abnormality occurs and the corresponding component type.

[0125] In the above steps, the three-dimensional point set of the component surface is a set of all data points that are precisely segmented from the local point cloud to characterize the three-dimensional shape of the outer surface of the specific component, based on the spatial correspondence between the component image region and the local point cloud, for each component image region determined on the image of the inspection data unit, and is used to provide the precise three-dimensional geometric data of the component.

[0126] Deviation information is a descriptive data on the degree of difference between the three-dimensional point set on the surface of a component and the standard normal three-dimensional shape of the corresponding type of component as defined in the prior geometric model. It is obtained by comparing and analyzing the three-dimensional point set on the surface of a component one by one with the standard normal three-dimensional shape of the corresponding type of component. It is used to objectively measure the degree of abnormality of the component's shape.

[0127] The defect identification result integrates the quantitative analysis conclusions of deviation information with the location and type information of the component image area. The resulting final output report contains the specific defective component identity, defect spatial location, and defect category determination, which is used to directly guide operation and maintenance work.

[0128] In this embodiment, firstly, through spatial mapping, for each determined component image region, three-dimensional data points that completely overlap with the physical component represented by the component image region in space are found in the local point cloud contained in the inspection data unit. These three-dimensional data points are completely extracted to form a component surface three-dimensional point set that specifically describes the shape of a single physical component. Secondly, through geometric measurement and comparison, the actual three-dimensional shape expressed by the component surface three-dimensional point set is compared with the allowable range of standard normal three-dimensional shape parameters in the prior geometric model database. All differences between the component surface three-dimensional point set in spatial orientation and the allowable range of standard normal three-dimensional shape parameters are calculated and recorded. These calculated difference data are the deviation information.

[0129] Finally, through logical judgment and result synthesis, the deviation information is evaluated. If the deviation information indicates that the difference of the three-dimensional point set on the surface of the component in the key dimension exceeds the threshold allowed by the normal state constraint defined by the model, it is determined that the component corresponding to the component image region has a morphological abnormality. The coordinate position of the component image region with morphological abnormality in the image and the specific component type corresponding to the component image region are integrated to generate the defect identification result.

[0130] For example, following the implementation case of the previous step, for the component image area representing the insulator string confirmed in the first inspection data unit, the system extracts dense three-dimensional points depicting the surfaces of all the skirts and steel caps of the insulator string based on the correspondence between the area and the local point cloud, forming a three-dimensional point set of the component surface of the insulator string; then, the system compares the point set with the three-dimensional morphology standard of "normal insulator string" in the model library, and finds that the point set as a whole presents a significant curvature, and the curvature value exceeds the normal range, thereby generating deviation information describing this curvature and direction; finally, the system determines that the insulator string has a "tilting" defect based on the deviation information, and combined with its position in the image, generates a defect identification result containing "Location: Middle phase of tower No. 12, Component: Insulator string, Defect: Tilt".

[0131] This step extracts the three-dimensional surface data that precisely corresponds to the component and performs a rigorous quantitative comparison with the standard model, thereby achieving objective and accurate detection of three-dimensional morphological defects in the component. This overcomes the excessive reliance of traditional methods on the appearance features of two-dimensional images and significantly improves the identification ability and reliability of spatial posture defects such as insulator tilt and vibration damper slippage, thus providing direct and effective decision support for the safety status assessment of transmission lines.

[0132] Figure 2 This application provides a structural schematic diagram of an unmanned aerial vehicle (UAV) autonomous inspection and defect identification system for power transmission lines, as shown in the figure. Figure 2 As shown, the system includes:

[0133] The first generation module 21 is used to generate target flight missions based on the three-dimensional point cloud data of the power transmission line.

[0134] The acquisition module 22 is used to execute the target flight mission, synchronously acquire high-resolution image sequences and three-dimensional point cloud sequences, and synchronously record the real-time positioning and attitude information of the UAV when acquiring the high-resolution image sequences and the three-dimensional point cloud sequences;

[0135] The second generation module 23 is used to pair each frame of the high-resolution image sequence with the local point cloud of the corresponding spatial range in the three-dimensional point cloud sequence according to the real-time positioning and the attitude information to generate an inspection data unit.

[0136] The determination module 24 is used to determine the component image region contained in the image of the inspection data unit according to the preset prior geometric model;

[0137] The third generation module 25 is used to generate defect identification results based on the component image region, associating the three-dimensional point cloud in the inspection data unit with the normal state constraints defined by the prior geometric model.

[0138] Figure 2 The aforementioned UAV autonomous inspection and defect identification system for power transmission lines can perform... Figure 1 The implementation principle and technical effects of the UAV autonomous inspection and defect identification method for power transmission lines described in the illustrated embodiment will not be repeated here. The specific operation methods of each module and unit in the UAV autonomous inspection and defect identification system for power transmission lines described in the above embodiments have been described in detail in the embodiments related to this method, and will not be elaborated upon here.

[0139] In one possible design, Figure 2 The illustrated embodiment of an unmanned aerial vehicle (UAV) autonomous inspection and defect identification system for power transmission lines can be implemented as a computing device, such as... Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32;

[0140] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are invoked and executed by the processing component 32.

[0141] The processing component 32 is used for the above Figure 1 The above embodiment provides a method for autonomous inspection and defect identification of power transmission lines using unmanned aerial vehicles (UAVs).

[0142] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above-described method. Alternatively, the processing component may be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0143] Storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0144] Of course, computing devices may also include other components, such as input / output interfaces, display components, communication components, etc.

[0145] Input / output interfaces provide interfaces between processing components and peripheral interface modules, which can be output devices, input devices, etc.

[0146] The communication components are configured to facilitate wired or wireless communication between computing devices and other devices.

[0147] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server, and the aforementioned processing components, storage components, etc., can be basic server resources rented or purchased from the cloud computing platform.

[0148] This application also provides a computer storage medium storing a computer program, which, when executed by a computer, can perform the above-described functions. Figure 1 The embodiment shown is a method for autonomous inspection and defect identification of power transmission lines using unmanned aerial vehicles (UAVs).

[0149] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0150] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0151] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for power transmission line oriented unmanned aerial vehicle autonomous inspection and defect identification, characterized in that, include: Based on the three-dimensional point cloud data of power transmission lines, a target flight mission is generated; During the execution of the target flight mission, high-resolution image sequences and three-dimensional point cloud sequences are acquired simultaneously, and the real-time positioning and attitude determination information of the UAV is recorded simultaneously when the high-resolution image sequences and the three-dimensional point cloud sequences are acquired. Based on the real-time positioning and the attitude information, each frame of the high-resolution image sequence is paired with the local point cloud of the corresponding spatial range in the three-dimensional point cloud sequence to generate an inspection data unit. Based on a preset prior geometric model, the component image regions contained in the images of the inspection data unit are determined, including: searching for a 3D point cloud structure in the local point cloud contained in the inspection data unit that matches the standard 3D shape of the preset prior geometric model; determining the pixel region corresponding to the 3D point cloud structure on the image in the inspection data unit based on the position and range of the 3D point cloud structure in 3D space; and verifying the spatial arrangement of the pixel region in the image based on the connection relationship between components in the prior geometric model to determine the component image regions. Based on the component image region, the three-dimensional point cloud in the inspection data unit and the normal state constraints defined by the prior geometric model are associated to generate defect identification results; The step of verifying the spatial arrangement of the pixel region in the image based on the connection relationship between components in the prior geometric model, and determining the component image region, includes: A spatial location index is established for all pixel regions; based on the spatial location index, the adjacency relationship between each pixel region and other pixel regions is found; the adjacency relationship is matched with the component connection relationship in the prior geometric model to obtain the matching result; according to the matching result, the set of pixel regions that satisfy the component connection relationship is determined as the component image region.

2. The method according to claim 1, characterized in that, Based on the 3D point cloud data of power transmission lines, a target flight mission is generated, including: From the 3D point cloud data, identify the set of prominent elevation points representing the tower body and the continuous linear point cloud representing the direction of the conductor; Based on the spatial location of the set of elevation protrusions, determine the tower coordinates, and based on the extension direction of the continuous linear point cloud, determine the traverse path; Connect the coordinates of adjacent towers and extend them along the conductor path to construct a line topology network; With each tower node in the aforementioned line topology network as the center, plan the tower circling trajectory; Based on each conductor connection edge in the aforementioned line topology network, a conductor tracking trajectory is planned; All the tower-encircling flight paths and conductor-tracking flight paths are combined according to the connection sequence of the line topology network to form a space flight path; Multi-angle shooting commands are configured for the tower circling track portion of the space flight track, and forward fixed-distance shooting commands are configured for the wire tracking track portion, together generating the target flight mission.

3. The method of claim 1, wherein, During the execution of the target flight mission, high-resolution image sequences and 3D point cloud sequences are acquired simultaneously, and the real-time positioning and attitude determination information of the UAV during the acquisition of the high-resolution image sequences and the 3D point cloud sequences is recorded simultaneously, including: During the flight mission of the UAV, the visible light sensor and the three-dimensional sensor on the UAV are triggered simultaneously to acquire high-resolution image sequences and three-dimensional point cloud sequences, respectively. Each frame of the high-resolution image sequence is paired with a frame of point cloud in the three-dimensional point cloud sequence acquired at the same acquisition time to form an acquisition data pair; While generating each data acquisition pair, the spatial coordinate information output by the UAV's global satellite navigation system at the acquisition time is recorded as the real-time positioning information at the acquisition time; While generating each data acquisition pair, the attitude angle information output by the UAV inertial measurement unit at the acquisition time is recorded as the attitude information at the acquisition time.

4. The method of claim 1, wherein, Based on the real-time positioning and the pose determination information, each frame of the high-resolution image sequence is paired with a local point cloud of the corresponding spatial range in the 3D point cloud sequence to generate an inspection data unit, including: Based on the real-time positioning and attitude information recorded synchronously with each frame of the high-resolution image sequence, the imaging field of view of the image in three-dimensional space is determined; From all the point cloud data contained in the three-dimensional point cloud sequence, extract all three-dimensional points whose spatial coordinates fall within the imaging field of view to form a local point cloud corresponding to the image. The image is associated with the corresponding local point cloud to generate an inspection data unit.

5. The method of claim 1, wherein, Based on the component image region, and by associating the 3D point cloud in the inspection data unit with the normal state constraints defined by the prior geometric model, defect identification results are generated, including: For each component image region, the three-dimensional point set of the component surface corresponding to the component image region is extracted from the local point cloud contained in the inspection data unit. The three-dimensional point set on the surface of the component is compared with the normal three-dimensional shape range in the prior geometric model to obtain the deviation information of the three-dimensional point set on the surface of the component relative to the normal three-dimensional shape range; Based on the deviation information, it is determined whether there is a morphological abnormality in the component image region. When a morphological abnormality is determined to exist, a defect identification result is generated based on the location of the component image region where the morphological abnormality occurs and the corresponding component type.

6. A UAV autonomous inspection and defect identification system for power transmission lines, applied to the UAV autonomous inspection and defect identification method for power transmission lines according to any one of claims 1-5, characterized in that... include: The first generation module is used to generate target flight missions based on the three-dimensional point cloud data of power transmission lines. The acquisition module is used to execute the target flight mission, simultaneously acquire high-resolution image sequences and three-dimensional point cloud sequences, and simultaneously record the real-time positioning and attitude determination information of the UAV when acquiring the high-resolution image sequences and the three-dimensional point cloud sequences; The second generation module is used to pair each frame of the high-resolution image sequence with the local point cloud of the corresponding spatial range in the three-dimensional point cloud sequence according to the real-time positioning and the attitude information to generate an inspection data unit. The determination module is used to determine the component image region contained in the image of the inspection data unit based on a preset prior geometric model. The third generation module is used to generate defect identification results based on the component image region, associating the three-dimensional point cloud in the inspection data unit with the normal state constraints defined by the prior geometric model.

7. A computing device, characterized in that, It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement the UAV autonomous inspection and defect identification method for power transmission lines as described in any one of claims 1 to 5.

8. A computer storage medium, characterized in that The system contains a computer program that, when executed by a computer, implements a method for autonomous inspection and defect identification of power transmission lines using unmanned aerial vehicles (UAVs) as described in any one of claims 1 to 5.

Images