Line patrol unmanned aerial vehicle power line patrol data fusion method based on multi-aerial vehicle cooperative control

Abstract

The invention relates to a line patrol unmanned aerial vehicle power line patrol data fusion method based on multi-aerial vehicle cooperative control, provides deep learning, and belongs to the field of image or video recognition or understanding. The method comprises the following steps: a ground cooperative control station synchronously issues an N-view-angle line patrol task of a target line segment of a set power line transmission line to N line patrol unmanned aerial vehicles; and adopting an AI inspection data fusion model of a set power line transmission line to intelligently judge various fault types existing in a target line segment at a preset acquisition moment and positioning information of occurrence positions of the fault types on the basis of multiple analysis data corresponding to the N-view-angle line inspection task. In order to solve the technical problem that the precision, efficiency and comprehensiveness of a power inspection result cannot be improved at the same time in the prior art, a hardware architecture for power line inspection data fusion of the unmanned aerial vehicle for multi-vehicle cooperative control is established, and synchronous intelligent analysis of multiple fault types and occurrence positions thereof is completed by adopting an intelligent mode. Therefore, the technical problem is solved.

Description

A Data Fusion Method for Power Line Inspection Using Unmanned Aerial Vehicles Based on Multi-Machine Cooperative Control Technical Field

[0001] The deep learning proposed in this invention belongs to the field of image or video recognition or understanding, and in particular relates to a data fusion method for power line inspection drones based on multi-machine collaborative control. Background Technology

[0002] Due to the high complexity and difficulty in recognizing image or video content, it is advisable to use artificial intelligence models with deep learning to perform image or video content recognition in various sub-application areas. This provides targeted solutions for scene detection in various sub-application areas. For example, for images or video content collected during power line inspection tasks, it is common for power line management departments to use artificial intelligence models with deep learning to perform fault detection and status analysis of power lines erected in fixed areas.

[0003] For example, Chinese invention patent publication CN119813518A proposes a deep learning-based intelligent power line inspection management system and method. The system includes: a data acquisition device installed within the inspection area to collect monitoring data and environmental data from different power lines; a server to receive real-time reports of monitoring and environmental data from different power lines from the data acquisition device; and a power line inspection model used to inspect the power lines based on the monitoring and environmental data for each power line within a preset time period, obtaining inspection results and pushing these results to relevant terminal devices. The inspection results include whether a fault will occur and the type of fault if it does. The power line inspection model is a model capable of predicting and identifying power line faults, obtained by optimizing and training a long short-term memory network using an improved population optimization algorithm. This system and method aim to reduce the reliance on manual labor in power grid safety inspections.

[0004] For example, Chinese invention patent publication CN 118539338A proposes an intelligent inspection system for power line installations. The input of this system is electrically connected to the output of a power supply module. The system includes a processor, whose input is electrically connected to the output of a data acquisition module, the output of a data transmission module, the input of a data processing and analysis module, the input of a fault handling module, and the input of an adaptive inspection planning module. Through continuous self-learning of fault phenomena, this intelligent inspection system for power lines can improve the efficiency of fault diagnosis during power cable installation, reduce labor costs, and ensure the safety and stability of the power cable installation process.

[0005] Therefore, it is evident that the existing power line inspection technologies described above all employ a single-fault detection mode, detecting only one type of power line fault at a time. This approach cannot detect multiple power line faults simultaneously. Crucially, these technologies only identify the type of power line fault but cannot simultaneously perform real-time location tracking of the fault's location. This necessitates secondary analysis by the power line management department to confirm the specific location of the detected fault. Furthermore, these technologies rely solely on on-site, non-aerial data acquisition mechanisms or single-drone data acquisition mechanisms. The inability to execute multi-view aerial data acquisition reduces the comprehensiveness and reliability of the detection results. Summary of the Invention

[0006] To address technical challenges in this field, this invention provides a data fusion method for power line inspection using multi-machine collaborative control drones. Based on a hardware architecture for multi-machine collaborative control of power line inspection data fusion and an AI-based inspection data fusion model specifically designed for designated power transmission lines, this method enables comprehensive, synchronous, and intelligent analysis of various fault types and their locations within a fixed area (target line segment) of the power transmission line. This enhances the accuracy and efficiency of power line inspection while simultaneously improving its automation and intelligence levels.

[0007] According to the present invention, a data fusion method for power line inspection using a multi-machine cooperative control UAV is provided, the method comprising:

[0008] The ground-based collaborative control station synchronously sends N-view line inspection tasks to N power line transmission line drones, specifying the target line segments. Each view inspection task includes a designated shooting position, gimbal angle, and preset acquisition time, where N is an integer greater than or equal to 3.

[0009] N line inspection drones simultaneously collect N frames of visible light images, N frames of infrared thermal images, and N sets of laser point cloud data according to the N-view line inspection task and transmit them back to the ground collaborative control station.

[0010] At the ground-based collaborative control station, acquire configuration data for each power line transmission line and multiple segment association information for each segment of the target line.

[0011] At the ground-based collaborative control station, an AI inspection data fusion model is adopted, based on the line segment visualization parameter set in N frames of visible light images, the line segment infrared parameter set in N frames of infrared thermal images, N sets of laser point cloud data, the positioning information of N designated shooting positions corresponding to N-view line inspection tasks, the configuration data of each set of power line transmission lines, and multiple sets of segment association information of the target line segment. The model intelligently determines the names of various fault types existing in the target line segment at the preset acquisition time and the fault location information corresponding to each fault type.

[0012] Among them, the AI ​​inspection data fusion model corresponding to the power line transmission line is set to be a convolutional neural network after deep learning, and the number of pooling layers in the convolutional neural network is positively correlated with the number of power lines erected for the power line transmission line.

[0013] Compared with the prior art, the present invention has at least the following five outstanding substantive features:

[0014] Substantive Feature 1: Establishing a hardware architecture for data fusion of UAV power line inspections using multi-drone collaborative control. This hardware architecture includes a single ground collaborative control station and N inspection UAVs that interact with the single ground collaborative control station. The ground collaborative control station synchronously issues N-view inspection tasks for target power line segments to the N inspection UAVs. The N inspection UAVs synchronously collect N sets of power line inspection data according to the N-view inspection tasks. The ground collaborative control station uses artificial intelligence to perform targeted data fusion on the synchronously collected N sets of power line inspection data to intelligently determine the names of various fault types existing in the target line segments at the preset collection time and the corresponding fault location information for each fault type. This completes the synchronous analysis of multiple types of power line faults and their location information, enhancing the accuracy and efficiency of power line inspection while improving the automation and intelligence levels of power line inspection.

[0015] Substantive Feature Two: To intelligently determine the names of various fault types and their corresponding fault location information for each segment of the target power line at a preset collection time, an AI inspection data fusion model corresponding to the power line transmission line is used. This model is a convolutional neural network (CNN) after deep learning. The number of pooling layers in the CNN is positively correlated with the number of power lines erected on the power line transmission line. The number of learning iterations of the CNN exceeds a preset threshold, and the number of learning iterations follows the same trend as the power line erection height. The CNN includes an input layer, a convolutional layer, multiple pooling layers, and a fully connected layer. The convolutional layers use the Sigmoid function as the activation function, and each pooling layer uses the ReLU function. The input layer, convolutional layer, multiple pooling layers, and fully connected layer are sequentially connected. This customized structural design ensures the stability and reliability of the synchronous parsing of multiple power line fault types and their location information by designing different AI inspection data fusion models for different power line transmission lines.

[0016] Substantial Feature Three: In each learning iteration of the convolutional neural network, the names of various fault types existing in a certain line segment of the power transmission line at a certain historical acquisition time, along with the corresponding fault location information, are used as the output data of the convolutional neural network. The N-view line inspection task configured for the certain line segment at the certain historical acquisition time is used as the N-view historical line inspection task. The line segment visualization parameter set in the N frames of visible light images corresponding to the N-view historical line inspection task, the line segment infrared parameter set in the N frames of infrared thermal images, N sets of laser point cloud data, the location information of the N specified shooting locations corresponding to the N-view historical line inspection task, the configuration data of the power transmission line, and the multiple sets of segment association information of the certain line segment are used as the input data of the convolutional neural network to complete the learning process, thereby ensuring the learning effect of the convolutional neural network in each iteration.

[0017] Substantial Feature Four: To intelligently determine the names of various fault types and their corresponding fault location information for each type of fault in the target line segment at the preset acquisition time, various basic data are used. These basic data include the line segment visualization parameter set in N frames of visible light images, the line segment infrared parameter set in N frames of infrared thermal images, N sets of laser point cloud data, the location information of N designated shooting positions corresponding to the N-view line inspection tasks, the configuration data of each set of power line transmission lines, and multiple sets of segment association information for the target line segment. The targeted screening of the above-mentioned basic data further ensures the stability and reliability of the synchronous analysis of multiple types of power line faults and their location information.

[0018] Substantive Feature Five: Specifically, the visual parameter set for line segments in each frame of visible light image includes the C-channel, M-channel, Y-channel, K-channel, coordinate, and depth values ​​of each pixel in the image block occupied by the target line segment, as well as the grayscale gradient, coordinate, and depth gradient values ​​of each edge pixel in the image block occupied by the target line segment. Similarly, the infrared parameter set for line segments in each frame of infrared thermal image includes the red channel values ​​of each pixel in the image block occupied by the target line segment, as well as the red channel values ​​of each edge pixel in the image block occupied by the target line segment. The gradient values ​​are used to set various configuration data for power line transmission lines, including the cross-sectional area of ​​the power lines, the total length of the power lines, the number of power lines, the insulator type code, the tension clamp type code, the suspension clamp type code, and the vibration damper type code. Additionally, multiple segment association information for the target line segment is provided, including the power line erection height, power line erection length, number of insulators, number of tension clamps, number of suspension clamps, and number of vibration dampers for that segment. This completes the customized data structure design for the synchronous analysis of various basic data for multiple types of power line faults and their location information. Attached Figure Description

[0019] The embodiments of the present invention will now be described with reference to the accompanying drawings, wherein:

[0020] Figure 1 is a schematic diagram of the working scenario of a power line inspection data fusion method based on multi-machine collaborative control of a power line inspection drone according to the present invention.

[0021] Figure 2 is a flowchart illustrating the steps of a power line inspection data fusion method based on multi-machine collaborative control of a power line inspection drone according to Embodiment 1 of the present invention.

[0022] Figure 3 is a flowchart illustrating the steps of a power line inspection data fusion method based on multi-machine collaborative control of a power line inspection drone according to Embodiment 2 of the present invention.

[0023] Figure 4 is a flowchart illustrating the steps of a power line inspection data fusion method based on multi-machine collaborative control of a power line inspection drone according to Embodiment 3 of the present invention.

[0024] Figure 5 is a flowchart illustrating the steps of a power line inspection data fusion method based on multi-machine collaborative control of a power line inspection drone according to Embodiment 4 of the present invention.

[0025] Figure 6 is a flowchart illustrating the steps of a power line inspection data fusion method based on multi-machine collaborative control of a power line inspection drone according to Embodiment 5 of the present invention. Detailed Implementation

[0026] Figure 1 illustrates a working scenario of a power line inspection data fusion method based on multi-machine cooperative control using a power line inspection drone, according to the present invention. The deep learning method proposed in this invention belongs to the field of image or video recognition or understanding.

[0027] The specific technical process of this invention is as follows:

[0028] Technical Process A: Build a hardware architecture for data fusion of UAV power line inspection for multi-machine collaborative control. The hardware architecture includes a single ground collaborative control station and N inspection UAVs that interact with the single ground collaborative control station, as shown in Figure 1.

[0029] Specifically, the ground-based collaborative control station synchronously sends N-view inspection tasks for the target line segments of the power transmission line to N inspection drones. The N inspection drones simultaneously collect N sets of power line inspection data according to the N-view inspection tasks. The N sets of power line inspection data provide a data foundation for the subsequent synchronous analysis of multiple types of power line faults and their location information.

[0030] Technical Process B: To intelligently determine the names of various fault types and the corresponding fault location information for each type of fault in the target line segment at the preset collection time; an AI inspection data fusion model corresponding to the power line transmission line is introduced, as shown in Figure 1.

[0031] Specifically, the modeling mechanism of the artificial intelligence model of the present invention can design AI inspection data fusion models with different structures for different power line transmission lines. The modeling mechanism mainly involves the following aspects:

[0032] Firstly, the AI ​​inspection data fusion model corresponding to the power line transmission line is set as a convolutional neural network after deep learning. The number of times the convolutional neural network performs learning exceeds a preset threshold, which shows that the artificial intelligence model of this invention is based on deep learning mode. More importantly, the number of times the convolutional neural network performs learning is the same as the numerical change trend of the set power line erection height of the power line transmission line.

[0033] For example, when the power line of a power transmission line is set to an elevation of 8, the corresponding convolutional neural network performs 800 training iterations; when the power line of a power transmission line is set to an elevation of 10, the corresponding convolutional neural network performs 1000 training iterations; when the power line of a power transmission line is set to an elevation of 15, the corresponding convolutional neural network performs 1500 training iterations; when the power line of a power transmission line is set to an elevation of 20, the corresponding convolutional neural network performs 2000 training iterations, and so on.

[0034] Secondly, the number of pooling layers in a convolutional neural network is positively correlated with the number of power lines erected in a given power line transmission line.

[0035] For example, when the number of power lines in a power transmission line is set to 3, the number of pooling layers in the selected convolutional neural network is 6; when the number of power lines in a power transmission line is set to 4, the number of pooling layers in the selected convolutional neural network is 8; when the number of power lines in a power transmission line is set to 5, the number of pooling layers in the selected convolutional neural network is 10; when the number of power lines in a power transmission line is set to 6, the number of pooling layers in the selected convolutional neural network is 12, and so on.

[0036] Thirdly: The convolutional neural network used includes an input layer, a convolutional layer, multiple pooling layers, and a fully connected layer. The convolutional layer uses the Sigmoid function as the activation function, and each pooling layer uses the ReLU function as the activation function. The input layer, convolutional layer, multiple pooling layers, and fully connected layer are connected sequentially.

[0037] Fourthly, in each learning process of the convolutional neural network, the names of various fault types existing in a certain line segment of the power transmission line at a certain historical acquisition time, as well as the fault location information corresponding to each fault type, are used as the output data of the convolutional neural network. The N-view line inspection task configured for the certain line segment at the certain historical acquisition time is used as the N-view historical line inspection task. The line segment visualization parameter set in the N frames of visible light images corresponding to the N-view historical line inspection task, the line segment infrared parameter set in the N frames of infrared thermal images, N sets of laser point cloud data, the location information of the N specified shooting positions corresponding to the N-view historical line inspection task, the configuration data of the power transmission line, and the multiple sets of segment association information of the certain line segment are used as the input data of the convolutional neural network to complete the learning process, thereby ensuring the learning effect of the convolutional neural network in each learning process.

[0038] It can be seen that, through the dynamic modeling design of the above aspects, the stability and reliability of the synchronous analysis of multiple types of power line faults and their location information are guaranteed.

[0039] Technical Process C: To intelligently determine the names of various fault types and the corresponding fault location information for each type of fault in the target line segment at the preset data acquisition time, various basic data were used.

[0040] Specifically, as shown in Figure 1, the various basic data include the line segment visualization parameter set in N frames of visible light images, the line segment infrared parameter set in N frames of infrared thermal images, N sets of laser point cloud data, and other auxiliary data. The other auxiliary data includes the positioning information of N designated shooting locations corresponding to the N-view line inspection tasks, the configuration data of each set of power line transmission lines, and multiple sets of segment association information of the target line segments.

[0041] More specifically, the line segment visualization parameter set in each frame of visible light image consists of the C channel value, M channel value, Y channel value, K channel value, coordinate value, and depth value of each pixel point of the target line segment occupying the image block in that frame of visible light image, as well as the grayscale gradient value, coordinate value, and depth gradient value of each edge pixel point of the target line segment occupying the image block in that frame of visible light image. The line segment infrared parameter set in each frame of infrared thermal image consists of the red channel value of each pixel point of the target line segment occupying the image block in that frame of infrared thermal image, as well as the red channel gradient value of each edge pixel point of the target line segment occupying the image block in that frame of infrared thermal image.

[0042] More specifically, the configuration data for each power line transmission line includes setting the cross-sectional area of ​​the power line, the total length of the power line, the number of power lines, the insulator type code, the tension clamp type code, the suspension clamp type code, and the vibration damper type code.

[0043] More specifically, the segmented information of the target line segment includes the power line erection height, power line erection length, number of insulators, number of tension clamps, number of suspension clamps, and number of vibration dampers for the target line segment.

[0044] In this way, through targeted screening of the above-mentioned basic data and customized data structure design, the stability and reliability of synchronous analysis of multiple types of power line faults and their location information are further guaranteed.

[0045] Technical Process D: Based on the hardware architecture for multi-machine collaborative control of UAV power line inspection data fusion built in Technical Process A, the AI ​​inspection data fusion model designed specifically for power line transmission lines in Technical Process B is used. Based on the various basic data selected in Technical Process C, the intelligent judgment of the names of various fault types and the fault location information corresponding to each fault type in the target line segment at the preset collection time is performed, as shown in Figure 1.

[0046] Specifically, the names of various fault types determined by the intelligent system and the fault location information corresponding to each fault type are all represented in binary numerical form. When the names of various fault types obtained by the intelligent system are all strings composed of empty characters, it is determined that there are no fault types in the target line segment at the preset acquisition time.

[0047] It is evident that by coordinating the above-mentioned multiple technical processes, more comprehensive basic data can be used, and AI inspection data fusion models with different structures dynamically designed for different power line transmission lines can be employed to complete the synchronous analysis of multiple types of power line faults and their location information. This overcomes the technical problem that the accuracy, efficiency, and comprehensiveness of power inspection results cannot be improved simultaneously in existing technologies.

[0048] The key points of this invention are: simultaneous improvement in the accuracy, efficiency and comprehensiveness of power line inspection results; hardware architecture for data fusion of UAV power line inspection for multi-machine collaborative control; AI inspection data fusion models with different structures for dynamic design of different power lines and transmission lines; targeted screening of various basic data; and customized data structure design.

[0049] The following will describe in detail, by way of an embodiment, a power line inspection data fusion method based on multi-machine collaborative control of the present invention.

[0050] Example 1

[0051] Figure 2 is a flowchart illustrating the steps of a power line inspection data fusion method based on multi-machine collaborative control of a power line inspection drone according to Embodiment 1 of the present invention.

[0052] As shown in Figure 2, the power line inspection data fusion method based on multi-machine cooperative control of UAVs includes the following specific steps:

[0053] Step S21: The ground collaborative control station synchronously sends N-view line inspection tasks for the target line segments of the power transmission line to N line inspection drones. Each view line inspection task includes a specified shooting position, gimbal angle and preset acquisition time, where N is an integer greater than or equal to 3.

[0054] For example, as shown in Figure 1, with the ground plane as the base plane, N evenly spaced positions on a circle surrounding the target line segment within a set horizontal plane parallel to the ground plane are the N airspace shooting positions for N line inspection drones to simultaneously perform shooting, i.e., data collection, at a preset collection time. The shooting target is the target line segment, generally the tower erected in the target line segment, and the gimbal angles of the gimbals at the bottom of the N line inspection drones used to fix the corresponding aerial photography devices are equal. For example, as shown in Figure 1, N can be 6, then the angle between two adjacent airspace shooting positions on the horizontal plane and the target line segment is 60 degrees, and N can also be 12, then the angle between two adjacent airspace shooting positions on the horizontal plane and the target line segment is 30 degrees.

[0055] As a further example, the angle between two adjacent airspace shooting positions on the horizontal plane relative to the target line segment is calculated as follows: the two adjacent airspace shooting positions are respectively designated as the first airspace shooting position and the second airspace shooting position. The first airspace shooting position and the center point of the three-dimensional model formed by the target line segment form a first straight line, and the second airspace shooting position and the center point of the three-dimensional model formed by the target line segment form a second straight line. The angle between the first straight line and the second straight line is 60 degrees (when N is 6) or 30 degrees (when N is 12).

[0056] Step S22: N line inspection drones simultaneously collect N frames of visible light images, N frames of infrared thermal images, and N sets of laser point cloud data according to the N-view line inspection task and transmit them back to the ground collaborative control station.

[0057] Specifically, N line inspection drones simultaneously collect N frames of visible light images, N frames of infrared thermal images, and N sets of laser point cloud data according to the N-view line inspection task, and wirelessly transmit the N frames of visible light images, N frames of infrared thermal images, and N sets of laser point cloud data back to the ground collaborative control station.

[0058] Step S23: Obtain the configuration data of each set of power line transmission lines and multiple segment association information of the target line segments at the ground collaborative control station;

[0059] Specifically, two different content acquisition components can be used to acquire each set of configuration data for the set power line transmission line and multiple sets of segment association information for the target line segment, respectively.

[0060] Step S24: At the ground collaborative control station, the AI ​​inspection data fusion model corresponding to the power line transmission line is set based on the line segment visualization parameter set in N frames of visible light images, the line segment infrared parameter set in N frames of infrared thermal images, N sets of laser point cloud data, the positioning information of N designated shooting positions corresponding to the N-view line inspection tasks, the configuration data of each set of the power line transmission line, and the multiple sets of segment association information of the target line segment. The model intelligently determines the names of various fault types existing in the target line segment at the preset acquisition time and the fault location information corresponding to each fault type.

[0061] Specifically, the line segment visualization parameter set in the N-frame visible light image is derived from the targeted selection of visualization parameters for each pixel of the target line segment in the image block in the N-frame visible light image, and the line segment infrared parameter set in the N-frame infrared thermal image is derived from the targeted selection of visualization parameters for each pixel of the target line segment in the image block in the N-frame infrared thermal image.

[0062] Specifically, the N sets of laser point cloud data originate from the three-dimensional positioning information of each entity point on the three-dimensional structure of the target line segment completed by using the laser scanning device on the UAV;

[0063] Among them, the AI ​​inspection data fusion model corresponding to the power line transmission line is set to be a convolutional neural network after deep learning, and the number of pooling layers in the convolutional neural network is positively correlated with the number of power lines erected for the power line transmission line.

[0064] For example, the positive correlation between the number of pooling layers in a convolutional neural network and the number of power lines in a power transmission line includes: when the number of power lines in a power transmission line is set to 3, the number of pooling layers in the selected convolutional neural network is 6; when the number of power lines in a power transmission line is set to 4, the number of pooling layers in the selected convolutional neural network is 8; when the number of power lines in a power transmission line is set to 5, the number of pooling layers in the selected convolutional neural network is 10; when the number of power lines in a power transmission line is set to 6, the number of pooling layers in the selected convolutional neural network is 12, and so on.

[0065] Among them, the N designated shooting positions corresponding to the N-view line-following task are each position evenly spaced on a circle that is divided around the target line in a set horizontal plane in the airspace, and the N gimbal angle values ​​corresponding to the N-view line-following task are equal, and the N preset acquisition time values ​​corresponding to the N-view line-following task are equal.

[0066] Among them, the line segment visualization parameter set in each frame of visible light image is the C channel value, M channel value, Y channel value, K channel value, coordinate value and depth value of each pixel point of the target line segment occupying the image block in the visible light image frame, as well as the gray level gradient value, coordinate value and depth gradient value of each edge pixel point of the target line segment occupying the image block.

[0067] Specifically, the values ​​of the C channel, M channel, Y channel, and K channel of each pixel are all between 0 and 255, and the coordinate value of each pixel is composed of the horizontal and vertical coordinate values ​​of the pixel. The C channel, M channel, Y channel, K channel, coordinate values, and depth values ​​of each pixel can all be represented using binary values.

[0068] Among them, the infrared parameter set of the line segment in each frame of infrared thermal image is the red channel value of each pixel point of the target line segment in the image block occupied by the target line segment and the red channel gradient value of each edge pixel point of the image block occupied by the target line segment.

[0069] Specifically, the red channel value of each pixel ranges from 0 to 255, and can also be represented using binary values;

[0070] Among them, the configuration data of the set power line transmission line and the segment association information of the target line segment obtained at the ground collaborative control station include: the configuration data of the set power line transmission line includes the cross-sectional area of ​​the power line, the total length of the power line, the number of power lines, the insulator type code value, the tension clamp type code value, the suspension clamp type code value and the vibration damper type code value.

[0071] Specifically, different insulator types have different insulator type codes, different tension clamp types have different tension clamp type codes, different suspension clamp types have different suspension clamp type codes, and different vibration damper types have different vibration damper type codes.

[0072] Among them, the acquisition of configuration data for each set of power line transmission lines and multiple segment association information for the target line segments at the ground collaborative control station also includes: multiple segment association information for the target line segments including the power line erection height, power line erection length, number of insulators, number of tension clamps, number of suspension clamps and number of vibration dampers for the target line segments.

[0073] Among them, when the names of various fault types obtained by intelligent judgment are all strings composed of empty characters, it is determined that there are no fault types in the target line segment at the preset collection time;

[0074] Thus, the actual number of various fault types obtained is between 0 and M, where M is the maximum possible number of various fault types. The maximum number of various fault types that are intelligently judged and output are the binary code streams whose ASCII codes of each type name are sequentially connected. When a certain type of fault does not exist, the ASCII code of the binary code stream corresponding to that type of fault is an empty character.

[0075] Among them, the AI ​​inspection data fusion model for power line transmission lines is set as a convolutional neural network after deep learning, including: the number of times the convolutional neural network performs learning exceeds a preset threshold, and the number of times the convolutional neural network performs learning has the same numerical trend as the power line erection height of the power line transmission line.

[0076] Specifically, if the number of times the convolutional neural network performs learning exceeds a preset threshold, it indicates that the convolutional neural network used in this invention has undergone deep learning before being converted into an AI inspection data fusion model corresponding to a set power line transmission line.

[0077] For example, the number of times the convolutional neural network performs training corresponds to the numerical trend of the set power line erection height of the power transmission line, including: when the set power line erection height is 8, the corresponding number of times the convolutional neural network performs training is 800; when the set power line erection height is 10, the corresponding number of times the convolutional neural network performs training is 1000; when the set power line erection height is 15, the corresponding number of times the convolutional neural network performs training is 1500; when the set power line erection height is 20, the corresponding number of times the convolutional neural network performs training is 2000, and so on.

[0078] The positive correlation between the number of pooling layers in the convolutional neural network and the number of power lines erected by the power line transmission line includes: the convolutional neural network includes an input layer, a convolutional layer, multiple pooling layers and a fully connected layer, the convolutional layer uses the Sigmoid function as the activation function, each pooling layer uses the ReLU function as the activation function, and the input layer, convolutional layer, multiple pooling layers and fully connected layer are connected sequentially;

[0079] In each learning iteration of the convolutional neural network, the names of various fault types existing in a certain line segment of a power transmission line at a certain historical acquisition time, as well as the fault location information corresponding to each fault type, are used as the output data of the convolutional neural network. The N-view line inspection task configured for the certain line segment at the certain historical acquisition time is used as the N-view historical line inspection task. The line segment visualization parameter set in the N frames of visible light images corresponding to the N-view historical line inspection task, the line segment infrared parameter set in the N frames of infrared thermal images, N sets of laser point cloud data, the location information of the N specified shooting positions corresponding to the N-view historical line inspection task, the configuration data of the power transmission line, and the multiple sets of segment association information of the certain line segment are used as the input data of the convolutional neural network to complete this learning iteration.

[0080] Specifically, numerical simulation can be used to test and simulate each learning process performed by the convolutional neural network.

[0081] Example 2

[0082] Figure 3 is a flowchart illustrating the steps of a power line inspection data fusion method based on multi-machine collaborative control of a power line inspection drone according to Embodiment 2 of the present invention.

[0083] As shown in Figure 3, unlike the embodiment in Figure 2, the method for data fusion of power line inspection by a multi-machine collaborative control UAV, in which an AI inspection data fusion model corresponding to the power line transmission line is set at the ground collaborative control station, based on the line segment visualization parameter set in N frames of visible light images, the line segment infrared parameter set in N frames of infrared thermal images, N sets of laser point cloud data, the positioning information of N designated shooting positions corresponding to the N-view inspection tasks, the configuration data of each set of the power line transmission line, and multiple sets of segment association information of the target line segment, intelligently determines the names of various fault types existing in the target line segment at the preset acquisition time and the fault location information corresponding to each fault type, that is, after step S24, the method further includes:

[0084] Step S25: Receive the names of various fault types and the corresponding fault location information of each type of fault at the preset collection time for the target line segment at the ground collaborative control station, and execute the synchronous display of the names of various fault types and the corresponding fault location information of each type of fault at the preset collection time for the target line segment at the ground collaborative control station.

[0085] Among them, the synchronous display of the names of various fault types and the corresponding fault location information of each type of fault at the preset collection time of the target line segment at the ground collaborative control station includes: using a giant screen display device set at the ground collaborative control station to synchronously display the names of various fault types and the corresponding fault location information of each type of fault at the preset collection time of the target line segment.

[0086] Example 3

[0087] Figure 4 is a flowchart illustrating the steps of a power line inspection data fusion method based on multi-machine collaborative control of a power line inspection drone according to Embodiment 3 of the present invention.

[0088] As shown in Figure 4, unlike the embodiment in Figure 2, in the above-described method for data fusion of power line inspection by a multi-machine collaborative control UAV, an AI inspection data fusion model corresponding to the power line transmission line is used at the ground collaborative control station. This model is based on the line segment visualization parameter set in N frames of visible light images, the line segment infrared parameter set in N frames of infrared thermal images, N sets of laser point cloud data, the positioning information of N designated shooting positions corresponding to the N-view inspection tasks, the configuration data of each set of the power line transmission line, and multiple sets of segment association information of the target line segment. After intelligently determining the names of various fault types and the fault location information corresponding to each fault type in the target line segment at the preset acquisition time, i.e., after step S24, the method further includes:

[0089] Step S26: At the ground collaborative control station, the names of various fault types existing in the target line segment at the preset collection time, as well as the fault location information corresponding to each fault type, are wirelessly transmitted to the remote line patrol monitoring server.

[0090] The process of wirelessly transmitting the names of various fault types and the corresponding fault location information of each fault type to the remote patrol and monitoring server at the ground collaborative control station includes: packaging the names of various fault types, the corresponding fault location information of each fault type, and the preset collection time into the same network data packet for wireless transmission to the remote patrol and monitoring server.

[0091] Example 4

[0092] Figure 5 is a flowchart illustrating the steps of a power line inspection data fusion method based on multi-machine collaborative control of a power line inspection drone according to Embodiment 4 of the present invention.

[0093] As shown in Figure 5, unlike the embodiment in Figure 2, in the above-described method for data fusion of power line inspection drones based on multi-machine collaborative control, before the ground collaborative control station synchronously issues the N-view inspection task of setting the target line segment of the power transmission line to N inspection drones, that is, before step S21, the method further includes:

[0094] Step S27: Perform multiple learning operations on the convolutional neural network at the ground collaborative control station to obtain the convolutional neural network after deep learning and use it as the output of the AI ​​inspection data fusion model corresponding to the set power line transmission line. The number of learning operations performed by the convolutional neural network exceeds the preset threshold, and the numerical change trend of the number of learning operations performed by the convolutional neural network is the same as that of the power line erection height of the set power line transmission line.

[0095] The process of performing multiple learning operations on the convolutional neural network at the ground-based collaborative control station to obtain a convolutional neural network after deep learning and outputting it as the AI ​​inspection data fusion model corresponding to the designated power line transmission line includes: storing various model parameters of the AI ​​inspection data fusion model corresponding to the designated power line transmission line to complete the model storage of the AI ​​inspection data fusion model corresponding to the designated power line transmission line.

[0096] Example 5

[0097] Figure 6 is a flowchart illustrating the steps of a power line inspection data fusion method based on multi-machine collaborative control of a power line inspection drone according to Embodiment 5 of the present invention.

[0098] As shown in Figure 6, unlike the embodiment in Figure 2, in the power line inspection data fusion method based on multi-machine collaborative control, after N inspection drones simultaneously collect N frames of visible light images, N frames of infrared thermal images, and N sets of laser point cloud data according to N perspective inspection tasks and transmit them back to the ground collaborative control station, that is, after step S22, the method further includes:

[0099] Step S28: At the ground-based collaborative control station, perform image analysis on N frames of visible light images to obtain a set of line segment visualization parameters in the N frames of visible light images, and perform image analysis on N frames of infrared thermal images to obtain a set of line segment infrared parameters in the N frames of infrared thermal images.

[0100] Specifically, at the ground-based collaborative control station, image analysis is performed on N frames of visible light images to obtain a set of line segment visualization parameters in the N frames of visible light images. Image analysis is also performed on N frames of infrared thermal images to obtain a set of line segment infrared parameters in the N frames of infrared thermal images. This includes: performing image analysis on each frame of visible light images to obtain a set of line segment visualization parameters in that frame of visible light images, and performing image analysis on each frame of infrared thermal images to obtain a set of line segment infrared parameters in that frame of infrared thermal images.

[0101] Next, the various method embodiments of the present invention will be described in detail.

[0102] In a power line inspection data fusion method based on multi-machine cooperative control according to various embodiments of the present invention:

[0103] Various types of faults include power line wear, power line arc burns, power line corrosion, insulator skirt corrosion, insulator sheath cracks, insulator surface flashover marks, insulator end sealant cracks, insulator surface contamination, tension clamp cracks, suspension clamp hull cracks, and vibration damper misalignment.

[0104] Therefore, as can be seen from the present invention, as mentioned above, M is the maximum value of various possible fault types. That is, in the present invention, the value of M can be 11, which corresponds to 11 types, namely, power line wear, power line arc burn, power line corrosion, insulator skirt corrosion, insulator sheath cracking, insulator surface flashover marks, insulator end sealant cracking, insulator surface contamination, tension clamp crack, suspension clamp hull crack, and vibration damper offset. Of course, the value of M is not limited to 11, and can also be an integer greater than 11.

[0105] Among them, the fault location information corresponding to each type of fault is the location information of the occurrence position of the fault type in the target line segment, and the location information of the occurrence position of the fault type in the target line segment is the three-dimensional relative distance of the occurrence position of the fault type in the target line segment relative to the location of the reference line inspection drone. The reference line inspection drone is one of N line inspection drones.

[0106] Specifically, once one of the N line inspection drones is selected as the reference line inspection drone, no other line inspection drone will be selected as the reference line inspection drone. The location information of each type of occurrence in the target line segment is analyzed with the reference line inspection drone as the reference point.

[0107] In each frame of the visible light image, each pixel has C channel, M channel, Y channel and K channel values ​​in the CMYK space, and each pixel in each frame of the infrared thermal image has R channel value, i.e. red channel value, in the RGB space.

[0108] Specifically, the C channel value, M channel value, Y channel value, and K channel value are the cyan channel value, magenta channel value, yellow channel value, and black channel value, respectively;

[0109] In this context, each pixel is taken as the target pixel, and its set type channel gradient value is the standard deviation of its set type channel value and the set type channel values ​​corresponding to each of its neighboring pixels.

[0110] In a power line inspection data fusion method based on multi-machine cooperative control according to various embodiments of the present invention:

[0111] The number of times the convolutional neural network performs learning is the same as the numerical trend of the set power line erection height of the power line transmission line, including: using a number change curve to represent the numerical trend of the number of times the convolutional neural network performs learning, and using a height change curve to represent the numerical trend of the set power line erection height of the power line transmission line.

[0112] The learning number of the convolutional neural network is the same as the numerical trend of the power line erection height of the set power line transmission line. It also includes: performing curve length normalization processing on the curve of number change and the curve of height change respectively to obtain the first conversion curve and the second conversion curve of equal length.

[0113] Among them, the number of learning operations performed by the convolutional neural network is the same as the numerical trend of the set power line erection height of the power line transmission line, and also includes: the first conversion curve and the second conversion curve of equal length completely overlap;

[0114] Specifically, a CPLD chip was selected to perform curve length normalization processing on the number change curve and the height change curve to obtain a first conversion curve and a second conversion curve of equal length.

[0115] For example, selecting a CPLD chip to perform curve length normalization processing on the number change curve and the height change curve respectively to obtain a first conversion curve and a second conversion curve of equal length includes: the CPLD chip is programmed using the VHDL language.

[0116] And in a power line inspection data fusion method based on multi-machine cooperative control of a UAV according to various embodiments of the present invention:

[0117] At the ground-based collaborative control station, an AI inspection data fusion model corresponding to the power line transmission line is adopted. This model is based on the line segment visualization parameter set in N frames of visible light images, the line segment infrared parameter set in N frames of infrared thermal images, N sets of laser point cloud data, the positioning information of N designated shooting positions corresponding to N-view line inspection tasks, the configuration data of the power line transmission line, and multiple sets of segment association information of the target line segment. The model intelligently determines the names of various fault types existing in the target line segment at the preset acquisition time and the fault location information corresponding to each fault type. The fault types existing in the target line segment at the preset acquisition time are all represented by binary numerical values ​​of ASCII codes, and the fault location information corresponding to each fault type is also represented by binary numerical values.

[0118] Specifically, for the output or input content of the AI ​​inspection data fusion model that does not use binary numerical representation, programmable logic devices can be used to convert the numerical format to obtain the corresponding binary value.

[0119] Among them, at the ground collaborative control station, an AI inspection data fusion model corresponding to the power line transmission line is adopted. Based on the line segment visualization parameter set in N frames of visible light images, the line segment infrared parameter set in N frames of infrared thermal images, N sets of laser point cloud data, the positioning information of N designated shooting positions corresponding to N-view line inspection tasks, the configuration data of the power line transmission line, and multiple sets of segment association information of the target line segment, it intelligently judges the names of various fault types existing in the target line segment at the preset acquisition time and the fault positioning information corresponding to each fault type. It also includes: the line segment visualization parameter set in N frames of visible light images, the line segment infrared parameter set in N frames of infrared thermal images, the N sets of laser point cloud data, the positioning information of N designated shooting positions corresponding to N-view line inspection tasks, the configuration data of the power line transmission line, and the multiple sets of segment association information of the target line segment are all represented in binary numerical form.

[0120] Specifically, the line segment visualization parameter set in N frames of visible light images, the line segment infrared parameter set in N frames of infrared thermal images, N sets of laser point cloud data, the positioning information of N designated shooting positions corresponding to the N-view line inspection tasks, the configuration data of each set of power line transmission lines, and the multiple sets of segment association information of the target line segments are input in parallel into the AI ​​inspection data fusion model corresponding to the set power line transmission lines. The AI ​​inspection data fusion model corresponding to the set power line transmission lines is then executed to obtain the names of various fault types existing in the target line segments at the preset acquisition time and the fault location information corresponding to each fault type, as output by the AI ​​inspection data fusion model corresponding to the set power line transmission lines.

[0121] Specifically, an ASIC chip can be selected to execute the parallel input of the line segment visualization parameter set in N frames of visible light images, the line segment infrared parameter set in N frames of infrared thermal images, N sets of laser point cloud data, the positioning information of N specified shooting positions corresponding to the N-view line inspection task, the configuration data of each set of power line transmission lines, and multiple sets of segment association information of the target line segment.

[0122] In addition, the following technical content can be cited to further highlight the essential features of the present invention:

[0123] The positive correlation between the number of pooling layers in a convolutional neural network and the number of power lines erected by a given power line transmission line also includes: using a content transformation function to represent the content transformation relationship between the number of pooling layers in a convolutional neural network and the number of power lines erected by a given power line transmission line;

[0124] The content conversion function used to represent the positive correlation between the number of pooling layers in the convolutional neural network and the number of power lines in the set power transmission line includes: in the content conversion function, the number of power lines in the set power transmission line is the input content of the content conversion function, and the number of pooling layers in the convolutional neural network that is positively correlated with the number of power lines in the set power transmission line is the output content of the content conversion function;

[0125] For example, in the content conversion function, the number of power lines erected by the power line transmission line is set as the input content of the content conversion function, and the number of pooling layers in the convolutional neural network that is positively correlated with the number of power lines erected by the power line transmission line is the output content of the content conversion function. This includes: optionally using a GAL device to test and simulate the content conversion process of the content conversion function.

[0126] Those skilled in the art will readily understand the additional advantages and modifications. Therefore, the invention in a broader sense is not limited to the specific details and representative embodiments shown and described herein. Thus, various modifications can be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.

Claims

1. A data fusion method for power line inspection using a multi-machine collaborative control UAV, characterized in that, The method includes: a ground-based collaborative control station simultaneously issuing N-view inspection tasks for a target segment of a power transmission line to N inspection drones. Each inspection task includes a specified shooting position, gimbal angle, and preset acquisition time, where N is an integer greater than or equal to 3. The N inspection drones simultaneously acquire N frames of visible light images, N frames of infrared thermal images, and N sets of laser point cloud data according to the N-view inspection tasks and transmit them back to the ground-based collaborative control station. At the ground-based collaborative control station, configuration data for the specified power transmission line and multiple sets of segment association information for the target line segment are obtained. At the ground-based collaborative control station, an AI inspection data fusion model corresponding to the specified power transmission line is used based on the N frames of visible light images. The system includes a set of line segment visualization parameters in optical images, a set of line segment infrared parameters in N frames of infrared thermal images, N sets of laser point cloud data, positioning information of N designated shooting locations corresponding to N-view line inspection tasks, configuration data of each set of power transmission lines, and multiple sets of segment association information of the target line segments. It intelligently determines the names of various fault types existing in the target line segments at the preset acquisition time and the fault location information corresponding to each fault type. Among them, the AI ​​inspection data fusion model corresponding to the power transmission lines is a convolutional neural network after deep learning, and the number of pooling layers in the convolutional neural network is positively correlated with the number of power lines erected for the power transmission lines.

2. The power line inspection data fusion method based on multi-machine cooperative control of unmanned aerial vehicles (UAVs) as described in claim 1, characterized in that: The N designated shooting positions for each N-view line-following task are evenly spaced positions on a circle surrounding the target line segment within a set horizontal plane in the airspace. The N gimbal angle values ​​and the N preset acquisition times for each N-view line-following task are also equal. The line segment visualization parameter set for each visible light image frame includes the C-channel, M-channel, Y-channel, K-channel, coordinate, and depth values ​​of each pixel in the image block occupied by the target line segment, as well as the grayscale gradient, coordinate, and depth gradient values ​​of each edge pixel in the image block occupied by the target line segment. The infrared parameter set for each line segment in each infrared thermal image frame includes the red channel values ​​of each pixel in the image block occupied by the target line segment and the red channel gradient values ​​of each edge pixel in the image block occupied by the target line segment. The configuration data of the set power line transmission line and the segment association information of the target line segment obtained at the ground collaborative control station include: the configuration data of the set power line transmission line includes the cross-sectional area of ​​the power line, the total length of the power line, the number of power lines, the insulator type code value, the tension clamp type code value, the suspension clamp type code value, and the vibration damper type code value; the segment association information of the target line segment also includes: the power line erection height, the power line erection length, the number of insulators, the number of tension clamps, the number of suspension clamps, and the number of vibration dampers of the target line segment; when the names of various fault types obtained by intelligent judgment are all strings composed of empty characters, it is determined that there are no fault types in the target line segment at the preset acquisition time.

3. The power line inspection data fusion method based on multi-machine cooperative control of unmanned aerial vehicles (UAVs) as described in claim 2, characterized in that: The AI ​​inspection data fusion model for power transmission lines is set as a convolutional neural network (CNN) after deep learning. This includes: the number of times the CNN performs learning exceeds a preset threshold, and the number of times the CNN performs learning follows the same trend as the numerical change of the power line's erection height. The number of pooling layers in the CNN is positively correlated with the number of power lines in the CNN. The CNN includes an input layer, convolutional layers, multiple pooling layers, and a fully connected layer. The convolutional layers use the Sigmoid function as the activation function, and each pooling layer uses the ReLU function as the activation function. The input layer, convolutional layers, multiple pooling layers, and fully connected layer are sequentially connected. In each learning iteration of the CNN, known data is processed. The system uses the names of various fault types existing in a certain line segment of a power transmission line at a certain historical acquisition time, along with the corresponding fault location information for each fault type, as the output data of a convolutional neural network. It also uses the N-view line inspection task configured for that line segment at that historical acquisition time as the N-view historical line inspection task. The system uses the line segment visualization parameter set from N frames of visible light images, the line segment infrared parameter set from N frames of infrared thermal images, N sets of laser point cloud data, the location information of N designated shooting locations corresponding to the N-view historical line inspection task, the configuration data of the power transmission line, and multiple sets of segment association information for that line segment as the input data of the convolutional neural network to complete this learning process.

4. The data fusion method for power line inspection using a multi-machine cooperative control UAV as described in claim 3, characterized in that, At the ground-based collaborative control station, an AI inspection data fusion model corresponding to the power line transmission line is used. This model is based on a set of line segment visualization parameters from N frames of visible light images, a set of line segment infrared parameters from N frames of infrared thermal images, N sets of laser point cloud data, positioning information of N designated shooting locations corresponding to N-view inspection tasks, configuration data of the power line transmission line, and multiple sets of segment association information of the target line segment. After intelligently determining the names of various fault types and the corresponding fault location information of each fault type in the target line segment at a preset acquisition time, the method further includes: receiving various fault types in the target line segment at the preset acquisition time at the ground-based collaborative control station. The fault type names and corresponding fault location information for each type of fault are displayed synchronously at the ground collaborative control station. This synchronous display includes using a large-screen display device located at the ground collaborative control station to display the fault type names and corresponding fault location information for each type of fault on the target line segment at the preset acquisition time.

5. The data fusion method for power line inspection using a multi-machine cooperative control UAV as described in claim 3, characterized in that, At the ground-based collaborative control station, an AI inspection data fusion model is used to intelligently determine the names of various fault types and their corresponding fault location information at a preset acquisition time for the target line segment. This is based on the line segment visualization parameter set in N frames of visible light images, the line segment infrared parameter set in N frames of infrared thermal images, N sets of laser point cloud data, the positioning information of N designated shooting locations corresponding to the N-view line inspection tasks, the configuration data of the power line transmission line, and multiple sets of segment association information for the target line segment. The method further includes: at the ground-based collaborative control station, the target line... The names of various fault types existing in the target line segment at the preset collection time, as well as the corresponding fault location information, are wirelessly transmitted to the remote line patrol monitoring server. Specifically, the wireless transmission of the names of various fault types existing in the target line segment at the preset collection time, as well as the corresponding fault location information, to the remote line patrol monitoring server at the ground collaborative control station includes: packaging the names of various fault types existing in the target line segment at the preset collection time, the corresponding fault location information, and the preset collection time into the same network data packet for wireless transmission to the remote line patrol monitoring server.

6. The power line inspection data fusion method based on multi-machine cooperative control of unmanned aerial vehicles (UAVs) as described in claim 3, characterized in that, Before the ground-based collaborative control station synchronously issues N-view line inspection tasks for the target power line transmission line segments to N inspection drones, the method further includes: performing multiple learning operations on a convolutional neural network at the ground-based collaborative control station to obtain a convolutional neural network after deep learning, which is then used as the output of the AI ​​inspection data fusion model corresponding to the target power line transmission line. The number of learning operations performed by the convolutional neural network exceeds a preset threshold, and the numerical trend of the number of learning operations performed by the convolutional neural network is the same as the numerical trend of the power line erection height of the target power line transmission line. The step of performing multiple learning operations on the convolutional neural network at the ground-based collaborative control station to obtain a convolutional neural network after deep learning, which is then used as the output of the AI ​​inspection data fusion model corresponding to the target power line transmission line, includes: storing various model parameters of the AI ​​inspection data fusion model corresponding to the target power line transmission line to complete the model storage of the AI ​​inspection data fusion model corresponding to the target power line transmission line.

7. The power line inspection data fusion method based on multi-machine cooperative control of unmanned aerial vehicles (UAVs) as described in claim 3, characterized in that, After N UAVs simultaneously collect N frames of visible light images, N frames of infrared thermal images, and N sets of laser point cloud data according to N perspectives for line inspection tasks and transmit them back to the ground collaborative control station, the method further includes: performing image analysis on the N frames of visible light images at the ground collaborative control station to obtain a set of line segment visualization parameters in the N frames of visible light images, and performing image analysis on the N frames of infrared thermal images to obtain a set of line segment infrared parameters in the N frames of infrared thermal images; wherein, performing image analysis on the N frames of visible light images at the ground collaborative control station to obtain a set of line segment visualization parameters in the N frames of visible light images, and performing image analysis on the N frames of infrared thermal images to obtain a set of line segment infrared parameters in the N frames of infrared thermal images includes: performing image analysis on each frame of visible light image to obtain a set of line segment visualization parameters in that frame of visible light image, and performing image analysis on each frame of infrared thermal image to obtain a set of line segment infrared parameters in that frame of infrared thermal image.

8. A data fusion method for power line inspection using a multi-machine cooperative control UAV, as described in any one of claims 3-7, characterized in that: The various fault types include power line wear, power line arc burns, power line corrosion, insulator skirt corrosion pits, insulator sheath cracks, flashover marks on the insulator surface, cracking of the sealant at the insulator end, contamination on the insulator surface, tension clamp cracks, suspension clamp hull cracks, and vibration damper misalignment. The fault location information for each fault type includes the location of the fault type within the target line segment, and the location information relative to the reference line inspection drone. The three-dimensional relative distance of the location, the reference line-following UAV is one of N line-following UAVs; wherein, each pixel in each frame of visible light image has C channel value, M channel value, Y channel value and K channel value in CMYK space, and each pixel in each frame of infrared thermal image has R channel value in RGB space, i.e. red channel value; wherein, each pixel is taken as a target pixel, and its set type channel gradient value is the standard deviation of its set type channel value and the set type channel values ​​corresponding to each of its neighboring pixels.

9. A data fusion method for power line inspection using a multi-machine cooperative control UAV, as described in any one of claims 3-7, characterized in that: The principle that the number of learning iterations performed by the convolutional neural network (CNN) is the same as the numerical trend of the power line erection height of the power transmission line includes: using a number-of-learnings variation curve to represent the numerical trend of the number of learning iterations performed by the CNN, and using a height variation curve to represent the numerical trend of the power line erection height of the power transmission line; furthermore, the principle that the number of learning iterations performed by the CNN is the same as the numerical trend of the power line erection height of the power transmission line also includes: performing curve length normalization processing on the number-of-learnings variation curve and the height variation curve respectively to obtain a first conversion curve and a second conversion curve of equal length; furthermore, the principle that the number of learning iterations performed by the CNN is the same as the numerical trend of the power line erection height of the power transmission line also includes: the first conversion curve and the second conversion curve of equal length completely overlap; and selecting an ASIC chip to perform curve length normalization processing on the number-of-learnings variation curve and the height variation curve respectively to obtain a first conversion curve and a second conversion curve of equal length.

10. A data fusion method for power line inspection using a multi-machine cooperative control UAV, as described in any one of claims 3-7, characterized in that: At the ground-based collaborative control station, an AI inspection data fusion model corresponding to the power line transmission line is adopted. This model is based on the line segment visualization parameter set in N frames of visible light images, the line segment infrared parameter set in N frames of infrared thermal images, N sets of laser point cloud data, the positioning information of N designated shooting positions corresponding to N-view inspection tasks, the configuration data of the power line transmission line, and multiple sets of segment association information of the target line segment. It intelligently determines the names of various fault types existing in the target line segment at the preset acquisition time, and the fault location information corresponding to each fault type, including: the target line segment at the preset acquisition time. All fault type names are represented by binary numerical values ​​in ASCII code, and the fault location information corresponding to each fault type is also represented by binary numerical values. At the ground-based collaborative control station, an AI inspection data fusion model is used, based on the line segment visualization parameter set in N frames of visible light images, the line segment infrared parameter set in N frames of infrared thermal images, N sets of laser point cloud data, the location information of N designated shooting locations corresponding to the N-view inspection tasks, the configuration data of the power line transmission line, and multiple segment data of the target line segment. The associated information, which intelligently determines the names of various fault types existing in the target line segment at the preset acquisition time, and the corresponding fault location information for each fault type, also includes: a set of line segment visualization parameters in N frames of visible light images, a set of line segment infrared parameters in N frames of infrared thermal images, N sets of laser point cloud data, location information of N specified shooting positions corresponding to N-view line inspection tasks, configuration data for each set of power line transmission lines, and multiple sets of segment association information for the target line segment, all in binary numerical representation; among them, the set of line segment visualization parameters in N frames of visible light images and N frames of infrared thermal images... The infrared parameter set of the line segment, N sets of laser point cloud data, positioning information of N designated shooting positions corresponding to the N-view line inspection tasks, configuration data of each set of power line transmission lines, and multiple sets of segment association information of the target line segment are input in parallel into the AI ​​inspection data fusion model corresponding to the set power line transmission line. The AI ​​inspection data fusion model corresponding to the set power line transmission line is then executed to obtain the names of various fault types existing in the target line segment at the preset acquisition time and the fault location information corresponding to each fault type, as output by the AI ​​inspection data fusion model corresponding to the set power line transmission line.

Images