Intelligent detection method for damage of power transmission line hardware fitting based on unmanned aerial vehicle

Abstract

The invention relates to the technical field of power transmission line detection, in particular to a power transmission line hardware fitting damage intelligent detection method based on an unmanned aerial vehicle. Comprising the steps that a main unmanned aerial vehicle takes off, an auxiliary unmanned aerial vehicle is assisted to take off, a to-be-detected hardware fitting is accurately determined through cooperation of the main unmanned aerial vehicle and the auxiliary unmanned aerial vehicle, and the main unmanned aerial vehicle and the auxiliary unmanned aerial vehicle conduct X-ray picture collection and detection on the hardware fitting; the unmanned aerial vehicle airborne detection platform is constructed, double unmanned aerial vehicles work cooperatively, clear X-ray images can be obtained quickly, the detection efficiency is far higher than that of traditional manual detection, and the power failure time is greatly shortened; according to the improved network structure, SPPF-DLKA and C2fG-Ghost modules and a CA attention mechanism are applied, the feature adaptation and extraction capability is enhanced, strong light interference is inhibited, and the damage type, degree and position of the hardware fitting are accurately judged; potential problems can be found in time, line faults are avoided, maintenance cost is reduced, and safety and reliability of power transmission lines are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power transmission line detection, and in particular to an intelligent detection method for power transmission line hardware damage based on an unmanned aerial vehicle. Background Art

[0002] Transmission line hardware plays an important role in the power system, and its status directly affects the safe operation of the transmission line. Traditional hardware detection methods have many limitations, such as the need for manual tower climbing, low efficiency and high risk; some detection methods are difficult to effectively detect hidden damage to the ground wire inside the hardware.

[0003] In recent years, drone technology has received extensive attention and application in the field of power transmission line inspection. With its advantages of strong maneuverability, high flexibility, and no need for power outages, drones can quickly approach power transmission lines for inspection, effectively improving inspection efficiency and reducing operational risks. However, the current transmission line hardware inspection technology based on drones still faces some challenges.

[0004] On the one hand, drones fly in a complex electromagnetic environment, and their communication and control signals are easily interfered with, affecting the stability and reliability of detection operations; on the other hand, the existing detection methods need to be further improved in terms of detection accuracy and intelligence. For example, the recognition and analysis technology of hardware images is not accurate enough, and it is difficult to accurately determine the type, degree and location of hardware damage; in addition, the detection equipment carried by drones needs to be further optimized in terms of performance, weight, volume, etc. to meet the needs of long-term and high-efficiency detection tasks.

[0005] In summary, with the continuous development of the power system and the increasing requirements for power supply reliability, there is an urgent need for an efficient, accurate, safe and intelligent method for detecting damage to transmission line hardware. Summary of the invention

[0006] In order to make up for the shortcomings of the prior art, the present invention aims to provide a transmission line hardware damage intelligent detection method based on drones to overcome the shortcomings of the prior art and provide a strong guarantee for the safe and stable operation of the transmission line. The technical solution adopted by the present invention to solve its technical problems is: a transmission line hardware damage intelligent detection method based on drones, comprising the following steps:

[0007] S1: The main drone takes off: The pilot controls the main drone to bypass a series of obstacles to reach the preset coordinate point, obtains the hardware position through the target detection algorithm and the binocular coordinate detection algorithm, and then hovers the main drone;

[0008] S2 The secondary drone takes off: The pilot controls the secondary drone to take off and turns on the wide-angle camera on the secondary drone to monitor the position of the hardware at this time. The pilot uses the visual assistance of the main drone to monitor the movement of the secondary drone and make the secondary drone reach the top of the hardware to be tested.

[0009] S3: Inspection preparation: After the slave drone reaches the top of the hardware to be inspected, the imaging plate carried by the slave drone is placed close to the hardware and placed on the high-voltage transmission line. At this time, the master drone is fine-tuned through the binocular visual distance detection algorithm so that the X-ray emission device of the DR imaging system on the master drone can be directly facing the center of the imaging plate;

[0010] S4: Get X-ray image: Turn on the X-ray transceiver of the DR imaging system. If the X-ray image is correctly obtained, it means that the detection is completed. At this time, the drone can be lowered to end the work. If the obtained image is blurry, it means that the X-ray image cannot be obtained. Continue to adjust the positions of the main drone, the auxiliary drone, and the imaging board until finally an X-ray image that can be used normally is obtained;

[0011] S5: Damage detection: Process the X-ray images obtained in S4 and perform damage detection on the hardware through the X-ray images.

[0012] Further, the damage detection described in S5 is based on the network structure of YOLOv8 to improve its SPPF module; the improved SPPF module is the SPPF-D_LKA module; the formula of the SPPF-D_LKA module is as follows:

[0013] Attention=Conv1×1(DDW-D-Conv(DDW-cONV(F′)))

[0014]

[0015] In the formula, the input features are and F′∈GELU(Conv(F)), the attention component represents the attention threshold, which is the attention value of the corresponding feature. Represents a multiplication operation.

[0016] Furthermore, the damage detection described in S5 adopts an improved C2f-Ghost module to convert the original n Bottleneck modules into G-GhostBottleneck.

[0017] Furthermore, the damage detection described in S5 also includes a CA attention mechanism module.

[0018] Furthermore, the main UAV and the slave UAV are additionally equipped with robotic arms; the robotic arms of the main UAV and the slave UAV constitute a dual-UAV-mounted gimbal; the dual-UAV-mounted gimbal includes a two-dimensional electric-controlled gimbal, a suspension connection device, a binocular vision module, an edge intelligent processing module, a wide-angle camera, a communication module, a host computer, and a slave computer.

[0019] Furthermore, the robotic arm is also equipped with a laser ranging module and a visual sensor module;

[0020] The laser ranging module is directly connected to the main control board of the edge intelligent processing module to provide positioning information;

[0021] The visual sensor module includes an infrared imager, a high-definition camera and a depth camera; the visual sensor module is directly connected to the main control board of the edge intelligent processing module to process the collected images to realize the information fusion of infrared and high-definition images, and to reorganize the depth information fed back by the depth camera to construct a three-dimensional stereo model of the working scene.

[0022] Furthermore, the lower computer adopts the serial communication protocol to communicate with STM32, and the upper computer communicates with the lower computer through the TCP / IP protocol, and controls the STM32 to control the operation of the robotic arm through serial port instructions. The visual sensor module obtains the detection frame and specific coordinates of the target to form a video stream and text information respectively. The video stream is first encoded by the lower computer, and then the encoded information is transmitted to the upper computer for decoding according to the TCP / IP protocol to form a stable output video stream; the text information is directly transmitted to the upper computer through the TCP / IP protocol; the X-ray transmitting device is remotely controlled to operate, the penetrating hardware is projected onto the imaging board and sent to the upper computer for display.

[0023] Furthermore, the robotic arm adopts a titanium tiger integrated joint module to form a dual-UAV mounted gimbal.

[0024] The beneficial effects of the present invention are as follows:

[0025] 1. The intelligent detection method for power transmission line hardware damage based on drones described in the present invention can quickly obtain clear X-ray images by constructing a drone-mounted detection platform and working with two drones in collaboration. The detection efficiency far exceeds that of traditional manual detection, greatly shortening the power outage time. The improved network structure uses SPPF-D_LKA, C2fG-Ghost modules and CA attention mechanism to enhance feature adaptation and extraction capabilities, suppress strong light interference, and accurately determine the type, degree and location of hardware damage. Potential problems can be discovered in a timely manner, line failures can be avoided, maintenance costs can be reduced, and the safety and reliability of transmission lines can be improved.

[0026] 2. The intelligent detection method for power transmission line hardware damage based on drones described in the present invention adopts drone detection and remote control by operators to avoid the risks of falling from high altitudes and electric shock when climbing towers manually. It is also adaptable to line detection under various harsh conditions and voltage levels, providing reliable and flexible solutions for operation and maintenance, enhancing the operating capabilities in complex environments, and ensuring the smooth implementation of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The present invention will be further described below in conjunction with the accompanying drawings.

[0028] Figure 1 is a flow chart of the method of the present invention;

[0029] Figure 2 It is a schematic diagram of the SPPF of the present invention being improved into SPPF-D_LKA;

[0030] Figure 3 It is the three-stage transformation diagram after adding G-Ghost to the CNN network of the present invention;

[0031] Figure 4 It is a flow chart of the mixing operation of the present invention;

[0032] Figure 5 It is a schematic diagram of the improved G-GhostBottleneck module of the present invention;

[0033] Figure 6 It is a schematic diagram of the C2fG-Ghsot module of the present invention;

[0034] Figure 7 Schematic diagram of the CA attention mechanism of the present invention;

[0035] Figure 8 It is a schematic diagram of the YOLOv8-TC network of the present invention. DETAILED DESCRIPTION

[0036] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0037] like Figure 1 As shown, the present invention proposes an intelligent detection method for power transmission line hardware damage based on UAV, which uses the airborne gimbals of the main UAV and the auxiliary UAV to realize the damage detection of the hardware; the method comprises the following steps:

[0038] S1: The main drone takes off: The pilot controls the main drone to bypass a series of obstacles to reach the preset coordinate point, obtains the hardware position through the target detection algorithm and the binocular coordinate detection algorithm, and then hovers the main drone;

[0039] S2 The secondary drone takes off: The pilot controls the secondary drone to take off and turns on the wide-angle camera on the secondary drone to monitor the position of the hardware at this time. The pilot uses the visual assistance of the main drone to monitor the movement of the secondary drone and make the secondary drone reach the top of the hardware to be tested.

[0040] S3: Inspection preparation: After the slave drone reaches the top of the hardware to be inspected, the imaging plate carried by the slave drone is placed close to the hardware and placed on the high-voltage transmission line. At this time, the master drone is fine-tuned through the binocular visual distance detection algorithm so that the X-ray emission device of the DR imaging system on the master drone can be directly facing the center of the imaging plate;

[0041] S4: Get X-ray image: Turn on the X-ray transceiver of the DR imaging system. If the X-ray image is correctly obtained, it means that the detection is completed. At this time, the drone can be lowered to end the work. If the obtained image is blurry, it means that the X-ray image cannot be obtained. Continue to adjust the positions of the main drone, the auxiliary drone, and the imaging board until finally an X-ray image that can be used normally is obtained;

[0042] S5: Damage detection: Process the X-ray images obtained in S4 and perform damage detection on the hardware through the X-ray images.

[0043] The main UAV and the auxiliary UAV both adopt the titanium tiger integrated joint module to form a mechanical arm, and the mechanical arm forms a dual UAV-mounted gimbal; the dual UAV-mounted gimbal includes a two-dimensional electric-controlled gimbal, a suspension connection device, a binocular vision module, an edge intelligent processing module, a wide-angle camera, a communication module, a host computer, a slave computer, and other hardware components; a laser ranging module and a visual sensor module are also installed;

[0044] The laser ranging module is directly connected to the main control board of the edge intelligent processing module to provide positioning information;

[0045] The visual sensor module includes an infrared imager, a high-definition camera and a depth camera; the visual sensor module is directly connected to the main control board of the edge intelligent processing module to process the collected images to achieve information fusion of infrared and high-definition images, and to reorganize the depth information fed back by the depth camera to construct a three-dimensional model of the working scene;

[0046] The lower computer uses a serial communication protocol to communicate with STM32, while the upper computer communicates with the lower computer through the TCP / IP protocol, and controls the operation of the STM32 control robot arm through serial port instructions. The visual sensor module obtains the detection frame and specific coordinates of the target to form a video stream and text information respectively. The video stream is first encoded by the lower computer, and then the encoded information is transmitted to the upper computer for decoding according to the TCP / IP protocol to form a stable output video stream; the text information is directly transmitted to the upper computer through the TCP / IP protocol; the X-ray transmitting device is remotely controlled to operate, the penetrating hardware is projected onto the imaging board and sent to the upper computer for display.

[0047] The damage detection scheme described in S5 includes the following:

[0048] Based on the network structure of YOLOv8, it has been improved in all aspects to enhance the generalization ability of the network, reduce parameter redundancy, and speed up detection and training.

[0049] Improved SPPF module --SPPF-D_LKA module:

[0050] In order to improve the generalization ability of the network, an improved SPPF-D_LKA module is proposed, such as Figure 2 (a) shows the structure of SPPF. In the figure, a layer of CBS module is used to expand the input features, and then three 5×5 convolution kernels are used to extract the feature points with the largest adjacent feature values. Finally, a CBS module is used to reduce the dimension of the extracted features. Since multiple convolutions are used in this module to extract key data layer by layer, the key information of the context is missing, which reduces the overall performance of the model.

[0051] like Figure 2 The D_LKA module shown in (c) combines deformable convolution and expands the use of large kernels for image segmentation. The deformable convolution is freely deformed by adjusting the sampling grid and integer offset. An additional convolution layer learns deformation from the feature map to create an offset field. The adaptive convolution kernel is obtained by learning deformation based on its own characteristics, which can enhance the adaptation effect to different data features of the hardware. The deformable large kernel attention mechanism is introduced to fully understand the context information. Since the attention mechanism opens up a self-aware domain space, running in this space will not increase the computer's memory. The module formula is as follows:

[0052] Attention=Conv1×1(DDW-D-Conv(DDW-cONV(F′)))

[0053]

[0054] In the formula, the input features are and F′∈GELU(Conv(F)), the attention component represents the attention threshold, which is the attention value of the corresponding feature. Represents multiplication operation;

[0055] like Figure 2 The improved SPPF-D_LKA module shown in (b) modifies the CBS module of the last layer in the SPPF module into D-LKA. After the SPPF module undergoes multiple layers of convolution, it can generate adaptive convolution kernels to adapt to different types of data features and improve the generalization of the network model.

[0056] Improved lightweight network module - C2fG-Ghost:

[0057] The Ghost module can use standard convolution to obtain a part of the feature map, and then each feature map can map out another feature map through linear operation, and superimpose the generated feature map with its own Ghost module to generate a new feature map, so as to achieve more mapping features with fewer parameters and models. However, this operation is only tested on CPU devices and cannot give full play to the advantages of GPU hardware devices. Therefore, the G-Ghost module is used to reduce some redundant feature maps and reduce activation through complex nonlinear operations to accelerate the computing efficiency on the GPU, reduce the delay on the GPU device, and increase the running speed to adapt to the deployment of embedded devices.

[0058] like Figure 3 As shown in the figure, when analyzing the G-Ghost module, it is assumed that a CNN network is used to analyze its three stages:

[0059] like Figure 3 As shown in (a), the first stage expression is:

[0060] Y1=L1(x);

[0061] Y n =L n (L n-1 (…L2(Y1)));

[0062] Where X represents the input feature map, {L1,L2…L n} represents n feature layers, Y1 represents the first feature, Y n Represents the nth block feature;

[0063] like Figure 3 As shown in (b), the second stage can generally divide the deep features into two types, expressed as:

[0064]

[0065] in, represents the input features, c, w, and h represent the number, width, and height of channels, respectively. Indicates complicated features, Representing ghost features and setting 0≤λ≤1, the final complicated feature layer is composed of n blocks superimposed as:

[0066]

[0067] Among them, L n '...L2' is represented as a block of (1-λ)×w width. The block can be represented as a ghost block similar to Y1, and its operation process C is expressed as:

[0068]

[0069] Combining the above equations, the final output features are:

[0070]

[0071] like Figure 3 As shown in (c), the third stage represents the use of information features in complex paths to enhance the information features of cheap operations;

[0072] like Figure 4 As shown, transform Z in the figure to The same range, and then a blending operation is performed to merge the information into A new It is expressed as:

[0073]

[0074] In the formula, τ is the transformation function and the parameter Z is expressed as c′ is the total number of channels;

[0075] In order to understand the meaning of τ, we design a global average pool to obtain aggregate features. The application connection layer will Transform to In the same area, τ(Z) can be expressed as:

[0076] τ(Z)=WPooling(Z)+b

[0077] in represents the weight, Indicates the amount of deviation.

[0078] like Figure 5As shown in FIG. 1 , the improved G-GhostBottleneck module is to modify two CBS modules into G-ghost_regnet. The first layer is used to expand the number of channels, and the second layer is used to reduce the number of channels. The two layers are superimposed to match the number of input and output channels.

[0079] like Figure 6 As shown in the figure, the C2f-Ghost module converts the original n Bottleneck modules into G-GhostBottleneck to improve the network's efficiency in extracting complex features and reduce the extraction of hidden features, thereby increasing the system's operating speed and reducing the number of network parameters.

[0080] Introducing CA attention mechanism:

[0081] Since the main drone and the auxiliary drone will encounter interference from strong light when inspecting the hardware, in order to suppress the interference of the strong light environment, a CA attention mechanism module is added;

[0082] The CA (Coordinate Attention) attention mechanism performs global average pooling operations on the input feature map in both width and length directions to obtain the corresponding width and height, thereby improving detection accuracy.

[0083] like Figure 7 As shown in the figure, the CA attention mechanism, given an input feature, its feature layer shape is [C, H, W]; first, a parallel operation is performed to perform an average pooling operation on the width, and the obtained feature layer shape is [C, H, 1], which is mapped to the height for alignment;

[0084] Then, the height is also averaged and pooled, and the obtained feature layer shape is [C, 1, W], which is mapped to the width for alignment;

[0085] The width and height obtained by the parallel operation are transposed and merged to the same latitude for superposition. The shape of the obtained feature layer is [C, 1, (W+H)]. Convolution, normalization, and activation functions are used to process the superimposed features.

[0086] Subsequently, a parallel separation operation is performed to process the width and height separately and transpose them to obtain two feature layers, [C, H, 1] and [C, 1, W]. Finally, a 1×1 convolution is used to adjust the number of channels and perform sigmoid to obtain the multiplication value in the width and height dimensions. The multiplication value is multiplied by the original feature to obtain the processed feature.

[0087] The CA attention mechanism can accurately obtain the rows and columns of interest in the image, enhance the network's feature extraction capabilities, and has high flexibility. It can be added anywhere in the network to achieve plug-and-play.

[0088] like Figure 8 As shown in the figure, the YOLOv8-TC network structure first modifies the SPPF module in the network to the SPPF-D_LKA module to enhance the generalization ability of the network for different image features; secondly, a large amount of channel information is generated in the Backbone, which includes deep features and shallow features in the image. When this information enters the Neck layer, the feature information will be fused to form channel information of three scales: p3, p4, and p5. In order to prevent information redundancy from causing large model calculations, the C2f module is replaced with a lightweight module C2fG-Ghost; then a CA module is added after each C2fG-Ghost to eliminate the problem that C2fG-Ghost will eliminate some intermediate features, resulting in insufficient feature extraction capabilities.

[0089] The average accuracy rate (AP) is generally used to measure the detection effect of the target detection model. The accuracy rate is the proportion of all predicted samples that correctly detect the target, and the expression is:

[0090]

[0091] The return rate is the proportion of all detected targets that are actually detected, and the expression is:

[0092]

[0093] In the formula, TP represents the number of positive classes predicted in the detection results, FP represents the number of positive classes predicted in the negative classes, and FN represents the number of negative classes predicted in the negative classes.

[0094] The average accuracy of the category can be calculated based on the precision and recall, expressed as:

[0095]

[0096] Then we can get the average accuracy of n categories and name it mAP, which is expressed as:

[0097]

[0098] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. An intelligent detection method for power transmission line hardware damage based on drones, characterized in that: The steps include: S1: The main drone takes off: The pilot controls the main drone to bypass a series of obstacles to reach the preset coordinate point, obtains the hardware position through the target detection algorithm and the binocular coordinate detection algorithm, and then hovers the main drone; S2: The secondary drone takes off: The pilot controls the secondary drone to take off and turns on the wide-angle camera on the secondary drone to monitor the position of the hardware at this time. The pilot uses the visual assistance of the main drone to monitor the movement of the secondary drone and make the secondary drone reach the top of the hardware to be tested. S3: Inspection preparation: After the slave drone reaches the top of the hardware to be inspected, the imaging plate carried by the slave drone is placed close to the hardware and placed on the high-voltage transmission line. At this time, the master drone is fine-tuned through the binocular visual distance detection algorithm so that the X-ray emission device of the DR imaging system on the master drone can be directly facing the center of the imaging plate; S4: Get X-ray image: Turn on the X-ray transceiver of the DR imaging system. If the X-ray image is correctly obtained, it means that the detection is completed. At this time, the drone can be lowered to end the work. If the obtained image is blurry, it means that the X-ray image cannot be obtained. Continue to adjust the positions of the main drone, the auxiliary drone, and the imaging board until finally an X-ray image that can be used normally is obtained; S5: Damage detection: Process the X-ray images obtained in S4 and perform damage detection on the hardware through the X-ray images.

2. According to claim 1, a method for intelligent detection of power transmission line hardware damage based on drones is characterized in that: The damage detection described in S5 is based on the network structure of YOLOv8 to improve its SPPF module; the improved SPPF module is the SPPF-D_LKA module; the formula of the SPPF-D_LKA module is as follows: Attention=Conv1×1(DDW-D-Conv(DDW-cONV(F′))) In the formula, the input features are and F′∈GELU(Conv(F)), represents the attention threshold, which is the attention value of the corresponding feature. Represents a multiplication operation.

3. The intelligent detection method for power transmission line hardware damage based on drone according to claim 2 is characterized in that: The damage detection described in S5 uses an improved C2f-Ghost module to convert the original n Bottleneck modules into G-GhostBottleneck.

4. The intelligent detection method for power transmission line hardware damage based on drone according to claim 3 is characterized in that: The damage detection described in S5 also includes a CA attention mechanism module.

5. The intelligent detection method for power transmission line hardware damage based on drone according to claim 1 is characterized in that: The main UAV and the secondary UAV are additionally equipped with mechanical arms; the mechanical arms of the main UAV and the secondary UAV form a dual UAV-mounted gimbal; The dual-UAV gimbal includes a two-dimensional electric-controlled gimbal, a suspension connection device, a binocular vision module, an edge intelligent processing module, a wide-angle camera, a communication module, a host computer, and a slave computer.

6. The intelligent detection method for power transmission line hardware damage based on drone according to claim 5 is characterized in that: The mechanical arm is also equipped with a laser ranging module and a visual sensor module; The laser ranging module is directly connected to the main control board of the edge intelligent processing module to provide positioning information; The visual sensor module includes an infrared imager, a high-definition camera and a depth camera; The visual sensor module is directly connected to the main control board of the edge intelligent processing module to process the collected images to realize the information fusion of infrared and high-definition images, and to reorganize the depth information fed back by the depth camera to build a three-dimensional model of the working scene.

7. The intelligent detection method for power transmission line hardware damage based on drone according to claim 6 is characterized in that: The lower computer uses a serial communication protocol to communicate with STM32, while the upper computer communicates with the lower computer through the TCP / IP protocol, and controls the operation of the STM32 control robot arm through serial port instructions. The visual sensor module obtains the detection frame and specific coordinates of the target to form a video stream and text information respectively. The video stream is first encoded by the lower computer, and then the encoded information is transmitted to the upper computer for decoding according to the TCP / IP protocol to form a stable output video stream; the text information is directly transmitted to the upper computer through the TCP / IP protocol; the X-ray transmitting device is remotely controlled to operate, the penetrating hardware is projected onto the imaging board and sent to the upper computer for display.

8. The intelligent detection method for power transmission line hardware damage based on drone according to claim 5 is characterized in that: The robotic arm adopts a titanium tiger integrated joint module to form a dual-UAV mounted gimbal.

Images