Defect detection method of unmanned aerial vehicle for insulator chain

Abstract

The invention provides an insulator chain-oriented defect detection method for an unmanned aerial vehicle, and the method comprises the steps: obtaining a target image of an insulator chain shot by the unmanned aerial vehicle, and improving a constructed defect detection model based on a YOLOv5 convolution layer; performing defect detection on the target image through a defect detection model to obtain a defect detection result of the insulator chain; wherein the defect detection model comprises a grouping dimension raising module, a multi-scale attention module and a double-path cooperative detection head, so that the parameter quantity of the model can be reduced, the high-dimensional feature extraction capability of an insulator string can be improved, and rapid detection and positioning of defects such as insulator cracks and dirt can be realized; complex background interference is effectively suppressed through a multi-scale attention module, and the robustness of defect detection is enhanced; and then reasoning is performed through the lightweight dual-path cooperative detection head, so that the feature diversity can be enhanced, the additional calculation overhead in the reasoning stage can be avoided, and the defect detection capability of the end-to-end insulator chain of the unmanned aerial vehicle is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of unmanned aerial vehicle power equipment detection, in particular to a defect detection method for insulator strings of unmanned aerial vehicles. BACKGROUND

[0002] Unmanned aerial vehicle power inspection has been widely used in high-voltage transmission lines and complex terrain scenes due to its efficiency, safety and flexibility. China's power system is large in scale, with a total length of transmission lines exceeding 1.8 million kilometers, and the proportion of high-voltage transmission lines is increasing year by year. As a key component of the transmission line, the state of the insulator string directly affects the stability and safety of the power grid. In recent years, hundreds of cases of failures caused by defects in insulator strings (such as porcelain chip damage and steel cap corrosion) have resulted in direct economic losses exceeding one hundred million yuan. These defects can lead to a decrease in insulating performance, a risk of electric leakage and short circuit, and even large-scale power outages in severe cases. Therefore, defect detection of insulator strings has attracted much attention, and how to efficiently and accurately detect defects has become a key to ensuring the safe operation of the power grid.

[0003] Traditional insulator string defect detection mainly relies on manual inspection, which is low in efficiency, high in cost and has safety hazards. Methods based on image processing have poor robustness and are difficult to cope with complex background interference. Although deep learning technology has made significant progress in defect detection, it still lacks the ability to recognize complex background interference and small defects in high-altitude images. In particular, in unmanned aerial vehicle power inspection, insulator strings are often interfered by obstructions such as tower fittings and wires, resulting in complex image backgrounds and difficulty in accurately extracting defect features. In addition, the unmanned aerial vehicle needs to process a large amount of image data in real time during the inspection process, and traditional deep learning models have high computational complexity and large memory occupancy, making it difficult to efficiently run on resource-constrained embedded devices. SUMMARY

[0004] The main purpose of the embodiments of the present application is to propose a defect detection method for insulator strings of unmanned aerial vehicles, which can improve the end-to-end defect detection capability of insulator strings of unmanned aerial vehicles, greatly improve the automation and intelligence level of unmanned aerial vehicle power inspection, reduce the failure rate of power systems, save manpower and material resources, and provide a strong guarantee for the safe and stable operation of the power grid.

[0005] To achieve the above purpose, a first aspect of the embodiments of the present application proposes a defect detection method for insulator strings of unmanned aerial vehicles, comprising: obtaining a target image of an insulator string shot by an unmanned aerial vehicle, and a defect detection model constructed based on an improved YOLOv5 convolution layer; detecting defects in the target image through the defect detection model to obtain a defect detection result of the insulator string; The construction process of the defect detection model comprises the following steps: obtaining a plurality of sample images, the sample images being insulator string images marked with defects; initializing a network architecture improved based on a YOLOv5 convolution layer, the network architecture comprising a group dimension increasing module for performing convolution processing on a target image, a multi-scale attention module for extracting defect features from the target image, and a double-path collaborative detection head for defect reasoning of the target image, the group dimension increasing module being communicatively connected with the multi-scale attention module, and the multi-scale attention module being communicatively connected with the double-path collaborative detection head; training the network architecture according to the plurality of sample images to obtain a defect detection model.

[0006] Further, in some embodiments, the defect detection model is obtained by iteratively training the network architecture according to the plurality of sample images, comprising the following steps: inputting the sample image into the group dimension increasing module to expand features to obtain a first feature image; inputting the first feature image into the multi-scale attention module to extract multi-scale features to obtain a second feature image; inputting the second feature image into the double-path collaborative detection head to predict defects to obtain a defect prediction image; pairing the defect prediction image with the sample image in position to obtain difference data between the sample image and the defect prediction image; based on a loss function, performing gradient difference calculation on the difference data to obtain a gradient loss value of the current iteration; according to the gradient loss value, updating the multi-layer network parameters of the network architecture by an iterative optimizer in a backward propagation manner; if the multi-layer network parameters of the network architecture meet preset training requirements, terminating the iterative training of the network architecture to obtain the defect detection model.

[0007] Further, in some embodiments, the group dimension increasing module comprises a double-channel expansion architecture and a group convolution module, the double-channel expansion architecture being communicatively connected with the group convolution module; inputting the sample image into the group dimension increasing module to expand features to obtain a first feature image, comprising the following steps: inputting the sample image into an input layer of the network architecture to extract features to obtain a second feature image, the second feature image comprising a plurality of feature channels; inputting the second feature image into the double-channel expansion architecture to expand features to obtain a third feature image, the third feature image having a larger number of feature channels than the second feature image; inputting the third feature image into the group convolution module to compress features to obtain the first feature image, the first feature image having a smaller number of feature channels than the third feature image.

[0008] Further, in some embodiments, the dual-channel expansion architecture comprises an expansion path and a compensation path, the expansion path comprising the first convolutional layer and the multi-layer perceptron, and the compensation path comprising the second convolutional layer and the feature decoupler, the expansion path being related in parallel with the compensation path; wherein the second feature image is input into the dual-channel expansion architecture for feature expansion to obtain a third feature image, comprising the following steps: the second feature image is input into the first convolutional layer for convolution operation, and the first feature image after convolution is subjected to feature expansion in a feature channel combination manner through the multi-layer perceptron to obtain a fourth feature image, the fourth feature image comprising a first feature channel for expressing fusion of local texture features and global structure features of the sample image; the second feature image is input into the second convolutional layer for convolution operation, and the first feature image after convolution is subjected to feature expansion in a feature channel decomposition manner through the feature decoupler to obtain a fifth feature image, the fifth feature image comprising a second feature channel for expressing edge detail features of the sample image; the fourth feature image and the fifth feature image are spliced in the channel dimension to obtain the third feature image.

[0009] Further, in some embodiments, the third feature image is input into the group convolution module for feature compression to obtain the first feature image, comprising: all feature channels of the third feature image are subjected to grouping processing to obtain a grouped third feature image, the grouped third feature image comprising a plurality of channel groups, and each channel group comprising a plurality of feature channels; all channel groups of the third feature image are subjected to convolution operation to obtain a first feature image set corresponding to the third feature image, the first feature image set comprising a plurality of feature sub-images, and each feature sub-image being mapped to a channel group; all feature sub-images in the first feature image set are subjected to image splicing to obtain the first feature image.

[0010] Further, in some embodiments, all feature channels of the third feature image are subjected to grouping processing to obtain a grouped third feature image, comprising: a preset feature channel number output by the group dimension increasing module is obtained, and a first feature channel number of the third feature image is determined; a greatest common divisor between the first feature channel number and the preset feature channel number is determined as a dynamic grouping number of the third feature image; all feature channels of the third feature image are evenly grouped according to the dynamic grouping number to obtain the grouped third feature image.

[0011] Further, in some embodiments, the multi-scale attention module comprises a multi-scale pooling layer and a pool attention mechanism layer, the multi-scale pooling layer is in communication connection with the pool attention mechanism layer, the multi-scale pooling layer comprises a spatial pyramid pooling layer, a multi-scale group convolution layer and an up-sampling layer, the spatial pyramid pooling layer is in communication connection with the multi-scale group convolution layer, and the multi-scale group convolution layer is in communication connection with the up-sampling layer; The first feature image is input into the multi-scale attention module for multi-scale feature extraction to obtain a second feature image, comprising the following steps: The first feature image is input into the spatial pyramid pooling layer for multi-scale feature extraction to obtain a plurality of sixth feature images, and the spatial scales of two sixth feature images are the same or different; Based on the multi-scale group convolution layer, a plurality of sixth feature images at the same spatial scale are compressed by group convolution to obtain a plurality of seventh feature images with different spatial scales; The plurality of seventh feature images are channel-spliced by the up-sampling layer to obtain an eighth feature image after multi-scale fusion; The eighth feature image is input into the pool attention mechanism layer for feature enhancement to obtain a second feature image for highlighting the defect region feature.

[0012] Further, in some embodiments, the multi-scale attention module further comprises a pool attention mechanism layer, the pool attention mechanism layer comprises a channel attention module and a spatial attention module, the channel attention module is in communication connection with an output layer of the multi-scale pooling layer, the spatial attention module is in communication connection with the channel attention module, the channel attention module comprises a full connection layer, and the spatial attention module comprises a convolution layer, and the convolution layer and the full connection layer are both provided with a Sigmoid activation function; The eighth feature image is input into the pool attention mechanism layer for feature enhancement to obtain a second feature image for highlighting the defect region feature, comprising the following steps: The eighth feature image is globally average-pooled by the channel attention module to obtain a channel description vector of the eighth feature image; The channel description vector is respectively calculated by a Sigmoid activation function in the full connection layer to obtain a channel attention weight corresponding to the eighth feature image; The channel attention weight is multiplied with each feature channel in the eighth feature image by the full connection layer to obtain a ninth feature image after channel weighting; The ninth feature image is input into the spatial attention module for maximum pooling and average pooling respectively to obtain a tenth feature image of maximum-pooled spatial description and an eleventh feature image of average-pooled spatial description; The spatial attention weight between the tenth feature image and the eleventh feature image is calculated through a sigmoid activation function in a full connection layer to obtain a spatial attention weight; The spatial attention weight is multiplied with each feature channel of the ninth feature image through a convolution layer to obtain a second feature image.

[0013] Further, in some embodiments, the dual-path collaborative detection head includes a diversified branch block, a positioning branch, and a classification branch, and the diversified branch block is in communication connection with the positioning branch and the classification branch respectively; The second feature image is input into the dual-path collaborative detection head for defect prediction to obtain a defect prediction image, including the following steps: The second feature image is input into the diversified branch block for feature enhancement to obtain a twelfth feature image; The type probability of the defect feature of the twelfth feature image is calculated through the classification branch to obtain a defect type probability distribution map; The defect feature of the twelfth feature image is mapped through the positioning branch to obtain a defect position distribution map; The output layer of the dual-path collaborative detection head is used to combine the defect type probability distribution map and the defect position distribution map to obtain the defect prediction image; Further, in some embodiments, the diversified branch block includes a first convolution branch, a second convolution branch, a third convolution branch, a fourth convolution branch, and a branch connection layer, the first convolution branch includes a first convolution kernel of 1×1, the second convolution branch includes a second convolution kernel of 1×1 and a third convolution kernel of K×K, the second convolution kernel is in communication connection with the third convolution kernel, the third convolution branch includes a fourth convolution kernel of 1×K, and the fourth convolution branch includes a fifth convolution kernel of K×1; Further, the eleventh feature image output by the multi-scale attention module is input into the diversified branch block for feature enhancement to obtain a twelfth feature image, including the following steps: The eleventh feature image is input into the first convolution branch to enable the first convolution kernel to perform average pooling on the eleventh feature image to obtain a twelfth feature image for expressing global context information; The eleventh feature image is input into the second convolution branch to enable the second convolution kernel and the third convolution kernel to capture spatial correlation of the eleventh feature image to obtain a thirteenth feature image for expressing local spatial correlation; The eleventh feature image is input into the third convolution branch respectively to enable the fourth convolution kernel to capture spatial correlation of the eleventh feature image to obtain a fourteenth feature image for expressing direction sensitivity; The eleventh feature image is input into the fourth convolution branch respectively, so that the fifth convolution kernel performs spatial correlation capture on the eleventh feature image, and a fifteenth feature image for expressing direction sensitivity is obtained. The twelfth feature image, the thirteenth feature image and the fourteenth feature image are connected through the branch connection layer to obtain the twelfth feature image.

[0014] And after obtaining the defect detection model, the following steps are included: Based on the zero padding and parameter superposition method, the first convolution branch, the second convolution branch, the third convolution branch and the fourth convolution branch are merged in the form of re-parameterization to obtain a fifth convolution branch, and the fifth convolution branch includes a sixth convolution kernel of KxK.

[0015] Further, in some embodiments, the positioning branch includes a bounding box convolution layer, a DFL loss layer and a CIOU loss layer, the bounding box convolution layer is in communication connection with the DFL loss layer, the DFL loss layer is in communication connection with the CIOU loss layer, the DFL module is provided with a cross-entropy loss function, and the defect features of the twelfth feature image are mapped to coordinates through the positioning branch to obtain a defect position distribution map, including the following steps: The twelfth feature image is adjusted in channel through the bounding box convolution layer to obtain a first defect position image with a bounding box attribute; The bounding box coordinate values in the first defect position image are optimized by regression in a discrete manner through the DFL loss layer to obtain a second defect position image; The overlapping area, center point distance and aspect ratio of the second defect position image are adjusted in weight through the CIOU loss layer to obtain the defect position distribution map.

[0016] Further, in some embodiments, the classification branch includes a classification convolution layer and a CLS loss layer, the classification convolution layer is in communication connection with the CLS loss layer, the classification convolution layer is provided with a binary cross-entropy loss function, and the defect features of the twelfth feature image are calculated in type probability through the classification branch to obtain a defect type probability distribution map, including the following steps: The defect features of the twelfth feature image are mapped in type through the compressed feature channel of the classification convolution layer to obtain a first defect type image; Each anchor point in the first defect type image is predicted in multi-type probability and focal loss through the CLS loss layer to obtain a second defect type image; The feature channels of the second defect type image are processed in weight through the channel attention mechanism of the classification branch to obtain the defect type probability distribution map.

[0017] The embodiments of the first aspect of the present application have the following beneficial effects: the feature expansion of the insulator string is performed through the group dimensionality increase group convolution module of the defect detection model, which can reduce the number of model parameters while improving the high-dimensional feature extraction capability of the insulator string, thereby realizing rapid detection and positioning of defects such as insulator cracks and contamination; and the feature extraction of the insulator string is performed through the multi-scale attention module, which can accurately distinguish the insulator target in a complex background without increasing the computational burden, effectively suppress the interference of the complex background, and enhance the robustness of defect detection; and then reasoning is performed through a lightweight dual-path collaborative detection head, which can enhance the feature diversity while avoiding the additional computational overhead in the reasoning stage, thereby realizing end-to-end insulator string defect detection by drones, greatly improving the automation and intelligence level of drone power inspections, reducing the occurrence rate of power system failures, saving manpower and material costs, and providing a strong guarantee for the safe and stable operation of the power grid. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is an optional flow chart of the defect detection method for insulator strings using a drone provided in an embodiment of the present application; Figure 2 This is an optional flowchart of the process of constructing the defect detection model provided in the embodiment of the present application; Figure 3 This embodiment of the present application provides Figure 2 An optional flowchart of step S203; Figure 4 This is an optional structural diagram of the sample image training defect detection model provided in the embodiment of the present application; Figure 5 This embodiment of the present application provides Figure 3 An optional flowchart of step S301; Figure 6 This embodiment of the present application provides Figure 5 An optional flowchart of step S502; Figure 7 This is an optional structural diagram of the dual-channel expansion architecture provided in an embodiment of the present application; Figure 8 This embodiment of the present application provides Figure 5 An optional flowchart of step S603; Figure 9 This is an optional schematic diagram of feature compression comparison between the group convolution module and the ordinary convolution module provided in an embodiment of the present application; Figure 10 This embodiment of the present application provides Figure 7 An optional flowchart of step S801; Figure 11is an optional flowchart of step S302 in the embodiment of the present application Figure 3 is an optional flowchart of step S302 in the embodiment of the present application Figure 12 is an optional structure diagram of the multi-scale attention module provided by the embodiment of the present application Figure 13 is an optional flowchart of step S1104 in the embodiment of the present application Figure 11 is an optional flowchart of step S1104 in the embodiment of the present application Figure 14 is an optional structure diagram of the pool attention mechanism layer provided by the embodiment of the present application Figure 15 is an optional flowchart of step S303 in the embodiment of the present application Figure 3 is an optional flowchart of step S303 in the embodiment of the present application Figure 16 is an optional structure diagram of the dual-path collaborative detection head provided by the embodiment of the present application Figure 17 is an optional flowchart of step S1501 in the embodiment of the present application Figure 15 is an optional flowchart of step S1501 in the embodiment of the present application Figure 18 is an optional flowchart of step S1502 in the embodiment of the present application Figure 15 is an optional flowchart of step S1502 in the embodiment of the present application Figure 19 is an optional flowchart of step S1503 in the embodiment of the present application Figure 15 is an optional flowchart of step S1503 in the embodiment of the present application Figure 20 is an optional hardware structure diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0020] In the description of the present application, it should be understood that the orientation description, such as the orientation or position relationship indicated by up, down, front, back, left, right, etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0021] It also needs to be explained that in the description of the present application, several meanings are one or more, the meaning of multiple is two or more, greater than, less than, more than, etc. Understand as not including the number, above, below, within, etc. Understand as including the number. If it is described to the first, the second is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or implicitly indicating the order of the indicated technical features.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification is for the purpose of describing embodiments of this application only and is not intended to be limiting of this application.

[0023] In the description of the present application, the description of the reference terms "one embodiment", "some embodiments", "exemplary embodiment", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the exemplary description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0024] Traditional insulator string defect detection mainly relies on manual inspection, which is low in efficiency, high in cost and has safety hazards. The method based on image processing has poor robustness and is difficult to cope with complex background interference. Although deep learning technology has made significant progress in defect detection, its ability to recognize complex background interference and small defects in high-altitude images is still insufficient. In particular, in unmanned aerial vehicle power inspection, insulator strings are often interfered by tower fittings, wires and other obstructions, resulting in complex image backgrounds and difficulty in accurately extracting defect features. In addition, unmanned aerial vehicles need to process a large amount of image data in real time during inspection. The traditional deep learning model has high computational complexity and large memory occupation, which is difficult to run efficiently on resource-limited embedded devices.

[0025] To address these issues, this application proposes a drone-based defect detection method for insulator strings. First, by building a group-based dimensionality increase module into the defect detection model, the computational efficiency and memory usage of the neural network are optimized, enabling efficient operation on embedded devices. This group-based dimensionality increase module replaces the fully connected convolution kernel with adaptive group convolution, significantly reducing the parameter size while maintaining cross-channel feature interaction, reducing computational complexity and memory usage while ensuring detection accuracy. Second, to address the issues of complex background interference and minor defect detection, a multi-scale attention module is built into the defect detection model to extract features from insulator strings. This module can accurately distinguish insulator targets in complex backgrounds without increasing the computational burden, effectively suppressing complex background interference and enhancing the robustness of defect detection. Furthermore, a lightweight dual-path collaborative detection head is constructed through the model for inference. This enhances feature diversity while avoiding additional computational overhead in the inference phase, thereby enabling end-to-end drone-based insulator string defect detection. This significantly improves the automation and intelligence level of drone-based power inspections, reduces the occurrence of power system failures, saves manpower and material costs, and provides a strong guarantee for the safe and stable operation of the power grid.

[0026] An embodiment of the present application provides a method for detecting defects in insulator strings using a drone, which is specifically described through the following embodiments.

[0027] First, refer to Figure 1 As shown, Figure 1 This is an optional flowchart of the defect detection method for insulator strings using a drone provided in an embodiment of the present application. The method may include but is not limited to steps S101 to S102.

[0028] Step S101: Obtain a target image of an insulator string taken by a drone and a defect detection model constructed based on an improved YOLOv5 convolutional layer.

[0029] Before acquiring the target image of the insulator string from a drone, the system first determines the starting point of the inspection area and the drone's flight path. This means clarifying the drone's departure and destination points, and ensuring the geographic information of the mission area and the inspection objectives are clear. Based on the preset mission parameters and regional map information, the backend server generates an inspection path that covers the edges of the target area. By using drones to collect images of the insulator string, a closed-loop preprocessing process, from path planning to data acquisition, is established, providing standardized input conditions for subsequent insulator defect analysis.

[0030] After the UAV arrives at the inspection area, it collects images of the insulator string at the inspection site and performs image preprocessing. First, according to the pre-planned flight path, the attitude and height of the UAV are adjusted to ensure that the camera can take pictures of the insulator string at the best angle and distance. The image data collected by the camera is transmitted in real time to the terminal display through 5G communication, and is also stored in the local storage device of the UAV for subsequent processing. In order to improve the accuracy and efficiency of defect detection, the collected images need to go through a series of preprocessing steps. Specifically, since the images taken by the UAV in complex environments may be affected by factors such as light changes, fog, backlight, etc., resulting in noise in the images, it is necessary to use methods such as Gaussian filtering or median filtering to remove random noise while preserving the key features of the insulator string. In order to ensure the consistency of the model input, the pixel values of the image are normalized to a uniform range to eliminate the differences in pixel values under different lighting conditions. At the same time, according to the input requirements of the defect detection model, the image is uniformly adjusted to 1920*1080 as input to meet the calculation requirements of the model. During the preprocessing stage, data augmentation is performed on the image, including random rotation, flipping, scaling, etc. to simulate image changes under different shooting conditions, thereby improving the robustness of the model. Through these preprocessing steps, the collected insulator string images can be optimized to a format suitable for the input of the defect detection model, providing high-quality input data for the subsequent insulator string defect detection model. The optimized images can more clearly present the crack, dirt and other defect features of the insulator string, thereby significantly improving the accuracy and efficiency of detection.

[0031] Step S102: performing defect detection on the target image by the defect detection model to obtain a defect detection result of the insulator string.

[0032] In one possible embodiment, after the defect detection model performs defect detection on the target image of the insulator string collected by the UAV, it automatically identifies typical fault types such as surface dirt, umbrella skirt damage, steel cap rust, etc., and generates a visual defect detection result map containing defect position, shape and contour information. The output detection result map will be presented in color, with each color corresponding to a specific abnormal category, making it easy to quickly distinguish different problem areas, and also reflecting the distribution range and appearance characteristics through contour outlining. When receiving the processed image, the on-site maintenance personnel can quickly filter the focus according to the color classification system and make a graded judgment on the influence degree of each type of abnormality based on professional experience. Based on this visual analysis result, subsequent targeted disposal plans will be formed to ensure that different conditions are reasonably addressed, effectively improving the accuracy and efficiency of facility maintenance.

[0033] Referring to Figure 2 , as shown, Figure 2This is an optional flowchart of the construction process of the defect detection model provided in an embodiment of the present application. The construction process may include but is not limited to steps S201 to S203.

[0034] Step S201: Acquire multiple sample images, where the sample images are images of insulator strings marked with defects.

[0035] Step S202: Initialize the network architecture improved based on the YOLOv5 convolutional layer.

[0036] Among them, the network architecture includes a group dimensionality increase module for performing convolution processing on the target image, a multi-scale attention module for extracting defect features of the target image, and a dual-path collaborative detection head for performing defect reasoning on the target image. The group dimensionality increase module is communicated with the multi-scale attention module, and the multi-scale attention module is communicated with the dual-path collaborative detection head.

[0037] Step S203: training the network architecture based on the multiple sample images to obtain a defect detection model.

[0038] It should be noted that by constructing a group dimensionality increase module in the network architecture, the computational efficiency and memory usage of the neural network are optimized, enabling it to run efficiently on embedded devices. This module replaces the fully connected convolution kernel with adaptive group convolution, significantly reducing the parameter scale while maintaining the cross-channel feature interaction capability, reducing computational complexity and memory usage while ensuring detection accuracy. Secondly, to address the problems of complex background interference and small defect detection, a multi-scale attention module is constructed in the network architecture. This multi-scale attention module enhances the model's response to small defects through hierarchical spatial compression and multi-level feature fusion, while suppressing background interference, reducing the false detection rate, and significantly improving the robustness of defect recognition.

[0039] Reference Figure 3 As shown, Figure 3 This embodiment of the present application provides Figure 2 An optional flowchart of step S203 in the method, the process of the method may include but is not limited to steps S301 to S307.

[0040] Step S301: Input the sample image into the grouping and dimensionality upgrading module for feature expansion to obtain a first feature image.

[0041] Step S302: Input the first feature image into the multi-scale attention module for multi-scale feature extraction to obtain a second feature image.

[0042] Step S303: inputting the second feature image into the dual-path collaborative detection head for defect prediction to obtain a defect prediction image.

[0043] Step S304: The defect prediction image is positionally paired with the sample image to obtain difference data between the sample image and the defect prediction image.

[0044] Step S305: Gradient difference calculation is performed on the difference data based on a loss function to obtain a gradient loss value of the current iteration.

[0045] Step S306: The multi-layer network parameters of the network architecture are updated by back propagation through an iterative optimizer according to the gradient loss value.

[0046] Step S307: If the multi-layer network parameters of the network architecture meet the preset training requirements, the iterative training of the network architecture is terminated to obtain the defect detection model.

[0047] It should be noted that, as shown in Figure 4 , the method can include but is not limited to steps S501 to S503. Figure 4 is an optional structure diagram of the sample image training defect detection model provided by the embodiments of the present application. By constructing a grouping dimension increasing module in the defect detection model, the calculation efficiency and memory occupation of the neural network are optimized, so that it can run efficiently on an embedded device. The grouping dimension increasing module replaces the full connection convolution kernel with an adaptive grouping convolution, significantly reduces the parameter size, while maintaining the cross-channel feature interaction capability, reduces the calculation complexity and memory occupation under the premise of ensuring the detection accuracy. Secondly, for the problem of complex background interference and micro defect detection, a multi-scale attention module is also constructed in the defect detection model to extract features from the insulator string, which can accurately distinguish the insulator target in the complex background without increasing the calculation burden, effectively suppresses the complex background interference, enhances the robustness of defect detection, and further infers through the model to construct a lightweight double-path collaborative detection head, which can enhance the feature diversity while avoiding additional calculation overhead in the inference stage, thereby realizing end-to-end insulator string defect detection of the unmanned aerial vehicle, greatly improving the automation and intelligence level of the unmanned aerial vehicle power inspection, reducing the power system failure rate, saving manpower and material resources, and providing a strong guarantee for the safe and stable operation of the power grid.

[0048] Further, the grouping dimension increasing module includes a double-channel expansion architecture and a group convolution module, which are in communication connection, as shown in Figure 5 , the method can include but is not limited to steps S501 to S503. Figure 5 is an optional flowchart of step S301 in Figure 3 the embodiments provided by the present application. The method can include but is not limited to steps S501 to S503.

[0049] Step S501: The sample image is input to the input layer of the network architecture for feature extraction to obtain a second feature image.

[0050] The second feature image includes multiple feature channels.

[0051] Step S502: inputting the second feature image into the dual-channel expansion architecture to perform feature expansion to obtain a third feature image.

[0052] Step S503: Input the third feature image into a group convolution module for feature compression to obtain a first feature image, where the number of feature channels of the first feature image is less than that of the third feature image.

[0053] Further, refer to Figure 6 and Figure 7 As shown, Figure 6 This embodiment of the present application provides Figure 5 An optional flowchart of step S502 in FIG. Figure 7 This is an optional structural diagram of a dual-channel expansion architecture provided in an embodiment of the present application. The dual-channel expansion architecture includes an expansion path and a compensation path. The expansion path includes a first convolutional layer and a multi-layer perceptron. The compensation path includes a second convolutional layer and a feature decoupler. The expansion path is related to the compensation path and is parallel. The method may include but is not limited to steps S601 to S603.

[0054] Step S601: Input the second feature image into the first convolution layer for convolution operation, and perform feature expansion on the convolved first feature image in a feature channel combination manner through a multi-layer perceptron to obtain a fourth feature image.

[0055] The fourth feature image includes a first feature channel for expressing the fusion of local texture features and global structural features of the sample image.

[0056] Step S602: Input the second feature image into the second convolutional layer for convolution operation, and perform feature expansion on the convolved first feature image in a feature channel decomposition manner through a feature decoupler to obtain a fifth feature image.

[0057] The fifth feature image includes a second feature channel for expressing edge detail features of the sample image.

[0058] Step S603: performing channel dimension splicing on the fourth feature image and the fifth feature image to obtain a third feature image.

[0059] In steps S601 to S603, based on the working condition of the unmanned aerial vehicle power inspection, a double-channel expansion architecture is proposed to further improve the accuracy of defect detection of the insulator string. The architecture adopts a cooperative mechanism of parallel expansion paths and compensation paths. In the expansion path, the multi-layer perception machine fuses the local texture features and global structure features of the first feature image in the form of feature channel combination. Meanwhile, in the compensation path, the feature decoupler expands the edge detail features of the second feature image in the form of feature channel decomposition, thereby suppressing complex background interference and significantly improving the detection performance of the insulator string in the unmanned aerial vehicle power inspection.

[0060] In an embodiment, the double-channel expansion architecture is as shown in Figure 7 The first convolutional layer of the expansion path expands the channel to 1.5 times the original by using a 3x3 convolutional kernel, constructs a wide channel space to fuse the local texture and global structure features of the insulator string (first feature image), and enhances the adaptability to multi-scale targets under aerial view. The compensation path suppresses complex background interference such as high-voltage line towers and vegetation through the feature decoupler, and enhances the ability to preserve the edge details of the insulator string (first feature image) in foggy, backlight and other scenes. The double-channel architecture balances the calculation efficiency and feature expression ability by optimizing the channel expansion coefficient, and is compatible with the multi-modal feature extraction requirements of metal heat spots in night infrared images.

[0061] The expansion path performs dimensionality increasing operation (i.e. feature expansion) as follows: (1) Wherein represents the number of expanded feature channels, is a 3x3 convolution kernel, and a is a channel expansion coefficient, represents the number of feature channels of the intermediate layer, represents the number of input channels of the intermediate layer.

[0062] Further, referring to Figure 8 and Figure 9 , Figure 8 is an optional flowchart of step S603 in Figure 5 provided by the embodiments of the present application, Figure 9 is an optional schematic diagram for comparing feature compression between the group convolution module and the ordinary convolution module provided by the embodiments of the present application. The method can include but is not limited to steps S801 to S803.

[0063] Step S801: Grouping processing is performed on all feature channels of the third feature image to obtain a grouped third feature image. The grouped third feature image includes a plurality of channel groups, and each channel group includes a plurality of feature channels.

[0064] Step S802: performing a convolution operation on all channel groups of the third feature image to obtain a first feature image set corresponding to the third feature image, wherein the first feature image set includes a plurality of feature sub-images, and the feature sub-images and the channel groups are mapped to each other.

[0065] Step S803: performing image stitching on all feature sub-images in the first feature image set to obtain a first feature image.

[0066] In steps S801 to S803, the embodiment of the present application uses the standard convolution layer in YOLOv5 as the basis and adopts the group convolution module to adapt to the computational limitations of the drone embedded device. Group convolution is to group the third feature image of the input layer and then use different convolution kernels to convolve each channel group. Among them, the group convolution module is compared with the ordinary convolution module. Figure 9 As shown in the figure, the group convolution module divides the feature channels of the third feature image of the input layer into several channel groups. Each channel group performs a convolution operation independently to obtain a set of first feature images corresponding to the third feature image. Finally, the feature sub-images of each channel group are spliced ​​to obtain the first feature image. This design significantly reduces the amount of computation and parameters to N times that of ordinary convolution, where N = 1 / number of channel groups, thereby significantly reducing memory usage and computing resource consumption. It is particularly suitable for resource-constrained scenarios such as drone inspections. At the same time, the grouping mechanism can also force different channel groups to learn diverse feature patterns. Using group convolution can reduce the amount of convolution computation and increase the accuracy of the model.

[0067] Reference Figure 10 As shown, Figure 10 This embodiment of the present application provides Figure 7 An optional flowchart of step S801 in the method may include but is not limited to steps S1001 to S1003.

[0068] Step S1001: obtaining the preset number of feature channels that the grouping and dimensionality upgrading module is preset to output, and determining the number of first feature channels of the third feature image.

[0069] Step S1002: determining the greatest common divisor between the number of the first feature channels and the number of preset feature channels as the number of dynamic groupings of the third feature image.

[0070] Step S1003: grouping all feature channels of the third feature image evenly according to the dynamic grouping number to obtain a grouped third feature image.

[0071] In step S1001 to step S1003, the embodiment of the present application significantly improves the real-time performance and lightweight deployment capability of insulator string defect identification in UAV power inspection by using the adaptive group convolution method. wherein, represents the number of groups, is the preset feature channel number of the output image of the group dimension increasing module, is the first feature channel number of the third feature image, and gcd represents the greatest common divisor between the first feature channel number and the preset feature channel number.

[0072] is the parameter scale of the group convolution module for realizing efficient fusion of cross-scale features under a power transmission line complex background. (2) wherein, is the size of the group convolution kernel, which is compressed by 1 / g times compared with the traditional convolution. Through the adaptive channel grouping strategy in steps S801 to S803, the model can dynamically capture the subtle contamination deposition, damage loss and other defect features between insulator pieces, and still maintain stable detection performance under changes in illumination and occlusion interference. Meanwhile, compared with the traditional convolution, the following optimizations exist: 1. Memory occupation optimization: based on the grouping parameter sharing mechanism, the model weight file volume is reduced to: (wherein represents the adjusted new value, is the reference value), after the model weight file volume is reduced, it can be stored in the limited storage space of the unmanned aerial vehicle on-board equipment.

[0073] 2. Computing performance improvement: the joint evaluation index is introduced: - wherein, represents the memory access cost, is the number of floating point operations, is the number of tensors or batch size.

[0074] (3) In steps S1001 to S1003, the method can effectively improve the detection frame rate of insulator cracks, contamination and other defects by optimizing memory occupation and improving computing performance.

[0075] Further, in a possible embodiment, referring to FIGS. 1, 2 and 3, Figure 11 and Figure 12 is an optional flowchart of step S302 in the method provided by the embodiments of the present application, Figure 11 is an optional flowchart of step S302 in the method provided by the embodiments of the present application, Figure 3 is an optional flowchart of step S302 in the method provided by the embodiments of the present application, Figure 12is an optional structural diagram of a multi-scale attention module provided by an embodiment of the present application. The multi-scale attention module comprises a multi-scale pooling layer and a pool attention mechanism layer. The multi-scale pooling layer is in communication connection with the pool attention mechanism layer. The multi-scale pooling layer comprises a spatial pyramid pooling layer, a multi-scale group convolution layer, and an upsampling layer. The spatial pyramid pooling layer is in communication connection with the multi-scale group convolution layer. The multi-scale group convolution layer is in communication connection with the upsampling layer. The method can comprise, but is not limited to, steps S1101 to S1104.

[0076] In step S1101, a first feature image is input to the spatial pyramid pooling layer for multi-scale feature extraction to obtain a plurality of sixth feature images.

[0077] The spatial scales of the two sixth feature images are the same or different.

[0078] In step S1102, based on the multi-scale group convolution layer, a plurality of sixth feature images at the same spatial scale are compressed by group convolution to obtain a plurality of seventh feature images of different spatial scales.

[0079] In step S1103, the plurality of seventh feature images are channel spliced by the upsampling layer to obtain an eighth feature image after multi-scale fusion.

[0080] In step S1104, the eighth feature image is input to the pool attention mechanism layer for feature enhancement to obtain a second feature image for highlighting the defect region features.

[0081] In steps S1101 to S1103, as shown in Figure 12 To further improve the model's perception ability of the insulator string multi-scale defects, a multi-scale attention module is also constructed in the model based on the efficient feature representation extracted by the group dimension increasing module. The module innovatively integrates a multi-scale pooling layer and a pool attention mechanism layer. The insulator string image collected by the unmanned aerial vehicle is expanded by the group dimension increasing module to obtain a first feature image F∈R H×W×C where H and W are the height and width of the first feature image respectively, and C is the number of feature channels of the first feature image. In the multi-scale pooling layer, the first feature image is pooled by different scale kernels in the spatial pyramid pooling layer to generate a plurality of sixth feature images. Then, based on the multi-scale group convolution layer, a plurality of sixth feature images at the same spatial scale are compressed by group convolution to obtain a plurality of seventh feature images of different spatial scales. Finally, the seventh feature images of different scales are restored to the original size of the image in the upsampling layer, and channel splicing is performed to obtain a second feature image for highlighting the defect region features. The calculation formula of the second feature image is as follows.

[0082] (4); in, ∈R H×W×3C is the second feature image after multi-scale fusion.

[0083] Further, in a possible embodiment, referring to Figure 12 、 Figure 13 、 Figure 14 As shown, Figure 13 This embodiment of the present application provides Figure 11 An optional flowchart of step S1104 in FIG. Figure 14 This is an optional structural diagram of the pool attention mechanism layer provided in an embodiment of the present application. The multi-scale attention module also includes a pool attention mechanism layer. The pool attention mechanism layer includes a channel attention module and a spatial attention module. The channel attention module is communicated with the output layer of the multi-scale pooling layer. The spatial attention module is communicated with the channel attention module. The channel attention module includes a fully connected layer. The spatial attention module includes a convolutional layer. The convolutional layer and the fully connected layer are both provided with a Sigmoid activation function. The method may include but is not limited to steps S1301 to S1306.

[0084] Step S1301: Perform global average pooling on the eighth feature image through the channel attention module to obtain a channel description vector of the eighth feature image.

[0085] Step S1302: Using the Sigmoid activation function in the fully connected layer, weight calculations are performed on the channel description vectors to obtain the channel attention weights corresponding to the eighth feature image.

[0086] Step S1303: Through the fully connected layer, the channel attention weight is multiplied by each feature channel in the eighth feature image to obtain a channel-weighted ninth feature image.

[0087] Step S1304: The ninth feature image is sent to the spatial attention module for maximum pooling and average pooling respectively, to obtain the tenth feature image described by the maximum pooling space and the eleventh feature image described by the average pooling space.

[0088] Step S1305: Calculate the spatial description weight between the tenth feature image and the eleventh feature image through the Sigmoid activation function in the fully connected layer to obtain the spatial attention weight.

[0089] Step S1306: Through the convolution layer, multiply the spatial attention weight by each feature channel of the ninth feature image to obtain the second feature image.

[0090] In steps S1301 to S1306, on the basis of feature fusion of the multi-scale pooling layer, weights are dynamically allocated through the pool attention mechanism layer to further highlight the defect area. The pool attention mechanism layer includes a channel attention module and a spatial attention module. First, the channel attention module is executed to perform global average pooling (GAP) on the second feature image e obtained by performing global average pooling (GAP) on the second feature image 3C , and the channel attention weight α = σ (FC (z)) of the eighth feature image is generated through a fully connected layer and a Sigmoid activation function, and multiplied with the second feature image to obtain the ninth feature image ∈R H×W×3C , wherein the ninth feature image is calculated as follows: (5) (6) Next, through the spatial attention module, the ninth feature image after channel weighting is respectively subjected to maximum pooling and average pooling in the channel dimension to obtain the tenth feature image F max ∈R H×W×1 about maximum pooling spatial description and the eleventh feature image F avg ∈R H×W× about average pooling spatial description. After splicing the two spatial description maps, the spatial attention weight between the two spatial description maps is generated through a fully connected layer and a Sigmoid function, and the spatial attention weight is multiplied with the ninth feature image after channel weighting to obtain the final second feature image F final ∈R H×W×3C . Wherein, the tenth feature image F max , the eleventh feature image F avg , and the second feature image F final are calculated as follows.

[0091] (7) (8) (9) Further, in one possible embodiment, referring to Figure 15 and Figure 16 , Figure 15 is an optional flowchart of step S303 in Figure 3 provided by the embodiments of the present application,Figure 16 is an optional structure diagram of a dual-path cooperative detection head provided by the embodiment of the present application. The dual-path cooperative detection head comprises a multi-element branch block, a positioning branch, and a classification branch. The multi-element branch block is in communication connection with the positioning branch and the classification branch respectively. The method can include but is not limited to steps S1501 to S1504.

[0092] Step S1501: input the second feature image into the multi-element branch block for feature enhancement to obtain a twelfth feature image.

[0093] Step S1502: through the classification branch, calculate the type probability of the defect feature of the twelfth feature image to obtain a defect type probability distribution map.

[0094] Step S1503: through the positioning branch, perform coordinate mapping on the defect feature of the twelfth feature image to obtain a defect position distribution map.

[0095] Step S1504: combine the defect type probability distribution map and the defect position distribution map through the output layer of the dual-path cooperative detection head to obtain a defect prediction image.

[0096] In steps S1501 to S1504, the dual-path cooperative detection head adopts a hierarchical progressive feature processing strategy, uses the multi-element branch block to extract the shared features of the second feature image to obtain the twelfth feature image, then performs defect prediction on the twelfth feature image through the classification branch and the positioning branch to obtain the defect type probability distribution map and the defect position distribution map, and finally combines the two maps to obtain the defect prediction image.

[0097] In the defect detection link of the insulator string by the classification branch, the model adopts a multi-stage detection framework based on deep learning to perform fine processing on the input features. First, the Softmax function is introduced in the classification branch to generate a high-precision initial defect type probability distribution map . For the probability distribution map, the dual-path cooperative detection head also implements a dynamic threshold segmentation strategy to convert the continuous probability value into a binary defect mask by setting a threshold θ to obtain the defect type probability distribution map after the defect mask , wherein the initial defect type probability distribution map , the defect type probability distribution map The calculation process is as follows.

[0098] (10) (11) In the defect positioning phase of the positioning branch, the model maps the image coordinates to the geographic coordinate system through the coordinate transformation matrix, realizes the accurate positioning of the defect position. At the same time, the model outputs the insulator string defect detection results of each position, realizes the accurate matching of the insulator string defect and the position, and greatly reduces the labor cost. The whole processing flow is deeply optimized at the algorithm level, which meets the real-time requirement of unmanned aerial vehicle inspection while ensuring the detection accuracy.

[0099] It should be noted that the dual-path cooperative detection head separates the classification task and the positioning task, and then the dual-path cooperative detection head can significantly improve the classification accuracy and positioning accuracy of the target in a complex background while reducing parameter redundancy.

[0100] Further, in one possible embodiment, referring again to Figure 16 As shown, the diversified branch block includes a first convolution branch, a second convolution branch, a third convolution branch, a fourth convolution branch, and a branch connection layer. The first convolution branch includes a first convolution kernel of 1x1. The second convolution branch includes a second convolution kernel of 1x1 and a third convolution kernel of KxK. The second convolution kernel is communicatively connected with the third convolution kernel. The third convolution branch includes a fourth convolution kernel of 1xK. The fourth convolution branch includes a fifth convolution kernel of Kx1. As shown, Figure 17 Figure 17 is an optional flowchart of step S1501 in Figure 15 The method can include, but is not limited to, steps S1701 to S1705.

[0101] Step S1701: input the eleventh feature image into the first convolution branch, so that the first convolution kernel performs average pooling on the eleventh feature image to obtain a twelfth feature image for expressing global context information.

[0102] Step S1702: input the eleventh feature image into the second convolution branch, so that the second convolution kernel and the third convolution kernel capture spatial correlation of the eleventh feature image to obtain a thirteenth feature image for expressing local spatial correlation.

[0103] Step S1703: input the eleventh feature image into the third convolution branch, so that the fourth convolution kernel captures spatial correlation of the eleventh feature image to obtain a fourteenth feature image for expressing direction sensitivity.

[0104] Step S1704: input the eleventh feature image into the fourth convolution branch, so that the fifth convolution kernel captures spatial correlation of the eleventh feature image to obtain a fifteenth feature image for expressing direction sensitivity.

[0105] ​Step S1705: connecting the twelfth feature image, the thirteenth feature image and the fourteenth feature image through the branch connection layer to obtain the twelfth feature image.

[0106] In the steps S1701 to S1705, the core module of the dual-path cooperative detection head includes a diversified branch block, a positioning branch and a classification branch. In the training stage, the dual-path cooperative detection head first extracts multi-scale shared features of the eleventh feature image through the diversified branch block to obtain the twelfth feature image.

[0107] It should be noted that after obtaining the defect detection model, the following steps are included: based on the zero padding and parameter superposition method, the first convolution branch, the second convolution branch, the third convolution branch and the fourth convolution branch are combined in a reparameterization form to obtain a fifth convolution branch. The fifth convolution branch includes a sixth convolution kernel of KxK, which realizes efficient calculation in the inference stage. This design expands the feature representation space in the training through branch diversity, and then converts the multi-branch structure into a single convolution operation through the reparameterization technology, thereby maintaining lightweight in the inference stage, enhancing the feature diversity while avoiding additional calculation overhead in the inference stage, solving the pain point of calculation redundancy of the traditional multi-branch structure.

[0108] Further, in one possible embodiment, referring again to Figure 16 , the classification branch includes a classification convolution layer and a CLS loss layer, the classification convolution layer is in communication connection with the CLS loss layer, the classification convolution layer is provided with a binary cross-entropy loss function, and referring to Figure 18 , the CLS loss layer is provided with a focal loss function. Figure 18 is an optional flowchart of step S1502 in Figure 15 the method provided by the embodiment of the present application, which can include but is not limited to steps S1801 to S1803.

[0109] Step S1801: performing type mapping on the defect features of the twelfth feature image through the compressed feature channel of the classification convolution layer to obtain a first defect type image.

[0110] Step S1802: performing multi-type probability prediction and focal loss on each anchor point in the first defect type image through the CLS loss layer to obtain a second defect type image.

[0111] Step S1803: performing weighted processing on the feature channels of the second defect type image through the channel attention mechanism of the classification branch to obtain a defect type probability distribution map.

[0112] In steps S1801 to S1803, the classification branch includes a classification convolutional layer and a CLS loss layer. For the twelfth feature image after feature enhancement of the diversified branch block, the classification convolutional layer maps the defect features of the twelfth feature image by compressing the feature channels through a 1x1 convolutional kernel to generate a first defect type image of the class-related feature mapping. Then the CLS loss layer adopts a binary cross-entropy loss function to perform multi-class probability prediction on each anchor point in the first defect type image, and combines a focal loss mechanism to relieve the class imbalance problem to obtain a second defect type image. To adapt to the singularity of image features, the classification branch also introduces a channel attention mechanism to dynamically weight the features of different channels of the second defect type image, enhancing the expression ability of key information.

[0113] Further, in one possible embodiment, referring again to Figure 16 Fig. 7, the positioning branch includes a detection frame convolutional layer, a DFL loss layer, and a CIOU loss layer. The detection frame convolutional layer is in communication connection with the DFL loss layer, and the DFL loss layer is in communication connection with the CIOU loss layer. The DFL module is provided with a cross-entropy loss function, as shown in Figure 19 Figure 19 is an optional flowchart of step S1503 in Figure 15 the method provided by the embodiments of the present application, which can include but is not limited to steps S1901 to S1903.

[0114] Step S1901: adjusting the channels of the twelfth feature image through the detection frame convolutional layer to obtain a first defect position image with a bounding box attribute.

[0115] Step S1902: discretely regressing and optimizing the bounding box coordinate values in the first defect position image through the DFL loss layer to obtain a second defect position image.

[0116] Step S1903: adjusting the weights of the overlapping area, center point distance, and aspect ratio of the second defect position image through the CIOU loss layer to obtain a defect position distribution map.

[0117] ​In steps S1901 to S1903, first, for the diversified branch block after the feature enhancement of the twelfth feature image, the bounding box convolution layer adjusts the channel of the twelfth feature image to generate a first defect position image with an initial prediction of the bounding box. Then, the DFL loss layer predicts the discrete distribution, optimizes the bounding box coordinate value in the first defect position image, converts the continuous bounding box coordinate value into a probability distribution of 16 reference values, and uses the cross-entropy loss function to guide the network to focus on the prediction area near the true value to obtain a second defect position image with optimized true coordinates. Finally, the CIOU loss layer dynamically adjusts the weights of the three geometric factors of the overlapping area, the center distance and the aspect ratio of the second defect position image through the CIOU loss function, and further improves the bounding box regression accuracy in the defect position distribution map. Through multi-level feature fusion and geometric constraint, the positioning branch can significantly improve the positioning robustness of fuzzy or small targets in the image.

[0118] The embodiments of the present application also provide an electronic device, which comprises a memory and a processor. The memory stores a computer program, and the processor implements the defect detection method for insulator strings of a UAV when executing the computer program. The electronic device can be any intelligent terminal, such as a mobile phone, a tablet computer, a vehicle-mounted computer, etc.

[0119] Please refer to Figure 20 , Figure 20 is a hardware structure schematic diagram of an electronic device provided by an embodiment of the present application. The electronic device comprises: The processor 2001 can be implemented in the form of a general-purpose CPU (Central Processing Unit, central processor), a microprocessor, an ASIC (Application Specific Integrated Circuit, application-specific integrated circuit), or one or more integrated circuits, etc., and is used to execute related programs to implement the defect detection method for insulator strings of a UAV provided by the embodiments of the present application. The memory 2002 can be implemented in the form of a ROM (ReadOnly Memory, read-only memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory, random access memory). The memory 2002 can store an operating system and other application programs. When the technology provided by the embodiments of the present application is implemented by software or firmware, the related program codes are saved in the memory 2002 and are called and executed by the processor 2001 to implement the defect detection method for insulator strings of a UAV provided by the embodiments of the present application. The input / output interface 2003 is used to realize information input and output. The communication interface 2004 is configured to realize the communication interaction between the device and other devices, and can realize the communication through a wired manner (for example, a USB, a network cable, or the like) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, or the like). The bus 2005 is configured to transmit information between various components (for example, the processor 2001, the memory 2002, the input / output interface 2003, and the communication interface 2004) of the device. The processor 2001, the memory 2002, the input / output interface 2003, and the communication interface 2004 are connected to each other through the bus 2005.

[0120] The application also provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method for detecting defects of an insulator string by a UAV is provided.

[0121] The memory is a non-transitory computer readable storage medium, and can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0122] The embodiments described in the application are used to more clearly illustrate the technical solutions of the application, and do not constitute a limitation on the technical solutions provided by the application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the application are also applicable to similar technical problems.

[0123] Those skilled in the art can understand that the technical solutions shown in the figure do not constitute a limitation on the application, and can include more or fewer steps than the figure, or combine certain steps, or different steps.

[0124] The device embodiments described above are only schematic, and the units described as separate components can be or can not be physically separated, that is, can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the modules can be selected to realize the purpose of the embodiments.

[0125] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the function modules / units in the system and the device can be implemented as software, firmware, hardware or appropriate combination thereof.

[0126] The terms "first", "second", "third", "fourth" and the like in the description of the application and in the claims, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of these terms herein is to be construed to cover a general order and / or structure unless otherwise indicated. Furthermore, the terms "comprise", "comprising", "has", "having", "includes", "including", "contain", "containing" or any other similar forms are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises, has, includes, contains items or components does not include items or components not explicitly recited. The terms "a" or "an", as used herein in the detailed description and in the claims, mean "one or more" or "at least one", unless otherwise indicated.

[0127] It should be understood that, in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are three cases: only A, only B, and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be singular or plural.

[0128] In several embodiments provided in the present application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the above-described system embodiments are only illustrative, for example, the division of the above-mentioned units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0129] The units described as separate components above can or can not be physically separate, and the components shown as units can or can not be physical units, that is, can be located in one place, or can be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0130] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0131] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer accessible storage medium. Based on such understanding, the technical scheme of the present application or the part that contributes to the prior art or the whole or part of the technical scheme can be embodied in the form of a software product. The computer software product is stored in a storage medium, including multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0132] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, but this does not limit the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A defect detection method for an insulator string using an unmanned aerial vehicle, characterized in that: include: Obtain target images of insulator strings taken by drones and a defect detection model based on the improved YOLOv5 convolutional layer; Performing defect detection on the target image using the defect detection model to obtain a defect detection result of the insulator string; The process of constructing the defect detection model includes the following steps: Acquire a plurality of sample images, wherein the sample images are images of insulator strings marked with defects; Initializing a network architecture improved based on the YOLOv5 convolutional layer, the network architecture comprising a grouping dimension-raising module for performing convolution processing on the target image, a multi-scale attention module for extracting defect features on the target image, and a dual-path collaborative detection head for performing defect reasoning on the target image, the grouping dimension-raising module being communicatively connected to the multi-scale attention module, and the multi-scale attention module being communicatively connected to the dual-path collaborative detection head; The network architecture is trained based on the plurality of sample images to obtain the defect detection model.

2. The defect detection method according to claim 1, characterized in that: The iterative training of the network architecture based on the plurality of sample images to obtain the defect detection model comprises the following steps: Inputting the sample image into the grouping and dimensionality increasing module for feature expansion to obtain a first feature image; Inputting the first feature image into the multi-scale attention module to perform multi-scale feature extraction to obtain a second feature image; Inputting the second feature image into the dual-path collaborative detection head for defect prediction to obtain a defect prediction image; Positionally pairing the defect prediction image with the sample image to obtain difference data between the sample image and the defect prediction image; Based on the loss function, a gradient difference calculation is performed on the difference data to obtain a gradient loss value of the current iteration; Based on the gradient loss value, back-propagation updates are performed on the multi-layer network parameters of the network architecture through an iterative optimizer; If the multi-layer network parameters of the network architecture meet the preset training requirements, the iterative training of the network architecture is terminated to obtain the defect detection model.

3. The defect detection method according to claim 2, characterized in that: The group dimension-raising module includes a dual-channel extension architecture and a group convolution module, wherein the dual-channel extension architecture is communicatively connected to the group convolution module; The step of inputting the sample image into the grouping and dimensionality increasing module for feature expansion to obtain a first feature image comprises the following steps: Inputting the sample image into the input layer of the network architecture for feature extraction to obtain a second feature image, where the second feature image includes a plurality of feature channels; Inputting the second feature image into the dual-channel expansion architecture for feature expansion to obtain a third feature image, wherein the number of feature channels of the third feature image is greater than the number of feature channels of the second feature image; The third feature image is input into the group convolution module for feature compression to obtain the first feature image, wherein the number of feature channels of the first feature image is less than the number of feature channels of the third feature image.

4. The defect detection method according to claim 3, characterized in that: The dual-channel expansion architecture includes an expansion path and a compensation path, the expansion path includes a first convolutional layer and a multi-layer perceptron, the compensation path includes a second convolutional layer and a feature decoupler, and the expansion path is related to and parallel to the compensation path; The step of inputting the second feature image into the dual-channel expansion architecture to perform feature expansion to obtain a third feature image comprises the following steps: Inputting the second feature image into the first convolutional layer for convolution operation, and performing feature expansion on the convolved first feature image in a feature channel combination manner through the multi-layer perceptron to obtain a fourth feature image, wherein the fourth feature image includes a first feature channel for expressing a fusion of local texture features and global structural features of the sample image; Inputting the second feature image into the second convolutional layer for convolution operation, and performing feature expansion on the convolved first feature image in a feature channel decomposition manner by the feature decoupler to obtain a fifth feature image, wherein the fifth feature image includes a second feature channel for expressing edge detail features of the sample image; The fourth feature image and the fifth feature image are spliced ​​in the channel dimension to obtain the third feature image.

5. The defect detection method according to claim 3, characterized in that: Inputting the third feature image into the group convolution module for feature compression to obtain the first feature image includes: performing grouping processing on all feature channels of the third feature image to obtain a grouped third feature image, wherein the grouped third feature image includes a plurality of channel groups, and the channel group includes a plurality of feature channels; Performing a convolution operation on all channel groups of the third feature image to obtain a first feature image set corresponding to the third feature image, where the first feature image set includes a plurality of feature sub-images, and the feature sub-images are mapped to the channel groups; Perform image stitching on all feature sub-images in the first feature image set to obtain the first feature image.

6. The defect detection method according to claim 5, characterized in that: The grouping process of all feature channels of the third feature image to obtain a grouped third feature image includes: Obtaining a preset number of feature channels output by the grouping and dimensionality-raising module, and determining a first number of feature channels of the third feature image; Determining the greatest common divisor between the number of the first feature channels and the number of the preset feature channels as the number of dynamic groupings of the third feature image; All feature channels of the third feature image are averagely grouped according to the dynamic grouping number to obtain the grouped third feature image.

7. The defect detection method according to claim 2, characterized in that: The multi-scale attention module includes a multi-scale pooling layer and a pool attention mechanism layer, the multi-scale pooling layer is communicatively connected to the pool attention mechanism layer, the multi-scale pooling layer includes a spatial pyramid pooling layer, a multi-scale group convolution layer and an upsampling layer, the spatial pyramid pooling layer is communicatively connected to the multi-scale group convolution layer, and the multi-scale group convolution layer is communicatively connected to the upsampling layer; The step of inputting the first feature image into the multi-scale attention module for multi-scale feature extraction to obtain the second feature image includes the following steps: Inputting the first feature image into the spatial pyramid pooling layer to perform multi-scale feature extraction to obtain a plurality of sixth feature images, where the spatial scales of two of the sixth feature images are the same or different; Based on the multi-scale group convolution layer, performing group convolution compression on multiple sixth feature images at the same spatial scale to obtain multiple seventh feature images with different spatial scales; Perform channel splicing on the plurality of seventh feature images through an upsampling layer to obtain a multi-scale fused eighth feature image; The eighth feature image is input into the pool attention mechanism layer for feature enhancement to obtain the second feature image for highlighting the features of the defect area.

8. The defect detection method according to claim 7, characterized in that: The multi-scale attention module also includes a pool attention mechanism layer, the pool attention mechanism layer includes a channel attention module and a spatial attention module, the channel attention module is communicatively connected to the output layer of the multi-scale pooling layer, the spatial attention module is communicatively connected to the channel attention module, the channel attention module includes a fully connected layer, the spatial attention module includes a convolutional layer, and the convolutional layer and the fully connected layer are both provided with a Sigmoid activation function; The step of inputting the eighth feature image into the pool attention mechanism layer for feature enhancement to obtain the second feature image for highlighting the features of the defect area includes the following steps: Performing global average pooling on the eighth feature image through the channel attention module to obtain a channel description vector of the eighth feature image; Performing weight calculations on the channel description vectors respectively through the Sigmoid activation function in the fully connected layer to obtain the channel attention weights corresponding to the eighth feature image; multiplying the channel attention weight by each feature channel in the eighth feature image through the fully connected layer to obtain a channel-weighted ninth feature image; Submitting the ninth feature image to the spatial attention module for maximum pooling and average pooling, respectively, to obtain a tenth feature image described by the maximum pooling space and an eleventh feature image described by the average pooling space; Calculating a spatial description weight between the tenth feature image and the eleventh feature image through a Sigmoid activation function in the fully connected layer to obtain a spatial attention weight; Through the convolution layer, the spatial attention weight is multiplied by each feature channel of the ninth feature image to obtain the second feature image.

9. The defect detection method according to claim 2, characterized in that: The dual-path collaborative detection head includes a diversified branch block, a positioning branch, and a classification branch, wherein the diversified branch block is communicatively connected to the positioning branch and the classification branch respectively; The step of inputting the second feature image into the dual-path collaborative detection head for defect prediction to obtain a defect prediction image comprises the following steps: Inputting the second feature image into the diversified branch block for feature enhancement to obtain a twelfth feature image; By using the classification branch, type probability calculation is performed on the defect features of the twelfth feature image to obtain a defect type probability distribution map; By means of the positioning branch, coordinate mapping is performed on the defect features of the twelfth feature image to obtain a defect position distribution map; The output layer of the dual-path collaborative detection head is combined with the defect type probability distribution map and the defect position distribution map to obtain the defect prediction image.

10. The defect detection method according to claim 9, characterized in that: The diversified branch block includes a first convolution branch, a second convolution branch, a third convolution branch, a fourth convolution branch and a branch connection layer, wherein the first convolution branch includes a 1×1 first convolution kernel, the second convolution branch includes a 1×1 second convolution kernel and a K×K third convolution kernel, the second convolution kernel is communicatively connected to the third convolution kernel, the third convolution branch includes a 1×K fourth convolution kernel, and the fourth convolution branch includes a K×1 fifth convolution kernel; Furthermore, the step of inputting the eleventh feature image output by the multi-scale attention module into the diversified branch block for feature enhancement to obtain the twelfth feature image comprises the following steps: Inputting the eleventh feature image into the first convolution branch so that the first convolution kernel performs average pooling on the eleventh feature image to obtain a twelfth feature image for expressing global context information; Inputting the eleventh feature image into the second convolution branch, so that the second convolution kernel and the third convolution kernel capture the spatial correlation of the eleventh feature image, to obtain a thirteenth feature image for expressing the local spatial correlation; Inputting the eleventh feature image into the third convolution branch respectively, so that the fourth convolution kernel captures the spatial correlation of the eleventh feature image, thereby obtaining a fourteenth feature image for expressing directional sensitivity; Inputting the eleventh feature image into the fourth convolution branch respectively, so that the fifth convolution kernel captures the spatial correlation of the eleventh feature image, thereby obtaining a fifteenth feature image for expressing directional sensitivity; connecting the twelfth characteristic image, the thirteenth characteristic image, and the fourteenth characteristic image through the branch connection layer to obtain the twelfth characteristic image; Furthermore, after obtaining the defect detection model, the following steps are included: Based on the zero padding and parameter superposition method, the first convolution branch, the second convolution branch, the third convolution branch and the fourth convolution branch are merged in a re-parameterized form to obtain a fifth convolution branch, where the fifth convolution branch includes a K×K sixth convolution kernel.

Images