Multi-size defect recognition method for transmission lines in haze scenes based on deep learning

Abstract

The present invention discloses a method for identifying multi-size defects of power transmission lines in haze scenes based on deep learning, and relates to the technical field of identification methods. The present invention comprises the following steps: Step 1: constructing a data set of power transmission line defects in haze scenes; Step 2: using an image atomization algorithm to perform data preprocessing on the data set of power transmission line defects; Step 3: constructing a transmission line defect recognition algorithm in haze scenes, which is used to identify fuzzy defect targets and small-size defect targets of the power transmission lines in haze scenes; Step 4: performing algorithm training and verification based on the transmission line defect recognition algorithm constructed in step 3. The present invention simulates the transmission line defect data in haze scenes by performing data preprocessing of adding fog to the image data after collecting the transmission line defect data set.

Description

Technical Field

[0001] The present invention belongs to the field of multi-size defect recognition methods, and specifically, relates to a multi-size defect recognition method for power transmission lines in haze scenarios based on deep learning. Background Art

[0002] With the rapid development of industrialization and urbanization, air pollution has become a global problem, and the impact of haze environment on transmission lines has gradually become prominent. In haze weather, transmission lines may be affected by factors such as covering, sediment and moisture, resulting in the formation or aggravation of line defects. The safe and stable operation of transmission lines is essential, and regular inspections are the key to ensuring this. The traditional manual inspection mode is costly and difficult in sparsely populated urban fringe areas or mountainous areas with complex terrain. With the continuous development of technology, more and more power grid units have turned to automated inspections. UAVs or fixed cameras are used to obtain inspection images of transmission lines, and line defects are discovered through artificial intelligence recognition algorithms. This "human inspection as a supplement, machine inspection as the main" model can not only reduce costs and improve efficiency, but also reduce dependence on personnel, increase the coverage and depth of inspections, and thus improve the overall safety and stability of transmission lines.

[0003] It is difficult to identify defects in transmission lines under haze conditions, which may cause power transmission safety problems. In order to ensure the stable operation and reliable power supply of the power system, it is necessary to identify defects in transmission lines under haze conditions. When identifying defects in transmission lines, fixed cameras are installed on transmission towers or other equipment, or drones are used to collect images of transmission lines and collect inspection images of transmission lines.

[0004] The identification and judgment of transmission line defects in traditional haze environments requires manual screening of data collected from inspection images. This process is inefficient and requires a lot of professional manpower and time costs. In view of the shortcomings of the traditional manual identification method of transmission line defects in haze environments, if the automatic algorithm such as deep learning is used to automatically screen and identify defects from the data extracted from the transmission line inspection images collected by drones or fixed cameras, the time consumption and human resources of the process can be significantly reduced. In general, the study of relevant methods for solving the problem of transmission line defect identification in haze environments through deep learning can significantly improve the efficiency and quality of transmission line defect identification in haze environments, reduce dependence on personnel, and improve the coverage and depth of inspections.

[0005] Chinese patent document number CN117470976A, published on January 30, 2024, describes a "transmission line defect detection method and system based on voiceprint features". This method collects the voiceprint feature matrix of the transmission line when it is working, and sends the collected voiceprint feature matrix to the transmission line defect detection model for detection to determine whether the transmission line has defects, thereby realizing the detection of transmission line defects.

[0006] Chinese patent document number CN117557504A, published on February 13, 2024, describes a "transmission line defect detection method". This method uses the Faster-RCNN network to train inspection transmission line images containing defects to obtain a model for detecting transmission line defects.

[0007] The prior art has the following technical problems:

[0008] (1) Visibility is poor in a haze environment, and traditional manual inspection methods are prone to missed inspections or misjudgments, and the reliability of the inspection results is low. In addition, traditional inspection methods are usually intermittent inspections, which makes it difficult to achieve real-time identification and timely processing of transmission line defects, and easily cause delays and losses due to line failures.

[0009] (2) At present, the identification of transmission line defects in most haze weather conditions requires a large number of special sensors, such as infrared sensors and laser sensors. Such sensors require a lot of professional costs. In addition, complex mathematical models need to be designed for such identification methods, making the identification process cumbersome and time-consuming.

[0010] In view of this, the present invention is proposed. Summary of the invention

[0011] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a method for identifying multi-size defects of power transmission lines in haze scenes based on deep learning, thereby solving the problems raised in the above-mentioned background technology.

[0012] In order to solve the above technical problems, the basic concept of the technical solution adopted by the present invention is:

[0013] A method for identifying multi-size defects of power transmission lines in haze scenes based on deep learning, comprising the following steps:

[0014] Step 1: Construct a data set of power transmission line defects under haze scenarios;

[0015] Step 2: Use image atomization algorithm to preprocess the data set of power transmission line defects;

[0016] Step 3: Construct a transmission line defect recognition algorithm in haze scenarios to identify fuzzy targets and small-sized defects of transmission lines in haze scenarios;

[0017] Step 4: Perform algorithm training and verification based on the transmission line defect recognition algorithm constructed in Step 3;

[0018] Step 5: After training the transmission line defect recognition algorithm, use the haze scene transmission line image data test set preprocessed in Step 2 to conduct a transmission line defect recognition test.

[0019] Optionally, in the process of constructing the power transmission line defect dataset under the haze scenario in Step 1, images of the power transmission lines with defects are collected through drone aerial photography or cameras fixed on the transmission towers. After the transmission line defect images are collected, all the transmission line defect images are labeled one by one according to coordinates and categories for subsequent algorithm training.

[0020] Optionally, in Step 2, fog is added to the power transmission line defect dataset using the numpy library and the opencv library to simulate the power transmission line scene under haze conditions and introduce random concentration changes in different areas of the fog.

[0021] Optionally, the modules used to identify multi-sized defects of power transmission lines in haze scenes include a location information grouping capture convolution module LGC, a small target defect enhanced recognition head PBS-Head, a lightweight defect target recognition head SBF-Head, and a coordinate attention module CA.

[0022] Optionally, Step 3 constructs a transmission line defect recognition algorithm in a haze scene, and the steps for identifying fuzzy targets and small-sized defects of the transmission line in the haze scene are as follows:

[0023] Step3.1: The position information grouping capture convolution module LGC is adopted. For the input feature map F1, the LGC module performs an SGCConv convolution on it, and then inputs it into the BN layer and the ReLU layer to obtain the feature map F2. Then F2 is input into a module that splits the feature map along the channel dimension. The channel dimension divides the feature map into two parts, one part is the feature map F3 that is not processed, and the other part of the feature map F4 is input into the combination of the SGCConv convolution, the BN layer and the ReLU layer to obtain the feature map F5. The feature map F5 is input into the coordinate attention CA module output and then the output feature map F6 is output. Then F6 is input into the convolution layer, the BN layer and the ReLU layer in sequence to obtain the output feature map F7. F7 is spliced ​​with the feature maps F2 and F3 to obtain the feature map F8. Finally, F8 is subjected to a layer of group convolution to obtain the final output feature map F9 of the LGC module; the position information grouping capture convolution module LGC is adopted to strengthen the attention to the position information of the fuzzy transmission line defect feature image in the haze scene, and better fit the fuzzy position in the feature map.

[0024] Step 3.2: Use the enhanced recognition head PBS-Head for small target defects to enhance the recognition effect of small-sized defect targets in the feature image, while reducing computational redundancy and memory access during the reasoning process.

[0025] Step 3.3: When identifying defective targets of larger size, the lightweight defect target recognition head SBF-Head does not need a highly accurate recognition head, so the network can be lightweighted by reducing the number of parameters in the recognition head;

[0026] Step 3.4: Combine the PBS-Head in Step 3.2 and the SBF-Head in Step 3.3 to construct a multi-scale defect recognition head module for the transmission line defect recognition network;

[0027] Step 3.5: Combine the multi-scale transmission line defect recognition module designed in Step 3.4 and the LGC module designed in Step 3.1 to construct a multi-scale defect recognition algorithm network for transmission lines in haze scenarios, such as Figure 2 .

[0028] Optionally, the steps of obtaining the final output feature map F9 through the convolution module LGC are:

[0029] Step 3.1.1: The transmission line defect feature map F1 is convolved by an SGCConv. SGCConv takes out the feature map of 1 / 2 channels of F1 for group convolution operation, and convolves the feature map of the other half of the channels. Finally, the results processed by SGCConv are concatenated in the channel dimension to obtain a feature map with half the number of input channels. Finally, the output feature map is input into a BN layer and a ReLU layer in sequence to obtain the transmission line defect feature map F2.

[0030] Step 3.1.2: Divide the feature map F2 into half of the number of input channels in the channel dimension, and obtain feature maps F3 and F4 with channel numbers 1 / 2 of the number of channels of F2.

[0031] Step 3.1.3: Input the feature map F4 into an SGCConv convolution, and then input it into the BN layer and ReLU layer in sequence to obtain the transmission line defect feature map F5 whose output channel number is twice the number of input channels.

[0032] Step 3.1.4: Input the feature map F5 into a coordinate attention module CA to obtain a transmission line defect feature map F6 with the same number of channels.

[0033] Step 3.1.5: Convolve the feature map F6 to obtain a transmission line defect feature map F7 whose output channel number is half the number of input channels.

[0034] Step 3.1.6: Concatenate features F2, F3, and F7 in the channel dimension to obtain a transmission line defect feature map F8 whose channel number is the sum of the channel numbers of F2, F3, and F7.

[0035] Step 3.1.7: Convolve the feature map F8, and then input it into the BN layer and ReLU layer in sequence to obtain a transmission line defect feature map F9 with an output channel number equal to the input channel number.

[0036] Optionally, the steps of using PBS-Head recognition to output feature maps are:

[0037] Step 3.2.1: Input the transmission line defect feature map X1 into two processing branches for regression and classification of transmission line defects;

[0038] Step 3.2.2: Input X1 into the PConv layer to obtain an output result in which the number of output channels is equal to the number of input channels; then input the result into the ReLU layer to obtain the power transmission line defect feature map X2, input X2 into the convolution layer, the number of output channels is equal to the number of input channels, and then input another BN layer to obtain the power transmission line defect feature map X3. Finally, input the feature map X3 into the convolution for target recognition frame regression, and output the feature map X4 with a channel number of 2, where the channel number 2 represents the upper left and lower right coordinates of the recognition frame;

[0039] Step 3.2.3: Input X1 to the PConv layer to obtain an output result with the same number of output channels as the number of input channels; then input the result to the ReLU layer to obtain the power line defect feature map X5, input X5 to the convolution layer, and then input it to a BN layer to obtain the power line defect feature map X6, and finally, input the feature map X6 to the convolution for target classification, and the output channel number is X7, which is the number of transmission line defect classifications;

[0040] Step3.2.4: Concatenate the output feature map X4 of Step3.1.2 and the output feature map X7 of Step3.1.3 along the channel dimension to obtain the output feature map X8. The number of channels of feature map X8 is the sum of the number of channels of X4 and X7.

[0041] Optionally, the steps of using a lightweight defect target recognition head SBF-Head to recognize the output feature map are:

[0042] Step 3.3.1: Input the transmission line defect feature Y1 into an SGCConv convolution with an input grouping number of 2 to obtain the output result. The number of output channels is half of the number of input channels. Then input the result into the ReLU layer to obtain the transmission line defect feature map Y2;

[0043] Step 3.3.2: Input the power transmission line defect feature map Y2 into the convolution layer, and then input a BN layer and a ReLU layer to obtain the power transmission line defect feature map Y3. The number of output channels is equal to the number of input channels, and the size becomes 1 / 2 of the original size.

[0044] Step 3.3.3: Input the feature map Y3 into the convolution for target regression and the convolution for target classification, respectively, to obtain an output feature map Y4 with an output channel number of 2 and an output feature map Y5 with an output channel number equal to the number of transmission line defect classifications;

[0045] Step3.3.4: Concatenate the output feature maps Y4 and Y5 of Step3.3.3 along the channel dimension to obtain the output feature map Y6.

[0046] Optionally, the steps of algorithm training and verification in Step 4 based on the transmission line defect recognition algorithm constructed in Step 3 are as follows:

[0047] Step 4.1: Initialize the number of training times, momentum factor, batch size, optimizer, learning rate type and initial learning rate of the transmission line defect recognition algorithm;

[0048] Step 4.2: Divide the transmission line defect recognition data set constructed in Step 1 into a training set, a validation set, and a test set for algorithm training. In each round of training, the algorithm grabs the transmission line defect recognition data in the training set according to the batch processing preset value for training. The algorithm optimizes the parameters of each module in the algorithm through classification loss and regression loss back propagation. The regression loss function uses the WIoUv3 loss function, and the classification loss function uses the VFL function;

[0049] Step 4.3: After each round of training is completed, the transmission line defect recognition data in the training set is captured according to the batch processing preset value for verification to check the effectiveness of each round of training. When the algorithm training reaches the convergence of the loss value, the transmission line defect recognition algorithm training is completed.

[0050] After adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art. Of course, any product implementing the present invention does not necessarily need to achieve all the advantages described below at the same time:

[0051] (1) After collecting the power transmission line defect data set, the present invention performs data preprocessing by adding fog to the image data to simulate the power transmission line defect data in a haze scene.

[0052] (2) The present invention designs an LGC module, in which SGCConv convolution is used to reduce the number of module parameters, and long connection operations are used to enhance the module's ability to extract features from blurred images of transmission line defects.

[0053] (3) The present invention designs a PBS-Head recognition head responsible for identifying small-sized defects in the transmission line, and uses partial convolution in the recognition head to enhance the recognition capability of fuzzy small targets in the transmission line.

[0054] (4) The present invention designs an SBF-Head recognition head to identify large-sized defects in transmission lines. The SBF-Head recognition head integrates the dual-branch structure of PBS-Head and replaces part of the convolution with SGCConv, which speeds up the algorithm's reasoning speed.

[0055] The specific implementation modes of the present invention are further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The drawings described below are only some embodiments. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0057] In the figure:

[0058] Figure 1 The flowchart of the multi-size defect identification method for transmission lines is shown;

[0059] Figure 2 Figure 1 is a diagram of the transmission line defect identification algorithm;

[0060] Figure 3 This is the LGC module structure diagram;

[0061] Figure 4 This is the structure diagram of PBS-Head;

[0062] Figure 5 This is the SBF-Head structure diagram.

[0063] It should be noted that these drawings and textual descriptions are not intended to limit the conceptual scope of the present invention in any way, but are intended to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0064] The present invention will now be described in further detail with reference to the accompanying drawings.

[0065] See also Figure 1-5 As shown, in this embodiment, a multi-size defect recognition method for power transmission lines in haze scenes based on deep learning is provided. The modules used include a position information grouping capture convolution module LGC, a small target defect enhanced recognition head PBS-Head, a lightweight defect target recognition head SBF-Head, and a coordinate attention module CA, including the following steps:

[0066] Step 1: Construct a data set of power transmission line defects under haze scenarios;

[0067] Step 1: In the process of constructing a transmission line defect dataset under haze scenarios, images of defective transmission lines are collected through drone aerial photography or cameras fixed on transmission towers. After the transmission line defect image collection is completed, all transmission line defect images are labeled one by one according to coordinates and categories for subsequent algorithm training.

[0068] Step 2: Use image atomization algorithm to preprocess the data set of power transmission line defects;

[0069] It should be noted that fog is added to the power transmission line defect dataset through the numpy library and the opencv library to simulate the power transmission line scene under haze conditions and introduce random concentration changes in different areas of the fog.

[0070] Example: Assume that the power transmission line defect image to be processed is read in as variable img. First, use the ones_like()*(200*200*200) method of the numpy library to create a fog image fog_density filled with a constant color, where (200*200*200) is the RGB color parameter for simulating haze. Then use the uniform(0.1,0.3) method of the random library to generate a random fog density fog_density for each image, where the random number range is set to 0.1 to 0.3. Finally, call the addWeighted(img*(1-fog_density),(mask*fog_density).astype(np.uint8)) method of the opencv library to apply fog to the image, where 1-fog_density is the weight of the original image.

[0071] Step 3: Construct a transmission line defect recognition algorithm in haze scenes, which is used to identify fuzzy targets and small-sized defects in transmission lines in haze scenes. The network structure of the transmission line defect recognition algorithm constructed by the present invention is as follows: Figure 2 shown.

[0072] Step 3: Construct a transmission line defect recognition algorithm in haze scenes. The steps for identifying fuzzy targets and small-sized defects of transmission lines in haze scenes are as follows:

[0073] Step 3.1: The location information group capture convolution module LGC is used. For the input feature map F1, the LGC module performs an SGCConv (Split Group Capture Conv) convolution on it, and then inputs it into the BN layer and the ReLU layer to obtain the feature map F2. Then F2 is input into a module that splits the feature map along the channel dimension. The channel dimension divides the feature map into two parts, one part is the unprocessed feature map F3, and the other part of the feature map F4 is input into the combination of the SGCConv convolution, the BN layer and the ReLU layer to obtain the feature map F5; the feature map F5 is input into the coordinate attention CA module output and then the output feature map F6 is output; F6 is input into the convolution layer, the BN layer and the ReLU layer in sequence to obtain the output feature map F7; F7 is spliced ​​with the feature maps F2 and F3 to obtain the feature map F8; F8 is finally subjected to a layer of group convolution to obtain the final output feature map F9 of the LGC module. The location information group capture convolution module designed by the present invention is as follows Figure 3 shown.

[0074] In this embodiment, the steps of obtaining the final output feature map F9 through the convolution module LGC are:

[0075] Step3.1.1: The transmission line defect feature map F1 is convolved by an SGCConv. SGCConv takes out the feature map of 1 / 2 channels of F1 for group convolution operation (set the number of group convolution groups to 2, divide the input feature map into 2 groups by channel, and then perform convolution with a kernel size of 3×3, a step size of 1, and zero padding of 1 on each group. Through group convolution operation, the number of parameters in the algorithm network can be reduced). The feature map of the other half of the channels is convolved with a kernel size of 3×3, a step size of 1, and zero padding of 1. Finally, the results processed by SGCConv are spliced ​​in the channel dimension to obtain a feature map with half the number of input channels as the output channel number. Finally, the output feature map is input into a BN layer and a ReLU layer in sequence to obtain the transmission line defect feature map F2.

[0076] Step 3.1.2: Divide the feature map F2 into half of the number of input channels in the channel dimension, and obtain feature maps F3 and F4 with channel numbers 1 / 2 of the number of channels of F2.

[0077] Step 3.1.3: Input the feature map F4 into an SGCConv convolution, and then input it into the BN layer and ReLU layer in sequence to obtain the transmission line defect feature map F5 whose output channel number is twice the number of input channels.

[0078] Step 3.1.4: Input the feature map F5 into a coordinate attention module CA to obtain a transmission line defect feature map F6 with the same number of channels.

[0079] Step 3.1.5: Convolve the feature F6 with a convolution kernel size of 3×3, a step size of 2, and a zero padding of 1 to obtain a transmission line defect feature map F7 with an output channel number that is half the number of input channels.

[0080] Step 3.1.6: Concatenate features F2, F3, and F7 in the channel dimension to obtain a transmission line defect feature map F8 whose channel number is the sum of the channel numbers of F2, F3, and F7.

[0081] Step 3.1.7: Convolution is performed on feature F8 with a kernel size of 3×3, a step size of 2, and zero padding of 1, and then input into the BN layer and the ReLU layer in sequence to obtain a transmission line defect feature map F9 with an output channel number equal to the input channel number.

[0082] Embodiment: Assume that the size of the transmission line defect feature map is H×W×C, where H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map.

[0083] For example, the size of the transmission line defect feature map F1 is 512×512×128. It is input into the SGCConv convolution with 2 groups, 16 group convolution kernels, 32 convolution kernels, size 3×3, step size 1 and zero padding 1 (in the SGCConv convolution, the feature map of 1 / 2 channels in F1 is taken as the first processing branch for group convolution operation: according to the group number 2, the channel dimension is evenly divided into 2 groups of 512×512×32 feature maps, and each group is convolved with 16 convolution kernels, size 3×3 and step size 1, and 2 groups of 512×512×16 feature maps are obtained. After that, the two groups of 512×512×16 feature maps are concatenated to obtain a 512×512×32 feature map; the feature map of the other 1 / 2 channels of F1 is operated as the second processing branch: a convolution with 32 convolution kernels, a size of 3×3, a step size of 1, and a zero padding of 1 is performed to obtain a 512×512×32 feature map; finally, the feature maps processed by the above group convolution and convolution branches are concatenated to obtain a 512×512×64 feature map), which is then input into the BN layer and the ReLU layer in sequence to obtain the transmission line defect feature map F2 with a size of 512×512×64; then Split F2 into two halves along the channel dimension to obtain the power line defect feature maps F3 and F4, both of which are 512×512×32 in size; input F4 into the SGCConv convolution with 2 groupings, 16 group convolution kernels, 32 convolution kernels, 3×3 size, 1 step size and 1 zero padding, and then input it to the BN layer and ReLU layer in turn to obtain the power line defect feature map F5, with a size of 512×512×64; input F5 into the position coordinate attention module CA to improve the extraction of position information features in the feature map, and obtain the power line defect feature map F6 with the same size; input the feature map F6 into the convolution A convolutional layer with 32 kernels, size 3×3, stride 1, and zero padding 1 is input into the BN layer and the ReLU layer in sequence to obtain the transmission line defect feature map F7 with a size of 512×512×32; F2, F3, and F7 are concatenated in the channel dimension to obtain the transmission line defect feature map F8 with a size of 512×512×128; finally, the feature map F8 is input into a convolutional layer with 128 convolution kernels, size 3×3, stride 2, and zero padding 1, and then input into the BN layer and the ReLU layer in sequence to obtain the final output feature map F9 of the LGC module with a size of 256×256×128.

[0084] Step 3.2: Use the enhanced recognition head PBS-Head for small target defects to enhance the recognition effect of small-sized defect targets in the feature image, while reducing computational redundancy and memory access during the reasoning process.

[0085] In this embodiment, the steps of using PBS-Head recognition to output the feature map are as follows:

[0086] Step 3.2.1: Input the transmission line defect feature map X1 into two processing branches for regression and classification of transmission line defects; the structure of the PBS-Head recognition head is as follows: Figure 4 shown.

[0087] Step 3.2.2: Input X1 to the PConv layer with an operation ratio of 1 / 4, a convolution kernel size of 3×3, a step size of 1, and a zero padding of 1, and obtain an output result with the number of output channels equal to the number of input channels. Then input the result to the ReLU layer to obtain the transmission line defect feature map X2, and input X2 to the convolution layer with a convolution kernel size of 3×3, a step size of 1, and a zero padding of 1, and the number of output channels is equal to the number of input channels. Then input another BN layer to obtain the transmission line defect feature map X3. Finally, input the feature map X3 into the convolution for target regression, and obtain the output feature map X4 with an output channel number of 2 (representing the upper left and lower right coordinate positions of the algorithm prediction box);

[0088] Step 3.2.3: Input X1 to the PConv layer with an operation ratio of 1 / 4, a convolution kernel size of 3×3, a step size of 1, and a zero padding of 1, and obtain an output result in which the number of output channels is equal to the number of input channels. Then input the result to the ReLU layer to obtain the transmission line defect feature map X5, and input X5 to the convolution layer with a convolution kernel size of 3×3, a step size of 1, and a zero padding of 1, and then input a BN layer to obtain the transmission line defect feature map X6. Finally, input the feature map X6 into the convolution for target classification, and the number of output channels is X7, the number of transmission line defect classifications;

[0089] Step 3.2.4: Concatenate the output feature map X4 of Step 3.2.2 and the output feature map X7 of Step 3.1.3 along the channel dimension to obtain the output feature map X8. The number of channels of feature map X8 is the sum of the number of channels of X4 and X7.

[0090] Embodiment: Assume that the size of the power transmission line defect feature map input to the PBS-Head recognition head is H×W×C, where H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map.

[0091] For example, the size of the transmission line defect feature map X1 input to the PBS-Head recognition head is 512×512×128. First, the input transmission line defect feature map X1 is respectively input into the upper branch for recognition frame regression and the lower branch for classification to perform regression and classification of transmission line defects, and finally merge and output. The following is a specific description of the embodiment of the final merge output process after the two branches are processed in Step 3.1.2, the upper branch processing process of the recognition frame regression, Step 3.1.3, and Step 3.1.4.

[0092] The first recognition box regression upper branch processing embodiment process in Step 3.2.2 is as follows: X1 is input into the PConv layer with an operation ratio of 1 / 4, a number of convolution kernels of 128, a size of 3×3, a step size of 1, and a zero padding of 1 to obtain an output feature map of size 512×512×128, and then the result is input into the ReLU layer to obtain a transmission line defect feature map X2 of size 512×512×128, and then X2 is input into the PConv layer with a convolution kernel of 1. 28, a convolution layer with a size of 3×3, a stride of 2, and zero padding of 1 is used to obtain an output feature map with a size of 256×256×128. The feature map is then input into a BN layer to obtain a transmission line defect feature map X3 with a size of 256×256×128. Finally, the feature map X3 is input into the convolution for target regression. The number of convolution kernels is 2, the size is 3×3, the stride is 1, and the zero padding is 1. The output feature map X4 has a size of 256×256×2.

[0093] The second branch processing implementation process in Step 3.2.3 is as follows: X1 is input into a PConv layer with an operation ratio of 1 / 4, a number of convolution kernels of 128, a size of 3×3, a step size of 1, and a zero padding of 1 to obtain an output feature map of size 512×512×128, and then the result is input into a ReLU layer to obtain a transmission line defect feature map X5 of size 512×512×128, and then X5 is input into a PConv layer with a number of convolution kernels of 128 and a size of A convolution layer with a size of 3×3, a step size of 2, and zero padding of 1 is used to obtain an output feature map of size 256×256×128. The feature map is then input into a BN layer to obtain a transmission line defect feature map X6 with a size of 256×256×128. Finally, the feature map X6 is input into the convolution for target regression. The number of convolution kernels is the number of transmission line defect classifications n, the size is 3×3, the step size is 1, the zero padding is 1, and the output feature map X7 has a size of 256×256×n.

[0094] The final merged output implementation process after processing the two branches in Step 3.2.4 is as follows: the output feature map X4 of Step 3.1.2 and the output feature map X7 of Step 3.1.3 are spliced ​​along the channel dimension to obtain the output feature map X8 with a size of 256×256×(n+2).

[0095] Step 3.3: When the lightweight defect target recognition head SBF-Head recognizes larger defect targets, it does not need a high-precision recognition head, so the network can be lightweighted by reducing the number of parameters in the recognition head. The SBF-Head designed by the present invention merges the two branches in the PBS-Head and replaces the PConv in the PBS-Head with the LGC module, reducing the parameter redundancy in the recognition head, so that the network can quickly recognize the target in the transmission line feature map, such as Figure 5 shown.

[0096] In this embodiment, the steps of using the lightweight defect target recognition head SBF-Head to recognize the output feature map are as follows:

[0097] Step 3.3.1: Input the transmission line defect feature Y1 into an SGCConv convolution with an input grouping number of 2 to obtain the output result. The number of output channels is half of the number of input channels. Then input the result into the ReLU layer to obtain the transmission line defect feature map Y2;

[0098] Step 3.3.2: Input the power transmission line defect feature map Y2 into a convolution layer with a convolution kernel size of 3×3, a step size of 2, and a zero padding of 1, and then input a BN layer and a ReLU layer to obtain the power transmission line defect feature map Y3. The number of output channels is equal to the number of input channels, and the size becomes 1 / 2 of the original size.

[0099] Step 3.3.3: Input the feature map Y3 into the convolution for target regression and the convolution for target classification respectively, the convolution kernel size is 3×3, the step size is 1, and the zero padding is 1, and the output feature map Y4 with an output channel number of 2 (representing the upper left and lower right coordinate positions of the algorithm prediction box) and the output feature map Y5 with an output channel number equal to the number of transmission line defect classifications are obtained;

[0100] Step3.3.4: Concatenate the output feature maps Y4 and Y5 of Step3.3.3 along the channel dimension to obtain the output feature map Y6.

[0101] Embodiment: Assume that the size of the power transmission line defect feature map input to the SBF-Head recognition head is H×W×C, where H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map.

[0102] Example: Input the SBF-Head recognition head transmission line defect feature map Y1 with a size of 512×512×128, input the transmission line defect feature Y1 into an SGCConv convolution with an input grouping number of 2, a group convolution kernel number of 16, a convolution kernel number of 32, a size of 3×3, a step size of 1, and a zero padding of 1, and get the output result, and then input the result into the ReLU layer to get the transmission line defect feature map Y2 with a size of 512×512×64. Then input the transmission line defect feature map Y2 into a convolution layer with a convolution kernel number of 64, a size of 3×3, a step size of 2, and a zero padding of 1, and then input a BN layer and a ReLU layer to get the transmission line defect feature map Y3 with a size of 256×256×64. Then input a BN layer and a ReLU layer to get the transmission line defect feature map Y3. The transmission line defect feature map Y3 is input into the convolution for target regression and the convolution for target classification respectively. The number of convolution kernels corresponding to the convolution for target regression is 2, and the number of convolution kernels corresponding to the convolution for target regression is the number of transmission line defect categories n. The convolution kernel size is 3×3, the step size is 1, and the zero padding is 1. The transmission line defect feature map Y4 with an output channel number of 2 and the transmission line defect feature map Y5 with an output channel number of n are obtained. Finally, the output feature maps Y4 and Y5 are spliced ​​along the channel dimension to obtain the output feature map Y6 with a size of 256×256×(n+2).

[0103] Step 3.4: Combine the PBS-Head in Step 3.2 and the SBF-Head in Step 3.3 to construct a multi-scale defect recognition head module for the transmission line defect recognition network;

[0104] Step 3.5: Combine the multi-scale transmission line defect recognition module designed in Step 3.4 and the LGC module designed in Step 3.1 to construct a multi-scale defect recognition algorithm network for transmission lines in haze scenarios, such as Figure 2 .

[0105] Step 4: Perform algorithm training and verification based on the transmission line defect recognition algorithm constructed in Step 3;

[0106] Step 4: The steps for algorithm training and verification based on the transmission line defect recognition algorithm constructed in Step 3 are as follows:

[0107] Step 4.1: Initialize the number of training times, momentum factor, batch size, optimizer, learning rate type and initial learning rate of the transmission line defect recognition algorithm;

[0108] Step 4.2: Divide the transmission line defect recognition data set constructed in Step 1 into a training set, a validation set, and a test set for algorithm training. In each round of training, the algorithm grabs the transmission line defect recognition data in the training set according to the batch processing preset value for training. The algorithm optimizes the parameters of each module in the algorithm through classification loss and regression loss back propagation. The regression loss function uses the WIoUv3 loss function, and the classification loss function uses the VFL function;

[0109] Step 4.3: After each round of training is completed, the transmission line defect recognition data in the training set is captured according to the batch processing preset value for verification to check the effectiveness of each round of training. When the algorithm training reaches the convergence of the loss value, the transmission line defect recognition algorithm training is completed.

[0110] Step 5: After training the transmission line defect recognition algorithm, use the haze scene transmission line image data test set preprocessed in Step 2 to conduct a transmission line defect recognition test.

[0111] It should be noted that: in practical applications, the algorithm is deployed to cameras or drones on transmission line towers or substations with haze conditions to monitor the transmission lines with haze in real time, and output the corresponding transmission line defect identification status and defect category to facilitate subsequent further processing.

[0112] The present invention is not limited to the above-mentioned embodiments. Anyone should be aware that any structural changes made under the enlightenment of the present invention, and any technical solutions that are the same or similar to the present invention, fall within the protection scope of the present invention. The technology, shape, and structural parts not described in detail in the present invention are all well-known technologies.

Claims

1. A multi-size defect recognition method for power transmission lines in haze scenes based on deep learning, characterized in that: The following steps are involved: Step 1: Construct a data set of power transmission line defects under haze scenarios; Step 2: Use image atomization algorithm to preprocess the data set of power transmission line defects; Step 3: Construct a transmission line defect recognition algorithm in haze scenes to identify fuzzy targets and small-sized defects of transmission lines in haze scenes. The modules used to identify multi-sized defects of transmission lines in haze scenes include the position information grouping capture convolution module LGC, the enhanced recognition head of small target defects PBS-Head, the lightweight defect target recognition head SBF-Head, and the coordinate attention module CA; The specific steps of constructing a transmission line defect recognition algorithm in a haze scene for identifying fuzzy targets and small-sized defects of the transmission line in the haze scene are as follows: Step3.1: The position information grouping capture convolution module LGC is used. For the input feature map F1, the LGC module performs an SGCConv convolution on it, and then inputs the BN layer and the ReLU layer to obtain the feature map F2. Then F2 is input into a module that splits the feature map along the channel dimension. The channel dimension divides the feature map into two parts, one part is the unprocessed feature map F3, and the other part of the feature map F4 is input into the combination of the SGCConv convolution, the BN layer and the ReLU layer to obtain the feature map F5. The feature map F5 is input into the coordinate attention CA module output and the output feature map F6 is output. Then F6 is input into the convolution layer, the BN layer and the ReLU layer in sequence to obtain the output feature map F7. F7 is spliced ​​with the feature maps F2 and F3 to obtain the feature map F8. Finally, F8 is subjected to a layer of group convolution to obtain the final output feature map F9 of the LGC module. Step 3.2: Use the enhanced recognition head PBS-Head for small target defects to enhance the recognition effect of small-sized defect targets in the feature image, while reducing computational redundancy and memory access during the reasoning process; Step 3.3: When identifying larger defect targets, the lightweight defect target recognition head SBF-Head does not need a high-precision recognition head, so the number of network parameters can be reduced by reducing the number of branches of the recognition head; Step 3.4: Combine the SBF-Head in Step 3.2 and the PBD-Head in Step 3.3 to construct a multi-scale defect recognition head module for the transmission line defect recognition network; Step 3.5: Combine the multi-scale transmission line defect recognition module designed in Step 3.4 and the LGC module designed in Step 3.1 to build a multi-scale transmission line defect recognition algorithm network under haze scenarios; Step 4: Perform algorithm training and verification based on the transmission line defect recognition algorithm constructed in Step 3; Step 5: After training the transmission line defect recognition algorithm, use the haze scene transmission line image data test set preprocessed in Step 2 to conduct a transmission line defect recognition test.

2. According to the method of multi-size defect recognition of power transmission lines in haze scenes based on deep learning in claim 1, it is characterized in that: Step 1: In the process of constructing a transmission line defect dataset under haze scenarios, images of defective transmission lines are collected through drone aerial photography or cameras fixed on transmission towers. After the transmission line defect image collection is completed, all transmission line defect images are labeled one by one according to coordinates and categories for subsequent algorithm training.

3. According to the method of multi-size defect recognition of power transmission lines in haze scenes based on deep learning in claim 1, it is characterized in that: In Step 2, fog is added to the power transmission line defect dataset through the numpy library and opencv library to simulate the power transmission line scene under haze and introduce random concentration changes in different areas of the fog.

4. According to the method of multi-size defect recognition of power transmission lines in haze scenes based on deep learning in claim 1, it is characterized in that: The steps to obtain the final output feature map F9 through the convolution module LGC are: Step 3.1.1: The power transmission line defect feature map F1 is convolved by an SGCConv. SGCConv takes out the feature map of 1 / 2 channels of F1 for group convolution operation, and convolves the feature map of the other half of the channels. Finally, the results processed by SGCConv are spliced ​​in the channel dimension to obtain a feature map with half the number of input channels. Finally, the output feature map is input into a BN layer and a ReLU layer in sequence to obtain the power transmission line defect feature map F2; Step 3.1.2: Divide the transmission line defect feature map F2 into half of the input channel number in the channel dimension, and obtain transmission line defect feature maps F3 and F4 with the number of channels being 1 / 2 of the number of channels of F2; Step 3.1.3: Input the power transmission line defect feature map F4 into a SGCConv convolution, and then input it into the BN layer and the ReLU layer in sequence to obtain the power transmission line defect feature map F5 whose output channel number is twice the number of input channels; Step 3.1.4: Input the transmission line defect feature map F5 into a coordinate attention module CA to obtain a transmission line defect feature map F6 with a constant number of channels; Step 3.1.5: Convolve the power transmission line defect feature map F6 to obtain a power transmission line defect feature map F7 whose output channel number is half of the input channel number; Step 3.1.6: Concatenate the transmission line defect feature maps F2, F3 and F7 in the channel dimension to obtain a transmission line defect feature map F8 whose channel number is the sum of the channel numbers of F2, F3 and F7; Step 3.1.7: Convolve the transmission line defect feature map F8, and then input it into the BN layer and the ReLU layer in sequence to obtain a transmission line defect feature map F9 with an output channel number equal to the input channel number.

5. According to the method of multi-size defect recognition of power transmission lines in haze scenes based on deep learning in claim 1, it is characterized in that: The steps of using PBS-Head recognition head to recognize the output feature map are: Step 3.2.1: Input the transmission line defect feature map X1 into two processing branches for regression and classification of transmission line defects; Step 3.2.2: Input X1 into the PConv layer to obtain an output power transmission line defect feature map with the same number of output channels as the number of input channels; then input the feature map into the ReLU layer to obtain the power transmission line defect feature map X2, input X2 into the convolution layer, and then input it into the BN layer to obtain the power transmission line defect feature map X3, and finally, input the feature map X3 into the convolution for target recognition frame regression, and output the feature map X4 with a channel number of 2, where the channel number 2 represents the upper left and lower right coordinates of the recognition frame; Step 3.2.3: Input X1 to the PConv layer to obtain an output result with the number of output channels equal to the number of input channels, input the result to the ReLU layer to obtain the transmission line defect feature map X5, input X5 to the convolution layer, and then input it to the BN layer to obtain the transmission line defect feature map X6, and finally, input the feature map X6 to the convolution for target classification, and the output channel number is the feature map X7 of the number of transmission line defect classifications; Step3.2.4: Concatenate the output feature map X4 of Step3.1.2 and the output feature map X7 of Step3.1.3 along the channel dimension to obtain the output feature map X8. The number of channels of feature map X8 is the sum of the number of channels of X4 and X7.

6. According to a method for identifying multi-size defects of power transmission lines in haze scenes based on deep learning in claim 1, it is characterized in that: The steps of using the lightweight defect target recognition head SBF-Head to identify the output feature map are as follows: Step 3.3.1: Input the transmission line defect feature Y1 into an SGCConv convolution with an input grouping number of 2 to obtain the output result. The number of output channels is half of the number of input channels. Then input the result into the ReLU layer to obtain the transmission line defect feature map Y2; Step 3.3.2: Input the power transmission line defect feature map Y2 into the convolution layer, and then input a BN layer and a ReLU layer to obtain the power transmission line defect feature map Y3. The number of output channels is equal to the number of input channels, and the size becomes 1 / 2 of the original size. Step 3.3.3: Input the feature map Y3 into the convolution for target regression and the convolution for target classification, respectively, to obtain an output feature map Y4 with an output channel number of 2 and an output feature map Y5 with an output channel number equal to the number of transmission line defect classifications; Step3.3.4: Concatenate the output feature maps Y4 and Y5 of Step3.3.3 along the channel dimension to obtain the output feature map Y6.

7. According to the method of multi-size defect recognition of power transmission lines in haze scenes based on deep learning in claim 1, it is characterized in that: Step 4: The steps for algorithm training and verification based on the transmission line defect recognition algorithm constructed in Step 3 are as follows: Step 4.1: Initialize the number of training times, momentum factor, batch size, optimizer, learning rate type and initial learning rate of the transmission line defect recognition algorithm; Step 4.2: Divide the transmission line defect recognition data set constructed in Step 1 into a training set, a validation set and a test set for algorithm training; in each round of training, the algorithm grabs the transmission line defect recognition data in the training set according to the batch processing preset value for training, and the algorithm optimizes the parameters of each module in the algorithm through classification loss and regression loss back propagation. The regression loss function uses the WIoUv3 loss function, and the classification loss function uses the VFL function; Step 4.3: After each round of training is completed, the transmission line defect recognition data in the training set is captured according to the batch preset value for verification to check the effectiveness of each round of training; when the algorithm is trained until the loss value tends to converge, the transmission line defect recognition algorithm training is completed.

Images