Insulator defect detection method based on improved YOLOv7

Abstract

The invention provides an insulator defect detection method GAC-YOLO based on YOLOv7, and relates to the field of identification and detection of insulator targets. Comprising the steps that a GSConv module is optimized, and the calculation amount and the memory requirement are reduced while the precision is guaranteed; an ACmix attention mechanism is introduced, convolution and self-attention are fused, the prediction ability of the model is enhanced, and then the detection effect is improved; the centralized feature pyramid is fused, information exchange and integration can be achieved through feature maps of different scales, more comprehensive insulator features are further obtained, and therefore the insulator detection precision is improved. And performing an ablation experiment on each module of the model, and performing a contrast experiment with a mainstream deep learning model to obtain a final insulator detection model. According to the model realized by the invention, on the basis of ensuring the original detection rate of the YOLOv7, the accuracy of the model in insulator defect detection is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent identification and relates to an insulator detection method based on YOLOv7. Background Art

[0002] With the development of economy, my country's electricity demand continues to grow, and ensuring the stable operation of transmission lines has become a key issue that the power system needs to solve urgently. Insulators in transmission lines play an important role, such as fixing transmission lines and preventing current from flowing back to the earth's crust. However, considering that insulators are exposed to harsh environments for a long time, such as wind erosion and high temperature, insulators may be damaged to varying degrees. Therefore, it is of great significance to regularly inspect and test insulators.

[0003] At present, deep learning algorithms are used in the recognition and detection of several types of targets on transmission lines. The algorithms for transmission line detection are mainly divided into two categories, namely two-stage detection algorithms and single-stage detection algorithms. Single-stage detection algorithms represented by RetinaNet, YOLO, and SSD, and two-stage detection algorithms represented by R-CNN, Fast R-CNN, and Faster R-CNN are common methods in the field of deep learning. Due to the small size of insulator defect targets and the influence of complex environmental background, the accuracy of insulator defect detection is not high. Summary of the invention

[0004] The present invention aims to solve the shortcomings of the existing intelligent recognition technology and proposes an improved YOLOv7 insulator defect detection method to cope with the influence of small insulator defect targets and complex environmental background, thereby improving the accuracy of insulator defect detection.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] An insulator defect detection method based on improved YOLOv7 comprises the following steps:

[0007] Step S1: Based on the “China Transmission Line Insulator String Dataset” as the original data set, the insulator defect images of the data set are provided by the State Grid Corporation of China, and the data set is produced by WANG Zi-Hao. The website is: https: / / github.com / InsulatorData / InsulatorDataSet. The data set is expanded by random color conversion, image inversion and rotation, etc. to create a complex data set;

[0008] Step S2: Perform image enhancement processing on the images of the complex data set, and divide the image-enhanced data set into a training set, a validation set, and a test set in a ratio of 8:1:1;

[0009] Step S3: Apply the feature pyramid module CFP to the neck of the YOLOv7 network model to improve the network's ability to extract target features in the image

[0010] Step S4: Optimize the GSConv module, improve the optimized GSConv module into the neck detection module in the YOLOv7 model, and build a target detection model based on the improved YOLOv7;

[0011] Step S5: Introduce the YOLOv7 model backbone network with ACmix attention mechanism to extract feature information of the dataset image;

[0012] Step S6: Put the divided training set into the improved target detection model for training to obtain the improved model. Based on the model parameter indicators, perform ablation experiments on each module of the model, and conduct comparative experiments with SSD, YOLOv3, YOLOv4, YOLOv5, YOLOx, YOLOv7-tiny, and YOLOv8 models to obtain the final insulator detection model.

[0013] Furthermore, the step S1 comprises the following steps:

[0014] Step S1a: adaptively scale the original dataset to 640×640×3, and expand the dataset by random color conversion, image inversion and rotation;

[0015] Step S1b: Use the LabelImg tool to label the entire insulator dataset and create a VOC format dataset containing the category name and the bounding box position of the target (coordinates of the upper left corner and lower right corner).

[0016] Furthermore, the step S2 comprises the following steps:

[0017] Step S2a: Randomly select 4 different images;

[0018] Step S2b: Generate a large image of 1280×1280, and randomly generate a center point in the large image;

[0019] Step S2c: stitch the four images together based on the center point of the large image;

[0020] Step S2d: performing matrix transformation on the spliced ​​images;

[0021] Step S2e: cropping a 640×640 area based on the upper left corner of the large image;

[0022] Step S2f: Convert the label coordinates of each independent image into label coordinates based on the large image, and then obtain the final coordinates after M matrix transformation;

[0023] Step S2g: After cropping the 640×640 area, the coordinates exceeding the area are restricted and truncated if they cross the boundary.

[0024] Step S2h: The synthesized 640×640 image and the corresponding label box are used for model training.

[0025] Furthermore, in step S3, the applied feature pyramid module CFP includes:

[0026] Input image, backbone network (CNN), explicit visual center (EVC), global centralization regulation (GCR);

[0027] The explicit visual center (EVC) consists of two parallel blocks, in which a lightweight MLP is used to capture the global long-range dependency (i.e., global information) of the top-level feature X4. At the same time, in order to preserve the local corner area (i.e., local information), a learnable visual center mechanism is used to act on the local area features in the aggregation layer. The resulting feature maps of these two blocks are concatenated along the channel dimension and passed to the downstream recognition model as the output of EVC:

[0028] X=cat(MLP(X in );LVC(X in )) (1)

[0029] The input of the MLP module is the output of the depthwise separable convolution module.

[0030] The input of the depthwise separable convolution module is the normalized feature map X in , depthwise separable convolution can improve feature expression capability while reducing computational cost. This process can be described by formula (2):

[0031]

[0032] in,

[0033] The GN module starts with the input feature maps, which enter the module through the output of the convolutional layer. Next, GN divides the number of channels into multiple groups to reduce computational complexity and improve normalization. For each group, the module calculates the mean and variance to ensure that the normalization of the features can be performed effectively. During normalization, the features are subtracted from the mean and divided by the standard deviation to make the distribution more consistent.

[0034] The LVC module structure is constructed by convolutional layers, fully connected layers, and dictionary encoders. The starting point of its operation is that the convolutional layer encodes the input features, and then uses the constructed CBR module (composed of convolution and Rel activation functions with normalization characteristics) to process the obtained code and transmit it to the dictionary encoder. With the help of the encoder, the full range of information of the encoded image can be obtained, and the results of the encoder are passed to the fully connected layer and the convolutional layer to predict the key class features m. The process is shown in formula (3):

[0035] X down =X in +Z (3)

[0036] The calculation formula of Z is shown in formula (4):

[0037]

[0038] Among them, X in represents the original features passed in the LVC module, X down Represents the features processed by the LVC module. Represents an element-by-element multiplication operation, Conv 1×1 represents a 1x1 convolution operation, K represents the group index of the current operation, φ represents the feature generated by the encoder, and for each group K feature response, e k represents the weighted response for the Kth group of features, and its calculation formula is shown in formula (5):

[0039]

[0040] where s k represents the scale parameter used to control the Gaussian distribution, bk represents the center point or benchmark feature of the kth group of features, and N represents the number of samples in the feature map.

[0041] The step S3 comprises the following steps:

[0042] Step S3a: Input the input image into the backbone network to extract a five-level feature pyramid X, where each layer of feature X i The spatial sizes of (i=0,1,2,3,4) are 1 / 2, 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the input image respectively;

[0043] Step S3b: Use a lightweight MLP architecture to capture the global long-range dependencies of X4 and use a learnable visual center mechanism to aggregate local corner regions of the input image;

[0044] Step S3c: Use global centering regulation (GCR) to enable shallow features of the feature pyramid to benefit from the visual centering information of the deepest features at the same time.

[0045] As an improved solution of the present invention, the network used for extracting feature information includes:

[0046] The first layer of CBS module has 32 channels, a convolution kernel size of 3×3, and a step size of 1. It performs convolution, normalization, and activation operations on the input image. The image resolution is 640×640, and the number of channels becomes 32.

[0047] The second layer of CBS module has 64 channels, a convolution kernel size of 3×3, and a step size of 2. It continues to perform convolution, normalization, and activation operations on the feature map. The number of channels becomes 64, and the feature map is reduced to half of the original size.

[0048] The third layer of CBS module has 64 channels, a convolution kernel size of 3×3, and a step size of 1. The feature map is convolved, normalized, and activated again, with the number of channels and feature map size unchanged.

[0049] The fourth layer of CBS module has 128 channels, a convolution kernel size of 3×3, and a step size of 2. It continues to perform convolution, normalization, and activation operations on the feature map. The number of channels becomes 128, and the feature map is reduced to half of the original size.

[0050] The fifth layer ELAN module has 256 channels, which fuses feature maps of different sizes, with a resolution of 160×160 and a channel number of 256.

[0051] The sixth MP module has 256 channels, which reduces the size of the feature map to half of its original size.

[0052] The seventh layer ELAN module has 512 channels and continues to fuse feature maps of different sizes. The resolution is 80×80 and the number of channels is changed to 512.

[0053] The eighth layer MP module has 512 channels and continues to reduce the size of the feature map to half of the original size.

[0054] The ninth layer of ELAN module has 1024 channels. The feature maps of different sizes are fused again. The resolution is 40×40 and the number of channels becomes 1024.

[0055] The tenth MP module has 1024 channels and continues to reduce the size of the feature map to half of the original size.

[0056] The eleventh layer ELAN module has 1024 channels. The feature maps of different sizes are fused again. The resolution is 20×20 and the number of channels becomes 1024.

[0057] The twelfth layer, the CFP feature pyramid module, enables the network to fully extract the specific features of the target. The output feature map has a resolution of 20×20 and the number of channels becomes 512.

[0058] Further, in step S4, optimizing the GSconv module includes:

[0059] Replace the DW conv in the GSConv module with Partial convolution, and its feature data is:

[0060]

[0061] Where ⊙ represents the dot multiplication operation, W represents the convolution kernel weight parameter, b represents the corresponding bias parameter, X represents the pixel feature in the sliding window corresponding to the current convolution kernel; M represents the binary mask corresponding to the pixel, and sum(1) represents the number of pixels corresponding to the sliding window, that is, the convolution kernel size. Represents the scaling factor, and uses appropriate scaling to adjust the change in effective input; the update rule of the mask is:

[0062]

[0063] If the sliding window corresponding to the current pixel contains at least one valid pixel, the position is marked as valid, that is, M at the pixel is set to 1.

[0064] The contents of the improvement module for neck detection in the YOLOv7 model in step S4 are as follows: Conv is improved to the optimized GSConv; based on GSConv, a GS bottleneck module is added, and a nested parallel connection method (one-shot-aggregation, OSA) named VoVNet (Voice-of-Voice Networks) is designed to construct a cross-level partial network (GSCSP) module, and the C3 module in Neck is replaced by the VoV-GSCSP module.

[0065] As an improved solution of the present invention, the network used for extracting feature information includes:

[0066] The thirteenth layer, Upsample, performs upsampling, changing the resolution of the feature map to 40×40 and the number of channels to 512;

[0067] The fourteenth layer, Concat layer, connects the feature map of the backbone network with other feature maps;

[0068] The fifteenth VoV-GSCSP layer has 512 channels and processes the feature map;

[0069] The sixteenth layer, Upsample, performs upsampling, changing the resolution of the feature map to 80×80 and the number of channels to 256;

[0070] The seventeenth layer, Concat, connects the feature map of the backbone network with other feature maps;

[0071] The eighteenth VoV-GSCSP layer has 512 channels and processes the feature map;

[0072] The nineteenth GSConv layer has 256 channels and a 3×3 convolution kernel. It performs GSConv operations and preserves the connections between channels.

[0073] The 20th Concat layer connects the feature map of the head network with other feature maps;

[0074] The 21st VoV-GSCSP layer has 512 channels and processes the feature map.

[0075] The 22nd GSConv layer has 512 channels and a 3×3 convolution kernel. It performs GSConv operations and preserves the connections between channels.

[0076] The twenty-third Concat layer connects the feature map of the head network with other feature maps;

[0077] The 24th VoV-GSCSP layer has 512 channels and processes the feature map.

[0078] The twenty-fifth layer is the Head layer, the feature map enters the Head layer.

[0079] Furthermore, in step S5, the ACmix attention mechanism code is added to the feature extraction network. The input features are projected through three 1×1 convolutions and reshaped into N segments respectively. A set of rich intermediate features containing 3×N feature maps is obtained. For the self-attention path, the intermediate features are grouped into N groups, each containing three features, one for each 1×1 convolution, and the corresponding three feature maps are used as query, key and value. Following the traditional multi-head self-attention module, for the convolution path with kernel size k, a fully connected layer is used and k are generated. 2 Feature mapping, generates the final output by aggregating and moving the generated features.

[0080] The formulas for the aggregation and movement process are as follows:

[0081]

[0082] in, Represents the tensor corresponding to the pixel point (i, j), As a synthesis of feature maps from different kernel positions, the Shift operation is defined as:

[0083]

[0084] Among them, △x and △y correspond to horizontal and vertical displacements respectively.

[0085] F out =αF att +βF conv (11)

[0086] In formula (11), F out Represents the output result of the final feature quantity, F att and F conv It represents the feature results extracted through different paths, and α and β are weight parameters that are both 1.

[0087] Compared with the existing intelligent recognition technology, the insulator defect detection method based on improved YOLOv7 of the present invention has the following beneficial effects:

[0088] 1. Apply the concentrated feature pyramid module CFP to the neck of the YOLOv7 network model so that the network model can receive more complete insulator features, thereby making target detection more accurate and enabling the network to fully extract the specific features of the target.

[0089] 2. The present invention optimizes the GSConv module, integrates the Partial convolution and GS bottleneck modules, and reduces the amount of computation and memory requirements. The Partial convolution only acts on part of the input channels, and the GS bottleneck module and the nested parallel connection (VoVNet) construct the GSCSP module, replace the C3 module in Neck with VoV-GSCSP, and replace the DW convolution with the Partial convolution, which significantly reduces the amount of parameter calculation and structural complexity.

[0090] 3. The present invention introduces the ACmix attention module, which enables the network to have the dual advantages of convolution and self-attention. It can fully utilize the global features and pay attention to local details. This improves the effectiveness and efficiency of the operator while avoiding the possible feature loss in the process of network feature processing and reduces the occurrence of missed detection events. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 It is a flow chart for realizing the present invention;

[0092] Figure 2 It is the overall framework diagram of the present invention;

[0093] Figure 3It is a structural diagram of the centralized feature pyramid module CFP of the present invention;

[0094] Figure 4 It is a schematic diagram of the Partial convolution process of the present invention;

[0095] Figure 5 The VoV-GSCSP module structure diagram of the present invention;

[0096] Figure 6 This is the optimized GSConv module structure diagram of the present invention;

[0097] Figure 7 This is a structural diagram of the ACmix module of the present invention. DETAILED DESCRIPTION

[0098] The specific implementation of the present invention is described below in conjunction with the accompanying drawings and embodiments:

[0099] Embodiment 1:

[0100] In order to solve the shortcomings of the existing intelligent recognition technology, the present invention proposes an improved YOLOv7 insulator defect detection method. Firstly, based on the "China Transmission Line Insulator String Dataset" as the original data set, the data set is expanded by random color conversion, image inversion and rotation, etc. to create a complex data set. After that, the images of the complex data set are enhanced to increase the data diversity, and the image-enhanced data set is divided into training set, validation set and test set in a ratio of 8:1:1. Then, the feature pyramid module CFP is applied to the neck of the YOLOv7 network model to improve the network's ability to extract target features in the image; the GSConv module is optimized, and the optimized GSConv module is improved into the neck detection module in the YOLOv7 model to build a target detection model based on the improved YOLOv7; the YOLOv7 model backbone network with ACmix attention mechanism is introduced to extract the feature information of the dataset image; finally, the divided training set is put into the improved target detection model for training to obtain the improved model, and the ablation experiment is carried out on each module of the model based on the model parameter indicators, and the comparison experiment is carried out with SSD, YOLOv3, YOLOv4, YOLOv5, YOLOx, YOLOv7-tiny, and YOLOv8 models to obtain the final insulator detection model.

[0101] As a specific embodiment of the present invention, the present invention discloses an insulator defect detection method based on improved YOLOv7, comprising the following steps:

[0102] Step S1: Based on the “China Transmission Line Insulator String Dataset” as the original dataset, the insulator defect images of this dataset are provided by the State Grid Corporation of China, and the dataset is produced by WANG Zi-Hao. The URL is: https: / / github.com / InsulatorData / InsulatorDataSet , augmenting the dataset by random color conversion, image inversion and rotation to create complex datasets;

[0103] Furthermore, the step S1 comprises the following steps:

[0104] Step S1a: adaptively scale the original dataset to 640×640×3, and expand the dataset by random color conversion, image inversion and rotation;

[0105] Step S1b: Use the LabelImg tool to label the entire insulator dataset and create a VOC format dataset containing the category name and the bounding box position of the target (coordinates of the upper left corner and lower right corner).

[0106] Step S2: Perform image enhancement processing on the images of the complex data set, and divide the image-enhanced data set into a training set, a validation set, and a test set in a ratio of 8:1:1;

[0107] Furthermore, the step S2 comprises the following steps:

[0108] Step S2a: Randomly select 4 different images;

[0109] Step S2b: Generate a large image of 1280×1280, and randomly generate a center point in the large image;

[0110] Step S2c: stitch the four images together based on the center point of the large image;

[0111] Step S2d: performing matrix transformation on the spliced ​​images;

[0112] Step S2e: cropping a 640×640 area based on the upper left corner of the large image;

[0113] Step S2f: Convert the label coordinates of each independent image into label coordinates based on the large image, and then obtain the final coordinates after M matrix transformation;

[0114] Step S2g: After cropping the 640×640 area, the coordinates exceeding the area are restricted, and if they cross the boundary, they are truncated;

[0115] Step S2h: The synthesized 640×640 image and the corresponding label box are used for model training.

[0116] Specifically, Figure 2 and Figure 3 As shown, the feature pyramid module CFP is applied to the neck of the YOLOv7 network model:

[0117] Step S3: Apply the feature pyramid module CFP to the neck of the YOLOv7 network model to enhance the network's ability to extract target features in the image.

[0118] Furthermore, in step S3, the applied feature pyramid module CFP includes:

[0119] Input image, backbone network (CNN), explicit visual center (EVC), global centralization regulation (GCR);

[0120] The explicit visual center (EVC) consists of two parallel blocks, in which a lightweight MLP is used to capture the global long-range dependency (i.e., global information) of the top-level feature X4. At the same time, in order to preserve the local corner area (i.e., local information), a learnable visual center mechanism is used to act on the local area features in the aggregation layer. The resulting feature maps of these two blocks are concatenated along the channel dimension and passed to the downstream recognition model as the output of EVC:

[0121] X=cat(MLP(X in );LVC(X in )) (1)

[0122] The input of the MLP module is the output of the depthwise separable convolution module.

[0123] The input of the depthwise separable convolution module is the normalized feature map X in , depthwise separable convolution can improve feature expression capability while reducing computational cost. This process can be described by formula (2):

[0124]

[0125] in, The GN module starts with the input feature maps, which enter the module through the output of the convolutional layer. Next, GN divides the number of channels into multiple groups to reduce computational complexity and improve normalization. For each group, the module calculates the mean and variance to ensure that the normalization of the features can be performed effectively. During normalization, the features are subtracted from the mean and divided by the standard deviation to make the distribution more consistent.

[0126] The LVC module structure is constructed by convolutional layers, fully connected layers, and dictionary encoders. The starting point of its operation is that the convolutional layer encodes the input features, and then uses the constructed CBR module (composed of convolution and Rel activation functions with normalization characteristics) to process the obtained code and transmit it to the dictionary encoder. With the help of the encoder, the full range of information of the encoded image can be obtained, and the results of the encoder are passed to the fully connected layer and the convolutional layer to predict the key class features m. The process is shown in formula (3):

[0127] X down =X in +Z (3)

[0128] The calculation formula of Z is shown in formula (4):

[0129]

[0130] Among them, X in represents the original features passed in the LVC module, X down Represents the features processed by the LVC module. Represents an element-by-element multiplication operation, Conv 1×1 represents a 1x1 convolution operation, K represents the group index of the current operation, φ represents the feature generated by the encoder, and for each group K feature response, e k represents the weighted response for the Kth group of features, and its calculation formula is shown in formula (5):

[0131]

[0132] where s k represents the scale parameter used to control the Gaussian distribution, bk represents the center point or benchmark feature of the kth group of features, and N represents the number of samples in the feature map.

[0133] The step S3 comprises the following steps:

[0134] Step S3a: Input the input image into the backbone network to extract a five-level feature pyramid X, where each layer of feature X i The spatial sizes of (i=0,1,2,3,4) are 1 / 2, 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the input image respectively;

[0135] Step S3b: Use a lightweight MLP architecture to capture the global long-range dependencies of X4 and use a learnable visual center mechanism to aggregate local corner regions of the input image;

[0136] Step S3c: Use global centering regulation (GCR) to enable shallow features of the feature pyramid to benefit from the visual centering information of the deepest features at the same time.

[0137] As an improved solution of the present invention, the network used for extracting feature information includes:

[0138] The first layer of CBS module has 32 channels, a convolution kernel size of 3×3, and a step size of 1. It performs convolution, normalization, and activation operations on the input image. The image resolution is 640×640, and the number of channels becomes 32.

[0139] The second layer of CBS module has 64 channels, a convolution kernel size of 3×3, and a step size of 2. It continues to perform convolution, normalization, and activation operations on the feature map. The number of channels becomes 64, and the feature map is reduced to half of the original size.

[0140] The third layer of CBS module has 64 channels, a convolution kernel size of 3×3, and a step size of 1. The feature map is convolved, normalized, and activated again, with the number of channels and feature map size unchanged.

[0141] The fourth layer of CBS module has 128 channels, a convolution kernel size of 3×3, and a step size of 2. It continues to perform convolution, normalization, and activation operations on the feature map. The number of channels becomes 128, and the feature map is reduced to half of the original size.

[0142] The fifth layer ELAN module has 256 channels, which fuses feature maps of different sizes, with a resolution of 160×160 and a channel number of 256.

[0143] The sixth MP module has 256 channels, which reduces the size of the feature map to half of its original size.

[0144] The seventh layer ELAN module has 512 channels and continues to fuse feature maps of different sizes. The resolution is 80×80 and the number of channels is changed to 512.

[0145] The eighth layer MP module has 512 channels and continues to reduce the size of the feature map to half of the original size.

[0146] The ninth layer of ELAN module has 1024 channels. The feature maps of different sizes are fused again. The resolution is 40×40 and the number of channels becomes 1024.

[0147] The tenth MP module has 1024 channels and continues to reduce the size of the feature map to half of the original size.

[0148] The eleventh layer ELAN module has 1024 channels. The feature maps of different sizes are fused again. The resolution is 20×20 and the number of channels becomes 1024.

[0149] The twelfth layer, the CFP feature pyramid module, enables the network to fully extract the specific features of the target. The output feature map has a resolution of 20×20 and the number of channels becomes 512.

[0150] Specifically, Figure 4 , Figure 5 and Figure 6 As shown in the figure, the GSConv module is optimized, and the optimized GSConv module is improved into the neck detection module in the YOLOv7 model:

[0151] Step S4: Optimize the GSConv module, improve the optimized GSConv module into the neck detection module in the YOLOv7 model, and build a target detection model based on the improved YOLOv7;

[0152] Further, in step S4, optimizing the GSconv module includes:

[0153] Replace the DW conv in the GSConv module with Partial convolution, and its feature data is:

[0154]

[0155] Where ⊙ represents the dot multiplication operation, W represents the convolution kernel weight parameter, b represents the corresponding bias parameter, X represents the pixel feature in the sliding window corresponding to the current convolution kernel; M represents the binary mask corresponding to the pixel, and sum(1) represents the number of pixels corresponding to the sliding window, that is, the convolution kernel size. Represents the scaling factor, and uses appropriate scaling to adjust the change in effective input; the update rule of the mask is:

[0156]

[0157] If the sliding window corresponding to the current pixel contains at least one valid pixel, the position is marked as valid, that is, M at the pixel is set to 1.

[0158] The contents of the improvement module for neck detection in the YOLOv7 model in step S4 are as follows: Conv is improved to the optimized GSConv; based on GSConv, a GS bottleneck module is added, and a nested parallel connection method (one-shot-aggregation, OSA) named VoVNet (Voice-of-Voice Networks) is designed to construct a cross-level partial network (GSCSP) module, and the C3 module in Neck is replaced by the VoV-GSCSP module.

[0159] As an improved solution of the present invention, the network used for extracting feature information includes:

[0160] The thirteenth layer, Upsample, performs upsampling, changing the resolution of the feature map to 40×40 and the number of channels to 512;

[0161] The fourteenth layer, Concat layer, connects the feature map of the backbone network with other feature maps;

[0162] The fifteenth VoV-GSCSP layer has 512 channels and processes the feature map;

[0163] The sixteenth layer, Upsample, performs upsampling, changing the resolution of the feature map to 80×80 and the number of channels to 256;

[0164] The seventeenth layer, Concat, connects the feature map of the backbone network with other feature maps;

[0165] The eighteenth VoV-GSCSP layer has 512 channels and processes the feature map;

[0166] The nineteenth GSConv layer has 256 channels and a 3×3 convolution kernel. It performs GSConv operations and preserves the connections between channels.

[0167] The 20th Concat layer connects the feature map of the head network with other feature maps;

[0168] The 21st VoV-GSCSP layer has 512 channels and processes the feature map.

[0169] The 22nd GSConv layer has 512 channels and a 3×3 convolution kernel. It performs GSConv operations and preserves the connections between channels.

[0170] The twenty-third Concat layer connects the feature map of the head network with other feature maps;

[0171] The 24th VoV-GSCSP layer has 512 channels and processes the feature map.

[0172] The twenty-fifth layer is the Head layer, the feature map enters the Head layer.

[0173] Specifically, Figure 7 As shown, the ACmix attention mechanism is introduced:

[0174] Step S5: Introduce the YOLOv7 model backbone network with ACmix attention mechanism to extract feature information of the dataset image;

[0175] Furthermore, in step S5, the ACmix attention mechanism code is added to the feature extraction network. The input features are projected through three 1×1 convolutions and reshaped into N segments respectively. A set of rich intermediate features containing 3×N feature maps is obtained. For the self-attention path, the intermediate features are grouped into N groups, each containing three features, one for each 1×1 convolution, and the corresponding three feature maps are used as query, key and value. Following the traditional multi-head self-attention module, for the convolution path with kernel size k, a fully connected layer is used and k are generated. 2 Feature mapping, generates the final output by aggregating and moving the generated features.

[0176] The formulas for the aggregation and movement process are as follows:

[0177]

[0178] in, Represents the tensor corresponding to the pixel point (i, j), As a synthesis of feature maps from different kernel positions, the Shift operation is defined as:

[0179]

[0180] Among them, △x and △y correspond to horizontal and vertical displacements respectively.

[0181] F out =αF att +βF conv (11)

[0182] In formula (11), F out Represents the output result of the final feature quantity, F att and F conv It represents the feature results extracted through different paths, and α and β are weight parameters that are both 1.

[0183] Step S6: Put the divided training set into the improved target detection model for training to obtain the improved model. Based on the model parameter indicators, perform ablation experiments on each module of the model, and conduct comparative experiments with SSD, YOLOv3, YOLOv4, YOLOv5, YOLOx, YOLOv7-tiny, and YOLOv8 models to obtain the final insulator detection model.

[0184] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge scope of ordinary technicians in the field without departing from the purpose of the present invention. These changes involve related technologies well known to those skilled in the art, which all fall within the scope of protection of the patent of the present invention.

[0185] Many other changes and modifications may be made without departing from the concept and scope of the present invention.It should be understood that the present invention is not limited to the specific embodiments, and the scope of the present invention is defined by the appended claims.

Claims

1. Insulator defect detection method based on improved YOLOv7, It is characterized in that The method comprises: Step S1: Based on the original data set, expand the data set to create a complex data set; Step S2: Perform image enhancement processing on the images of the complex data set, and divide the image-enhanced data set into a training set, a validation set, and a test set in a ratio of 8:1:1; Step S3: applying a centralized feature pyramid module CFP to the neck of the YOLOv7 network model to enhance the network's ability to extract target features in the image; Step S4: Optimize the GSConv module, improve the optimized GSConv module into the neck detection module in the YOLOv7 model, and build a target detection model based on the improved YOLOv7; Step S5: Introduce the YOLOv7 model backbone network with ACmix attention mechanism to extract feature information of the dataset image; Step S6: Put the divided training set into the improved target detection model for training to obtain the improved model. Based on the model parameter indicators, perform ablation experiments on each module of the model, and conduct comparative experiments with SSD, YOLOv3, YOLOv4, YOLOv5, YOLOx, YOLOv7-tiny, and YOLOv8 models to obtain the final insulator detection model.

2. The insulator defect detection method based on improved YOLOv7 according to claim 1, It is characterized in that The original dataset is "China Transmission Line Insulator String Dataset"; The step S1 of expanding the data set is to expand the original data set by random color conversion, image inversion and rotation. The step S1 comprises the following steps: Step S1a: The original dataset is adaptively scaled to 640×640×3, and the dataset is expanded by random color conversion, image inversion and rotation. Step S1b: Use the LabelImg tool to label the entire insulator dataset and create a VOC format dataset containing the category name and the bounding box location of the target.

3. The insulator defect detection method based on improved YOLOv7 according to claim 1, It is characterized in that The data enhancement method in step S2 is a Mosaic image enhancement method, which specifically includes the following steps: Step S2a: Randomly select 4 different images; Step S2b: Generate a large image of 1280×1280, and randomly generate a center point in the large image; Step S2c: stitch the four images together based on the center point of the large image; Step S2d: performing matrix transformation on the spliced ​​images; Step S2e: cropping a 640×640 area based on the upper left corner of the large image; Step S2f: Convert the label coordinates of each independent image into label coordinates based on the large image, and then obtain the final coordinates after M matrix transformation; Step S2g: After cropping the 640×640 area, the coordinates exceeding the area are restricted and truncated if they cross the boundary. Step S2h: The synthesized 640×640 image and the corresponding label box are used for model training.

4. According to the insulator defect detection method based on improved YOLOv7 according to claim 1, It is characterized in that In step S3, the applied concentrated feature pyramid module CFP includes: Input image, backbone network (CNN), explicit visual center (EVC), global centralization regulation (GCR); Step S3a: Input the input image into the backbone network to extract a five-level feature pyramid X, where each layer of feature X i The spatial sizes of (i=0,1,2,3,4) are 1 / 2, 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the input image respectively; Step S3b: Use a lightweight MLP architecture to capture the global long-range dependencies of X4 and use a learnable visual center mechanism to aggregate local corner regions of the input image; Step S3c: Use global centering adjustment to enable shallow features of the feature pyramid to benefit from the visual centering information of the deepest features at the same time.

5. According to the insulator defect detection method based on improved YOLOv7 according to claim 4, It is characterized in that The explicit visual center consists of two parallel blocks, in which a lightweight MLP is used to capture the global long-range dependencies of the top-level features X4. At the same time, in order to preserve the local corner areas, a learnable visual center mechanism is used to act on the local area features in the aggregation layer; the resulting feature maps of these two blocks are concatenated along the channel dimension and passed to the downstream recognition model as the output of EVC: X=cat(MLP(X in );LVC(X in )) (1) In formula (1), the input of the MLP module is the output of the depthwise separable convolution module; The input of the depthwise separable convolution module is the normalized feature map X in , depthwise separable convolution can improve feature expression capability while reducing computational cost. This process is described by formula (2): In formula (2), The LVC module structure is constructed by convolutional layer, fully connected layer and dictionary encoder. The starting point of its operation is that the convolutional layer encodes the input features, and then uses the constructed CBR module to process the obtained code and transmit it to the dictionary encoder. With the help of the encoder, the full range of information of the encoded image can be obtained. The result of the encoder is passed to the fully connected layer and convolutional layer to predict the key class feature m. The process is shown in formula (3): X down =X in +Z (3) In formula (3), the calculation formula of Z is shown in formula (4): In formula (4), X in represents the original features passed in the LVC module, X down Represents the features processed by the LVC module. Represents an element-by-element multiplication operation, Conv 1×1 represents a 1x1 convolution operation, K represents the group index of the current operation, φ represents the feature generated by the encoder, and for each group K feature response, e k represents the weighted response for the Kth group of features, and its calculation formula is shown in formula (5): In formula (5), s k represents the scale parameter used to control the Gaussian distribution, bk represents the center point or benchmark feature of the kth group of features, and N represents the number of samples in the feature map.

6. According to the insulator defect detection method based on improved YOLOv7 according to claim 5, It is characterized in that The network for extracting target features from the image in step S3 includes: The first layer of CBS module has 32 channels, a convolution kernel size of 3×3, and a step size of 1. It performs convolution, normalization, and activation operations on the input image. The image resolution is 640×640, and the number of channels becomes 32. The second layer of CBS module has 64 channels, a convolution kernel size of 3×3, and a step size of 2. It continues to perform convolution, normalization, and activation operations on the feature map. The number of channels becomes 64, and the feature map is reduced to half of the original size. The third layer of CBS module has 64 channels, a convolution kernel size of 3×3, and a step size of 1. The feature map is convolved, normalized, and activated again, with the number of channels and feature map size unchanged. The fourth layer of CBS module has 128 channels, a convolution kernel size of 3×3, and a step size of 2. It continues to perform convolution, normalization, and activation operations on the feature map. The number of channels becomes 128, and the feature map is reduced to half of the original size. The fifth layer ELAN module has 256 channels, which fuses feature maps of different sizes, with a resolution of 160×160 and a channel number of 256. The sixth MP module has 256 channels, which reduces the size of the feature map to half of its original size. The seventh layer ELAN module has 512 channels and continues to fuse feature maps of different sizes. The resolution is 80×80 and the number of channels is changed to 512. The eighth layer MP module has 512 channels and continues to reduce the size of the feature map to half of the original size. The ninth layer of ELAN module has 1024 channels. The feature maps of different sizes are fused again. The resolution is 40×40 and the number of channels becomes 1024. The tenth MP module has 1024 channels and continues to reduce the size of the feature map to half of the original size. The eleventh layer ELAN module has 1024 channels. The feature maps of different sizes are fused again. The resolution is 20×20 and the number of channels becomes 1024. The twelfth layer is the centralized feature pyramid CFP module. Through this layer, the network can fully extract the specific features of the target. The output feature map has a resolution of 20×20 and the number of channels becomes 512.

7. According to the insulator defect detection method based on improved YOLOv7 according to claim 1, It is characterized in that The optimization of the GSconv module in step S4 includes: Replace the DW conv in the GSConv module with Partial convolution, and its feature data is: In formula (6), ⊙ represents the dot multiplication operation, W represents the convolution kernel weight parameter, b represents the corresponding bias parameter, X represents the pixel feature in the sliding window corresponding to the current convolution kernel; M represents the binary mask corresponding to the pixel, sym(1) represents the number of pixels corresponding to the sliding window, that is, the convolution kernel size, Represents the scaling factor, and uses appropriate scaling to adjust the change in effective input; the update rule of the mask is: If the sliding window corresponding to the current pixel contains at least one valid pixel, the position is marked as valid, that is, M at the pixel is set to 1.

8. According to the insulator defect detection method based on improved YOLOv7 according to claim 7, It is characterized in that In step S4, the content of the neck detection improvement module in the YOLOv7 model is as follows: Improve Conv to optimized GSConv; Based on GSConv, add GS bottleneck module, design a nested parallel connection method (one-shot-aggregation, OSA) named VoVNet (Voice-of-Voice Networks) to build a cross-level partial network (GSCSP) module, and replace the C3 module in Neck with the VoV-GSCSP module; The network is as follows: The thirteenth layer, Upsample, performs upsampling, changing the resolution of the feature map to 40×40 and the number of channels to 512; The fourteenth layer, Concat layer, connects the feature map of the backbone network with other feature maps; The fifteenth VoV-GSCSP layer has 512 channels and processes the feature map. The sixteenth layer, Upsample, performs upsampling, changing the resolution of the feature map to 80×80 and the number of channels to 256; The seventeenth layer, Concat, connects the feature map of the backbone network with other feature maps; The eighteenth VoV-GSCSP layer has 512 channels and processes the feature map; The nineteenth GSConv layer has 256 channels and a 3×3 convolution kernel. It performs GSConv operations and preserves the connections between channels. The 20th Concat layer connects the feature map of the head network with other feature maps; The 21st VoV-GSCSP layer has 512 channels and processes the feature map. The 22nd GSConv layer has 512 channels and a 3×3 convolution kernel. It performs GSConv operations and preserves the connections between channels. The twenty-third Concat layer connects the feature map of the head network with other feature maps; The 24th VoV-GSCSP layer has 512 channels and processes the feature map. The twenty-fifth layer is the Head layer, the feature map enters the Head layer.

9. According to the insulator defect detection method based on improved YOLOv7 according to claim 1, It is characterized in that In step S5, the ACmix attention mechanism code is added to the feature extraction network. The input features are projected through three 1×1 convolutions and reshaped into N segments respectively. A set of rich intermediate features containing 3×N feature maps is obtained. For the self-attention path, the intermediate features are grouped into N groups, each containing three features, one for each 1×1 convolution, and the corresponding three feature maps are used as query, key and value. Following the traditional multi-head self-attention module, for the convolution path with kernel size k, a fully connected layer is used and k are generated. 2 Feature mapping, generates the final output by aggregating and moving the generated features.

10. According to the insulator defect detection method based on improved YOLOv7 according to claim 9, It is characterized in that The formulas for the aggregation and movement process are as follows: In formula (8), Represents the tensor corresponding to the pixel point (i, j), As a synthesis of feature maps from different kernel positions, the Shift operation is defined as: In formula (10), △x and △y correspond to horizontal and vertical displacements respectively; F out =αF att +βF conv (11) In formula (11), F out Represents the output result of the final feature quantity, F att and F conv It represents the feature results extracted through different paths, and α and β are weight parameters that are both 1.

Images