Multi-type insulator defect identification method based on multi-scale dynamic YOLOv8

By combining technologies such as multi-scale dynamic YOLOv8 model and image enhancement module, the problem of inaccurate identification of insulator defects in complex environments is solved, and the accurate identification and positioning of multiple types of insulator defects is achieved, and the recognition accuracy and efficiency are improved.

CN118968248APending Publication Date: 2024-11-15GUANGXI UNIV

Patent Information

Application Number
CN202411203647.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The prior art does not accurately identify multiple types of insulator defects in complex environments, and the deep learning-based object detection method relies on high-quality images and appropriate shooting angles, and has limited adaptability and cannot effectively identify insulator defects in complex backgrounds.

Method used

Using a multi-scale dynamic YOLOv8 method, the YOLOv8 model is combined with the image enhancement module, the receptive field attention convolution feature extraction module, the multi-scale target pyramid feature fusion module and the deformable large-nuclear attention mechanism, and the rotation, scaling, and translation and contrast adjustments to improve offline data enhancement, improve feature extraction and recognition accuracy, and enhance the model's adaptability to complex environments.

Benefits of technology

It improves the robustness and generalization ability of insulator defect recognition, can more accurately identify and locate insulator defects in complex backgrounds, reduce feature interference, enhance the detection ability of small targets, avoid chessboard effects, and improve recognition accuracy and efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968248A_ABST
    Figure CN118968248A_ABST
Patent Text Reader

Abstract

The invention provides a multi-type insulator defect identification method based on multi-scale dynamic YOLOv8. The method comprises the following steps: firstly, performing offline data enhancement on a data set acquired by an unmanned aerial vehicle; thirdly, after data are loaded in a multi-thread mode, the size of an image is adjusted to be 640 * 640, and the image is input into a model for training; then, loss calculation is carried out by using the output target existence, position and classification information; and finally, network parameters are updated and optimized through a weighted attenuation adaptive moment estimation optimizer and a learning rate attenuation strategy. The method provided by the invention can solve the problem of low accuracy of multi-type insulator defect identification in a complex background in the prior art, realizes the function of identifying the defects of different types of insulators, optimizes the insulator defect detection of the power transmission line, improves the accuracy of identifying the defects of the multi-type insulators in the complex background, and reduces the cost. And an important guarantee is provided for safe operation of a power system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of multi-type insulator defect identification and positioning in transmission lines, and relates to a multi-type insulator defect identification method based on multi-scale dynamic YOLOv8, which is suitable for the application of transmission line inspection in power systems. Background Art

[0002] As one of the basic components on the transmission line, insulators play an important role in electrical insulation and mechanical fixation. However, insulators installed in a complex and changeable natural environment are easily affected by voltage, machinery, and environment, causing damage, flashover, self-explosion, and foreign matter. These potential risks can lead to power transmission interruptions and cause large-scale power outages. Therefore, accurately and quickly identifying defective insulators on transmission lines is crucial to the stable operation of the power system.

[0003] In the early days, the identification of insulator defects in transmission lines was carried out through manual inspection methods. The increase in the number of transmission lines has made the inspection scale larger and the inspection environment more complex, magnifying the shortcomings of the manual inspection method, which is high labor cost and low detection efficiency. The new drone inspection method relies on the target detection method to identify insulator defects, which can effectively deal with the shortcomings of manual inspection. The target detection method is divided into a target detection method based on manual features and a target detection method based on deep learning. The target detection method based on manual features relies on manual design to extract target features and then uses a classifier to identify defects. However, this method is heavily dependent on high-quality drone images and appropriate shooting angles. It is only applicable to simple backgrounds or large target images and has limited adaptability. The target detection method based on deep learning first uses an annotation tool to annotate insulator defects, then uses a deep learning model for training, and adjusts the parameters to optimize the model based on the training results. The target detection method based on deep learning effectively solves the problem that the target detection method based on manual features cannot extract deep feature information and the extraction process is prone to information loss. It is the current development trend of insulator defect identification. However, the target detection method based on deep learning is not accurate enough for the identification of multiple types of insulator defects in complex environments.

[0004] Therefore, a multi-type insulator defect recognition method based on multi-scale dynamic YOLOv8 is proposed to accurately identify insulator defects in complex environmental backgrounds. Summary of the invention

[0005] The present invention proposes a multi-type insulator defect recognition method based on multi-scale dynamic YOLOv8, which combines the YOLOv8 model, image enhancement module, receptive field attention convolution feature extraction module, multi-scale target pyramid feature fusion module, deformable large core attention mechanism and dynamic upsampling for multi-type insulator defect detection, and can enhance the extraction and attention of target features in the backbone network, effectively fuse important feature information in the neck network and the detection head, and improve the meticulousness of feature map restoration; the steps of a multi-type insulator defect recognition method based on multi-scale dynamic YOLOv8 during use are:

[0006] Step (i): Screen out defective insulator images from the insulator dataset, and enhance the image samples using offline data enhancement methods including rotation, scaling, translation, flipping, and contrast adjustment to improve the diversity of image samples;

[0007] The new coordinates of the image pixels after rotation are:

[0008] (1)

[0009] In the formula, is the row coordinate of the pixel in the image, is the column coordinate of the pixel in the image, is the rotation angle, is the rotation angle The sine value of is the rotation angle The cosine value of is the row coordinate of the pixel after rotation transformation, is the column coordinate of the pixel after rotation transformation;

[0010] The new row coordinates of the image pixels after scaling are:

[0011] (2)

[0012] The new column coordinates of the image pixels after scaling are:

[0013] (3)

[0014] In the formula, is the row coordinate of the pixel in the image, is the column coordinate of the pixel in the image, is the horizontal scaling factor, is the scaling factor in the vertical direction, is the row coordinate of the pixel after horizontal scaling transformation, The column coordinates of the pixel after vertical scaling transformation;

[0015] The new row coordinates of the image pixel after translation are:

[0016] (4)

[0017] The new column coordinates of the image pixel after translation are:

[0018] (5)

[0019] In the formula, is the row coordinate of the pixel in the image, is the column coordinate of the pixel in the image, is the horizontal translation, is the vertical translation, is the row coordinate of the pixel after horizontal translation, is the column coordinate of the pixel after vertical translation;

[0020] The new row coordinates of the image pixels after horizontal flipping are:

[0021] (6)

[0022] The new column coordinates of the image pixels after horizontal flipping are:

[0023] (7)

[0024] In the formula, is the row coordinate of the pixel in the image, is the column coordinate of the pixel in the image, is the width of the image, is the row coordinate of the pixel after horizontal flipping, is the column coordinate of the pixel after horizontal flipping;

[0025] The new row coordinates of the image pixels after vertical flipping are:

[0026] (8)

[0027] The new column coordinates of the image pixels after vertical flipping are:

[0028] (9)

[0029] In the formula, is the row coordinate of the pixel in the image, is the column coordinate of the pixel in the image, is the height of the image, is the row coordinate of the pixel after vertical flipping, is the column coordinate of the pixel after vertical flip;

[0030] The color value of the image pixel after contrast adjustment is:

[0031] (10)

[0032] In the formula, is the row coordinate of the pixel in the image, is the column coordinate of the pixel in the image, is the original color value of the pixel, is the color value of the pixel after contrast adjustment, is the contrast adjustment factor, is the overall brightness level of the original image;

[0033] Step (ii): normalize and letterbox-size the enhanced images, uniformly scale the images to a size of 640×640×3, and divide the insulator defect images into a training set, a validation set, and a test set in a ratio of 8:1:1;

[0034] Step (iii): Establishing the RMDDYOLOv8 network framework, wherein the network model of the RMDDYOLOv8 includes input, image enhancement module, convolution layer, receptive field attention convolution feature extraction module, residual block, spatial pyramid pooling, deformable large core attention mechanism, dynamic upsampling, multi-scale target pyramid feature fusion module and detection head;

[0035] Among them, the convolution layer includes two-dimensional convolution, batch normalization and activation function; the residual block includes convolution layer, channel splitting, residual bottleneck module and feature fusion; the residual bottleneck module includes 2 convolution layers; spatial pyramid pooling includes convolution layer, maximum pooling and channel splicing; the detection head includes convolution layer, two-dimensional convolution, bounding box loss and classification loss; each module is combined into two links to calculate bounding box loss and classification loss respectively;

[0036] The image enhancement module is composed of Gaussian pyramid decomposition and Laplacian pyramid decomposition, which decomposes the image into multi-frequency, multi-level pyramid sub-images, and then applies fusion rules to superimpose the sub-images layer by layer to enhance the edge and detail information of the image before model training;

[0037] Gaussian pyramid The construction process of the layer image is:

[0038] (11)

[0039] In the formula, is the Gaussian pyramid layer image, is the current image layer number, is the horizontal offset relative to the center pixel in the Gaussian filter window, is the vertical offset relative to the center pixel in the Gaussian filter window, is a Gaussian filter at position The weight of is the row coordinate of the pixel in the image, is the column coordinate of the pixel in the image, For the Gaussian pyramid Layer Position The pixel value of

[0040] The Laplace pyramid The construction process of the layer image is:

[0041] (12)

[0042] In the formula, is the first layer image, is the current image layer number, is the horizontal offset relative to the center pixel in the Gaussian filter window, is the vertical offset relative to the center pixel in the Gaussian filter window, is a Gaussian filter at position The weight of is the row coordinate of the pixel in the image, is the column coordinate of the pixel in the image, For the Laplace pyramid Layer Position The pixel value of For the Layer Position The reconstructed image pixel value of

[0043] The receptive field attention convolution feature extraction module can use grouped convolutions of different sizes to dynamically generate expanded features based on the receptive field size, and sort different features within the receptive field according to importance, thereby enhancing the network's understanding of defect features and improving the effectiveness of feature extraction;

[0044] The feature output of the receptive field attention convolution feature extraction module is:

[0045] (13)

[0046] In the formula, The size is The grouped convolution, The size is The grouped convolution, For normalization, is the input feature map, is the activation function, is the normalization function, is the average pooling function, For the feature output in different sliders, is the attention map, is the receptive field feature after transformation;

[0047] The deformable large kernel attention mechanism can effectively capture a wide range of contextual relationships by combining large convolution kernels and optimized deformable convolutions, flexibly adapt to the transformation of target size and shape, and improve the detection accuracy of the model;

[0048] The attention map generated by the deformable large core attention mechanism is:

[0049] (14)

[0050] The output feature map generated by the deformable large core attention mechanism is:

[0051] (15)

[0052] In the formula, is the original input feature, for After the features processed by convolution operation and GELU activation function, is the attention map, is the output feature map, is the depthwise convolution, is the depth-scalable convolution, is a 1×1 convolution, It is an element-by-element multiplication operation;

[0053] The dynamic upsampling can effectively restore feature detail information based on the generated content-aware sampling point sampling feature map;

[0054] The offset of dynamic upsampling is:

[0055] (16)

[0056] The sampling set for dynamic upsampling is:

[0057] (17)

[0058] The output feature map of dynamic upsampling is:

[0059] (18)

[0060] In the formula, is the input feature map, is the linear transformation layer, is the activation function, is the offset, is the original sampling grid, is the sampling set, is the sampling function, Output feature map for upsampling;

[0061] The multi-scale target pyramid feature fusion module can suppress the consistency of features at different levels by spatially filtering conflicting information, improve the scale invariance of features, and enhance the expression ability of small target features in feature maps by adding a small target detection layer;

[0062] The feature map used for prediction after fusion by the multi-scale target pyramid feature fusion module is:

[0063] (19)

[0064] In the formula, is the number of layers of the current feature map, is the weight parameter of the first layer feature map, is the weight parameter of the second layer feature map, is the weight parameter of the third layer feature map, is the feature map used for prediction after fusion, To adjust the first layer feature map to the The size of the layer feature map, To adjust the second layer feature map to the The size of the layer feature map, To adjust the third layer feature map to the The size of the layer feature map;

[0065] Step (iv): Input the divided training set image samples into the preset RMDDYOLOv8 network model for training, use the output target existence, position and classification information to calculate the loss, and update the network parameters through the weighted attenuated adaptive moment estimation optimizer according to the loss; then use the validation set image samples to evaluate the performance of the model after each iterative training, and adjust the hyperparameters according to the evaluation results; then input the test set image samples into the trained network to obtain the detection results.

[0066] Compared with the prior art, the present invention has the following advantages and effects:

[0067] (1) Existing target detection methods based on deep learning often use a relatively single data set to train models, which is not close enough to the actual detection environment. The present invention enhances image samples through offline data enhancement methods such as rotation, scaling, translation, flipping and contrast adjustment. Rotation corresponds to the change in direction of the defective target in the image, scaling corresponds to the change in size of the defective target in the image, translation corresponds to the change in position of the defective target in the image, flipping corresponds to the change in orientation and perspective of the defective target in the image, and contrast adjustment corresponds to the change in lighting conditions when shooting with a drone. These operations can increase the diversity of image samples and improve the robustness and generalization ability of the model in identifying insulator defects in complex environments.

[0068] (2) The present invention proposes a multi-type insulator defect recognition method based on multi-scale dynamic YOLOv8, and processes the insulator image through the preset RMDDYOLOv8. The YOLOv8 network model is used as the basic model of the preset RMDDYOLOv8, which consists of a convolution layer, a pooling layer, a residual block, an upsampling layer, a fully connected layer and a detection head. Before the YOLOv8 model is trained, an image enhancement module is introduced to decompose the input image into multi-frequency and multi-level tower sub-graphs, and then the fusion rules are applied to superimpose the sub-graphs layer by layer to achieve image deblurring and high-frequency information enhancement.

[0069] (3) The convolutional layers in the backbone network of the original YOLOv8 model cannot effectively understand and process the local area of ​​the insulator defect in the image. The present invention introduces a receptive field attention convolutional feature extraction module to replace the last four convolutional layers in the backbone network, which can significantly reduce feature interference irrelevant to the recognition target and enhance the model's attention to the insulator defect features.

[0070] (4) The residual block of the original YOLOv8 model easily misses key inter-layer information, and the model's feature understanding ability is limited. The present invention embeds a deformable large kernel attention mechanism in the residual block, effectively captures a wide range of contextual relationships through a large convolution kernel, and the optimized deformable convolution can more flexibly adapt to the transformation of the target scale and shape, thereby improving the recognition accuracy of the model.

[0071] (5) The original YOLOv8 model performs upsampling based on deconvolution, which easily produces chessboard effect and spatial discontinuity of feature maps, reducing the quality of restoration of low-resolution feature maps. The present invention introduces dynamic upsampling to avoid the parameter sharing problem in deconvolution, effectively restore feature detail information, and further improve model performance.

[0072] (6) The detection head of the original YOLOv8 model only uses a single-level feature map. The insulator defect targets are of different sizes, and the inconsistency of features between different layers will affect the final detection results. The present invention proposes a multi-scale target pyramid feature fusion module, which suppresses the inconsistency of features at different levels by adaptively fusing feature maps at different levels. At the same time, the multi-scale target pyramid feature fusion module adds a small target detection layer on the basis of adaptive fusion, allowing the network to improve the feature extraction capability of small targets and improve the recognition effect.

[0073] (7) After using the enhanced data to train the preset model, the present invention can more accurately identify and locate insulator defects under various complex backgrounds, which is beneficial for power inspection and operation and maintenance personnel to accurately discover hidden dangers of transmission lines and prevent transmission failures. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 is a flow chart of the method of the present invention.

[0075] Figure 2 It is the overall structure diagram of the RMDDYOLOv8 model preset by the method of the present invention.

[0076] Figure 3 It is a structural diagram of the image enhancement module of the method of the present invention.

[0077] Figure 4 It is a structural diagram of the receptive field attention convolution feature extraction module of the method of the present invention.

[0078] Figure 5 This is a structural diagram of the deformable large-core attention mechanism of the method of the present invention.

[0079] Figure 6 It is a dynamic upsampling structure diagram of the method of the present invention.

[0080] Figure 7 It is a structural diagram of the multi-scale target pyramid feature fusion module of the method of the present invention.

[0081] Figure 8 It is a convolutional layer structure diagram of the method of the present invention.

[0082] Fig. 9 It is a residual block structure diagram of the method of the present invention.

[0083] Fig.10 It is a diagram of the spatial pyramid pooling layer structure of the method of the present invention.

[0084] Fig.11 It is a structural diagram of the detection head of the method of the present invention. DETAILED DESCRIPTION

[0085] The present invention proposes a multi-type insulator defect recognition method based on multi-scale dynamic YOLOv8, which is described in detail in conjunction with the accompanying drawings as follows:

[0086] Figure 1 This is a flow chart of the method of the present invention. First, the original insulator defect data is enhanced by offline data enhancement methods such as rotation, scaling, translation, flipping and contrast adjustment as input; the model is loaded and the parameters are initialized; the input data is preprocessed, namely, the image size is scaled to 640×640×3 and normalized; the model is iteratively trained after multi-threaded data loading; then, the loss is calculated using the output target existence, location and classification information; the network parameters are updated according to the loss through the weighted decay adaptive moment estimation optimizer and the learning rate decay strategy; the performance of the model is verified using the validation set after each iterative training; finally, after the iteration is completed, the optimal parameter model after multiple iterative training is output, and the optimal model is tested using the test set, and the test results are visualized.

[0087] Figure 2 This is the overall structure diagram of the RMDDYOLOv8 model preset by the method of the present invention. The specific contents of the preset network model are:

[0088] Input original image;

[0089] Access the image enhancement module;

[0090] Then connect to 1 convolutional layer;

[0091] Then connect a receptive field attention convolution feature extraction module;

[0092] Then connect 1 residual block;

[0093] It is further divided into 2 branches;

[0094] Branch 1 accesses 1 connection layer

[0095] Branch 2 is connected to a receptive field attention convolution feature extraction module;

[0096] Then, a residual block is connected after the receptive field attention convolution feature extraction module;

[0097] It is further divided into 2 branches;

[0098] Branch 3 accesses 1 connection layer

[0099] Branch 4 is connected to a receptive field attention convolution feature extraction module;

[0100] Then, a residual block is connected after the receptive field attention convolution feature extraction module;

[0101] It is further divided into 2 branches;

[0102] Branch 5 accesses 1 connection layer

[0103] Branch 6 is connected to a receptive field attention convolution feature extraction module;

[0104] Then, a residual block is connected after the receptive field attention convolution feature extraction module;

[0105] Then connect a spatial pyramid pooling module;

[0106] Then connect a dynamic upsampling module;

[0107] Then concatenate branch 5 and the dynamic upsampling module output;

[0108] Then connect 1 residual block;

[0109] It is further divided into 2 branches;

[0110] Branch 7 accesses 1 connection layer

[0111] Branch 8 is connected to a dynamic upsampling module;

[0112] Then concatenate branch 3 and the dynamic upsampling module output;

[0113] Then connect 1 residual block;

[0114] It is further divided into 2 branches;

[0115] Branch 9 accesses 1 connection layer

[0116] Branch 10 is connected to a dynamic upsampling module;

[0117] Then concatenate the output of branch 1 and the dynamic upsampling module;

[0118] Then connect a residual block containing a deformable large core attention mechanism;

[0119] Then connect to 1 convolutional layer;

[0120] Then concatenate branch 9 and the convolutional layer output;

[0121] Then connect a residual block containing a deformable large core attention mechanism;

[0122] Then connect to 1 convolutional layer;

[0123] Then concatenate branch 7 and the convolutional layer output;

[0124] Then connect a residual block containing a deformable large core attention mechanism;

[0125] Then connect to 1 convolutional layer;

[0126] Then concatenate the output of the spatial pyramid pooling module and the output of the convolutional layer;

[0127] Then connect a residual block containing a deformable large core attention mechanism;

[0128] Then, multi-scale target pyramid features are fused after all residual blocks containing deformable large-core attention mechanisms and then connected to the detection head;

[0129] Output the detection results.

[0130] Figure 3 : is a structural diagram of the image enhancement module of the method of the present invention. The specific contents of the image enhancement module structure are:

[0131] After input, it is divided into branches 1 and 2;

[0132] Branch 2 is connected to a Gaussian downsampling layer;

[0133] Then divide the Gaussian downsampling module output into 3 and 4 branches;

[0134] Branch 4 is connected to a Gaussian downsampling layer;

[0135] Then divide the Gaussian downsampling module output into 5 and 6 branches;

[0136] Branch 6 is connected to a Gaussian upsampling layer;

[0137] The output of the Gaussian upsampling module is differentiated from branch 5 to obtain the third layer of the Laplacian pyramid;

[0138] Branch 5 is connected to a Gaussian upsampling layer;

[0139] The output of the Gaussian upsampling module is differentiated from branch 3 to obtain the second layer of the Laplacian pyramid;

[0140] Branch 3 is connected to a Gaussian upsampling layer;

[0141] The output of the Gaussian upsampling module is differentiated from branch 1 to obtain the first layer of the Laplacian pyramid;

[0142] Branch 6 is connected to a Laplace upsampling layer;

[0143] The output of the Laplacian upsampling module is added to the third level of the Laplacian pyramid;

[0144] The added output is connected to a Laplace upsampling layer;

[0145] The output of the Laplacian upsampling module is added to the second level of the Laplacian pyramid;

[0146] The added output is connected to a Laplace upsampling layer;

[0147] The output of the Laplacian upsampling module is added to the first layer of the Laplacian pyramid;

[0148] Final output.

[0149] Figure 4 : is a structural diagram of the receptive field attention convolution feature extraction module of the method of the present invention. The specific contents of the receptive field attention convolution feature extraction module structure are:

[0150] After input, it is divided into branches 1 and 2;

[0151] Branch 1 is connected to a group convolutional layer;

[0152] Branch 2 is connected to an average pooling layer;

[0153] The output of the average pooling layer is further divided into 3, 4 and 5 branches and connected to 1 group convolution layer respectively;

[0154] Then connect to one activation function layer, namely Softmax;

[0155] The outputs of the three activation function layers are multiplied element-by-element with the output of branch 1;

[0156] Then connect another channel splicing layer;

[0157] Then add a layer to adjust and transform the tensor shape;

[0158] Then connect a 3×3 two-dimensional convolutional layer;

[0159] Final output.

[0160] Figure 5 : is a structural diagram of the deformable large core attention mechanism of the method of the present invention. The specific content of the deformable large core attention mechanism structure is:

[0161] After input, it is divided into branches 1 and 2;

[0162] Branch 2 is connected to a 2D convolutional layer;

[0163] Then connect one more activation function layer, namely GELU;

[0164] The activation function layer output is divided into 3 and 4 branches;

[0165] Branch 4 is connected to one deformable depth convolution layer, one deformable depth scalable convolution layer and one 2D convolution layer in sequence;

[0166] The output of branch 4 is multiplied element-wise with the output of branch 3;

[0167] I want to multiply the output and connect it to a 2D convolutional layer;

[0168] The 2D convolutional layer output is added to branch 1;

[0169] Final output.

[0170] Figure 6 : is a dynamic upsampling structure diagram of the method of the present invention. The specific content of the dynamic upsampling structure is:

[0171] The input is connected to a linear layer;

[0172] The linear layer output is divided into branches 1 and 2;

[0173] Branch 1 is connected to an activation function layer, namely sigmoid;

[0174] The activation function layer output is multiplied by 0.5;

[0175] We want to multiply the output and branch 2 element by element;

[0176] I want to multiply the output and feed it into a pixel shuffle layer;

[0177] The output of the pixel shuffling layer is connected to a 2D sampling layer;

[0178] Final output.

[0179] Figure 7 : is a multi-scale target pyramid feature fusion module structure diagram of the method of the present invention. The specific content of the structure of level 1 in the multi-scale target pyramid feature fusion module is:

[0180] Input feature maps of 1, 2, 3, and 4 levels;

[0181] Levels 2 and 3 are connected to one nearest neighbor interpolation layer respectively;

[0182] The outputs of the two nearest neighbor interpolation layers and level 4 are connected to a 1×1 convolutional layer respectively;

[0183] The convolution layer output is channel-joined;

[0184] The concatenated output is then connected to a 1×1 convolutional layer;

[0185] Then connect an activation function layer, namely Softmax;

[0186] Then connect to a weighted fusion layer;

[0187] Final output.

[0188] Figure 8 It is a convolutional layer structure diagram of the method of the present invention. The specific content of the convolutional layer structure is:

[0189] After the input, connect to a convolutional layer;

[0190] Then connect a batch normalization layer;

[0191] Then connect another activation function, namely SiLu;

[0192] Final output.

[0193] Fig. 9 : is a residual block structure diagram of the method of the present invention. The specific content of the residual block structure is:

[0194] After the input, connect to a convolutional layer;

[0195] Split the convolutional layer output into channels;

[0196] Then connect multiple residual bottleneck modules;

[0197] Perform feature fusion on the outputs of multiple residual bottleneck modules;

[0198] Then connect to 1 convolutional layer;

[0199] Final output.

[0200] Fig.10 : is a diagram of the spatial pyramid pooling layer structure of the method of the present invention. The specific content of the spatial pyramid pooling layer structure is:

[0201] After the input, connect to a convolutional layer;

[0202] The output of the convolutional layer is divided into branches 1 and 2;

[0203] Branch 1 is connected to three maximum pooling layers in sequence;

[0204] Concatenate the branch 2 and pooling layer output channels;

[0205] Then connect to 1 convolutional layer;

[0206] Final output.

[0207] Fig.11 : is a structural diagram of the detection head of the method of the present invention. The specific contents of the detection head structure are:

[0208] After input, it is divided into branches 1 and 2;

[0209] Branch 1 and branch 2 are connected to two convolutional layers and one two-dimensional convolution in sequence;

[0210] Branch 1 outputs the bounding box loss;

[0211] Branch 2 outputs the classification loss.

[0212] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A multi-type insulator defect identification method based on multi-scale dynamic YOLOv8, characterized in that: The YOLOv8 model, image enhancement module, receptive field attention convolution feature extraction module, multi-scale target pyramid feature fusion module, deformable large core attention mechanism and dynamic upsampling are combined for multi-type insulator defect detection, which can enhance the extraction and attention of target features in the backbone network, effectively fuse important feature information in the neck network and the detection head, and improve the meticulousness of feature map restoration; a multi-type insulator defect recognition method based on multi-scale dynamic YOLOv8 has the following steps during use: Step (i): Screen out defective insulator images from the insulator dataset, and enhance the image samples using offline data enhancement methods including rotation, scaling, translation, flipping, and contrast adjustment to improve the diversity of image samples; The new coordinates of the image pixels after rotation are: (1) In the formula, is the row coordinate of the pixel in the image, is the column coordinate of the pixel in the image, is the rotation angle, is the rotation angle The sine value of is the rotation angle The cosine value of is the row coordinate of the pixel after rotation transformation, is the column coordinate of the pixel after rotation transformation; The new row coordinates of the image pixels after scaling are: (2) The new column coordinates of the image pixels after scaling are: (3) In the formula, is the row coordinate of the pixel in the image, is the column coordinate of the pixel in the image, is the horizontal scaling factor, is the scaling factor in the vertical direction, is the row coordinate of the pixel after horizontal scaling transformation, The column coordinates of the pixel after vertical scaling transformation; The new row coordinates of the image pixel after translation are: (4) The new column coordinates of the image pixel after translation are: (5) In the formula, is the row coordinate of the pixel in the image, is the column coordinate of the pixel in the image, is the horizontal translation, is the vertical translation, is the row coordinate of the pixel after horizontal translation, is the column coordinate of the pixel after vertical translation; The new row coordinates of the image pixels after horizontal flipping are: (6) The new column coordinates of the image pixels after horizontal flipping are: (7) In the formula, is the row coordinate of the pixel in the image, is the column coordinate of the pixel in the image, is the width of the image, is the row coordinate of the pixel after horizontal flipping, is the column coordinate of the pixel after horizontal flipping; The new row coordinates of the image pixels after vertical flipping are: (8) The new column coordinates of the image pixels after vertical flipping are: (9) In the formula, is the row coordinate of the pixel in the image, is the column coordinate of the pixel in the image, is the height of the image, is the row coordinate of the pixel after vertical flipping, is the column coordinate of the pixel after vertical flip; The color value of the image pixel after contrast adjustment is: (10) In the formula, is the row coordinate of the pixel in the image, is the column coordinate of the pixel in the image, is the original color value of the pixel, is the color value of the pixel after contrast adjustment, is the contrast adjustment factor, is the overall brightness level of the original image; Step (ii): normalize and letterbox-size the enhanced images, uniformly scale the images to a size of 640×640×3, and divide the insulator defect images into a training set, a validation set, and a test set in a ratio of 8:1:1; Step (iii): Establishing the RMDDYOLOv8 network framework, wherein the network model of the RMDDYOLOv8 includes input, image enhancement module, convolution layer, receptive field attention convolution feature extraction module, residual block, spatial pyramid pooling, deformable large core attention mechanism, dynamic upsampling, multi-scale target pyramid feature fusion module and detection head; Among them, the convolution layer includes two-dimensional convolution, batch normalization and activation function; the residual block includes convolution layer, channel splitting, residual bottleneck module and feature fusion; the residual bottleneck module includes 2 convolution layers; spatial pyramid pooling includes convolution layer, maximum pooling and channel splicing; the detection head includes convolution layer, two-dimensional convolution, bounding box loss and classification loss; each module is combined into two links to calculate bounding box loss and classification loss respectively; The image enhancement module is composed of Gaussian pyramid decomposition and Laplacian pyramid decomposition, which decomposes the image into multi-frequency, multi-level pyramid sub-images, and then applies fusion rules to superimpose the sub-images layer by layer to enhance the edge and detail information of the image before model training; Gaussian pyramid The construction process of the layer image is: (11) In the formula, is the Gaussian pyramid layer image, is the current image layer number, is the horizontal offset relative to the center pixel in the Gaussian filter window, is the vertical offset relative to the center pixel in the Gaussian filter window, is a Gaussian filter at position The weight of is the row coordinate of the pixel in the image, is the column coordinate of the pixel in the image, For the Gaussian pyramid Layer Position The pixel value of The Laplace pyramid The construction process of the layer image is: (12) In the formula, is the first layer image, is the current image layer number, is the horizontal offset relative to the center pixel in the Gaussian filter window, is the vertical offset relative to the center pixel in the Gaussian filter window, is a Gaussian filter at position The weight of is the row coordinate of the pixel in the image, is the column coordinate of the pixel in the image, For the Laplace pyramid Layer Position The pixel value of For the Layer Position The reconstructed image pixel value of The receptive field attention convolution feature extraction module can use grouped convolutions of different sizes to dynamically generate expanded features based on the receptive field size, and sort different features within the receptive field according to importance, thereby enhancing the network's understanding of defect features and improving the effectiveness of feature extraction; The feature output of the receptive field attention convolution feature extraction module is: (13) In the formula, The size is The grouped convolution, The size is The grouped convolution, For normalization, is the input feature map, is the activation function, is the normalization function, is the average pooling function, For the feature output in different sliders, is the attention map, is the receptive field feature after transformation; The deformable large kernel attention mechanism can effectively capture a wide range of contextual relationships by combining large convolution kernels and optimized deformable convolutions, flexibly adapt to the transformation of target size and shape, and improve the detection accuracy of the model; The attention map generated by the deformable large core attention mechanism is: (14) The output feature map generated by the deformable large core attention mechanism is: (15) In the formula, is the original input feature, for After the features processed by convolution operation and GELU activation function, is the attention map, is the output feature map, is the depthwise convolution, is the depth-scalable convolution, is a 1×1 convolution, It is an element-by-element multiplication operation; The dynamic upsampling can effectively restore feature detail information based on the generated content-aware sampling point sampling feature map; The offset of dynamic upsampling is: (16) The sampling set for dynamic upsampling is: (17) The output feature map of dynamic upsampling is: (18) In the formula, is the input feature map, is the linear transformation layer, is the activation function, is the offset, is the original sampling grid, is the sampling set, is the sampling function, Output feature map for upsampling; The multi-scale target pyramid feature fusion module can suppress the consistency of features at different levels by spatially filtering conflicting information, improve the scale invariance of features, and enhance the expression ability of small target features in feature maps by adding a small target detection layer; The feature map used for prediction after fusion by the multi-scale target pyramid feature fusion module is: (19) In the formula, is the number of layers of the current feature map, is the weight parameter of the first layer feature map, is the weight parameter of the second layer feature map, is the weight parameter of the third layer feature map, is the feature map used for prediction after fusion, To adjust the first layer feature map to the The size of the layer feature map, To adjust the second layer feature map to the The size of the layer feature map, To adjust the third layer feature map to the The size of the layer feature map; Step (iv): Input the divided training set image samples into the preset RMDDYOLOv8 network model for training, use the output target existence, position and classification information to calculate the loss, and update the network parameters through the weighted attenuated adaptive moment estimation optimizer according to the loss; then use the validation set image samples to evaluate the performance of the model after each iterative training, and adjust the hyperparameters according to the evaluation results; then input the test set image samples into the trained network to obtain the detection results.

Citation Information

Patent Citations

  • Training set acquisition method and electric power facility detection method and device

    CN111428753A

  • Insulator defect detection method based on improved YOLOv3 convolutional neural network

    CN114283117A

  • Steel surface defect detection method based on YOLOv8 multi-scale feature fusion

    CN118229655A

  • Power transmission line insulator equipment based on improved YOLOv8 neural network and defect detection method thereof

    CN118505672A

  • Wire clamp defect detection model and wire clamp detection method based on YOLOv51 frame

    CN118552502A

Cited By

  • Passenger car passenger safety monitoring method and system

    CN119693926A

  • Defect detection method of unmanned aerial vehicle for insulator chain

    CN120807402A

  • Automatic board selection method and system based on magnesium oxide board surface defect recognition

    CN121305193A