Typical distribution network material image recognition method based on MSA-YOLOv5x

Abstract

The invention relates to an MSA-YOLOv5x-based image recognition method for typical distribution network materials, and solves the problem of low precision of distribution network material image recognition based on YOLOv5 at present. Comprising the following steps: acquiring distribution network material image data, and labeling and preprocessing an image; expanding the processed image data set; constructing an MSA-YOLOv5x model, including introducing an ACON-C activation function and a D-SPP module, embedding a mask self-attention mechanism module, and improving a loss function; the MSA-YOLOv5x model is trained, and an optimal model is obtained; and predicting the distribution network material image by using the optimal model to obtain an identification result. By improving the YOLOv5x model, the improved model is used for training, verifying and testing a data set, a high-precision and high-speed detection model is obtained, and a good effect is achieved on distribution network material image recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to a typical distribution network material image recognition method based on MSA-YOLOv5. BACKGROUND

[0002] The current common distribution network material image recognition method mainly has a recognition method based on machine learning and a recognition method based on deep learning. Compared with the traditional machine learning method, the deep learning technology has two advantages: first, the deep learning technology can continuously improve its performance with the increase of data size, while the traditional machine learning algorithm is difficult to continuously improve its performance by using massive data. Second, the deep learning technology can directly extract features from data, reducing the work of designing a feature extractor for each problem, while the traditional machine learning algorithm needs to extract features manually.

[0003] Based on the deep learning technology, the distribution network material image recognition actually belongs to the category of target detection. The target detection algorithm based on deep learning mainly includes two categories: one is a two-stage algorithm based on candidate regions, and the other is a single-stage algorithm based on bounding box regression. The detection process of the two-stage algorithm includes two steps: the first step is to find the candidate region from the input image, i.e. the region where the target may exist, and the second step is to classify and regress the candidate region, and to correct the position of the candidate region. Finally, the class of the target is output. Typical algorithms mainly include R-CNN, Fast R-CNN, Faster R-CNN, etc. The single-stage target detection algorithm directly outputs the class probability and position coordinate information of the object by using the regression method, and the representative algorithms include YOLOv1-YOLOv5, SSD, etc. The two-stage algorithm has slow detection speed, while the single-stage algorithm belongs to end-to-end detection, which improves the speed of target detection and meets the requirement of real-time detection. The YOLOv5 network structure is multi-layered, which is convenient for embedding mobile devices, and has fast recognition speed and high accuracy for distribution network material image recognition.

[0004] At present, the distribution network material image recognition is easily affected by factors such as light, environment and different target scales, resulting in low recognition accuracy. With the continuous iteration of the YOLO series algorithm, the detection accuracy and speed are improved, and YOLOv5 is widely used in image recognition. However, the distribution network material image recognition based on YOLOv5 still has the problem of insufficient accuracy, which will affect the subsequent picking work. SUMMARY

[0005] The present application mainly solves the problem of insufficient accuracy of the current distribution network material image recognition based on YOLOv5, and provides a typical distribution network material image recognition method based on MSA-YOLOv5x.

[0006] The above technical problems of the present invention are mainly solved by the following technical solutions: A typical distribution network material image recognition method based on MSA-YOLOv5x, comprising the following steps:

[0007] S1. Collect distribution network material image data, label and preprocess the image data;

[0008] S2. Expand the processed image dataset;

[0009] S3. Build the MSA-YOLOv5x model based on the YOLOv5x model, including:

[0010] Introduce the ACON-C activation function in the CSP module,

[0011] Replace the SPP module with the D-SPP module,

[0012] A masked self-attention mechanism module is embedded after the output of the effective feature layer of the neck network and before the input of the prediction layer.

[0013] Improve the loss function;

[0014] S4. Use the image dataset to train the MSA-YOLOv5x model and obtain the optimal model;

[0015] S5. Use the optimal model to predict the distribution network material image and obtain the recognition result.

[0016] The present invention improves the YOLOv5x model and uses the improved model to train, verify, and test a data set, ultimately obtaining a detection model with high precision and high speed, achieving good results in distribution network material image recognition. The MSA-YOLOv5x model is obtained based on the improved YOLOv5x model. The improvements include introducing the ACON-C activation function, introducing the D-SPP module, embedding a masked self-attention mechanism module, and improving the loss function. The introduction of the ACON-C activation function improves the network's feature extraction capability; the introduction of the D-SPP module allows convolutional pooling operations to generate adaptive spatial weights for different receptive field features, and by combining different channel information, different receptive field features can be efficiently fused, enhancing feature representation capabilities; the embedding of the masked self-attention mechanism module performs masked self-attention operations on feature maps of large, medium, and small sizes, enhancing the network's ability to capture global feature relationships, completing the recognition of multi-scale distribution network materials, improving the algorithm's detection accuracy, and enhancing the model's robustness.

[0017] As a preferred solution, the ACON-C activation function is introduced after the concat module of the CSP module. The ACON-C activation function is:

[0018] fACON-C (x) = (p1-p2)x*σ[β(p1-p2)x]+p2x

[0019] where x is the input value, p1 and p2 are learnable parameters, σ is a Sigmoid function, and β is a switching factor.

[0020] The ACON-C activation function can learn whether to activate neurons, scale features using hyperparameters, and introducing the ACON-C activation function can improve the feature extraction capability of the network.

[0021]

[0022] In the formula, S is a smooth maximum function, η a (x),η b (x) is a linear function.

[0023] Let η a (x) = p1x, η b (x) = p2x (p1≠p2)

[0024] Then f ACON- (x) = S β (p1x, p2x).

[0025] As a preferred solution, the D-SPP module includes an SPP module, two CBL modules, three max pooling modules, and a concat module. The CBL module, the first max pooling module, the SPP module, the second max pooling module, and the third max pooling module are sequentially connected to the concat module. The first max pooling module, the SPP module, and the second max pooling module are respectively connected to the concat module. The concat module outputs are connected to the CBL module. The YOLOv5x model uses the SPP module to obtain different receptive field information in the backbone network. The SPP module cannot sufficiently fuse local information and global information, which easily causes information loss. The present solution proposes a D-SPP module, which nests an SPP between the first max pooling module and the second max pooling module in the SPPF module structure, generates adaptive weights for different size feature maps, and combines different channel information to efficiently fuse different receptive field features and enhance feature representation capability. The D-SPP module generates adaptive spatial weights for different receptive field features through convolutional pooling operations. First, a fixed scale pooling branch is used, and then the SPP module is used to extract information, embed spatial information, fuse context features through weighting, generate new features containing multi-scale context information, and fuse the feature maps obtained through two max pooling operations with the previously obtained feature maps.

[0026] As a preferred solution, the mask self-attention mechanism module comprises multiplying the input feature map and the convolution kernel W Q , W K , W V to generate a query vector matrix Q, a key vector matrix K and a value vector matrix V respectively, performing inner product operation on the query vector matrix Q and the transpose of the key vector matrix K through scaled dot product, performing normalization through a Softmax activation function to obtain attention scores, performing mask calculation on the obtained attention scores, and finally multiplying the value vector matrix V to obtain the self-attention output.

[0027] As a preferred solution, the improved loss function comprises a confidence loss, a classification loss and a bounding box regression loss, and the improved loss function formula is as follows:

[0028]

[0029] wherein L Focal-EIo represents Focal-EIoU Loss, and respectively represent the probability that the jth prior box of the ith network contains the network material and does not contain the network material, the value of the containing is 1 and the value of the not containing is 0, represents the confidence level of the jth predicted box of the ith grid, C i represents the confidence level of the real box, represents the class probability value of the predicted box, p i represents the class probability value of the real box.

[0030] As a preferred solution, the preprocessing in S1 comprises filtering and denoising processing on the image.

[0031] Since the collected network material image contains salt and pepper noise, the median filtering algorithm can eliminate isolated noise values and is suitable for processing spot noise and salt and pepper noise. In this solution, the median filtering method is specifically used for image denoising. A 3*3 filtering window is adopted, the original network material image sub-region at the filtering window (i,j) is set as I i,j , and the pixel value at the position I(s,t) is calculated according to the following formula,

[0032]

[0033] As a preferred solution, the processed image dataset in S2 is expanded, specifically comprising:

[0034] The image is translated, flipped, rotated, scaled, contrast transformed and brightness transformed to obtain an expanded image dataset.

[0035] Assume that the size of the distribution network material image is M×N, I0(x,y) represents the coordinates of a point on the distribution network material image before enhancement, and I0(x′,y′) represents the coordinates of the point after enhancement.

[0036] Translation: Move all pixels in the distribution network material image horizontally or vertically according to a certain translation amount. The calculation formula is as follows:

[0037]

[0038] Wherein, Δx represents the distance that the distribution network material image moves in the horizontal direction, and Δy represents the distance that the distribution network material image moves in the vertical direction.

[0039] Flip: Transform the distribution network material image along the axis in an axisymmetric manner. The horizontal flip type flips the distribution network material image left and right along the vertical center axis, and the vertical flip type flips the distribution network material attributes up and down along the horizontal center axis. The calculation formulas are as follows:

[0040]

[0041]

[0042] To rotate, first select the rotation center, then rotate 30 degrees or 60 degrees clockwise or counterclockwise. The formula for calculating the counterclockwise rotation angle θ is as follows:

[0043]

[0044] Scaling transformation: enlarge the distribution network material image by 1.5 times, or reduce it by 0.5 times.

[0045] Contrast transformation: adjust the contrast of the distribution network material image according to the contrast change factor.

[0046] Brightness conversion: enhance or reduce the brightness of the distribution network material image according to the ratio of 1.2 or 0.8.

[0047] By expanding the collected 1,000 images to 10,000 images, the amount of training data is effectively increased, which enables each model to learn sufficient features in the subsequent training process, without being affected by factors such as lighting, background, and different target scales, thereby enhancing the robustness of the model.

[0048] Therefore, the advantages of the present invention are:

[0049] The YOLOv5x model was improved and the data set was trained, verified, and tested using the improved model. Finally, a detection model with high precision and high speed was obtained, achieving good results in distribution network material image recognition.

[0050] Improvements include the introduction of the ACON-C activation function, the D-SPP module, the embedding of a masked self-attention mechanism module, and an improved loss function. The introduction of the ACON-C activation function enhances the network's feature extraction capabilities; the introduction of the D-SPP module allows convolutional pooling to generate adaptive spatial weights for features with different receptive fields, and by combining information from different channels, enables efficient fusion of features with different receptive fields, enhancing feature representation capabilities; the embedding of the masked self-attention mechanism module performs masked self-attention operations on feature maps of large, medium, and small sizes, enhancing the network's ability to capture global feature relationships and complete the identification of multi-scale distribution network materials, improving the algorithm's detection accuracy and enhancing the model's robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of a process of the present invention;

[0052] Figure 2 This is a structural diagram of the MSA-YOLOv5x model of the present invention;

[0053] Figure 3 This is a structural diagram of the masked self-attention mechanism module in the present invention. DETAILED DESCRIPTION

[0054] The technical solution of the present invention will be further specifically described below through embodiments and in conjunction with the accompanying drawings.

[0055] Example:

[0056] This embodiment provides an image recognition method for typical distribution network materials based on MSA-YOLOv5x, including the following steps:

[0057] S1. Collect distribution network material image data, label and preprocess the image data;

[0058] The collected distribution network material images are labeled using LabelImg software. Then preprocessing is performed. The preprocessing in this embodiment mainly involves filtering and denoising the image. Since the collected distribution network material images contain salt and pepper noise, the median filter algorithm can eliminate isolated noise values ​​and is suitable for processing speckle noise and salt and pepper noise. Specifically, this embodiment uses the median filter method to denoise the image. A 3×3 filter window is used, and the sub-region of the original distribution network material image at the filter window (i, j) is I i,j , the pixel value at position I(s,t), the material image of the material network after median filtering is calculated as shown in the following formula,

[0059]

[0060] S2. Expand the processed image dataset;

[0061] The image is mainly translated, flipped, rotated, scaled, contrasted, and brightly transformed to obtain an expanded image dataset.

[0062] Assume that the size of the distribution network material image is M×N, I0(x,y) represents the coordinates of a point on the distribution network material image before enhancement, and I0(x′,y′) represents the coordinates of the point after enhancement.

[0063] Translation: Move all pixels in the distribution network material image horizontally or vertically according to a certain translation amount. The calculation formula is as follows:

[0064]

[0065] Wherein, Δx represents the distance that the distribution network material image moves in the horizontal direction, and Δy represents the distance that the distribution network material image moves in the vertical direction.

[0066] Flip: Transform the distribution network material image along the axis in an axisymmetric manner. The horizontal flip type flips the distribution network material image left and right along the vertical center axis, and the vertical flip type flips the distribution network material attributes up and down along the horizontal center axis. The calculation formulas are as follows:

[0067]

[0068]

[0069] To rotate, first select the rotation center, then rotate 30 degrees or 60 degrees clockwise or counterclockwise. The formula for calculating the counterclockwise rotation angle θ is as follows:

[0070]

[0071] Scaling transformation: enlarge the distribution network material image by 1.5 times, or reduce it by 0.5 times.

[0072] Contrast transformation: adjust the contrast of the distribution network material image according to the contrast change factor.

[0073] Brightness conversion: enhance or reduce the brightness of the distribution network material image according to the ratio of 1.2 or 0.8.

[0074] By expanding the collected 1,000 images to 10,000 images, the amount of training data is effectively increased, which enables each model to learn sufficient features in the subsequent training process, without being affected by factors such as lighting, background, and different target scales, thereby enhancing the robustness of the model.

[0075] S3. The MSA-YOLOv5x model is constructed based on the YOLOv5x model. The improved model includes introducing the ACON-C activation function in the CSP module, replacing the SPP module with the D-SPP module, embedding the masked self-attention mechanism module after the output of the neck network effective feature layer and before the input of the prediction layer, and improving the loss function. The improved YOLOv5x is named MSA-YOLOv5x, and its structure is as follows: Figure 2 shown.

[0076] (1) Introducing the ACON-C activation function into the CSP module;

[0077] Specifically, the ACON-C activation function is introduced after the concat module of the CSP module. The ACON-C activation function can learn whether to activate neurons and scale features using hyperparameters. The introduction of the ACON-C activation function can improve the feature extraction ability of the network. It is expressed as follows:

[0078]

[0079] Where S is the smooth maximum function, η a (x),η b (x) is a linear function;

[0080] Let η a (x) = p1x, η b (x) = p2x (p1≠p2), then the ACON-C output is derived as:

[0081] f ACON- (x) = S β (p1x,p2x)=(p1-p2)x*σ[β(p1-p2)x]+p2x.

[0082] (2) Replace the SPP module with the D-SPP module;

[0083] The YOLOv5x model uses the SPP module in the backbone network to obtain different receptive field information. The SPP module cannot fully integrate local information and global information, which easily causes information loss. This embodiment proposes a D-SPP module to replace the SPP module in the backbone network, such as Figure 2 As shown in the figure, the D-SPP module includes an SPP module, two CBL modules, three maximum pooling modules, and a concat module. The CBL module, the first maximum pooling module, the SPP module, the second maximum pooling module, and the third maximum pooling module are connected in sequence and then connected to the concat module. The first maximum pooling module, the SPP module, and the second maximum pooling module are respectively connected to the concat module, and the output of the concat module is connected to the CBL module.

[0084] The D-SPP module is structured by nesting SPP on top of SPPF (Fast Spatial Pyramid Pooling), generating adaptive spatial weights for features with different receptive fields through convolutional pooling operations. First, a fixed-scale pooling branch is used, followed by the SPP module to extract information and embed spatial information. By weighted fusion of contextual features, new features containing multi-scale contextual information are generated. The feature maps obtained through two maximum pooling steps are then fused with the previously obtained feature maps, enabling efficient fusion of features with different receptive fields and enhancing feature capabilities.

[0085] (3) A masked self-attention mechanism module is embedded after the output of the effective feature layer of the neck network and before the input of the prediction layer;

[0086] The masked self-attention mechanism module, namely the MSA module, performs masked self-attention operations on feature maps of three sizes: large, medium, and small, enhancing the network's ability to capture global feature relationships, completing the detection and recognition of multi-scale distribution network materials, and improving the algorithm's detection accuracy. The masked self-attention mechanism uses parallel query to extract the correlation degree from the input feature vector to different subspaces. Its structure is as follows: Figure 3 As mentioned above, the input feature map X and the convolution kernel W Q 、W K 、W V Multiply them to generate the query vector matrix Q, key vector matrix K, and value vector matrix V respectively. The calculation is shown in the following formula:

[0087]

[0088] By scaling the dot product, we perform an inner product operation on the transpose of the query vector matrix Q and the key vector matrix K, normalize it with the Softmax activation function to get the attention score, perform mask calculation on the obtained attention score, and finally multiply it with the value vector matrix V to get the output of the self-attention. The output of the masked self-attention mechanism module is shown in the following formula:

[0089]

[0090] (4) Improve the loss function;

[0091] The improved YOLOv5x model loss function includes confidence loss, classification loss, and bounding box regression loss. Confidence loss and classification loss are calculated using the binary cross-entropy loss function, while bounding box regression loss is calculated using Focal-EIoU. The width-height loss in Focal-EIoU Loss directly minimizes the difference between the width and height of the predicted box and the true box, resulting in faster convergence and resolving the ambiguous definition of the CIoU aspect ratio. Focal Loss is also added to address sample imbalance in bounding box regression. The Focal-EIoU Loss calculation formula is as follows:

[0092]

[0093] L Focal-EIoU =IoUγL EIoU

[0094] Among them, w, h, and b are the width, height, and center coordinate parameters of the prediction box respectively. gt 、h gt 、b gt are the width, height, and center coordinate parameters of the real frame, respectively, and ρ is b and b gt The Euclidean distance between the predicted box and the real box is , c is the diagonal length of the minimum circumscribed rectangle between the predicted box and the real box, and IoU is the intersection-over-union ratio between the candidate box and the real box. The formula for IoU is as follows:

[0095]

[0096] Where A is the candidate box and B is the real box.

[0097] The improved loss function is shown in the following formula:

[0098]

[0099] Among them L Focal-EIoU represents Focal-EIoU Loss, and They represent the probability that the j-th prior frame of the i-th network contains the distribution network material and does not contain the distribution network material, respectively. The value of inclusion is 1, and the value of non-inclusion is 0. represents the confidence level of the j-th prediction box of the i-th grid, C i represents the confidence level of the ground-truth box, Represents the category probability value of the prediction box, p i Represents the category probability value of the true box.

[0100] S4. Use the image dataset to train the MSA-YOLOv5x model and obtain the optimal model;

[0101] 1) Determine evaluation indicators

[0102] Taking the dry-type transformer as an example, the confusion matrix is ​​shown in the following table:

[0103] Taking the dry-type transformer as an example, the confusion matrix is ​​shown in the following table:

[0104]

[0105] a) P (accuracy)

[0106]

[0107] b) R (Recall)

[0108]

[0109] c) AP (Average Detection Precision)

[0110]

[0111] d) mAP (mean average detection precision)

[0112]

[0113] Among them, M represents the number of types of distribution network materials. Since 10 types of distribution network materials need to be identified, M=10, AP i It is the average accuracy of each type of distribution network material.

[0114] e) FPS (frames per second)

[0115]

[0116] Wherein, N is the total number of test samples, and T is the time consumed for testing the test samples.

[0117] 2) Model training

[0118] The dataset is divided into training set, validation set, and test set in a ratio of 6:2:2. Assuming the batch size is 20 and the number of iterations is 100, the number of batches is 400 and the number of iterations is 40,000. Training is performed based on the dataset, and the weight parameters are continuously updated. At the end of the training, the optimal detection model is obtained.

[0119] S5. Use the optimal model to predict the distribution network material image and obtain the recognition result.

[0120] The trained model is used to make predictions on the distribution network material test set. The recognition results are shown in the following table:

[0121]

[0122] This embodiment recognizes typical distribution network material images based on the MSA-YOLOv5x model, and improves the network's feature extraction capability by introducing the ACON-C activation function. The D-SPP module is introduced, and the convolution pooling operation generates adaptive spatial weights for different receptive field features. By combining different channel information, different receptive field features can be efficiently fused, thereby enhancing the feature representation capability. By introducing the masked self-attention mechanism module, masked self-attention operations are performed on feature maps of large, medium, and small sizes, respectively, enhancing the network's ability to capture global feature relationships, completing the recognition of multi-scale distribution network materials, improving the detection accuracy of the algorithm, and enhancing the robustness of the model.

[0123] The specific embodiments described herein are merely illustrative of the spirit of the present invention. Persons skilled in the art may make various modifications, additions, or substitutions to the described specific embodiments without departing from the spirit of the present invention or exceeding the scope of the appended claims.

Claims

1. A typical distribution network material image recognition method based on MSA-YOLOv5x, characterized by: The following steps are involved: S1. Collect distribution network material image data, label and preprocess the image data; S2. Expand the processed image dataset; S3. Build the MSA-YOLOv5x model based on the YOLOv5x model, including: Introduce the ACON-C activation function in the CSP module, Replace the SPP module with the D-SPP module, A masked self-attention mechanism module is embedded after the output of the effective feature layer of the neck network and before the input of the prediction layer. Improve the loss function; S4. Use the image dataset to train the MSA-YOLOv5x model and obtain the optimal model; S5. Use the optimal model to predict the distribution network material image and obtain the recognition result.

2. The image recognition method of a typical distribution network material based on MSA-YOLOv5x according to claim 1 is characterized in that The ACON-C activation function is introduced after the concat module of the CSP module. The ACON-C activation function is: f ACON-C (x)=(p1-p2)x*σ[β(p1-p2)x]+p2x Where x is the input value, p1 and p2 are learnable parameters, σ is the Sigmoid function, and β is the switching factor.

3. The image recognition method of a typical distribution network material based on MSA-YOLOv5x according to claim 1 is characterized in that The D-SPP module includes an SPP module, two CBL modules, three maximum pooling modules, and a concat module. The CBL module, the first maximum pooling module, the SPP module, the second maximum pooling module, and the third maximum pooling module are connected in sequence and then connected to the concat module. The first maximum pooling module, the SPP module, and the second maximum pooling module are respectively connected to the concat module, and the output of the concat module is connected to the CBL module.

4. The image recognition method of a typical distribution network material based on MSA-YOLOv5x according to claim 1 is characterized in that The mask self-attention mechanism module includes the input feature map and the convolution kernel W Q 、W K 、W V Multiply them to generate the query vector matrix Q, key vector matrix K, and value vector matrix V respectively. Perform the inner product operation on the transpose of the query vector matrix Q and the key vector matrix K through the scaled dot product, use the Softmax activation function to normalize it to get the attention score, perform mask calculation on the obtained attention score, and finally multiply it with the value vector matrix V to get the self-attention output.

5. The image recognition method of a typical distribution network material based on MSA-YOLOv5x according to claim 1 is characterized in that The improved loss function includes confidence loss, classification loss and bounding box regression loss. The improved loss function formula is as follows: Among them L Focal-EIoU represents Focal-EIoU Loss, and They represent the probability that the j-th prior frame of the i-th network contains the distribution network material and does not contain the distribution network material, respectively. The value of inclusion is 1, and the value of non-inclusion is 0. represents the confidence level of the j-th prediction box of the i-th grid, C i represents the confidence level of the ground-truth box, Represents the category probability value of the prediction box, p i Represents the category probability value of the true box.

6. The image recognition method for typical distribution network materials based on MSA-YOLOv5x according to any one of claims 1 to 5, characterized in that The preprocessing in S1 includes filtering and denoising the image.

7. The image recognition method for typical distribution network materials based on MSA-YOLOv5x according to any one of claims 1 to 5, characterized in that S2 The processed image dataset is expanded in the following ways: The image is translated, flipped, rotated, scaled, contrast-transformed, and brightness-transformed to obtain an expanded image dataset.

Images