Strip steel surface defect detection method based on RAC-YOLO algorithm

Abstract

The invention discloses a strip steel surface defect detection method based on an RAC-YOLO algorithm, and belongs to the technical field of industrial automatic detection and computer vision. Comprising the steps that an image data set is acquired and preprocessed; the method comprises the following steps: constructing a RAC-YOLO network based on YOLOv8, and replacing an original feature pyramid module SPPF with an intra-scale feature interaction module AIFI; meanwhile, an attention mechanism module SE is introduced behind the AIFI module; a lightweight upsampling operator CARAFE is used for replacing an original upsampling model; replacing an original C2f module with a context anchor attention feature extraction and fusion module; meanwhile, the network loss function is replaced by the MPDIOU; and completing the training of the RAC-YOLO network by using the preprocessed image data set. Various defects on the surface of the strip steel can be rapidly and accurately detected on a high-speed production line, the production efficiency and the product quality are effectively improved, and a more reliable quality control solution is provided for industrial manufacturing.

Description

Technical Field

[0001] The present invention relates to the field of industrial automation detection and computer vision technology, and in particular to a strip steel surface defect detection method based on a RAC-YOLO algorithm. Background Art

[0002] With the rapid development of modern industrial manufacturing technology, the steel industry has increasingly higher requirements for product quality. As an important metal material, the surface quality of strip steel directly affects the performance and service life of downstream products. Therefore, accurate and rapid detection and classification of strip surface defects is a key link in improving production efficiency and product quality.

[0003] In recent years, with the development of deep learning technology, target detection algorithms based on convolutional neural networks (CNN) have gradually been applied to strip surface defect detection. These algorithms can automatically extract image features, classify and locate defects, greatly improving detection accuracy and efficiency. Among them, the YOLO algorithm has been widely used and received attention due to its fast detection and end-to-end training characteristics. However, there are also some significant disadvantages. The convolutional neural network has a large amount of calculation and slow processing speed due to its complex network structure, which makes it difficult to meet the needs of real-time detection. Secondly, although the YOLOv8 algorithm is fast, it performs poorly in detecting small-sized and complex-shaped defects, and is prone to missed detection and false detection problems, especially for the complex background and large changes in illumination on the strip surface, and its robustness is insufficient. Summary of the invention

[0004] In view of the defects of the prior art, the present invention adopts contextual anchor attention feature extraction fusion, intra-scale feature interaction and content-aware feature reorganization upsampling to improve the YOLOv8 network, which can quickly and accurately detect various defects on the surface of strip steel on high-speed production lines, effectively improve production efficiency and product quality, and provide a more reliable quality control solution for industrial manufacturing.

[0005] In order to achieve the above object, the present invention provides a strip surface defect detection method based on RAC-YOLO algorithm, comprising the following steps:

[0006] (1) Obtain image data set and perform preprocessing;

[0007] (2) Build a RAC-YOLO network based on YOLOv8, including Backbone module, Neck module and Head module;

[0008] In the Backbone module, the original feature pyramid module SPPF is replaced by the intra-scale feature interaction module AIFI; and the attention mechanism module SE is introduced after the AIFI module;

[0009] In the Neck module, the lightweight upsampling operator CARAFE is used to replace the original upsampling model;

[0010] In the Backbone module and the Neck module, the original C2f module is replaced by a contextual anchor attention feature extraction fusion module;

[0011] At the same time, the network loss function is replaced with the MPDIoU loss function to improve the training effect of bounding box regression;

[0012] (3) The RAC-YOLO network is trained using the preprocessed image data set, and the trained RAC-YOLO network is used to detect surface defects of the strip.

[0013] Further, the preprocessing includes:

[0014] The images in the image dataset are processed using a histogram equalization method;

[0015] Adjusting the contrast and brightness distribution of the equalized image data set so that the distribution of pixels at different gray levels is more uniform;

[0016] The adjusted image dataset is divided into a training set and a test set in a ratio of 7:3.

[0017] Furthermore, the Backbone module includes two convolution modules, four context anchor attention feature extraction and fusion modules, three downsampling modules Adown, one AIFI module and one SE module;

[0018] The two convolution modules in the backbone network are the 0th and 1st layers respectively; the four context anchor attention feature extraction and fusion modules are the 2nd, 4th, 6th and 8th layers respectively; the three downsampling modules Adown are the 3rd, 5th and 7th layers; the AIFI module is the 9th layer; the SE module is the 10th layer; among them, the feature information extracted by the 4th and 6th layer context anchor attention feature extraction and fusion modules and the 9th layer AIFI module is output to the Neck module.

[0019] Furthermore, the AIFI module is specifically:

[0020] The input two-dimensional image is converted into a one-dimensional vector, multi-head attention processing is applied to the one-dimensional vector, the processed sequence is residually connected with the original one-dimensional vector and normalized, nonlinear learning and normalization are performed through a feedforward network, and finally the normalized one-dimensional vector is converted back into a two-dimensional image;

[0021] The formula is:

[0022] Q=K=V=Flatten(Input)

[0023] Output=Reshape(FFN(MultiHead(Q, K, V)))

[0024] Among them: MultiHead is the multi-head attention function; FFN is the feedforward network.

[0025] Furthermore, the contextual anchor attention feature extraction fusion module RepNCSPELAN4_CAA is a network module formed by integrating the feature extraction module RepNCSPELAN4 with the contextual anchor attention mechanism CAAttention; the RepNCSPELAN4 combines two neural network structures CSPNet and ELAN, uses the segmentation and reorganization of CSPNet, and introduces the hierarchical convolution of ELAN in each part; the two parts of the segmented feature map are respectively subjected to different convolution processing and recombined into a new feature map, and then input into the CAAttention module for contextual anchor attention processing, and then the final output result is obtained through the last layer of convolution.

[0026] Further, the lightweight upsampling operator CARAFE includes a kernel prediction module and a content-aware reconstruction module;

[0027] The kernel prediction module consists of three submodules: channel compressor, content encoder and kernel normalizer; first, the channel compression submodule is used to compress the input feature channel, secondly, the content encoder submodule encodes the content of the compressed feature map to generate a reconstructed kernel, and finally, the kernel normalizer submodule applies a softmax function to each reconstructed kernel.

[0028] Furthermore, the MPDIoU loss function uses the minimum point distance to directly predict the distance between the upper left corner and the lower right corner of the bounding box and the real box, thereby improving the efficiency and accuracy of bounding box regression; specifically:

[0029]

[0030]

[0031] in: represents the bounding box coordinates of the prediction box, Represents the coordinates of the bounding box of the real box; d1 represents the distance between the upper left corner coordinates of the predicted box and the upper left corner coordinates of the real box, d2 represents the distance between the upper right corner coordinates of the predicted box and the upper right corner coordinates of the real box, w and h represent the width and height of the input image respectively.

[0032] Beneficial effects of the present invention:

[0033] The RAC-YOLO network proposed in the present invention adopts the RepNCSPELAN4_CAA module to replace the C2f module in the original backbone, thereby enhancing the network's ability to extract surface defect features of strip steel; adopts the AIFI module to replace the pyramid pooling layer SPPF module in the original backbone, further optimizing the multi-scale feature fusion process; adopts the lightweight upsampling CARAFE operator to replace the traditional upsampling method in Neck, thereby improving the accuracy of the model, reducing the amount of calculation and enhancing the generalization ability; adopts the bounding box regression loss function MPDIoU to replace the original loss function CIoU, thereby accelerating the convergence speed of the model and improving the accuracy of regression. The present invention can quickly and accurately detect various defects on the surface of strip steel on a high-speed production line, effectively improving production efficiency and product quality, and providing a more reliable quality control solution for industrial manufacturing. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic flow chart of a strip steel surface defect detection method based on the RAC-YOLO algorithm according to an embodiment of the present invention.

[0035] Figure 2 Schematic diagram of the RAC-YOLO network structure according to an embodiment of the present invention.

[0036] Figure 3 It is a schematic diagram of the network module structure of the context anchor attention feature extraction fusion module RepNCSPELAN4_CAA of an embodiment of the present invention.

[0037] Figure 4 It is a structural diagram of the scale feature interaction module AIFI in an embodiment of the present invention.

[0038] Figure 5 It is a schematic diagram of the structure of the lightweight upsampling CARAFE operator according to an embodiment of the present invention.

[0039] Figure 6 It is a schematic diagram of the MPDIoU structure of the bounding box loss function according to an embodiment of the present invention.

[0040] Figure 7 It is a detection result diagram of the strip surface defect detection method based on the RAC-YOLO algorithm adopted in the embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be further explained below in conjunction with the accompanying drawings and embodiments.

[0042] like Figure 1 As shown, an embodiment of the present invention provides a strip surface defect detection method based on the RAC-YOLO algorithm, comprising the following steps:

[0043] S101, obtaining an image data set and performing preprocessing.

[0044] The public dataset NEU-DET is obtained. This dataset contains six typical surface defects of hot-rolled steel strip, namely rolling scale (Rs), plaque (Pa), crack (Cr), pitting surface (PS), inclusion (In) and scratch (Sc). The dataset contains 1800 grayscale images, and each defect type contains 300 samples. The dataset is preprocessed, including:

[0045] The histogram equalization method is used to process the images in the image dataset, and the contrast and image brightness distribution are adjusted to make the pixel distribution of different gray levels more uniform.

[0046] The processed image dataset will be divided into training set and test set in a ratio of 7:3.

[0047] S102. Build a RAC-YOLO network based on improved YOLOv8, including a Backbone module, a Neck module, and a Head module.

[0048] like Figure 2 As shown in the figure, in the Backbone module, the intra-scale feature interaction module AIFI is used to replace the original feature pyramid module SPPF; at the same time, the attention mechanism module SE is introduced after the AIFI module; in the Neck module, the lightweight upsampling operator CARAFE is used to replace the original upsampling model; in the Backbone module and the Neck module, the context anchor attention feature extraction fusion module is used to replace the original C2f module.

[0049] The Backbone module consists of two convolution modules, four contextual anchor attention feature extraction fusion modules, three downsampling modules Adown, an AIFI module and an SE module.

[0050] The two convolution modules in the backbone network are the 0th and 1st layers respectively; the four context anchor attention feature extraction and fusion modules are the 2nd, 4th, 6th and 8th layers respectively; the three downsampling modules Adown are the 3rd, 5th and 7th layers; the AIFI module is the 9th layer; the SE module is the 10th layer; among them, the feature information extracted by the 4th and 6th layer context anchor attention feature extraction and fusion modules and the 9th layer AIFI module is output to the Neck module.

[0051] The intra-scale feature interaction module AIFI uses a self-attention mechanism for high-level features with rich semantic concepts to coordinate the intra-scale interactions between these features. This enables the network to better capture the relationship between conceptual entities in the image, thereby helping subsequent modules to more effectively detect and identify objects in the image. Specifically:

[0052] like Figure 4 As shown in the figure, first, the input two-dimensional image is converted into a one-dimensional vector for subsequent processing; then, multi-head attention is used to process the input one-dimensional vector, allowing the model to perform multiple sets of self-attention on the input sequence, helping the model to better capture information at different positions in the sequence and improve its ability to handle long-range dependencies; next, the processed sequence is residually connected to the original one-dimensional vector and normalized, and nonlinear learning is introduced through a feedforward network, and normalization is performed, so that the network can learn the complex relationship between different elements in the input sequence; finally, the normalized one-dimensional vector is converted back to a two-dimensional image for subsequent network processing.

[0053] The formula is:

[0054] Q=K=V=Flatten(Input)

[0055] Output=Reshape(FFN(MultiHead(Q, K, V)))

[0056] Among them: MultiHead is the multi-head attention function; FFN is the feedforward network.

[0057] like Figure 3 As shown in Figure 1, the contextual anchor attention feature extraction fusion module RepNCSPELAN4_CAA is a block architecture that is composed of the feature extraction module RepNCSPELAN4 fused with the contextual anchor attention mechanism CAAttention. RepNCSPELAN4_CAA enhances feature representation by combining efficient feature extraction with the attention mechanism.

[0058] RepNCSPELAN4 enhances functionality by combining two neural network structures, CSPNet and ELAN. It uses the segmentation and reorganization of CSPNet, and introduces the hierarchical convolution of ELAN in each part. The CSPNet connection promotes the flow of information between different stages of the network, and ELAN enhances feature representation through the attention mechanism. The two parts of the segmented feature map are reorganized into new feature maps after different convolution processing, and then input into the CAAttention module for contextual anchor attention processing, and then the final output result is obtained after the last layer of convolution. The contextual anchor attention mechanism aggregates and displays contextual information, considering factors such as position information, to obtain a more accurate representation of user interests, thereby improving the performance and accuracy of the model.

[0059] like Figure 5As shown in the figure, the lightweight upsampling operator CARAFE introduces content awareness by adaptively predicting the upsampling kernel based on the content of the feature map. Different from the traditional methods, this mechanism highlights two main components: the kernel prediction module and the content-aware reconstruction module; CARAFE not only has a wide receptive field, but also cleverly utilizes the input features for reorganization, ensuring a harmonious and efficient upsampling process.

[0060] The kernel prediction module consists of three submodules: channel compressor, content encoder, and kernel normalizer. First, the input feature channel is compressed by the channel compression submodule. Second, the content encoder submodule encodes the content of the compressed feature map to generate a reconstructed kernel. Finally, the kernel normalizer submodule applies a softmax function to each reconstructed kernel. In the upsampling kernel prediction module, the upsampling factor is σ, and the number of channels is first compressed to Cm. Then, a convolution operation of size kencoder×kencoder is performed to achieve content encoding. Finally, the spatial dimension is expanded and the kernel is normalized so that the weight sum of the convolution kernel is 1.

[0061] In the feature reorganization module, take out the kup×kup region centered at each position of the feature map and the reorganization kernel W of the upsampling prediction module l ′ is dot-producted, and finally the upsampled feature map is obtained; the lightweight upsampling operator CARAFE expands the receptive field, adapts to the input content and fully retains the small target information of the upsampled feature map.

[0062] The upsampled kernel prediction module takes as input a C×H×W input feature map X, upsampled by a factor of σ, and the resulting feature map X′ spans C×σH×σW. A key aspect of the CARAFE operation involves the relationship between a target location l′=(i′, j′) in X′ and its analogous source location l=(i, j) in X, expressed as:

[0063]

[0064] N(Xl,k) is designated as a kencoder×kencoder subregion centered at X position l, and the kernel prediction module ψ uses this subregion to infer the reorganization kernel Wl' for each target position l' and combines it with the content-aware reorganization module Combined, the upsampled features of position l' are generated:

[0065] W l′ =ψ(N(X l , k encoder ))

[0066] X l″ =φ(N(X l , k up ), Wl′ )

[0067] ψ(x)=x*ω+b

[0068] Among them, ω represents weight, b represents bias, both of which are improved during training; ψ represents the prediction module.

[0069] CARAFE is integrated into the original architecture to replace the traditional nearest neighbor interpolation method and obtain the content-aware upsampled output. The parameter count is expressed as:

[0070]

[0071] Where: C in Represents the number of input channels, C m Represents the number of channels after compression, Represents the size of the convolution kernel generated by the encoder, Reorganize the convolution kernel size, σ represents the upsampling ratio, C i Represents the number of reorganized convolution kernel channels.

[0072] In order to further accelerate the convergence speed of the model and improve the accuracy of regression, the RAC-YOLO network uses the MPDIoU loss function instead of the original CIoU loss function to improve the training effect of bounding box regression. Figure 6 As shown in Figure 1, MPDIoU uses the minimum point distance to directly predict the distance between the upper left corner and the lower right corner of the bounding box and the real box, thereby improving the efficiency and accuracy of bounding box regression. Compared with the CIoU loss function, MPDIoU not only inherits the advantages in overlapping areas, center point positioning, width and height differences, but also simplifies the comparison process of the similarity between the predicted box and the real box. The specific formula is expressed as:

[0073]

[0074]

[0075] in: represents the bounding box coordinates of the prediction box, Represents the coordinates of the ground-truth bounding box; is the coordinate of the upper left corner of the prediction box, is the coordinate of the upper left corner of the prediction box; is the coordinate of the upper right corner of the real box, is the coordinate of the upper right corner of the prediction box; d1 represents the distance between the coordinate of the upper left corner of the prediction box and the coordinate of the upper left corner of the real box, d2 represents the distance between the coordinate of the upper right corner of the prediction box and the coordinate of the upper right corner of the real box, w and h represent the width and height of the input image respectively.

[0076] S103, using the preprocessed image data set to train the RAC-YOLO network, and using the trained RAC-YOLO network to perform strip surface defect detection.

[0077] The preprocessed NEU-DET dataset is used, and the RAC-YOLO model is trained and verified using the MPDIou loss function and AdamW optimizer.

[0078] The accuracy, recall and mean average precision (mAP) used in target detection are used as evaluation indicators of the model defect detection effect.

[0079]

[0080]

[0081] Among them, TP is true positives, which means the number of correctly predicted positive samples; FP is false positives, which means the number of negative samples detected as positive samples; FN is the number of positive samples mistakenly judged as negative samples; AP is the PR curve area, and mAP is the mean of AP of all categories.

[0082] At the same time, in order to analyze the effect of the model in small-scale, medium-scale and large-scale object recognition, the standard COCO performance index is used. In addition, in order to better compare RAC-YOLO with other models, the number of frames per second FPS and floating-point operations FLOPS are added as evaluation indicators. Figure 6 The test results are given.

[0083] Obtain the surface image of the strip to be inspected, and use the trained RAC-YOLO network to detect surface defects of the strip.

[0084] Although specific embodiments of the present invention are described above, those skilled in the art should understand that these are merely examples and that various changes or modifications may be made to these embodiments without departing from the present invention and its essence. Therefore, the scope of protection of the present invention is limited by the appended claims.

Claims

1. A strip steel surface defect detection method based on RAC-YOLO algorithm, characterized in that: The steps include: (1) Obtain image data set and perform preprocessing; (2) Build a RAC-YOLO network based on YOLOv8, including Backbone module, Neck module and Head module; In the Backbone module, the original feature pyramid module SPPF is replaced by the intra-scale feature interaction module AIFI; and the attention mechanism module SE is introduced after the AIFI module; In the Neck module, the lightweight upsampling operator CARAFE is used to replace the original upsampling model; In the Backbone module and the Neck module, the original C2f module is replaced by a contextual anchor attention feature extraction fusion module; At the same time, the network loss function is replaced with the MPDIoU loss function to improve the training effect of bounding box regression; (3) The RAC-YOLO network is trained using the preprocessed image data set, and the trained RAC-YOLO network is used to detect surface defects of the strip.

2. The strip surface defect detection method based on the RAC-YOLO algorithm according to claim 1 is characterized in that: The pre-processing comprises: The images in the image dataset are processed using a histogram equalization method; Adjusting the contrast and brightness distribution of the equalized image data set so that the distribution of pixels at different gray levels is more uniform; The adjusted image dataset is divided into a training set and a test set in a ratio of 7:

3.

3. The strip steel surface defect detection method based on the RAC-YOLO algorithm according to claim 1, characterized in that: The Backbone module includes two convolution modules, four context anchor attention feature extraction and fusion modules, three downsampling modules Adown, an AIFI module and an SE module; The two convolution modules in the backbone network are the 0th and 1st layers respectively; the four context anchor attention feature extraction and fusion modules are the 2nd, 4th, 6th and 8th layers respectively; the three downsampling modules Adown are the 3rd, 5th and 7th layers; the AIFI module is the 9th layer; the SE module is the 10th layer; among them, the feature information extracted by the 4th and 6th layer context anchor attention feature extraction and fusion modules and the 9th layer AIFI module is output to the Neck module.

4. The strip surface defect detection method based on the RAC-YOLO algorithm according to claim 1 is characterized in that: The AIFI module is specifically: The input two-dimensional image is converted into a one-dimensional vector, multi-head attention processing is applied to the one-dimensional vector, the processed sequence is residually connected with the original one-dimensional vector and normalized, nonlinear learning and normalization are performed through a feedforward network, and finally the normalized one-dimensional vector is converted back into a two-dimensional image; The formula is: Q=K=V=Flatten(Input) Output=Reshape(FFN(MultiHead(Q, K, V))) Among them: MultiHead is the multi-head attention function; FFN is the feedforward network.

5. The strip surface defect detection method based on the RAC-YOLO algorithm according to claim 1, characterized in that: The contextual anchor attention feature extraction fusion module RepNCSPELAN4_CAA is a network module formed by integrating the feature extraction module RepNCSPELAN4 with the contextual anchor attention mechanism CAAttention; the RepNCSPELAN4 combines two neural network structures CSPNet and ELAN, uses the segmentation and reorganization of CSPNet, and introduces the hierarchical convolution of ELAN in each part; the two parts of the segmented feature map are respectively subjected to different convolution processing and recombined into a new feature map, and then input into the CAAttention module for contextual anchor attention processing, and then the final output result is obtained through the last layer of convolution.

6. The method for detecting surface defects of strip steel based on the RAC-YOLO algorithm according to claim 1, characterized in that: The lightweight upsampling operator CARAFE includes a kernel prediction module and a content-aware reorganization module; The kernel prediction module consists of three submodules: channel compressor, content encoder and kernel normalizer; first, the channel compression submodule is used to compress the input feature channel, secondly, the content encoder submodule encodes the content of the compressed feature map to generate a reconstructed kernel, and finally, the kernel normalizer submodule applies a softmax function to each reconstructed kernel.

7. The strip steel surface defect detection method based on the RAC-YOLO algorithm according to claim 1, characterized in that: The MPDIoU loss function uses the minimum point distance to directly predict the distance between the upper left corner and the lower right corner of the bounding box and the real box, thereby improving the efficiency and accuracy of bounding box regression; specifically: in: represents the bounding box coordinates of the prediction box, Represents the coordinates of the bounding box of the real box; d1 represents the distance between the upper left corner coordinates of the predicted box and the upper left corner coordinates of the real box, d2 represents the distance between the upper right corner coordinates of the predicted box and the upper right corner coordinates of the real box, w and h represent the width and height of the input image respectively.

Images