Strip steel surface defect area size measuring method

Abstract

A strip steel surface defect area dimension measurement method relates to a defect dimension measurement method, and comprises the following steps: calibrating a camera coordinate system, and shooting a strip steel surface image by using a camera; identifying defects on the surface of the strip steel in the image by using a defect identification model, dividing a defective area in the image, measuring the size of the defective area, and calculating the actual size of the defective area according to the coordinate conversion relation; the defect identification model is based on YOLOv8s, a linear focusing attention mechanism is fused into a C2F module of a backbone network to form a C2F-FLA module, global features of an image are concerned, and more defect related features are captured; a bidirectional weighted feature pyramid network structure is adopted in the Neck network part, and a wavelet convolution module is utilized to respond to low-frequency information in the image; and performing down-sampling by adopting a Haar wavelet down-sampling module. The feature extraction capability is improved while the model parameter quantity, the large calculation amount and the calculation complexity are reduced, so that the detection result is more accurate.

Description

Technical Field

[0001] The invention relates to a defect size measuring method, in particular to a method for measuring the size of a defect area on a strip steel surface. Background Art

[0002] During the production process, the surface of the steel strip is prone to defects such as plaques, scratches, inclusions, pitting, etc. These defects will affect the intrinsic properties of the steel strip to a certain extent, such as reducing the fatigue strength, wear resistance and corrosion resistance of the steel strip. On the one hand, detecting the surface defects of the steel strip and identifying the defective locations are crucial to ensure high-quality steel production. On the other hand, measuring the size of the defective area and determining the proportion of the defective area in the produced steel strip is of great significance for production management and adjustment of process parameters.

[0003] High-precision defect detection algorithms are required to detect surface defects of steel strips and measure the size of defective areas. Deep learning not only solves the complexity and uncertainty of manual feature extraction in traditional methods, but also continuously transforms the underlying basic features of the image into more abstract semantic features, making weight learning more efficient. In deep learning algorithms, commonly used defect detection models include the YOLO series, such as YOLOv7 and YOLOv8. However, when it comes to surface defect detection of steel strips, there are problems with large number of parameters, large amount of calculation, and high complexity, which has not yet met the lightweight requirements in actual application scenarios. Moreover, it does not pay enough attention to the global features and partial information of the image, resulting in low detection accuracy. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the defects of large number of detection model parameters, large amount of calculation and low detection accuracy, and provide a method for measuring the size of defective area on the surface of strip steel.

[0005] The technical solution adopted by the present invention to solve the above technical problems is: a method for measuring the size of the defective area on the surface of a steel strip, calibrating the camera coordinate system, and taking an image of the surface of the steel strip with a camera; using a trained defect recognition model to identify the defects on the surface of the steel strip in the image, and dividing the defective area in the image, measuring the size of the defective area in the image, and calculating the actual size of the defective area based on the coordinate transformation relationship.

[0006] The defect recognition model is based on YOLOv8s, and the feature information in the input image is extracted through the backbone network of YOLOv8s, and is processed and fused through the Neck network part and then output through the head network; wherein, the linear focus attention mechanism is integrated into the C2F module of the backbone network to form a C2F-FLA module, which is used to extract features of the input image; a bidirectional weighted feature pyramid network structure is adopted in the Neck network part, and a wavelet convolution module is adopted in the bidirectional weighted feature pyramid network structure to replace the C2F module of the Neck network part in YOLOv8s, and the wavelet convolution module is used to respond to the low-frequency information in the image; the Haar wavelet downsampling module is adopted in the backbone network and the Neck network part for downsampling.

[0007] In the C2F-FLA module, its input is first processed by a convolution layer and then divided into two parts, one of which is processed by multiple FLABottleNeck modules that incorporate a linear focus attention mechanism and then spliced ​​with the other part in the channel dimension, and then output after being processed by a convolution layer; wherein, the FLABottleNeck module first processes its input through a convolution layer to make its number of channels half of the original, and then restores the number of channels through a convolution layer, and then uses the linear focus attention module to calculate the global weight, and the result is spliced ​​with the input of the FLABottleNeck module to obtain the output.

[0008] The kernel size of the convolutional layer in the FLABottleNeck module is 3×3.

[0009] The linear focus attention module passes its input tensor through a one-dimensional convolutional layer W Q , W K , W V The tensors Q, K, and V obtained by the operation are processed using the kernel function, and the similarity between Q, K, and V is calculated to obtain the weight matrix of the image; then V, which has undergone a layer of deep convolution processing, is superimposed with the previous result to obtain the output.

[0010] The wavelet convolution module divides its input into two paths for processing, one of which is subjected to wavelet decomposition and processed by a deep convolution layer to obtain a low-frequency and three high-frequency channels, each of which has a resolution of half of the original. The four channels are then subjected to inverse wavelet transformation to restore the original resolution size; the other path is processed using a deep convolution layer, and the processing result is superimposed element by element with the processing result of the first path to obtain the output of the wavelet convolution module.

[0011] The wavelet convolution module uses a first-order Daubechies wavelet to perform wavelet decomposition on the input.

[0012] The wavelet convolution module uses a 5×5 depth convolution layer for processing.

[0013] The Haar wavelet downsampling module performs wavelet decomposition on its input, concatenates a low-frequency part and three high-frequency parts, and then processes it through a feature representation learning block consisting of a 1×1 convolution layer, a batch normalization layer and a RelU activation function, and uses point-by-point convolution to adjust the number of channels of the feature map.

[0014] After the defects on the strip surface in the image are identified, two defect positions located at the outermost sides in the length direction of the strip are selected, and lines are drawn along the transverse direction of the strip with the two selected defect positions as base points to divide the defective area.

[0015] The beneficial effects of the present invention are as follows: a linear focus attention mechanism is integrated into the C2F module of the backbone network, a global dependency of image features is established, global features of the image are focused on, more defect-related features are captured, and more details are focused on, so that the backbone network of the model has a stronger feature extraction capability. A bidirectional weighted feature pyramid network structure is introduced in the Neck network part, and it is combined with wavelet convolution to reduce the model parameter amount and the amount of calculation, while adaptively fusing the features of each scale in the network, the ability of the model to capture the defect contour is improved, and the task of more accurately identifying the contours of defects of different scales is solved together. Haar wavelet downsampling is used to replace the ordinary convolution downsampling module, and the advantages of Haar wavelet multi-scale lossless fast signal decomposition are utilized, so that the model retains effective information as much as possible during the downsampling process, that is, while reducing the feature resolution, more defect-related feature information is retained, and the model parameter amount and amount of calculation are reduced at the same time. By improving various parts of YOLOv8s, while reducing the model parameter amount, large amount of calculation and computational complexity, the feature extraction capability is improved, and the detection result is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a structural schematic diagram of the defect recognition model of the present invention.

[0017] Figure 2 It is a schematic diagram of the structure of the C2F-FLA module of the present invention.

[0018] Figure 3 It is a schematic diagram of the wavelet convolution module processing process.

[0019] Figure 4 It is a schematic diagram of the processing process of the Haar wavelet downsampling module.

[0020] Figure 5 This is a schematic diagram of the processing process of the Merge module in the Neck network part.

[0021] Figure 6 It is a heat map comparison between C2F module and C2F-FLA.

[0022] Figure 7 It is a comparison of the test results of the model of the present invention and other models. DETAILED DESCRIPTION

[0023] The technical solution of the present invention is described clearly and completely below in conjunction with the accompanying drawings and specific implementation methods. The specific contents listed in the following embodiments are not limited to the technical features necessary for the technical problems to be solved by the technical solutions recorded in the claims. At the same time, the enumerated embodiments are only part of the present invention, not all embodiments.

[0024] The method for measuring the size of the defective area on the surface of the steel strip of the present invention uses a defect recognition model to identify the defects on the surface of the steel strip, and after dividing the defective area, the size of the defective area is determined by measurement and calculation. The image used for the defect recognition model detection is photographed by a camera, and the coordinate system is calibrated according to the camera parameters, installation position and distance from the surface of the steel strip before shooting. The specific method can refer to the prior art. After the camera calibration is completed, the surface image of the steel strip is photographed by the camera, and then the defects on the surface of the steel strip in the image are identified by the trained defect recognition model, and the defective area is divided in the image. The division method of the defective area is carried out according to specific needs. For example, the contour of the defective area is determined according to the outermost defective position in the image and the defective area is divided. This method is more accurate in area division, but the calculation complexity is high. This method can be used when it is necessary to obtain the area of ​​the defective area. Or after identifying the defects on the surface of the steel strip in the image, two defective positions located at the outermost sides in the length direction of the steel strip are selected, and the two selected defective positions are used as base points to draw lines along the transverse direction of the steel strip to divide the defective area. This division method is simple and easy to calculate. This method can be used when it is necessary to obtain the length of the defective part of the steel strip. In an image, a threshold can be set for the distance between adjacent defects. When the distance between two defects exceeds the threshold and there are no other defects between them, two or more defect areas can be divided in the image. After the defect area division is completed, the size of the defect area in the image is measured, and the actual size of the defect area is calculated based on the coordinate transformation relationship. The specific calculation method refers to the existing visual measurement technology, which is not the focus of the present invention and will not be described in detail.

[0025] like Figure 1As shown, the defect recognition model of the present invention is based on YOLOv8s, which includes a backbone network (Backbone), a Neck network part (Neck feature fusion network) and a head network (Head prediction layer). The backbone network is used to extract feature information from the input image, the Neck network part is used to process and fuse the extracted features of different dimensions, and the head network predicts and classifies the target position according to the output of the Neck network part and outputs the result.

[0026] like Figure 1 As shown, the present invention uses a C2F-FLA module in the backbone network to extract features from the input image, wherein the C2F-FLA module is formed by integrating a linear focused attention mechanism (Focused LinearAttention, FLA) into the C2F module of the original YOLOv8s, thereby establishing a global dependency of image features and improving feature extraction capabilities.

[0027] like Figure 2 As shown in the figure, in the C2F-FLA module, its input is first processed by a Conv convolution layer, and the processing result is split into two parts. One part is processed by multiple FLABottleNeck modules that incorporate the linear focus attention mechanism and then concatenated with the other part in the channel dimension, and then output after being processed by a Conv convolution layer. Among them, the FLABottleNeck module first processes its input through a Conv convolution layer with a kernel size of 3×3, so that its number of channels will be half of the original, and then restores the number of channels through a 3×3 Conv convolution layer, and then uses the linear focus attention module (FLA) to calculate the global weight, and the result is concatenated with the input of the FLABottleNeck module to obtain the output.

[0028] The calculation process formula of the linear focused attention module (FLA) is: Among them, O is the output; , , The input tensor x is passed through a one-dimensional convolutional layer W Q , W K , W V The tensor obtained by the operation; The h in it represents the number of attention heads of K, n represents the number of one-dimensional sequences formed by linear transformation after the image is divided into several image blocks, and c represents the number of channels of the attention head of K; DWC is depthwise convolution; is the kernel function, The T stands for transpose operation.

[0029] , where RELU is the activation function, is the focusing function.

[0030] ,in It represents the pth power of its element x, and the present invention sets p to 3.

[0031] The linear focus attention module passes its input tensor through a one-dimensional convolutional layer W Q , W K , W V The tensors Q, K, and V obtained by calculation use the kernel function When processing Q and K, the kernel function only changes the direction of the feature, without changing the modulus of the feature, thus increasing the similarity of related query-key pairs and reducing the similarity of unrelated query-key pairs, thereby improving the feature focusing ability of the module. Secondly, the similarity between Q, K, and V is calculated to obtain the weight matrix; finally, V, which has been processed by a layer of deep convolution, is superimposed with the previous result to obtain the final output. The deep convolution layer can restore the rank of the feature matrix to a higher number, alleviate the phenomenon of feature homogeneity, and achieve the diversity of extracted features.

[0032] Since the scale of the surface defects of the strip steel varies greatly and the contour information is complex, it is difficult for the general convolutional neural network to accurately identify defects of different resolutions. In addition, the general convolutional layer pays more attention to the high-frequency information of the image and ignores some low-frequency information representing the contour features of the image. Figure 1 As shown in the figure, a bidirectional weighted feature pyramid network structure (BiFPN) is used in the Neck network part to replace the original PAFPN structure of YOLOv8s. The importance of different input features is learned by introducing learnable weights, and multi-scale feature fusion from top to bottom and bottom to top is repeatedly applied to adapt to the large scale changes of surface defects of strip steel. At the same time, a wavelet convolution module (WTConv) is used in the BiFPN structure to replace the C2F module of the Neck network part in the original YOLOv8s, forming the overall structure of WT-BiFPN in the Neck network part. The wavelet convolution module is used to decompose and process its input, so that the model can better respond to the low-frequency information in the image, thereby more accurately identifying the contours of defects of different scales, improving the model's ability to capture defect contours, and reducing the number of parameters and calculations of the model.

[0033] like Figure 1 As shown in the figure, in the WT-BiFPN structure of the Neck network part, the Merge module is used for data fusion. The calculation process formula of the Merge module is: ; Among them, O is the output, is the input layer to be fused, , are the learnable weights corresponding to the input layer. Depending on the number of output layers, the maximum values ​​of i and j are 2 or 3.

[0034] The Merge module first uses the Relu activation function to process the weights so that they are greater than or equal to 0. Then the learnable weights are normalized to obtain a set of weights ranging from 0 to 1. Finally, this set of weights is multiplied by the corresponding feature layer to be fused, and the sum is accumulated. The dimension of the final output remains unchanged. Figure 5 As shown, the input end is two or three feature layers to be fused, and different small cubes represent different learnable weights.

[0035] The calculation process formula of the wavelet convolution module (WTConv) is: ; Among them, O is the output, X is the input, WT is the Daubechies 1 wavelet transform, IWT is the inverse wavelet transform achieved by transposed convolution, and DWC is the depthwise convolution. The depthwise convolution layer uses a 5×5 depthwise convolution, which has a larger receptive field and higher computational efficiency than a 3×3 ordinary convolution.

[0036] like Figure 3 As shown in the figure, the calculation of the wavelet convolution module is carried out in two steps. In the first step, the input is decomposed by wavelet (WT) using the first-order Daubechies wavelet, and processed by the deep convolution layer DWC to obtain a low-frequency and three high-frequency channels. The resolution of each channel is half of the original. Then, these four parts are inversely transformed (IWT) to restore the original resolution size. In the second step, the input is processed by the deep convolution layer DWC, and then element-by-element superposition with the result of the first step is obtained to obtain the output.

[0037] The surface defects of steel strips, such as rolled scale (RS) and pitted surface (PS), have fine feature granularity, so that a large amount of information will be lost in the general target detection model after layers of feature extraction and spatial transformation, resulting in poor detection effect. This problem is particularly obvious in the downsampling layer. The present invention uses the Haar wavelet downsampling module HWD to replace the original ordinary convolution downsampling module of the backbone network and the Neck network. This module utilizes the advantages of Haar wavelet multi-scale lossless fast signal decomposition, so that the model retains effective information as much as possible during the downsampling process, reduces the loss of surface defect information of steel strips, that is, while reducing the feature resolution, it retains more defect-related feature information, and at the same time reduces the number of parameters and the amount of calculation of the model.

[0038] The calculation process of wavelet downsampling module HWD is as follows Figure 4 As shown in the figure, the first is the lossless feature encoding block, which is responsible for feature conversion and spatial resolution reduction. The input is decomposed by Haar wavelet to obtain a low-frequency part LL and three high-frequency parts LH, HL, and HH. These four parts are spliced ​​together to reduce the resolution of the feature map to half of the original, while the number of channels becomes four times the original. This process encodes part of the information in the spatial dimension into the channel dimension without information loss. Then there is the feature representation learning block, which consists of a standard 1 × 1 convolution layer, a batch normalization layer, and a ReLU activation function. In this block, point-by-point convolution is used to adjust the number of channels of the feature map to make it consistent with the original input, ensuring computational efficiency. This block has two main functions: one is to adjust the number of channels of the feature map to align it with the subsequent layers; the other is to filter as much redundant information as possible so that the subsequent layers can learn representative features more effectively.

[0039] The present invention uses the public dataset NEU-DET to train and test the model. The NEU-DET dataset contains a total of six types of steel surface defects, including patches (Pa), scratches (Sc), cracks (Cr), inclusions (In), pits (PS) and rolled-in-scale (RS). There are 300 images of each defect, a total of 1800 images, and the image resolution is 200×200. The training set and the validation set are divided according to a ratio of 9:1, that is, 1620 training sets and 180 validation sets.

[0040] The evaluation indicators used are mean average precision (mAP), number of parameters (Params), amount of computation (GFLOPs), and model weight file size (size). The IOU threshold is set to 0.5. When the intersection of the detection box and the ground truth box is greater than 0.5, the prediction is considered correct. The following table shows the ablation experiment conducted on the NEU-DET dataset, with a total of 8 groups of experiments, each with different modules added. The first group is the YOLOv8s baseline model without any improvement strategy, and the last group is the final model proposed in this paper.

[0041] Among them, Cr, In, Pa, Ps, Rs, and Sr represent defect types crazing, inclusion, patches, pitted-surface, rolled-in-scale, and scratches, respectively.

[0042] From the comparison between the first and second groups, the third and fourth groups, and the fifth and eighth groups, it can be seen that the introduction of the C2F-FLA module increased the mAP by 1.07%, 0.26% and 1.04% respectively, proving that the C2F-FLA module effectively improved the feature extraction capability of the model backbone network. From the comparison between the first and third groups, it can be seen that the introduction of the WT-BiFPN module increased the mAP by 2.54%, and the mAP of Crazing increased by 23.46%, proving that the WT-BiFPN module improved the detection capability of defects with large scale changes such as Crazing, that is, it effectively improved the multi-scale fusion capability of the model and the ability to capture defect contour information. From the comparison between the third and fifth groups, and the fourth and eighth groups, it can be seen that the introduction of the HWD module increased the mAP by 0.521% and 1.30% respectively, proving that the HWD module can retain more effective information of defects and improve the accuracy of the model in detecting defects. From the comparison between the first group and the eighth group, it can be seen that the overall mAP has increased by 4.14%, among which the mAP of Crazing has increased by 24.30%, and the mAP of Inclusion has increased by 6.52%, proving that the method of the present invention has improved the problem of low accuracy in strip surface defect detection. At the same time, compared with the benchmark model, the method of the present invention has reduced the number of parameters by 44.44%, the amount of calculation by 32.75%, and the model volume by 40.93%, achieving lightweight, reducing the model deployment cost, and facilitating practical applications in industrial scenarios.

[0043] The model of the present invention is compared with the classic target detection algorithm and an excellent algorithm for strip surface defect detection, and the results are shown in the following table.

[0044] From the results in the table, it can be seen that in the two-stage target detector, the parameter amount and computation amount of Faster R-CNN are too large, which is not suitable for practical application in industrial scenarios. In the one-stage target detector, SSD and RetinaNet perform worse than the YOLO series in strip surface defect detection. In the YOLO series, the parameter amount and computation amount of v3 and v7 are relatively large, and the detection effect is not ideal; v5s, v8s and v9s are all lightweight versions, and their detection accuracy increases with the increase of parameter amount and computation amount. SRN-YOLO and SCRL-EMD-FR are improvements made by previous researchers on the problem of strip surface defect detection based on the YOLO algorithm. It can be seen that both have achieved very good results in detection accuracy. The detection accuracy of the method of the present invention is second only to SRN-YOLO, and the mAP (%) is 3.2 lower than that of SRN-YOLO, but the parameter amount is only 11.5% of it, and the computation amount is only 15.3% of it. It can be seen that the method of the present invention realizes the organic combination of detection accuracy and lightweight, and achieves more efficient detection.

[0045] Figure 6 The heat map comparison of the features captured by the C2F and C2F-FLA modules on three pictures is shown. It can be seen that the C2F-FLA module proposed in the present invention can capture more defect-related features, pay attention to more details, and make the backbone network of the model have stronger feature extraction capabilities.

[0046] Figure 7 The detection results of the benchmark model YOLOv8s and the model of the present invention on three pictures are shown. The first column in the figure is the true label of the original input data of the three pictures, and the second and third columns are the detection results of the benchmark model YOLOv8s and the present invention on the three pictures. By comparison, it can be seen that YOLOv8s can detect most defects, but there are some missed detections and scale mismatches. The detection frame position of the method of the present invention is more accurate, the confidence of the detection target is higher, and better detection accuracy is achieved.

[0047] The above description of the specific implementation mode is only used to help understand the technical concept and core idea of ​​the present invention. Although the technical solution is described and illustrated using a specific preferred embodiment, it should not be understood as a limitation of the present invention itself. Those skilled in the art may make various changes in form and details without departing from the technical concept of the present invention. These easily conceived changes or substitutions should all be included in the protection scope of the present invention.

Claims

1. A method for measuring the size of defective areas on a strip surface, characterized in that: Calibrate the camera coordinate system and use the camera to take images of the strip surface; use the trained defect recognition model to identify the defects on the strip surface in the image, divide the defective area in the image, measure the size of the defective area in the image, and calculate the actual size of the defective area based on the coordinate transformation relationship; The defect recognition model is based on YOLOv8s, and extracts feature information in the input image through the backbone network of YOLOv8s, and outputs it through the head network after processing and fusion through the Neck network part; wherein, the linear focus attention mechanism is integrated into the C2F module of the backbone network to form a C2F-FLA module, which is used to extract features of the input image; a bidirectional weighted feature pyramid network structure is adopted in the Neck network part, and a wavelet convolution module is adopted in the bidirectional weighted feature pyramid network structure to replace the C2F module of the Neck network part in YOLOv8s, and the wavelet convolution module is used to decompose and process its input to respond to the low-frequency information in the image; the Haar wavelet downsampling module is adopted in the backbone network and the Neck network part for downsampling.

2. A method for measuring the size of a strip surface defect area according to claim 1, characterized in that: In the C2F-FLA module, its input is first processed by a convolution layer and then divided into two parts, one of which is processed by multiple FLABottleNeck modules that incorporate a linear focus attention mechanism and then spliced ​​with the other part in the channel dimension, and then output after being processed by a convolution layer; wherein, the FLABottleNeck module first processes its input through a convolution layer to make its number of channels half of the original, and then restores the number of channels through a convolution layer, and then uses the linear focus attention module to calculate the global weight, and the result is spliced ​​with the input of the FLABottleNeck module to obtain the output.

3. A method for measuring the size of a strip surface defect area according to claim 2, characterized in that: The kernel size of the convolutional layer in the FLABottleNeck module is 3×3.

4. A method for measuring the size of a strip surface defect area according to claim 2, characterized in that: The linear focus attention module passes its input tensor through a one-dimensional convolutional layer W Q , W K , W V The tensors Q, K, and V obtained by the operation are processed using the kernel function, and the similarity between Q, K, and V is calculated to obtain the weight matrix of the image; then V, which has undergone a layer of deep convolution processing, is superimposed with the previous result to obtain the output.

5. The method for measuring the size of a strip steel surface defect area according to claim 1, characterized in that: The wavelet convolution module divides its input into two paths for processing, one of which is subjected to wavelet decomposition and processed by a deep convolution layer to obtain a low-frequency and three high-frequency channels, each of which has a resolution of half of the original. The four channels are then subjected to inverse wavelet transformation to restore the original resolution size; the other path is processed using a deep convolution layer, and the processing result is superimposed element by element with the processing result of the first path to obtain the output of the wavelet convolution module.

6. A method for measuring the size of a strip steel surface defect area according to claim 5, characterized in that: The wavelet convolution module uses a first-order Daubechies wavelet to perform wavelet decomposition on the input.

7. A method for measuring the size of a strip steel surface defect area according to claim 5, characterized in that: The wavelet convolution module uses a 5×5 depth convolution layer for processing.

8. A method for measuring the size of a strip surface defect area according to claim 1, characterized in that: The Haar wavelet downsampling module performs wavelet decomposition on its input, concatenates a low-frequency part and three high-frequency parts, and then processes it through a feature representation learning block consisting of a 1×1 convolution layer, a batch normalization layer and a RelU activation function, and uses point-by-point convolution to adjust the number of channels of the feature map.

9. A method for measuring the size of a strip surface defect area according to claim 1, characterized in that: After the defects on the strip surface in the image are identified, two defect positions located at the outermost sides in the length direction of the strip are selected, and lines are drawn along the transverse direction of the strip with the two selected defect positions as base points to divide the defective area.

Images