Scale sequence and triple feature coding fused steel coil end face defect detection method

Abstract

The invention relates to a scale sequence and triple feature coding fused steel coil end face defect detection method, and belongs to the technical field of quality detection. According to the method, by introducing a scale sequence feature fusion module and a triple feature coding module, the problems of missing detection and false detection in steel coil end face defect detection are effectively solved. And the scale sequence feature fusion module can be combined with global information from images with different proportions to capture different spatial scales of different sizes and shapes of different targets, so that the robustness of detection is improved. The triple feature coding module can capture feature information of large, medium and small sizes, and generates a more accurate segmentation image through splicing and convolution operations, thereby improving the segmentation accuracy. Experimental results show that the method is superior to an existing method in the aspects of steel coil end face instance segmentation accuracy, reasoning speed and the like, and an efficient and accurate solution is provided for steel coil end face defect detection.

Description

Technical Field

[0001] The invention belongs to the technical field of quality inspection and relates to a steel coil end surface defect detection method integrating a scale sequence and triple feature coding. Background Art

[0002] Traditional methods for detecting defects on the end face of steel coils mainly rely on artificial vision or rule-based methods, which have disadvantages such as low efficiency, poor accuracy, and susceptibility to subjective factors. With the rapid development of deep learning technology, target detection methods based on deep learning are gradually applied to the field of steel coil end face defect detection. Existing target detection methods based on deep learning are mainly divided into two categories:

[0003] Single-stage algorithms: such as YOLO (You Only Look Once) and SSD, have fast detection speed but relatively low accuracy and are prone to missed detection and false detection.

[0004] Two-stage algorithm: such as MaskR-CNN, FasterR-CNN, etc., which has high detection accuracy but slow speed and cannot meet the needs of real-time detection.

[0005] The defects on the end surface of steel coils are usually small target defects, such as scratches, center cracks, folds, tail cracks, etc. In the feature learning process of deep learning, small target defects are easily ignored, resulting in missed detection and false detection. The existing target detection methods based on deep learning are difficult to effectively solve this problem. Therefore, a method that can effectively detect small target defects on the end surface of steel coils is urgently needed. Summary of the invention

[0006] In view of this, the purpose of the present invention is to provide a steel coil end surface defect detection method that integrates scale sequence and triple feature encoding, effectively improving the accuracy of steel coil end surface defect detection and reducing missed detection and false detection. The key to the present invention is that we added two new modules to the neck network of the original YOLOv8 detection and segmentation model: one is the scale sequence fusion module (scalesequencefeaturefusionmodule), which combines the global information of multiple scale images to capture different spatial scales of various sizes and shapes of different targets; the other is the triple feature encoding module (triplefeatureencodingmodule), which splices three different sizes of features, large, medium and small, in the spatial dimension to capture detailed information of small targets.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] A method for detecting defects on the end surface of a steel coil by integrating scale sequence and triple feature coding comprises the following steps:

[0009] Input the end surface image of the steel coil to be inspected;

[0010] Use the pre-trained backbone network to extract features from the image and obtain feature maps of different scales, including P3, P4 and P5; among them, P3 is a feature map in the feature pyramid, which is a feature map extracted from the third feature layer in the YOLO backbone network. The feature map corresponding to P3 is a feature map of the original input image downsampled 8 times, P4 is a feature map extracted from the fourth feature layer in the backbone network, which is 16 times the size of the original input image downsampled, and P5 is a feature map extracted from the fourth feature layer in the backbone network, which is 32 times the size of the original input image downsampled;

[0011] Adjust the channels and dimensions of the feature information of the three layers P3, P4 and P5 so that they have the same number of channels and dimensions;

[0012] The adjusted P3, P4 and P5 feature maps are stacked to form a 3D feature map;

[0013] The 3D feature map is processed using 3D convolution, 3D batch normalization and SiLU activation function operations to obtain scale sequence features;

[0014] Extract features of three sizes, large, medium and small, from two sets of feature maps respectively; the two sets of feature maps are P2, P3 and P4 and P3, P4 and P5 respectively; P2 is a feature map in the feature pyramid, which comes from the second layer in the backbone network and is a feature map of the original input image downsampled 4 times;

[0015] Adjust the channels of the feature maps of the three sizes to have the same number of channels;

[0016] The feature maps of three sizes are concatenated to form a triple feature map;

[0017] The triple feature map is processed using a convolution operation to obtain the final detection and segmentation result.

[0018] Furthermore, the scale sequence feature fusion module includes:

[0019] Channel adjustment module, adjusts the number of channels of P4 and P5 feature levels to 256;

[0020] The dimension adjustment module uses the Unsqueeze method to change the P3, P4, and P5 feature maps from 3D tensors [height, width, channels] to 4D tensors [depth, height, width, channels];

[0021] A stacking module is used to stack the adjusted feature maps along the depth dimension to form a 3D feature map;

[0022] 3D convolution module, used to perform convolution operations on 3D feature maps;

[0023] 3D batch normalization module, used to perform batch normalization on the output of the 3D convolution module;

[0024] The SiLU activation function module is used to perform activation function processing on the output of the 3D batch normalization module.

[0025] Furthermore, the triple feature encoding module includes:

[0026] The large-size feature processing module includes: a channel adjustment module for adjusting the number of channels of the large-size feature map to the same value as the main-scale feature map; a downsampling module for downsampling the large-size feature map;

[0027] The medium-size feature processing module includes: a convolution processing module that does not adjust the number of channels;

[0028] The small-size feature processing module includes: a channel adjustment module for adjusting the number of channels of the small-size feature map to the same value as that of the main-scale feature map; an upsampling module for performing an upsampling operation on the small-size feature map;

[0029] A splicing module is used to splice large, medium and small size feature maps to form a triple feature map;

[0030] Convolution module, used to perform convolution operations on triple feature maps.

[0031] Furthermore, the input of the scale sequence feature fusion module is expressed by the following formula:

[0032] F0(w,h)=G0(w,h)×f(w,h)

[0033]

[0034] Where f(w,h) refers to the 2D input image with width w and height h, F0(w,h) represents the image generated by smoothing a series of convolutions using a 2D Gaussian filter G0(w,h); σ is a scaling parameter for the standard deviation of the 2D Gaussian filter used for convolution.

[0035] Furthermore, in the triple feature encoding module, before feature encoding, the number of feature channels is adjusted to be consistent with the main scale feature;

[0036] For large-size feature maps, they are first processed through a convolution module to adjust the number of channels to 1C, and then maximum pooling and average pooling are used in the downsampling process to retain the high-resolution features of the surface defects of the steel coil;

[0037] For small-size feature maps, the convolution module is first used to adjust the number of channels to 1C, and then the “nearest neighbor interpolation method” is used for downsampling to maintain the richness of local features of low-resolution images and prevent the loss of small target feature information;

[0038] Convolution is performed on large, medium and small feature maps of the same size respectively, and concatenated in the channel dimension. This process is expressed by the following formula:

[0039] F zoom_cat =Concat(F l ,F m ,F s )

[0040] Among them, F zoom_cat It represents the output feature map of the triple feature encoding module, F l , F m and F s Represents large, medium and small size feature maps respectively; F zoom_cat By F l , F m and F s Splicing is generated, Concat means splicing.

[0041] Furthermore, the backbone network is a YOLOv8 network.

[0042] Furthermore, the detection and segmentation result includes the category, position and segmentation mask of the defect target.

[0043] The beneficial effects of the present invention are:

[0044] (1) The scale sequence feature fusion module can capture different spatial scales of different targets of various sizes and shapes, thereby improving the robustness of detection. The triple feature encoding module can capture feature information of three different sizes: large, medium, and small, and generate a more accurate segmentation map through splicing and convolution operations, thereby improving the accuracy of segmentation. Experimental results show that the present invention can significantly improve the accuracy of small target defect segmentation and effectively reduce missed detection and false detection.

[0045] (2) The scale sequence feature fusion module can combine global information from images of different scales, thereby improving the robustness of detection to complex scenes such as illumination and occlusion. The triple feature encoding module can capture feature information of different scales, thereby improving the robustness of detection to complex situations such as changes in target size and shape.

[0046] (3) The present invention is based on the YOLOv8 network, which can maintain a high detection speed while ensuring detection accuracy, meeting the needs of real-time detection.

[0047] (4) The method of the present invention is simple, easy to deploy and apply, and can be widely used in the field of steel coil end surface defect detection.

[0048] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0050] Figure 1 It is the overall structure of the ST-YOLOv8 model;

[0051] Figure 2 It is the scale sequence feature fusion module;

[0052] Figure 3 It is a triple feature encoding module. DETAILED DESCRIPTION

[0053] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0054] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0055] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0056] 1. Overall architecture

[0057] Figure 1 The overall architecture of the proposed ST-YOLOv8 model is shown in Figure 1. In order to study a model that can be used for defect segmentation detection on the surface of steel coils, we developed a new feature fusion network, which mainly consists of two parts: (1) scale sequence feature fusion module, which can combine global information or high-level semantic information from images of various scales. Here, feature maps of the three levels P3, P4, and P5 are selected because the feature maps of these levels have low resolution, but large receptive fields and contain more semantic information. The original input image size is 640*640, P1 is the feature map obtained by further convolution and downsampling the original input image by 2 times, and the feature map size is 1 / 2 of the original image, that is, 320*320; P2 is the feature map obtained by further convolution and downsampling P1 by 2 times, and the feature map size is 1 / 4 of the original image, that is, 160*160; P3 is the feature map obtained by further convolution and downsampling P2 by 2 times, and the feature map size is 1 / 8 of the original image, that is, 80*80; P4 is the feature map obtained by further convolution and downsampling P3 by 2 times, and the feature map size is 1 / 16 of the original image, that is, 40*40; P5 is the feature map obtained by further convolution and downsampling P4 by 2 times, and the feature map size is 1 / 32 of the original image, that is, 20*20. The scale sequence feature fusion module reshapes the feature information of the three layers P3, P4 and P5 to the same size, and then stacks them together as the input of the 3D convolution, and then obtains an output through batch normalization, SiLU activation function and other methods. (2) Triple feature encoding module: This module can capture more detailed image information. It can capture the feature information of large, medium and small sizes of the target, and use the local feature information and global features of the small target to generate a more accurate segmentation map.

[0058] Figure 1The overall structure of the ST-YOLOv8 model. ScalSeq is the scale sequence feature fusion module; Zoom_cat is the triple feature encoding module. The C2f and Concat modules are from YOLOv8.

[0059] 2. Scale sequence feature fusion module

[0060] At present, there are relatively few studies on surface defect detection of steel coils, and the existing research is mostly based on the application of basic models. The new scale sequence fusion module we proposed can combine the high-dimensional information of the deep feature map with the detailed information of the shallow feature map. During image downsampling, the size of the image will change, but the scale will not change. The scale space is established along the scale axis, which not only represents a specific scale, but also covers a variety of scale ranges that objects may have. Scale refers to the details in the image. Blurred images may lose these details, but can still retain the structural characteristics of the image. The input of the scale sequence feature fusion model can be expressed by the following formula:

[0061] F0(w,h)=C0(w,h)×f(w,h)

[0062]

[0063] Where f(w,h) is the 2D input image with width w(weight) and height h(height), F0(w,h) is the image generated by smoothing a series of convolutions using a 2D Gaussian filter G0(w,h). σ is a scaling parameter for the standard deviation of the 2D Gaussian filter used for convolution.

[0064] These generated images have the same resolution and different scales. Therefore, feature maps of different sizes can be regarded as different scale spaces. We stack feature maps of different scales horizontally and then use 3D convolution to extract scale sequence features. Because the high-resolution feature map P3 level contains more key detection and segmentation information for small targets, our design is based on the P3 level. Figure 2 As shown in the figure, a convolution is first used to change the number of channels of the P3 and P4 feature levels to 256, and then the "nearest neighbor interpolation method" is used to change its size to the size of the P3 level. The unsqueeze method is used to change the dimension of each feature layer from 3D [height, width, channel] to 4D [depth, height, width, channel], and then the 4D feature maps are concatenated along the depth dimension to form a 3D feature map. Finally, this 3D feature map will undergo a series of operations such as 3D convolution, 3D batch normalization, and SiLU activation function to complete the extraction of scale sequence features.

[0065] Figure 2 is the scale sequence feature fusion module. BN is BatchNormalization, and SiLU is the SiLU activation function.

[0066] 3. Triple Feature Encoding Module

[0067] In the defect detection on the surface of steel coils, most of them are small, and there are also some dense targets. We can compare the shape and appearance changes of different scales by enlarging the image. Since different feature layers of the backbone network have different sizes, the traditional FPN network fusion mechanism only upsamples the small-sized feature map and then separates or adds it to the previous feature layer. This approach ignores the large amount of detailed information contained in the large-sized feature map. Therefore, we propose a triple feature encoding module, which separates the features of large, medium and small-sized feature maps, adds large-sized feature maps, and enlarges the features to improve the detailed feature information.

[0068] like Figure 3 As shown in the figure, the entire structure of the triple feature encoding module is shown. For the input of the two triple feature encoding modules, we have selected two groups of inputs (P2, P3, P4) and (P3, P4, P5) here, and their large, medium, and small are relative to the large, medium, and small of each group of input feature maps. Before feature encoding, the number of feature channels is first adjusted so that it can be consistent with the main scale feature. As can be seen from the figure, for the large-size feature map, it first passes through the convolution module to adjust its channel number to 1C, and then uses the maximum pooling and average pooling in the downsampling process, which helps to retain the high-resolution features of the surface defects of the steel coil; for the small-size feature map, the convolution module is also used to adjust its channel number to 1C, and then the "nearest neighbor interpolation method" is used for downsampling, which helps to maintain the richness of the local features of the low-resolution image and prevent the loss of small target feature information; for the medium-size feature map, only one convolution module is performed on it to keep its channel number unchanged. Finally, the large, medium and small feature maps of the same size are convolved respectively, and then spliced ​​in the channel dimension. This process can be expressed by the following formula:

[0069] F zoom_cat =Concat(F l ,F m ,F s )

[0070] Among them, F zoom_cat It represents the output feature map of the triple feature encoding module, F l , F m and F s Represents large, medium and small size feature maps respectively. zoom_cat By F l , Fm and F s Concatenation is produced.

[0071] Figure 3 is a triple feature encoding module, where C represents the number of channels and S represents the feature map size. Each triple feature encoding uses feature maps of different sizes as input.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A steel coil end surface defect detection method integrating scale sequence and triple feature coding, Features: The method comprises the following steps: Input the end surface image of the steel coil to be inspected; Use the pre-trained backbone network to extract features from the image and obtain feature maps of different scales, including P3, P4 and P5; among them, P3 is a feature map in the feature pyramid, which is a feature map extracted from the third feature layer in the YOLO backbone network. The feature map corresponding to P3 is a feature map of the original input image downsampled 8 times, P4 is a feature map extracted from the fourth feature layer in the backbone network, which is 16 times the size of the original input image downsampled, and P5 is a feature map extracted from the fourth feature layer in the backbone network, which is 32 times the size of the original input image downsampled; Adjust the channels and dimensions of the feature information of the three layers P3, P4 and P5 so that they have the same number of channels and dimensions; The adjusted P3, P4 and P5 feature maps are stacked to form a 3D feature map; The 3D feature map is processed using 3D convolution, 3D batch normalization and SiLU activation function operations to obtain scale sequence features; Extract features of three sizes, large, medium and small, from two sets of feature maps respectively; the two sets of feature maps are P2, P3 and P4 and P3, P4 and P5 respectively; P2 is a feature map in the feature pyramid, which comes from the second layer in the backbone network and is a feature map of the original input image downsampled 4 times; Adjust the channels of the feature maps of the three sizes to have the same number of channels; The feature maps of three sizes are concatenated to form a triple feature map; The triple feature map is processed using a convolution operation to obtain the final detection and segmentation result.

2. According to claim 1, a steel coil end surface defect detection method integrating scale sequence and triple feature coding, Features: The scale sequence feature fusion module includes: Channel adjustment module, adjusts the number of channels of P4 and P5 feature levels to 256; The dimension adjustment module uses the Unsqueeze method to change the P3, P4, and P5 feature maps from 3D tensors [height, width, channels] to 4D tensors [depth, height, width, channels]; A stacking module is used to stack the adjusted feature maps along the depth dimension to form a 3D feature map; 3D convolution module, used to perform convolution operations on 3D feature maps; 3D batch normalization module, used to perform batch normalization on the output of the 3D convolution module; The SiLU activation function module is used to perform activation function processing on the output of the 3D batch normalization module.

3. According to the method for detecting defects on the end surface of a steel coil by integrating scale sequence and triple feature coding as described in claim 1, Features: The triple feature encoding module includes: The large-size feature processing module includes: a channel adjustment module for adjusting the number of channels of the large-size feature map to the same value as the main-scale feature map; a downsampling module for downsampling the large-size feature map; The medium-size feature processing module includes: a convolution processing module that does not adjust the number of channels; The small-size feature processing module includes: a channel adjustment module for adjusting the number of channels of the small-size feature map to the same value as that of the main-scale feature map; an upsampling module for performing an upsampling operation on the small-size feature map; A splicing module is used to splice large, medium and small size feature maps to form a triple feature map; Convolution module, used to perform convolution operations on triple feature maps.

4. According to claim 2, a method for detecting defects on the end surface of a steel coil by integrating a scale sequence and triple feature coding, Features: The input of the scale sequence feature fusion module is expressed by the following formula: F 0 (w,h)=G 0 (w,h)×f(w,h) Where f(w,h) refers to a 2D input image with a width of w and a height of h, and F 0 (w, h) represents the sum of the two dimensions by using a 2D Gaussian filter G 0 (w, h) The image produced by smoothing a series of convolutions; σ is the scaling parameter for the standard deviation of the 2D Gaussian filter used for the convolution.

5. According to claim 3, a method for detecting defects on the end surface of a steel coil by integrating scale sequence and triple feature coding, Features: In the triple feature encoding module, before feature encoding, the number of feature channels is adjusted to be consistent with the main scale feature; For large-size feature maps, they are first processed through a convolution module to adjust the number of channels to 1C, and then maximum pooling and average pooling are used in the downsampling process to retain the high-resolution features of the surface defects of the steel coil; For small-size feature maps, the convolution module is first used to adjust the number of channels to 1C, and then the "nearest neighbor interpolation method" is used for downsampling to maintain the richness of local features of low-resolution images and prevent the loss of small target feature information; Convolution is performed on large, medium and small feature maps of the same size respectively, and concatenated in the channel dimension. This process is expressed by the following formula: F zoom_cat =Concat(F l ,F m ,F s ) Among them, F zoom_cat It represents the output feature map of the triple feature encoding module, F l , F m and F s Represents large, medium and small size feature maps respectively; F zoom_cat By F l , F m and F s Splicing is generated, Concat means splicing.

6. A method for detecting defects on the end surface of a steel coil by integrating a scale sequence and triple feature coding according to any one of claims 1 to 3, Features: The backbone network is a YOLOv8 network.

7. A method for detecting defects on the end surface of a steel coil by integrating a scale sequence and triple feature coding according to any one of claims 1 to 3, Features: The detection and segmentation results include the category, position and segmentation mask of the defect target.

Images