A strip steel defect prediction method based on salient object detection

Abstract

The application discloses a strip steel defect prediction method based on salient object detection, which comprises the following steps: S1, five layers of features in a picture are extracted by an encoder composed of five encoder layers, and are respectively denoted as h1, h2, h3, h4 and h5; S2, a bridging layer is used to connect the encoder, and hbg is output through the bridging layer; S3, decoding is performed by using a decoder module, the decoder module comprises a decoder and five FEE modules, the decoder is composed of five decoder layers, the output h5 of the encoder is input into the corresponding fifth FEE module, the h4 and the output of the fifth layer of the decoder and the hbg are input into the upper FEE module, and the output of the upper FEE module is used as the input of the upper decoder; and S4, the output of the uppermost decoder layer is output into 1 through a 1*1 convolution, and the channel is output into 1, that is, the final output result. The method based on the salient object detection method can automatically identify and locate potential defect areas, and reduces the demand for manual operation.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically relating to a method for predicting defects in strip steel based on salient target detection. Background Technology

[0002] Currently, traditional methods for detecting surface defects in strip steel typically rely on manual operation or relatively simple image processing techniques. However, with industrial development, these methods exhibit increasingly significant limitations:

[0003] Subjectivity: Manual operation is easily affected by subjective factors. Different operators may have different judgment criteria, leading to inconsistent results.

[0004] Inefficient: Manual inspection of large-scale steel production is often very time-consuming, especially on high-volume production lines.

[0005] Limited accuracy: Traditional methods may not be able to accurately detect small or complex defects, and may easily miss potential problems.

[0006] Not suitable for large-scale production: With the continuous development of the steel industry and the continuous increase in production volume, traditional methods are difficult to meet the needs of rapid production.

[0007] Most cutting-edge saliency prediction models do not effectively integrate information from the encoder and decoder. Furthermore, the information fused through the interaction of the encoder and decoder can largely and accurately reconstruct the image to be segmented. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention proposes a method for predicting defects in strip steel based on salient target detection. This method can automatically identify and locate potential defect areas, reducing the need for manual operation.

[0009] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0010] A method for predicting defects in strip steel based on salient target detection includes the following steps:

[0011] S1. Extract five layers of features from the image using an encoder consisting of five encoder layers, denoted as h1, h2, h3, h4, and h5 respectively;

[0012] S2. Connect the encoder using a bridging layer, and output hbg through the bridging layer.

[0013] S2-1. Input the feature h5 output by the encoder into the bridging layer, and obtain multi-scale information through several convolutional branches in the bridging layer;

[0014] S2-2. Merge the multi-scale information and feature h5, then pass it through a 1*1 convolution, BN and ReLU, and use it as the final output, denoted as hbg;

[0015] S3. Decoding is performed using a decoder module, which includes a decoder and a 5-layer FEE Module. The decoder consists of 5 decoder layers.

[0016] The encoder output h5 is input into the corresponding 5th layer FEE Module. h4, the output of the 5th layer of the decoder, and hbg are upsampled and sent to the upper layer FEE Module. The output of the channel attention and spatial attention modules in the FEE Module is used as the input of the upper layer decoder. This process is repeated, and a total of 5 layers of FEE Modules and 5 layers of decoder modules are combined into a decoder module.

[0017] S4. The output of the top decoder layer is processed by a 1*1 convolution to output 1 for each channel, which is the final output result.

[0018] Preferably, all five encoder layers use a ResNet34 network.

[0019] As a preferred method, the feature extraction method of the 5-layer ResNet34 network is as follows: each layer passes through two 3*3 convolutions from the input, then through a channel attention module and a spatial attention module, and finally adds the input features to obtain the output value. The output of each layer is recorded as h1, h2, h3, h4, and h5 respectively.

[0020] Preferably, the bridging layer has four branching points in parallel: conv3*3 convolution dilation=1, padding=1; conv3*3 convolution dilation=2, padding=2; conv3*3 convolution dilation=3, padding=3; and conv3*3 convolution dilation=4, padding=4.

[0021] Preferably, in step S2, h5 is used as input and convolved at four bifurcations to obtain information at four scales.

[0022] Preferably, the FEE Module includes two network branches, one of which consists of 3x3 convolutions and 1x1 convolutions, and the other consists of upsampling and 1x1 convolutions.

[0023] Preferably, the FEE Module processing method is as follows: the output of the encoder layer is input to one network branch, and the output of the decoder layer is input to another network branch. The output of the encoder layer is subjected to a 3*3 convolution, upsampled with the output of the decoder layer, and then fused using concat. The output of the encoder layer is then subjected to a 1*1 convolution, and the output of the decoder layer is also subjected to a 1*1 convolution to ensure the number of channels is consistent. These convolutions are then passed to the channel attention and spatial attention modules. The outputs of the channel attention and spatial attention modules are then fused with hbg using concat and output as out.

[0024] This invention has the following characteristics and beneficial effects:

[0025] Methods based on salient target detection can automatically identify and locate potential defect areas, reducing the need for manual operation. This approach can process large numbers of steel images in a short time, improving production line efficiency. Deep learning models can learn complex features and patterns, thereby achieving accurate detection and classification of various types of defects.

[0026] The salient target detection-based method is applicable not only to defects of different sizes and shapes but also to various steel production environments. The strip steel defect prediction method based on salient target detection represents an advanced and efficient detection approach that can significantly improve the level of steel quality control. With the continuous development of deep learning technology and advancements in computer vision, this method will provide the steel industry with a more reliable and efficient defect detection solution, thereby ensuring safety and reliability across various sectors. Future research and applications will continue to drive development and innovation in this field.

[0027] Surface defects such as scratches, inclusions, and blemishes can degrade the quality of steel, affecting its performance and appearance. Detecting and eliminating these defects helps ensure the production of steel products with high quality, consistency, and reliability. Defects can lead to structural fatigue, fracture, or instability in steel during use, posing a hazard to materials used in construction, transportation (such as ships and automobiles), and other applications. Therefore, surface defect detection helps ensure the safety and reliability of using these steels. Early detection and treatment of surface defects can reduce scrap rates and improve production efficiency. This helps reduce the costs of scrap disposal and reprocessing, thereby reducing production costs.

[0028] High-quality steel products help build a manufacturer's reputation. A good reputation in the market increases customer trust and fosters more business partnerships. Many industries and countries have stringent quality standards and regulations requiring manufacturers to conduct comprehensive quality testing on their products. Surface defect detection for strip steel is a crucial component in meeting these requirements. In-depth research and detection of surface defects can help manufacturers improve production processes and drive the development of new materials and technologies. This contributes to innovation and progress within the industry. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a diagram illustrating the overall model architecture of an embodiment of the present invention.

[0031] Figure 2 This is a diagram of the ResNet34 residual block architecture.

[0032] Figure 3 This is the architecture diagram of the bridging layer.

[0033] Figure 4 This is an architecture diagram of the decoder module. Detailed Implementation

[0034] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0035] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0036] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0037] This invention provides a method for predicting defects in strip steel based on salient target detection, such as... Figure 1 As shown, the steps are as follows:

[0038] Step 1: Extract features from the image.

[0039] The encoder backbone uses a ResNet34 network with a total of 5 layers. For example... Figure 2 As shown, each layer passes through two 3x3 convolutions from the input, then through a channel attention module and a spatial attention module, and finally adds the input features to obtain the output value. Specifically, the outputs of each layer are recorded as h1, h2, h3, h4, and h5, respectively.

[0040] Specifically, the encoder consists of 5 layers using a ResNet34 network, labeled encoder1, encoder2, encoder3, encoder4, and encoder5. The image passes through encoder1 to output feature h1, feature h1 enters encoder2 to output feature h2, feature h2 enters encoder3 to output feature h3, feature h3 enters encoder4 to output feature h4, and feature h4 enters encoder5 to output feature h5.

[0041] Step 2: Connect the codec using a bridging layer

[0042] The input (h5) of the last layer of the encoder is fed into the bridging layer. For example... Figure 3 As shown, the bridging layer has four branching points. These are respectively a conv3*3 (dilation=1, padding=1) convolution, a conv3*3 (dilation=2, padding=2) convolution, a conv3*3 (dilation=3, padding=3) convolution, and a conv3*3 (dilation=4, padding=4) convolution. Finally, the outputs of the four branches are merged with the original input image. The final output, denoted as hbg, is then passed through a 1*1 convolution, followed by BN and ReLU.

[0043] Step 3: Decoder

[0044] The decoder consists of two parts: the FEE Module and the decoder itself. For example... Figure 4 As shown, specifically, the encoder layer output (EN) undergoes a 3x3 convolution, which is then upsampled and concatenated with the decoder layer output (DE). The upper and lower layers undergo 1x1 convolutions to ensure consistent channel counts, before being cross-passed into the channel attention and spatial attention modules. The outputs of these two modules are then concatenated with the bridging layer output (hbg) to obtain the module's output. This process is repeated for a total of five decoder combination modules.

[0045] The decoder module consists of a decoder and five FFE modules. The decoder itself comprises five decoder layers, denoted as decoder1, decoder2, decoder3, decoder4, and decoder5. It should be noted that all five decoder layers are ordinary convolutional decoder layers.

[0046] Specifically, the output feature h5 of encoder5 is input into FEE Module5. The output of the encoder layer is passed through a 3*3 convolution; the output of the 3*3 convolution is passed through a 1*1 convolution, and then cross-passed into the channel attention and spatial attention modules. The outputs of the channel attention and spatial attention modules are fused with hbg through concat to output out5.

[0047] The encoder4 output feature h4 is input into FEE Module 4. The encoder4 output feature h4 is processed by a 3*3 convolution, and upsampled with the decoder5 output hd5. Then, they are fused by concat. The encoder layer output is processed by a 1*1 convolution, and the decoder layer output is also processed by a 1*1 convolution to make the number of channels consistent. Then, they are cross-passed into the channel attention and spatial attention modules. The outputs of the channel attention and spatial attention modules are fused with hbg by concat and output out4.

[0048] The encoder3 output feature h3 is input into FEE Module 3. The encoder3 output feature h3 is processed by a 3*3 convolution, and upsampled with the decoder4 output hd4. Then, they are fused by concat. The encoder layer output is processed by a 1*1 convolution, and the decoder layer output is also processed by a 1*1 convolution to make the number of channels consistent. Then, they are cross-passed into the channel attention and spatial attention modules. The outputs of the channel attention and spatial attention modules are fused with hbg by concat and output out3.

[0049] The encoder2 output feature h2 is input into FEE Module 2. The encoder2 output feature h2 is processed by a 3*3 convolution, and upsampled with the decoder3 output hd3. Then, they are fused by concat. The encoder layer output is processed by a 1*1 convolution, and the decoder layer output is also processed by a 1*1 convolution to make the number of channels consistent. Then, they are cross-passed into the channel attention and spatial attention modules. The outputs of the channel attention and spatial attention modules are fused with hbg by concat to output out2.

[0050] The output feature h1 of encoder1 is input into FEE Module1. The output feature h1 of encoder1 is processed by a 3*3 convolution, and upsampled with the output hd2 of decoder2. Then, they are fused by concat. The output of the encoder layer is processed by a 1*1 convolution, and the output of the decoder layer is also processed by a 1*1 convolution to make the number of channels consistent. Then, they are cross-passed into the channel attention and spatial attention modules. The outputs of the channel attention and spatial attention modules are fused with hbg by concat and output out1.

[0051] Understandably, in this embodiment, a typical convolutional layer consists of two 3x3 convolutions used to extract feature values. The FEE Module, however, aims to fuse information from the encoder and decoder to the greatest extent possible, extracting both detailed and global information through the application of an attention mechanism.

[0052] Step 4: Output

[0053] The output out1 of the top-level decoder module (decoder1) is processed by a 1*1 convolution to output 1 for each channel, which is the final output result output.

[0054] It should be noted that:

[0055] The model presented in this embodiment was implemented using PyTorch on a server with an NVIDIA TITAN X Pascal. The model parameters were initialized using PyTorch's default settings. The optimizer used during training was Adam, with a fixed learning rate of 0.00001. The batch size was set to 1, meaning 32 PNG images were input into the model per iteration, with the resolution adjusted to 224×224.

[0056] This embodiment uses the training set of the publicly available dataset SD-saliency-900 for training.

[0057] The loss function used is BCE loss, which is widely used for binary task classification and is defined as follows:

[0058]

[0059] Where G is the manually annotated supervised image, and S is the predicted saliency map.

[0060] The method in this embodiment first applies salient target detection technology to identify potential defect areas on the steel surface as salient targets, thereby quickly and accurately locating possible problem areas. Next, we use a deep learning model to further analyze these salient targets to detect and classify various types of surface defects, such as scratches, inclusions, and patches. Our method not only improves detection accuracy but also significantly reduces manual intervention, thereby increasing production efficiency.

[0061] The defect prediction method for strip steel based on salient target detection in this embodiment has achieved significant success in quality control. It can not only detect potential defects in a timely manner but also provide crucial information for early intervention in the production process, thereby reducing scrap rates and production costs. This method has broad application prospects and can be used in various industrial fields, including steel manufacturing, construction, and manufacturing, providing a feasible solution for the production of high-quality steel. By combining computer vision and advanced deep learning technologies, our research provides a new perspective and method for the field of defect prediction.

[0062] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments, including components, without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A method for predicting defects in strip steel based on salient target detection, characterized in that, Includes the following steps: S1. Extract five layers of features from the image using an encoder consisting of five encoder layers, denoted as h1, h2, h3, h4, and h5 respectively; S2. Connect the encoder using a bridging layer, and output hbg through the bridging layer. S2-1. Input the feature h5 output by the encoder into the bridging layer, and obtain multi-scale information through several convolutional branches in the bridging layer; S2-2. Merge the multi-scale information and feature h5, then pass it through a 1*1 convolution, BN and ReLU, and use it as the final output, denoted as hbg; S3. Decoding is performed using a decoder module, which includes a decoder and a 5-layer FEE Module. The decoder consists of 5 decoder layers. The encoder output h5 is input into the corresponding 5th layer FEE Module. h4, the output of the 5th layer of the decoder, and hbg are sent to the upper layer FEE Module. The output of the upper layer FEE Module is used as the input of the upper layer decoder. This process is repeated in a loop, resulting in a decoder module consisting of 5 layers of FEE Modules and 5 layers of decoder. S4. The output of the top decoder layer is processed by a 1*1 convolution to output 1 for each channel, which is the final output result. The encoder consists of a 5-layer ResNet34 network; The method for extracting features in the 5-layer ResNet34 network is as follows: each layer passes through two 3*3 convolutions from the input, then through a channel attention module and a spatial attention module, and finally adds the input features to obtain the output value. The output of each layer is recorded as h1, h2, h3, h4, and h5 respectively. The bridging layer has four branching points in parallel: conv3*3 convolution dilation=1, padding=1; conv3*3 convolution dilation=2, padding=2; conv3*3 convolution dilation=3, padding=3; and conv3*3 convolution dilation=4, padding=4. In step S2, h5 is used as input and is convolved at four bifurcations to obtain information at four scales. The FEE Module includes two network branches, one of which consists of 3*3 convolutions and 1*1 convolutions, and the other consists of upsampling and 1*1 convolutions. The FEE Module processing method is as follows: the output of the encoder layer is input into one network branch, the output of the decoder layer is input into another network branch, the output of the encoder layer is passed through a 3*3 convolution, and upsampled with the output of the decoder layer, and then fused by concat; the output of the encoder layer is passed through a 1*1 convolution, and the output of the decoder layer is also passed through a 1*1 convolution to make the number of channels consistent, and then passed to the channel attention and spatial attention modules. The outputs of the channel attention and spatial attention modules are fused with hbg by concat and output out.

Images