A strip steel surface defect detection method and device based on multi-level feature fusion

Abstract

The present application relates to a kind of strip surface defect detection method and equipment based on multi-level feature fusion, the method comprises the following steps: step S1, strip surface image is collected, and pre-processing;Step S2, the image after pre-processing is input into trained strip surface defect detection and is detected for defect, and defect detection result is obtained;The strip surface defect detection model based on multi-level feature fusion includes sequentially connected preliminary feature extraction module and multi-level feature pyramid module.Compared with prior art, the present application has the advantages of improving the detection accuracy of different size defects on strip surface and improving detection speed.

Description

Technical Field

[0001] This invention relates to the field of industrial defect detection, and in particular to a method and device for detecting surface defects in strip steel based on multi-level feature fusion. Background Technology

[0002] Steel strip is a fundamental product of modern industrialized countries, widely used in aerospace, automotive manufacturing, and defense industries. Timely and accurate detection of surface defects in steel strip is crucial for ensuring product quality, which plays a vital role in the safety and reliability of other products. However, due to the complexity of its manufacturing process and the numerous steps involved, defects such as cracks, blemishes, and scratches inevitably occur on the surface of steel strip during production. These defects not only affect the aesthetics of the product but also reduce its corrosion resistance and wear resistance, leading to decreased profits for the company. Therefore, it is essential to strengthen the timely and accurate detection of surface defects in steel strip to ensure the reliable operation of related equipment.

[0003] Traditional manual inspection is time-consuming and prone to missed or false detections. Currently, machine vision-based methods are often used to replace manual inspection for defect detection in some products. However, due to issues such as unclear edge information, low contrast between defects and background, numerous defect types, large intra-class differences among similar defects, and small inter-class differences among dissimilar defects, some mainstream defect detection methods perform poorly in this scenario, failing to meet practical operational requirements. Therefore, it is necessary to further improve the generalization ability and detection accuracy of the models.

[0004] A search revealed Chinese invention patent publication number CN113506239A, which discloses a method for detecting surface defects in strip steel based on multi-level feature fusion using a cross-stage local network. This method includes: constructing a dataset for detecting surface defects in strip steel, containing 12 types of surface defects; constructing a CSP-YOLO network model based on the YOLOv4 deep neural network model for detecting surface defects; training and testing the CSP-YOLO model, and outputting the optimal model to detect the location and type of defects in the acquired strip steel images. This existing patent suffers from low detection accuracy.

[0005] Improving the accuracy of steel surface defect detection has become a technical problem that needs to be solved. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method and device for detecting surface defects in strip steel based on multi-level feature fusion.

[0007] The objective of this invention can be achieved through the following technical solutions: According to one aspect of the present invention, a method for detecting surface defects in strip steel based on multi-level feature fusion is provided, the method comprising the following steps: Step S1: Acquire images of the strip surface and preprocess them; Step S2: Input the preprocessed image into the trained strip surface defect detection system for defect detection and obtain the defect detection result; The strip surface defect detection model is based on multi-level feature fusion, including a preliminary feature extraction module and a multi-level feature pyramid module connected in sequence.

[0008] Preferably, the preliminary feature extraction module includes five sets of feature extraction sub-modules connected in sequence, namely the first set of feature extraction sub-modules, the second set of feature extraction sub-modules, the third set of feature extraction sub-modules, the fourth set of feature extraction sub-modules and the fifth set of feature extraction sub-modules.

[0009] More preferably, the first set of feature extraction sub-modules consists of three stacked convolutional networks, the second set of feature extraction sub-modules consists of one stacked convolutional network and two stacked cross-order spatial networks, the third set of feature extraction sub-modules and the fourth set of feature extraction sub-modules each consist of one stacked downsampling unit, one stacked average spatial attention mechanism unit and one stacked cross-order spatial network, and the fifth set of feature extraction sub-modules consists of one stacked downsampling unit, one stacked average spatial attention mechanism unit, one stacked cross-order spatial network and one stacked spatial pyramid pooling structure network.

[0010] Preferably, the multi-level feature pyramid module incorporates a feature pyramid structure network into the cross-level spatial network.

[0011] More preferably, the multi-level feature pyramid module includes three average spatial attention mechanism sub-modules and a feature pyramid structure network, which minimizes the loss of semantic and detailed information caused by inter-layer transmission by fusing multi-scale information.

[0012] More preferably, the calculation process of multi-level feature fusion by the multi-level feature pyramid module is represented by the following formula: In the formula, F i This represents a shallow feature map, and the corresponding feature map size is large. F i+1 This represents a deep feature map, and the corresponding feature map size is small. Cat Representative feature map channel splicing operation; U( ) This indicates an upsampling operation performed using bilinear interpolation.

[0013] More preferably, the overall calculation process of the average spatial attention mechanism is as follows: In the formula, F and F in The convolutional feature map representing the input. AvgPool Represents average pooling. MaxPool Represents max pooling. This represents a convolution with an n×n kernel. F c Represents a fully connected network. It is the Sigmoid activation function. A c This represents the average channel attention mechanism. A s Represents spatial attention mechanism, F out This represents the output of the average spatial attention mechanism submodule.

[0014] More preferably, the average spatial attention mechanism assigns different weights to different feature information and different location points for automatic information filtering.

[0015] Preferably, the training process of the strip steel surface defect detection model is as follows: the preprocessed image is input into the strip steel surface defect detection model, the result information containing defect category and defect location is calculated, the result information and image label are used to calculate the loss value, and then backpropagation is performed to update the model parameters until the model converges.

[0016] According to another aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described thereon.

[0017] Compared with the prior art, the present invention has the following beneficial effects: 1) This invention first performs effective preliminary feature extraction of defect information from the surface defect image of strip steel, and then obtains multi-scale information of the defect through multi-level feature fusion and performs feature fusion to obtain more significant image features, thereby effectively improving the detection accuracy of defects of different sizes on the surface of strip steel.

[0018] 2) Both the preliminary feature extraction module and the multi-level feature pyramid module of this invention use the average spatial attention mechanism to assign different weights to different feature information and different location points, thereby enabling the screening, extraction and retention of key information, and feature enhancement of effective defect features, which can accurately and quickly detect surface defects of strip steel in complex scenarios. Attached Figure Description

[0019] Figure 1 This is a schematic flowchart of the strip steel surface defect detection method according to an embodiment of the present invention; Figure 2 This is a diagram of the defect detection network structure according to an embodiment of the present invention; Figure 3 This is a preliminary feature extraction module diagram according to an embodiment of the present invention; Figure 4 This is a diagram of a multi-level feature pyramid module according to an embodiment of the present invention; Figure 5 This is a diagram of the average spatial attention mechanism submodule in an embodiment of the present invention; In the attached diagram, BCB: Convolutional Network, CSPlayer: Cross-order Spatial Network, Transition: Downsampling, ASAM: Average Spatial Attention Mechanism, CSP-SPP: Spatial Pyramid Pooling Structure Network, Upsample: Upsampling. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0021] This embodiment relates to a method for detecting surface defects in strip steel based on multi-level feature fusion. This method is based on multi-level feature fusion and average spatial attention mechanism, which can solve the problems of false detection and missed detection in the prior art. At the same time, the method has a fast detection speed and can meet the needs of real-time detection.

[0022] like Figure 1 The method includes the following steps: Step S1: Preprocess the defect images and divide them into training set, validation set and test set; Step S2: Construct a strip steel surface defect detection model based on multi-level feature fusion and average spatial attention mechanism. The model includes a preliminary feature extraction module, a multi-level feature pyramid module, and an average spatial attention mechanism sub-module, such as... Figure 2 ; Step S3: Input the image after image preprocessing into the defect detection model, train and update the parameters in the model until the model converges, and obtain the trained model. Step S4: Input the defect image into the trained model to obtain the defect detection result.

[0023] In step S1, the defect image is preprocessed, specifically by normalizing the image pixel values, scaling, rotating, cropping, and flipping. The dataset is divided as follows: the training set, validation set, and test set account for 81%, 9%, and 10% of the total dataset, respectively.

[0024] In step S2, a defect detection model is constructed, wherein the preliminary feature extraction module includes five sets of feature extraction sub-modules, such as... Figure 3 Specifically: The first feature extraction submodule consists of three stacked convolutional networks. The second feature extraction submodule consists of a convolutional network and two cross-order spatial networks stacked together. The third and fourth feature extraction submodules are each composed of a downsampling unit (Transition), an average spatial attention mechanism unit, and a cross-order spatial network stacked together. The fifth feature extraction submodule consists of a downsampling unit (Transition), an average spatial attention mechanism unit, a cross-order spatial network, and a spatial pyramid pooling structure network stacked together.

[0025] After the input image passes through the preliminary feature extraction module, it outputs three sets of feature maps, which are then sent to the multi-level feature pyramid module for further processing.

[0026] The multi-level feature pyramid module adds a feature pyramid structure network to the Cross-Stage Partial Network (CSP). This module includes three average spatial attention mechanism sub-modules and the feature pyramid structure network. It minimizes the loss of semantic and detail information caused by inter-layer transmission by fusing multi-scale information. Figure 4 The three feature maps are processed through a multi-level feature pyramid module to form three enhanced feature maps.

[0027] In the multi-level feature pyramid module, the calculation process for multi-level feature fusion can be represented by the following formula: In the formula, F i This represents a shallow feature map, and the corresponding feature map size is large. F i+1 This represents a deep feature map, and the corresponding feature map size is small. Cat Representative feature map channel splicing operation; U( ) This indicates an upsampling operation performed using bilinear interpolation.

[0028] like Figure 5In the average spatial attention mechanism submodule, the overall calculation process is represented by the following formula: In the formula, F and F in The convolutional feature map representing the input. AvgPool Represents average pooling. MaxPool Represents max pooling. This represents a convolution with an n×n kernel. F c Represents a fully connected network. It is the Sigmoid activation function. A c This represents the average channel attention mechanism. A s Represents spatial attention mechanism, F out This represents the output of the average spatial attention mechanism submodule.

[0029] In step S3, the image after image preprocessing is input into the defect detection model. The model calculates the result information containing the defect category and defect location. The result information and image labels are used to calculate the loss value. Then, backpropagation is performed to update the model parameters until the model converges.

[0030] By incorporating an attention mechanism (ASAM) into the preliminary feature extraction module and the multi-level feature pyramid module for automatic information filtering, the loss generated during information extraction is reduced, and the effective fusion of multi-scale information is promoted. Therefore, this invention can accurately classify complex defects and detect them with high precision.

[0031] Example 2 This embodiment involves a comparison between the method of the present invention and other methods. The average accuracy index values ​​are shown in Table 1: Table 1 As shown in Table 1, the average accuracy of the multi-level feature fusion-based strip surface defect detection method based on multi-level feature fusion and average spatial attention mechanism in this embodiment is 80.00%, significantly higher than other methods. This indicates that the defect detection method in this embodiment can effectively improve defect detection accuracy and has high application value.

[0032] The electronic device of this invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM may also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0033] Multiple components in the device are connected to the I / O interface, including: input units such as keyboards and mice; output units such as various types of displays and speakers; storage units such as disks and optical discs; and communication units such as network interface cards (NICs), modems, and wireless transceivers. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0034] The processing unit executes the various methods and processes described above, such as methods S1 to S4. For example, in some embodiments, methods S1 to S4 may be implemented as computer software programs tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of the methods S1 to S4 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S4 by any other suitable means (e.g., by means of firmware).

[0035] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting surface defects in strip steel based on multi-level feature fusion, characterized in that, The method includes the following steps: Step S1: Acquire images of the strip surface and preprocess them; Step S2: Input the preprocessed image into the trained strip surface defect detection model to perform defect detection and obtain the defect detection results; Step S3: Input the image after image preprocessing into the strip steel surface defect detection model. The model calculates the result information containing the defect category and defect location. The result information and image labels are used to calculate the loss value. Then, backpropagation is performed to update the model parameters until the model converges. The strip surface defect detection model is based on multi-level feature fusion, including a preliminary feature extraction module and a multi-level feature pyramid module connected in sequence. The preliminary feature extraction module includes five feature extraction sub-modules, specifically: The first feature extraction submodule consists of three stacked convolutional networks. The second feature extraction submodule consists of a convolutional network and two cross-order spatial networks stacked together. The third and fourth feature extraction submodules are each composed of a downsampling unit, an average spatial attention mechanism unit, and a cross-order spatial network stacked together. The fifth feature extraction submodule consists of a downsampling unit, an average spatial attention mechanism unit, a cross-order local network layer, and a spatial pyramid pooling structure network stacked together. After the input image passes through the preliminary feature extraction module, it outputs three sets of feature maps, which are then sent to the multi-level feature pyramid module for further processing. The multi-level feature pyramid module adds a feature pyramid structure network to the cross-level spatial network. The multi-level feature pyramid module includes three average spatial attention mechanism sub-modules and a feature pyramid structure network. It minimizes the loss of semantic and detail information caused by inter-layer transmission by fusing multi-scale information.

2. The method for detecting surface defects in strip steel based on multi-level feature fusion according to claim 1, characterized in that, The calculation process of multi-level feature fusion by the multi-level feature pyramid module is expressed by the following formula: In the formula, F i This represents a shallow feature map, and the corresponding feature map size is large. F i+1 This represents a deep feature map, and the corresponding feature map size is small. Cat Representative feature map channel splicing operation; U( ) This indicates an upsampling operation performed using bilinear interpolation.

3. The method for detecting surface defects in strip steel based on multi-level feature fusion according to claim 1, characterized in that, The overall calculation process of the average spatial attention mechanism is as follows: In the formula, F and F in The convolutional feature map representing the input. AvgPool Represents average pooling. MaxPool Represents max pooling. This represents a convolution with an n×n kernel. F c Represents a fully connected network. It is the Sigmoid activation function. A c This represents the average channel attention mechanism. A s Represents spatial attention mechanism, F out This represents the output of the average spatial attention mechanism submodule.

4. The method for detecting surface defects in strip steel based on multi-level feature fusion according to claim 3, characterized in that, The average spatial attention mechanism assigns different weights to different feature information and different location points to automatically filter information.

5. The method for detecting surface defects in strip steel based on multi-level feature fusion according to claim 1, characterized in that, The training process of the strip steel surface defect detection model is as follows: the preprocessed image is input into the strip steel surface defect detection model, the result information containing defect category and defect location is calculated, the result information and image label are used to calculate the loss value, and then backpropagation is performed to update the model parameters until the model converges.

6. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 5.

Images