Target detection method for steel plate surface defects

Abstract

The invention discloses a target detection method for steel plate surface defects, which comprises the following steps: acquiring steel plate surface defect image data before and after marking to form a sample set; based on a multi-scale step-by-step fusion residual network and a double-branch decoding strategy, a target detection model is constructed, and the double-branch decoding strategy comprises a refined defect characterization decoding branch and an enhanced defect characterization decoding branch; inputting the sample set into the target detection model for processing, and generating a saliency map; performing loss parameter adjustment on the generated saliency map based on the label truth value of the defect image in the sample set to obtain an optimized target detection model; and carrying out real-time detection on the surface defects of the steel plate based on the optimized target detection model. According to the method model provided by the invention, the calculation overhead is greatly reduced while high accuracy is maintained, and an efficient and practical solution is provided for a real-time industrial detection system.

Description

Technical Field

[0001] The invention belongs to the technical field of machine vision and industrial automation, and in particular relates to a target detection method for surface defects of a steel plate. Background Art

[0002] With the rapid development of industrial automation, the demand for quality inspection in the production process continues to increase, especially for industries that require high-quality surface treatment, such as steel plate manufacturing. Traditional steel plate surface defect detection usually relies on manual or rule-based algorithms. However, these methods are inefficient, inaccurate, and easily affected by human factors, making it difficult to meet the requirements of real-time and high precision, especially in high-throughput production environments. In recent years, deep learning-based defect detection methods have made significant progress in the field of image processing, making it possible to achieve automated and intelligent steel plate surface defect detection.

[0003] Steel plate surface defect images have various defect types, low contrast between defects and background, and large differences in defect scale. This requires specialized technology to accurately capture details and filter out noise and background interference. However, although existing machine vision technologies perform well in some defect detection, they often have difficulty in balancing high precision and real-time performance, and there are still some challenges and limitations:

[0004] Existing detection methods that pursue high precision perform well in the detection of steel plate defect images, but most of them rely on models that require pre-training and huge computational effort, and require a lot of hardware resources to achieve high detection accuracy. This limits their real-time detection in actual factory applications, especially in environments with limited hardware resources, making it difficult to deploy and apply them effectively.

[0005] Although many existing lightweight defect detection models can achieve a relatively high detection speed, they still face challenges when dealing with complex industrial environments with diverse defects. They have problems such as limited adaptability to defect types and insufficient detection accuracy, making them difficult to apply in actual production.

[0006] Therefore, how to optimize computational efficiency while ensuring detection accuracy has become a key challenge in steel plate surface defect detection. Especially in high-efficiency application scenarios, such as high-throughput production lines, how to balance the contradiction between detection accuracy and efficiency has become the main bottleneck limiting system performance. Therefore, developing a lightweight and efficient model to achieve high-precision steel plate defect detection with minimal computational overhead has become an urgent problem to be solved. Summary of the invention

[0007] In order to solve the above technical problems, the present invention proposes a target detection method for steel plate surface defects. This method realizes high-precision detection of steel plate surface defects by designing a lightweight deep learning model and an optimized saliency detection algorithm, and significantly reduces the computational cost, meeting the needs of real-time detection, thereby solving the problems existing in the above-mentioned prior art.

[0008] To achieve the above object, the present invention provides a target detection method for surface defects of a steel plate, comprising the following steps:

[0009] Obtaining image data of steel plate surface defects before and after annotation to form a sample set;

[0010] Based on a multi-scale step-by-step fusion residual network and a dual-branch decoding strategy, a target detection model is constructed, wherein the dual-branch decoding strategy includes a refined defect representation decoding branch and an enhanced defect representation decoding branch;

[0011] Inputting the sample set into the target detection model for processing to generate a saliency map;

[0012] Based on the true value of the label of the defect image in the sample set, the loss parameter of the generated saliency map is adjusted to obtain the optimized object detection model;

[0013] Based on the optimized target detection model, the surface defects of the steel plate are detected in real time.

[0014] Optionally, obtaining the image data of the steel plate surface defects before and after the annotation, and forming the sample set further includes:

[0015] The sample set is subjected to data augmentation operations such as random horizontal flipping, random rotation, and random color jittering.

[0016] Optionally, the multi-scale step-by-step fusion residual network includes several multi-scale residual fusion modules.

[0017] Optionally, the refined defect representation decoding branch includes a feature extraction module, a defect representation refinement module and a multi-scale attention fusion module;

[0018] Among them, the feature extraction module includes a 1×1 convolution layer and a 3×3 depth-separable convolution layer; the defect representation refinement module includes two 3×3 depth-separable convolution layers; the multi-scale attention fusion module includes different convolution layers, a coordinate attention layer, an average pooling layer and a feature enhancement layer.

[0019] Optionally, the enhanced defect representation decoding branch includes a multi-scale attention fusion module and a CBR module;

[0020] The CBR module includes a 3×3 convolution layer, a batch normalization layer and a ReLU activation function layer.

[0021] Optionally, the sample set is input into the target detection model for processing, and the process of generating a saliency map includes:

[0022] The sample set is input into the multi-scale step-by-step fusion residual network to perform step-by-step feature extraction to obtain multi-level feature maps of different scales; the multi-level feature maps of different scales are input into the feature extraction module in the refined defect representation decoding branch to obtain a first refined feature map and a highest-level refined feature map; the multi-level feature maps of different scales are input into the defect representation refinement module in the refined defect representation decoding branch to obtain a second refined feature map; the first refined feature map and the second refined feature map are input into the multi-scale attention fusion module to perform the first multi-level feature fusion to obtain a preliminary enhanced feature map; the highest-level refined feature map and the preliminary enhanced feature map are input into the enhanced defect representation decoding branch to perform feature fusion to obtain a fused saliency map.

[0023] Optionally, a hybrid loss function is used to adjust the loss parameters of the generated saliency map based on the true value of the label of the defect image in the sample set, and the hybrid loss function includes a binary cross entropy loss function, a structural similarity loss function, an intersection-over-union loss function and a cross entropy loss function.

[0024] The present invention also provides a computer device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method.

[0025] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method are implemented.

[0026] The present invention also provides a computer program product, comprising a computer program, which implements the steps of the method when executed by a processor.

[0027] Compared with the prior art, the present invention has the following advantages and technical effects:

[0028] The proposed method for detecting surface defects of steel plates uses a multi-scale step-by-step fusion residual network as the backbone network. Even without pre-training, it can achieve competitive defect detection accuracy and significantly reduce model size and inference time. Combined with a dual-branch decoding architecture and a multi-scale attention fusion module, efficient and high-precision detection performance can be achieved.

[0029] In general, the method model proposed in the present invention greatly reduces the computational overhead while maintaining high accuracy, providing an efficient and practical solution for real-time industrial detection systems.

[0030] The present invention can achieve higher detection accuracy with a lower number of model parameters, can effectively address key challenges in industrial defect detection, and is suitable for industrial environments with limited resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0032] Figure 1 Schematic diagram of the overall process of the target detection method according to an embodiment of the present invention;

[0033] Figure 2 Schematic diagram of the overall architecture of the target detection method according to an embodiment of the present invention;

[0034] Figure 3 Schematic diagram of the overall architecture of a multi-scale residual fusion module according to an embodiment of the present invention;

[0035] Figure 4 A schematic diagram of the overall architecture of a multi-scale attention fusion module according to an embodiment of the present invention;

[0036] Figure 5 It is a schematic diagram of the qualitative effect of real-time significant target detection of steel plate surface defects according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0038] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] Embodiment 1

[0040] like Figure 1 As shown, this embodiment provides a target detection method for steel plate surface defects, comprising the following steps performed in sequence:

[0041] Obtaining image data of steel plate surface defects before and after annotation to form a sample set;

[0042] Based on a multi-scale step-by-step fusion residual network and a dual-branch decoding strategy, a target detection model is constructed, wherein the dual-branch decoding strategy includes a refined defect representation decoding branch and an enhanced defect representation decoding branch;

[0043] Inputting the sample set into the target detection model for processing to generate a saliency map;

[0044] Based on the true value of the label of the defect image in the sample set, the loss parameter of the generated saliency map is adjusted to obtain the optimized object detection model;

[0045] Based on the optimized target detection model, the surface defects of the steel plate are detected in real time.

[0046] As a specific embodiment, the specific process includes:

[0047] Step 1: Steel plate defect image data processing:

[0048] The pre-acquired steel plate surface defect image data and the annotated steel plate surface defect image data form a sample set, and the sample set is divided into a training set and a test set according to a set ratio; then the training set is used as a model input image and preprocessed.

[0049] Step 2: Based on the multi-scale step-by-step fusion residual network and dual-branch decoding strategy, a target detection model is constructed. The multi-scale step-by-step fusion residual network is used as the backbone network, and the backbone network extracts multi-scale features:

[0050] The divided training set data is input into the backbone network to suppress background interference and refine defect details, and obtain significant defect feature maps under different receptive fields of the defect image.

[0051] Step 3: Decoder network refines defect representation:

[0052] This embodiment adopts a dual-branch decoding strategy to improve the detection accuracy of the model, further refines the defect representation and strengthens the defect representation processing of the 5-level scale defect features extracted by the backbone network, so as to realize multi-scale and multi-stage significant feature fusion and enhance the defect area recognition and feature expression capabilities of the model. Among them, the refined defect representation (RDR) decoding branch is composed of the feature extraction module (IFE), the defect representation refinement module (DRR) and the multi-scale attention fusion module (MAF) to realize the preliminary extraction of defect features and the refinement of defect representation.

[0053] Step 4: Decoder network enhancement defect characterization:

[0054] The enhanced defect representation (EDR) decoding branch is mainly composed of a multi-scale attention fusion module (MAF) and a CBR module. It enhances the extracted defect features and further characterizes the output saliency map by fusing multi-scale features in a bottom-up and step-by-step manner.

[0055] Step 5: Loss training model:

[0056] This embodiment uses the true value of the labels of various defects in the training set to perform a reverse network propagation loss adjustment on the saliency map Si finally output in the decoder network to optimize the training of the method model of this embodiment.

[0057] Step 6: Model testing to detect defective images:

[0058] This embodiment inputs the defect images in the test set into the optimal target detection model for steel plate surface defects and outputs the detection results of the method model.

[0059] Furthermore, in step 1, the steel plate defect image data processing process includes in detail:

[0060] Step 1.1: First, the data set consisting of the acquired steel plate surface defect image data and its label true value is divided into a training set and a test set according to a set ratio;

[0061] Step 1.2: Use the training set as input images for the method model The input image I is preprocessed. The preprocessing is to apply three data enhancement operations, namely, random horizontal flipping, random rotation and random color jittering, to the input image to improve the generalization ability of the model.

[0062] Furthermore, in step 2, the backbone network extracts multi-scale features including:

[0063] Step 2.1: By using the multi-level feature extraction method of the multi-scale progressive fusion residual network (MPARNet), the background and noise interference of the input image data I is suppressed layer by layer, the defect feature information is emphasized, and multi-level feature maps of different scales are extracted.

[0064] The multi-scale progressive fusion residual network (MPARNet) is a lightweight backbone network designed in this embodiment to meet the needs of efficient and high-precision industrial detection. Figure 2 As shown in Figure 1, MPARNet consists of a Stem layer and four Layer layers. Figure 3 As shown in the figure, multiple multi-scale residual fusion modules (MRF) are stacked to ensure that the backbone network can highlight the target area while remaining lightweight and generate high-quality feature maps that focus more on salient targets.

[0065] Step 2.2: Process the input image layer by layer through MPARNet This embodiment extracts defect feature maps of 5 different scales Where i∈{1,2,3,4,5}, and the size settings of each level of features are shown in Table 1.

[0066] Table 1

[0067]

[0068] Furthermore, in step 3, the decoder network refines the defect representation in detail including:

[0069] Step 3.1: 5-level defect feature map The input is sent to the feature extraction module (IFE) in the refined defect representation (RDR) decoding branch to further process the defect features and obtain the first refined feature map. where i∈{1,2,3,4,5}.

[0070] The feature extraction module (IFE) is composed of a 1×1 convolutional layer (Conv) and a 3×3 depth-separable convolutional layer (DSConv). Input, get the initial refined features where i∈{1,2,3,4,5}.

[0071] Step 3.2: 5-level defect feature map The input is sent to the Defect Refinement Module (DRR) in the Refined Defect Representation (RDR) decoding branch to further process the defect features and obtain the second refined feature map. where i∈{2,3,4,5}.

[0072] The defect representation refinement module (DRR) is composed of two 3×3 depthwise separable convolutional layers to expand the receptive field to improve the feature representation capability, and an attention convolutional module layer (CBAM) is added in the middle to further refine the features to accelerate the convergence of the model. Other defect characteristics Input, get the second refined feature where i∈{2,3,4,5}.

[0073] Step 3.3: Remove thin features In addition, the first refinement feature With the second refinement feature The flow goes to the multi-scale attention fusion module (MAF) for the first multi-level feature fusion to capture the semantic dependency between defect features at all levels to aggregate and obtain preliminary enhanced features. where i∈{1,2,3,4}.

[0074] The multi-scale attention fusion module (MAF) is composed of different convolutional layers, coordinate attention layers (CA attention layers), average pooling layers and feature enhancement layers. Figure 4As shown in the figure, this module can combine the fine-grained details from the shallow layer with the abstract semantic information from the deep layer in a multi-scale fusion manner, ensuring that information at different scales is effectively fused, which helps the model cope with defect features of different complexities and scales, thereby improving the detection performance of the model.

[0075] As an additional embodiment, the high-level feature f g and low-level features f d are added to the channel attention (CA) module to enhance the important channel features, and f' g =CA(CBR(f g ))) and f' d =CA(f d ), CBR(·) represents a convolution layer, a batch normalization layer, and a ReLU activation function layer; then the high-level feature f' g By using multi-scale convolution kernels, features of different scales can be extracted to obtain preliminary fusion features. Indicates the concatenation operation; to further enhance the initial fusion features In order to improve the feature expression ability, this embodiment uses the average pooling operation to provide important weight information for high and low layer features. represents the element-by-element product operation, s(·) represents the Sigmoid activation function, Avg 3×3 (·) represents the average pooling operation; then more convolutional features are fused again to obtain secondary fusion features Finally, in order to prevent the information of significant defects from being diluted, especially for the low contrast characteristics of strip defects, this embodiment performs feature enhancement on the fused feature map and finally generates a multi-scale fusion feature map.

[0076] Furthermore, in step 4, the decoder network enhancement defect characterization includes in detail:

[0077] Step 4.1: Preliminary enhanced features from the output of the refined defect characterization (RDR) decoding branch and Features This embodiment uses high-level features As input, the features are then gradually aggregated using the MAF module and the CBR module. To achieve multi-scale deep attention feature fusion to highlight defects.

[0078] The CBR module is composed of a 3×3 convolution layer, a batch normalization layer, and a ReLU activation function layer.

[0079] Step 4.2: This embodiment uses the fusion output features of each level and Features The saliency map of is taken as the output of the model, where i∈{1,2,3,4}.

[0080] The salient map Si is outputted. In order to generate a high-quality clear salient map, this embodiment uses a bilinear interpolation upsampling operation to adjust the predicted salient map Si to the same size as the original input image I, where i∈{1,2,3,4,5}.

[0081] Furthermore, in step 5, the loss training model includes in detail:

[0082] Step 5.1: This example uses a hybrid loss function To process the true value of the label of the defect image and the salient map Si generated by the model, so as to perform an effective optimization training on the method model of this embodiment.

[0083] The hybrid loss is composed of a binary cross entropy loss function (BCE loss), a structural similarity loss function (SSIM loss), an intersection over union loss function (IOU loss) and a cross entropy loss function (CEL loss). An effective loss function can not only quantify the gap between the model and the target, but also accelerate convergence, avoid overfitting, and enhance robustness during model training.

[0084] Furthermore, in order to fully demonstrate the efficiency and accuracy of the method of the present invention, the following evaluation indicators are specially set to quantitatively analyze the defect detection performance of the method model:

[0085] S-measure(S α ,α=0.5): It can effectively reflect the performance of the image segmentation model in terms of structure and content. α The larger the value, the better.

[0086] MAE: The difference between the saliency map S and the true map G is measured by calculating the average of the absolute values ​​of the prediction errors. The smaller the MAE, the better.

[0087] E-measure(E ξ ): It can comprehensively consider the accuracy of the salient map S region and the boundary precision, and pay special attention to the boundary and regional consistency of the segmentation results. This embodiment uses adaptive E-measure mean E-measure and maximum E-measure As evaluation indicators, the larger the values ​​of these three evaluation indicators, the better.

[0088] F-measure(F β): It can comprehensively consider the precision and recall rate and comprehensively evaluate the quality of the saliency map. This embodiment uses adaptive F-measure mean F-measure and maximum F-measure As evaluation indicators, the larger the values ​​of these three evaluation indicators, the better.

[0089] Weighted F-measure The weighting mechanism can better reflect the performance of the model on key categories. The larger the value, the better.

[0090] Table 2 shows the quantitative effect of real-time significant target detection of steel plate surface defects. It can be seen from Table 2 that the model-L based on VGG16 as the backbone network has the best performance in detection performance.

[0091] Table 2

[0092]

[0093] The present invention proposes a real-time significant target detection model for surface defects of steel plates, which can achieve higher detection accuracy with a lower number of model parameters, can effectively cope with key challenges in industrial defect detection, and is suitable for industrial environments with limited resources. Among them, a multi-scale progressive fusion residual network (MPARNet) is designed as the backbone network, which can achieve competitive defect detection accuracy even without pre-training, and significantly reduce the model size and inference time. Combined with the dual-branch decoding architecture and the multi-scale attention fusion module (MAF), efficient and high-precision detection performance can be achieved. In general, the method model proposed in the present invention greatly reduces the computational overhead while maintaining high accuracy, providing an efficient and practical solution for real-time industrial detection systems.

[0094] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A target detection method for steel plate surface defects, characterized in that: The following steps are involved: Obtaining image data of steel plate surface defects before and after annotation to form a sample set; Based on a multi-scale step-by-step fusion residual network and a dual-branch decoding strategy, a target detection model is constructed, wherein the dual-branch decoding strategy includes a refined defect representation decoding branch and an enhanced defect representation decoding branch; Inputting the sample set into the target detection model for processing to generate a saliency map; Based on the true value of the label of the defect image in the sample set, the loss parameter of the generated saliency map is adjusted to obtain the optimized object detection model; Based on the optimized target detection model, the surface defects of the steel plate are detected in real time.

2. The method according to claim 1, characterized in that Obtain the image data of steel plate surface defects before and after annotation, and after forming the sample set, it also includes: The sample set is subjected to data augmentation operations such as random horizontal flipping, random rotation, and random color jittering.

3. The method according to claim 1, characterized in that: The multi-scale step-by-step fusion residual network includes a plurality of multi-scale residual fusion modules.

4. The method according to claim 1, characterized in that The refined defect representation decoding branch includes a feature extraction module, a defect representation refinement module and a multi-scale attention fusion module; Among them, the feature extraction module includes a 1×1 convolution layer and a 3×3 depth-separable convolution layer; the defect representation refinement module includes two 3×3 depth-separable convolution layers; the multi-scale attention fusion module includes different convolution layers, a coordinate attention layer, an average pooling layer and a feature enhancement layer.

5. The method according to claim 4, characterized in that The enhanced defect representation decoding branch includes a multi-scale attention fusion module and a CBR module; The CBR module includes a 3×3 convolution layer, a batch normalization layer and a ReLU activation function layer.

6. The method according to claim 5, characterized in that The sample set is input into the target detection model for processing, and the process of generating a saliency map includes: The sample set is input into the multi-scale step-by-step fusion residual network to perform step-by-step feature extraction to obtain multi-level feature maps of different scales; the multi-level feature maps of different scales are input into the feature extraction module in the refined defect representation decoding branch to obtain a first refined feature map and a highest-level refined feature map; the multi-level feature maps of different scales are input into the defect representation refinement module in the refined defect representation decoding branch to obtain a second refined feature map; the first refined feature map and the second refined feature map are input into the multi-scale attention fusion module to perform the first multi-level feature fusion to obtain a preliminary enhanced feature map; the highest-level refined feature map and the preliminary enhanced feature map are input into the enhanced defect representation decoding branch to perform feature fusion to obtain a fused saliency map.

7. The method according to claim 1, characterized in that A hybrid loss function is used to adjust the loss parameters of the generated saliency map based on the true value of the label of the defect image in the sample set. The hybrid loss function includes a binary cross entropy loss function, a structural similarity loss function, an intersection-over-union loss function and a cross entropy loss function.

8. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images