A metal surface scratch defect segmentation method

Abstract

The present application relates to the technical field of computer vision, and particularly relates to a metal surface scratch defect segmentation method, comprising the following steps: obtaining metal surface image data to be segmented to construct a scratch defect data set for model training and model testing; training the model through the scratch defect data set to obtain an MSDD-UNet model; inputting the metal surface image data to be segmented into the MSDD-UNet model to obtain a pixel-level segmentation result of the metal surface scratch defect. The present application solves the problems of the existing metal surface scratch defect segmentation method, such as insufficient detection accuracy under a complex background, weak representation ability for slender and tiny scratch defects, easy loss of detail information in the down-sampling process, and insufficient global context modeling ability in the feature fusion process, and can provide more accurate and stable detection and segmentation results for the metal surface scratch defect in the actual environment.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method for segmenting scratch defects on metal surfaces. Background Technology

[0002] Metals and metal products, as core basic materials in modern industrial manufacturing, are widely used in many key sectors of the national economy, such as aerospace, automobile manufacturing, precision instruments, and rail transportation. Their surface quality directly determines the assembly precision, service life, and safety performance of downstream products, and is one of the important indicators for measuring the level of industrial manufacturing.

[0003] However, throughout the entire lifecycle of metal materials—from smelting and rolling to processing, transportation, and storage—surface scratches are difficult to completely avoid due to various factors, including limitations in production processes, natural equipment wear, human error, and external impacts and compression. These defects not only damage the product's appearance and increase the difficulty of cleaning and maintenance, but also become stress concentration sources, significantly reducing key mechanical properties such as dimensional stability, surface roughness, and fatigue limit. Industry statistics show that even tiny scratches on raw materials with a depth of only 0.1 mm can be stretched and extended into deeper surface grooves after plastic processing processes such as cold drawing and cold rolling. This causes the surface roughness Ra value of the finished product to increase from the standard 0.8 μm to over 2.5 μm, severely exceeding the assembly requirements of precision equipment. The pass rate for raw materials containing surface fold defects is only 53%, far lower than the pass rate of over 98% for normal raw materials.

[0004] Therefore, timely detection and repair of scratches on metal surfaces is crucial for ensuring the performance of downstream products, reducing production costs, and enhancing a company's market competitiveness. Currently, manufacturers primarily repair defects by recoating, polishing, and laser cladding to restore the product's aesthetics and durability. However, these repair measures rely on accurate and efficient scratch defect detection.

[0005] Traditional metal surface scratch inspection mainly relies on manual visual inspection, which has inherent drawbacks such as low efficiency, high subjectivity, and high false negative rate. Each person can only inspect 3-5 products per minute, with a false negative rate as high as 15%-20%. Furthermore, it is easily affected by factors such as human fatigue, experience level, and emotional state. It is also difficult to standardize inspection standards, which cannot meet the high-speed, continuous, and stable quality inspection requirements of modern industrial production lines.

[0006] In recent years, computer vision technologies, represented by deep learning, have brought revolutionary breakthroughs to the detection of scratches on metal surfaces. By constructing deep neural network models, the microscopic texture and geometric features of scratches can be automatically learned, achieving accurate identification of minute defects. Both detection accuracy and efficiency are significantly improved compared to traditional methods. However, current deep learning-based scratch detection technology still faces many pressing challenges:

[0007] Model efficiency is mismatched with industrial needs: In pursuit of high-precision detection results, existing technologies typically design large and complex deep learning models. These models contain massive amounts of parameters, require extremely high computing resources, and have long training and inference times, making them difficult to adapt to the stringent requirements of detection efficiency in industrial production environments. Some models have a single-image detection time exceeding 50ms, which cannot meet the high-speed production line cycle of more than 120 pieces per minute.

[0008] The problem of data class imbalance is prominent: In industrial production, the number of samples with scratch defects on metal surfaces is far less than that of normal samples, resulting in a serious foreground-background class imbalance. This data distribution characteristic causes deep learning models to tend to learn the features of normal samples, resulting in insufficient ability to identify scratch defects, especially prone to missing low-proportion, small-sized fine scratches.

[0009] Detecting complex targets is challenging: scratches on metal surfaces are characterized by extremely small scale (width typically between 0.05mm and 0.5mm), extremely low contrast (contrast with the background is usually less than 10%), and irregular shapes (including straight lines, curves, intersections, discontinuous shapes, and other forms). They are also easily affected by natural textures, water stains, oil, dust, etc. on the metal surface. This is a typical task for detecting small, low-contrast, and complex-shaped targets. Existing models have significant shortcomings in feature extraction and generalization capabilities, making it difficult to achieve accurate detection and localization of various scratch defects. Summary of the Invention

[0010] To address the aforementioned issues, this invention provides a method for segmenting scratch defects on metal surfaces. This method solves the problems of insufficient detection accuracy in complex backgrounds, weak characterization of slender and minute scratch defects, easy loss of detailed information during downsampling, and insufficient global context modeling capability during feature fusion. It can provide more accurate and stable detection and segmentation results for scratch defects on metal surfaces in real-world environments.

[0011] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for segmenting scratch defects on a metal surface includes the following steps: S1. Obtain the metal surface image data to be segmented, and preprocess the metal surface image data to construct a scratch defect dataset for model training and model testing; S2. Train the model using the scratch defect dataset to obtain the MSDD-UNet model; S3. Input the metal surface image data to be segmented into the MSDD-UNet model, and perform feature extraction, frequency domain decomposition, self-attention feature fusion, decoding upsampling reconstruction in sequence to obtain pixel-level segmentation results of metal surface scratch defects.

[0012] Further, in step S1, the preprocessing step of the metal surface image data includes: S1.1 Acquire original images containing surfaces of various metals and alloys under different lighting conditions, and after filtering and cropping the original images, obtain metal surface image data containing scratches and defects; S1.2 Perform size unification preprocessing on the acquired metal surface image data to adjust the image space size of the metal surface image data to be the same, and mark the scratch defect areas in the metal surface image data to generate a scratch defect mask label file corresponding to the original image; S1.3 Divide the label files into training and testing sets according to the proportion to construct a scratch defect dataset.

[0013] Furthermore, in step S1.3, random data augmentation is performed on each supervised training image and its mask label in the training set.

[0014] Further, in step S3, the MSDD-UNet model includes an encoder path, a decoder path, and a skip connection structure connecting the encoder path and the decoder path. The encoder path will include a downsampling module that includes a spatial-to-depth module and a lightweight channel attention module, replacing the traditional max pooling operation; In the skip connection structure, a cross-channel self-attention mechanism based on frequency domain decomposition is set; In the decoder path, the fused features are upsampled and reconstructed step by step to restore spatial resolution and obtain pixel-level segmentation results.

[0015] Furthermore, in the downsampling module, the input feature map is sliced ​​to divide it into several sub-feature maps in the spatial dimension. The spatial-to-depth module then concatenates the sub-feature maps in the depth dimension to obtain a concatenated feature map. The lightweight channel attention module assigns channel weights to each channel of the concatenated feature map. Based on the channel weights, the depth of the concatenated feature map is restored to its original size using a convolution with a kernel size of (1, 1) to downsample the feature map during the encoding process.

[0016] Furthermore, when the size of the feature map is During the transformation from S-space to depth, the slicing operation on the feature map is as follows: Formula (1); Where X is the feature map; The i-th row and j-th column of the output f of the slicing operation are represented; scale is the downsampling scaling factor.

[0017] Furthermore, in the self-attention mechanism, the features extracted from the encoder path are implicitly decomposed into low-frequency components representing global information and high-frequency components carrying local details through a convolution with a kernel size of (1, 1). Self-attention calculation is performed on the low-frequency components along the channel dimension to extract global features. The global features and the high-frequency components are then concatenated and fused with the feature map of the decoder path to obtain fused features.

[0018] Furthermore, the self-attention is calculated as follows: Formula (2); Where X is the input feature map; Q, K, V and , , These represent the outputs and weights of the query, key, and value fully connected layers, respectively. is the transpose of Q, Softmax is the normalized exponential function, and Attention is the output of the attention calculation.

[0019] Furthermore, in step S2, the loss function of the MSDD-UNet model is a hybrid loss function that combines Focal loss and Dice loss. The hybrid loss function is used to dynamically weight the difficult-to-identify scratch samples during network training, and it is also used to evaluate the shape similarity between the segmented region and the real mask.

[0020] Furthermore, the hybrid loss function is: Formula (3); in, L is a hyperparameter used to balance focal loss and Dice loss; focal Let L be the Focal loss function; Dice The Dice loss function; The expression for the focal loss function is: Formula (4); The Dice loss function is expressed as follows: Formula (5); Among them, P t This represents the value of the i-th pixel in the model segmentation result; , t represents the probability that the model predicts for the target. i To supervise the value of the i-th pixel in the annotated scratch mask; For hyperparameters, when When set to 0, It degenerates into the original cross-entropy loss function; This is used to reduce the weight of easily identifiable samples while increasing the weight of difficult-to-identify samples; N is the total number of pixels in the image.

[0021] The beneficial effects of this invention are: In the MSDD-UNet model, a downsampling module, including a spatial-to-depth module and a lightweight channel attention module, is embedded into U-Net. This reduces the loss of contextual information in the feature map after multiple convolutions and pooling in the encoder, thereby acquiring more information. For the geometric features of scratches, such as direction and thickness, and their inherent high-frequency components, a cross-channel self-attention mechanism based on frequency domain decomposition is embedded in the skip connection structure connecting the encoder and decoder paths. This self-attention mechanism is a lightweight, redundancy-resistant structure that refines edge information, serving as a precursor to the encoder-decoder feature fusion process. Furthermore, addressing the inherent foreground-background class imbalance in metal surface scratch defect data, this invention introduces a hybrid loss function combining focal loss and Dice loss, effectively utilizing their unique and superior characteristics to train the model for accurate scratch segmentation. Attached Figure Description

[0022] Figure 1 This is a flowchart of a preferred embodiment of the present invention.

[0023] Figure 2 This is a diagram of the overall network structure of the MSDD-UNet model according to a preferred embodiment of the present invention.

[0024] Figure 3 This is a flowchart illustrating the downsampling mode processing of a preferred embodiment of the present invention.

[0025] Figure 4 A forward propagation process diagram of a lightweight channel attention module according to a preferred embodiment of the present invention.

[0026] Figure 5 A network structure diagram of a self-attention mechanism according to a preferred embodiment of the present invention.

[0027] Figure 6 A partial image of a metal scratch dataset according to a preferred embodiment of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0030] Please also see Figures 1 to 6 A preferred embodiment of the present invention provides a method for segmenting scratch defects on a metal surface, comprising the following steps: S1. Obtain the image data of the metal surface to be segmented, and preprocess the metal surface image data to construct a scratch defect dataset for model training and model testing.

[0031] In step S1, the preprocessing steps for the metal surface image data include: S1.1 Acquire raw images of surfaces of various metals and alloys under different lighting conditions. After screening and cropping, obtain metal surface image data containing scratches and defects. In this embodiment, a total of 1575 metal surface image data containing scratches and defects are obtained. The metal surface image data covers scratches and defects with diverse shapes and color distributions, mainly long and thin scratches, and also includes some short and micro scratches. Figure 5 The image shown is a partial image from the metal scratch dataset.

[0032] S1.2 The acquired metal surface image data undergoes size unification preprocessing to adjust the image spatial dimensions of the metal surface image data to be the same, and scratch defect areas in the metal surface image data are labeled to generate scratch defect mask label files corresponding to the original images. In this embodiment, the spatial dimensions of all images are adjusted to 256×256 pixels. Pixel-level annotation tools are used to perform fine annotation of scratch defect areas in the preprocessed images, generating scratch defect mask label files corresponding to the original images. Figure 1 A corresponding scratch defect mask label file (Mask).

[0033] S1.3 The label files are divided into training and test sets according to a ratio to construct a scratch defect dataset. In this embodiment, 1575 preprocessed and labeled images and their corresponding label files are divided into training and test sets in an 8:2 ratio; the training set contains 1260 images for parameter learning and optimization of the network model; the test set contains 315 images for evaluating the segmentation accuracy and generalization performance of the trained model.

[0034] In step S1.3, random data augmentation is performed on each supervised training image and its mask label in the training set. Before the model officially starts training, random data augmentation is performed on the training set, and the specific methods of data augmentation include: random horizontal flipping, random vertical flipping, or a combination of horizontal and vertical flipping operations.

[0035] S2. Train the model using the scratch defect dataset to obtain the MSDD-UNet model.

[0036] In step S2, the loss function of the MSDD-UNet model is a hybrid loss function that combines Focal loss and Dice loss. The hybrid loss function is used to dynamically weight the difficult-to-identify scratch samples during network training, and it is also used to evaluate the shape similarity between the segmented region and the real mask.

[0037] The hybrid loss function is: Formula (1); in, L is a hyperparameter used to balance focal loss and Dice loss; focal Let L be the Focal loss function; Dice The Dice loss function; The expression for the focal loss function is: Formula (2); The Dice loss function is expressed as follows: Formula (3); Among them, P t This represents the value of the i-th pixel in the model segmentation result; , t represents the probability that the model predicts for the target. i To supervise the value of the i-th pixel in the annotated scratch mask; For hyperparameters, when When set to 0, It degenerates into the original cross-entropy loss function; This is used to reduce the weight of easily identifiable samples while increasing the weight of difficult-to-identify samples; N is the total number of pixels in the image.

[0038] In this embodiment, the hybrid loss function simultaneously measures the similarity between the predicted segmented region and the ground truth labeled mask, thereby improving the model's adaptability to class imbalance problems and the segmentation accuracy for minor scratch defects. The hybrid loss function combines the sensitivity of the BCE loss function to pixel misclassification, the sensitivity of the Dice loss function to the degree of matching between the segmentation result and the ground truth scratch region, and the characteristic of the focal loss function to weighted processing of hard-to-identify samples.

[0039] S3. Input the image data of the metal surface to be segmented into the MSDD-UNet model, and perform feature extraction, frequency domain decomposition, self-attention feature fusion, decoding upsampling reconstruction, etc., to obtain pixel-level segmentation results of scratch defects on the metal surface. The MSDD-UNet model is as follows: Figure 2 As shown, the MSDD-UNet model is used to sequentially perform feature extraction based on efficient downsampling, self-attention feature fusion based on frequency domain decomposition, and decoding reconstruction on the metal surface images to be segmented, thereby outputting pixel-level segmentation results of metal surface scratch defects.

[0040] In step S3, the MSDD-UNet model includes encoder paths, decoder paths, and skip connection structures connecting encoder paths and decoder paths.

[0041] The encoder path will include a downsampling module that includes a spatial-to-depth module and a lightweight channel attention module, replacing the traditional max pooling operation. The efficient downsampling module, consisting of the spatial-to-depth module SPD and the lightweight channel attention module LCAM, can reduce the spatial resolution of the feature map while preserving as much image detail as possible.

[0042] like Figure 3As shown, in the downsampling module, the input feature map is sliced ​​to divide it into several sub-feature maps in the spatial dimension. The spatial-to-depth module then concatenates the sub-feature maps in the depth dimension to obtain a concatenated feature map. The lightweight channel attention module assigns channel weights to each channel of the concatenated feature map. Based on the channel weights, the depth of the concatenated feature map is restored to its original size through a convolution with a kernel size of (1, 1) to downsample the feature map during the encoding process.

[0043] The spatial-to-depth module is used to stitch feature slices from the spatial dimension to the channel dimension to preserve complete detail information. For example... Figure 4 As shown, the lightweight channel attention module is used to enhance the importance weights of each channel feature in the metal surface image data. The lightweight nature is mainly reflected in the fact that the MLP of this module has only a single node.

[0044] The feature map size is During the transformation from S-space to depth, the slicing operation on the feature map is as follows: Formula (4); Where X is the feature map; The i-th row and j-th column of the output f of the slicing operation are represented; scale is the downsampling scaling factor.

[0045] For the input feature map, a slicing operation is first used to divide the feature map into four parts in the spatial dimension, each part being 1 / 4 the size of the original feature map. Simultaneously, a spatial-to-depth module is used to concatenate the four sliced ​​feature maps in the depth dimension, increasing the depth to four times that of the original feature map. Then, a lightweight channel attention module is used to assign weights to each channel of the feature map. Finally, a convolution with a kernel size of (1, 1) is used to restore the depth of the feature map to its original size. Through this processing mode, the length and width of the input feature map are reduced to half their original size, while the depth dimension remains unchanged. This module is designed so that MSDD-UNet can effectively downsample the feature map during the encoding process, thereby replacing the max-pooling operation used in the original U-Net.

[0046] In the skip connection structure, a cross-channel self-attention mechanism based on frequency domain decomposition is set up.

[0047] like Figure 5 As shown, in the self-attention mechanism, the features extracted from the encoder path are implicitly decomposed into low-frequency components representing global information and high-frequency components carrying local details through a convolution with a kernel size of (1, 1). Self-attention calculation is performed on the low-frequency components along the channel dimension to extract global features. The global features and high-frequency components are then concatenated and fused with the feature map of the decoder path to obtain fused features.

[0048] 8. A method for segmenting scratch defects on a metal surface according to claim 7, characterized in that: the self-attention calculation is as follows: Formula (5); Where X is the input feature map; Q, K, V and , , These represent the outputs and weights of the query, key, and value fully connected layers, respectively. is the transpose of Q, Softmax is the normalized exponential function, and Attention is the output of the attention calculation.

[0049] In this embodiment, the feature map output by the convolutional encoder can be regarded as a mixture of high-frequency and low-frequency features. Therefore, before the encoding-decoding feature fusion, a pair of convolutions with a kernel size of (1, 1) are introduced to implicitly decompose the feature map extracted by the encoder into low-frequency and high-frequency components. Then, the low-frequency components are input into the self-attention module to extract their global features, and then concatenated with the high-frequency components and the decoder feature map. By replacing the cross-spatial dimension calculation with the cross-channel dimension calculation of formula (5), the computational load of the self-attention mechanism is reduced, and the global information can be implicitly encoded effectively. Thus, while suppressing the generation of redundant information in the self-attention block, the dependency relationship between the low-frequency component pixels is established, thereby obtaining efficient global features, forming dense contextual information, and emphasizing the target region through the high-frequency components.

[0050] In the decoder path, the fused features are upsampled and reconstructed step by step to restore spatial resolution and obtain pixel-level segmentation results.

Claims

1. A method for segmenting scratch defects on a metal surface, characterized in that, Includes the following steps: S1. Obtain the metal surface image data to be segmented, and preprocess the metal surface image data to construct a scratch defect dataset for model training and model testing; S2. Train the model using the scratch defect dataset to obtain the MSDD-UNet model; S3. Input the metal surface image data to be segmented into the MSDD-UNet model, and perform feature extraction, frequency domain decomposition, self-attention feature fusion, decoding upsampling reconstruction in sequence to obtain pixel-level segmentation results of metal surface scratch defects.

2. The method for segmenting scratch defects on a metal surface according to claim 1, characterized in that: In step S1, the preprocessing step for the metal surface image data includes: S1.1 Acquire original images containing surfaces of various metals and alloys under different lighting conditions, and after filtering and cropping the original images, obtain metal surface image data containing scratches and defects; S1.2 Perform size unification preprocessing on the acquired metal surface image data to adjust the image space size of the metal surface image data to be the same, and mark the scratch defect areas in the metal surface image data to generate a scratch defect mask label file corresponding to the original image; S1.3 Divide the label files into training and testing sets according to the proportion to construct a scratch defect dataset.

3. The method for segmenting scratch defects on a metal surface according to claim 2, characterized in that: In step S1.3, random data augmentation is performed on each supervised training image and its mask label in the training set.

4. The method for segmenting scratch defects on a metal surface according to claim 1, characterized in that: In step S3, the MSDD-UNet model includes an encoder path, a decoder path, and a skip connection structure connecting the encoder path and the decoder path. The encoder path will include a downsampling module that includes a spatial-to-depth module and a lightweight channel attention module, replacing the traditional max pooling operation; In the skip connection structure, a cross-channel self-attention mechanism based on frequency domain decomposition is set; In the decoder path, the fused features are upsampled and reconstructed step by step to restore spatial resolution and obtain pixel-level segmentation results.

5. The method for segmenting scratch defects on a metal surface according to claim 4, characterized in that: In the downsampling module, the input feature map is sliced ​​to divide it into several sub-feature maps in the spatial dimension. The spatial-to-depth module then concatenates these sub-feature maps in the depth dimension to obtain a concatenated feature map. The lightweight channel attention module assigns channel weights to each channel of the concatenated feature map. Based on these channel weights, the depth of the concatenated feature map is restored to its original size using a convolution with a kernel size of (1, 1), thereby downsampling the feature map during the encoding process.

6. The method for segmenting scratch defects on a metal surface according to claim 5, characterized in that: The feature map size is During the transformation from S-space to depth, the slicing operation on the feature map is as follows: Official (1); Where X is the feature map; The i-th row and j-th column of the output f of the slicing operation are represented; scale is the downsampling scaling factor.

7. The method for segmenting scratch defects on a metal surface according to claim 4, characterized in that: In the self-attention mechanism, the features extracted from the encoder path are implicitly decomposed into low-frequency components representing global information and high-frequency components carrying local details through a convolution with a kernel size of (1, 1). Self-attention calculation is performed on the low-frequency components along the channel dimension to extract global features. The global features and the high-frequency components are then concatenated and fused with the feature map of the decoder path to obtain fused features.

8. The method for segmenting scratch defects on a metal surface according to claim 7, characterized in that: The self-attention is calculated as follows: Official (2); Where X is the input feature map; Q, K, V and , , These represent the outputs and weights of the query, key, and value fully connected layers, respectively. is the transpose of Q, Softmax is the normalized exponential function, and Attention is the output of the attention calculation.

9. The method for segmenting scratch defects on a metal surface according to claim 1, characterized in that: In step S2, the loss function of the MSDD-UNet model is a hybrid loss function that combines Focal loss and Dice loss. The hybrid loss function is used to dynamically weight the difficult-to-identify scratch samples during network training, and it is also used to evaluate the shape similarity between the segmented region and the real mask.

10. A method for segmenting scratch defects on a metal surface according to claim 9, characterized in that: The hybrid loss function is: Official (3); in, L is a hyperparameter used to balance focal loss and Dice loss; focal Let L be the Focal loss function; Dice The Dice loss function; The expression for the focal loss function is: Official (4); The Dice loss function is expressed as follows: Official (5); Among them, P t This represents the value of the i-th pixel in the model segmentation result; , t represents the probability that the model predicts for the target. i To supervise the value of the i-th pixel in the annotated scratch mask; For hyperparameters, when When set to 0, It degenerates into the original cross-entropy loss function; This is used to reduce the weight of easily identifiable samples while increasing the weight of difficult-to-identify samples; N is the total number of pixels in the image.

Images