A Deep Learning-Based Classification Method for Surface Defects in Metal Parts

Abstract

This invention discloses a deep learning-based method for classifying surface defects in metal parts. First, a customized data augmentation process is designed, integrating... Gamma Correction, color jitter and Laplace Sharpening was performed to improve the model's adaptability to changes in lighting and defect edges. Then, a spatial attention dynamic fusion network was constructed to... ResNet101 Using a backbone network, key features are selected through channel and spatial attention mechanisms, and dynamic fusion is achieved by combining multi-scale feature pyramids, enhancing the ability to locate and represent defective regions. Finally, an improved loss function is employed during training, combined with label smoothing and class weighting, to alleviate the data imbalance problem. The beneficial effects of this invention are: enabling the model to adaptively focus on subtle defective regions, thereby significantly improving classification accuracy; and enhancing the model's robustness and generalization ability under reflective interference by simulating different lighting and shooting conditions.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent industrial manufacturing, and in particular to a method for classifying surface defects of metal parts based on deep learning. Background Technology

[0002] The surface quality of high-precision metal parts is a key factor affecting product assembly accuracy and service life. Metal part surface defect classification is a product of the deep integration of computer vision and intelligent industrial manufacturing, and has significant application value in fields such as machining, automobile manufacturing, and precision measurement. In traditional production processes, defect detection of metal parts mainly relies on manual visual inspection, which has limitations such as low efficiency, inconsistent standards, and susceptibility to subjective factors, making it difficult to meet the requirements of high-speed, high-consistency quality inspection in modern production lines. Although deep learning technology has been widely applied in the field of metal part defect detection, traditional models often cannot automatically focus on key image regions related to the classification task, frequently requiring additional labeled feature points to assist model training, resulting in low feature extraction efficiency and susceptibility to inconsistent labeling standards.

[0003] In recent years, with the widespread application of deep learning technology in computer vision, classification models based on convolutional neural networks (such as AlexNet and VGG) have been applied to object classification tasks. However, most of these models are designed for object classification tasks with significant features and relatively obvious defect morphologies. Their model structures generally lack effective fusion of multi-scale features, often resulting in poor performance when detecting and classifying subtle defects on metal surfaces (such as scratches and burrs). In addition to their subtle features and diverse morphologies, surface defects on metal parts are often affected by lighting and reflections, leading to insufficient recognition accuracy and weak robustness in practical industrial scenarios.

[0004] In summary, surface defects in metal parts have the following characteristics: First, the defects are minute in shape, such as scratches and burrs, which are small in size and have low contrast; second, the surface of metal parts is easily affected by reflections, interfering with the extraction and identification of defect features. Traditional deep learning methods face two challenges in such tasks: on the one hand, the model struggles to focus on minute defect areas, leading to a high false negative rate; on the other hand, they suffer from poor robustness and generalization, with the false positive rate often significantly increasing due to variations in lighting and shooting angles. Summary of the Invention

[0005] This invention aims to overcome the shortcomings of existing classification models in the task of classifying surface defects of metal parts, and provides a deep learning-based method for classifying surface defects of metal parts that can improve the classification accuracy, recognition robustness and generalization ability of the model.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: The deep learning-based method for classifying surface defects in metal parts includes the following steps: (1) Construct a metal parts surface image dataset: Divide the input metal parts surface image dataset into a training set and a test set. Use customized data augmentation processing on the training set to simulate the fluctuations of real-world scenes and enhance defect features; perform standardization processing on the test set, including target scaling, grayscale conversion and normalization processing. (2) Constructing a spatial attention dynamic fusion network: Input the processed image tensor, extract features through ResNet101, and input the extracted feature map into the constructed spatial attention dynamic fusion network. The specific operation process includes: using channel enhancement to filter key feature channels, using the spatial attention module to enhance the features of defective regions and suppress irrelevant region features, and using the dynamic fusion module to realize the dynamic global coordination and adaptive weight fusion of multi-dimensional features, and outputting the final feature map. (3) Model training: The image after the customized data augmentation process in step (1) is input into the spatial attention dynamic fusion network constructed in step (2), and the learning rate decay and loss function optimization are combined to finally obtain the metal part surface defect classification model; (4) Classification of surface defects of metal parts: Deployment steps (3) The trained metal part surface defect classification model is used to input the metal surface image to be classified into the ResNet101 network, and then the customized enhanced data is input into the spatial attention dynamic fusion mechanism. Finally, the feature map output by the spatial attention dynamic fusion is input into the fully connected layer, and then the classification head of the fully connected layer obtains the classification result, thereby realizing the automatic classification of surface defects of metal parts.

[0007] This invention designs a network architecture based on Spatial Attention with Dynamic Fusion (SADF) and combines it with a deep convolutional neural network, enabling the model to adaptively focus on subtle defect areas, thereby significantly improving classification accuracy. Furthermore, this invention proposes a customized data processing method for metal part images, enhancing the model's robustness and generalization ability under reflective interference by simulating different lighting and shooting conditions.

[0008] As a preferred option, in step (1), specifically: collect images of the surface of metal parts through public channels and collect them independently to construct a dataset of surface defects of metal parts. The images in the dataset are grayscale images of level [0, 255]. According to the types and levels of defects on the surface of metal parts, the dataset is divided into training set and test set according to the proportion.

[0009] Preferably, in step (1), the customized data augmentation processing method is as follows: (11) The dimensions are length and width. Original image I src Scale to match the dimensions of the target image Same intermediate image Then to All pixels were processed using bicubic interpolation to obtain The grayscale value of the middle pixel; medium pixel grayscale value Defined as: in Defined as: in, , These represent the pixel value rounded down in the x-direction and the pixel value rounded down in the y-direction, respectively. , Let i and j represent the bicubic interpolation weight coefficients for pixel steps of size i in the x-direction and j in the y-direction, respectively. For a step size of t, the bicubic interpolation weight function is... Defined as: To adapt the model to differences in reflectivity under different lighting conditions, Normalization process The interval is used to obtain the normalized gray value. : To adjust the reflectivity, Gamma correction was applied to obtain the corrected grayscale value. : in, It is the Gamma coefficient, and its value range is... ;when When, it enhances shadow details and adapts to strong light environments; when At the same time, it enhances details in bright areas and adapts to low-light environments; (12) The corrected grayscale values Obtained through brightness adjustment : in, This is the brightness offset, and its value range is... ; To avoid the model relying on fixed color features, further... Contrast adjustment was used to simulate changes in lighting during the shooting process, resulting in grayscale values ​​after contrast adjustment. : in, The average gray level of the image; This is the contrast offset, and its value range is... ; To enhance the contrast of the surface edges of metal parts and make it easier for the model to capture defect features, Laplace sharpening is performed to obtain : Where k is the sharpening intensity; edge detection operator Defined as: .

[0010] Preferably, in step (2), ResNet101 is used as the backbone network architecture. A spatial attention module is constructed, combined with a dynamic fusion module, and these modules interact and fuse among the backbone network architectures. Specifically: (21) Constructing a spatial attention mechanism: The feature map output by the backbone network ResNet101 is filtered out for key feature channels with defects through channel enhancement. The key channels are pooled to obtain two description vectors. The description vectors are then compressed and activated through two layers of MLP. Feature fusion and Sigmoid activation are then used to obtain the feature map after spatial feature enhancement. (22) Construct a dynamic fusion mechanism: Perform bicubic interpolation on the feature map after spatial feature enhancement to generate multi-scale features, then perform multi-scale splicing and fusion convolution on the feature vectors at different scales, and compress the spliced ​​and fused feature vectors into fixed-length feature vectors by global average pooling, and output the final feature map.

[0011] Preferably, the specific operation method in step (21) is as follows: Image tensors after customized data processing The output feature vector is obtained after normalization processing by the backbone network ResNet101. ,in To further enhance the customized data, channel enhancement is used to filter out key feature channels of defects, which are then used as input feature maps for the spatial attention module. : Where R represents All elements in the array are real numbers; B represents the batch size; C represents the number of feature channels; H and W represent the length and width of the feature map. The specific operation of screening key feature channels for defects is to convert the feature vectors... After reactivation using the shared two-layer MLP compression channels, a global average feature vector is obtained. With the global maximum eigenvector They are defined as follows: Where ReLU represents activation processing; Conv represents channel compression processing; and They represent respectively to The two channel description vectors after performing global average pooling and global max pooling; Output feature vector of MLP and The spatial attention mask is obtained by summing the results and then activating them with a Sigmoid function. : Then, the feature vectors after ResNet101 normalization are processed. Further weighted fusion yields the input feature vector. : Through the above operations, the defective area corresponds to... A weight close to 1 enhances defect features; background and reflective areas correspond to... Weights close to 0 suppress irrelevant region features; Finally, the input feature map will be displayed. Spatial attention mask Element-wise multiplication is performed to enhance the spatial dimension of features, resulting in an output feature map. .

[0012] Preferably, the specific operation method in step (22) is as follows: By setting model=bilinear and the upsampling factor s to 2, 4, and 8 respectively, the following parameters are obtained: Bilinear interpolation upsampling generates three feature maps at different scales, which are then combined with the original feature maps to form a four-scale feature pyramid; scale feature vectors Defined as: Among them, the upsampling scale factor Corresponding generation Scaled feature map, original feature map As a benchmark The sampling mode bilinear is bilinear sampling. To ensure that the model can learn defect features at every scale, the scale feature vector is used. The four attention feature maps after alignment The feature map is obtained by splicing the channels. : in, This is a channel-level splicing operation; Then Each scale is equipped with an independent Convolution, fusing multi-scale features and reducing dimensionality, yields a feature map after fusing multi-dimensional features. : in, , To fuse convolutional layer parameters; Then, all channels are fused and compressed to obtain the final output feature map. : in, This is the global average pooling function.

[0013] Preferably, in step (3), the image after the customized data augmentation processing in step (1) is input into the spatial attention dynamic fusion mechanism network constructed in step (2), and learning rate decay and loss function optimization are combined. The specific optimization strategy is to add label smoothing, class weighting and modulation to the original loss function Focal Loss to construct a new loss function. To focus on difficult-to-classify defect samples, the model is trained iteratively for 1 epoch until convergence, and finally a surface defect classification model for metal parts is obtained.

[0014] Preferably, the specific operation method in step (3) is as follows: Input the training set data into the network constructed through step (2) using the spatial attention dynamic fusion mechanism. To reduce the problem of data imbalance, label smoothing and class weighting are added to the original loss function Focal Loss to obtain a new loss function. To focus on difficult-to-classify defective samples: Where i represents the number of categories; Indicates category weight; Indicates the modulation coefficient; prediction probability Defined as: in, This represents the predicted probability that the i-th sample belongs to the k-th class after label smoothing. This is the label smoothing coefficient.

[0015] Preferably, the specific operation method in step (4) is as follows: Before actual use, the classification model trained in step (3) is deployed, the metal surface image to be classified is input into the ResNet101 network, and the data after customization and enhancement is input into the spatial attention dynamic fusion mechanism. Finally, the classification result is output through the fully connected layer to realize the automatic classification of surface defects of metal parts.

[0016] The beneficial effects of this invention are: (1) A customized data augmentation strategy designed for the characteristics of metal parts, such as surface reflection and minor defects. By adjusting the brightness, the reflection interference of the metal surface and the difference in light intensity are reproduced, which solves the problem of traditional models misjudging the background as defects due to reflection. In addition, by using defect enhancement operations to highlight the edge features of scratches and the contour features of burrs and randomly erase them, the problem of low contrast and easy to ignore minor defects of metal parts in the image is solved, and the model is prevented from relying too much on local features, thereby improving the recognition and generalization ability of minor defects.

[0017] (2) Traditional models have difficulty accurately focusing on defect areas and are prone to misjudging reflective areas on metal surfaces as defect areas. By combining spatial attention mechanisms to generate attention masks, high weights are assigned to defect areas and low weights are assigned to irrelevant areas such as background and reflections. This achieves enhancement of defect areas and suppression of irrelevant areas in the feature map, effectively reducing the interference of irrelevant information and significantly improving the accurate localization of defect areas.

[0018] (3) To address the large size differences in surface defect features of metal parts, a multi-scale feature pyramid is generated, with an independent attention learning branch configured for each scale. This enables the model to capture subtle textures of minor defects at small scales, allowing it to autonomously learn the weights of defect features at different scales and avoid the limitations of single-scale surface features in identifying defects at different scales. Furthermore, through feature normalization and channel splicing fusion, attention features at different scales are integrated into a unified fusion feature. This preserves the detailed information at multiple scales and achieves feature refinement through convolutional dimensionality reduction, thereby improving the discriminative power of the features and providing reliable results for the classification head. Attached Figure Description

[0019] Figure 1This is a flowchart of a deep learning-based method for classifying surface defects in metal parts. Figure 2 It is a flowchart of customized data augmentation processing; Figure 3 This is a flowchart of the spatial attention mechanism; Figure 4 This is a flowchart of the dynamic fusion mechanism; Figure 5 This is a flowchart of the loss function optimization process. Detailed Implementation

[0020] The present invention will now be further described with reference to the accompanying drawings and specific embodiments.

[0021] The deep learning-based method for classifying surface defects in metal parts includes the following steps: (1) Constructing a metal parts surface image dataset: The input metal parts surface image dataset is divided into a training set and a test set. The test set is standardized, including target scaling, grayscale conversion, and normalization. Customized data augmentation is applied to the training set to simulate fluctuations in real-world scenarios and enhance defect features. Specifically, metal parts surface images are collected. Through public channels and self-collection, a metal parts surface defect image dataset is constructed. The images in the dataset are grayscale images of level [0, 255]. Based on the defect types, defect levels, and annotations of the metal parts surface, the dataset is divided into a training set and a validation set according to the proportions. The customized data augmentation method is as follows: (11) The dimensions are length and width. Original image I src Scale to match the dimensions of the target image Same intermediate image Then to All pixels were processed using bicubic interpolation to obtain The grayscale value of a pixel. medium pixel grayscale value Defined as: in Defined as: in, , These represent the pixel value rounded down in the x-direction and the pixel value rounded down in the y-direction, respectively. , Let i and j represent the bicubic interpolation weight coefficients for pixel steps of size i in the x-direction and j in the y-direction, respectively. For a step size of t, the bicubic interpolation weight function is... Defined as: To adapt the model to differences in reflectivity under different lighting conditions, Normalization process The interval is used to obtain the normalized gray value. : To adjust the reflectivity, Gamma correction was applied to obtain the corrected grayscale value. : in, It is the Gamma coefficient, and its value range is... (Values ​​are obtained using random sampling). When When, it enhances shadow details and adapts to strong light environments; when At the same time, it enhances details in bright areas and adapts to low-light environments.

[0022] (12) The corrected grayscale values Obtained through brightness adjustment : in, This is the brightness offset, and its value range is... (Values ​​are obtained using a random sampling method).

[0023] To avoid the model relying on fixed color features, further... Contrast adjustment was used to simulate changes in lighting during the shooting process, resulting in grayscale values ​​after contrast adjustment. : in, The average gray level of the image; This is the contrast offset, and its value range is... (Values ​​are obtained using a random sampling method).

[0024] To enhance the contrast of edges such as scratches and burrs on the surface of metal parts, making it easier for the model to capture defect features, Laplace sharpening is performed to obtain : Among them, sharpening intensity Edge detection operator Defined as: .

[0025] (2) Constructing a spatial attention dynamic fusion network: Input the processed image tensor The feature map is extracted by ResNet101 and then input into the constructed spatial attention dynamic fusion network. The specific operation process includes: using channel enhancement to select key feature channels, using the spatial attention module to enhance the features of defective regions and suppress irrelevant features, and using the dynamic fusion module to realize the dynamic global coordination and adaptive weight fusion of multi-dimensional features, and outputting the final feature map.

[0026] In terms of network architecture design, this invention uses ResNet101 as the backbone network architecture, constructs a spatial attention module, combines it with a dynamic fusion module, and integrates and fuses these modules across the backbone network architecture. Specifically: (21) Constructing a spatial attention mechanism: First, the feature map output by the backbone network ResNet101 is filtered to identify key feature channels with defects through channel enhancement. Then, the selected key channels are pooled to obtain two description vectors, which are then compressed and activated through two layers of MLP. After that, feature fusion and Sigmoid activation are further applied to obtain the feature map with spatial feature enhancement. The specific operation method is as follows: Image tensors after customized data processing The output feature vector is obtained after normalization processing by the backbone network ResNet101. Then, channel enhancement is used to filter out the key feature channels of defects, which are then used as the input feature maps for the spatial attention module. : Where R represents All elements in the array are real numbers; B represents the batch size; C represents the number of feature channels; H and W represent the length and width of the feature map.

[0027] The specific operation of screening key feature channels for defects is to convert the feature vectors... After compressing the number of channels and reactivating them using a shared two-layer MLP (Multilayer Perceptron), a globally averaged feature vector is obtained. With the global maximum eigenvector They are defined as follows: Where ReLU represents activation processing; Conv represents channel compression processing; and They represent respectively to The two channel description vectors after performing global average pooling and global max pooling.

[0028] Output feature vector of MLP and The spatial attention mask is obtained by summing the results and then activating them with a Sigmoid function. : Then, the feature vectors after ResNet101 normalization are processed. Further weighted fusion yields the input feature vector. : Through the above operations, the defective area corresponds to... A weight close to 1 enhances defect features; background and reflective areas correspond to... Weights close to 0 suppress irrelevant regional features.

[0029] Finally, the input feature map Spatial attention mask Element-wise multiplication is performed to enhance the spatial dimension of features, resulting in an output feature map. : .

[0030] (22) Constructing a dynamic fusion mechanism: First, the feature map after spatial feature enhancement is subjected to bicubic interpolation to generate multi-scale features. Then, the feature vectors at different scales are concatenated and fused using multi-scale convolution. Finally, the concatenated and fused feature vectors are compressed into fixed-length feature vectors using global average pooling, and the final feature map is output. The specific operation method is as follows: By setting model=bilinear (sampling mode is bilinear), and the upsampling factors s are 2, 4, and 8 respectively, the following applies: Bilinear interpolation upsampling generates three feature maps at different scales, which are then combined with the original feature maps to form a four-scale feature pyramid. (Scale feature vector) Defined as: Among them, the upsampling scale factor Corresponding generation Scaled feature map, original feature map As a benchmark The sampling mode bilinear is bilinear sampling.

[0031] To ensure that the model can learn defect features at every scale, the scale feature vector is used. The four attention feature maps after alignment The feature map is obtained by splicing the channels. : in, This is a channel-dimensional splicing operation.

[0032] Then Each scale is equipped with an independent Convolution, fusing multi-scale features and reducing dimensionality, yields a feature map after fusing multi-dimensional features. : in, , To fuse the parameters of the convolutional layers.

[0033] Then, all channels are fused and compressed to obtain the final output feature map. : in, This is the global average pooling function.

[0034] (3) Model training: The image after the customized data augmentation processing in step (1) is input into the spatial attention dynamic fusion network constructed in step (2), and learning rate decay and loss function optimization are combined. The specific optimization strategy is to add label smoothing, class weighting and modulation to the original loss function Focal Loss to construct a new loss function. To focus on difficult-to-classify defect samples, the model is trained iteratively through n epochs (n rounds) until convergence, ultimately yielding a surface defect classification model for metal parts. The specific operation method is as follows: The training set data is input into the network constructed through step (2) using the spatial attention dynamic fusion mechanism. In addition, to reduce the problem of data imbalance, label smoothing and class weighting are added to the original loss function Focal Loss to obtain a new loss function. To focus on difficult-to-classify defective samples: Where i represents the number of categories; Indicates category weight; Indicates the modulation coefficient; prediction probability Defined as: in, This represents the predicted probability that the i-th sample belongs to the k-th class after label smoothing. This is the label smoothing coefficient.

[0035] (4) Classification of surface defects of metal parts: Before actual use, the metal part surface defect classification model trained in step (3) is deployed, the metal surface image to be classified is input into the ResNet101 network, and the data after customization enhancement is input into the spatial attention dynamic fusion mechanism. Finally, the feature map output by the spatial attention dynamic fusion is input into the fully connected layer, and then the classification head of the fully connected layer obtains the classification result, thereby realizing the automatic classification of surface defects of metal parts.

[0036] Based on the above method and the accompanying drawings, an implementation example is as follows: like Figure 1 The flowchart shown illustrates the overall process of a deep learning-based method for classifying surface defects in metal parts. This includes customized data augmentation, constructing a spatial attention mechanism, building a dynamic fusion mechanism, and optimizing the loss function of the fully connected layer. The specific steps are as follows: Step 1: Customized Data Augmentation Images of metal parts' surfaces were collected, and a large-scale image dataset of surface defects was constructed using publicly available and self-collected data. The images were then categorized and labeled according to the damage level of the surface defects. The public datasets included images from the NEU-CLS Dataset, NEU-DET Dataset, GC10-DET Dataset, and Metal-AUTH. The image data was then categorized... The dataset is divided into training and testing sets for model training and performance testing.

[0037] Among them, customized data augmentation strategies such as Figure 2 As shown, the original image is input using data divided into training set portions. First, the dimensions are... The original image Scale to the same dimensions as the target image Consistent intermediate images And then All pixels are subjected to bicubic interpolation to obtain the target image. grayscale values ​​of each pixel Secondly, in order to solve the problems of excessive surface reflection and excessive darkness in low-light environments, [further measures were taken]. Perform Gamma correction when At times, it enhances bright areas and compensates for excessively dark surfaces in low-light environments; when At the same time, enhance the shadows to reduce excessive reflections from the metallic surface. Furthermore, due to changes in lighting during shooting, avoid relying on fixed color characteristics for the model by adjusting the corrected grayscale values. Color dithering was used to adjust brightness and contrast. Finally, to enhance the edge contrast of scratches, burrs, and other defects on the metal parts' surface, making it easier for the model to capture these features, the grayscale values ​​after color dithering were adjusted. Further Laplace sharpening is then performed, ultimately outputting customized enhanced data. .

[0038] Step 2: Construct a spatial attention dynamic fusion network Step 2.1: Constructing a spatial attention mechanism like Figure 3 As shown, the image customized in step 1 is processed by a ResNet101 network to extract features and output a feature map. The key feature channels are selected using a channel feature enhancement method and used as input features for the constructed spatial attention module. The specific steps are as follows: First, the feature map... Global average pooling and global max pooling are performed to obtain and Secondly, the feature vector after pooling is compressed and reactivated through two shared perceptron channels to obtain the globally averaged feature vector. With the global maximum eigenvector Then, channel-weighted fusion and Sigmoid activation are further performed to obtain channel attention weights. To make the defective area A value close to 1 enhances defect characteristics; it also improves the appearance of background, reflective areas, etc. Values ​​close to 0 suppress irrelevant region features. Finally, to achieve feature enhancement in the spatial dimension, the feature map... Spatial attention weights The output feature map is obtained by performing element-wise multiplication. .

[0039] Step 2.2: Construct a dynamic fusion mechanism like Figure 4 As shown, the feature vector output by the spatial attention mechanism in step 2.1 will be... As input features for the dynamic fusion module. First, for In sampling factor Bilinear interpolation is performed on the top feature map, and combined with the original feature map to form a corresponding 4-scale feature map pyramid. In addition, to ensure that the model can learn defect features at each scale, a separate feature is provided for each scale. The convolution process is performed. Next, all convolutionally processed feature vectors are concatenated and fused to obtain the feature map. Finally, the feature maps will be spliced ​​and fused through channels. Perform global average pooling to obtain the final output feature map. .

[0040] Step 3: Model Training The image processed by the customized data augmentation in step 1 is input into the spatial attention dynamic fusion mechanism network constructed in step 2, and optimized using learning rate decay and loss function. Furthermore, to reduce the problem of data imbalance, the following measures are taken: Figure 5 The loss function optimization strategy is shown. A new loss function is constructed by adding label smoothing, class weighting, and modulation to the original Focal Loss. To focus on difficult-to-classify defect samples, the model is trained iteratively for n epochs (n rounds) until convergence, and finally a surface defect classification model for metal parts is obtained.

[0041] Step 4: Classification of surface defects in metal parts To verify the effectiveness of this method, comparative experiments were conducted on a unified dataset of surface defects in metal parts. ResNet101 was used as the baseline model, and AlexNet, VGG16, EfficientNetV2, and the EfficientNetV2+SADF model (which also incorporates an attention mechanism) were selected for comparison. Evaluation metrics included accuracy, precision, recall, and F1 score. The experimental results are shown in Table 1.

[0042] Table 1. Experimental results of surface defect classification for metal parts Continued from Table 1: Experimental Results of Surface Defect Classification for Metal Parts Experimental results show that the proposed method based on spatial attention dynamic fusion mechanism achieves superior performance across all metrics. Specifically, the accuracy of this method is improved by 9.8% compared to the baseline model ResNet101. Simultaneously, precision, recall, and F1 score are all significantly improved compared to the baseline model. This fully demonstrates that the proposed method, through the synergistic effect of spatial attention dynamic fusion mechanism and customized data augmentation strategy, can more effectively focus on the key features of defective regions, thereby significantly improving classification performance.

[0043] Compared to traditional convolutional neural networks such as AlexNet and VGG16, this method exhibits a more significant performance advantage, improving accuracy by 15.9% and 14.3%, respectively. This demonstrates that in complex industrial defect detection scenarios, deep networks combined with targeted enhancement and attention mechanisms are necessary and effective. Furthermore, compared to the improved model EfficientNetV2, which also employs the spatial attention dynamic fusion mechanism SADF, this method still achieves a 7.8% improvement in accuracy. This proves that the multi-scale feature dynamic fusion mechanism combined with spatial attention in this invention offers greater advantages in feature extraction and fusion.

[0044] Furthermore, this method exhibits balanced and excellent overall performance in terms of precision, recall, and F1 score. This indicates that while suppressing background interference and enhancing defect features, the method effectively balances false negatives and false negatives, making it more suitable for practical industrial testing environments with high reliability requirements.

[0045] In summary, the experimental results fully demonstrate that the method proposed in this invention, which combines customized data augmentation, spatial attention dynamic fusion network, and improved loss function, exhibits strong robustness, generalization ability, and accuracy when dealing with tasks that involve small inter-class differences, strong illumination interference, variable defect scales, and class imbalance. It provides an effective solution for the automated and high-precision classification of surface defects in metal parts.

Claims

1. A deep learning-based method for classifying surface defects in metal parts, characterized by: Specifically, the steps include the following: (1) Construct a metal parts surface image dataset: Divide the input metal parts surface image dataset into a training set and a test set. Use customized data augmentation processing on the training set to simulate the fluctuations of real-world scenes and enhance defect features; perform standardization processing on the test set, including target scaling, grayscale conversion and normalization processing. (2) Constructing a spatial attention dynamic fusion network: Input the processed image tensor, extract features through ResNet101, and input the extracted feature map into the constructed spatial attention dynamic fusion network. The specific operation process includes: using channel enhancement to filter key feature channels, using the spatial attention module to enhance the features of defective regions and suppress irrelevant region features, and using the dynamic fusion module to realize the dynamic global coordination and adaptive weight fusion of multi-dimensional features, and outputting the final feature map. (3) Model training: The image after the customized data augmentation process in step (1) is input into the spatial attention dynamic fusion network constructed in step (2), and the learning rate decay and loss function optimization are combined to finally obtain the metal part surface defect classification model; (4) Classification of surface defects of metal parts: Deploy the metal part surface defect classification model trained in step (3), input the metal surface image to be classified into the ResNet101 network, input the customized enhanced data into the spatial attention dynamic fusion mechanism, and finally input the feature map output by the spatial attention dynamic fusion into the fully connected layer. Then the classification head of the fully connected layer obtains the classification result, thereby realizing the automatic classification of surface defects of metal parts.

2. The method for classifying surface defects of metal parts based on deep learning according to claim 1, characterized in that, In step (1), specifically: collect images of the surface of metal parts, construct a dataset of surface defects of metal parts through public channels and self-collection, the images in the dataset are grayscale images of level [0, 255], and divide the dataset into training set and test set according to the defect type, defect level classification and labeling of the surface of metal parts.

3. The deep learning-based method for classifying surface defects in metal parts according to claim 1 or 2, characterized in that, in In step (1), the customized data augmentation processing method is as follows: (11) The dimensions are length and width. Original image I src Scale to match the dimensions of the target image Same intermediate image Then to All pixels were processed using bicubic interpolation to obtain grayscale value of medium pixel To adapt the model to differences in reflectivity under different lighting conditions, Normalization process The interval is used to obtain the normalized gray value. ; To adjust the reflectivity, Gamma correction was applied to obtain the corrected grayscale value. : in, It is the Gamma coefficient, and its value range is... ;when When, it enhances shadow details and adapts to strong light environments; when At the same time, it enhances details in bright areas and adapts to low-light environments; (12) The corrected grayscale values Obtained through brightness adjustment To avoid the model relying on fixed color features, further... Contrast adjustment was used to simulate changes in lighting during the shooting process, resulting in grayscale values ​​after contrast adjustment. To enhance the contrast of the edges of metal parts and make it easier for the model to capture defect features, Laplace sharpening is performed to obtain .

4. The method for classifying surface defects of metal parts based on deep learning according to claim 1, characterized in that, in In step (2), ResNet101 is used as the backbone network architecture. A spatial attention module is constructed, combined with a dynamic fusion module, and these modules interact and fuse among the backbone network architectures. Specifically: (21) Constructing a spatial attention mechanism: The feature map output by the backbone network ResNet101 is filtered out for key feature channels with defects through channel enhancement. The key channels are pooled to obtain two description vectors. The description vectors are then compressed and activated through two layers of MLP. Feature fusion and Sigmoid activation are then used to obtain the feature map after spatial feature enhancement. (22) Construct a dynamic fusion mechanism: Perform bicubic interpolation on the feature map after spatial feature enhancement to generate multi-scale features, then perform multi-scale splicing and fusion convolution on the feature vectors at different scales, and compress the spliced ​​and fused feature vectors into fixed-length feature vectors by global average pooling, and output the final feature map.

5. The method for classifying surface defects of metal parts based on deep learning according to claim 4, characterized in that, in In step (21), the specific operation method is as follows: Image tensors after customized data processing The output feature vector is obtained after normalization processing by the backbone network ResNet101. ,in To further enhance the customized data, channel enhancement is used to filter out key feature channels of defects, which are then used as input feature maps for the spatial attention module. ; The specific operation of screening key feature channels for defects is to convert the feature vectors... After reactivation using the shared two-layer MLP compression channels, a global average feature vector is obtained. With the global maximum eigenvector ; output feature vector of MLP and The spatial attention mask is obtained by summing the results and then activating them with a Sigmoid function. ; Then, the feature vectors after ResNet101 normalization are processed. Further weighted fusion yields the input feature vector. : Through the above operations, the defective area corresponds to... A weight close to 1 enhances defect characteristics; Background, reflective areas corresponding to Weights close to 0 suppress irrelevant region features; Finally, the input feature map will be displayed. Spatial attention mask Element-wise multiplication is performed to enhance the spatial dimension of features, resulting in an output feature map. .

6. The method for classifying surface defects of metal parts based on deep learning according to claim 5, characterized in that, in In step (22), the specific operation method is as follows: By setting model=bilinear and the upsampling factor s to 2, 4, and 8 respectively, the following parameters are obtained: Bilinear interpolation upsampling is used to generate feature maps at three different scales, which are then combined with the original feature maps to form a four-scale feature pyramid. scale Feature vector Defined as: Among them, the upsampling scale factor Corresponding generation Scaled feature map, original feature map As a benchmark The sampling mode bilinear is bilinear sampling. To ensure that the model can learn defect features at every scale, the scale feature vector is used. The four attention feature maps after alignment The feature map is obtained by splicing the channels. Then Each scale is equipped with an independent Convolution, fusing multi-scale features and reducing dimensionality, yields a feature map after fusing multi-dimensional features. ; Then, all channels are fused and compressed to obtain the final output feature map. .

7. The method for classifying surface defects of metal parts based on deep learning according to claim 1, characterized in that, in In step (3), the image after the customized data augmentation processing in step (1) is input into the spatial attention dynamic fusion mechanism network constructed in step (2), and learning rate decay and loss function optimization are combined. The specific optimization strategy is to add label smoothing, class weighting and modulation to the original loss function Focal Loss to construct a new loss function. To focus on difficult-to-classify defect samples, the model is trained iteratively for 1 epoch until convergence, and finally a surface defect classification model for metal parts is obtained.

8. The method for classifying surface defects of metal parts based on deep learning according to claim 7, characterized in that, in In step (3), the specific operation method is as follows: Input the training set data into the network constructed through step (2) using the spatial attention dynamic fusion mechanism. To reduce the problem of data imbalance, label smoothing and class weighting are added to the original loss function Focal Loss to obtain a new loss function. To focus on difficult-to-classify defective samples: Where i represents the number of categories; Indicates category weight; Indicates the modulation coefficient; prediction probability Defined as: in, This represents the predicted probability that the i-th sample belongs to the k-th class after label smoothing. This is the label smoothing coefficient.

Images