A surface defect detection method for aluminum profiles based on DMSA-Swin-Unet network model

Abstract

The present invention discloses a method for detecting surface defects of aluminum profiles based on a DMSA-Swin-Unet network model, which relates to the field of industrial detection technology. The method comprises the following steps: pre-processing the collected surface image of the aluminum profile; using a convolution operation with a predetermined step size to divide the image into image blocks and perform linear projection to generate an embedded feature sequence; inputting the features into a Swin Unet encoder module to extract multi-level local and global features, and the multi-layer nested Swin Transformer module adopts an alternating window attention and shifted window attention mechanism; during the decoding process, the coding features of corresponding scales are fused through a jump connection; a DMSA module is configured after the jump connection to enhance the ability to perceive contextual information after fusion; and step-by-step upsampling is performed through image block expansion to output a pixel-level defect segmentation map that is consistent with the original image size. The present invention has high detection accuracy, strong robustness, good adaptability, and strong deployability, and is suitable for automatic identification and intelligent quality inspection scenarios of aluminum profile surface defects in industrial sites.

Description

Technical Field

[0001] The present invention relates to the field of industrial detection technology, and in particular to a method for detecting surface defects of aluminum profiles based on a DMSA-Swin-Unet network model. Background Art

[0002] In industrial production, aluminum profiles are an important material widely used in construction, transportation, electronics and other fields. Their surface quality directly affects the performance and appearance of the final product. However, during the production process, defects such as cracks, peeling, and scratches often appear on the surface of aluminum profiles. At present, surface defect detection of aluminum profiles is still mainly based on manual visual inspection, which has problems such as low efficiency, strong subjectivity, and high error rate, making the efficiency of quality inspection difficult to control. Therefore, the research of efficient and accurate surface defect detection technology has become an important topic in industrial automation and intelligent manufacturing. Traditional image processing methods rely on preset rules and features, and are difficult to adapt to the complex and changing defect types and actual environmental noise.

[0003] In recent years, deep learning has achieved rapid progress in fields such as image recognition. Researchers have proposed online visual inspection methods based on deep learning, using convolutional neural networks (CNNs) to automatically extract image features. This overcomes the traditional methods' reliance on feature engineering and improves the robustness and versatility of detection. For example, the improved YOLOv3 algorithm incorporates an attention mechanism and a multi-scale feature fusion structure, enhancing the detection capabilities of complex textures and multi-scale defects. However, while CNNs excel at extracting local features, they have inherent disadvantages in modeling global image information and long-range dependencies, which can lead to incomplete defect recognition.

[0004] Therefore, technicians in this field are committed to developing a method for detecting surface defects of aluminum profiles based on the DMSA-Swin-Unet network model. Summary of the Invention

[0005] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is that the traditional methods have weak detection capabilities and poor robustness for complex, tiny, and diverse defect features.

[0006] To achieve the above object, the present invention provides a method for detecting surface defects of aluminum profiles based on a DMSA-Swin-Unet network model, the method comprising the following steps:

[0007] S101: Preprocessing the collected aluminum profile surface image, wherein the preprocessing includes image size normalization, color enhancement, and data standardization;

[0008] S103: performing a convolution operation with a predetermined step length on the preprocessed image, dividing the image into image blocks and performing linear projection to generate an embedded feature sequence;

[0009] S105: extracting multi-level local and global features through a Swin Unet encoder module, wherein the multi-layer nested Swin Transformer in the encoder adopts alternating window attention and shifted window attention mechanisms;

[0010] S107: In the decoding process of each layer, the coding features of the corresponding scale are fused through skip connections;

[0011] S109: configuring a DMSA module after the skip connection, wherein the DMSA module is composed of a channel attention module and a spatial attention module in parallel;

[0012] S111: performing step-by-step upsampling by image block expansion, and finally outputting a pixel-level defect segmentation map with the same size as the original image.

[0013] Furthermore, in step S105, the Swin Transformer module is a multi-layer nested structure, each layer including a window attention module and a shifted window attention module, the window attention module calculates self-attention in the local window, and the shifted window attention module interacts with cross-region features through window shifting.

[0014] Furthermore, the step S105 includes the following sub-steps:

[0015] S1051: Input feature map and calculate self-attention through the window attention module:

[0016]

[0017]

[0018] S1052: The shifted window attention module introduces cross-window feature dependencies to perform cross-region feature interaction:

[0019]

[0020]

[0021] in, For the The W-MSA module output of the Swin Transformer module of the layer, For the l The MLP layer output of the SwinTransformer module, For the +1 layer of Swin Transformer’s SW-MSA module output, For the l+1 The MLP layer output of the Swin Transformer module, LN is layer normalization, MLP is multi-layer perceptron, is the window multi-head self-attention, It is a shift window multi-head attention mechanism;

[0022] The self-attention mechanism is calculated as follows:

[0023]

[0024] in, Attention is the attention coefficient of the key-value pair, is the query matrix, is the bond matrix, is the value matrix, is the transposed matrix of matrix K, is the normalized exponential function, is the dimension of the query or key, is the bias matrix.

[0025] Furthermore, each layer output in the Swin Transformer module is subjected to resolution compression and channel expansion through image block merging to extract multi-scale features.

[0026] Furthermore, in step S109, the channel attention module uses global average pooling and multi-layer perceptron to weight the channel features to obtain output features;

[0027] Among them, the channel attention map is:

[0028]

[0029]

[0030] The final output is:

[0031]

[0032]

[0033] in, is the attention weight of channel c, is the global feature of channel c, is the input feature map of channel c, is the channel height, is the channel width, is the number of channels, and is the fully connected layer weight, represents the activation function, is the activation function, exp is an exponential function, is the channel identifier, Output features for the channel attention module, is the channel attention map, which represents the influence of the i-th channel on the j-th channel. The original features of the i-th channel, is the original feature of the j-th channel, is the proportional coefficient.

[0034] Furthermore, the spatial attention module uses convolution operations to construct a spatial attention map for weighting the original features. The spatial attention map is:

[0035]

[0036] The final output is:

[0037]

[0038] in, is the spatial attention map, which represents the influence of the i-th channel on the j-th channel. Output features for the spatial attention module, Local features The new feature map generated by the convolutional layer, is the channel identifier, is the number of channels, is the proportionality coefficient, exp is an exponential function.

[0039] Furthermore, in step S111, the output end of the DMSA-Swin-Unet network uses 1×1 convolution to perform category mapping, and uses a normalized exponential function to normalize the multi-category probability of each pixel.

[0040] Furthermore, the training of the DMSA-Swin-Unet network model uses a joint loss function, which adopts a weighted combination of a cross entropy loss function and a Dice loss function. The joint loss function is:

[0041]

[0042]

[0043]

[0044] in: is the joint loss function, is the cross entropy loss function, is the Dice loss function, is the weighting coefficient, is the true label, is the predicted probability, A is the predicted result, and S represents the true label.

[0045] Furthermore, the DMSA-Swin-Unet network model uses the Adam optimizer during training, combined with a learning rate scheduler to dynamically adjust the learning rate to improve convergence speed and detection accuracy. The learning rate scheduling strategy used by the learning rate scheduler is:

[0046]

[0047] in, For the round learning rate, is the minimum learning rate, is the maximum learning rate, is the training round, is the total number of training rounds, and cos is the cosine function.

[0048] Furthermore, the method supports processing multiple types of complex surface defect images, including convex powder, jet flow, dirty spots, scratches, pits and exposed bottom.

[0049] In a preferred embodiment of the present invention, compared with the prior art, the present invention has the following beneficial effects:

[0050] 1. This invention introduces the Swin-Unet structure to replace the traditional U-Net, achieving more effective feature extraction and semantic fusion. It uses the Swin Transformer as the backbone network structure and adopts a multi-level local window attention structure to improve the multi-scale feature expression capability. The skip connection ensures the retention of low-level semantics and spatial information. Compared with the traditional CNN structure, the defect recognition accuracy is improved by more than 5%;

[0051] 2. This invention introduces the window-based attention (W-MSA) and shifted window-based attention (SW-MSA) mechanisms to enhance local and global modeling capabilities. W-MSA calculates self-attention in a local window, while SW-MSA achieves cross-regional feature interaction through window shifting, enabling the model to simultaneously model local and contextual relationships. The model is better able to perceive context and improve the ability to locate irregular targets such as small defects and fuzzy edges.

[0052] 3. This invention integrates the DMSANet module into the decoder and introduces a "channel + spatial" dual attention mechanism. The DMSA module constructs channel attention (weighting feature channels) and spatial attention (enhancing key positions) in parallel, enhancing skip connection features, improving the recognition of significant areas, and improving the model's response to minor defects and weak boundaries, avoiding feature dilution and enhancing model generalization.

[0053] 4. The present invention proposes a DMSA-Swin-Unet fusion structure to realize a multi-scale information extraction mechanism of semantic enhancement + spatial restoration. Through the joint optimization of the multi-scale encoder and the fusion attention module, the model has stronger category distinction ability and regional edge expression ability; the up and down sampling structure helps to restore space, improves detection stability, and has a stronger ability to recognize complex scenes and targets with blurred boundaries; the image segmentation results have smoother edges and clearer structures.

[0054] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 1 is a flow chart of a detection method according to a preferred embodiment of the present invention;

[0056] Figure 2 Schematic diagram of the DMSA-Swin-Unet network structure of a preferred embodiment of the present invention;

[0057] Figure 3 Schematic diagram of the overall architecture of a Swin Transformer according to a preferred embodiment of the present invention;

[0058] Figure 4 Schematic diagram of a Swin Transformer module of a preferred embodiment of the present invention;

[0059] Figure 5 Schematic diagram of the DMSANet structure of a preferred embodiment of the present invention;

[0060] Figure 6 is a loss function curve diagram of a preferred embodiment of the present invention;

[0061] Figure 7 Schematic diagram of the semantic segmentation results of convex powder defects in a preferred embodiment of the present invention, wherein (a) is the original image of the convex powder surface defect, (b) is the original label image of the convex powder, and (c) is the semantic segmentation image of the convex powder obtained by applying the model defect detection method of the present invention;

[0062] Figure 8This is a schematic diagram of the semantic segmentation results of dirty spot defects in a preferred embodiment of the present invention, wherein (a) is the original dirty spot surface defect image, (b) is the original dirty spot label image, and (c) is the dirty spot semantic segmentation image obtained by applying the model defect detection method of the present invention. DETAILED DESCRIPTION

[0063] The following describes several preferred embodiments of the present invention with reference to the accompanying drawings to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0064] In the drawings, components with identical structures are denoted by the same reference numerals, and components with similar structures or functions are denoted by similar reference numerals. The size and thickness of each component shown in the drawings are arbitrary and are not limited by the present invention. For clarity, the thickness of components in some places in the drawings is appropriately exaggerated.

[0065] To address the weak detection and robustness of traditional methods for complex, small, and diverse defects, this paper proposes a Swin-Unet network model (DMSA-Swin-Unet) that incorporates a dual multi-scale attention mechanism (DMSA) for semantic segmentation of aluminum profile surface defects. This architecture combines the powerful spatial context modeling capabilities of the Swin Transformer with the enhanced spatial and channel attention capabilities of DMSANet. It simultaneously focuses on local details and global structure, significantly improving defect detection accuracy and detail restoration capabilities. It is particularly suitable for segmenting multi-scale and multi-morphological defects.

[0066] like Figure 1 As shown, an embodiment of the present invention provides a method for detecting surface defects of aluminum profiles based on a DMSA-Swin-Unet network model, the method comprising the following steps:

[0067] S101: Preprocessing the collected aluminum profile surface image.

[0068] In this embodiment, the preprocessing includes image size normalization, color enhancement, and data standardization.

[0069] Since the collected images come from diverse sources and have inconsistent sizes, directly inputting them into the model will cause a waste of computing resources and unstable training. Therefore, the original images need to be cropped to a uniform size, and a mosaic data augmentation method is used to expand the training samples.

[0070] S103: Using a convolution operation with a predetermined step size on the preprocessed image, the image is divided into image blocks and linearly projected to generate an embedded feature sequence.

[0071] After the image is normalized, the model input is divided into non-overlapping image blocks using Conv-Patch Embedding and then fed into the Swin Unet encoder. The specific method is as follows:

[0072] The normalized image is input into a convolution kernel with a stride of P and a size of P. In a 2D convolutional layer of P, where P is the image patch size (4 in this case), the convolution operation transforms the image, embedding the channel dimension within the image patch. This process completes the image patch division and also implements preliminary feature extraction and downsampling through convolution.

[0073] S105: Extract multi-level local and global features through the Swin Unet encoder module. The multi-layer nested Swin Transformer in the encoder adopts alternating window attention and shifted window attention mechanism.

[0074] In this embodiment, the Swin Transformer module has a multi-layer nested structure, each layer includes a window attention module and a shifted window attention module. The window attention module calculates self-attention in the local window, and the shifted window attention module realizes cross-region feature interaction through window shifting.

[0075] The specific steps include the following:

[0076] S1051: Input feature map passes through the window attention module to calculate self-attention:

[0077]

[0078]

[0079] S1052: The shifted window attention module introduces cross-window feature dependencies and performs cross-region feature interaction:

[0080]

[0081]

[0082] in, For the The W-MSA module output of the Swin Transformer module of the layer, For the The output of the MLP layer of the SwinTransformer module, For the The output of the SW-MSA module of the Swin Transformer module at the layer, is the final output of the current Swin Transformer module, LN is layer normalization, MLP is multi-layer perceptron, is the window multi-head self-attention, It is a shift window multi-head attention mechanism;

[0083] The self-attention mechanism is calculated as follows:

[0084]

[0085] in, Attention is the attention coefficient of the key-value pair, is the query matrix, is the bond matrix, is the value matrix, is the transposed matrix of matrix K, is the normalized exponential function, is the dimension of the query or key, is the bias matrix.

[0086] In this embodiment, each layer output in the Swin Transformer module is subjected to resolution compression and channel expansion through image block merging to extract multi-scale features.

[0087] S107: In the decoding process of each layer, the coding features of the corresponding scale are fused through jump connections.

[0088] S109: Configure the DMSA module after the jump connection. The DMSA module consists of a channel attention module and a spatial attention module in parallel.

[0089] In this embodiment, the channel attention module uses global average pooling and multi-layer perceptron to weight the channel features to obtain output features;

[0090] Among them, the channel attention weight is:

[0091]

[0092]

[0093] The final output is:

[0094]

[0095]

[0096] in, is the attention weight of channel c, is the global feature of channel c, is the input feature map of channel c, is the channel height, is the channel width, is the number of channels, and is the fully connected layer weight, represents the activation function, is the activation function, exp is an exponential function, is the channel identifier, Output features for the channel attention module, is the channel attention map, which measures the impact of the i-th channel on the j-th channel. The original features of the i-th channel, is the original feature of the j-th channel, is the proportional coefficient.

[0097] The spatial attention module uses convolution operations to construct a spatial attention map for weighting the original features. The spatial attention map is:

[0098]

[0099] The final output is:

[0100]

[0101] in, is the spatial attention map, which measures the impact of the i-th channel on the j-th channel. Output features for the spatial attention module, Local features The new feature map generated by the convolutional layer, is the channel identifier, is the number of channels, is the proportionality coefficient, exp is an exponential function.

[0102] S111: performing step-by-step upsampling by image block expansion, and finally outputting a pixel-level defect segmentation map with the same size as the original image.

[0103] In this embodiment, the output end of the DMSA-Swin-Unet network uses 1×1 convolution to perform category mapping, and uses the Softmax function to normalize the multi-category probability of each pixel.

[0104] In the training of the DMSA-Swin-Unet network model, a joint loss function is used. The joint loss function adopts a weighted combination of the cross entropy loss function and the Dice loss function. The joint loss function is:

[0105]

[0106]

[0107]

[0108] in: is the joint loss function, is the cross entropy loss function, is the Dice loss function, is the weighting coefficient, is the true label, is the predicted probability, A is the predicted result, and S represents the true label.

[0109] During the DMSA-Swin-Unet network model training process, the optimizer uses the Adam optimizer and combines it with the learning rate scheduler to dynamically adjust the learning rate to improve convergence speed and detection accuracy. The learning rate scheduling strategy used by the learning rate scheduler is:

[0110]

[0111] in, For the round learning rate, is the minimum learning rate, is the maximum learning rate, is the training round, is the total number of training rounds, and cos is the cosine function.

[0112] An embodiment of the present invention provides an aluminum profile surface defect detection method based on the DMSA-Swin-Unet network model, which supports processing multiple types of complex surface defect images, including aluminum profile surface defects such as convex powder, jet flow, dirty spots, scratches, pits and exposed bottom.

[0113] Compared with the prior art, the aluminum profile surface defect detection method based on the DMSA-Swin-Unet network model provided in the embodiment of the present invention has the following characteristics:

[0114] 1. The present invention uses the Swin-Unet structure to replace the traditional U-Net, which can achieve more effective feature extraction and semantic fusion. By using the Swin Transformer as the backbone network structure and adopting a multi-level local window attention structure, the multi-scale feature expression capability is improved. The skip connection ensures the retention of low-level semantics and spatial information. Compared with the traditional CNN structure, the defect recognition accuracy is improved by more than 5%;

[0115] 2. This invention introduces window attention and shifted window attention mechanisms to enhance local and global modeling capabilities. W-MSA calculates self-attention in a local window, and SW-MSA achieves cross-regional feature interaction through window shifting, enabling the model to simultaneously model local and contextual relationships, better perceive context, and improve the ability to locate irregular targets such as small defects and fuzzy edges.

[0116] 3. The present invention integrates the DMSANet module into the decoder and introduces the "channel + space" dual attention mechanism. The DMSA module constructs channel attention and spatial attention in parallel, enhances the skip connection features, improves the recognition of significant areas, improves the model's response to minor defects and weak boundaries, avoids feature dilution, and enhances the generalization of the model.

[0117] 4. The DMSA-Swin-Unet fusion structure proposed in this invention realizes a multi-scale information extraction mechanism for semantic enhancement and spatial restoration. Through the joint optimization of the multi-scale encoder and the fusion attention module, the model has stronger category distinction ability and regional edge expression ability; the up and down sampling structure helps to restore space, improves detection stability, and has a stronger ability to recognize complex scenes and targets with blurred boundaries; the image segmentation results have smoother edges and clearer structures.

[0118] The present invention is described in detail below in conjunction with the preferred embodiments of the present invention.

[0119] 1. Dataset Introduction

[0120] The Tianchi aluminum profile surface defect dataset is provided by the Alibaba Cloud Tianchi platform. It contains 10 typical defect types (paint bubbles, blasts, dirty spots, exposed bottom, scratches, orange peel, non-conductivity, pitting, convex powder, and exposed bottom corners). It contains more than 10,000 images of defective aluminum profile monitoring data from actual production, and each image contains one or more defects.

[0121] 2. Data preprocessing

[0122] Since the collected images come from diverse sources and have inconsistent sizes, directly inputting them into the model will cause a waste of computing resources and unstable training. Therefore, the original images need to be cropped to a uniform size, and a mosaic data augmentation method is used to expand the training samples.

[0123] 2-1. Image size standardization

[0124] All images are scaled or center-cropped to a uniform size of 512×512. Bilinear interpolation or nearest neighbor interpolation is used to ensure that the image structure is not distorted during the scaling process, improve the batch training efficiency of the model, and maintain the matching of the convolution kernel and image features.

[0125] 2-2. Mosaic Image Enhancement

[0126] To improve the model's generalization and robustness, we simulated the appearance of defects at different angles on the actual production line and used mosaic augmentation for data augmentation. Four images were randomly selected from the training dataset for processing.

[0127] Random scaling and cropping: Each image is randomly scaled and cropped to fit the input size of the model.

[0128] Random permutation: Randomly arrange the four images into a rectangular area to form a new training sample. Data expansion: Further expand the dataset through operations such as flipping and rotation.

[0129] 3. Model Input

[0130] After the image is normalized, the model input uses Conv-PatchEmbedding to divide the image into non-overlapping patches and input it to the Swin Unet encoder. The specific method is as follows:

[0131] The normalized image Input to a convolution kernel with stride P and size P In the 2D convolution layer of P, where P is the image block size, which is 4 in this case. The convolution operation transforms the image into a shape of Where C is the number of output feature channels, i.e., the embedding dimension of the image block. This completes the image block division and implements preliminary feature extraction and downsampling through convolution.

[0132] 4. Feature extraction and fusion

[0133] The encoder extracts multi-level local and global features, the decoder gradually upsamples through the image block expansion module and combines jump connections to restore the spatial structure, and the DMSA module enhances the spatial and channel expression capabilities of the fused features.

[0134] 4-1. Swin Unet Encoder

[0135] In the encoder, the resolution is The C-dimensional input is fed into the multi-layer nested structure of Swin Transformer for representation learning. Each layer includes a window attention (W-MSA) and a shifted window attention (SW-MSA) module. The feature dimension and resolution remain unchanged. At the same time, the image block merging layer reduces the number of features (2x downsampling) and increases the feature dimension to 2x. Here .

[0136] First, for the l-th layer encoding, the input feature map After multi-head self-attention calculation within the window:

[0137]

[0138]

[0139] Then, the shifted window attention introduces feature dependencies across windows, and the output is:

[0140]

[0141]

[0142] in, For the The W-MSA module output of the Swin Transformer module of the layer, For the The output of the MLP layer of the SwinTransformer module, For the The output of the SW-MSA module of the Swin Transformer module at the layer, is the final output of the current Swin Transformer module, LN is layer normalization, MLP is multi-layer perceptron, is the window multi-head self-attention, It is a shift window multi-head attention mechanism;

[0143] The self-attention mechanism is calculated as follows:

[0144]

[0145] in, Attention is the attention coefficient of the key-value pair, , is the query matrix, is the bond matrix, is the value matrix, is the transposed matrix of matrix K, is the normalized exponential function, is the dimension of the query or key, and B comes from the bias matrix .

[0146] Between each stage, image blocks are merged to reduce the spatial dimension and increase the channel dimension.

[0147] The input image patch is divided into 4 parts and connected together through the image patch merging layer. At this time, the feature resolution is downsampled by 2 times and the feature dimension is increased by 4 times. Therefore, a linear layer is applied to the connected features to unify the feature dimension to 2 times the original dimension:

[0148]

[0149] 4-2. Swin Unet bottleneck area

[0150] Since the Transformer is too deep to converge, only two consecutive Swin Transformer blocks are used to build a bottleneck to learn deep feature representation. In the bottleneck area, the feature size and resolution remain unchanged.

[0151] 4-3. Swin Unet Decoder

[0152] The decoder structure is symmetrical to the encoder. In the decoder, the image block expansion layer is used to upsample the extracted deep features. The image block expansion layer reshapes the feature maps of adjacent dimensions into higher resolution feature maps (2x upsampling) and correspondingly reduces the feature dimension to half of the original dimension, gradually restoring the spatial structure. Taking the first image block expansion layer as an example, before upsampling, the input feature ( ) to increase the feature dimension to the original dimension ( ), and then the reordering operation is used to expand the resolution of the input features to 2× the input resolution, reducing the feature dimension to a quarter of the input dimension ( ).

[0153] For the k-th layer decoding:

[0154]

[0155] in, is the skip connection feature from the same layer of the encoder, It is an upsampling module, and the addition represents the fusion of decoder and encoder features.

[0156] 4-4. Skip Connection

[0157] Similar to U-Net, skip connections are used to fuse multi-scale features from the encoder with upsampled features. Shallow and deep features are concatenated to reduce the loss of spatial information caused by downsampling. After the linear layer, the size of the concatenated features remains the same as the size of the upsampled features.

[0158] 4-5. DMSANet

[0159] The DMSA module is introduced after each jump connection of the decoder, and the fused features is fed into the DMSA module.

[0160] The channel attention module is used to selectively weight the importance of each channel to produce the best output features. Global average pooling or global maximum pooling is performed on the input feature map to obtain the global features of each channel:

[0161]

[0162] Attention weight of the c-th channel:

[0163]

[0164] Calculation from original features Channel attention map , apply the softmax layer to obtain the channel attention map :

[0165]

[0166] The matrix multiplication result of it and a is multiplied by the scale parameter β and the element-wise summation operation is performed on a to obtain the final output :

[0167]

[0168] in, Represents the input feature map, H, W, and C represent the height, width, and number of input channels, respectively. and Denotes the weight of the fully connected (FC) layer. Represents the activation function, in which the Sigmoid function is usually used. exp is an exponential function. It indicates measuring the influence of the i-th channel on the j-th channel.

[0169] The spatial attention module is used to emphasize important spatial regions in the image.

[0170] The local features represented by A are fed into the convolutional layer to generate two new feature maps. and , apply a softmax layer to compute the spatial attention map :

[0171]

[0172] Input feature A into the convolutional layer to generate a new feature map . Multiply it by the matrix of S and the scale parameter , and perform element-wise sum operation on feature a to obtain the final output:

[0173]

[0174] in, It indicates measuring the influence of the i-th channel on the j-th channel.

[0175] Finally, all sub-features are aggregated and the output is:

[0176]

[0177] in is the obtained multi-scale feature map.

[0178] 5. Defect segmentation output

[0179] Guided by the feature descriptor, the attention module adaptively selects different spatial scales across channels:

[0180]

[0181] Multi-scale channel attention The recalibration weight is multiplied by the corresponding scale Feature map:

[0182]

[0183] 6. Training Strategy

[0184] During the model training process, the following optimization configuration is adopted to improve the convergence speed and detection accuracy.

[0185] 6-1. Optimizer selection and parameter setting

[0186] The optimizer uses Adam, the learning rate is set to 1e-4, and it is dynamically adjusted in combination with the learning rate scheduler. The weight decay coefficient is 0.01 for regularization to prevent overfitting, and the momentum factor parameter is set to =0.9, =0.999.

[0187] 6-2. Learning rate scheduling strategy

[0188] In order to improve the stability in the early stage of training and the accuracy in the later stage, the Cosine Annealing learning rate adjustment strategy is used to gradually reduce the learning rate from the initial value to the minimum threshold:

[0189]

[0190] in, is the learning rate for the tth round, T is the total number of training rounds, and is the minimum and maximum learning rate.

[0191] 6-3. Loss Function Design

[0192] The cross-entropy loss (CrossEntropy Loss) and Dice Loss are jointly designed to effectively alleviate the problems of category imbalance and small target defect segmentation. The joint loss function is defined as follows:

[0193]

[0194]

[0195]

[0196] in: is the joint loss function, is the cross entropy loss function, is the Dice loss function, is the weighting coefficient, is the true label, is the predicted probability, A is the predicted result, and S represents the true label.

[0197] 6-4. Overfitting Control Strategy

[0198] Use the validation set early stopping strategy (Early Stopping) to terminate training early when the validation performance does not improve for 10 consecutive rounds to avoid overfitting.

[0199] 6-5. Training Platform and Configuration

[0200] Training platform: implemented in PyTorch and running on NVIDIA RTX A5000 / GPU cluster;

[0201] Batch size: 32, dynamically adjusted based on video memory size;

[0202] Total training rounds: 50;

[0203] Supports breakpoint resumption and model saving.

[0204] 7. Evaluation Metrics

[0205] Evaluate model performance using mIoU, F1-score, Precision, Recall, etc.:

[0206]

[0207]

[0208]

[0209]

[0210] Among them, TP is the number of pixels predicted to be of this class and actually of this class (True Positive), FP is the number of pixels predicted to be of this class but actually are not of this class (False Positive), FN is the number of pixels that are actually of this class but are wrongly predicted to be other classes (False Negative), and N is the total number of classes.

[0211] like Figure 2 As shown in the figure, the Swin-Unet network structure diagram of this embodiment is a U-Net variant based on the Swin Transformer. It replaces the original convolutional UNet with a Swin transformer module. The maximum pooling is replaced by image block merging, which groups adjacent minimum feature blocks and splices them by depth. It is a convolution-free downsampling technology. At the same time, image block expansion is used to achieve upsampling of feature maps. The DMSANet dual multi-scale attention mechanism is introduced in the decoding stage of the Swin-Unet structure, focusing on both spatial attention and channel attention.

[0212] like Figure 3 The figure below shows the overall architecture of the Swin Transformer of this embodiment. It introduces hierarchical feature mapping and windowed attention transformation. The network divides the image into different image blocks through image block segmentation and outputs them as embeddings. Each stage first implements convolution-free downsampling by merging image blocks, gradually increasing the receptive field. Each stage consists of image block merging and multiple blocks. In the first stage, the number of channels is adjusted to c through linear embedding and local linear transformation. Subsequently, the number of channels is doubled while reducing the resolution.

[0213] like Figure 4 Figure 2 shows the Swin Transformer module of this embodiment, which mainly consists of LayerNorm, MLP, WindowAttention, and Shifted Window Attention. W-MSA and SW-MSA represent window-based multi-head self-attention using regular and shifted window split configurations, respectively. The shifted window split method introduces connections between adjacent non-overlapping windows in the previous layer.

[0214] like Figure 5 Figure 2 shows the DMSANet architecture of this embodiment. The dual multi-scale attention network consists of two parts: the first part separates the input image, extracts features at different scales, and aggregates them; the second part uses spatial and channel attention modules to adaptively combine local features with their global dependencies in parallel.

[0215] like Figure 6The figure below shows the loss function curve for this example. As the number of training rounds increases, both the training loss and the validation loss show an overall downward trend, indicating that the model's performance on both the training and validation sets is continuously improving, and its ability to fit the data is gradually increasing.

[0216] like Figure 7 The figure shows the result of convex powder semantic segmentation in this embodiment. Figure 7 (a) is the original image of convex powder surface defects. Figure 7 (b) is the original label map of convex powder, Figure 7 (c) is the convex powder semantic segmentation map obtained by applying the defect detection method of the model of the present invention. Figure 7 (a) the original state and Figure 7 (b) is for reference only. Figure 7 (c) It can accurately segment defect areas such as convex powder, realize effective detection and positioning of defects, and improve the efficiency and accuracy of defect detection.

[0217] like Figure 8 The figure shows the result of semantic segmentation of dirty points in this embodiment. Figure 8 (a) is the original image of dirty surface defects. Figure 8 (b) is the original label map of dirty points, Figure 8 (c) is the semantic segmentation map of dirty points obtained by applying the defect detection method of the model of the present invention. Figure 8 (a) the original state and Figure 8 (b) is for reference only. Figure 8 (c) It can accurately segment defect areas such as dirty spots, achieve effective detection and positioning of defects, and improve the efficiency and accuracy of defect detection.

[0218] Compared with the existing technology, the aluminum profile surface defect detection method based on the DMSA-Swin-Unet network model provided by the present invention has broad industrial applicability and promotion prospects.

[0219] 1. Significant technological advantages

[0220] (1) Able to adapt to various types of surface defects of aluminum profiles, such as linear scratches, surface peeling, point dirt, regional wrinkles, edge damage, etc.;

[0221] (2) The model has strong robustness and fault tolerance, and can maintain high recognition accuracy even in the presence of complex background interference such as uneven lighting, complex surface textures, and weak contrast defects;

[0222] (3) The combination of Transformer and attention mechanism is used to effectively overcome the problem of missing minor defects in traditional CNN methods.

[0223] 2. Flexible deployment

[0224] (1) Support deployment of embedded devices, such as Jetson NX, Raspberry Pi, domestic industrial control motherboards and other low-power platforms;

[0225] (2) Adapt to cloud server deployment and collaborative operation with edge terminals, and can be combined with production line PLC and MES systems to achieve data closed loop;

[0226] (3) Provide inference modules and front-end interface (API) to facilitate access and operation in different industrial control systems.

[0227] 3. Stable performance

[0228] (1) The model has been tested on samples collected from various industrial aluminum surfaces and is suitable for various treatment processes such as anodizing, spraying, brushing, and electrophoresis;

[0229] (2) It has been verified in actual environments that it can still output stable test results under complex working conditions such as vibration, dust, noise, and electromagnetic interference;

[0230] (3) The system supports online automatic updates and model fine-tuning, and has continuous learning and self-adaptation capabilities.

[0231] 4. Strong scalability

[0232] (1) The network structure is highly versatile and can be expanded to other metal surface detection such as stainless steel, titanium alloy, copper strip, etc. through lightweight parameter adjustment;

[0233] (2) Modular design supports subsequent access to subsystems such as depth determination (such as defect severity assessment), traceability identification (QR code / barcode), and robot control;

[0234] (3) It can be used in conjunction with industrial camera arrays to achieve high-end detection needs such as multi-angle detection and multi-channel fusion.

[0235] The above describes in detail the preferred embodiments of the present invention. It should be understood that those skilled in the art can make numerous modifications and variations based on the concepts of the present invention without inventive effort. Therefore, any technical solutions that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A method for detecting surface defects of aluminum profiles based on the DMSA-Swin-Unet network model, characterized in that: The method comprises the following steps: S101: Preprocessing the collected aluminum profile surface image, wherein the preprocessing includes image size normalization, color enhancement, and data standardization; S103: performing a convolution operation with a predetermined step length on the preprocessed image, dividing the image into image blocks and performing linear projection to generate an embedded feature sequence; S105: extracting multi-level local and global features through a Swin Unet encoder module, wherein the multi-layer nested Swin Transformer in the encoder module adopts alternating window attention and shifted window attention mechanisms; S107: In the decoding process of each layer, the coding features of the corresponding scale are fused through skip connections; S109: configuring a DMSA module after the skip connection, wherein the DMSA module is composed of a channel attention module and a spatial attention module in parallel; S111: performing step-by-step upsampling by image block expansion, and finally outputting a pixel-level defect segmentation map with the same size as the original image.

2. The method according to claim 1, wherein In step S105, the Swin Transformer module is a multi-layer nested structure, each layer includes a window attention module and a shift window attention module, the window attention module calculates self-attention in the local window, and the shift window attention module realizes cross-region feature interaction through window shifting.

3. The method according to claim 2, wherein The step S105 includes the following sub-steps: S1051: Input feature map and calculate self-attention through the window attention module: S1052: The shifted window attention module introduces cross-window feature dependencies to perform cross-region feature interaction: in, For the The SW_MSA module output of the Swin Transformer module of the layer, For the l The MLP layer output of the SwinTransformer module, For the +1 layer of Swin Transformer’s SW-MSA module output, For the l+1 The MLP layer output of the Swin Transformer module, LN is layer normalization, MLP is multi-layer perceptron, is the window multi-head self-attention, It is a shift window multi-head attention mechanism; The self-attention mechanism is calculated as follows: in, Attention is the attention coefficient of the key-value pair, is the query matrix, is the bond matrix, is the value matrix, is the transposed matrix of matrix K, is the normalized exponential function, is the dimension of the query or key, is the bias matrix.

4. The method according to claim 3, wherein Each layer output in the Swin Transformer module is subjected to resolution compression and channel expansion through image block merging to extract multi-scale features.

5. The method according to claim 4, characterized in that In step S109, the channel attention module uses global average pooling and multi-layer perceptron to weight the channel features to obtain output features; Among them, the channel attention map is: The final output is: in, is the attention weight of channel c, is the global feature of channel c, is the input feature map of channel c, is the channel height, is the channel width, is the number of channels, and is the fully connected layer weight, represents the activation function, is the activation function, exp is an exponential function, is the channel identifier, Output features for the channel attention module, is the channel attention map, expressed as i Channel pair j The impact of the channels, is the original feature of the i-th channel, is the original feature of the j-th channel, is the proportional coefficient.

6. The method according to claim 5, characterized in that The spatial attention module uses convolution operations to construct a spatial attention map for weighting the original features. The spatial attention map is: The final output is: in, is the spatial attention map, which represents the influence of the i-th channel on the j-th channel. Output features for the spatial attention module, Local features The new feature map generated by the convolutional layer, is the channel identifier, is the number of channels, is the proportionality coefficient, exp is an exponential function.

7. The method according to claim 6, characterized in that In step S111, the output end of the DMSA-Swin-Unet network uses 1×1 convolution to perform category mapping, and uses a normalized exponential function to normalize the multi-category probability of each pixel.

8. The method according to claim 7, characterized in that The training of the DMSA-Swin-Unet network model uses a joint loss function, which adopts a weighted combination of the cross entropy loss function and the Dice loss function. The joint loss function is: in, is the joint loss function, is the cross entropy loss function, is the Dice loss function, is the weighting coefficient, is the true label, is the predicted probability, A is the predicted result, and S represents the true label.

9. The method according to claim 8, characterized in that The DMSA-Swin-Unet network model uses the Adam optimizer during training, and dynamically adjusts the learning rate in combination with the learning rate scheduler. The learning rate scheduling strategy used by the learning rate scheduler is: in, For the round learning rate, is the minimum learning rate, is the maximum learning rate, is the training round, is the total number of training rounds, and cos is the cosine function.

10. The method according to claim 9, characterized in that The method supports processing multiple types of complex surface defect images, including convex powder, jet flow, dirty spots, scratches, pits and exposed bottom.

Images