Aluminum profile surface defect classification method and device based on deep learning

Abstract

The application discloses a kind of based on deep learning's aluminium profile surface defect classification method and device, for improving the detection performance of aluminium profile surface defect classification.The method of the application includes inputting defect sample image to initial model, extracting features to first feature map by residual network first section to defect sample image;Second feature map is obtained by feature enhancement to first feature map through weight non-local module and residual network second section;Second feature map is input into residual network third section and feature extraction is carried out to obtain third feature map;Fourth feature map is obtained by feature fusion to second feature map and third feature map through feature fusion auxiliary classifier;Loss calculation is carried out to third feature map and fourth feature map, and model parameter optimization is carried out using back propagation algorithm;The above steps are iteratively trained until loss converges or reaches preset iteration number, and obtain defect classification model, and the model is used for the classification detection of aluminium profile surface defect.

Description

Technical Field

[0001] This application relates to the field of image detection, and in particular to a method and apparatus for classifying surface defects of aluminum profiles based on deep learning. Background Technology

[0002] Currently, defect detection is an indispensable part of industrial production, with aluminum profiles serving as a fundamental material in industrial products. During the production of aluminum profiles, defects can arise due to various reasons, resulting in a variety of defect types, such as scratches, orange peel texture, exposed substrate, paint splatter, paint bubbles, pitting, discoloration, and dirt spots. To ensure product quality, surface defect detection of aluminum profiles is necessary so that different processing methods can be applied based on the detected defect types.

[0003] Previously, aluminum surface inspection mainly relied on manual visual inspection or traditional non-destructive testing. However, aluminum surfaces inherently contain textures that are difficult to distinguish from defects. Manual visual inspection is laborious and cannot accurately identify surface defects in a timely manner, making it difficult to control the efficiency of quality inspection. Traditional non-destructive testing methods include eddy current testing, infrared testing, magnetic flux leakage testing, and laser testing, but these methods detect a limited range of defects and lack real-time accuracy in classification.

[0004] With the development of deep learning, especially the application of convolutional neural networks in image recognition, image detection, and image segmentation in computer vision, the results achieved are unparalleled by traditional algorithms. Image detection, situated between image recognition and image segmentation, is a method that can detect the location of objects of a specific category in an image and provide corresponding category confidence scores. Currently, commonly used aluminum defect detection algorithms, such as binary classification algorithms and support vector machine classification, achieve aluminum defect detection by performing binary classification on images or pixels.

[0005] However, due to the varying sizes, shapes, and extreme aspect ratios of surface defects in aluminum materials, the performance of commonly used detection algorithms for detecting surface defects in aluminum materials is not satisfactory. Summary of the Invention

[0006] This application provides a deep learning-based method and apparatus for classifying surface defects of aluminum profiles, which enhances the detection effect of the defect classification model on aluminum profile defects and improves the model's classification performance on aluminum surface defects.

[0007] The first aspect of this application provides a deep learning-based method for classifying surface defects in aluminum profiles, including:

[0008] Obtain defect sample images carrying defect labels from the aluminum profile image training set;

[0009] The defect sample image is input into the initial model, which includes a residual network, a weighted nonlocal module, a feature fusion auxiliary classifier, and a feature mapping module.

[0010] The first feature map output by the first segment of the residual network is obtained by extracting features from the defect sample image.

[0011] The first feature map is enhanced by the weighted nonlocal module and the second segment of the residual network to obtain the second feature map;

[0012] The second feature map is input into the third segment of the residual network for feature extraction to obtain the third feature map;

[0013] The second feature map and the third feature map are fused using the feature fusion auxiliary classifier to obtain a fourth feature map;

[0014] The feature mapping module performs loss calculations on the third and fourth feature maps, and updates and optimizes the model parameters based on the loss results using the backpropagation algorithm.

[0015] Input defect sample images from the aluminum profile image training set into the initial model and perform the above steps for iterative training until the loss converges or the preset number of iterations is reached to obtain the trained defect classification model. The defect classification model is used for the classification and detection of surface defects of aluminum profiles.

[0016] Optionally, the residual network includes a backbone module, a first-stage module, a second-stage module, a third-stage module, and a fourth-stage module, wherein the first-stage module, the second-stage module, the third-stage module, and the fourth-stage module are composed of 3, 4, 6, and 3 residual structure modules, respectively.

[0017] The step of extracting features from the defect sample image through the first segment of the residual network to obtain the first feature map output by the first segment includes:

[0018] The defect sample image is sequentially input into the main module and the first stage module for image feature extraction to obtain the first feature map;

[0019] The step of enhancing the first feature map using the weighted nonlocal module and the second segment of the residual network to obtain the second feature map includes:

[0020] The first feature map is input into the weighted nonlocal module for feature enhancement to obtain the target feature map;

[0021] The target feature map is input into the second stage module for feature extraction to obtain the second feature map;

[0022] The step of inputting the second feature map into the third segment of the residual network for feature extraction to obtain the third feature map includes:

[0023] The second feature map is sequentially input into the third-stage module and the fourth-stage module for image feature extraction to obtain the third feature map;

[0024] The loss calculation of the third feature map and the fourth feature map by the feature mapping module includes:

[0025] The feature mapping module performs flattening, full connection, and softmax normalization operations on the third feature map to determine the first prediction probability vector of the third feature map; the cross-entropy loss is calculated based on the first prediction probability vector and the true label value to obtain the first loss.

[0026] The fourth feature map is flattened, fully connected, and normalized using softmax to determine the second prediction probability vector of the fourth feature map; the cross-entropy loss is calculated based on the second prediction probability vector and the true label value to obtain the second loss.

[0027] Calculate the total loss based on the first loss and the second loss.

[0028] Optionally, the residual structure module includes two 1×1 convolutional modules and one 3×3 convolutional module, and the backbone module is a 7×7 convolutional module.

[0029] Optionally, the weighted nonlocal module includes a channel attention module, and the step of inputting the first feature map into the weighted nonlocal module for feature enhancement to obtain the target feature map includes:

[0030] A first feature map of size H×W×C is input into the weighted nonlocal module, and then subjected to θ linear transformation, ... Linear transformation and g-linear transformation yield characteristics Figure X θ Feature map and characteristics Figure X g ;

[0031] The feature Figure X θ The input is fed into the channel attention module to obtain the channel attention weight vector λ = [λ 1 ,λ 2 ,…,λ k ,…,λ C ], where C is the feature Figure Xθ The number of channels, where k is the index value;

[0032] The channel attention weight vector λ is compared with the feature Figure X θ Perform element-wise multiplication, then combine with the feature map. Perform matrix multiplication to obtain the similarity matrix S M ;

[0033] The similarity matrix S M Softmax normalization is performed to obtain the spatial attention map A. M ;

[0034] The spatial attention map A M With the aforementioned features Figure X g After performing matrix multiplication, a 1×1 convolution module is used for convolution;

[0035] The convolutional feature map is added to the first feature map element by element to obtain the target feature map.

[0036] Optionally, the feature Figure X θ The input is fed into the channel attention module to obtain the channel attention weight vector λ = [λ 1 ,λ 2 ,…,λ k ,…,λ C ]include:

[0037] The feature Figure X θ Input to the channel attention module;

[0038] The features are analyzed using a global average pooling formula. Figure X θ Perform a global average pooling operation to ensure that the features are... Figure X θ Compressed into feature real numbers Z = [z] with global information 1 ,z 2 ,…,z k ,…,z C ];

[0039] According to the characteristic real number Z = [z 1 ,z 2 ,…,z k ,…,z C Based on the fully connected formula, perform fully connected operations to obtain the channel attention weight vector λ = [λ]. 1 ,λ 2 ,…,λ k ,…,λ C].

[0040] Optionally, the global average pooling formula is defined as follows:

[0041]

[0042] Where H and W are the height and width of the feature map, and i and j represent the pixel indices in the feature map space;

[0043] The fully connected formula is defined as follows:

[0044] F ex (Z,W)=σ(W2δ(W1Z))

[0045] Where δ and σ represent the ReLU and Sigmoid activation functions, respectively, and W1∈R (C / r)×C W2∈R C×(C / r) R represents the fully connected real number space, and r represents the preset parameter;

[0046] The channel attention weight vector λ is calculated as follows:

[0047]

[0048] Optionally, the step of fusing the second feature map and the third feature map using the feature fusion auxiliary classifier to obtain the fourth feature map includes:

[0049] The second feature map and the third feature map are input into the feature fusion auxiliary classifier, and the third feature map is upsampled by subpixel convolution so that the third feature map and the second feature map have the same spatial size.

[0050] The second feature map and the sampled third feature map are concatenated to obtain the fourth feature map.

[0051] The second aspect of this application provides a deep learning-based method for classifying surface defects in aluminum profiles. The defect classification model used in this method is trained using the method described in the first aspect. The method includes:

[0052] Acquire an image of the aluminum material to be inspected;

[0053] The image of the aluminum material to be detected is input into the defect classification model, and the first feature map output by the first segment of the residual network is obtained by extracting features from the image of the aluminum material to be detected.

[0054] The first feature map is enhanced by using a weighted nonlocal module and the second segment of the residual network to obtain a second feature map;

[0055] The second feature map is input into the third segment of the residual network for feature extraction to obtain the third feature map;

[0056] The third feature map is subjected to category prediction probability calculation by the feature mapping module, and the surface defect category of the aluminum material image to be detected is determined based on the calculation result.

[0057] A third aspect of this application provides a deep learning-based device for classifying surface defects in aluminum profiles, the device comprising:

[0058] The acquisition unit is used to acquire defect sample images carrying defect labels from the aluminum profile image training set;

[0059] An input unit is used to input the defect sample image into an initial model, the initial model including a residual network, a weighted nonlocal module, a feature fusion auxiliary classifier, and a feature mapping module;

[0060] The first extraction unit is used to extract features from the defect sample image through the first segment of the residual network and obtain the first feature map output by the first segment.

[0061] The second extraction unit is used to perform feature enhancement on the first feature map through the weighted nonlocal module and the second segment of the residual network to obtain a second feature map;

[0062] The third extraction unit is used to input the second feature map into the third segment of the residual network for feature extraction to obtain the third feature map;

[0063] The fusion unit is used to fuse the second feature map and the third feature map using the feature fusion auxiliary classifier to obtain a fourth feature map;

[0064] The optimization unit is used to perform loss calculation on the third feature map and the fourth feature map through the feature mapping module, and update and optimize the model parameters based on the loss result using the backpropagation algorithm.

[0065] The input unit is also used to input defect sample images from the aluminum profile image training set into the initial model and perform the above steps for iterative training until the loss converges or the preset number of iterations is reached, so as to obtain the trained defect classification model. The defect classification model is used for the classification and detection of surface defects of aluminum profiles.

[0066] Optionally, the residual network includes a backbone module, a first-stage module, a second-stage module, a third-stage module, and a fourth-stage module, wherein the first-stage module, the second-stage module, the third-stage module, and the fourth-stage module are composed of 3, 4, 6, and 3 residual structure modules, respectively.

[0067] The first extraction unit is specifically used to sequentially input the defect sample image into the main module and the first stage module for image feature extraction to obtain a first feature map;

[0068] The second extraction unit is specifically used to input the first feature map into the weighted nonlocal module for feature enhancement to obtain the target feature map;

[0069] The target feature map is input into the second stage module for feature extraction to obtain the second feature map;

[0070] The third extraction unit is specifically used to input the second feature map into the third stage module and the fourth stage module in sequence for image feature extraction to obtain the third feature map;

[0071] The optimization unit is specifically used to perform flattening, full connection and softmax normalization operations on the third feature map through the feature mapping module to determine the first prediction probability vector of the third feature map.

[0072] The first loss is obtained by calculating the cross-entropy loss based on the first predicted probability vector and the true label value.

[0073] The fourth feature map is flattened, fully connected, and normalized using softmax to determine the second prediction probability vector of the fourth feature map.

[0074] The cross-entropy loss is calculated based on the second predicted probability vector and the true label value to obtain the second loss;

[0075] The total loss is calculated based on the first loss and the second loss, and the model parameters are updated and optimized using the backpropagation algorithm based on the total loss result.

[0076] Optionally, the weighted nonlocal module includes a channel attention module, and the second extraction unit is specifically used to input a first feature map of size H×W×C into the weighted nonlocal module, and perform θ linear transformation, Linear transformation and g-linear transformation yield characteristics Figure X θ Feature map and characteristics Figure X g ;

[0077] The feature Figure X θ The input is fed into the channel attention module to obtain the channel attention weight vector λ = [λ 1 ,λ 2 ,…,λ k ,…,λ C ], where C is the feature Figure X θ The number of channels, where k is the index value;

[0078] The channel attention weight vector λ is compared with the feature Figure X θ Perform element-wise multiplication, then combine with the feature map. Perform matrix multiplication to obtain the similarity matrix S M ;

[0079] The similarity matrix S M Softmax normalization is performed to obtain the spatial attention map A. M ;

[0080] The spatial attention map A M With the aforementioned features Figure X g After performing matrix multiplication, a 1×1 convolution module is used for convolution;

[0081] The convolutional feature map is added to the first feature map element by element to obtain the target feature map.

[0082] Optionally, the second extraction unit is specifically used to extract the features Figure X θ Input to the channel attention module;

[0083] The features are analyzed using a global average pooling formula. Figure X θ Perform a global average pooling operation to ensure that the features are... Figure X θ Compressed into feature real numbers Z = [z] with global information 1 ,z 2 ,…,z k ,…,z C ];

[0084] According to the characteristic real number Z = [z 1 ,z 2 ,…,z k ,…,z C Based on the fully connected formula, perform fully connected operations to obtain the channel attention weight vector λ = [λ]. 1 ,λ 2 ,…,λ k ,…,λ C ].

[0085] Optionally, the fusion unit is specifically used to input the second feature map and the third feature map into the feature fusion auxiliary classifier, and to upsample the third feature map through sub-pixel convolution so that the third feature map and the second feature map have the same spatial size;

[0086] The second feature map and the sampled third feature map are concatenated to obtain the fourth feature map.

[0087] A fourth aspect of this application provides a deep learning-based device for classifying surface defects in aluminum profiles, the device comprising:

[0088] The acquisition unit is used to acquire an image of the aluminum material to be inspected;

[0089] The input unit is used to input the image of the aluminum material to be detected into the defect classification model, and to extract features from the image of the aluminum material to be detected through the first segment of the residual network to obtain the first feature map output by the first segment.

[0090] The feature enhancement unit is used to enhance the first feature map by using a weighted nonlocal module and the second segment of the residual network to obtain a second feature map.

[0091] An extraction unit is used to input the second feature map into the third segment of the residual network for feature extraction to obtain a third feature map;

[0092] The output unit is used to perform category prediction probability calculation on the third feature map through the feature mapping module, and determine the surface defect category of the aluminum material image to be detected output by the defect classification model based on the calculation result.

[0093] The fifth aspect of this application provides a deep learning-based device for classifying surface defects in aluminum profiles, the device comprising:

[0094] Processor, memory, input / output units, and bus;

[0095] The processor is connected to the memory, the input / output unit, and the bus;

[0096] The memory stores a program, which the processor calls to execute the first aspect and any optional aluminum profile surface defect classification method in the first aspect, or to execute the aluminum profile surface defect classification method of the second aspect.

[0097] The sixth aspect of this application provides a computer-readable storage medium storing a program that, when executed on a computer, performs the aluminum profile surface defect classification method of the first aspect and any optional method of the first aspect, or performs the aluminum profile surface defect classification method of the second aspect.

[0098] As can be seen from the above technical solutions, this application has the following advantages:

[0099] This application first inputs defect sample images into an initial model, which includes a residual network, a weighted nonlocal module, a feature fusion auxiliary classifier, and a feature mapping module. Then, the first segment of the residual network extracts features from the defect sample images, obtaining a first feature map. The first feature map is then enhanced using the weighted nonlocal module and the second segment of the residual network, resulting in a second feature map. This second feature map is then input into the third segment of the residual network for feature extraction, yielding a third feature map. The feature fusion auxiliary classifier fuses the second and third feature maps, resulting in a fourth feature map. The feature mapping module calculates the loss between the third and fourth feature maps, and the model parameters are updated and optimized using backpropagation based on the loss result. Finally, defect sample images from the aluminum profile image training set are input into the initial model, and the above steps are performed iteratively until the loss converges or a preset number of iterations is reached, resulting in a trained defect classification model. Therefore, the defect classification model of this application can be used to detect surface defects in aluminum profiles. Compared with manual visual inspection, it has higher recognition efficiency and accuracy. Compared with traditional inspection, it can adapt to the recognition of various defect types, has stronger adaptability, and can input detection in real time. In addition, the defect classification model of this application uses a residual network as the backbone network and introduces a weighted nonlocal module for feature enhancement to calculate the similarity between different pixels in the feature map, thereby improving the model's ability to capture long-distance dependencies and contextual information of features. At the same time, feature fusion is used to assist the classifier in feature fusion, thereby improving the model's ability to mine detailed features such as texture and edges, enhancing the model's detection effect for aluminum profile defects, and improving the model's classification performance for aluminum surface defects. Attached Figure Description

[0100] To more clearly illustrate the technical solutions in this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0101] Figure 1 A schematic flowchart of an embodiment of the deep learning-based aluminum profile surface defect classification method provided in this application;

[0102] Figure 2 A schematic flowchart of another embodiment of the deep learning-based aluminum profile surface defect classification method provided in this application;

[0103] Figure 3 A schematic diagram of the structure of an embodiment of the defect classification model in the deep learning-based aluminum profile surface defect classification method provided in this application;

[0104] Figure 4 A schematic diagram of an embodiment of the weighted nonlocal module in the defect classification model of the deep learning-based aluminum profile surface defect classification method provided in this application;

[0105] Figure 5 A schematic diagram of the structure of a channel attention module in a weighted nonlocal module provided in this application;

[0106] Figure 6 A schematic diagram of an embodiment of the feature fusion auxiliary classifier in the defect classification model of the deep learning-based aluminum profile surface defect classification method provided in this application.

[0107] Figure 7 A schematic flowchart of another embodiment of the deep learning-based aluminum profile surface defect classification method provided in this application;

[0108] Figure 8 A schematic diagram of an embodiment of the deep learning-based aluminum profile surface defect classification device provided in this application;

[0109] Figure 9 A schematic diagram of another embodiment of the deep learning-based aluminum profile surface defect classification device provided in this application;

[0110] Figure 10 A schematic diagram of an embodiment of the deep learning-based aluminum profile surface defect classification device provided in this application. Detailed Implementation

[0111] This application provides a deep learning-based aluminum profile surface defect classification and device to improve the detection performance of aluminum profile surface defect classification.

[0112] It should be noted that the deep learning-based aluminum profile surface defect classification method provided in this application can be applied to terminals as well as servers. For example, the terminal can be a smartphone, computer, tablet, smart TV, smartwatch, portable computer terminal, or a desktop computer or other fixed terminal. For ease of explanation, this application uses the terminal as the execution subject for illustration.

[0113] Please see Figure 1 , Figure 1 An embodiment of the deep learning-based aluminum profile surface defect classification method provided in this application includes:

[0114] 101. Obtain defect sample images carrying defect labels from the aluminum profile image training set;

[0115] First, the terminal stores a training set of aluminum profile images for model training. This training set contains a large number of defect sample images of aluminum profiles, each labeled with a corresponding defect tag. Specifically, the defect tag may include the coordinate position of the defect in the image, the defect category, etc. The defect category includes basic defect types of aluminum profiles, such as scratches, corner defects, orange peel, bottom defects, spraying, paint bubbles, pitting, discoloration, and dirt spots. It is important to note that this aluminum profile image training set also includes defect sample images without defects; that is, if an aluminum profile sample image has no defects, the defect category is labeled as "normal" or "normal". Then, the terminal retrieves defect sample images from the aluminum profile image training set so that these defect sample images can be input into the initial model for training. In a possible embodiment, defect sample images in the aluminum profile image training set can be sequentially numbered. For example, if the training set stores 10,000 defect sample images with defect labels, these 10,000 defect sample images can be renamed and numbered as 1.jpg, 2.jpg, ..., 10000.jpg, etc., and then the defect sample images in the training set can be obtained in sequence.

[0116] 102. Input the defect sample image into the initial model, which includes a residual network, a weighted nonlocal module, a feature fusion auxiliary classifier, and a feature mapping module;

[0117] After acquiring defect sample images from the aluminum profile image training set, the terminal inputs these images into the initial model for training. This initial model uses a residual network as its backbone, comprising three segments. Furthermore, a weighted nonlocal module is introduced into the residual network to calculate the similarity between different pixels in the feature maps obtained from the residual network, thereby improving the initial model's ability to capture long-range dependencies and contextual information. Simultaneously, the initial model also incorporates a feature fusion auxiliary classifier to fuse the feature maps obtained before and after the residual network, enhancing the model's ability to extract detailed information from shallow features, such as texture and edge information. Additionally, the initial model includes a feature mapping module to map the feature maps obtained from the residual network or the feature fusion auxiliary classifier. Operations such as flattening and fully connected layers convert the feature maps into label type prediction probabilities to determine the defect category of the image.

[0118] 103. Extract features from the defect sample image using the first segment of the residual network to obtain the first feature map output by the first segment;

[0119] After the terminal inputs the defect sample image into the initial model, it first inputs it into the first segment of the residual network to extract features from the defect sample image and convert it into a first feature map at a preset image size.

[0120] 104. The first feature map is enhanced by using a weighted nonlocal module and the second segment of the residual network to obtain the second feature map;

[0121] After the terminal obtains the first feature map through the first segment of the residual network, it inputs the first feature map into the weighted nonlocal module for feature enhancement. Specifically, the weighted nonlocal module extracts features from the first feature map, calculates the similarity of different pixels of the features, determines the weight vector of different feature pixels, and enhances different features based on the weight vector. Then, the feature-enhanced feature map is input into the second segment of the residual network for feature extraction to obtain a high-resolution second feature map containing rich detailed information but relatively insufficient semantic information.

[0122] 105. Input the second feature map into the third segment of the residual network for feature extraction to obtain the third feature map;

[0123] The terminal inputs the second feature map into the third segment of the residual network for feature extraction, obtaining a low-resolution third feature map that has some loss of detail information but is rich in semantic information.

[0124] 106. The second and third feature maps are fused using a feature fusion auxiliary classifier to obtain the fourth feature map;

[0125] The terminal fuses a high-resolution second feature map (which contains rich detail information but lacks semantic information) obtained from the second segment of the weighted nonlocal module and residual network, and a third feature map (which loses detail information but is rich in semantic information) obtained from the third segment of the residual network, to obtain a fused fourth feature map.

[0126] 107. The loss is calculated on the third and fourth feature maps respectively through the feature mapping module, and the model parameters are updated and optimized based on the loss results using the backpropagation algorithm;

[0127] The terminal uses a feature mapping module to flatten and fully connect the third and fourth feature maps, transforming them into corresponding label type prediction probabilities. Then, it calculates the cross-entropy loss based on the corresponding label type prediction probability vector and the true label values. Based on the calculated cross-entropy loss, the backpropagation algorithm is used to update and optimize the model parameters of the initial model.

[0128] 108. Input defect sample images from the aluminum profile image training set into the initial model and perform the above steps for iterative training until the loss converges or the preset number of iterations is reached to obtain the trained defect classification model. This defect classification model is used for the classification and detection of surface defects of aluminum profiles.

[0129] After updating and optimizing the model parameters of the initial model, the terminal repeats the above steps: obtaining defect sample images from the aluminum profile image training set and inputting them back into the updated initial model for training. Once the loss converges, or after iterative training with the input defect sample images reaches a preset number of iterations (e.g., 10,000 iterations), the model training is complete, resulting in a trained defect classification model capable of classifying and detecting defects in the input aluminum profile images.

[0130] In this embodiment, the defect classification model of this application can be used to detect surface defects in aluminum profiles. Compared with manual visual inspection, it has higher recognition efficiency and accuracy. Compared with traditional inspection, it can adapt to the identification of various defect types, has stronger adaptability, and can input detection in real time. In addition, the defect classification model in this embodiment uses a residual network as the backbone network and introduces a weighted nonlocal module for feature enhancement to calculate the similarity between different pixels in the feature map, thereby improving the model's ability to capture long-distance dependencies and contextual information of features. At the same time, feature fusion is used to assist the classifier in feature fusion, thereby improving the model's ability to mine detailed information features such as texture and edges, reducing the amount of computation, and enhancing the model's detection effect for aluminum profile defect classification. This improves the model's classification performance for aluminum surface defects with less computational cost.

[0131] To make the deep learning-based aluminum profile surface defect classification method provided in this application more obvious and easy to understand, the following is a detailed description of the deep learning-based aluminum profile surface defect classification method provided in this application:

[0132] Please see Figures 2 to 6 , Figure 2 Another embodiment of the deep learning-based aluminum profile surface defect classification method provided in this application includes:

[0133] 201. Obtain defect sample images carrying defect labels from the aluminum profile image training set;

[0134] Step 201 in this embodiment is the same as described above. Figure 1 Step 101 in the illustrated embodiment is similar and will not be described in detail here.

[0135] 202. Input the defect sample image into the initial model, which includes a residual network, a weighted nonlocal module, a feature fusion auxiliary classifier, and a feature mapping module;

[0136] After the terminal obtains defect sample images from the aluminum profile image training set, it inputs these defect sample images into the initial model for model training. (See also: [link to relevant documentation]). Figure 3The initial model, FWACNet, uses a residual network as its backbone and introduces a weighted nonlocal module, a feature fusion auxiliary classifier, and a feature mapping module. Figure 3 (Not shown). The residual network consists of a first segment, a second segment, and a third segment. The first segment includes a backbone module and a first-stage module (Stage 1). The second segment includes a second-stage module (Stage 2), and the third segment includes a third-stage module (Stage 3) and a fourth-stage module (Stage 4). The backbone module consists of a 7×7 convolutional module. The first-stage, second-stage, third-stage, and fourth-stage modules are composed of 3, 4, 6, and 3 residual structure modules, respectively. Each residual structure module (Stage) consists of a 1×1 convolutional module, a 3×3 convolutional module, and a 1×1 convolutional module, respectively.

[0137] 203. Input the defect sample image into the main module and the first stage module in sequence for image feature extraction to obtain the first feature map;

[0138] After the terminal inputs a defect sample image of size H×W×C into the initial model FWACNet, the defect sample image undergoes convolution processing in the backbone module before entering the first-stage module for feature extraction, resulting in the first feature map. Here, H×W represents the image dimensions (height × width), and C represents the number of channels. For example, inputting a defect sample image of size 384×384×3 into a 7×7 convolution module yields a feature map with a spatial size of 96×96×64. Figure X stage1 Then, in Stage 1, feature extraction is performed to obtain the first feature with a spatial size of 96×96×256. Figure X stage2 .

[0139] 204. Input the first feature map into the weighted nonlocal module for feature enhancement to obtain the target feature map;

[0140] Considering that the computational cost of the weighted nonlocal module is affected by the spatial size of the feature map, a weighted nonlocal module is introduced into the second-stage module of the residual network. The first feature map is then input into the weighted nonlocal module for feature enhancement. The data flow process of the first feature map in the weighted nonlocal module is as follows (please refer to [link]). Figure 4 , Figure 4 Here is a diagram of the weighted nonlocal module structure:

[0141] Input a first feature of size H×W×C into the weighted nonlocal module Figure X =[x1,x2,…,x i ], and respectively underwent θ linear transformation, Linear transformation and g-linear transformation yield characteristics Figure Xθ Feature map and characteristics Figure X g .

[0142] Specifically, the linear transformation θ(x) i ) = W θ x i , Linear transformation g linear transformation It is generally implemented using 1×1 convolution, where W θ , X represents the parameters in the convolution kernel. i Represents image pixels.

[0143] Features Figure X θ The input is fed into the channel attention module to obtain the channel attention weight vector λ = [λ 1 ,λ 2 ,…,λ k ,…,λ C ], where C is a feature Figure X θ The number of channels, where k is its index value. The channel attention module is as follows: Figure 5 As shown, this includes the global average pooling operation F. sq With fully connected structure F ex Specifically, the global average pooling formula is defined as follows:

[0144]

[0145] Where H and W are the height and width of the feature map, and i and j represent the pixel indices in the feature map space. Global average pooling operation F sq Input features Figure X θ Perform global average pooling operation to combine features Figure X θ The feature map of size H×W on each channel is compressed into a single real feature number containing global information. Let the compressed real feature number be Z = [z]. 1 ,z 2 ,…,z k ,…,z C ].

[0146] Then, based on the characteristic real number Z = [z 1 ,z 2 ,…,z k ,…,z C Perform a fully connected operation to obtain the channel attention weight vector λ = [λ 1 ,λ 2 ,…,λk ,…,λ C Specifically, the full connection operation is performed based on the following full connection formula:

[0147] F ex (Z,W)=σ(W2δ(W1Z))

[0148] Where δ and σ represent the ReLU and Sigmoid activation functions, respectively, and W1∈R (C / r)×C and W2∈R C×(C / r) Represents fully connected FC1 and FC2, R represents the real number domain space of the fully connected system, and r represents the preset parameter. r can be set manually according to the actual situation, and the specific setting is not limited here.

[0149] Therefore, the channel attention weight vector λ is calculated as follows:

[0150]

[0151] The channel attention weight vector λ is compared with the feature Figure X θ Perform element-wise multiplication, then combine with the feature map. Perform matrix multiplication to obtain the similarity matrix S M The similarity matrix S M Softmax normalization is performed to obtain the spatial attention map A. M The specific calculation method is as follows:

[0152]

[0153] A M (i,j)=Softmax(S M (i,j))

[0154] Where i,j represent pixel indices in the feature map space, 1≤i≤H×W, 1≤j≤H×W.

[0155] Then, the spatial attention map A of size HW×HW is... M With features Figure X g Matrix multiplication is performed, followed by convolution using a 1×1 convolution module to increase the channel dimension, allowing element-wise addition with the first feature map. Finally, the convolutional feature map is compared with the first feature map. Figure X Perform element-wise addition to obtain the target feature map V = [v1, v2, ..., v]. i ].

[0156] Specifically, the calculation method is as follows:

[0157]

[0158] In this embodiment, the similarity between different pixels is calculated using a weighted nonlocal module, for example, for the first feature. Figure X For a given pixel A, its similarity is calculated with all pixels in the feature map. Therefore, by calculating the similarity between different pixels, the model's ability to capture long-range dependencies and contextual information of features is improved.

[0159] 205. Input the target feature map into the second-stage module for feature extraction to obtain the second feature map;

[0160] The terminal inputs the target feature map output from the weighted nonlocal module into the second-stage module to extract features from the target feature map, obtaining a high-resolution second feature map containing rich detailed information but insufficient semantic information. For example, the first feature map with a spatial size of 96×96×256... Figure X stage2 After obtaining a target feature map of the same size through the weighted nonlocal module, the target feature map is input into the second-stage module Stage2 to obtain a large-resolution second target feature map with a spatial size of 48×48×512, which contains rich details such as texture and edge information but lacks semantic information. Figure X stage3 .

[0161] 206. Input the second feature map into the third stage module and the fourth stage module in sequence for image feature extraction to obtain the third feature map;

[0162] The terminal inputs the second feature map into the third and fourth stages of the residual network for feature extraction, obtaining a small-resolution third feature map that has lost some detail information but is rich in semantic information. For example, the second feature map with a size of 48×48×512 is input into the third stage module Stage3 to obtain a feature map with a size of 24×24×1024, and then input into the fourth stage module Stage4 to obtain a third feature map with a size of 12×12×2048.

[0163] 207. Input the second feature map and the third feature map into the feature fusion auxiliary classifier, and upsample the third feature map by subpixel convolution so that the spatial size of the third feature map is the same as that of the second feature map;

[0164] Please see Figure 6 , Figure 6This is a schematic diagram of the feature fusion auxiliary classifier structure introduced in the model. In this embodiment, the terminal fuses the high-resolution second feature map (which contains rich detail but insufficient semantic information after Stage 2) with the third feature map (which loses detail but is rich in semantic information after Stages 3 and 4) to obtain the fused fourth feature map. Specifically, D represents the deep feature extracted by the residual network, i.e., the third feature map. S represents the shallow feature extracted by the residual network, i.e., the second feature map. The data flow transmission process of the feature fusion auxiliary classifier is as follows:

[0165] First, the third feature map is upsampled using subpixel convolution, making its feature map spatial size similar to that of the second feature map. Figure 1 This yields the third feature map after sampling. Specifically, sub-pixel convolution utilizes channel information to fill in the spatial representation of the feature map, achieving feature map D∈R through the filling formula. H×W×C to D1∈R 4H×4W×(C / 16) The mapping. The expression for the filling formula is as follows:

[0166]

[0167] Among them, PS(D) x,y,c This function outputs the feature pixels with index (x, y, c) of the feature map. RoundDown is the floor function, and mod is the modulo function.

[0168] 208. The second feature map and the sampled third feature map are concatenated to obtain the fourth feature map;

[0169] The terminal concatenates the sampled third feature map, output after 207 steps, with the second feature map along the channel dimension to obtain the fourth feature map. Specifically, the two feature maps, namely the third and second feature maps, are superimposed and fused into a single feature map, namely the fourth feature map. In this embodiment, by using a feature fusion-assisted classifier to fuse the feature maps of deep features extracted by the residual network and the feature maps of shallow features, the model's ability to extract detailed information from shallow features can be improved, while reducing the computational load of the model.

[0170] 209. The third feature map is flattened, fully connected, and normalized using the feature mapping module to determine the first prediction probability vector of the third feature map; the cross-entropy loss is calculated based on the first prediction probability vector and the true label value to obtain the first loss.

[0171] The feature mapping module includes a flattening module, a fully connected module, and a softmax batch normalization module. The terminal uses the feature mapping module to sequentially perform flattening, fully connected, and softmax normalization operations on the third feature map to determine the first predicted probability vector of the third feature map. Specifically, the third feature map is flattened into a one-dimensional vector, passed to a fully connected layer for a fully connected operation, and then the output of the fully connected layer is fed into the softmax normalization module to obtain the first predicted probability vector of the input image, i.e., the defect sample image. The length of this first predicted probability vector is determined by the number of defect label categories in the aluminum profile image training set. Then, the cross-entropy loss is calculated using this first predicted probability vector and the true values ​​of the labels carried by the defect sample image to obtain the first loss.

[0172] 210. Perform flattening, fully connected, and softmax normalization operations on the fourth feature map to determine the second prediction probability vector of the fourth feature map; calculate the cross-entropy loss based on the second prediction probability vector and the true label value to obtain the second loss;

[0173] The terminal takes the fourth feature map, obtained by fusing the second and fourth feature maps after step 208, and flattens and fully connected the map after stacking two convolutional blocks to determine the second prediction probability vector of the fourth feature map. This convolutional block includes a 3×3 convolutional block, a batch normalization module, and a ReLU module. Specifically, the formula for calculating the second prediction probability vector is as follows:

[0174] Predict Probability=W5δ(W4δ(W3[PS(D),S]))

[0175] Where W3 and W4 are convolution parameters, W5 is the fully connected layer parameter, and δ is the ReLU activation function. The length of this second predicted probability vector is determined by the number of defect label categories in the aluminum profile image training set. Then, the cross-entropy loss is calculated using this second predicted probability vector and the ground truth label values ​​carried by the defect sample images to obtain the second loss.

[0176] 211. Calculate the total loss based on the first loss and the second loss, and update and optimize the model parameters using the backpropagation algorithm based on the total loss result;

[0177] The terminal adds the first loss and the second loss to obtain the total loss. Based on the calculated total loss, the backpropagation algorithm is used to update and optimize the model parameters of the initial model.

[0178] 212. Input defect sample images from the aluminum profile image training set into the initial model and perform the above steps for iterative training until the loss converges or the preset number of iterations is reached to obtain the trained defect classification model. This defect classification model is used for the classification and detection of surface defects of aluminum profiles.

[0179] Step 212 in this embodiment is the same as described above. Figure 1 Step 108 in the illustrated embodiment is similar and will not be described in detail here.

[0180] In this embodiment, the defect sample image is first input into the initial model. Features are extracted from the defect sample image through the backbone module and the first-stage module of the residual network, resulting in a first feature map. The first feature map is then input into the weighted nonlocal module and the second-stage module of the residual network to obtain a second feature map. The second feature map is then sequentially input into the third-stage module and the fourth-stage module to obtain a third feature map. The third feature map is then flattened, fully connected, and softmaxed sequentially to obtain a first predicted probability vector for the defect sample image. The first predicted probability vector is then compared with the true label value to calculate the cross-entropy loss, resulting in a first loss. Simultaneously, the second and third feature maps are input into a feature fusion auxiliary classifier for feature fusion, and a second loss is calculated. The first and second losses are added to obtain the total loss. Based on the total loss, the backpropagation algorithm is used to automatically optimize and update the model parameters. After the model parameters are updated, the above steps are repeated, and defect sample images from the training set are input again for training. When the total loss converges or the number of iterations reaches a preset number, the model training is complete, resulting in the final defect classification model. This defect classification model can then be used for the automatic detection and classification of surface defects on aluminum materials.

[0181] In this embodiment, the defect classification model uses a residual network as the backbone network and introduces a weighted nonlocal module for feature enhancement to calculate the similarity between different pixels in the feature map, thereby improving the model's ability to capture long-distance dependencies and contextual information of features. At the same time, a feature fusion-assisted classifier is used to perform feature fusion, thereby improving the model's ability to mine detailed features such as texture and edges, reducing computational load, and enhancing the model's detection effect on aluminum profile defect classification. This improves the model's classification performance of aluminum surface defects with less computational cost.

[0182] The following is an example of using the defect classification model provided in this application for surface defect classification and detection of aluminum profiles. Please refer to... Figure 7 , Figure 7 Another embodiment of the deep learning-based aluminum profile surface defect classification method provided in this application includes:

[0183] 301. Obtain an image of the aluminum material to be inspected;

[0184] When a surface defect classification and inspection of a certain aluminum profile is required, a camera is used to capture an image of the aluminum profile to be inspected.

[0185] 302. Input the image of the aluminum material to be inspected into the defect classification model, and extract features from the image of the aluminum material to be inspected through the first segment of the residual network to obtain the first feature map output by the first segment;

[0186] The terminal inputs the image of the aluminum material to be inspected into the defect classification model, which then uses the above-mentioned... Figure 1 or Figure 2 The method shown in the embodiment is used for training. First, the image of the aluminum material to be detected is input into the first segment of the model's residual network to obtain the first feature map. Specifically, the image of the aluminum material to be detected is input into the backbone module and the first stage module to obtain the first feature map.

[0187] 303. The first feature map is enhanced by using a weighted nonlocal module and the second segment of the residual network to obtain a second feature map;

[0188] The first feature map is sequentially input into the second-stage module of the weighted nonlocal module and the residual network for feature extraction to obtain the second feature map.

[0189] 304. Input the second feature map into the third segment of the residual network for feature extraction to obtain the third feature map;

[0190] The second feature map is input into the third segment of the residual network for feature extraction to obtain the third feature map. Specifically, the second feature map is sequentially input into the third-stage module and the fourth-stage module of the residual network for feature extraction to obtain the third feature map.

[0191] 305. The third feature map is predicted by the feature mapping module to determine the category of surface defects in the aluminum material image to be detected by the defect classification model based on the calculation results.

[0192] In this embodiment, the terminal performs flattening, fully connected, and softmax normalization operations on the third feature map sequentially through the model's feature mapping module to determine the category prediction probability of the third feature map. Based on the defect category with the highest probability among the category prediction probabilities, the surface defect category of the aluminum image to be detected is output.

[0193] In this embodiment, the defect classification model of this application can be used to detect surface defects in aluminum profiles. Compared with manual visual inspection, it has higher recognition efficiency and accuracy. Compared with traditional detection, it can adapt to the recognition of various defect types, has stronger adaptability, and can input detection in real time. In addition, since the defect classification model has been trained, after determining the category prediction probability of the third feature map by performing flattening, fully connected operation, and softmax normalization operation on the obtained third feature map in sequence, the surface defect category of the aluminum image to be detected can be directly output according to the defect category with the highest probability among the category prediction probabilities. There is no need for a feature fusion auxiliary classifier to participate in the recognition, thus improving the model's classification performance for defect images with less computational cost.

[0194] The above describes the deep learning-based aluminum profile surface defect classification method provided in this application. The following describes the deep learning-based aluminum profile surface defect classification device provided in this application:

[0195] Please see Figure 8 , Figure 8 An embodiment of the deep learning-based aluminum profile surface defect classification device provided in this application includes:

[0196] The acquisition unit 401 is used to acquire defect sample images carrying defect labels from the aluminum profile image training set;

[0197] The input unit 402 is used to input the defect sample image into the initial model, which includes a residual network, a weighted nonlocal module, a feature fusion auxiliary classifier, and a feature mapping module.

[0198] The first extraction unit 403 is used to extract features from the defect sample image through the first segment of the residual network and obtain the first feature map output by the first segment.

[0199] The second extraction unit 404 is used to perform feature enhancement on the first feature map through the second segment of the weighted nonlocal module and the residual network to obtain the second feature map;

[0200] The third extraction unit 405 is used to input the second feature map into the third segment of the residual network for feature extraction to obtain the third feature map;

[0201] The fusion unit 406 is used to fuse the second feature map and the third feature map through a feature fusion auxiliary classifier to obtain a fourth feature map.

[0202] The optimization unit 407 is used to calculate the loss of the third feature map and the fourth feature map through the feature mapping module, and update and optimize the model parameters based on the loss result using the backpropagation algorithm.

[0203] The input unit 402 is also used to input defect sample images from the aluminum profile image training set into the initial model and perform the above steps for iterative training until the loss converges or the preset number of iterations is reached, so as to obtain the trained defect classification model. The defect classification model is used to classify and detect surface defects of aluminum profiles.

[0204] Optionally, the residual network includes a backbone module, a first-stage module, a second-stage module, a third-stage module, and a fourth-stage module, wherein the first-stage module, the second-stage module, the third-stage module, and the fourth-stage module are composed of 3, 4, 6, and 3 residual structure modules, respectively.

[0205] The first extraction unit 403 is specifically used to input the defect sample image into the main module and the first stage module in sequence for image feature extraction to obtain the first feature map;

[0206] The second extraction unit 404 is specifically used to input the first feature map into the weighted nonlocal module for feature enhancement to obtain the target feature map;

[0207] The target feature map is input into the second-stage module for feature extraction to obtain the second feature map;

[0208] The third extraction unit 405 is specifically used to input the second feature map into the third stage module and the fourth stage module in sequence for image feature extraction to obtain the third feature map;

[0209] The optimization unit 407 is specifically used to perform flattening, fully connected and softmax normalization operations on the third feature map through the feature mapping module to determine the first prediction probability vector of the third feature map; and to calculate the cross-entropy loss based on the first prediction probability vector and the true label value to obtain the first loss.

[0210] The fourth feature map is flattened, fully connected, and normalized using softmax to determine the second prediction probability vector of the fourth feature map; the cross-entropy loss is calculated based on the second prediction probability vector and the true label value to obtain the second loss.

[0211] The total loss is calculated based on the first loss and the second loss, and the model parameters are updated and optimized using the backpropagation algorithm based on the total loss result.

[0212] Optionally, the weighted nonlocal module includes a channel attention module, and the second extraction unit 404 is specifically used to input a first feature map of size H×W×C into the weighted nonlocal module, and perform θ linear transformation, Linear transformation and g-linear transformation yield characteristics Figure X θ Feature map and characteristics Figure X g ;

[0213] Features Figure X θ The input is fed into the channel attention module to obtain the channel attention weight vector λ = [λ 1 ,λ 2 ,…,λ k ,…,λ C ], where C is a feature Figure X θ The number of channels, where k is the index value;

[0214] The channel attention weight vector λ is compared with the feature Figure X θ Perform element-wise multiplication, then combine with the feature map. Perform matrix multiplication to obtain the similarity matrix S M ;

[0215] The similarity matrix S M Softmax normalization is performed to obtain the spatial attention map A. M ;

[0216] Spatial attention map A M With features Figure X g After performing matrix multiplication, a 1×1 convolution module is used for convolution;

[0217] The target feature map is obtained by adding corresponding elements of the convolutional feature map to the first feature map.

[0218] Optionally, the second extraction unit 404 is specifically used to extract features Figure X θ Input to the channel attention module;

[0219] Features are analyzed using the global average pooling formula. Figure X θ Perform global average pooling to ensure that the features are... Figure X θ Compressed into feature real numbers Z = [z] with global information 1 ,z 2 ,…,z k ,…,z C ];

[0220] Based on the characteristic real number Z = [z 1 ,z 2 ,…,z k ,…,z C Based on the fully connected formula, perform fully connected operations to obtain the channel attention weight vector λ = [λ]. 1 ,λ 2 ,…,λ k ,…,λ C].

[0221] Optionally, the fusion unit 406 is specifically used to input the second feature map and the third feature map into the feature fusion auxiliary classifier, and to upsample the third feature map through sub-pixel convolution so that the spatial size of the third feature map is the same as that of the second feature map.

[0222] The second feature map and the sampled third feature map are concatenated to obtain the fourth feature map.

[0223] In this embodiment, the functions performed by each unit are the same as those described above. Figure 1 or Figure 2 The steps in the method embodiments shown correspond to each other, and will not be repeated here.

[0224] In this embodiment, the trained defect classification model uses a residual network as the backbone network and introduces a weighted nonlocal module for feature enhancement to calculate the similarity between different pixels in the feature map, thereby improving the model's ability to capture long-distance dependencies and contextual information of features. At the same time, a feature fusion auxiliary classifier is used to perform feature fusion, thereby improving the model's ability to mine detailed features such as texture and edges, reducing computational load, and enhancing the model's detection effect on aluminum profile defect classification. This improves the model's classification performance of aluminum surface defects with less computational cost.

[0225] Please see Figure 9 , Figure 9 Another embodiment of the deep learning-based aluminum profile surface defect classification device provided in this application includes:

[0226] Acquisition unit 501 is used to acquire an image of the aluminum material to be detected;

[0227] The input unit 502 is used to input the image of the aluminum material to be detected into the defect classification model, and to extract features from the image of the aluminum material to be detected through the first segment of the residual network to obtain the first feature map output by the first segment.

[0228] The feature enhancement unit 503 is used to enhance the features of the first feature map through the second segment of the weighted nonlocal module and the residual network to obtain the second feature map.

[0229] Extraction unit 504 is used to input the second feature map into the third segment of the residual network for feature extraction to obtain the third feature map;

[0230] The output unit 505 is used to calculate the category prediction probability of the third feature map through the feature mapping module, and determine the surface defect category of the aluminum material image to be detected output by the defect classification model based on the calculation result.

[0231] In this embodiment, the functions performed by each unit are the same as those described above. Figure 7The steps in the method embodiments shown correspond to each other, and will not be repeated here.

[0232] In this embodiment, the defect classification model of this application can be used to detect surface defects in aluminum profiles. Compared with manual visual inspection, it has higher recognition efficiency and accuracy. Compared with traditional detection, it can adapt to the recognition of various defect types, has stronger adaptability, and can input detection in real time. In addition, since the defect classification model has been trained, after determining the category prediction probability of the third feature map by performing flattening, fully connected operation, and softmax normalization operation on the obtained third feature map in sequence, the surface defect category of the aluminum image to be detected can be directly output according to the defect category with the highest probability among the category prediction probabilities. There is no need for a feature fusion auxiliary classifier to participate in the recognition, thus improving the model's classification performance for defect images with less computational cost.

[0233] This application also provides a deep learning-based device for classifying surface defects in aluminum profiles. Please refer to [link to relevant documentation]. Figure 10 , Figure 10 An embodiment of the deep learning-based aluminum profile surface defect classification device provided in this application includes:

[0234] Processor 601, memory 602, input / output unit 603, bus 604;

[0235] The processor 601 is connected to the memory 602, the input / output unit 603, and the bus 604;

[0236] The memory 602 stores a program, and the processor 601 calls the program to execute any of the deep learning-based aluminum profile surface defect classification methods described above.

[0237] This application also relates to a computer-readable storage medium storing a program that, when run on a computer, causes the computer to execute any of the deep learning-based aluminum profile surface defect classification methods described above.

[0238] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0239] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0240] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0241] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0242] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for classifying surface defects of aluminum profiles based on deep learning, characterized in that, The method includes: Obtain defect sample images carrying defect labels from the aluminum profile image training set; The defect sample image is input into the initial model, which includes a residual network, a weighted nonlocal module, a feature fusion auxiliary classifier, and a feature mapping module. The first feature map output by the first segment of the residual network is obtained by extracting features from the defect sample image. The first feature map is enhanced by the weighted nonlocal module and the second segment of the residual network to obtain the second feature map; The second feature map is input into the third segment of the residual network for feature extraction to obtain the third feature map; The second feature map and the third feature map are fused using the feature fusion auxiliary classifier to obtain a fourth feature map; The feature mapping module performs loss calculations on the third and fourth feature maps, and updates and optimizes the model parameters based on the loss results using the backpropagation algorithm. Input defect sample images from the aluminum profile image training set into the initial model and perform the above steps for iterative training until the loss converges or the preset number of iterations is reached to obtain the trained defect classification model. The defect classification model is used for the classification and detection of surface defects of aluminum profiles. The residual network includes a backbone module, a first-stage module, a second-stage module, a third-stage module, and a fourth-stage module. The first-stage module, the second-stage module, the third-stage module, and the fourth-stage module are composed of 3, 4, 6, and 3 residual structure modules, respectively. The step of extracting features from the defect sample image through the first segment of the residual network to obtain the first feature map output by the first segment includes: The defect sample image is sequentially input into the main module and the first stage module for image feature extraction to obtain the first feature map; The step of enhancing the first feature map using the weighted nonlocal module and the second segment of the residual network to obtain the second feature map includes: The first feature map is input into the weighted nonlocal module for feature enhancement to obtain the target feature map; The target feature map is input into the second stage module for feature extraction to obtain the second feature map; The step of inputting the second feature map into the third segment of the residual network for feature extraction to obtain the third feature map includes: The second feature map is sequentially input into the third-stage module and the fourth-stage module for image feature extraction to obtain the third feature map; The loss calculation of the third feature map and the fourth feature map by the feature mapping module includes: The feature mapping module performs flattening, full connection, and softmax normalization operations on the third feature map to determine the first prediction probability vector of the third feature map; the cross-entropy loss is calculated based on the first prediction probability vector and the true label value to obtain the first loss. The fourth feature map is flattened, fully connected, and normalized using softmax to determine the second prediction probability vector of the fourth feature map; the cross-entropy loss is calculated based on the second prediction probability vector and the true label value to obtain the second loss. Calculate the total loss based on the first loss and the second loss; The residual structure module includes two 1×1 convolutional modules and one 3×3 convolutional module, and the backbone module is a 7×7 convolutional module. The weighted nonlocal module includes a channel attention module. The step of inputting the first feature map into the weighted nonlocal module for feature enhancement to obtain the target feature map includes: A first feature map of size H×W×C is input into the weighted nonlocal module, and then subjected to linear transformations θ, φ, and g to obtain the feature map X. θ Feature map X φ and feature map X g ; The feature map X θ The input is fed into the channel attention module to obtain the channel attention weight vector λ=[λ 1 ,λ 2 ,…,λ k ,…,λ C ], where C is the feature map X θ The number of channels, where k is the index value; The channel attention weight vector λ is compared with the feature map X. θ Perform element-wise multiplication, then multiply with the feature map X. φ Perform matrix multiplication to obtain the similarity matrix S M ; The similarity matrix S M Softmax normalization is performed to obtain the spatial attention map A. M ; The spatial attention map A M With the feature map X g After performing matrix multiplication, a 1×1 convolution module is used for convolution; The convolutional feature map is added to the first feature map element by element to obtain the target feature map. The step of fusing the second feature map and the third feature map using the feature fusion auxiliary classifier to obtain the fourth feature map includes: The second feature map and the third feature map are input into the feature fusion auxiliary classifier, and the third feature map is upsampled by subpixel convolution so that the third feature map and the second feature map have the same spatial size. The second feature map and the sampled third feature map are concatenated to obtain the fourth feature map.

2. The method for classifying surface defects of aluminum profiles according to claim 1, characterized in that, The feature map X θ The input is fed into the channel attention module to obtain the channel attention weight vector λ=[λ 1 ,λ 2 ,…,λ k ,…,λ C ]include: The feature map X θ Input to the channel attention module; The feature map X is processed using the global average pooling formula. θ Perform a global average pooling operation to make the feature map X... θ Compressed into feature real numbers Z=[z] that have global information 1 ,z 2, …,z k ,…,z C ]; According to the characteristic real number Z=[z 1 ,z 2 ,…,z k ,…,z C Based on the fully connected formula, a fully connected operation is performed to obtain the channel attention weight vector λ=[λ 1 ,λ 2 ,…,λ k ,…,λ C ].

3. The method for classifying surface defects of aluminum profiles according to claim 2, characterized in that, The global average pooling formula is defined as follows: Where H and W are the height and width of the feature map, and i and j represent the pixel indices in the feature map space; The fully connected formula is defined as follows: Where δ and σ represent the ReLU and Sigmoid activation functions, respectively, and W1∈R (C / r)×C W2∈R C×(C / r) R represents the fully connected real number space, and r represents the preset parameter; The channel attention weight vector λ is calculated as follows: 。 4. A method for classifying surface defects of aluminum profiles based on deep learning, characterized in that, The defect classification model used in the method is trained using the method described in any one of claims 1 to 3, wherein the method comprises: Acquire an image of the aluminum material to be inspected; The image of the aluminum material to be detected is input into the defect classification model, and the first feature map output by the first segment of the residual network is obtained by extracting features from the image of the aluminum material to be detected. The first feature map is enhanced by using a weighted nonlocal module and the second segment of the residual network to obtain a second feature map; The second feature map is input into the third segment of the residual network for feature extraction to obtain the third feature map; The third feature map is subjected to category prediction probability calculation by the feature mapping module, and the surface defect category of the aluminum material image to be detected is determined based on the calculation result.

5. A deep learning-based surface defect classification device for aluminum profiles, characterized in that, The apparatus for performing the method as described in any one of claims 1 to 3 includes: The acquisition unit is used to acquire defect sample images carrying defect labels from the aluminum profile image training set; An input unit is used to input the defect sample image into an initial model, the initial model including a residual network, a weighted nonlocal module, a feature fusion auxiliary classifier, and a feature mapping module; The first extraction unit is used to extract features from the defect sample image through the first segment of the residual network and obtain the first feature map output by the first segment. The second extraction unit is used to perform feature enhancement on the first feature map through the weighted nonlocal module and the second segment of the residual network to obtain a second feature map; The third extraction unit is used to input the second feature map into the third segment of the residual network for feature extraction to obtain the third feature map; The fusion unit is used to fuse the second feature map and the third feature map using the feature fusion auxiliary classifier to obtain a fourth feature map; The optimization unit is used to perform loss calculation on the third feature map and the fourth feature map through the feature mapping module, and update and optimize the model parameters based on the loss result using the backpropagation algorithm. The input unit is also used to input defect sample images from the aluminum profile image training set into the initial model and perform the above steps for iterative training until the loss converges or the preset number of iterations is reached, so as to obtain the trained defect classification model. The defect classification model is used for the classification and detection of surface defects of aluminum profiles.

6. A deep learning-based surface defect classification device for aluminum profiles, characterized in that, The defect classification model used in the device is trained by the method described in any one of claims 1 to 3. The device comprises: The acquisition unit is used to acquire an image of the aluminum material to be inspected; The input unit is used to input the image of the aluminum material to be detected into the defect classification model, and to extract features from the image of the aluminum material to be detected through the first segment of the residual network to obtain the first feature map output by the first segment. The feature enhancement unit is used to enhance the first feature map by using a weighted nonlocal module and the second segment of the residual network to obtain a second feature map. An extraction unit is used to input the second feature map into the third segment of the residual network for feature extraction to obtain a third feature map; The output unit is used to perform category prediction probability calculation on the third feature map through the feature mapping module, and determine the surface defect category of the aluminum material image to be detected output by the defect classification model based on the calculation result.

Images