An industrial defect detection method based on incremental training and CBAM enhancement
By using incremental training and CBAM enhancement, a SuperSimpleNet network integrating CBAM modules was constructed, which solved the problems of low detail recognition accuracy and easy forgetting during incremental training in industrial defect detection of lightweight networks, and achieved efficient and accurate industrial defect detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TONGYOU INTELLIGENT EQUIP (JIANGSU) CO LTD
- Filing Date
- 2026-02-05
- Publication Date
- 2026-06-09
AI Technical Summary
Lightweight networks struggle to simultaneously address the issues of low accuracy in identifying detailed defects and the tendency to forget historical defect knowledge during incremental training in industrial defect detection, thus failing to meet the flexible demands of industrial production.
We employ an incremental training and CBAM enhancement approach. By constructing a SuperSimpleNet network that integrates CBAM modules and combining Fisher information matrix and EWC regularization term, we achieve lightweight feature extraction and incremental expansion, avoiding the forgetting of existing defective features.
It significantly improves the accuracy of identifying minor defects, reduces the false negative rate, reduces training time and data requirements, adapts to the real-time detection requirements of industry, and meets the needs of flexible production.
Smart Images
Figure CN122175873A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of industrial defect detection technology, specifically an industrial defect detection method based on incremental training and CBAM enhancement. Background Technology
[0002] In industrial production, surface defect detection is a core component in ensuring product quality. As the manufacturing industry moves towards higher efficiency and greater flexibility, automated detection methods based on convolutional neural networks (CNNs) are gradually replacing traditional methods that rely on manual visual inspection, meeting the quality inspection efficiency requirements of modern production lines.
[0003] Lightweight networks (such as SuperSimpleNet) have become the preferred solution for real-time industrial inspection scenarios due to their advantages of fewer parameters, faster inference speed, and support for incremental training. However, in pursuit of lightweight design, these networks have limited feature extraction depth and insufficient ability to capture features of minor defects such as tiny scratches and pinholes, resulting in generally low accuracy in identifying such defects. At the same time, the types of defects in industrial production are dynamically added as raw materials change and processes are adjusted. When lightweight networks undergo incremental training, they are prone to the "catastrophic forgetting" phenomenon, that is, when learning new defect knowledge, they lose the ability to identify historical defects. This requires full training to make up for the loss, which is both time-consuming and difficult to adapt to the needs of flexible production.
[0004] In summary, while lightweight incremental networks maintain lightweight design and incremental learning capabilities, they suffer from insufficient accuracy in identifying detailed defects and are prone to forgetting historical defect knowledge during incremental training. They cannot simultaneously meet the comprehensive requirements of industrial inspection for accuracy in detail identification, incremental adaptability, and real-time performance, making them a key pain point that urgently needs to be addressed in the field of industrial defect detection. Summary of the Invention
[0005] The purpose of this invention is to provide an industrial defect detection method based on incremental training and CBAM enhancement to solve the problems raised in the prior art.
[0006] To achieve the above objectives, the present invention provides the following technical solution: an industrial defect detection method based on incremental training and CBAM enhancement, the method comprising the following steps: Step 1: Obtain surface images of industrial products containing existing defects and newly added defects. After annotation, data augmentation, and dataset partitioning, obtain existing defect datasets and newly added defect datasets. To provide high-quality, comprehensive training data for network training, ensuring that the model can learn existing defect features and adapt to newly added defect types. Industrial product surface images can be acquired through industrial cameras, CCD sensors, and other devices, covering surface images of various industrial products such as metal sheets, plastic casings, and electronic components. Labeling operations can utilize mainstream labeling tools such as LabelImg and VGGImageAnnotator, classifying and labeling according to defect types (e.g., scratches, dents, pinholes, hairline cracks, etc.). Data augmentation aims to expand the dataset size and improve the model's generalization ability. Normalization specifically maps image pixel values to the [0,1] interval, random flipping includes horizontal and vertical flipping, and random cropping extracts fixed-size sub-images (e.g., 64×64) from the original image. Dataset partitioning is based on the labeling results of defect types. For example, images and labels corresponding to long-standing defects such as scratches and dents are classified as existing defect datasets, while images and labels corresponding to newly added defects such as pinholes due to raw material changes and hairline cracks due to process adjustments are classified as new defect datasets.
[0007] Step 2: Construct an incremental SuperSimpleNet network that integrates the CBAM module and initialize the network weights; A basic network architecture combining lightweight design, feature enhancement capabilities, and incremental expansion is constructed. The SuperSimpleNet network structure consists of an input layer, M lightweight convolutional blocks, N max-pooling layers (GAP), a classification layer, and an output layer. M and N are preset numbers, preferably M=3 and N=3, which ensures sufficient feature extraction depth while maintaining the network's lightweight characteristics. The lightweight convolutional blocks employ a Conv-BN-ReLU structure, where Conv represents the convolution operation, BN (Batch Normalization) is used to accelerate network training and alleviate the gradient vanishing problem, and ReLU is the activation function used to introduce non-linear features. The CBAM module includes a channel attention submodule (CAM) and a spatial... The attention submodule (SAM) is integrated sequentially into the SuperSimpleNet network after the lightweight convolutional block and before the max pooling layer. This integration position enhances effective features after feature extraction and before feature compression, avoiding the loss of subtle defect features due to pooling operations. The channel attention submodule evaluates the importance of each channel of the feature map, enhancing key channel features (such as channels corresponding to edges and textures). The spatial attention submodule evaluates the spatial dimension importance of the feature map after channel attention enhancement, enhancing the spatial features of defective regions (such as the location and shape of defects), ultimately yielding the enhanced feature map output by the CBAM module. Step 3: Train the network based on the existing defect dataset, optimize the network weights, and save the initial model and Fisher information matrix; The network learns the features of existing defects to form a stable initial model, and records the key weight information for identifying existing defects, providing a basis for anti-forgetting constraints in subsequent incremental training. During training, gradient descent is used to minimize the cross-entropy loss function, continuously adjusting the network weights θ to make the model's predicted probability of existing defects as close as possible to the true label. The Fisher information matrix F is used to quantify the importance of each weight to the initial defect identification. For example, if a weight corresponds to the extraction of defect edge features, and it has a significant impact on the identification accuracy in the initial training, then the corresponding Fisher value F is [value missing]. j The value is relatively high, and the update range of this weight needs to be carefully constrained in subsequent incremental training to avoid forgetting the ability to extract edge features; Step 4: When adding a new defect type, expand the network classification layer dimension, load the initial model weights, and use the new defect dataset to incrementally train and update the network in conjunction with the EWC regularization term. Without reconstructing the network or forgetting existing defect knowledge, the model can quickly learn new defect features. The core of the classification layer extension is to adapt to the number of new defect categories without changing the structure of the network's feature extraction part, significantly saving training resources. The EWC regularization term constrains the update of key weights, with λ controlling the constraint strength (preferably λ within the range of 10). 2 ~10 3 This approach allows for only minor adjustments to the weights with high Fisher scores, ensuring the model can learn new defect features while avoiding a decline in performance in recognizing existing defects. Incremental training uses the Adam optimizer with a learning rate set to 10%. -4 This learning rate is lower than the initial training rate, which can further avoid forgetting caused by large fluctuations in weights.
[0008] Step 5: Preprocess the image to be detected, input it into the trained network, and output the defect category and confidence level.
[0009] This technology enables rapid and accurate detection of surface defects in industrial products, providing a basis for quality inspection decisions on the production line. Preprocessing operations must be consistent with the preprocessing of the training data to ensure a uniform distribution of input features. Confidence level indicates the model's reliability in predicting categories; for example, a confidence level of 0.95 indicates that the model is 95% certain the defect belongs to the predicted category. In practical applications, a confidence threshold (e.g., 0.8) can be set. Detection results below the threshold can trigger manual review, further improving detection reliability. Furthermore, in step 1, the surface image of the industrial product includes images corresponding to defects that have already appeared in industrial production and images corresponding to newly added defects; The annotation specifically refers to the type of defect in the image; The data augmentation specifically includes: normalizing the image, randomly flipping it, and randomly cropping it; The dataset partitioning refers to dividing the images and their annotations based on the annotation information to obtain existing defect datasets and newly added defect datasets.
[0010] Furthermore, in step 2, the SuperSimpleNet network structure consists of an input layer, M lightweight convolutional blocks, N max pooling layers (GAP), a classification layer, and an output layer. Wherein, M and N represent preset quantities; the lightweight convolutional block is a Conv-BN-ReLU structure; The CBAM module has a channel attention submodule CAM and a spatial attention submodule SAM, which are integrated into the SuperSimpleNet network after the lightweight convolutional block and before the max pooling layer using a serial embedding method. The channel attention submodule enhances key channel features by evaluating the importance of each channel in the feature map; The spatial attention submodule performs spatial dimension importance assessment on the feature map after channel attention enhancement, thereby strengthening the spatial features of the defective region. The feature map output by the CBAM module is obtained.
[0011] The channel attention submodule enhances key channel features by evaluating the importance of each channel in the feature map, specifically including: For the input feature map F∈[C,H,W], global max pooling and global average pooling are performed on the feature map respectively. Channel weights are obtained from the shared multilayer perceptron (MLP). After weighting, the features of defect-related channels are enhanced, while the background channels are suppressed: M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))); Where F is the input feature map; C is the number of channels; H is the height; W is the width; σ(·) is the Sigmoid activation function; M c represents the channel attention weights; AvgPool(·) is the average pooling operation; MaxPool(·) is the max pooling operation.
[0012] The spatial attention submodule performs a spatial dimension importance assessment on the feature map after channel attention enhancement, and the enhancement of spatial features of defective regions specifically includes: Feature map M after channel attention weighting c (F) is multiplied by the original input feature map F to obtain feature map F', F'=M c (F)⨂F, performs max pooling and average pooling on the channel dimension respectively, and after concatenation, generates spatial attention weights through m×n and n×m convolutions: M s (F')=σ(Convn×m (Conv m×n (Concat(MaxPool(F');AvgPool(F'))))); In this design, m=1 and n=7 are preferred, meaning 1×7 and 7×1 asymmetric convolution kernels are used instead of the traditional 3×3 convolution kernel. While maintaining the receptive field of the 7×7 kernel (which is much larger than that of the 3×3 kernel), the number of parameters is only slightly increased. This allows for more accurate capture of common industrial defects such as long strips (e.g., scratches, hairline cracks) or irregular minute defects. Simultaneously, convolution decomposition reduces memory access costs and improves inference speed. ⨂ represents element-wise multiplication; m and n are preset convolution kernel size parameters; Conv n×m Represents an n×m convolutional layer, Conv m×n This represents an m×n convolutional layer; Concat(·) represents the channel concatenation operation; M s Spatial attention weights; The feature map output by the CBAM module is represented as F''=F⨁M s (M c (F)⨂F).
[0013] Furthermore, in step 3, the network is trained based on the existing defect dataset, and the network weights are saved. The specific steps are as follows: The existing defect dataset is represented as ; Where, x i For an existing defective image, y i Label existing defect categories; y i ∈{1,2,...,C init}, C init N represents the initial number of defect categories. init Let i be the total number of samples in the existing defective dataset; i∈{1,2,...,N} init}; Using cross-entropy loss as the optimization objective, SuperSimpleNet, which incorporates CBAM, is trained. The cross-entropy objective function is as follows: ; Where θ is the set of all trainable parameters of the network; p(y i |x i ;θ) represents the model's application to existing defective image samples x i Existing defect category y i The predicted probability; Save the initial weights θ after training. * ={θ1 * ,θ2 * ,...,θ k *}; where θ1* ,θ2 * ,...,θ k * Let each of the parameters in set θ represent a parameter, and k be the total number of parameters; Calculate the Fisher information matrix F Quantify the importance of each weight for initial defect identification: ; Among them, F j The j-th element of the Fisher matrix corresponds to the j-th weight θ. j Importance; j∈{1,2,...,k}; In step 4, the newly added defect dataset is represented as follows: Add a new defect category label y t ∈{C init +1,...,C init +C inc}; Among them, C inc N represents the number of new defect categories. inc The total number of samples in the newly added defect dataset; t∈{1,2,...,N inc}; The expanded output dimension of the classification layer is C. init +C inc ; The weight initialization rule for the expanded classification layer is: the first C values in the new classification layer weight matrix... init The top C of the weights and bias vectors corresponding to all columns in the row init The Cth element inherits the weights and biases of the original classification layer, respectively. The Cth element in the new classification layer's weight matrix... init In the Cth column of the weight and bias vector corresponding to all columns in the last row init The elements from the first to the last element are determined using a random initialization method; Category layer expansion: Original classification layer: fc old :R D →R Cinit (R) D →R Cinit The input to the classification layer is a D-dimensional feature vector, and the output is a probability vector of the "initial number of categories" dimension. Extended classification layer: fc new :R D →R Cinit+Cinc (R) D →R Cinit+Cinc The input to the classification layer is a D-dimensional feature vector, and the output is a probability vector of dimensions "initial number of categories + new number of categories". Weight initialization rule: fc new .weight[:Cinit ,:]=fc old .weight,fc new .bias[:C init ]=fc old .bias, fc new .weight[C init :,:] = Random initialization, fc new .bias[C init :] = Random initialization; Among them, fc new .weight[:C init ,:] represents the top C values in the new classification layer weight matrix. init The weights of all row and column categories, fc old .weight represents all the weights of the old classification layer; fc new .bias[:C init ] represents the first C in the new classification layer bias vector. init elements, fc old .bias represents all bias vectors of the old classification layer; fc new .weight[C init :,:] represents C in the new classification layer weight matrix. init The weights of all column categories up to the last row, fc new .bias[C init :] represents C in the new classification layer bias vector. init The bias corresponding to the last element.
[0014] Network parameter update: The expanded network parameters are denoted as θ'={θ1',θ2',...,θ k '}, where k is the total number of parameters. The parameters of the non-classification layer inherit the initial weights θ. * The newly added part of the classification layer is randomly initialized.
[0015] The loss function for incremental training is L. inc =L ce +L ewc ; in, , representing the cross-entropy loss of the newly added defect data, x t This represents the newly added defect image, and θ' is the expanded set of network parameters; Here, λ is the EWC regularization term, θ is the EWC regularization coefficient, and θ is the EWC regularization term. j ' represents the j-th parameter of the expanded network, θ j * Let F be the weight of the j-th parameter in the initial model. jThis is the j-th element of the Fisher information matrix; The incremental training employs the Adam optimizer. During training, the newly added weights of the classification layer and the non-core weights of the CBAM module are updated, and the weights of convolutional layers with high Fisher values are fine-tuned. After training, the final model parameters θ are saved. final .
[0016] Furthermore, in step 5, the preprocessing specifically includes normalization and adjustment to a preset size; The process of obtaining the defect category and confidence level specifically includes: The features F of the image to be detected are extracted by SuperSimpleNet, which incorporates the CBAM module. feat =CBAM-SuperSimpleNet feat (x test ;θ final ); where x test The image to be detected; After processing by the incremental classification layer, the class probability p = Softmax(fc) is obtained through the Softmax activation function. new (F feat )); where fc new To expand the classification layer; Output the category y'=argmax(p) corresponding to the maximum probability and its confidence level.
[0017] Compared with existing technologies, the beneficial effects of this invention are: the channel-spatial dual attention mechanism of the CBAM module can accurately capture the features of subtle defects (such as scratches and pinholes) on the surface of industrial products, significantly improving defect recognition accuracy and reducing false negative rate compared to the original SuperSimpleNet; when adding new defect types, only incremental training is required, without the need for full network reconstruction, reducing training time and data requirements, and adapting to flexible industrial production; lightweight adaptation to industrial scenarios: the CBAM module is lightweighted to meet the requirements of real-time industrial detection; combined with the incremental training strategy of EWC regularization, existing defect features are avoided when training for new defects. Attached Figure Description
[0018] Figure 1 This is a schematic diagram of the overall network architecture of CBAM-SuperSimpleNet, an industrial defect detection method based on incremental training and CBAM enhancement according to the present invention. Figure 2 This is a schematic diagram of the CBAM module structure of an industrial defect detection method based on incremental training and CBAM enhancement according to the present invention. Figure 3This is a schematic diagram of the model training process for an industrial defect detection method based on incremental training and CBAM enhancement according to the present invention. Detailed Implementation
[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0020] Example: Figure 1-3 As shown, this invention provides a technical solution: an industrial defect detection method based on incremental training and CBAM enhancement, which includes the following steps: Step 1: Obtain surface images of industrial products containing existing defects and newly added defects. After annotation, data augmentation, and dataset partitioning, obtain existing defect datasets and newly added defect datasets. To provide high-quality, comprehensive training data for network training, ensuring that the model can learn existing defect features and adapt to newly added defect types. Industrial product surface images can be acquired through industrial cameras, CCD sensors, and other devices, covering surface images of various industrial products such as metal sheets, plastic casings, and electronic components. Labeling operations can utilize mainstream labeling tools such as LabelImg and VGGImageAnnotator, classifying and labeling according to defect types (e.g., scratches, dents, pinholes, hairline cracks, etc.). Data augmentation aims to expand the dataset size and improve the model's generalization ability. Normalization specifically maps image pixel values to the [0,1] interval, random flipping includes horizontal and vertical flipping, and random cropping extracts fixed-size sub-images (e.g., 64×64) from the original image. Dataset partitioning is based on the labeling results of defect types. For example, images and labels corresponding to long-standing defects such as scratches and dents are classified as existing defect datasets, while images and labels corresponding to newly added defects such as pinholes due to raw material changes and hairline cracks due to process adjustments are classified as new defect datasets.
[0021] Step 2: Construct an incremental SuperSimpleNet network that integrates the CBAM module and initialize the network weights; A basic network architecture combining lightweight design, feature enhancement capabilities, and incremental expansion is constructed. The SuperSimpleNet network structure consists of an input layer, M lightweight convolutional blocks, N max-pooling layers (GAP), a classification layer, and an output layer. M and N are preset numbers, preferably M=3 and N=3, which ensures sufficient feature extraction depth while maintaining the network's lightweight characteristics. The lightweight convolutional blocks employ a Conv-BN-ReLU structure, where Conv represents the convolution operation, BN (Batch Normalization) is used to accelerate network training and alleviate the gradient vanishing problem, and ReLU is the activation function used to introduce non-linear features. The CBAM module includes a channel attention submodule (CAM) and a spatial... The attention submodule (SAM) is integrated sequentially into the SuperSimpleNet network after the lightweight convolutional block and before the max pooling layer. This integration position enhances effective features after feature extraction and before feature compression, avoiding the loss of subtle defect features due to pooling operations. The channel attention submodule evaluates the importance of each channel of the feature map, enhancing key channel features (such as channels corresponding to edges and textures). The spatial attention submodule evaluates the spatial dimension importance of the feature map after channel attention enhancement, enhancing the spatial features of defective regions (such as the location and shape of defects), ultimately yielding the enhanced feature map output by the CBAM module. Step 3: Train the network based on the existing defect dataset, optimize the network weights, and save the initial model and Fisher information matrix; The network learns the features of existing defects to form a stable initial model, and records the key weight information for identifying existing defects, providing a basis for anti-forgetting constraints in subsequent incremental training. During training, gradient descent is used to minimize the cross-entropy loss function, continuously adjusting the network weights θ to make the model's predicted probability of existing defects as close as possible to the true label. The Fisher information matrix F is used to quantify the importance of each weight to the initial defect identification. For example, if a weight corresponds to the extraction of defect edge features, and it has a significant impact on the identification accuracy in the initial training, then the corresponding Fisher value F is [value missing]. j The value is relatively high, and the update range of this weight needs to be carefully constrained in subsequent incremental training to avoid forgetting the ability to extract edge features; Step 4: When adding a new defect type, expand the network classification layer dimension, load the initial model weights, and use the new defect dataset to incrementally train and update the network in conjunction with the EWC regularization term. Without reconstructing the network or forgetting existing defect knowledge, the model can quickly learn new defect features. The core of the classification layer extension is to adapt to the number of new defect categories without changing the structure of the network's feature extraction part, significantly saving training resources. The EWC regularization term constrains the update of key weights, with λ controlling the constraint strength (preferably λ within the range of 10). 2 ~10 3 This approach allows for only minor adjustments to the weights with high Fisher scores, ensuring the model can learn new defect features while avoiding a decline in performance in recognizing existing defects. Incremental training uses the Adam optimizer with a learning rate set to 10%. -4 This learning rate is lower than the initial training rate, which can further avoid forgetting caused by large fluctuations in weights.
[0022] Step 5: Preprocess the image to be detected, input it into the trained network, and output the defect category and confidence level.
[0023] This technology enables rapid and accurate detection of surface defects in industrial products, providing a basis for quality inspection decisions on the production line. Preprocessing operations must be consistent with the preprocessing of the training data to ensure a uniform distribution of input features. Confidence level indicates the model's reliability in predicting categories; for example, a confidence level of 0.95 indicates that the model is 95% certain the defect belongs to the predicted category. In practical applications, a confidence threshold (e.g., 0.8) can be set. Detection results below the threshold can trigger manual review, further improving detection reliability. Furthermore, in step 1, the surface image of the industrial product includes images corresponding to defects that have already appeared in industrial production and images corresponding to newly added defects; The annotation specifically refers to the type of defect in the image; The data augmentation specifically includes: normalizing the image, randomly flipping it, and randomly cropping it; The dataset partitioning refers to dividing the images and their annotations based on the annotation information to obtain existing defect datasets and newly added defect datasets.
[0024] Furthermore, in step 2, the SuperSimpleNet network structure consists of an input layer, M lightweight convolutional blocks, N max pooling layers (GAP), a classification layer, and an output layer. Wherein, M and N represent preset quantities; the lightweight convolutional block is a Conv-BN-ReLU structure; The CBAM module has a channel attention submodule CAM and a spatial attention submodule SAM, which are integrated into the SuperSimpleNet network after the lightweight convolutional block and before the max pooling layer using a serial embedding method. The channel attention submodule enhances key channel features by evaluating the importance of each channel in the feature map; The spatial attention submodule performs spatial dimension importance assessment on the feature map after channel attention enhancement, thereby strengthening the spatial features of the defective region. The feature map output by the CBAM module is obtained.
[0025] The channel attention submodule enhances key channel features by evaluating the importance of each channel in the feature map, specifically including: For the input feature map F∈[C,H,W], global max pooling and global average pooling are performed on the feature map respectively. Channel weights are obtained from the shared multilayer perceptron (MLP). After weighting, the features of defect-related channels are enhanced, while the background channels are suppressed: M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))); Where F is the input feature map; C is the number of channels; H is the height; W is the width; σ(·) is the Sigmoid activation function; M c represents the channel attention weights; AvgPool(·) is the average pooling operation; MaxPool(·) is the max pooling operation.
[0026] The spatial attention submodule performs a spatial dimension importance assessment on the feature map after channel attention enhancement, and the enhancement of spatial features of defective regions specifically includes: Feature map M after channel attention weighting c (F) is multiplied by the original input feature map F to obtain feature map F', F'=M c (F)⨂F, performs max pooling and average pooling on the channel dimension respectively, and after concatenation, generates spatial attention weights through m×n and n×m convolutions: M s (F')=σ(Conv n×m (Conv m×n (Concat(MaxPool(F');AvgPool(F'))))); In this design, m=1 and n=7 are preferred, meaning 1×7 and 7×1 asymmetric convolution kernels are used instead of the traditional 3×3 convolution kernel. While maintaining the receptive field of the 7×7 kernel (which is much larger than that of the 3×3 kernel), the number of parameters is only slightly increased. This allows for more accurate capture of common industrial defects such as long strips (e.g., scratches, hairline cracks) or irregular minute defects. Simultaneously, convolution decomposition reduces memory access costs and improves inference speed. ⨂ represents element-wise multiplication; m and n are preset convolution kernel size parameters; Conv n×m Represents an n×m convolutional layer, Conv m×n This represents an m×n convolutional layer; Concat(·) represents the channel concatenation operation; M s Spatial attention weights; The feature map output by the CBAM module is represented as F''=F⨁M s (M c (F)⨂F).
[0027] Furthermore, in step 3, the network is trained based on the existing defect dataset, and the network weights are saved. The specific steps are as follows: The existing defect dataset is represented as ; Where, x i For an existing defective image, y i Label existing defect categories; y i ∈{1,2,...,C init}, C init N represents the initial number of defect categories. init Let i be the total number of samples in the existing defective dataset; i∈{1,2,...,N} init}; Using cross-entropy loss as the optimization objective, SuperSimpleNet, which incorporates CBAM, is trained. The cross-entropy objective function is as follows: ; Where θ is the set of all trainable parameters of the network; p(y i |x i ;θ) represents the model's application to existing defective image samples x i Existing defect category y i The predicted probability; Save the initial weights θ after training. * ={θ1 * ,θ2 * ,...,θ k *}; where θ1 * ,θ2 * ,...,θ k * Let each of the parameters in set θ represent a parameter, and k be the total number of parameters; Calculate the Fisher information matrix F Quantify the importance of each weight for initial defect identification: ; Among them, F j The j-th element of the Fisher matrix corresponds to the j-th weight θ. j Importance; j∈{1,2,...,k}; In step 4, the newly added defect dataset is represented as follows: Add a new defect category label y t ∈{C init +1,...,C init +C inc}; Among them, C inc N represents the number of new defect categories. inc The total number of samples in the newly added defect dataset; t∈{1,2,...,N inc}; The expanded output dimension of the classification layer is C. init +C inc ; The weight initialization rule for the expanded classification layer is: the first C values in the new classification layer weight matrix... init The top C of the weights and bias vectors corresponding to all columns in the row init The Cth element inherits the weights and biases of the original classification layer, respectively. The Cth element in the new classification layer's weight matrix... init In the Cth column of the weight and bias vector corresponding to all columns in the last row init The elements from the first to the last element are determined using a random initialization method; Category layer expansion: Original classification layer: fc old :R D →R Cinit (R) D →R Cinit The input to the classification layer is a D-dimensional feature vector, and the output is a probability vector of the "initial number of categories" dimension. Extended classification layer: fc new :R D →R Cinit+Cinc (R) D →R Cinit+Cinc The input to the classification layer is a D-dimensional feature vector, and the output is a probability vector of dimensions "initial number of categories + new number of categories". Weight initialization rule: fc new .weight[:C init ,:]=fc old .weight,fc new .bias[:C init ]=fc old .bias, fc new .weight[C init :,:] = Random initialization, fc new .bias[C init :] = Random initialization; Among them, fc new .weight[:C init ,:] represents the top C values in the new classification layer weight matrix. init The weights of all row and column categories, fc old .weight represents all the weights of the old classification layer; fc new .bias[:C init] represents the first C in the new classification layer bias vector. init elements, fc old .bias represents all bias vectors of the old classification layer; fc new .weight[C init :,:] represents C in the new classification layer weight matrix. init The weights of all column categories up to the last row, fc new .bias[C init :] represents C in the new classification layer bias vector. init The bias corresponding to the last element.
[0028] Network parameter update: The expanded network parameters are denoted as θ'={θ1',θ2',...,θ k '}, where k is the total number of parameters. The parameters of the non-classification layer inherit the initial weights θ. * The newly added part of the classification layer is randomly initialized.
[0029] The loss function for incremental training is L. inc =L ce +L ewc ; in, , representing the cross-entropy loss of the newly added defect data, x t This represents the newly added defect image, and θ' is the expanded set of network parameters; Here, λ is the EWC regularization term, θ is the EWC regularization coefficient, and θ is the EWC regularization term. j ' represents the j-th parameter of the expanded network, θ j * Let F be the weight of the j-th parameter in the initial model. j This is the j-th element of the Fisher information matrix; The incremental training employs the Adam optimizer. During training, the newly added weights of the classification layer and the non-core weights of the CBAM module are updated, and the weights of convolutional layers with high Fisher values are fine-tuned. After training, the final model parameters θ are saved. final .
[0030] Furthermore, in step 5, the preprocessing specifically includes normalization and adjustment to a preset size; The process of obtaining the defect category and confidence level specifically includes: The features F of the image to be detected are extracted by SuperSimpleNet, which incorporates the CBAM module. feat =CBAM-SuperSimpleNet feat (x test ;θ final ); where x test The image to be detected; After processing by the incremental classification layer, the class probability p = Softmax(fc) is obtained through the Softmax activation function. new (F feat )); where fc new To expand the classification layer; Output the category y'=argmax(p) corresponding to the maximum probability and its confidence level.
[0031] In this embodiment, a certain smartphone metal frame production line has long faced two core problems: First, minute scratches and pinholes with a surface width of <0.1mm on the metal frame are easily missed; second, due to changes in upstream raw material suppliers (such as adjustments to the aluminum alloy ingot composition) and optimization of anodizing process parameters, when new defect types such as hairline cracks and oxide spots are added, the original detection model needs to be fully retrained, which is not only time-consuming but also prone to forgetting the ability to identify historical defects. This solution is applied to improve the above problems: The production line uses an industrial CCD camera (with an image resolution of 0.01mm) to capture images of the surface of the mobile phone's metal frame, including both existing and newly added defects. Existing defects include scratches and dents that have been present on the production line for a long time, while newly added defects include pinholes that appeared after the raw materials were changed and hairline cracks that appeared after the process was adjusted.
[0032] The LabelImg tool was used to label defect types. Data augmentation operations such as normalization, random horizontal / vertical flipping, and random cropping were performed on all images. The dataset was divided according to the defect type labeling results: the images and labels corresponding to scratches and dents were assigned to the existing defect dataset, and the images and labels corresponding to pinholes and hairline cracks were assigned to the newly added defect dataset.
[0033] An incremental SuperSimpleNet network integrating the CBAM module was constructed. The network consists of an input layer, three Conv-BN-ReLU lightweight convolutional blocks, three max pooling layers, a classification layer, and an output layer.
[0034] The CBAM module is integrated sequentially after each lightweight convolutional block and before the max pooling layer: the Channel Attention Submodule (CAM) enhances the channel features corresponding to defect edges and textures through global max pooling, global average pooling, and shared MLP, while suppressing background channels in defect-free uniform regions; the Spatial Attention Submodule (SAM) uses 1×7 and 7×1 asymmetric convolutional kernels to generate spatial attention weights through channel dimension splicing and asymmetric convolution, accurately capturing the spatial features of long scratches, hairline cracks, and tiny pinholes, while maintaining the network's lightweight characteristics.
[0035] The network was trained using an existing defect dataset (scratches and dents), with cross-entropy loss as the optimization objective. All trainable parameters of the network (including the CBAM module, convolutional layers, and initial classification layer weights) were adjusted. After training, the initial model weights and Fisher information matrix were saved. This matrix quantifies the importance of each network weight for identifying scratches and dents, providing a basis for anti-forgetting constraints in subsequent incremental training.
[0036] When two new defects, pinholes and hairline cracks, appear on the production line, the output dimension of the network classification layer is expanded (from the initial 2 categories to 4 categories). The classification layer weights are initialized according to the rule of "the weights of the old classification layer are directly inherited and the weights of the new categories are randomly initialized".
[0037] Initial model weights are loaded, and incremental training is performed using a newly added defect dataset (pinholes, hairline cracks). During training, a composite loss function of "cross-entropy loss + EWC regularization" is used. The EWC regularization term constrains the update magnitude of highly important weights in the Fisher information matrix, preventing the forgetting of scratch and dent recognition features. Training only updates the newly added weights in the classification layer and non-core weights in the CBAM module, without reconstructing the entire network, significantly shortening the training cycle.
[0038] The final trained model is deployed to the quality inspection station on the production line. After the metal frame of the mobile phone to be inspected is transported to the inspection area by a conveyor belt, an industrial CCD camera captures surface images. The images are then normalized and preprocessed by size adjustment before being input into the model.
[0039] The model extracts image features by integrating the CBAM module into SuperSimpleNet, and outputs the probability distribution of four types of defects through an incremental classification layer and a Softmax activation function. Finally, it outputs the defect category (scratches, dents, pinholes, hairline cracks) and confidence level corresponding to the maximum probability, thus achieving real-time and accurate identification of all defects.
[0040] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.
Claims
1. An industrial defect detection method based on incremental training and CBAM enhancement, characterized in that: The method includes the following steps: Step 1: Obtain surface images of industrial products containing existing defects and newly added defects. After annotation, data augmentation, and dataset partitioning, obtain existing defect datasets and newly added defect datasets. Step 2: Construct an incremental SuperSimpleNet network that integrates the CBAM module and initialize the network weights; Step 3: Train the network based on the existing defect dataset, optimize the network weights, and save the initial model and Fisher information matrix; Step 4: When adding a new defect type, expand the network classification layer dimension, load the initial model weights, and use the new defect dataset to incrementally train and update the network in conjunction with the EWC regularization term. Step 5: Preprocess the image to be detected, input it into the trained network, and output the defect category and confidence level.
2. The industrial defect detection method based on incremental training and CBAM enhancement according to claim 1, characterized in that: In step 1, the surface image of the industrial product includes images corresponding to defects that have already appeared in industrial production and images corresponding to newly added defects; The annotation specifically refers to the type of defect in the image; The data augmentation specifically includes: normalizing the image, randomly flipping it, and randomly cropping it; The dataset partitioning refers to dividing the images and their annotations based on the annotation information to obtain existing defect datasets and newly added defect datasets.
3. The industrial defect detection method based on incremental training and CBAM enhancement according to claim 2, characterized in that: In step 2, the SuperSimpleNet network structure consists of an input layer, M lightweight convolutional blocks, N max pooling layers (GAP), a classification layer, and an output layer. Wherein, M and N represent preset quantities; the lightweight convolutional block is a Conv-BN-ReLU structure; The CBAM module has a channel attention submodule CAM and a spatial attention submodule SAM, which are integrated into the SuperSimpleNet network after the lightweight convolutional block and before the max pooling layer using a serial embedding method. The channel attention submodule enhances key channel features by evaluating the importance of each channel in the feature map; The spatial attention submodule performs spatial dimension importance assessment on the feature map after channel attention enhancement, thereby strengthening the spatial features of the defective region. The feature map output by the CBAM module is obtained.
4. The industrial defect detection method based on incremental training and CBAM enhancement according to claim 3, characterized in that: In step 2, the channel attention submodule enhances key channel features by evaluating the importance of each channel in the feature map, specifically including: For the input feature map F∈[C,H,W], global max pooling and global average pooling are performed on the feature map respectively. Channel weights are obtained from the shared multilayer perceptron (MLP). After weighting, the features of defect-related channels are enhanced, while the background channels are suppressed: M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))); Where F is the input feature map; C is the number of channels; H is the height; W is the width; σ(·) is the Sigmoid activation function; M c represents the channel attention weights; AvgPool(·) is the average pooling operation; MaxPool(·) is the max pooling operation.
5. The industrial defect detection method based on incremental training and CBAM enhancement according to claim 4, characterized in that: In step 2, the spatial attention submodule performs a spatial dimension importance assessment on the feature map after channel attention enhancement. Specifically, enhancing the spatial features of the defect region includes: Feature map M after channel attention weighting c (F) is multiplied by the original input feature map F to obtain feature map F', F'=M c (F)⨂F, performs max pooling and average pooling on the channel dimension respectively, and after concatenation, generates spatial attention weights through m×n and n×m convolutions: M s (F')=σ(Conv n×m (Conv m×n (Concat(MaxPool(F');AvgPool(F'))))); Where ⨂ represents element-wise multiplication; m and n are preset convolution kernel size parameters; Conv n×m Represents an n×m convolutional layer, Conv m×n This represents an m×n convolutional layer; Concat(·) represents the channel concatenation operation; M s Spatial attention weights; The feature map output by the CBAM module is represented as F''=F⨁M s (M c (F)⨂F).
6. The industrial defect detection method based on incremental training and CBAM enhancement according to claim 5, characterized in that: In step 3, the network is trained based on the existing defect dataset, and the network weights are saved. The specific steps are as follows: The existing defect dataset is represented as ; Where, x i For an existing defective image, y i Label existing defect categories; y i ∈{1,2,...,C init }, C init N represents the initial number of defect categories. init Let i be the total number of samples in the existing defective dataset; i∈{1,2,...,N} init }; Using cross-entropy loss as the optimization objective, SuperSimpleNet, which incorporates CBAM, is trained. The cross-entropy objective function is as follows: ; Where θ is the set of all trainable parameters of the network; p(y i |x i ;θ) represents the model's application to existing defective image samples x i Existing defect category y i The predicted probability; Save the initial weights θ after training. * ={θ1 * ,θ2 * ,...,θ k * }; where θ1 * ,θ2 * ,...,θ k * Let each of the parameters in set θ represent a parameter, and k be the total number of parameters; Calculate the Fisher information matrix F Quantify the importance of each weight for initial defect identification: ; Among them, F j The j-th element of the Fisher matrix corresponds to the j-th weight θ. j The importance of j∈{1,2,...,k}.
7. The industrial defect detection method based on incremental training and CBAM enhancement according to claim 6, characterized in that: In step 4, the newly added defect dataset is represented as follows: Add a new defect category label y t ∈{C init +1,...,C init +C inc }; Among them, C inc N represents the number of new defect categories. inc The total number of samples in the newly added defect dataset; t∈{1,2,...,N inc }; The expanded output dimension of the classification layer is C. init +C inc ; The weight initialization rule for the expanded classification layer is: the first C values in the new classification layer weight matrix... init The top C of the weights and bias vectors corresponding to all columns in the row init The Cth element inherits the weights and biases of the original classification layer, respectively. The Cth element in the new classification layer's weight matrix... init In the Cth column of the weights and bias vectors corresponding to all columns in the last row... init The elements from the first to the last are determined using a random initialization method.
8. The industrial defect detection method based on incremental training and CBAM enhancement according to claim 7, characterized in that: In step 4, the loss function for incremental training is L. inc =L ce +L ewc ; in, , representing the cross-entropy loss of the newly added defect data, x t This represents the newly added defect image, and θ' is the expanded set of network parameters; Here, λ is the EWC regularization term, θ is the EWC regularization coefficient, and θ is the EWC regularization term. j ' represents the j-th parameter of the expanded network, θ j * Let F be the weight of the j-th parameter in the initial model. j This is the j-th element of the Fisher information matrix; The incremental training employs the Adam optimizer. During training, the newly added weights of the classification layer and the non-core weights of the CBAM module are updated, and the weights of convolutional layers with high Fisher values are fine-tuned. After training, the final model parameters θ are saved. final .
9. The industrial defect detection method based on incremental training and CBAM enhancement according to claim 8, characterized in that: In step 5, the preprocessing specifically includes normalization and adjustment to a preset size; The process of obtaining the defect category and confidence level specifically includes: The features F of the image to be detected are extracted by SuperSimpleNet, which incorporates the CBAM module. feat =CBAM-SuperSimpleNet feat (x test ;θ final ); where x test The image to be detected; After processing by the incremental classification layer, the class probability p = Softmax(fc) is obtained through the Softmax activation function. new (F feat )); where fc new To expand the classification layer; Output the category y'=argmax(p) corresponding to the maximum probability and its confidence level.