Industrial defect detection method based on cross-layer cooperative multi-level attention and progressive feature enhancement

Abstract

The application discloses an industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement, and belongs to the field of computer vision. The method first constructs a cross-layer collaborative multi-level attention feature extraction network, weights different level feature maps through slight, medium and severe three-level defect degree attention modules, and divides defect samples into three levels of simple, medium and difficult based on training loss value and confidence. Secondly, a conditional generative adversarial network is constructed, defect feature vectors, defect category labels, difficulty levels and random noise are taken as generator inputs, an enhanced defect feature vector of a specified category and difficulty is generated in a feature space, and an adversary is trained through a discriminator to optimize the generation quality. Finally, a progressive weighted training strategy is adopted to retrain the defect classifier, training data is introduced in stages according to the sample difficulty level, the loss weights of various samples are dynamically adjusted, and progressive model optimization from easy to difficult is realized.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and industrial automation quality inspection technology, and relates to an intelligent detection method for surface defects of industrial products based on deep learning, specifically an industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement. Background Technology

[0002] Surface defect detection in industrial products is a crucial step in ensuring product quality and improving production efficiency. In modern industrial production, surface defects such as scratches, stains, dents, and cracks not only affect the appearance of products but can also lead to functional failures, causing serious economic losses and safety hazards. Traditional manual visual inspection methods suffer from low efficiency, high subjectivity, and poor consistency, making them unsuitable for the needs of large-scale automated production. Therefore, developing efficient and accurate automated defect detection systems has significant engineering application value.

[0003] In recent years, with the rapid development of deep learning technology, defect detection methods based on convolutional neural networks (CNNs) have gradually become a research hotspot. However, existing technologies still face many challenges in industrial defect detection tasks. First, the yield rate on industrial production lines is usually high, resulting in a small number of defect samples that are difficult to collect. Deep learning methods typically require a large amount of labeled data for training, and insufficient samples can lead to model overfitting and poor generalization ability. Second, industrial defects are characterized by diverse shapes, sizes, and low contrast. Defects of the same type may present different visual appearances, and different types of defects may have visual similarities. Traditional convolutional neural networks are limited by local receptive fields and lack long-range dependency modeling capabilities, making it difficult to adapt to the defect detection needs of different scales and shapes simultaneously. During layer-by-layer downsampling and convolution stacking, edge and texture information of small defects is easily lost, leading to incomplete detection results. In addition, although traditional data augmentation methods (such as rotation, flipping, scaling, brightness adjustment, etc.) can expand the training data to a certain extent, the generated samples differ from the real defect distribution and cannot generate new types of defect patterns, thus having limited improvement on model performance. Especially in industrial settings, the shape and distribution of defects follow specific patterns, making it difficult for simple geometric transformations to simulate real-world defect variations.

[0004] In summary, existing technologies still face challenges such as sample scarcity and poor multi-scale adaptability in industrial defect detection tasks, necessitating a new technical solution to address these challenges. Summary of the Invention

[0005] To address the aforementioned problems in existing technologies, this invention proposes an industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement. This method constructs a detection framework that integrates cross-layer multi-level attention mechanisms, conditional feature adversarial enhancement, and progressive weighted training. By constructing a cross-layer collaborative multi-level attention feature extraction network, it fully captures multi-scale defect features. It utilizes a conditional generative adversarial network to synthesize high-quality defect samples to expand the training data, and employs a progressive weighted training strategy to optimize the model from easy to difficult, thereby significantly improving the accuracy, robustness, and practicality of industrial defect detection.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] The first aspect of this invention provides an industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement, comprising the following steps:

[0008] S1. Collect surface images of qualified industrial products and those with different types of defects, and label the surface images of industrial products with defect category labels to construct an industrial defect dataset; the defect categories include scratches, cracks, stains, dents and holes, etc.

[0009] S2. Construct a cross-layer collaborative multi-level attention feature extraction network, which includes a feature pyramid backbone network, a multi-level defect severity attention module, and a cross-layer feature fusion module. The feature pyramid backbone network is used to extract multi-level feature maps of industrial product surface images. The multi-level defect severity attention module is used to perform attention weighting processing on the multi-level feature maps to obtain slight defect attention features, moderate defect attention features, and severe defect attention features. The cross-layer feature fusion module is used to perform weighted fusion of the slight defect attention features, moderate defect attention features, and severe defect attention features to obtain a fused feature map.

[0010] S3. The cross-layer collaborative multi-level attention feature extraction network and the defect classifier are initially trained using the industrial defect dataset constructed in S1 to obtain the basic feature extractor and the basic defect classifier. During the initial training process, the difficulty is evaluated based on the training loss value and confidence score of each defect sample in the industrial defect dataset. The defect samples are divided into three levels: simple samples, medium difficulty samples and hard samples, to obtain a defect sample set with difficulty labels and defect category labels.

[0011] S4. Use the basic feature extractor to extract multi-level features of each defect sample in the defect sample set and fuse the features of different levels to obtain a real defect feature map of the same dimension, thus forming a defect feature dataset with difficulty label and defect category label.

[0012] S5. Construct a conditional generative adversarial network, which includes a generator and a discriminator. The generator takes a random noise vector and a specified defect category and difficulty level as input to generate an enhanced defect feature map with the specified defect category and difficulty. The discriminator performs adversarial training on the enhanced defect feature map and the real defect feature map to optimize the generator parameters.

[0013] S6. Adopt a progressive weighted training strategy and retrain the basic defect classifier using real defect feature maps and enhanced defect feature maps: In the early stage of training, simple samples are used first. As the training rounds increase, medium and difficult samples are gradually introduced, and the loss weights of various samples are dynamically adjusted to achieve progressive model optimization from easy to difficult, thus obtaining an enhanced defect classifier.

[0014] S7. Input the surface image of the industrial product to be detected into the basic feature extractor and the enhanced defect classifier, and output the defect detection result.

[0015] Furthermore, in S2, the specific structure of the cross-layer collaborative multi-level attention feature extraction network is as follows:

[0016] The feature pyramid backbone network adopts an improved ResNet framework, and the processing flow is as follows: The input industrial product surface image is first subjected to large-scale convolution and nonlinear activation to obtain an initial feature map; then the initial feature map is downsampled to obtain a pooled feature map; the pooled feature map is then passed through four residual module blocks in sequence to output the first to fourth level feature maps respectively.

[0017] The multi-level defect severity attention module includes cascaded minor defect attention modules, moderate defect attention modules, and severe defect attention modules. The processing flow is as follows: First, the pooled feature map is input into the minor defect attention module. A minor defect attention map is generated using the first-level feature map and weighted to obtain the minor defect attention feature. Then, the minor defect attention feature is input into the moderate defect attention module. A moderate defect attention map is generated using the second-level and third-level feature maps and weighted to obtain the moderate defect attention feature. Finally, the moderate defect attention feature is input into the severe defect attention module. A severe defect attention map is generated using the fourth-level feature map and weighted to obtain the severe defect attention feature.

[0018] The cross-layer feature fusion module performs weighted fusion of the minor flaw attention features, the moderate flaw attention features, and the severe flaw attention features to obtain a fused feature map.

[0019] Furthermore, in the multi-level defect severity attention module, the processing flow of the minor defect attention module is expressed as follows:

[0020]

[0021]

[0022] in, This indicates a minor flaw attention map. express Activation function express Convolution operation, This represents the first-level feature map. This represents element-wise multiplication; Indicates minor flaws and attention characteristics. This represents the feature map after pooling;

[0023] The processing flow of the medium defect attention module is represented as follows:

[0024]

[0025]

[0026] in, This represents a medium-level defect attention map. and These represent the second-level feature map and the third-level feature map, respectively. This indicates a moderate level of attentional impairment.

[0027] The processing flow of the severe defect attention module is represented as follows:

[0028]

[0029]

[0030] in, Attention graph indicating serious defects This represents the fourth-level feature map; This indicates a serious flaw in attention characteristics.

[0031] Furthermore, in S3, the initial training process of the cross-layer collaborative multi-level attention feature extraction network and the defect classifier includes:

[0032] The industrial defect dataset is input into a cross-layer collaborative multi-level attention feature extraction network and a defect classifier for forward propagation, using the classification cross-entropy loss function. End-to-end training is performed, where The number of samples in the industrial defect dataset. This represents the total number of defect categories. For sample index, For defect category index, For the sample In the category of defects The real labels on For the sample In the category of defects The predicted probability;

[0033] The gradient is calculated and the network parameters are updated using the backpropagation algorithm; the training is repeated iteratively until the model converges, resulting in a basic feature extractor and a basic defect classifier.

[0034] Furthermore, in S3, the difficulty of defective samples is assessed based on the following method:

[0035] At the end of the initial training Each round, statistical analysis of each defect sample. Average loss value and average confidence score :

[0036]

[0037]

[0038] in, For training round index, For the first Flawed samples during training rounds The loss value, For the first Flawed samples during training rounds The predicted probability;

[0039] Calculate defect samples Overall difficulty rating ,in and These are the weighting coefficients;

[0040] Calculate the mean of the difficulty scores for all defect samples. and standard deviation ,in This represents the total number of defect samples in the industrial defect dataset.

[0041] Based on the overall difficulty score, the defective samples are divided into three levels:

[0042] like Then the defective sample Difficulty level , which are simple samples;

[0043] like Then the defective sample Difficulty level This is a sample of medium difficulty.

[0044] like Then the defective sample Difficulty level This is a difficult sample.

[0045] Furthermore, in step S4, the process of constructing the defect feature dataset includes:

[0046] Use the trained basic feature extractor to extract each flawed sample in the flawed sample set. Feature extraction and fusion are performed to obtain a true defect feature map. Construct a defect feature dataset with difficulty labels and defect category labels. ,in , Defective samples Defect category labels and difficulty levels.

[0047] Furthermore, in S5, the specific structure of the conditional generative adversarial network includes:

[0048] The generator adopts an encoder-decoder structure, and the processing flow is as follows: A random noise vector is input into the encoder. Specified defect category labels and difficulty level ,in, and First, it is expanded into a random noise vector through an embedding mapping. A vector with consistent dimensions, then compared with a random noise vector. Concatenate to obtain the concatenated vector ; Concatenate vectors The latent variables are obtained after being processed through two fully connected layers and a batch normalization layer. ; hidden variables The input to the decoder is processed through two fully connected layers and a transposed convolutional layer to obtain the enhanced defect feature map. ;

[0049] The discriminator processing flow is as follows: The enhanced defect feature map generated by the generator is processed... Defect Category Label and difficulty level Or a real image showing the characteristics of the defects. and its defect category labels and difficulty level As input, the defect category label and difficulty level are converted into vector representations through an embedding mapping layer, and then the vector representations are compared with the input defect feature map. or The joint features are obtained by splicing them together. Joint features First, feature extraction is performed through the first convolutional layer. Subsequently passed The activation function is used to perform a nonlinear mapping to obtain Then, a downsampling operation is performed through a pooling layer to obtain... Then, features are further extracted through a second convolutional layer. Finally, the features are processed through a fully connected layer. Mapping to the discriminant space yields the discriminant result. ,in Enhanced defect feature map representing the input Or a real image showing the characteristics of the defects. The probability of being a real sample.

[0050] Furthermore, in S5, the training process of the conditional generative adversarial network includes:

[0051] The loss function of the generator is: ,in, This represents the probability that the feature map is true under given conditions. This represents the enhanced defect feature map generated by the generator. This represents the log-expected probability that the generated enhanced defect feature map is judged as real;

[0052] The loss function of the discriminator is: , It represents the logarithmic expectation of the probability that a true defect feature map is judged as true;

[0053] The discriminator and generator are trained alternately, and training is stopped when the training loss no longer decreases after 10 consecutive rounds.

[0054] Furthermore, in step S6, a progressive weighted training strategy is used to retrain the basic defect classifier, including:

[0055] After the conditional generative adversarial network is trained, the generator is fixed. The parameters are used to generate enhanced defect feature maps with specified defect categories and difficulties using a generator, thus constructing an enhanced defect feature dataset. ; Utilizing defect feature datasets and Enhanced Flaw Feature Dataset Retrain the base defect classifier, with the retraining batch including real defect samples (defect feature dataset). (sample data) and enhanced defect samples (enhanced defect feature dataset) Two types of data (sample data) are used for training weights. Divided into real defect sample weights and enhance the weight of defective samples The gradient is calculated and the network parameters are updated using the backpropagation algorithm; the training is repeated iteratively until the model converges, resulting in an enhanced defect classifier.

[0056] The retraining loss function of the basic defect classifier is: ;in, Indicates the first The weighted cross-entropy loss value of the training rounds, This indicates the total number of samples in the current batch. Indicates the first Training samples The loss weighting coefficient, Indicates the first During round training, the model tests the samples It belongs to the category of defects The predicted probability, For the sample In the category of defects The real labels on it;

[0057] The weights of the actual defect samples are calculated as follows:

[0058] The total number of retraining rounds It is divided into three stages: the first stage ( Using only simple samples, weights Phase Two ( Introduce samples of medium difficulty, and weight simple samples. Medium difficulty sample weights The third stage ( Introducing difficult samples and weighting simple samples Medium difficulty sample weights Difficult sample weights ,in, This indicates the total number of retraining rounds. Indicates the index of the current training round;

[0059] The method for calculating the weight of the enhanced defective samples is as follows: , This indicates that the enhanced defect features are in the first... Wheel weights; proportional coefficients The range of values ​​is .

[0060] Furthermore, the enhanced defect feature dataset The construction process is as follows:

[0061] For each defect category and each difficulty level Sampling from the standard normal distribution random noise vectors ,in Indicates the category to be generated Difficulty Level The number of enhanced defect samples can be set according to the defect categories and difficulty distribution of real defect samples to balance the dataset;

[0062] random noise vector Defect Category Label and difficulty level Input the trained generator Generate enhanced defect feature maps ;

[0063] Construct an enhanced defect feature dataset ,in To increase the total number of samples in the defect feature dataset, and Enhanced defect samples Defect category labels and difficulty levels.

[0064] A second aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor; when the processor executes the computer program, the electronic device performs the industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement.

[0065] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon; when the computer program is executed by a processor, it implements the industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement.

[0066] The beneficial effects of this invention are as follows: The industrial defect detection method proposed in this invention achieves high-precision detection and grade assessment of industrial defects through the synergistic effect of a cross-layer collaborative multi-level attention feature extraction network, a conditional generative adversarial network, and a progressively weighted training strategy. First, the cross-layer collaborative multi-level attention feature extraction network, through attention modules for minor, moderate, and severe defect levels, can simultaneously capture shallow detail information and deep semantic information, enabling adaptive detection of defects of different sizes and shapes, significantly improving sensitivity to minute defects. Second, the conditional generative adversarial network can generate enhanced defect feature vectors of specified categories and difficulties in the feature space, significantly expanding the training dataset without increasing actual data acquisition costs, effectively alleviating the model overfitting problem caused by insufficient samples. Third, the progressively weighted training strategy simulates the human cognitive process of learning from easy to difficult, allowing the model to first master basic concepts before tackling complex problems. By dynamically adjusting the loss weights of various samples, it avoids simple samples dominating gradient updates, ensuring that the model fully learns the feature representations of difficult samples, ultimately establishing a more robust decision boundary. This invention effectively alleviates the problems of data scarcity and category imbalance in industrial quality inspection scenarios, significantly improves the accuracy and robustness of defect detection, and can be widely applied to automated quality inspection of industrial products such as metal surfaces, textiles, and electronic components. Attached Figure Description

[0067] Figure 1 This is an overall framework diagram of the industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement in an embodiment of the present invention.

[0068] Figure 2 This is a schematic diagram of the cross-layer collaborative multi-level attention feature extraction network in an embodiment of the present invention, illustrating the cascading relationship and feature fusion process of the multi-level defect level attention modules.

[0069] Figure 3 This is a schematic diagram of the structure of a conditional generative adversarial network in an embodiment of the present invention, including the encoder-decoder structure of the generator and the conditional discrimination mechanism of the discriminator.

[0070] Figure 4 The flowchart of the progressive weighted training strategy in this embodiment of the invention shows the order of introduction of various samples and the dynamic changes of their weights during the three-stage training process. Detailed Implementation

[0071] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings.

[0072] Industrial defect detection is a core task in the fields of intelligent manufacturing and quality control. Its goal is to automatically and accurately detect and classify defective areas in surface images of industrial products using computer vision, thereby providing crucial information for quality assessment and production decisions in industrial products such as metal surfaces, textiles, and electronic components. Addressing the inherent limitations of traditional convolutional networks, such as scarce defect samples, poor multi-scale adaptability, difficulty in grade evaluation, and limited training strategies, this invention provides an industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement. It employs a novel detection framework that integrates a cross-layer collaborative multi-level attention feature extraction network, a conditional generative adversarial network, and a progressive weighted training strategy. The framework consists of three main stages: feature extraction, feature enhancement, and classifier optimization. The feature extraction stage first extracts multi-level primary features using a ResNet backbone network pre-trained on ImageNet. Then, it constructs a multi-level defect severity attention module, using a three-level attention mechanism (mild, moderate, and severe) to weight feature maps at different levels, thereby enhancing sensitivity to multi-scale defect features and simultaneously suppressing irrelevant background interference. The feature enhancement stage utilizes a conditional generative adversarial network to generate enhanced defect feature vectors of specified categories and difficulties in the feature space, effectively expanding the training data. The classifier optimization stage employs a progressively weighted training strategy, introducing training data in stages according to sample difficulty levels and dynamically adjusting the loss weights for each type of sample to achieve model optimization from easy to difficult. Figure 1 As shown, the method specifically includes the following steps:

[0073] S1. Construct an industrial defect dataset, as detailed below:

[0074] This embodiment uses the MVTec AD (MVTec Anomaly Detection) dataset for model training and testing. The MVTec AD dataset is an authoritative benchmark dataset in the field of industrial surface defect detection, released in 2019 by MVTec Software GmbH in Germany, and is widely used for evaluating and comparing industrial quality inspection algorithms. This dataset contains 15 categories, including 5 texture types (carpet, mesh, leather, tile, wood grain) and 10 object types (bottle, cable, capsule, nut, metal screw, metal sheet, transistor, zipper, pill, toothbrush), totaling 5354 high-resolution RGB images. The dataset covers more than 70 different defect types, including scratches, cracks, stains, deformation, color anomalies, etc., with defect morphologies ranging from minor surface scratches to obvious structural defects. Following the official classification standards, the training set contains normal samples and a small number of defect samples, while the test set contains normal samples and various defect samples, simulating a real industrial scenario where normal samples are plentiful and defect samples are scarce.

[0075] The experiment was conducted on a workstation equipped with an Intel® Xeon® Gold 6248R CPU (20 cores 2.50 GHz), 64 GB DDR4 memory, and an NVIDIA RTX 4090 24 GB GPU; the operating system was Ubuntu 22.04 LTS, CUDA version 11.8, cuDNN 8.7, and PyTorch 2.0 framework.

[0076] S2. Construct a cross-layer collaborative multi-level attention feature extraction network, which includes a feature pyramid backbone network, a multi-level defect level attention module, and a cross-layer feature fusion module.

[0077] The feature pyramid backbone network adopts an improved ResNet framework, and the processing flow is as follows:

[0078] Input industrial product surface image First, large-scale convolution and non-linear activation are applied to obtain the initial feature map. :

[0079]

[0080] in, Indicates the use of 64 Convolution kernels for industrial product surface images Perform convolution operations. This indicates a batch normalization operation. Represents the linear rectification activation function;

[0081] Then the initial feature map Perform downsampling to obtain the pooled feature map. :

[0082]

[0083] in, express Max pooling operation;

[0084] pooled feature maps The data passes through four residual module blocks in sequence, outputting four levels of feature maps respectively. Each residual module is composed of multiple stacked residual units, and the calculation formula for the residual unit is as follows: ,in and These represent the input and output of the residual unit, respectively. Represents residual mapping, These are the weight parameters for the convolutional layer.

[0085] The multi-level defect severity attention module includes cascaded minor defect attention modules, medium defect attention modules, and severe defect attention modules; the processing flow is as follows:

[0086] First, the pooled feature map Input a minor defect attention module and utilize the first-level feature map. Generate minor defect attention map pairs Weighting is performed to obtain the attention features for minor defects. The processing flow of the minor defect attention module is as follows:

[0087]

[0088]

[0089] in, This indicates a minor flaw attention map. express Activation function express Convolution operation, This represents the first-level feature map. This represents element-wise multiplication;

[0090] Then, focus on minor flaws. Input medium-impact attention module, utilizing Generate medium-defect attention map pairs Weighting is performed to obtain the attention features with moderate defects. The processing flow of the medium defect attention module is expressed as follows:

[0091]

[0092]

[0093] in, This represents a medium-level defect attention map. and These represent the second-level feature map and the third-level feature map, respectively.

[0094] Finally, the attention features of moderate defects Input a severely flawed attention module and utilize the fourth-level feature map. Generate severe defect attention map pairs Weighting is performed to obtain severely flawed attention features. The processing flow of the severe defect attention module is expressed as follows:

[0095]

[0096]

[0097] in, Attention graph indicating serious defects This represents the fourth-level feature map.

[0098] The cross-layer feature fusion module fuses the minor, moderate, and severe defect attention features output by the multi-level defect severity attention module to obtain a fused feature map. :

[0099]

[0100] in, These are learnable weights.

[0101] S3. Using the industrial defect dataset constructed in S1, the cross-layer collaborative multi-level attention feature extraction network and the defect classifier are initially trained to obtain the basic feature extractor and the basic defect classifier. The processing flow of the defect classifier includes:

[0102] First, convolution is used to compress the channels and extract discriminative features from the fused feature map. ;

[0103] Then, discriminative features Perform global average pooling to obtain a vector. ;

[0104] Next, two fully connected layers are used to further extract higher-order features and enhance nonlinear expressive power. The calculation process is represented as follows:

[0105]

[0106]

[0107] in, and These are the hidden layer outputs of the first and second fully connected layers, respectively. , These are the weight matrix and bias vector of the first fully connected layer, respectively. , These are the weight matrix and bias vector of the second fully connected layer, respectively.

[0108] Finally, the output layer passes through Obtain the predicted probability distribution ,in , These are the weight matrix and bias vector of the output layer, respectively.

[0109] The network model was initially trained using the SGD optimizer with a momentum of 0.9 and a weight decay of 0.0001. The initial learning rate was set to 0.01 and smoothly decayed to 0.00001 over 100 epochs using cosine annealing. The batch size was set to 32. The loss function used was the classification cross-entropy loss function. End-to-end training is performed, where The number of samples in the industrial defect dataset. This represents the total number of defect categories. For sample index, For defect category index, For the sample In the category of defects The real labels on it For the sample In the category of defects The predicted probability.

[0110] The gradient is calculated and the network parameters are updated using the backpropagation algorithm; training is repeated iteratively until the model converges, resulting in a basic feature extractor and a basic defect classifier; the parameter update formulas for the backpropagation of the cross-layer collaborative multi-level attention feature extraction network and the defect classifier are as follows: ,in, Indicates the first During round training, the entire set of learnable parameters for the cross-layer collaborative multi-level attention feature extraction network and the defect classifier is used. This represents the updated parameters of the cross-layer collaborative multi-level attention feature extraction network and the defect classifier. Represents the loss function For parameters gradient, This represents the learning rate.

[0111] At the end of the initial training Each round, statistical analysis of each defect sample. Average loss value and average confidence score :

[0112]

[0113]

[0114] in, For training round index, For the first Flawed samples during training rounds The loss value, For the first Flawed samples during training rounds The predicted probability;

[0115] Calculate defect samples Overall difficulty rating ,in and These are the weighting coefficients;

[0116] Calculate the mean of the difficulty scores for all defect samples. and standard deviation ,in This represents the total number of defect samples in the industrial defect dataset.

[0117] Based on the overall difficulty score, the defective samples are divided into three levels:

[0118] like Then the defective sample Difficulty level , which are simple samples;

[0119] like Then the defective sample Difficulty level This is a sample of medium difficulty.

[0120] like Then the defective sample Difficulty level This is a difficult sample.

[0121] Finally, the defective samples are compiled into a defective sample set with difficulty labels and defect category labels.

[0122] S4. Use the trained basic feature extractor to extract each defective sample in the defective sample set. Feature extraction and fusion are performed to obtain a true defect feature map. Construct a defect feature dataset with difficulty labels and defect category labels. ,in , Defective samples Defect category labels and difficulty levels.

[0123] S5. Construct a conditional generative adversarial network for feature enhancement; details are as follows:

[0124] The conditional generative adversarial network includes a generator and a discriminator; wherein, the generator adopts an encoder-decoder structure, which includes two parts: an encoder and a decoder;

[0125] The encoder processing flow is as follows: Input random noise vector Specified defect category labels and difficulty level ,in, and First, it is expanded into a random noise vector through an embedding mapping. A vector with consistent dimensions, then compared with a random noise vector. Concatenate to obtain the concatenated vector ; Concatenate vectors First, a linear transformation is performed through the first fully connected layer to obtain... Subsequently passed The activation function is used to perform a nonlinear mapping to obtain Then, it is further transformed through a second fully connected layer to obtain... Finally, the output is standardized using a batch normalization layer to obtain the latent variables. .

[0126] The decoder processing flow is as follows: implicit variables... As the input to the decoder, it first undergoes a linear transformation through the third fully connected layer to obtain... Subsequently passed Activation function for nonlinear mapping Then, after further transformation through the fourth fully connected layer, it becomes... Finally, an upsampling operation is performed through a transposed convolutional layer to obtain the enhanced defect feature map. .

[0127] The discriminator processing flow is as follows: The enhanced defect feature map generated by the generator is processed... Defect Category Label and difficulty level Or a real image showing the characteristics of the defects. and its defect category labels and difficulty level As input, the defect category label and difficulty level are converted into vector representations through an embedding mapping layer, and then the vector representations are compared with the input defect feature map. or The joint features are obtained by splicing them together. Joint features First, feature extraction is performed through the first convolutional layer. Subsequently passed The activation function is used to perform a nonlinear mapping to obtain Then, a downsampling operation is performed through a pooling layer to obtain... Then, features are further extracted through a second convolutional layer. Finally, the features are processed through a fully connected layer. Mapping to the discriminant space yields the discriminant result. ,in Enhanced defect feature map representing the input Or a real image showing the characteristics of the defects. The probability of being a real sample.

[0128] The loss function of the generator is: ,in, This represents the probability that the feature map is true under given conditions. This represents the enhanced defect feature map generated by the generator; the loss function of the discriminator is... The discriminator and generator are trained alternately; for example, the discriminator is trained 5 times, and the generator is trained once. Training stops when the training loss no longer decreases after 10 consecutive epochs. The Adam optimizer is used to train the network model with a learning rate of 2e-4. , The training rounds are 200.

[0129] S6. Retrain the basic defect classifier using a progressive weighted training strategy; the specific process is as follows:

[0130] After the conditional generative adversarial network is trained, the generator is fixed. The parameters are used to generate enhanced defect feature maps with specified defect categories and difficulties using a generator, thus constructing an enhanced defect feature dataset. ; Utilizing defect feature datasets and Enhanced Flaw Feature Dataset Retrain the base defect classifier, with the retraining batch including real defect samples (defect feature dataset). (sample data) and enhanced defect samples (enhanced defect feature dataset) Two types of data (sample data) are used for training weights. Divided into real defect sample weights and enhance the weight of defective samples The gradient is calculated and the network parameters are updated using the backpropagation algorithm; the training is repeated iteratively until the model converges, resulting in an enhanced defect classifier.

[0131] The process of constructing the enhanced defect feature dataset is as follows:

[0132] For each defect category and each difficulty level Sampling from the standard normal distribution random noise vectors ,in Indicates the category to be generated Difficulty Level The number of enhanced defect samples can be set according to the defect categories and difficulty distribution of real defect samples to balance the dataset;

[0133] random noise vector Defect Category Label and difficulty level Input the trained generator Generate enhanced defect feature maps ;

[0134] Construct an enhanced defect feature dataset ,in To increase the total number of samples in the defect feature dataset, and Enhanced defect samples Defect category labels and difficulty levels.

[0135] The retraining loss function of the basic defect classifier is: ;in, Indicates the first The weighted cross-entropy loss value of the training rounds, This indicates the total number of samples in the current batch. Indicates the first Training samples The loss weighting coefficient, Indicates the first During round training, the model tests the samples It belongs to the category of defects The predicted probability, For the sample In the category of defects The real label on it.

[0136] The weights of the actual defect samples are calculated as follows:

[0137] The total number of retraining rounds (In this embodiment, the value is set to 120) It is divided into three stages: the first stage ( Using only simple samples, weights Phase Two ( Introduce samples of medium difficulty, and weight simple samples. Medium difficulty sample weights The third stage ( Introducing difficult samples and weighting simple samples Medium difficulty sample weights Difficult sample weights ,in, This indicates the total number of retraining rounds. Indicates the index of the current training round;

[0138] The weighting of the enhanced defective samples is calculated as follows: , This indicates that the enhanced defect features are in the first... Wheel weights; proportional coefficients The range of values ​​is , The scaling factor is dynamically increased with each training round. It is used to control the contribution of augmented features to the loss function. In the early stages of training, the weight of augmented features is low to avoid interfering with the learning of basic features. As training progresses, the weight of augmented features is gradually increased to make full use of their diversity.

[0139] S7. Input the surface image of the industrial product to be detected into the trained cross-layer collaborative multi-level attention feature extraction network and defect classifier, and output the defect detection results. Determine the existence and type of defects based on the probability distribution output by the classifier; locate the defect location through the activation region of the feature map; evaluate the severity level of the defect based on the confidence score and feature response intensity. The inference time for a single image is approximately 25ms (RTX 4090), which meets the requirements for real-time detection (>30 FPS).

[0140] To comprehensively and objectively verify the superiority of the proposed industrial defect detection method, a systematic comparative experiment was conducted on the MVTec AD public benchmark against the most representative industrial defect detection algorithms. The reference models included classic architectures (ResNet-50), as well as the latest industrial defect detection methods proposed between 2021 and 2024 (PADIM, PatchCore, CFA, SimpleNet, DevNet, DGAD, RD4AD, UniAD), and Transformer-based methods (ViT-AD, AnoViT), ensuring broad representativeness of the comparison results in terms of time period, structure, and methodology. All experiments were reproduced under the same hardware environment and unified hyperparameter configuration to eliminate interference from implementation differences.

[0141] Table 1. Performance comparison of each method on the MVTec AD dataset

[0142]

[0143] As shown in the quantitative results in Table 1, the method of this invention achieves the best performance in all four core metrics: accuracy, precision, recall, and F1 score, reaching 96.80%, 95.30%, 94.78%, and 95.05%, respectively. Compared with the second-best UniAD method (F1 score 93.43%), the F1 score is improved by 1.62 percentage points; compared with AnoViT based on the Transformer architecture (F1 score 92.13%), the improvement is 2.92 percentage points; compared with the classic ResNet-50 (F1 score 86.43%), the advantage further expands to 8.62 percentage points; and compared with the early industrial defect detection method PADIM (F1 score 89.21%), the improvement is 5.84 percentage points. This significant gain fully demonstrates that this invention outperforms existing mainstream solutions in industrial defect detection tasks.

[0144] Further analysis of the balance between precision and recall reveals that most comparison algorithms often experience a significant drop in precision when achieving high recall. In contrast, the method of this invention maintains a recall rate close to 95% while achieving a precision rate as high as 95.30%, thus achieving dual suppression of false positives and false negatives. This advantage directly translates into improved reliability in defect detection: visualization results show that this invention effectively avoids false negatives and false negatives in cases of minor scratches, low-contrast stains, and complex textured backgrounds, significantly reducing the number of missed pixels and fragmented false positive areas.

[0145] In summary, the experimental results validated the method of this invention from both quantitative and qualitative perspectives. The performance improvement is mainly attributed to the accurate capture of multi-scale defect features by the cross-layer collaborative multi-level attention feature extraction network, the effective expansion of training data by the conditional generative adversarial network, and the fine-tuning of model optimization through the progressive weighted training strategy. These three factors work together to enable the model to robustly detect various defects even in complex industrial scenarios, achieving the optimal balance between accuracy, recall, and robustness.

[0146] To systematically analyze the contributions of cross-layer collaborative multi-level attention feature extraction network, conditional generative adversarial network and progressive weighted training strategy to the overall performance, this invention designed and executed multiple sets of controlled ablation experiments on the MVTec AD benchmark, and the results are summarized in Table 2.

[0147] Table 2 Ablation Experiment Results

[0148]

[0149] First, the "Base" model with ResNet-50 as its backbone has demonstrated a competitive baseline performance without introducing any additional modules: accuracy of 89.21%, precision of 85.62%, recall of 87.34%, and F1 score of 86.43%, which fully demonstrates that the pre-trained convolutional backbone itself has a certain representational ability.

[0150] Building upon this, embedding a separate Multi-Level Defect Attention Module (MLDA) further improved accuracy to 92.35%, precision to 89.14%, recall to 90.28%, and F1 score to 89.72%, representing increases of 3.14 and 3.29 percentage points, respectively. These results demonstrate that the MLDA mechanism can simultaneously capture both shallow detail information and deep semantic information, enabling adaptive detection of defects of varying sizes and shapes, and significantly enhancing sensitivity to minute defects.

[0151] Furthermore, after adding a Conditional Generative Adversarial Network (cGAN), the accuracy improved to 95.62%, precision to 93.24%, recall to 94.35%, and F1 score to 93.84%. These results demonstrate that cGANs can generate enhanced defect feature vectors of specified categories and difficulty in the feature space, significantly expanding the training dataset without increasing actual data acquisition costs, and effectively alleviating the model overfitting problem caused by insufficient samples.

[0152] When the multi-level defect attention module, conditional generative adversarial network, and progressively weighted training strategy are applied in synergy, the complete method achieves optimal performance across all metrics: accuracy 96.80%, precision 95.30%, recall 94.78%, and F1 score 95.05%. Compared to the baseline, accuracy is improved by 7.59 percentage points, and the F1 score by 8.62 percentage points. This gain reflects the complementary effect between feature extraction, data augmentation, and training strategy optimization, significantly enhancing the model's ability to detect various defects.

[0153] In summary, the industrial defect detection method proposed in this invention, based on cross-layer collaborative multi-level attention and progressive feature enhancement, has been validated through system experiments on the MVTec AD dataset. The results demonstrate that this method still achieves state-of-the-art performance with an F1 score of 95.05% in complex industrial scenarios. Ablation analysis further shows that the cross-layer collaborative multi-level attention feature extraction network can effectively capture multi-scale defect features, the conditional generative adversarial network can effectively expand the training data, and the progressive weighted training strategy can significantly improve the model's generalization ability. Comprehensive comparison and visualization analysis both show that this invention achieves the best balance in accuracy, recall, and robustness, providing a highly reliable and deployable intelligent detection tool for automated quality inspection of industrial products such as metal surfaces, textiles, and electronic components.

[0154] Finally, it should be noted that the above embodiments are intended to illustrate the technical solutions of the present invention and do not constitute any limitation on the present invention. Those skilled in the art should fully understand that modifications to the technical solutions described in the foregoing embodiments or equivalent substitutions for any part or all of the technical features are entirely feasible. Such modifications or substitutions, as long as they do not depart from the scope of protection defined by the claims of the present invention, should be considered reasonable extensions of the present invention.

Claims

1. An industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement, characterized in that, include: S1. Collect surface images of qualified industrial products and those with different types of defects, and label the surface images of industrial products with defect category labels to construct an industrial defect dataset; S2. Construct a cross-layer collaborative multi-level attention feature extraction network, which includes a feature pyramid backbone network, a multi-level defect degree attention module, and a cross-layer feature fusion module. The feature pyramid backbone network is used to extract multi-level feature maps from the surface image of industrial products; the multi-level defect severity attention module is used to perform attention weighting processing on the multi-level feature maps to obtain slight defect attention features, medium defect attention features, and severe defect attention features; the cross-layer feature fusion module is used to perform weighted fusion of the slight defect attention features, medium defect attention features, and severe defect attention features to obtain a fused feature map. S3. The industrial defect dataset constructed in S1 is used to initially train the cross-layer collaborative multi-level attention feature extraction network and the defect classifier to obtain the basic feature extractor and the basic defect classifier. During the initial training process, the difficulty is assessed based on the training loss value and confidence score of each defect sample in the industrial defect dataset. The defect samples are divided into three levels: simple samples, medium difficulty samples, and difficult samples, resulting in a defect sample set with difficulty labels and defect category labels. S4. Use the basic feature extractor to extract multi-level features of each defect sample in the defect sample set and fuse the features of different levels to obtain a real defect feature map of the same dimension, thus forming a defect feature dataset with difficulty label and defect category label. S5. Construct a conditional generative adversarial network, which includes a generator and a discriminator. The generator takes a random noise vector and a specified defect category and difficulty level as input to generate an enhanced defect feature map with the specified defect category and difficulty. The discriminator performs adversarial training on the enhanced defect feature map and the real defect feature map to optimize the generator parameters. S6. Adopt a progressive weighted training strategy and retrain the basic defect classifier using real defect feature maps and enhanced defect feature maps: In the early stage of training, simple samples are used first. As the training rounds increase, medium and difficult samples are gradually introduced, and the loss weights of various samples are dynamically adjusted to achieve progressive model optimization from easy to difficult, thus obtaining an enhanced defect classifier. S7. Input the surface image of the industrial product to be detected into the basic feature extractor and the enhanced defect classifier, and output the defect detection result.

2. The industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement according to claim 1, characterized in that, The specific structure of the cross-layer collaborative multi-level attention feature extraction network is as follows: The feature pyramid backbone network adopts an improved ResNet framework, and the processing flow is as follows: The input industrial product surface image is first subjected to large-scale convolution and nonlinear activation to obtain an initial feature map; then the initial feature map is downsampled to obtain a pooled feature map; the pooled feature map is then passed through four residual module blocks in sequence to output the first to fourth level feature maps respectively. The multi-level defect severity attention module includes cascaded minor defect attention modules, moderate defect attention modules, and severe defect attention modules. The processing flow is as follows: First, the pooled feature map is input into the minor defect attention module. A minor defect attention map is generated using the first-level feature map and weighted to obtain the minor defect attention feature. Then, the minor defect attention feature is input into the moderate defect attention module. A moderate defect attention map is generated using the second-level and third-level feature maps and weighted to obtain the moderate defect attention feature. Finally, the moderate defect attention feature is input into the severe defect attention module. A severe defect attention map is generated using the fourth-level feature map and weighted to obtain the severe defect attention feature. The cross-layer feature fusion module performs weighted fusion of the minor flaw attention features, the moderate flaw attention features, and the severe flaw attention features to obtain a fused feature map.

3. The industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement according to claim 2, characterized in that, In the multi-level defect severity attention module, the processing flow of the minor defect attention module is as follows: in, This indicates a minor flaw attention map. express Activation function express Convolution operation, This represents the first-level feature map. This represents element-wise multiplication; Indicates minor flaws and attention characteristics. This represents the feature map after pooling; The processing flow of the medium defect attention module is represented as follows: in, This represents a medium-level defect attention map. and These represent the second-level feature map and the third-level feature map, respectively. This indicates a moderate level of attentional impairment. The processing flow of the severe defect attention module is represented as follows: in, Attention graph indicating serious defects This represents the fourth-level feature map; This indicates a serious flaw in attention characteristics.

4. The industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement according to claim 1, 2, or 3, characterized in that, In step S3, the initial training process of the cross-layer collaborative multi-level attention feature extraction network and the defect classifier includes: The industrial defect dataset is input into a cross-layer collaborative multi-level attention feature extraction network and a defect classifier for forward propagation, using the classification cross-entropy loss function. End-to-end training is performed, where The number of samples in the industrial defect dataset. This represents the total number of defect categories. For sample index, For defect category index, For the sample In the category of defects The real labels on it For the sample In the category of defects The predicted probability; The gradient is calculated and the network parameters are updated using the backpropagation algorithm; the training is repeated iteratively until the model converges, resulting in a basic feature extractor and a basic defect classifier.

5. The industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement according to claim 4, characterized in that, In step S3, the difficulty of the defective samples is assessed based on the following method: At the end of the initial training Each round, statistical analysis of each defect sample. Average loss value and average confidence score : in, For training round index, For the first Flawed samples during training rounds The loss value, For the first Flawed samples during training rounds The predicted probability; Calculate defect samples Overall difficulty rating ,in and These are the weighting coefficients; Calculate the mean of the difficulty scores for all defect samples. and standard deviation ,in This represents the total number of defect samples in the industrial defect dataset. Based on the overall difficulty score, the defective samples are divided into three levels: like Then the defective sample Difficulty level , which are simple samples; like Then the defective sample Difficulty level This is a sample of medium difficulty. like Then the defective sample Difficulty level This is a difficult sample.

6. The industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement according to claim 5, characterized in that, In step S5, the construction process of the conditional generative adversarial network includes: The generator adopts an encoder-decoder structure, and the processing flow is as follows: A random noise vector is input into the encoder. Specified defect category labels and difficulty level ,in, and First, it is expanded into a random noise vector through an embedding mapping. A vector with consistent dimensions, then compared with a random noise vector. Concatenate to obtain the concatenated vector ; Concatenate vectors The latent variables are obtained after being processed through two fully connected layers and a batch normalization layer. ; hidden variables The input to the decoder is processed through two fully connected layers and a transposed convolutional layer to obtain the enhanced defect feature map. ; The discriminator processing flow is as follows: The enhanced defect feature map generated by the generator is processed... Defect Category Label and difficulty level Or a real image showing the characteristics of the defects. and its defect category labels and difficulty level As input, the defect category label and difficulty level are converted into vector representations through an embedding mapping layer, and then the vector representations are concatenated with the input defect feature map to obtain joint features. Joint features First, feature extraction is performed through the first convolutional layer. Subsequently passed The activation function is used to perform a nonlinear mapping to obtain Then, a downsampling operation is performed through a pooling layer to obtain... Then, features are further extracted through a second convolutional layer. Finally, the features are processed through a fully connected layer. Mapping to the discriminant space yields the discriminant result. ,in Enhanced defect feature map representing the input Or a real image showing the characteristics of the defects. The probability of being a real sample.

7. The industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement according to claim 6, characterized in that, In step S5, the training process of the conditional generative adversarial network includes: The loss function of the generator is: ,in, This represents the probability that the feature map is true under given conditions. This represents the enhanced defect feature map generated by the generator. This represents the log-expected probability that the generated enhanced defect feature map is judged as real; The loss function of the discriminator is: , It represents the logarithmic expectation of the probability that a true defect feature map is judged as true; The discriminator and generator are trained alternately, and training is stopped when the training loss no longer decreases after 10 consecutive rounds.

8. The industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement according to claim 7, characterized in that, In step S6, the basic defect classifier is retrained using a progressive weighted training strategy, including: The trained generator is used to generate enhanced defect feature maps of specified defect categories and difficulties, thus constructing an enhanced defect feature dataset. The base defect classifier is then retrained using the defect feature dataset and the enhanced defect feature dataset. The retraining batch includes real defect samples (sample data from the defect feature dataset) and enhanced defect samples (sample data from the enhanced defect feature dataset). The gradient is calculated and the network parameters are updated using the backpropagation algorithm. The training is repeated iteratively until the model converges, resulting in the enhanced defect classifier. The retraining loss function of the basic defect classifier is: ;in, Indicates the first The weighted cross-entropy loss value for each round of training; This indicates the total number of samples in the current batch; Indicates the first During round training, the model tests the samples It belongs to the category of defects The predicted probability; For the sample In the category of defects The real labels on it; Indicates the first Training samples The loss weighting coefficient, Divided into real defect sample weights and enhance the weight of defective samples ; The weights of the real defective samples are calculated as follows: The total number of retraining rounds... It is divided into three stages: In this phase, only simple samples are used, and the weights are... ; In this phase, medium-difficulty samples are introduced, and simple samples are weighted. Medium difficulty sample weights ; In this phase, difficult samples are introduced, and simple samples are weighted. Medium difficulty sample weights Difficult sample weights ,in, This indicates the total number of retraining rounds. Indicates the index of the current training round; The method for calculating the weight of the enhanced defective samples is as follows: , This indicates that the enhanced defect features are in the first... Wheel weights; proportional coefficients The range of values ​​is .

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor; characterized in that, When the processor executes the computer program, it causes the electronic device to perform the industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon; characterized in that, When the computer program is executed by the processor, it implements the industrial defect detection method based on cross-layer collaborative multi-level attention and progressive feature enhancement as described in any one of claims 1 to 8.

Images