Object surface defect detection method based on abnormal synthesis strategy

Abstract

The invention provides an object surface defect detection method based on an abnormal synthesis strategy, and the method comprises the following steps: obtaining a normal sample image, and obtaining a normal image feature, a global abnormal feature and a local abnormal feature based on the normal sample image; training a discriminator based on the normal image features, the global abnormal features and the local abnormal features corresponding to the training set; training an FDM discriminator by aligning the feature distribution of the test set with the feature distribution of the training set; and obtaining a to-be-detected target image, performing feature extraction on the to-be-detected target image based on the feature extraction network, performing feature distribution adjustment on the extracted features based on the FDM discriminator, and inputting the adjusted features into the trained discriminator to obtain a cigarette case defect detection result.

Description

Technical Field

[0001] This invention relates to the field of appearance inspection technology, and more specifically, to a method for detecting surface defects of objects based on an anomaly synthesis strategy. Background Art

[0002] Cigarette boxes and cigarettes are the main products of the tobacco industry, and their appearance quality directly reflects the production level of cigarette factories. During high-speed assembly line production, due to the complexity and variability of the factory environment and the limitations of production technology, various defects such as scratches, spots, and stains will inevitably occur on cigarette boxes and cigarettes, thus affecting the appearance and brand image of cigarettes.

[0003] With the rapid development of industrial automation and quality control, anomaly detection (AD) technology is playing an increasingly important role in manufacturing and consumer product inspection. Machine vision-based anomaly detection technology has shown significant advantages in replacing traditional manual inspection, greatly improving accuracy and efficiency. With the rapid development of deep learning technology, especially its breakthroughs in object detection, deep learning-based defect detection has gradually become a research and application hotspot. Deep learning technology, by constructing deep neural networks, can automatically learn and extract features from data, thus overcoming the dependence on manually designed features in traditional methods and exhibiting better generalization ability and adaptability.

[0004] In recent years, target appearance anomaly detection technologies based on computer vision and deep learning have been applied in the tobacco industry. Currently, commonly used deep learning-based anomaly detection methods include autoencoders (AEs), generative adversarial networks (GANs), variational autoencoders (VAEs), support vector machines (SVMs), and isolation forests. These methods have achieved significant results in various industries and are also applicable to target anomaly detection in the tobacco industry, showing promising application prospects. In the tobacco industry, many studies have utilized traditional image processing techniques for target detection and analysis. However, target appearance anomaly detection is characterized by small changes in imaging pose, stable lighting conditions, and simple backgrounds. Therefore, traditional image processing algorithms such as threshold segmentation, contour detection, and template matching can accurately extract target features and achieve high detection accuracy. However, these algorithms heavily rely on researchers' prior knowledge and feature engineering design capabilities, and are prone to misjudgment when external conditions such as product color and pose change slightly.

[0005] With the successful application of convolutional neural networks in image classification, the tobacco industry has also begun to widely adopt deep learning technology for product quality inspection. For example, Gao Zhenyu et al. used convolutional neural networks to identify the proportion of tobacco shreds; Chen Tongyu et al. used an improved YOLOv3 model to achieve online defect detection of cigarette boxes, with an average detection accuracy of 97.2%; Shan Yuxiang et al. used the Mask R-CNN model to detect and identify cigarette boxes in complex scenes, with an average accuracy of 95%. Furthermore, Peng et al. proposed an improved YOLOv5 algorithm for defect detection in cigarette box appearance. This algorithm enhances the ability to express and fuse defect features at different scales by optimizing the model structure and introduces an efficient focusing learning mechanism, enabling the model to accurately detect defect areas even in complex backgrounds, with an average accuracy of 91.6%. However, this model has high computational complexity, and its sensitivity to small defects needs to be improved.

[0006] To further improve detection accuracy, Yuan et al., building upon YOLOv4, significantly enhanced the detection accuracy of cigarette appearance defects by introducing attention mechanisms and Spatial Pyramid Pooling (SPP) techniques, achieving an average detection accuracy of 91.7%. Simultaneously, Yuan et al. proposed a real-time, high-precision cigarette appearance defect detection method based on the YOLOv7-tiny model. By introducing techniques such as variable convolutional networks into the model, they further improved its performance, achieving an average detection accuracy of 94.1% for cigarette appearance defects.

[0007] The aforementioned methods are of great significance for anomaly detection research in the tobacco industry. However, anomaly detection in the tobacco industry still faces many challenges: First, because target images are confidential information for each tobacco company, these companies typically exercise strict control over image data during the production process. This involves trade secrets and intellectual property protection, further leading to insufficient target defect sample datasets and making model training difficult. Second, in actual production, defects are diverse and vary widely in scale, ranging from tiny stains on the target surface to obvious defects such as holes and wrinkles. Furthermore, with the increasing complexity of production lines, subtle defects in images are often difficult to detect, especially under the influence of natural factors such as changes in lighting, object posture, or background, significantly impacting the model's prediction efficiency. Simultaneously, with continuous advancements in production technology, the difference between defective and normal areas of the target is gradually decreasing. The existence of these weak defects further increases the false detection rate of the model, severely affecting detection efficiency.

[0008] In order to solve the above problems, people have been seeking an ideal technological solution. Summary of the Invention

[0009] Therefore, it is necessary to provide a method for detecting surface defects and identifying diseases based on an anomaly synthesis strategy to address the aforementioned technical problems.

[0010] To achieve the above objectives, the first aspect of the present invention provides a method for detecting surface defects of an object based on an anomaly synthesis strategy, comprising the following steps:

[0011] Obtain normal sample images to form the training and test sets;

[0012] Feature extraction networks are used to extract features from normal sample images to obtain normal image features;

[0013] Normal image features are processed based on Gaussian noise to obtain Gaussian anomalous features. Gradient ascent and truncation projection are then applied to the Gaussian anomalous features to obtain global anomalous features.

[0014] Local anomaly images are generated based on a local anomaly synthesis strategy and anomaly sample images. Feature extraction networks are used to extract features from the local anomaly images to obtain local anomaly features.

[0015] During the training phase, the discriminator is trained based on the normal image features, global anomaly features, and local anomaly features corresponding to the training set.

[0016] During the testing phase, the FDM discriminator is trained by aligning the feature distribution of the test set with the feature distribution of the training set.

[0017] In the detection phase, the target image to be detected is acquired, and features are extracted from the target image based on the trained feature extraction network. The extracted features are then adjusted based on the trained FDM discriminator, and the adjusted features are input into the trained discriminator to obtain the cigarette box defect detection result.

[0018] In one possible embodiment, during the testing phase, the specific steps for training the FDM discriminator by aligning the feature distribution of the test set with the feature distribution of the training set include:

[0019] The test samples of the test set are fed into the feature extraction network for feature extraction to obtain the test image features;

[0020] A training sample is randomly selected from the training set and fed into the feature extraction network for feature extraction to obtain the training image features.

[0021] The test image features and training image features are fed into the FDM discriminator to calculate the feature distribution difference. When the maximum mean difference between the training image features and the test image features is less than the MMD threshold, the test image features are fed into the discriminator for classification. When the maximum mean difference between the training image features and the test image features is greater than or equal to the MMD threshold, the test image features are adjusted so that the maximum mean difference between the training image features and the test image features is less than the MMD threshold. max The formula is adjusted as follows:

[0022]

[0023] in It is a hyperparameter used to control the mixing ratio of training image features and test image features; and These are the ascending sort values ​​of the training image features and the test image features, respectively, where n is the number of feature vector elements.

[0024] In one possible embodiment, when training the FDM discriminator by aligning the feature distribution of the test set with the feature distribution of the training set, high-frequency statistical features are extracted and saved. When adjusting the feature distribution of the extracted features based on the FDM discriminator, the extracted features are linearly registered based on the saved high-frequency statistical features.

[0025] In one possible embodiment, the feature extraction network uses WideResNet50 as the backbone network, and introduces the channel attention module in the CBAM attention mechanism into each residual block of WideResNet50 to perform global average pooling and max pooling on the input features to obtain global average pooling features and global max pooling features.

[0026] The channel attention weights are obtained by fusing the global average pooling features and the global max pooling features. Finally, the channel attention weights are multiplied by the original input features F and then output.

[0027] In one possible embodiment, the discriminator is a multilayer perceptron with a single hidden layer and a sigmoid activation function, which directly outputs the anomaly confidence score for each feature point and calculates the model loss function based on the anomaly confidence score.

[0028] L total =L n +L gas +L las +L disc +L feat

[0029] Among them, L n For normal feature loss function,

[0030] Among them, f BCE It is a binary cross-entropy loss function. It is the discriminator's recognition of normal features F n The output of represents the confidence level of normal features, e i,n The closer the value is to 0, the closer the discriminator considers the feature to be a normal feature; i,n The closer a value is to 1, the closer the discriminator considers this feature to be an anomalous feature;

[0031] L gas The global anomaly feature loss function is given by the following formula:

[0032] in It is the discriminator's assessment of global anomaly features F gas The output of represents the confidence level of the anomalous feature, e i,gas The closer an e is to 0, the closer the discriminator considers the feature to be a normal feature; conversely, if e is closer to 0, the discriminator considers the feature to be closer to a normal feature. i,gas A value close to 1 indicates that the discriminator's prediction is relatively accurate, and the feature is considered to be closer to an abnormal feature.

[0033] L las The loss function for local anomaly features is given by the following formula:

[0034] in m i It is a mask for a localized anomalous image;

[0035] The loss function of the FDM discriminator is:

[0036]

[0037] The loss function of the feature extractor is:

[0038]

[0039] Where D is the FDM discriminator, The expected value representing the features of the training image. λ·MMD(P,Q) represents the expected value of the test image features; λ·MMD(P,Q) is the regularization term of the loss function of the feature extraction network, where λ is a hyperparameter used to control the weight of the MMD regularization term.

[0040]

[0041] To achieve the above objectives, a second aspect of the present invention provides a method for detecting defects in cigarette boxes based on an anomaly synthesis strategy, comprising the following steps:

[0042] After obtaining a normal cigarette box image, high-brightness patches and specular spots are randomly added to a local area of ​​the normal cigarette box image, and affine changes at the tilt angle are added to simulate the perspective distortion when the cigarette box is not perpendicular to the camera. The image obtained in the previous step is used as the cigarette box image to be detected.

[0043] The method for detecting defects in cigarette boxes as described in the first aspect is used to detect defects in cigarette boxes.

[0044] The feature extraction network uses WideResNet50 as the backbone network. The CBAM attention mechanism is introduced into each residual block of WideResNet50, and a highlight suppression branch is added to the CBAM attention mechanism to shield the weights of reflective areas.

[0045] The processing steps of the highlight suppression branch are as follows: obtain the brightness map of the input features, and determine the bright area when the brightness value is greater than the brightness threshold; convert the bright area into a mask map, and fuse the mask map with the spatial attention map obtained by the spatial attention mechanism to obtain the spatial attention feature map after reflection shielding.

[0046] To achieve the above objectives, a third aspect of the present invention provides a method for detecting cigarette defects based on an anomaly synthesis strategy, comprising the following steps:

[0047] The method for detecting defects in cigarette boxes as described in the first aspect is used to detect defects in cigarette sticks.

[0048] In this process, after obtaining a normal cigarette image, edge detection and contour extraction methods are used to obtain the contour boundary of the cigarette. Within the obtained contour boundary, a region is randomly selected and local anomalies are superimposed. At the same time, affine transformation and local bending techniques are used to flexibly distort the abnormal region to obtain an abnormal cigarette image.

[0049] Furthermore, Gaussian noise is injected using a non-uniform distribution method when acquiring global anomaly features.

[0050] To achieve the above objectives, a fourth aspect of the present invention provides a surface defect detection device based on an anomaly synthesis strategy, comprising:

[0051] The acquisition unit is used to acquire normal sample images to form the training set and the test set;

[0052] The local anomaly image generation module generates local anomaly images based on a local anomaly synthesis strategy and anomaly sample images.

[0053] The feature extraction network is used to extract features from normal sample images, images with local anomalies, and images of the target to be detected, respectively.

[0054] The global anomaly feature acquisition module is used to process normal image features based on Gaussian noise to obtain Gaussian anomaly features, and then perform gradient ascent and truncation on the Gaussian anomaly features to obtain global anomaly features.

[0055] The training module is used to train the discriminator based on the normal image features, global abnormal features and local abnormal features corresponding to the training set, and to train the FDM discriminator by aligning the feature distribution of the test set with the feature distribution of the training set.

[0056] The defect detection module is used to call the feature extraction network to extract features from the target image to be detected, adjust the feature distribution of the extracted features based on the FDM discriminator, and input the adjusted features into the trained discriminator to obtain the defect detection result of the cigarette box.

[0057] The beneficial effects of this invention are as follows:

[0058] This invention applies global anomaly synthesis and local anomaly synthesis strategies to the task of detecting surface defects. By coordinating these two anomaly synthesis strategies, the model can generate more representative and diverse anomaly samples, significantly enhancing the model's ability to identify weak defects.

[0059] By introducing the channel attention mechanism from the CBAM attention mechanism, this model can adaptively focus on key regions in the image, reduce background noise interference, and thus improve the quality of feature representation.

[0060] By introducing FDM feature distribution matching technology, the distribution difference between the test set and the training set is reduced during the testing phase, which significantly improves the model's generalization ability and robustness in real industrial environments.

[0061] In terms of FDM module optimization, a "feature alignment caching strategy" is introduced in the inference stage. This means that the high-frequency statistical features extracted during training are saved, and only one linear registration is performed during inference, instead of deep convolutional alignment.

[0062] Furthermore, when applied to cigarette box defect detection, high-brightness patches and mirror simulations are randomly added to local areas of normal cigarette box images. At the same time, affine changes in tilt angle are added to simulate the perspective distortion when the cigarette box is not perpendicular to the camera, so as to improve the detection accuracy of cigarette boxes. Meanwhile, a complete CBAM attention mechanism is introduced into the feature extraction network, and a high-brightness suppression branch is added to the CBAM attention mechanism to shield the weights of reflective areas.

[0063] When applied to cigarette defect detection, after acquiring a normal cigarette image, the outline boundary of the cigarette image is extracted, and local anomalies are synthesized within the boundary region. At the same time, the abnormal region is flexibly distorted using affine transformation and local bending techniques to obtain an abnormal cigarette image. When acquiring global anomaly features, Gaussian noise is injected in a non-uniform distribution manner. Through the above methods, the accuracy of cigarette detection is improved. Attached Figure Description

[0064] Figure 1 This is a flowchart illustrating the surface defect detection method of the present invention.

[0065] Figure 2 This is a schematic diagram of the process for synthesizing local anomalies according to the present invention.

[0066] Figure 3 This is a graph of the loss function of the present invention.

[0067] Figure 4 This is a comparison chart of FPS and AUROC of the present invention.

[0068] Figure 5 This is a comparison chart of FPS and PRO of the present invention.

[0069] Figure 6 This is a comparison diagram of defect detection in cigarette boxes according to the present invention.

[0070] Figure 7 This is a comparison chart of cigarette defect detection according to the present invention. Detailed Implementation

[0071] The technical solution of the present invention will be further described in detail below through specific embodiments.

[0072] Example 1

[0073] To achieve the above objectives, this embodiment provides a method for detecting surface defects of objects based on an anomaly synthesis strategy, such as... Figure 1 As shown, the following steps are included:

[0074] Training and testing phase

[0075] Step 1: Obtain normal sample images to form a training set and a test set. Extract features from the normal sample images based on the feature extraction network to obtain normal image features.

[0076] Specifically, the feature extraction network uses WideResNet50 as the backbone network, and introduces the CBAM attention mechanism into each residual block of WideResNet50. The CBAM module is pruned and simplified, and the original dual-branch structure is replaced with lightweight channel attention.

[0077] Specifically, pruning and simplifying the CBAM module means introducing the CBAM attention mechanism, but only retaining the channel attention module within the CBAM attention mechanism.

[0078] CBAM is a pluggable attention module that improves the quality of feature representations by adaptively focusing on important regions in an image. The CBAM module consists of two sub-modules: a channel attention module and a spatial attention module. The spatial attention module learns spatially important regions, reinforcing the focus on key areas. The channel attention mechanism learns inter-channel dependencies through two fully connected layers and an activation function.

[0079] In this embodiment, considering that channel attention can provide sufficient performance improvement in many tasks, the spatial attention mechanism is removed in order to reduce computational complexity and the number of parameters.

[0080] Specifically, the working principle of the channel attention module is as follows:

[0081] Global average pooling and max pooling are performed on the input features to obtain the global average pooled features F. gap and global max pooling feature F gmp ;

[0082] F gap =GlobalAveragePooling(F)

[0083] F gmp =GlobalMaxPooling(F)

[0084] The global average pooling feature F gap and global max pooling feature F gmp The channel attention weights F are obtained by fusion. cat =concat(F gap ,F gmp Finally, the channel attention weights F are... cat The output F is obtained by multiplying the original input feature F. out =F×F cat .

[0085] In this embodiment, by introducing the channel attention module from CBAM into each residual block of WideResNet50, the model can more effectively focus on important regions in the image. Especially when detecting weak defects, it can reduce the interference of background noise and improve the ability to identify defective regions. In the task of detecting surface defects, CBAM helps the model to more accurately locate and identify these anomalies when faced with tiny defects in the image, thereby improving detection accuracy and robustness.

[0086] In addition, before the input image enters the feature extraction process, interference from illumination and contrast should be minimized as much as possible. Specific techniques used include Gamma correction and adaptive contrast enhancement (CLAHE).

[0087] Gamma correction: Automatically identifies the brightness state of an image and adjusts the grayscale distribution; softens overly bright areas and sharpens overly dark areas.

[0088] Adaptive Contrast Enhancement (CLAHE): Processes image contrast using local histogram equalization; while preserving detail, it enhances texture contrast, making defects more visible.

[0089] Step 2: Process the normal image features based on Gaussian noise to obtain Gaussian anomalous features, and perform gradient ascent and truncation projection processing on the Gaussian anomalous features to obtain global anomalous features.

[0090] Global anomaly synthesis enhances a model's ability to identify weak defects by generating global anomaly samples in the feature space. The core idea of ​​global anomaly synthesis is to introduce Gaussian noise into the feature space, and combine gradient ascent and truncated projection techniques to generate anomaly samples that approximate the normal sample distribution.

[0091] Specifically, the steps in step 2 are as follows:

[0092] Gaussian noise is added to the image features of the training images to obtain Gaussian anomaly features:

[0093] F noise =F+θθ~N(μ,σ 2 )

[0094] Gradient ascent processing is applied to Gaussian anomaly features to obtain global anomaly features. The gradient ascent direction is determined by the discriminator's loss function, and the gradient ascent step size is calculated using the following formula:

[0095]

[0096] Where ω is the learning rate, which controls the step size of gradient ascent; L gas The loss function of the discriminator;

[0097] The truncation projection method is used to process global outliers, restricting them to a neighborhood of the normal sample distribution. The formula for truncation projection is as follows:

[0098] F final =clip(F grad ,F-υ,F+υ)

[0099]

[0100] Where v is the set gradient threshold, determined through model performance tuning; clip() is the cutoff function, F final This represents the final global anomaly features obtained.

[0101] Step 3: Generate local anomaly images based on the local anomaly synthesis strategy and anomaly sample images, and extract the anomalous features of the local anomaly images based on the feature extraction network.

[0102] like Figure 2 As shown, local anomaly synthesis simulates local defects (such as small stains, scratches, etc.) on a target surface by generating local anomaly samples in the image space. The core idea of ​​local anomaly synthesis is to synthesize local anomaly regions on a normal image through texture overlay and mask generation techniques.

[0103] The specific steps of step 3 are as follows:

[0104] A mask for local anomaly regions is generated using Burmester noise to simulate local defects on the object's surface; the formula for generating the mask M is:

[0105] M(x,y)=PerNose(x,y,sclae,octaves,persistence,lacunarity)

[0106] Where x and y are image coordinates, and scale, octaves, persistence, and lacunairerity are parameters of the Berlin noise, which control the detail and complexity of the noise.

[0107] Add the mask to the normal sample image;

[0108] Randomly select a texture T from the texture library and overlay it onto the mask to obtain a local anomaly image;

[0109] The formula for texture overlay is as follows:

[0110]

[0111] Among them, I normal T is a normal sample image, and T is a texture image. This indicates element-wise multiplication, where α represents the transparency coefficient. The smaller α is, the less noticeable the local defects are.

[0112] Step 4, Training Phase: The discriminator is trained based on the normal image features, global anomaly features, and local anomaly features corresponding to the training set.

[0113] Step 5, the testing phase, involves training the FDM discriminator by aligning the feature distribution of the test set with the feature distribution of the training set.

[0114] In one possible embodiment, the specific steps of training the FDM discriminator by aligning the feature distribution of the test set with the feature distribution of the training set include:

[0115] The test samples of the test set are fed into the feature extraction network for feature extraction to obtain the test image features;

[0116] A training sample is randomly selected from the training set and fed into the feature extraction network for feature extraction to obtain the training image features.

[0117] The test image features and training image features are fed into the FDM discriminator to calculate the feature distribution difference. When the maximum mean difference between the training image features and the test image features is less than the MMD threshold, the test image features are fed into the discriminator for classification. When the maximum mean difference between the training image features and the test image features is greater than or equal to the MMD threshold, the test image features are adjusted so that the maximum mean difference between the training image features and the test image features is less than the MMD threshold. max The formula is adjusted as follows:

[0118]

[0119] in It is a hyperparameter used to control the mixing ratio of training image features and test image features; and These are the ascending sort values ​​of the training image features and the test image features, respectively, where n is the number of feature vector elements.

[0120] Specifically, the FDM discriminator loss function:

[0121]

[0122] Feature extractor loss function:

[0123]

[0124] Where D is the FDM discriminator, The expected value representing the features of the training set. This represents the expected value of the test set features. λ·MMD(P,Q) serves as the regularization term in the feature extractor's loss function, where λ is a hyperparameter used to control the weight of the MMD regularization term. By continuously optimizing the feature extractor parameters and the FDM discriminator parameters, the maximum mean difference of MMD is continuously reduced.

[0125] Specifically,

[0126] Where P and Q represent the feature distributions of the training set and the test set, respectively, and F i P and Fj Q φ(·) is the feature representation of the training set and the test set, respectively. φ(·) is the sum function that maps the features to a high-dimensional feature space, and Γ is the reproducing kernel Hilbert space.

[0127] Understandably, existing anomaly detection algorithms have demonstrated good performance in the field of anomaly detection. However, these algorithms are typically based on a key assumption: that the training and test sets come from the same data distribution. However, in cigarette box defect detection tasks, the training set used in the model's training process is often collected under specific conditions, while the test set may face more complex and diverse industrial environments. This difference between the training and testing environments leads to a significant shift in the data distribution of the training and test sets, which in turn causes a substantial decrease in the model's performance in practical applications.

[0128] Specifically, in order to make the feature extractor more robust when extracting features, an FDM discriminator is introduced. Adversarial training techniques are used to minimize distribution differences, while the feature extractor is trained to deceive the discriminator, so that the distribution features of the training set and the test set tend to be consistent.

[0129] FDM effectively reduces the distributional discrepancy between the training and test sets by matching their distributions in the feature space. Specifically, FDM aligns the feature distributions of the training and test sets, enabling the model to better adapt to changes in the testing environment, thereby improving its generalization ability and robustness in real-world industrial scenarios. This technology provides an effective solution for cigarette box defect detection tasks, significantly improving the model's detection performance in complex environments and giving the feature extractor better robustness when extracting features.

[0130] Application reasoning stage

[0131] Step 6: Obtain the target image to be detected, extract features from the target image based on the feature extraction network, adjust the feature distribution of the extracted features based on the FDM discriminator, and input the adjusted features into the trained discriminator to obtain the cigarette box defect detection result.

[0132] Furthermore, when aligning the normal image feature distributions of the test set and training set using the FDM discriminator, high-frequency statistical features are extracted and saved. When adjusting the feature distribution of the extracted features based on the FDM discriminator, the extracted features are linearly registered based on the saved high-frequency statistical features.

[0133] It is understandable that this embodiment trains the FDM feature distribution matching technique during the training phase, enabling the model to adjust the features of each test image data during the inference phase to make them closer to the features of the training image data. This greatly improves the robustness of the model and plays a certain role in the anomaly detection of the target.

[0134] Furthermore, such as Figure 1 As shown in this embodiment, during the training phase of the model, the model has a total of three branches: the normal image branch is used to train the discriminator to recognize normal image features, the global anomaly branch is used to train the discriminator to recognize global anomaly features, and the local anomaly branch is used to train the discriminator to recognize local anomaly features. The three branches obtain three features, which are then sent to the discriminator respectively.

[0135] Preferably, the discriminator in this embodiment is a multilayer perceptron with a single hidden layer and a sigmoid activation function, which directly outputs the anomaly confidence score of each feature point.

[0136] The training objective consists of three parts, and therefore there are three basic loss functions to control the entire model training process.

[0137] Normal feature loss L n The abnormal confidence e is obtained by using binary cross-entropy loss to calculate the normal training image features from the FDM discriminator, which are then fed into discriminator D. i,n :

[0138]

[0139] Among them, f BCE It is a binary cross-entropy loss function. It is the discriminator D that distinguishes normal features F. n The output of represents the confidence level of normal features, e i,n The closer an expression is to 0, the closer the discriminator considers the feature to be a normal feature. Conversely, e i,n The closer a value is to 1, the closer the discriminator considers the feature to be an anomalous.

[0140] Global anomaly feature loss L gas Similar to the normal feature loss, the binary cross-entropy loss function is also used for calculation. The global anomalous features are then fed into the discriminator D to obtain the anomalous confidence e. i,gas :

[0141]

[0142] in It is the discriminator D that evaluates the global anomaly features F. gas The output of represents the confidence level of the anomalous feature, e i,gasThe closer an e is to 0, the closer the discriminator considers the feature to be a normal feature; conversely, if e is closer to 0, the discriminator considers the feature to be closer to a normal feature. i,gas A value close to 1 indicates that the discriminator's prediction is relatively accurate, and the feature is considered to be closer to an abnormal feature.

[0143] For local anomaly features, this embodiment takes into account the imbalance between normal sample features and anomaly sample features, and therefore uses Focal Loss to calculate the loss of local anomaly features:

[0144]

[0145] in m i It is a mask for localized anomalous images.

[0146] The FDM technique mentioned also uses a discriminator and a loss function for training, so the final overall loss function is:

[0147] L total =L n +L gas +L las +L disc +L feat

[0148] During the experiment, the model was trained and its parameters were continuously adjusted to achieve the best results by minimizing the overall loss function.

[0149] Example 2

[0150] This embodiment provides a method for detecting defects in cigarette boxes based on an anomaly synthesis strategy, including the following steps:

[0151] After obtaining a normal cigarette box image, high-brightness patches and specular spots are randomly added to a local area of ​​the normal cigarette box image, and affine changes at the tilt angle are added to simulate the perspective distortion when the cigarette box is not perpendicular to the camera. The image obtained in the previous step is used as the cigarette box image to be detected.

[0152] The surface defect detection method described in Example 1 is used to detect defects in cigarette boxes.

[0153] The feature extraction network uses WideResNet50 as the backbone network. The CBAM attention mechanism is introduced into each residual block of WideResNet50, and a highlight suppression branch is added to the CBAM attention mechanism to shield the weights of reflective areas.

[0154] The steps for processing specular suppression branches are as follows:

[0155] The input features are used to obtain a brightness map, and areas with brightness values ​​greater than a brightness threshold are identified as bright areas.

[0156] The highlighted areas are converted into mask images, and then fused with the spatial attention map obtained by the spatial attention mechanism to obtain a spatial attention feature map after reflection shielding, thereby improving the robustness of the model in the disturbed areas.

[0157] Verification Example:

[0158] Due to the scarcity of defect samples in actual production environments, this embodiment transforms some normal cigarette box and cigarette images by artificially simulating weak defects (such as minor scratches and small stains), and uses an industrial line scan camera for high-precision imaging, thereby expanding the scale of the defect dataset. This data augmentation strategy provides a solid foundation for model training and validation.

[0159] Regarding data annotation, this embodiment employs a general annotation method that distinguishes only between normal and abnormal samples. This strategy better aligns with actual production needs, enabling the rapid and accurate removal of defective samples without requiring detailed classification of defect types. Furthermore, this annotation method reduces the workload of manual annotation, avoids errors introduced by ambiguous classifications, and improves the reliability of the dataset. It should be noted that this embodiment uses an unsupervised learning method for training; therefore, the training set only contains normal images. Detailed data information is shown in Table 1.

[0160] Table 1. Dataset Information.

[0161]

[0162] To comprehensively evaluate model performance, this embodiment employs the following evaluation metrics: image-level AUROC, pixel-level AUROC, and PRO (Per-Region-Overlap). These metrics assess the model's anomaly detection capabilities at the image and pixel levels, respectively. Before calculating AUROC, the model's anomaly confidence level e on the output must first be calculated. i and real label y i Calculate the True Positive Rate (TPR) and the False Positive Rate (FPR):

[0163]

[0164] Plot the ROC curve with FPR on the horizontal axis and TPR on the vertical axis, and calculate the area under the ROC curve to obtain the image-level AUROC. The pixel-level AUROC is calculated using the same method. The formula is as follows:

[0165]

[0166] The PRO metric is used to evaluate the overlap rate of a model when detecting local anomalies. First, it's necessary to obtain the pixel-level anomaly confidence score e from the model's output. i And real pixel-level label images M i Anomaly confidence plot e of the model output i Thresholding is performed to generate a binary prediction map. in Indicates an anomaly in the prediction. This indicates the prediction is normal. Calculate the true label image M. i The true anomaly region R in k and prediction chart Predicted anomaly areas Calculate the overlap rate:

[0167]

[0168] in Represents the number of pixels in the overlapping region, |R k | Represents the number of pixels in the actual anomaly region. The PRO value is obtained by averaging the overlap rates of all actual anomaly regions.

[0169]

[0170] Where K represents the total number of true anomaly regions. The closer the PRO value is to 1, the stronger the model's ability to detect local anomaly regions.

[0171] Experimental environment

[0172] This experiment was conducted on an Ubuntu system using CUDA version 12.2, PyTorch version 2.1.2, Python version 3.9, and an NVIDIA GeForce RTX 4090 GPU. The training epochs were set to 300, the batch size to 4, the learning rate to 0.001, and the image resolution to 512×512. To reduce the computational cost and parameter count, and thus the model's complexity, the second and third layers of Wideresnet50 were used as the backbone network. This design avoids redundant computations that may exist in deep networks, making the model more lightweight.

[0173] Comparative experiment

[0174] This embodiment compares the WideResNet50 base model with the improved model proposed in this embodiment, and plots the curve of the loss function changing with the number of training epochs, as shown below. Figure 3As shown in the figure, in the early stages of training, the loss function values ​​of both models decrease rapidly with the increase of the number of training epochs; while in the later stages of training, the decreasing trend of the loss function gradually flattens out. Finally, both models reach convergence around the 300th epoch, therefore, this embodiment sets the number of training epochs to 300. By comparison, it can be found that the model proposed in this embodiment exhibits a lower loss value during training, indicating that it has superior performance and stronger convergence ability.

[0175] To verify that the model proposed in this embodiment has good performance under the same conditions, this embodiment selected a variety of classic models and methods for comparative experiments, including: Autoencoder (AE), GAN, YOLOv5, YOLOv7, Mask R-CNN, Patchcore, and EfficientAD. The experimental results of each model are shown in Table 2.

[0176] As can be seen from the evaluation results in Table 2, the method proposed in this embodiment significantly outperforms the comparative methods in all three metrics: image-level AUROC, pixel-level AUROC, and PRO, especially in pixel-level AUROC and PRO. Although the detection speed of the method proposed in this embodiment is slightly slower than that of the YOLOv5 and YOLOv7 models, it still meets the actual production requirements. Considering all evaluation metrics—image-level AUROC, pixel-level AUROC, PRO, and detection speed—the method proposed in this embodiment has the best overall performance and can effectively meet the actual needs of cigarette factory production lines.

[0177] Table 2.Comparison experiment with other models.

[0178]

[0179] Figure 4 and Figure 5 The comparisons of inference speed (FPS) with AUROC and PRO are shown separately. It is clearly evident that the method proposed in this embodiment has significant advantages in inference speed and AUROC and PRO metrics.

[0180] ablation experiment

[0181] To verify the effectiveness of each improvement in the model of this embodiment, a series of ablation experiments were specifically designed to evaluate the contribution of each improvement. In the experiments, all comparison models used the same parameter settings and datasets. First, this embodiment conducted an experimental comparison of the selection of the number of layers in the backbone network WideResNet50, and the results are shown in Table 3. Shallow features typically contain more low-level information, such as edge textures and colors. While these features are sensitive to details, they lack semantic information and are difficult to capture complex global structures. Deeper features, while containing more high-level semantic information, lose a significant amount of detailed information. Therefore, in this experiment, this embodiment selected features from the intermediate layers 2 and 3. These features achieve a good balance between low-level and high-level semantic information, thus exhibiting optimal overall performance.

[0182] Table3.Perfoemance of Backbone Settings

[0183]

[0184] This embodiment also conducted ablation experiments on various improvements to the model, and the experimental results are shown in Table 4. As can be seen from the experimental data in the table, all improvements to the WidenesNet50 model in this embodiment significantly improved the model's detection performance. Removing any one of the modules—CBAM, FDM, or the anomaly synthesis strategy—resulted in varying degrees of performance degradation. Specifically, when the CBAM attention mechanism was introduced, the model focused more on key regions in the image, resulting in increases in the model's three metrics—Img_Auroc, Pix_Auroc, and PRO—by 2.9%, 3%, and 3.7%, respectively. Further introduction of global and local anomaly synthesis strategies further improved the model's performance by 5.1%, 5.4%, and 7.8% in these three metrics compared to the most basic model, indicating that the anomaly synthesis strategy effectively enhanced the model's ability to identify weak defects.

[0185] Finally, by introducing FDM feature distribution matching technology, the distribution difference between the training set and the test set was further reduced, improving the three indicators by 6.9%, 6.6%, and 9.4% respectively, significantly enhancing the overall performance of the model. Experimental results show that the method proposed in this embodiment performs excellently in the cigarette box defect detection task and has outstanding performance advantages among similar methods.

[0186] Table 4.Results of ablation experiments

[0187]

[0188] In summary, to improve industrial production efficiency and avoid missed or false detections of surface defects during manual inspection, this embodiment proposes a surface defect detection method based on anomaly synthesis strategies. It is the first to apply global and local anomaly synthesis strategies to target anomaly detection. By synergistically combining these two strategies, the model can generate more representative and diverse anomaly samples, significantly enhancing its ability to identify weak defects. By introducing the CBAM attention mechanism, the model can adaptively focus on key regions in the image, reducing background noise interference and improving the quality of feature representation. Finally, this embodiment introduces FDM feature distribution matching technology to reduce the distribution difference between the test set and the training set during the testing phase, significantly improving the model's generalization ability and robustness in real industrial environments. Ablation experiments show that each improvement proposed in this embodiment significantly improves the model's detection performance on image-level Aurio, pixel-level Aurio, and PRO metrics. Comparative experiments show that this model significantly outperforms existing classical methods, meets practical production needs, and exhibits optimal overall performance.

[0189] Furthermore, to visually demonstrate the performance of the method in this embodiment in the task of detecting defects in cigarette boxes, this embodiment is visually compared with the existing classic model Patchcore. Figure 6 The visualizations demonstrate the effectiveness of cigarette box defect detection. In each image, the three rows on the left show the results of cigarette box defect detection using the method proposed in this embodiment, while the three rows on the right show the results of cigarette box defect detection using the classic Patchcore model.

[0190] As seen in the visualized results, the method in this embodiment can effectively detect common abnormal defects in cigarette boxes, such as spots on the surface and folds in the packaging, with high detection accuracy and precise defect localization. The heatmap shows that the method clearly marks the defect areas, demonstrating strong anomaly localization capabilities. In contrast, while the PatchCore model's detection results are generally accurate, the detection area is not precise enough, and its ability to detect subtle defects such as small holes or spots on the cigarette box surface is weak. In summary, the method in this embodiment outperforms the PatchCore model in both anomaly detection accuracy and detail capture.

[0191] Example 3

[0192] This embodiment provides a method for detecting cigarette defects based on an anomaly synthesis strategy, including the following steps:

[0193] The cigarette box defect detection method described in Example 1 is used to detect cigarette defects.

[0194] In this process, after obtaining a normal cigarette image, edge detection and contour extraction methods are used to obtain the contour boundary of the cigarette. Within the obtained contour boundary, a region is randomly selected and local anomalies are superimposed. At the same time, affine transformation and local bending techniques are used to flexibly distort the abnormal region to obtain an abnormal cigarette image.

[0195] Specifically, within the obtained contour boundary, a region is randomly selected, and local anomalies are overlaid, including:

[0196] Randomly select a region within the extracted contour boundary, using the following formula:

[0197] anomaly_x=np.random.randint(x,x+w-anomaly_size)

[0198] anomaly_y=np.random.randint(y,y+h-anomaly_size)

[0199] anomaly_type == 'noise';

[0200] To overlay localized anomalies such as spots, scratches, and stains onto an image, the formula is:

[0201] image[anomaly_y:anomal_y+anomaly_size,anomaly_x:anomaly_x+anomaly_size]=anomaly.

[0202] When acquiring global anomaly features, Gaussian noise is injected in a non-uniform distribution manner to preferentially interfere with "slender regions", which is closer to the real defects.

[0203] Specifically, the Gaussian noise injection formula is as follows:

[0204] anomaly=np.random.randint(0,256,(anomaly_size,anomaly_size,3),dtype=np.uint8);

[0205] In the formula, np.random.randint() is the random generation function, 0 and 256 are the minimum and maximum pixel values, (anomaly_size,anomaly_size,3) is the selected image size, and dtype = np.uint8 is an unsigned integer data type.

[0206] Figure 7The visualizations demonstrate the effectiveness of cigarette defect detection. Each image shows the detection results of the proposed method on the left and the results of the Patchcore model on the right. The visualizations show that the proposed method effectively detects common cigarette defects such as wrinkles, spots, and breaks, and the heatmaps show precise and accurate detection locations. In contrast, the Patchcore model is less precise in its detection and exhibits false positives. In conclusion, the surface defect detection method based on anomaly synthesis proposed in this embodiment demonstrates significant advantages in anomaly detection accuracy and defect localization.

[0207] Example 4

[0208] This embodiment provides a surface defect detection device for objects based on an anomaly synthesis strategy, including:

[0209] The acquisition unit is used to acquire normal sample images to form the training set and the test set;

[0210] The local anomaly image generation module generates local anomaly images based on a local anomaly synthesis strategy and anomaly sample images.

[0211] The feature extraction network is used to extract features from normal sample images, images with local anomalies, and images of the target to be detected, respectively.

[0212] The global anomaly feature acquisition module is used to process normal image features based on Gaussian noise to obtain Gaussian anomaly features, and then perform gradient ascent and truncation on the Gaussian anomaly features to obtain global anomaly features.

[0213] The training module is used to train the discriminator based on the normal image features, global abnormal features and local abnormal features corresponding to the training set, and to train the FDM discriminator by aligning the feature distribution of the test set with the feature distribution of the training set.

[0214] The defect detection module is used to call the feature extraction network to extract features from the target image to be detected, adjust the feature distribution of the extracted features based on the FDM discriminator, and input the adjusted features into the trained discriminator to obtain the defect detection result of the cigarette box.

[0215] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them; although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can still be made to the specific implementation of the present invention or equivalent substitutions can be made to some technical features without departing from the spirit of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the scope of the technical solutions claimed in the present invention.

Claims

1. A method for detecting surface defects of an object based on an anomaly synthesis strategy, characterized in that, The following steps are involved: Obtain normal sample images to form the training and test sets; Feature extraction networks are used to extract features from normal sample images to obtain normal image features; Normal image features are processed based on Gaussian noise to obtain Gaussian anomalous features. Gradient ascent and truncation projection are then applied to the Gaussian anomalous features to obtain global anomalous features. Local anomaly images are generated based on a local anomaly synthesis strategy and anomaly sample images. Feature extraction networks are used to extract features from the local anomaly images to obtain local anomaly features. During the training phase, the discriminator is trained based on the normal image features, global anomaly features, and local anomaly features corresponding to the training set. During the testing phase, the FDM discriminator is trained by aligning the feature distribution of the test set with the feature distribution of the training set. In the detection phase, the target image to be detected is acquired, and features are extracted from the target image based on the trained feature extraction network. The extracted features are then adjusted based on the trained FDM discriminator, and the adjusted features are input into the trained discriminator to obtain the cigarette box defect detection result.

2. The method for detecting surface defects of an object based on an anomaly synthesis strategy according to claim 1, characterized in that, During the testing phase, the specific steps for training the FDM discriminator by aligning the feature distribution of the test set with the feature distribution of the training set include: The test samples of the test set are fed into the feature extraction network for feature extraction to obtain the test image features; A training sample is randomly selected from the training set and fed into the feature extraction network for feature extraction to obtain the training image features. The test image features and training image features are fed into the FDM discriminator to calculate the feature distribution difference. When the maximum mean difference between the training image features and the test image features is less than the MMD threshold, the test image features are fed into the discriminator for classification. When the maximum mean difference between the training image features and the test image features is greater than or equal to the MMD threshold, the test image features are adjusted so that the maximum mean difference between the training image features and the test image features is less than the MMD threshold. max The formula is adjusted as follows: in It is a hyperparameter used to control the mixing ratio of training image features and test image features; and These are the ascending sort values ​​of the training image features and the test image features, respectively, where n is the number of feature vector elements.

3. The method for detecting surface defects of an object based on an anomaly synthesis strategy according to claim 2, characterized in that, When training the FDM discriminator by aligning the feature distribution of the test set with the feature distribution of the training set, high-frequency statistical features are extracted and saved. When adjusting the feature distribution of the extracted features based on the FDM discriminator, the extracted features are linearly registered based on the saved high-frequency statistical features.

4. A method for detecting surface defects of an object based on an anomaly synthesis strategy according to claim 1, 2, or 3, characterized in that, The feature extraction network uses WideResNet50 as the backbone network. The channel attention module in the CBAM attention mechanism is introduced into each residual block of WideResNet50 to perform global average pooling and max pooling on the input features to obtain global average pooling features and global max pooling features. The channel attention weights are obtained by fusing global average pooling features and global max pooling features, and finally multiplying the channel attention weights with the original input features to output the result.

5. The method for detecting surface defects of an object based on an anomaly synthesis strategy according to claim 4, characterized in that, The image features of normal sample images are processed based on Gaussian noise to obtain Gaussian anomaly features. The specific steps for obtaining global anomaly features by performing gradient ascent and truncation projection processing on the Gaussian anomaly features are as follows: Adding Gaussian noise to normal image features yields Gaussian anomaly features: F noise =F+θθ~N(μ,σ 2 ) Gradient ascent is applied to Gaussian anomaly features to obtain global anomaly features. The gradient ascent direction is determined by the discriminator's loss function, and the gradient ascent step size is calculated using the following formula: Where ω is the learning rate, which controls the step size of gradient ascent; L gas The loss function of the discriminator; The truncation projection method is used to process global outliers, restricting them to a neighborhood of the normal sample distribution. The formula for truncation projection is as follows: F final =clip(F grad ,F-υ,F+υ) Where v is the set gradient threshold, clip() is the cutoff function, and F final This represents the final global anomaly features obtained.

6. A method for detecting surface defects of an object based on an anomaly synthesis strategy according to claim 1, 2, 3, or 5, characterized in that, The generation of local anomalous images based on a local anomalous synthesis strategy and anomalous sample images includes the following steps: A mask for local anomaly regions is generated using Burmester noise to simulate local defects on the object's surface; the formula for generating the mask M is: M(x,y)=PerNose(x,y,sclae,octaves,persistence,lacunarity) Where x and y are image coordinates, and scale, octaves, persistence, and lacunairerity are parameters of the Berlin noise, which control the detail and complexity of the noise. Add the mask to the normal sample image; Randomly select a texture T from the texture library and overlay it onto the mask to obtain a local anomaly image; The formula for texture overlay is as follows: Among them, I normal T is a normal sample image, and T is a texture image. This indicates element-wise multiplication, where α represents the transparency coefficient. The smaller α is, the less noticeable the local defects are.

7. A method for detecting surface defects of an object based on an anomaly synthesis strategy according to claim 2 or 3, characterized in that, The discriminator is a multilayer perceptron with a single hidden layer and a sigmoid activation function. It directly outputs the anomaly confidence score for each feature point and calculates the model loss function based on the anomaly confidence score. L total =L n +L gas +L las +L disc +L feat Among them, L n For normal feature loss function, Among them, f BCE It is a binary cross-entropy loss function. It is the discriminator's recognition of normal features F n The output of represents the confidence level of normal features, e i,n The closer the value is to 0, the closer the discriminator considers the feature to be a normal feature; i,n The closer a value is to 1, the closer the discriminator considers this feature to be an anomalous feature; L gas The global anomaly feature loss function is given by the following formula: in It is the discriminator's assessment of global anomaly features F gas The output of represents the confidence level of the anomalous feature, e i,gas The closer an e is to 0, the closer the discriminator considers the feature to be a normal feature; conversely, if e is closer to 0, the discriminator considers the feature to be closer to a normal feature. i,gas A value close to 1 indicates that the discriminator's prediction is relatively accurate, and the feature is considered to be closer to an abnormal feature. L las The loss function for local anomaly features is given by the following formula: in m i It is a mask for a localized anomalous image; The loss function of the FDM discriminator is: The loss function of the feature extractor is: Where D is the FDM discriminator, The expected value representing the features of the training image. λ·MMD(P,Q) represents the expected value of the test image features; λ·MMD(P,Q) is the regularization term of the loss function of the feature extraction network, where λ is a hyperparameter used to control the weight of the MMD regularization term. Where P and Q represent the feature distributions of the training set and the test set, respectively, and F i P and F j Q φ(·) is the feature representation of the training set and the test set, respectively; φ(·) is the sum function that maps the features to the high-dimensional feature space; and Γ is the reproducing kernel Hilbert space.

8. A method for detecting defects in cigarette boxes based on an anomaly synthesis strategy, characterized in that, The following steps are involved: After obtaining a normal cigarette box image, high-brightness patches and specular spots are randomly added to a local area of ​​the normal cigarette box image, and affine changes at the tilt angle are added to simulate the perspective distortion when the cigarette box is not perpendicular to the camera. The image obtained in the previous step is used as the cigarette box image to be detected. The method for detecting surface defects of an object as described in any one of claims 1-7 is used to detect defects in cigarette boxes. The feature extraction network uses WideResNet50 as the backbone network. The CBAM attention mechanism is introduced into each residual block of WideResNet50, and a highlight suppression branch is added to the CBAM attention mechanism to shield the weights of reflective areas. The steps for processing specular suppression branches are as follows: The input features are used to obtain a brightness map, and areas with brightness values ​​greater than a brightness threshold are identified as bright areas. The highlighted areas are converted into a mask image, and then fused with the spatial attention image obtained by the spatial attention mechanism to obtain a spatial attention feature map after reflection shielding.

9. A method for detecting cigarette defects based on an anomaly synthesis strategy, characterized in that, The following steps are involved: The method for detecting surface defects of an object as described in any one of claims 1-7 is used to detect defects in cigarettes. In this process, after obtaining a normal cigarette image, edge detection and contour extraction methods are used to obtain the contour boundary of the cigarette. Within the obtained contour boundary, a region is randomly selected and local anomalies are superimposed. At the same time, affine transformation and local bending techniques are used to flexibly distort the abnormal region to obtain an abnormal cigarette image. Furthermore, Gaussian noise is injected using a non-uniform distribution method when acquiring global anomaly features.

10. A surface defect detection device for objects based on an anomaly synthesis strategy, characterized in that, include: The acquisition unit is used to acquire normal sample images to form the training set and the test set; The local anomaly image generation module generates local anomaly images based on a local anomaly synthesis strategy and anomaly sample images. The feature extraction network is used to extract features from normal sample images, images with local anomalies, and images of the target to be detected, respectively. The global anomaly feature acquisition module is used to process normal image features based on Gaussian noise to obtain Gaussian anomaly features, and then perform gradient ascent and truncation projection processing on the Gaussian anomaly features to obtain global anomaly features. The training module is used to train the discriminator based on the normal image features, global abnormal features and local abnormal features corresponding to the training set, and to train the FDM discriminator by aligning the feature distribution of the test set with the feature distribution of the training set. The defect detection module is used to call the feature extraction network to extract features from the target image to be detected, adjust the feature distribution of the extracted features based on the FDM discriminator, and input the adjusted features into the trained discriminator to obtain the defect detection result of the cigarette box.

Images