Industrial product defect detection method and system based on masking strategy

Abstract

The invention discloses an industrial product defect detection method and system based on a masking strategy, and the method comprises the steps: randomly adding a plurality of masking blocks into each original image, and obtaining a first image set; according to a preset dynamic noise ratio, resetting the noise intensity of the masking area in the first image set to obtain a second image set; according to a preset dynamic weight loss function, respectively applying weights to the masking area and the non-masking area in the second image set to obtain a training image set; based on the training image set, training a preset model to obtain a trained anomaly detection model; and inputting a product data set into the anomaly detection model to obtain a product defect identification result. According to the method, the problem of sample imbalance is solved by adding the masking block and applying a larger weight, the quality of training data is improved, and the defect detection model with more optimized performance is obtained.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to an industrial product defect detection method and system based on a masking strategy. Background Art

[0002] Industrial surface defect detection is of great importance to industrial product production. It can not only help enterprises improve product quality to meet the growing needs of consumers, but also help enterprises locate problems in a timely manner and reduce production costs. Because of human subjective consciousness and long-term eye fatigue, it is easy to cause product misdetection and missed detection. Therefore, relying solely on manpower to complete visual inspection cannot guarantee efficient industrial production requirements. It is necessary to use computer vision technology to realize intelligent detection of industrial product appearance defects.

[0003] Research on solving the problem of industrial defect visual inspection by using traditional image processing technology can be divided into two categories. One is to extract pixel-level or structure-level image features through manually designed feature extractors, and then input them into traditional classifiers for defect recognition. However, if the extracted image features are not accurate enough or the data is unbalanced, the defect recognition based on them will inevitably be inaccurate. The other is to calculate the feature difference between the sample to be tested and the normal template through template matching, so as to determine whether the sample to be tested has defects. However, the limitations of template selection and matching algorithms often affect the performance of the model and make it impossible to identify unknown defects. Summary of the invention

[0004] This application proposes an industrial product defect detection method and system based on a masking strategy, which solves the sample imbalance problem by adding masking blocks and applying greater weights, improves the quality of training data, and obtains a defect detection model with more optimized performance.

[0005] The first aspect of the present application provides an industrial product defect detection method based on a masking strategy, the method comprising:

[0006] Randomly adding a number of masking blocks to each original image to obtain a first image set; wherein the masking blocks exist in the first image set in the form of masking regions;

[0007] Resetting the noise intensity of the masked area in the first image set according to a preset dynamic noise ratio to obtain a second image set;

[0008] According to a preset dynamic weight loss function, weights are applied to the masked area and the non-masked area in the second image set respectively to obtain a training image set;

[0009] Based on the training image set, the preset model is trained to obtain a trained anomaly detection model;

[0010] The product data set is input into the anomaly detection model to obtain product defect recognition results.

[0011] In the above scheme, when training the model, there is generally an imbalance of samples because there are fewer defective samples. The trained model has low performance due to overfitting of the training results. Therefore, in order to solve the problem of fewer defective samples, different numbers of masking blocks of set shapes and sizes are randomly added to each original image to increase the number of defective samples. And by adjusting the noise intensity of the masked area corresponding to the masking block, the model will pay more attention to the masked area in the future, improve the sensitivity of the model to key features, and further reduce the negative impact of sample imbalance. Then, different weights are applied to the masked area and the non-masked area respectively, so that the model can accurately and quickly identify the defective area of ​​the sample during the training process, and can pay more attention to the repair and reconstruction of the defective area, so as to obtain a higher performance anomaly detection model that can accurately identify the defects of industrial products.

[0012] In a possible implementation method of the first aspect, a number of mask blocks are randomly added to each original image to obtain a first image set, specifically:

[0013] Assigning values ​​to pixels corresponding to a preset masking block to obtain a binary mask of the masking block;

[0014] Randomly combining the binary mask with each original image to obtain the original image with the mask block added;

[0015] A first image set is obtained according to the original image to which the mask block is added.

[0016] The above scheme introduces the pixel-level masking block into the original image by randomly combining the binary mask of the masking block with the mask matrix of each original image, thereby increasing the proportion of defective samples in the original image and alleviating the problem of sample imbalance.

[0017] In a possible implementation method of the first aspect, adding the original image of the mask block is specifically:

[0018]

[0019] In the formula, is the binary mask of the original image to which the masking block is to be added, is the i-th original image, The i-th original image with the mask block added.

[0020] In a possible implementation method of the first aspect, according to a preset dynamic noise ratio, the noise intensity of the masked area in the first image set is reset to obtain the second image set, specifically:

[0021] Obtaining a mask matrix corresponding to the first image set;

[0022] Based on the dynamic noise ratio, the noise intensity of the masked area is adjusted by the original noise intensity of the mask matrix to obtain a reconstructed image set;

[0023] The reconstructed image set is denoised according to a preset denoising function to obtain a second image set.

[0024] The above scheme first obtains a mask matrix with a masking block, and then reduces the noise intensity of the masked area in the mask matrix according to the dynamic noise ratio, so that the masked area is easier to pay attention to during model training, thereby improving the sensitivity of the model to the defective area.

[0025] In a possible implementation method of the first aspect, adjusting the noise intensity of the masked area is specifically:

[0026]

[0027] Where η i is the noise intensity of the mask matrix after adjustment, σ t is the dynamic noise ratio, t is the current step number, T is the total trajectory step number, M i is the mask of the i-th pixel in the first image set, M i When it is 0, it represents the mask of the masked area, M i When it is 1, it represents the mask of the non-masked area.

[0028] In a possible implementation method of the first aspect, according to a preset dynamic weight loss function, weights are applied to the masked area and the non-masked area in the second image set respectively to obtain a training image set, specifically:

[0029] According to the dynamic weight loss function, weighting the mask of the masked area by a first weight and weighting the mask of the non-masked area by a second weight to complete the reconstruction of the second image set to obtain a training image set;

[0030] The first weight is greater than the second weight.

[0031] In order to distinguish different areas in the image samples and improve the model's attention to the key areas of the image, namely the defective areas, the above scheme applies different weights to the masked areas and the non-masked areas through the dynamic weight loss function. The masked areas are given a larger weight, which allows the model to pay more attention to these marked important areas during the training process, thereby improving the model's recognition accuracy of defective areas.

[0032] In a possible implementation method of the first aspect, the dynamic weight loss function is specifically:

[0033]

[0034] Where L is the dynamic weight loss function, N is the total number of samples in the second image set, and M i is the mask of the i-th pixel in the second image set, w m is the weight corresponding to the mask, X 0i is the i-th sample in the second image set, X after reconstruction 0i .

[0035] In a possible implementation method of the first aspect, the method further includes:

[0036] According to the acquired model accuracy requirement, the shape and size of the mask block and the number of the mask blocks on each original image are set.

[0037] The above scheme adjusts the accuracy of the model by adjusting the shape, size and number of masking blocks in the image. By simulating different types of defect areas through masking blocks, the model can more accurately identify various defect types and improve the generalization ability of the model in different application scenarios.

[0038] The second aspect of the present application provides an industrial product defect detection system based on a masking strategy, the system comprising: a masking block adding module, a noise intensity adjustment module, an image reconstruction module, a model training module and a defect detection module;

[0039] The mask block adding module is used to randomly add a number of mask blocks to each original image to obtain a first image set; wherein the mask blocks exist in the first image set in the form of mask areas;

[0040] The noise intensity adjustment module is used to reset the noise intensity of the masked area in the first image set according to a preset dynamic noise ratio to obtain a second image set;

[0041] The image reconstruction module is used to apply weights to the masked area and the non-masked area in the second image set according to a preset dynamic weight loss function to obtain a training image set;

[0042] The model training module is used to train the preset model based on the training image set to obtain a trained anomaly detection model;

[0043] The defect detection module is used to input the product data set into the anomaly detection model to obtain product defect identification results.

[0044] A fourth aspect of the present application provides a storage medium, which stores computer-readable program code. When the computer-readable program code is executed, the steps of an industrial product defect detection method based on a masking strategy as described in any one of the embodiments of the present application are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the implementation manner will be briefly introduced below. Obviously, the drawings described below are only some implementation manners of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 This is a specific flow chart of an industrial product defect detection method based on a masking strategy provided in a certain embodiment of the present application;

[0047] Figure 2 It is a flowchart of adding a masking block of an industrial product defect detection method based on a masking strategy provided by a certain embodiment of the present application;

[0048] Figure 3 It is a defect image reconstruction flow chart of an industrial product defect detection method based on a masking strategy provided in a certain embodiment of the present application;

[0049] Figure 4 This is a visualization comparison result diagram of an industrial product defect detection method based on a masking strategy provided in a certain embodiment of the present application;

[0050] Figure 5 It is a masking block size influence diagram of an industrial product defect detection method based on a masking strategy provided in a certain embodiment of the present application;

[0051] Figure 6 It is a masking block quantity influence diagram of an industrial product defect detection method based on a masking strategy provided in a certain embodiment of the present application;

[0052] Figure 7 It is a specific structural diagram of an industrial product defect detection system based on a masking strategy provided in a certain embodiment of the present application. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0054] It should be understood that the step numbers used in this article are only for the convenience of description and are not intended to limit the order in which the steps are executed.

[0055] First embodiment

[0056] In the industrial production process, the probability of product defects is low, resulting in serious imbalance in sample data. Small samples and unbalanced data can easily lead to overfitting of the target detection model, making it impossible to effectively identify defective samples. Moreover, a wide variety of industrial defects are prone to occur in complex working environments, and the same type of defects also have multi-scale differences, which requires the detection model to have stronger generalization capabilities and be able to cope with the randomness caused by the type, location, size and severity of industrial defects. In response to the above problems, the embodiments of the present application introduce masking blocks and weight assignments to design a higher-performance anomaly detection model for industrial products, thereby solving the problems of sample imbalance and difficulty in detecting unknown defects in existing defect detection technologies.

[0057] like Figure 1 As shown, Figure 1 A specific flow chart of an industrial product defect detection method based on a masking strategy is provided for a certain embodiment of the present application. The industrial product defect detection method based on a masking strategy of this embodiment includes steps S1 to S5, which are described in detail as follows:

[0058] Step S1, randomly adding several mask blocks to each original image.

[0059] In an embodiment of the present application, a normal original image is first converted into an image with artificial occlusion, mainly by randomly adding a number of masking blocks to each original image so that there are masked areas and non-masked areas in the original image, thereby increasing the proportion of defective samples in all original images and preliminarily solving the problem of sample imbalance.

[0060] Specifically, the IPM method is used to randomly add several masking blocks to each original image, where the full name of the IPM method is Image Patch Masking. The specific process is: obtain several masking blocks and corresponding binary masks, and then randomly assign pixels in the mask matrix of each original image, assigning pixels that introduce masking blocks to 0, and pixels that do not introduce masking blocks to 1, to obtain the original image to which the masking blocks are to be added. Then, the original image to which the masking blocks are to be added is combined with the masking blocks using element-by-element multiplication to obtain the original image to which the masking blocks are to be added, specifically:

[0061]

[0062] In the formula, is the binary mask of the original image to which the masking block is to be added, is the i-th original image, is the i-th original image with the mask block added, and ⊙ is the Hadamard product.

[0063] In addition, when assigning values ​​to the mask matrix of the original image, the area of ​​the masked block is first determined, and then the pixels of the corresponding area in the original image are assigned values ​​according to the area of ​​the masked block.

[0064] Finally, a first image set is obtained according to the original image to which the mask block is added.

[0065] To better demonstrate how to add a masking block, Figure 2 A flowchart for adding mask blocks is provided. As shown in the figure, the upper left corner is an original image containing a chestnut. After being processed by the IPM module, 5 mask blocks are introduced into the original image, and the original image X'0 containing 5 mask blocks is obtained.

[0066] Figure 3 It is a defective image reconstruction flowchart, which shows how to repair a picture with visual defects and transform it into a picture without visual defects. Figure 2 and Figure 3 The addition of mask blocks and image reconstruction are both implemented based on the mDDAD model, showing the basic framework of the mDDAD model.

[0067] Optionally, the embodiment of the present application selects a DDAD model, i.e., a denoising diffusion anomaly detection model, as the basis for the anomaly detection model subsequently constructed. The model can identify abnormal areas by adding noise to the image and then reconstructing the image through a denoising process. In addition, the embodiment of the present application also constructs an mDDAD module based on the DDAD model, constructs an mDDAD model, and uses the MVTec-AD public dataset as the original image set.

[0068] Optionally, in some embodiments, the VisA public dataset is also selected as the original dataset.

[0069] Figure 4A visualization comparison result diagram of the DDAD model and the mDDAD model is provided. Defect detection is performed on six different images to demonstrate the accuracy of the two different models in noise prediction. The first two columns in the figure are the input images of the DDAD model and the mDDAD model, as well as the images with GT deep labels (the yellow part is the noise of the input image). The third, fourth and fifth columns are the reconstructed images, noise prediction results and heat maps of the DDAD model. Each row of the first five columns represents the display diagram of the input data at each processing stage in the DDAD model. Comparing the second column with the fourth column, it can be found that the noise prediction results of the DDAD model are quite different from the actual ones. The last three columns in the figure are the reconstructed images, noise prediction results and heat maps of the mDDAD model. Comparing the data in the second column with the data in the seventh column, it can be found that the noise prediction results of the mDDAD model are close to the actual ones, which can show that the detection accuracy of the mDDAD model is higher than that of the DDAD model.

[0070] Furthermore, because the shape, size and number of masking blocks in each original image will affect the performance of the defect detection model, before introducing the masking blocks into the original image, it is necessary to set the shape, size and number of masking blocks on each original image according to the model accuracy requirements.

[0071] Exemplarily, the embodiments of the present application select various masking block shapes including rectangles, circles and polygons. Different shapes can simulate different types of defect areas to better understand the impact of different masking block shapes on the detection accuracy of various defect types in the MVTec-AD public data set.

[0072] The following data descriptions provide insights into specific masked block categories:

[0073] 1. "Capsule", "Pill", "Carpet" and "Leather": These categories mainly show hole defects, so using this circular masking block can more accurately measure hole defects first;

[0074] 2. "Metal_nut", "Till" and "Wood": These categories mainly show crack-type defects, so using this polygonal mask can more accurately detect crack-type defects;

[0075] 3. Hybrid Shape: Combining the advantages of circular and polygonal masking blocks, this method performs well in all categories of defect detection.

[0076] The following table shows the AUROC of images with different shapes of mask blocks in the MVTee-AD public dataset. The bold data is the highest model accuracy value corresponding to each different shape of mask block:

[0077] Table 1 The influence of the shape of the masking block on the AUROC of the model

[0078]

[0079]

[0080] AUROC stands for the area under the receiver characteristic curve, which is used to evaluate the performance of anomaly detection models. In the embodiment of the present application, it is used to evaluate the performance of anomaly detection models based on mDDAD. The higher the AUROC, the better the model performance.

[0081] Table 1 shows that different shapes of masks have a significant impact on the model performance of various categories in the MVTec-AD dataset. It is obvious that for the categories of "capsule", "pill", "carpet", and "leather" with hole-type defects, the accuracy of the circular mask is significantly better than the square and polygonal masks. Especially for the leather category, the image AUROC reaches 100.0%. In contrast, for the categories of metal nuts, tiles, and wood with crack-type defects, the accuracy of the polygonal mask is significantly better than the circular and square masks. Therefore, it is found that the circular mask performs best in detecting hole-type defects, while the polygonal mask performs well in detecting crack-type defects. The mixed-shape masks combine the advantages of each shape and perform well in all categories. Their average performance reaches 97.3%, which is significantly better than the masks of other shapes. These findings provide valuable guidance for selecting and designing mask shapes and improve the performance of models in detecting different types of defects.

[0082] In addition, because the size of the masking block will also directly affect the model's ability to identify and reconstruct defect areas, the embodiment of the present application also sets the size of the masking block to 10×10, 50×50, 100×100, etc. Table 2 below shows the changes in model accuracy for the "toothbrush" and "hazelnut" categories under masking blocks of different sizes, where "toothbrush" represents a smaller type of defect and "hazelnut" represents a larger type of defect. Mask Size represents the size of the masking block, which are 10×10, 50×50, 100×100, 150×150, 200×200 and 250×250, respectively, gradually increasing from left to right. The bold data are the highest values ​​of model accuracy corresponding to masking blocks of two different sizes.

[0083] Table 2 Effect of masking block size on model AUROC

[0084]

[0085] To better illustrate the impact of masking block size on model performance, Figure 5A masking block size influence diagram is also provided to display the data in Table 2, where the blue line represents the toothbrush, the orange line represents the hazelnut, the horizontal axis is the size of the masking block, and the vertical axis is the image AUROC.

[0086] The results show that the size of the masking block significantly affects the model's ability to identify defects and reconstruct defect areas, specifically:

[0087] 1. Smaller masking blocks help the model learn finer defect features, but are limited when dealing with larger defects;

[0088] 2. Larger masking blocks can better simulate large-area defects, but may cause the model to ignore small defect details.

[0089] Table 2 and Figure 5 It clearly illustrates the performance of the model for two sample categories (“toothbrush” and “hazelnut”) at different mask sizes. Figure 5 In the figure, for the “hazelnut” class representing larger defects, the image AUROC increases with the increase of the masking block size, rising from 98.2% for the 10×10 size to 99.0% for the 200×200 size. When the masking block size continues to increase, the image AUROC drops sharply. In contrast, for the “toothbrush” class representing small defects, the best performance is achieved using the 10×10 size, reaching 100.0%. As the masking block size increases, the image AUROC begins to steadily decrease. Therefore, it is found that small masking blocks perform better in identifying small defects, while large masking blocks perform well in handling large area defects. By combining different masking block sizes, the overall performance of the model is significantly improved. In summary, choosing a suitable combination of masking block sizes is crucial to enhancing the defect detection and reconstruction capabilities of the model.

[0090] During the model training process, the number of masking blocks determines the density of the defect area, so the embodiment of the present application also tests the impact of different numbers of masking blocks in each original image on the model's ability to detect actual defects.

[0091] Optionally, in the embodiment of the present application, 3, 5 or 7 masking blocks are added to each original image.

[0092] Table 3 shows the impact of the number of masking blocks of different shapes on the model detection accuracy. Each column shows the accuracy of the model under different numbers of masking blocks. The number of masking blocks increases from 1 to 10. The bold data is the highest model accuracy corresponding to each shape of masking block, specifically:

[0093] Table 3 Effect of the number of masked blocks on the AUROC of the model

[0094]

[0095] The present application also provides Figure 6 The relationship between the number of masked blocks and the image AUROC in Table 3 is more intuitively shown in a line graph, reflecting the trend of the model's accuracy. As can be seen from the graph and table, moderately increasing the number of masked blocks can enhance the model's sensitivity to defects; however, too many masked blocks may cause the model to overfit the masked area, thereby reducing its ability to detect actual defects.

[0096] In general, for different sample categories, there is a balance point in the number of masked blocks. Before this balance point, the model accuracy generally increases, but after exceeding the balance point, the accuracy curve begins to decline. Therefore, finding the balance point in the number of masked blocks is crucial to improving the performance of the model.

[0097] In summary, after obtaining the first image set including the masked block, the masked block is saved in the first image set in the form of a masked area, and the first image set is input into a preset mDDAD model for noise addition.

[0098] Step S2: reset the noise intensity of the masked area in the first image set according to a preset dynamic noise ratio to obtain a second image set.

[0099] In the embodiment of the present application, an innovative mask-based dynamic noise adjustment mechanism is introduced to accurately control the noise intensity applied to the masked area in the image. This mechanism adjusts the addition of noise by combining mask information and noise level to optimize the reconstruction quality of the model for a specific area.

[0100] First, define the dynamic noise ratio, the specific expression is:

[0101]

[0102] In the formula, σ t is the dynamic noise ratio, t is the current step number, and T is the total trajectory step number. The dynamic noise ratio changes linearly with the change of t within the total trajectory step number T, and T is usually set to trajectory_steps in the model configuration.

[0103] Next, obtain the mask matrix M corresponding to the first image set, and set M i is the mask of the i-th pixel in the first image set, M i ∈{0,1},M i When it is 0, it represents the mask of the masked area in the first image set, M i When it is 1, it represents the mask of the non-masked area in the first image set.

[0104] Then, based on the dynamic noise ratio, the noise intensity of the masked area is adjusted, and the noise intensity of the non-masked area remains unchanged, to obtain the adjusted noise intensity of the mask matrix, and to obtain a reconstructed image set, specifically:

[0105]

[0106] Where η i is the noise intensity of the mask matrix after adjustment.

[0107] In the embodiment of the present application, the noise intensity of the masked area is set to 10% of the original noise intensity.

[0108] Finally, according to the preset noise adding function, the reconstructed image set is denoised by the adjusted noise intensity, that is, the reconstructed image set is resampled to generate the second image set. The specific expression is:

[0109]

[0110] In the formula, α t-1 is the attenuation coefficient, f θ (x t ) is the feature extraction function learned from the mDDAD model, ε θ (x t ) is the noise predicted by the mDDAD model at the current step number t, η i is the noise intensity obtained according to the dynamic noise ratio, which is in the masked area (M i =0) to reduce the noise impact in these areas, and in the non-masked area (M i =1) to keep the original intensity. In this way, the model will adjust the noise accordingly according to the mask matrix and the current step number t at each time step, so that the model can pay more attention to specific areas.

[0111] Step S3: according to a preset dynamic weight loss function, weights are applied to the masked area and the non-masked area in the second image set respectively to obtain a training image set.

[0112] In the embodiments of the present application, by introducing mask weights and mask-based dynamic noise adjustment strategies into the loss function, the model's attention to and processing capabilities of key image areas are greatly enhanced.

[0113] Specifically, traditional anomaly detection models usually do not distinguish between different areas in an image, which may cause the model to pay insufficient attention to key areas of the image (such as areas that may contain anomalies or defects) during training. To solve this problem, the present application embodiment designs a new loss function, namely a dynamic weight loss function, which applies greater reconstruction pressure to image areas with masked blocks. Compared with non-masked areas, higher weights are applied to masked areas, allowing the model to pay more attention to these areas that are artificially marked as important during training.

[0114] According to the dynamic weight loss function, the mask of the masked area is weighted by a first weight, and the mask of the non-masked area is weighted by a second weight, so as to complete the reconstruction of the second image set and obtain a training image set, wherein the first weight is greater than the second weight.

[0115] The dynamic weight loss function is specifically:

[0116]

[0117] Where L is the dynamic weight loss function, N is the total number of samples in the second image set, and M i is the mask of the i-th pixel in the second image set, w m is the weight corresponding to the mask, X 0i is the i-th sample in the second image set, X after reconstruction 0i .

[0118] The dynamic weight loss function weights the square error of each pixel according to its weight in the mask, which enhances the reconstruction accuracy of the model in the masked area, thereby more effectively learning and optimizing the key areas. m , the strength of the model's reconstruction of the masked area can be flexibly controlled, thereby optimizing the performance of the model in specific application scenarios.

[0119] Step S4: Based on the training image set, the preset model is trained to obtain a trained anomaly detection model.

[0120] In an embodiment of the present application, the mDDAD model is trained based on a training image set containing a masked area to obtain a high-precision anomaly detection model.

[0121] In order to demonstrate the excellent performance of the anomaly detection model, the following Table 4 provides a comparison of the AUROC index of the mDDAD model and the prior art on the MVTec-AD dataset:

[0122] Table 4 AUROC index comparison between mDDAD model and existing technology

[0123] Task GANomaly AE SkipG DRAEM DDAD mxD Cable 45.7 56.0 47.7 98.1 98.7 98.4 Tile 40.8 77.0 74.0 99.2 100.0 100.0 Leather 41.3 54.0 47.7 98.5 99.1 99.8 Hazelnut 14.4 53.0 56.0 98.4 99.5 98.2 bottle 25.1 80.0 85.2 95.2 96.6 100.0 Wood 74.4 74.0 19.9 95.1 94.7 99.4 Transistor 44.0 71.0 84.5 93.1 99.6 99.8 Grid 40.4 78.0 77.0 100.0 100.0 99.7 Carpet 20.3 50.0 70.9 97.0 99.4 92.3 Pill 18.0 88.5 78.9 92.9 92.3 93.7 Metal_nut 27.7 92.6 84.5 97.7 97.2 97.8 Capsule 68.2 62.0 54.3 98.1 97.0 97.0 screw 23.1 76.0 94.2 97.7 97.1 98.5 toothbrush 70.0 98.7 98.7 100.0 99.7 100.0 zipper 43.4 93.6 92.7 99.9 99.8 100.0

[0124] In Table 4, the vertical column represented by Task is masking blocks of different shapes, the second to fifth columns are the AUROC indicators of the GANomaly model, AE model, SkipG model, DREAM model and DDAD model in the prior art after training based on data containing masking blocks of different shapes, and the last column is the AUROC indicator of the embodiment of the present application after training with the mDDAD model; it can be seen from the table that when detecting defect areas of most shapes, the mDDAD model has a higher AUROC indicator than other models, indicating that the mDDAD model has higher detection accuracy than other models as a whole.

[0125] Step S5, inputting the product data set into the anomaly detection model to obtain a product defect recognition result.

[0126] Implementing the embodiments of the present application has the following beneficial effects:

[0127] The embodiment of the present application takes into account that in the general model training process, there is a sample imbalance due to the small number of defective samples, and the trained model has low performance due to overfitting of the training results. Therefore, in order to solve the problem of small number of defective samples, different numbers of masking blocks of set shapes and sizes are randomly added to each original image to increase the number of defective samples. And by adjusting the noise intensity of the masked area corresponding to the masking block, the noise intensity of the non-masked area is kept unchanged, so that the model will pay more attention to the masked area in the future, improve the sensitivity of the model to key features, and further reduce the negative impact of sample imbalance. Then, different weights are applied to the masks of the masked area and the non-masked area respectively, so that the model can accurately and quickly identify the defective area of ​​the sample during the training process due to the different mask sizes, and can pay more attention to the repair and reconstruction of the defective area, so as to obtain a higher performance anomaly detection model that can accurately identify the defects of industrial products.

[0128] Second embodiment

[0129] Furthermore, in order to implement the industrial product defect detection system based on the masking strategy corresponding to the above method embodiment to achieve the corresponding functions and technical effects, Figure 7 A structural diagram of an industrial product defect detection system based on a masking strategy is provided. For ease of description, only the parts related to this embodiment are shown. The industrial product defect detection system based on a masking strategy provided by the embodiment of the present application includes:

[0130] The mask block adding module 201 is used to randomly add a number of mask blocks to each original image to obtain a first image set; wherein the mask blocks exist in the first image set in the form of mask areas.

[0131] In the embodiment of the present application, the pixels corresponding to the preset masking block are assigned values ​​to obtain a binary mask of the masking block;

[0132] Randomly combining the binary mask with each original image to obtain the original image with the mask block added;

[0133] A first image set is obtained according to the original image to which the mask block is added.

[0134] The noise intensity adjustment module 202 is used to reset the noise intensity of the masked area in the first image set according to a preset dynamic noise ratio to obtain a second image set.

[0135] In an embodiment of the present application, a mask matrix corresponding to the first image set is obtained;

[0136] Based on the dynamic noise ratio, the noise intensity of the masked area is adjusted by the original noise intensity of the mask matrix to obtain a reconstructed image set;

[0137] The reconstructed image set is denoised according to a preset denoising function to obtain a second image set.

[0138] The image reconstruction module 203 is used to apply weights to the masked area and the non-masked area in the second image set according to a preset dynamic weight loss function to obtain a training image set.

[0139] In an embodiment of the present application, according to the dynamic weight loss function, the mask of the masked area is weighted by a first weight, and the mask of the non-masked area is weighted by a second weight, so as to complete the reconstruction of the second image set and obtain a training image set;

[0140] The first weight is greater than the second weight.

[0141] The model training module 204 is used to train a preset model based on a training image set to obtain a trained anomaly detection model.

[0142] In an embodiment of the present application, the mDDAD model is trained based on a training image set containing a masked area to obtain a high-precision anomaly detection model.

[0143] The defect detection module 205 is used to input the product data set into the anomaly detection model to obtain a product defect identification result.

[0144] In some embodiments, the mask block adding module 201 further includes:

[0145] In an embodiment of the present application, a normal original image is first converted into an image with artificial occlusion, mainly by randomly adding a number of masking blocks to each original image so that there are masked areas and non-masked areas in the original image, thereby increasing the proportion of defective samples in all original images and preliminarily solving the problem of sample imbalance.

[0146] Specifically, the IPM (Image Patch Masking) method is used to randomly add several masking blocks to each original image. Specifically, several masking blocks and corresponding binary masks are obtained, and then pixels are randomly assigned in the mask matrix of each original image, and the pixels that introduce the masking block are assigned to 0, and the pixels that do not introduce the masking block are assigned to 1, so as to obtain the original image to which the masking block is to be added. Then, the original image to which the masking block is to be added is combined with the masking block using element-by-element multiplication to obtain the original image to which the masking block is added, specifically:

[0147]

[0148] In the formula, is the binary mask of the original image to which the masking block is to be added, is the i-th original image, is the i-th original image with the mask block added, and ⊙ is the Hadamard product.

[0149] In addition, when assigning values ​​to the mask matrix of the original image, the area of ​​the masked block is first determined, and then the pixels of the corresponding area in the original image are assigned values ​​according to the area of ​​the masked block.

[0150] Finally, a first image set is obtained according to the original image to which the mask block is added.

[0151] Optionally, the embodiment of the present application selects a DDAD model, i.e., a denoising diffusion anomaly detection model, as the basis for the anomaly detection model subsequently constructed. The model can identify abnormal areas by adding noise to the image and then reconstructing the image through a denoising process. In addition, the embodiment of the present application also constructs an mDDAD module based on the DDAD model, constructs an mDDAD model, and uses the MVTec-AD public dataset as the original image set.

[0152] Optionally, in some embodiments, the VisA public dataset is also selected as the original dataset.

[0153] Furthermore, because the shape, size and number of masking blocks in each original image will affect the performance of the defect detection model, before introducing the masking blocks into the original image, it is necessary to set the shape, size and number of masking blocks on each original image according to the model accuracy requirements.

[0154] Exemplarily, the embodiments of the present application select various masking block shapes including rectangles, circles and polygons. Different shapes can simulate different types of defect areas to better understand the impact of different masking block shapes on the detection accuracy of various defect types in the MVTec-AD public data set.

[0155] The following data descriptions provide insights into specific masked block categories:

[0156] 4. "Capsule", "Pill", "Carpet" and "Leather": These categories mainly show hole defects, so using this circular masking block can more accurately measure hole defects first;

[0157] 5. "Metal_nut", "Till" and "Wood": These categories mainly show crack-type defects, so using this polygonal mask can more accurately detect crack-type defects first;

[0158] Hybrid Shape: Combining the strengths of circular and polygonal masking blocks, this approach performs well in all categories of defect detection.

[0159] In addition, AUROC is used to evaluate the performance of the anomaly detection model. AUROC stands for the area under the receiver characteristic curve, which is used to evaluate the performance of the anomaly detection model. In the embodiment of the present application, it is used to evaluate the performance of the anomaly detection model based on mDDAD. The higher the AUROC, the better the model performance.

[0160] In addition, because the size of the masking block will also directly affect the model's ability to identify and reconstruct defect areas, the embodiment of the present application also sets the size of the masking block to 10×10, 50×50, 100×100, etc. The size of the masking block significantly affects the model's ability to identify defects and reconstruct defect areas, specifically:

[0161] 1. Smaller masking blocks help the model learn finer defect features, but are limited when dealing with larger defects;

[0162] 2. Larger masking blocks can better simulate large-area defects, but may cause the model to ignore small defect details.

[0163] In summary, small masking blocks perform better in identifying small defects, while large masking blocks perform well in dealing with large area defects. By combining different masking block sizes, the overall performance of the model is significantly improved. In conclusion, choosing a suitable combination of masking block sizes is crucial to enhancing the defect detection and reconstruction capabilities of the model.

[0164] In addition, during the model training process, the number of masking blocks determines the density of the defect area, so the embodiment of the present application also tests the impact of different numbers of masking blocks in each original image on the model's ability to detect actual defects.

[0165] Optionally, in the embodiment of the present application, 3, 5 or 7 masking blocks are added to each original image.

[0166] Specifically, for different sample categories, there is a balance point in the number of masked blocks. Before this balance point, the model accuracy generally increases, but after exceeding the balance point, the accuracy curve begins to decline. Therefore, finding the balance point in the number of masked blocks is crucial to improving the performance of the model.

[0167] In summary, after obtaining the first image set including the masked block, the masked block is saved in the first image set in the form of a masked area, and the first image set is input into a preset mDDAD model for noise addition.

[0168] In some embodiments, the noise intensity adjustment module 202 further includes:

[0169] In the embodiment of the present application, an innovative mask-based dynamic noise adjustment mechanism is introduced to accurately control the noise intensity applied to the masked area in the image. This mechanism adjusts the addition of noise by combining mask information and noise level to optimize the reconstruction quality of the model for a specific area.

[0170] First, define the dynamic noise ratio, the specific expression is:

[0171]

[0172] In the formula, σ t is the dynamic noise ratio, t is the current step number, and T is the total trajectory step number. The dynamic noise ratio changes linearly with the change of t within the total trajectory step number T, and T is usually set to trajectory_steps in the model configuration.

[0173] Next, obtain the mask matrix M corresponding to the first image set, and set M i is the mask of the i-th pixel in the first image set, M i ∈{0,1},M i When it is 0, it represents the mask of the masked area in the first image set, M i When it is 1, it represents the mask of the non-masked area in the first image set.

[0174] Then, based on the dynamic noise ratio, the noise intensity of the masked area is adjusted, and the noise intensity of the non-masked area remains unchanged, to obtain the adjusted noise intensity of the mask matrix, and to obtain a reconstructed image set, specifically:

[0175]

[0176] Where η i is the noise intensity of the mask matrix after adjustment.

[0177] In the embodiment of the present application, the noise intensity of the masked area is set to 10% of the original noise intensity.

[0178] Finally, according to the preset noise adding function, the reconstructed image set is denoised by the adjusted noise intensity, that is, the reconstructed image set is resampled to generate the second image set. The specific expression is:

[0179]

[0180] In the formula, α t-1 is the attenuation coefficient, f θ (x t ) is the feature extraction function learned from the mDDAD model, ε θ (x t ) is the noise predicted by the mDDAD model at the current step number t, η i is the noise intensity obtained according to the dynamic noise ratio, which is in the masked area (M i =0) to reduce the noise impact in these areas, and in the non-masked area (M i =1) to keep the original intensity. In this way, the model will adjust the noise accordingly according to the mask matrix and the current step number t at each time step, so that the model can pay more attention to specific areas.

[0181] In some embodiments, the image reconstruction module 203 further includes:

[0182] In the embodiments of the present application, by introducing mask weights and mask-based dynamic noise adjustment strategies into the loss function, the model's attention to and processing capabilities of key image areas are greatly enhanced.

[0183] Specifically, traditional anomaly detection models usually do not distinguish between different areas in an image, which may cause the model to pay insufficient attention to key areas of the image (such as areas that may contain anomalies or defects) during training. To solve this problem, the present application embodiment designs a new loss function, namely a dynamic weight loss function, which applies greater reconstruction pressure to image areas with masked blocks. Compared with non-masked areas, higher weights are applied to masked areas, allowing the model to pay more attention to these areas that are artificially marked as important during training.

[0184] According to the dynamic weight loss function, the mask of the masked area is weighted by a first weight, and the mask of the non-masked area is weighted by a second weight, so as to complete the reconstruction of the second image set and obtain a training image set, wherein the first weight is greater than the second weight.

[0185] The dynamic weight loss function is specifically:

[0186]

[0187] Where L is the dynamic weight loss function, N is the total number of samples in the second image set, and M i is the mask of the i-th pixel in the second image set, w m is the weight corresponding to the mask, X 0i is the i-th sample in the second image set, X after reconstruction 0i .

[0188] The dynamic weight loss function weights the square error of each pixel according to its weight in the mask, which enhances the reconstruction accuracy of the model in the masked area, thereby more effectively learning and optimizing the key areas. m , the strength of the model's reconstruction of the masked area can be flexibly controlled, thereby optimizing the performance of the model in specific application scenarios.

[0189] Implementing the embodiments of the present application has the following beneficial effects:

[0190] The embodiment of the present application takes into account that in the general model training process, there is a sample imbalance due to the small number of defective samples, and the trained model has low performance due to overfitting of the training results. Therefore, in order to solve the problem of small number of defective samples, different numbers of masking blocks of set shapes and sizes are randomly added to each original image to increase the number of defective samples. And by adjusting the noise intensity of the masked area corresponding to the masking block, the noise intensity of the non-masked area is kept unchanged, so that the model will pay more attention to the masked area in the future, improve the sensitivity of the model to key features, and further reduce the negative impact of sample imbalance. Then, different weights are applied to the masks of the masked area and the non-masked area respectively, so that the model can accurately and quickly identify the defective area of ​​the sample during the training process due to the different mask sizes, and can pay more attention to the repair and reconstruction of the defective area, so as to obtain a higher performance anomaly detection model that can accurately identify the defects of industrial products.

[0191] An embodiment of the present application provides a storage medium, which stores computer-readable program codes. When the computer-readable program codes are executed, the steps of the above-mentioned industrial product defect detection method based on masking strategy are implemented.

[0192] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A method for detecting industrial product defects based on a masking strategy, characterized in that: include: Randomly adding a number of masking blocks to each original image to obtain a first image set; wherein the masking blocks exist in the first image set in the form of masking regions; Resetting the noise intensity of the masked area in the first image set according to a preset dynamic noise ratio to obtain a second image set; According to a preset dynamic weight loss function, weights are applied to the masked area and the non-masked area in the second image set to obtain a training image set; Based on the training image set, the preset model is trained to obtain a trained anomaly detection model; The product data set is input into the anomaly detection model to obtain product defect recognition results.

2. The industrial product defect detection method based on masking strategy according to claim 1 is characterized in that: The method of randomly adding a number of mask blocks to each original image to obtain a first image set is specifically as follows: Assigning values ​​to pixels corresponding to a preset masking block to obtain a binary mask of the masking block; Randomly combining the binary mask with each original image to obtain the original image with the mask block added; A first image set is obtained according to the original image to which the mask block is added.

3. The industrial product defect detection method based on masking strategy according to claim 2 is characterized in that: The original image to which the mask block is added is specifically: In the formula, is the binary mask of the original image to which the masking block is to be added, is the i-th original image, The i-th original image with the mask block added.

4. The industrial product defect detection method based on masking strategy according to claim 1, characterized in that: The method of resetting the noise intensity of the masked area in the first image set according to the preset dynamic noise ratio to obtain the second image set is specifically as follows: Obtaining a mask matrix corresponding to the first image set; Based on the dynamic noise ratio, the noise intensity of the masked area is adjusted by the original noise intensity of the mask matrix to obtain a reconstructed image set; The reconstructed image set is denoised according to a preset denoising function to obtain a second image set.

5. The industrial product defect detection method based on masking strategy according to claim 4 is characterized in that: The step of adjusting the noise intensity of the masked area is specifically: Where η i is the noise intensity of the mask matrix after adjustment, σ t is the dynamic noise ratio, t is the current step number, T is the total trajectory step number, M i is the mask of the i-th pixel in the first image set, M i When it is 0, it represents the mask of the masked area, M i When it is 1, it represents the mask of the non-masked area.

6. The industrial product defect detection method based on masking strategy according to claim 1, characterized in that: According to the preset dynamic weight loss function, weights are applied to the masked area and the non-masked area in the second image set respectively to obtain a training image set, specifically: According to the dynamic weight loss function, weighting the mask of the masked area by a first weight and weighting the mask of the non-masked area by a second weight to complete the reconstruction of the second image set to obtain a training image set; The first weight is greater than the second weight.

7. The industrial product defect detection method based on masking strategy according to claim 6 is characterized in that: The dynamic weight loss function is specifically: Where L is the dynamic weight loss function, N is the total number of samples in the second image set, and M i is the mask of the i-th pixel in the second image set, w m is the weight corresponding to the mask, X 0i is the i-th sample in the second image set, X after reconstruction 0i .

8. The industrial product defect detection method based on masking strategy according to any one of claims 1 to 7, characterized in that: Also includes: According to the acquired model accuracy requirement, the shape and size of the mask block and the number of the mask blocks on each original image are set.

9. An industrial product defect detection system based on masking strategy, characterized in that: include: Masking block adding module, noise intensity adjusting module, image reconstruction module, model training module and defect detection module; The mask block adding module is used to randomly add a number of mask blocks to each original image to obtain a first image set; wherein the mask blocks exist in the first image set in the form of mask areas; The noise intensity adjustment module is used to reset the noise intensity of the masked area in the first image set according to a preset dynamic noise ratio to obtain a second image set; The image reconstruction module is used to apply weights to the masked area and the non-masked area in the second image set according to a preset dynamic weight loss function to obtain a training image set; The model training module is used to train the preset model based on the training image set to obtain a trained anomaly detection model; The defect detection module is used to input the product data set into the anomaly detection model to obtain product defect identification results.

10. A storage medium, characterized in that: The storage medium stores computer-readable program codes, and when the computer-readable program codes are executed, the steps of an industrial product defect detection method based on a masking strategy according to any one of claims 1 to 8 are implemented.

Images