An Industrial Image Anomaly Detection Method Based on Prior Knowledge and Image Reconstruction

Abstract

This invention provides an industrial image anomaly detection method based on prior knowledge and image reconstruction, comprising the following steps: Step 10) Inputting the image to be tested into an anomaly judgment model based on multivariate Gaussian distribution to obtain a mask of the image to be tested, and using the mask to occlude the image to be tested, obtaining an occluded image; Step 20) Inputting the occluded image into an image reconstruction model for reconstruction, obtaining a reconstructed image; Step 30) Calculating the multi-scale gradient magnitude similarity map between the image to be tested and the reconstructed image to obtain the anomaly detection result. This invention provides an industrial image anomaly detection method based on prior knowledge and image reconstruction, which makes the image reconstruction process unaffected by abnormal pixels, improving detection efficiency and accuracy.

Description

Technical Field

[0001] This invention belongs to the field of image anomaly detection technology, specifically, it relates to an industrial image anomaly detection method based on prior knowledge and image reconstruction. Background Technology

[0002] In industrial production environments, surface anomaly detection plays a crucial role, serving as a vital means to prevent defective products from entering the market. In recent years, due to the rapid advancements in computer technology, deep learning has become an important and widely applied industrial image anomaly detection technique. In actual industrial production, most acquired images are normal, anomaly-free samples, while only a small number are anomaly samples, and it's difficult to collect all anomaly types. Supervised learning methods for anomaly detection require collecting as many anomaly samples as possible to balance positive and negative samples, and manually labeling the data, which is labor-intensive and impractical in real-world applications. Unsupervised learning, on the other hand, only requires normal samples and no labels. During training, it captures the distribution of normal samples; when a sample deviates significantly from the normal sample distribution, it is detected as an anomaly. Theoretically, it can generalize to the detection of all types of anomalies, even unknown anomaly types. These advantages make it more suitable for applications involving surface anomaly detection in industrial products.

[0003] In the field of image anomaly detection, image reconstruction-based methods are widely used. These methods typically construct a network with an encoder-decoder structure to perform the reconstruction task, determining whether an anomaly sample is real by analyzing the pixel-level differences before and after reconstruction. Reconstruction-based methods rely on the assumption that the reconstruction model can reconstruct normal images well but cannot accurately reconstruct anomaly samples. However, due to the high versatility and robustness of convolutional autoencoders, they can also reconstruct anomalous parts of images well, causing this assumption to fail and resulting in low accuracy. Some methods use image reconstruction instead of image reconstruction for anomaly detection, where the distribution of anomalous regions is inconsistent with other regions in the image. If normal regions are masked, the recovered content should be highly similar to the corresponding region in the input image. If anomalous regions are masked, the recovered content should have low similarity to the corresponding region. This leads to increased differences before and after reconstruction of anomalous samples. These methods have demonstrated effectiveness on multiple datasets, but these models randomly mask image regions, introducing uncertainty, and the reconstruction process is still affected by anomalous pixels during testing. In addition, image restoration models can only reconstruct a portion of the image at a time. Therefore, in order to reconstruct a complete image, multiple calculations are required, both during the training and testing phases, which leads to a significant increase in computational load and requires substantial computational and memory overhead. Summary of the Invention

[0004] The technical problem to be solved by this invention is: an industrial image anomaly detection method based on prior knowledge and image reconstruction, which makes the image reconstruction process unaffected by abnormal pixels, thereby improving detection efficiency and accuracy.

[0005] To solve the above-mentioned technical problems, the embodiments of the present invention adopt the following technical solutions:

[0006] This invention provides an industrial image anomaly detection method based on prior knowledge and image reconstruction, comprising the following steps:

[0007] Step 10) Input the image to be tested into the anomaly detection model based on multivariate Gaussian distribution to obtain the mask of the image to be tested, and use the mask to occlude the image to be tested to obtain the occluded image.

[0008] Step 20) Input the occluded image into the image reconstruction model for reconstruction to obtain the reconstructed image;

[0009] Step 30) Calculate the multi-scale gradient magnitude similarity map between the image to be tested and the reconstructed image to obtain the anomaly detection result.

[0010] As a further improvement to this embodiment of the invention, the anomaly detection model based on multivariate Gaussian distribution is obtained through the following steps:

[0011] Step 011) Pre-train the neural network on a classic dataset to obtain a pre-trained network;

[0012] Step 012) Use a pre-trained network to extract feature vectors from the training images, and concatenate the feature vectors extracted from different levels to obtain the embedding vector of each image patch in the training images;

[0013] Step 013) Model the embedding vectors of image patches at the same location in all training images using a multivariate Gaussian distribution to obtain the multivariate Gaussian distribution that the embedding vector of each image patch follows.

[0014] As a further improvement to this embodiment of the invention, step 10) specifically includes:

[0015] Step 101) Use the pre-trained network obtained in step 011) to extract the feature vector of the image to be tested, and concatenate the feature vectors extracted from different levels to obtain the embedding vector of each image block of the image to be tested.

[0016] Step 102) Use equation (1) to obtain the Mahalanobis distance of each image patch;

[0017]

[0018] In the formula, M(W) ij W represents the Mahalanobis distance of the image patch located at position (i,j) in the image to be tested.ij μ represents the embedding vector of the image patch located at position (i,j) in the image to be tested. ij This represents the sample mean of the embedding vector in the multivariate Gaussian distribution that the image patch located at position (i,j) obtained in step 013) is followed by;

[0019] Step 103) Compare the Mahalanobis distance of each image block with the anomaly threshold, set the image blocks with a Mahalanobis distance greater than the anomaly threshold to 1, and set the other image blocks to 0, to obtain the mask;

[0020] Step 104) Use a mask to mask the image to be tested to obtain a masked image.

[0021] As a further improvement to this embodiment of the invention, step 104) specifically includes:

[0022] Using equation (2), a mask is used to mask the image to be tested, resulting in a masked image.

[0023] W′=W*(1-M) Equation (2)

[0024] In the formula, W′ represents the masked image, W represents the image to be tested, and M represents the mask.

[0025] As a further improvement to this embodiment of the invention, the abnormal threshold is obtained by the following steps:

[0026] Step 021) Use the pre-trained network obtained in step 011) to extract the feature vector of the verification image, and concatenate the feature vectors extracted from different levels to obtain the embedding vector of each image block of the verification image.

[0027] Step 022) Calculate the Mahalanobis distance for each image block of the verification image;

[0028] Step 023) Calculate the average Mahalanobis distance of pixel blocks at the same position in all verification images, and obtain the anomaly threshold based on the maximum value of the average Mahalanobis distance.

[0029] As a further improvement to this embodiment of the invention, the image reconstruction model is obtained through the following steps:

[0030] Step 031) Construct the image reconstruction model;

[0031] Step 032) Use a randomly generated mask to mask the training image, and input the masked training image into the image reconstruction model for training to obtain the trained image reconstruction model.

[0032] As a further improvement to this embodiment of the invention, step 031) specifically includes:

[0033] Step 0311) Construct the first-stage sub-network model, which is represented by equation (3):

[0034]

[0035] In the formula, F C f represents the obtained coarse reconstruction result. CB () represents a regular convolutional block, f ACB () represents a dilated convolution block, f in G' represents the interpolation operation, and G' represents the input masked image.

[0036] Step 0312) Construct the second-stage sub-network model, which is represented by equation (4):

[0037]

[0038] In the formula, F represents the final reconstruction result obtained. f represents the dilated convolution branch. CON-ATT () indicates a contextual attention branch;

[0039] Step 0313) Construct the loss function shown in equation (5) using mean squared error loss, structural similarity loss, and multi-scale gradient magnitude similarity loss:

[0040] L re =λL um +(1-λ)L m Equation (5)

[0041] In the formula, L re L represents the overall reconstruction loss. um L represents the reconstruction loss of the unshaded area. m λ represents the reconstruction loss of the occluded region, and λ represents the loss balance hyperparameter.

[0042] L um =αL SSIM (V&(1-A),V r &(1-A)+βL MSE (V&(1-A),V r &(1-A))+(1-α-β)L MSGMS (V&(1-A),V r &(1-A)) Equation (6)

[0043] L m =αL SSIM (V&M,V r &A)+βL MSE (V&A,V r &A)+(1-α-β)LMSGMS (V&A,V r &A) Equation (7)

[0044] In the formula, V represents the original image, V r This represents the reconstructed image obtained through the image reconstruction model, α and β represent the loss balance hyperparameters, A represents the mask of the original image, & represents bitwise multiplication, and L represents the reconstructed image obtained through the image reconstruction model. MSE (I,I r ) represents image I and image I r Mean squared error loss, L SSIM (I,I r ) represents image I and image I r Structural similarity loss, L MSGMS (I,I r ) represents image I and image I r Multi-scale gradient magnitude similarity loss.

[0045] As a further improvement to the embodiments of the present invention, Image I and Image I r Structural similarity loss L SSIM (I,I r Obtained through the following steps:

[0046] First, use equation (9) to calculate image I and image I r Structural similarity graph SSIM(I,I) r ):

[0047]

[0048] In the formula, η I This represents the average gray value of image I. Image I r The average gray value, σ I This represents the standard deviation of the gray levels in image I. Image I r The standard deviation of gray level Representing image I and image I r The covariance between them, c1 and c2 represent constants used to maintain stability;

[0049] Then use equation (10) to calculate image I and image I r Structural similarity loss L SSIM (I,I r ):

[0050]

[0051] In the formula, SSIM(I,I) r ) (m,n) Representing image I and image Ir The structural similarity value at pixel (m,n) is represented by ·, where · indicates multiplication, and E and F represent the resolution of the original image.

[0052] As a further improvement to the embodiments of the present invention, Image I and Image I r Multi-scale gradient magnitude similarity loss L MSGMS (I,I r Obtained through the following steps:

[0053] Using equation (11), calculate image I and image I at a single scale. r Gradient magnitude similarity plot between them:

[0054]

[0055] In the formula, g(I) represents the gradient magnitude map of the image, h x and h y This represents a 3×3 filter along the x and y dimensions, * represents a convolution operation, and c represents a constant to ensure numerical stability.

[0056] Image I and image I at multiple scales obtained through calculation r The gradient magnitude similarity maps between the two images are obtained by upsampling the gradient magnitude similarity maps to the original image resolution and averaging them to obtain image I and image I2. r Multiscale gradient magnitude similarity graph MSGMS(I,I) r );

[0057] Calculate image I and image I using equation (12). r Multi-scale gradient magnitude similarity loss L MSGMS (I,I r ):

[0058]

[0059] In the formula, MSGMS(I,I) r ) (m,n) Representing image I and image I r The multi-scale gradient magnitude similarity value at pixel (m,n), where · indicates multiplication, and E and F are the resolutions of the original image.

[0060] As a further improvement to this embodiment of the invention, step 30) specifically includes:

[0061] Step 301) Obtain the multi-scale gradient magnitude similarity map between the image to be tested and the reconstructed image;

[0062] Step 302) Obtain the anomaly maps of the image to be tested and the reconstructed image using equation (10):

[0063]

[0064] In the formula, f sf×sf This indicates a mean filter, * indicates a convolution operation, 1 P×Q This represents a filter matrix of size P×Q. Represents the image to be tested W and the reconstructed image Multi-scale gradient magnitude similarity map;

[0065] Step 303) The pixel-level anomaly score of the image to be tested is the value of the corresponding pixel in the anomaly map, and the image-level anomaly score of the image to be tested is the maximum value of all pixels in the anomaly map.

[0066] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0067] This invention provides an industrial image anomaly detection method based on prior knowledge and image reconstruction. First, a mask for the image to be tested is obtained using an anomaly judgment model based on a multivariate Gaussian distribution. This mask is then used to occlude the image, resulting in an occluded image. Next, an image reconstruction model is used to reconstruct the occluded image, yielding a reconstructed image. Finally, a multi-scale gradient magnitude similarity map between the image to be tested and the reconstructed image is calculated to obtain the anomaly detection result. This image anomaly detection method utilizes prior knowledge to mask the image, ensuring that the image reconstruction process is unaffected by abnormal pixels. This prevents the reconstruction model from accurately reconstructing abnormal regions, thus reconstructing non-abnormal regions within the abnormal areas. Anomaly detection and localization are achieved by comparing the reconstruction errors, further improving detection efficiency and accuracy. This image anomaly detection method requires only one image reconstruction to complete pixel-level anomaly detection and image-level anomaly localization, improving detection speed and reducing memory overhead. Attached Figure Description

[0068] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 This is a flowchart of the industrial image anomaly detection method based on prior knowledge and image reconstruction of the present invention;

[0070] Figure 2 This is a schematic diagram of the anomaly detection model based on multivariate Gaussian distribution in the method of this invention;

[0071] Figure 3This is a schematic diagram of the image reconstruction model in the method of the present invention. Detailed Implementation

[0072] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0073] This invention provides an industrial image anomaly detection method based on prior knowledge and image reconstruction, such as... Figure 1 As shown, it includes the following steps:

[0074] Step 10) Input the image to be tested into the anomaly detection model based on multivariate Gaussian distribution to obtain the mask of the image to be tested, and use the mask to occlude the image to be tested to obtain the occluded image.

[0075] Step 20) Input the occluded image into the image reconstruction model for reconstruction to obtain the reconstructed image.

[0076] Step 30) Calculate the multi-scale gradient magnitude similarity map between the image to be tested and the reconstructed image to obtain the anomaly detection result.

[0077] The anomaly detection method of this invention first inputs the image to be tested into an anomaly judgment model based on multivariate Gaussian distribution to obtain a mask, and then uses the mask to perform occlusion processing on the image to be tested to obtain an occluded image; then the occluded image is sent into a trained image reconstruction model for reconstruction to obtain an anomaly-free reconstructed image; finally, the multi-scale gradient magnitude similarity map between the image to be tested and the reconstructed image is calculated to obtain the final anomaly detection result.

[0078] The anomaly detection method of this invention obtains a mask through an anomaly judgment model based on a multivariate Gaussian distribution. This mask suppresses the influence of anomalous pixels during image reconstruction, enabling the image reconstruction model to achieve excellent anomaly-free reconstruction results for anomalous pixels and improving the accuracy of anomaly detection. This method only requires one image reconstruction to complete pixel-level anomaly detection and image-level anomaly localization, improving detection speed and reducing memory overhead.

[0079] Preferably, the anomaly detection model based on multivariate Gaussian distribution in the method of this embodiment of the invention, such as... Figure 2 As shown, the following steps are used to obtain:

[0080] Step 011) Pre-train the neural network on a classic dataset to obtain a pre-trained network. The neural network can be a ResNet18 neural network, a Wide ResNet-50-2 neural network, or an EfficientNet-B5 neural network, etc., and the classic dataset can be the ImageNet dataset, the CIFAR100 dataset, or the Pascal VOC dataset, etc.

[0081] Step 012) Use a pre-trained network to extract feature vectors from the training images, and concatenate the feature vectors extracted from different levels to obtain the embedding vector of each image patch in the training images.

[0082] Specifically, for each training image, a pre-trained ResNet18 neural network is used to extract the feature vector of the training image. The feature vectors of the first three layers are concatenated to obtain an embedding vector U containing information at different semantic levels and resolutions. ij Thus, a feature map U = {U} encoding the global image features is obtained. ij}

[0083] Since the feature map U has a lower resolution than the original image, the embedding vectors of many pixels are consistent, which allows the original image to be divided into k*k non-overlapping image blocks.

[0084]

[0085] U ij This represents the embedding vector of the image patch at position (i,j). This represents the feature vectors extracted from different network layers, and `concat()` is the concatenation operation. Here, 1 ≤ i ≤ E / k, 1 ≤ j ≤ F / k, and E and F are the resolutions of the original image.

[0086] This process yields the embedding vectors for all image patches in all training images. The training images are normal images.

[0087] Step 013) Model the embedding vectors of image patches at the same location in all training images using a multivariate Gaussian distribution to obtain the multivariate Gaussian distribution that the embedding vector of each image patch follows.

[0088] Specifically, assuming U ij It is a multivariate Gaussian distribution N(μ) ij ,∑ ij Generated by N(μ), which follows a multivariate Gaussian distribution. ij ,∑ ij ), where μ ij The mean of the embedding vectors of the image patch located at position (i,j) in all training images is ∑_{i=1}^{j} sample covariance. ij The estimate is as follows:

[0089]

[0090] In order to ensure the sample covariance matrix ∑ ij It is full-rank and invertible; a regularization term ∈ I is added to the above equation. N represents the number of training images. Let represent the embedding vector of the image patch located at position (i,j) in the b-th training image.

[0091] Based on this, the multivariate Gaussian distribution of the embedding vectors corresponding to each image patch of all training images is calculated.

[0092] Preferably, the image reconstruction model in the method of this embodiment of the invention is, for example... Figure 3 As shown, the following steps are used to obtain:

[0093] Step 031) Build an image reconstruction model.

[0094] Specifically, it includes:

[0095] Step 0311) Construct the first-stage sub-network model, which is represented by equation (3):

[0096]

[0097] In the formula, F C f represents the obtained coarse reconstruction result. CB () represents a regular convolutional block, f ACB () represents a dilated convolution block, f in G' represents the interpolation operation, and G′ represents the input masked image.

[0098] The dilation rates of dilated convolutions were set to 2, 4, 8, and 16, respectively. Setting different dilation rates will result in receptive fields of different sizes, which can obtain multi-scale information without introducing additional parameters.

[0099] The first-stage sub-network model roughly estimates the missing content using ordinary convolutions and dilated convolutions. Dilated convolutions can expand the receptive field without losing information, making them suitable for coarsely generating missing content using contextual information. This process includes thirteen ordinary convolutional blocks, four dilated convolutional blocks, and two interpolation operations. The ordinary convolutional blocks consist of Convolution+ELU activation functions, while the dilated convolutional blocks consist of Atrous Convolution+ELU activation functions.

[0100] Step 0312) Construct the second-stage sub-network model, which is represented by equation (4):

[0101]

[0102] In the formula, F represents the final reconstruction result obtained. This indicates a dilated convolution branch.

[0103] f CON-ATT() represents the context attention branch. First, the generated region image patch is matched with the known context region image patch. A matching score is calculated using a normalized inner product (cosine similarity). Then, channel softmax and deconvolution are used to reconstruct the missing parts of the image using contextual information. Specifically, the known region is considered the background region, and the part to be repaired is considered the foreground region. First, a 3×3 image patch is extracted from the background and reshaped into a convolutional filter. Then, the normalized inner product (cosine similarity) is used to measure the foreground image patch {f}. x,y} and background image block {b x′,y′ Match degree of}:

[0104]

[0105] Among them, s x,y，x′,y′ This represents the similarity of image patches centered on the background (x', y') and foreground (x, y). Then, softmax is performed across all channels to find the background region with the highest similarity to the region to be repaired. Finally, deconvolution is used to process information from this most similar region to reconstruct the occluded region. The deconvolution block consists of a Transposed Convolution + ELU activation function. An averaging operation is performed on overlapping regions during the deconvolution process.

[0106] The second-stage sub-network model processes the coarse results estimated by the first-stage sub-network model. First, it extracts features through six layers of ordinary convolutional blocks. Then, it goes through two branches, a context attention branch, and a dilated convolutional branch. Finally, the outputs of the context attention branch and the dilated convolutional branch are concatenated and passed through seven ordinary convolutional blocks and two interpolation operations to produce the final repaired image result.

[0107] Step 0313) Use mean square error loss L MSE Structural similarity loss L SSIM And multi-scale gradient magnitude similarity loss L MSGMS Construct the loss function shown in equation (5):

[0108] L re =λL um +(1-λ)L m Equation (5)

[0109] In the formula, L re L represents the overall reconstruction loss. um L represents the reconstruction loss of the unshaded area. m λ represents the reconstruction loss of the occluded region, and λ represents the loss balance hyperparameter.

[0110] L um =αL SSIM(V&(1-A),V r &(1-A))+βL MSE (V&(1-A),V r &(1-A))+(1-α-β)L MSGMS (V&(1-A),V r &(1-A)) Equation (6)

[0111] L m =αL SSIM (V&A,V r &A)+βL MSE (V&A,V r &A)+(1-α-β)L MSGMS (V&A,V r &A) Equation (7)

[0112] In the formula, V represents the original image, V r This represents the reconstructed image obtained through the image reconstruction model, α and β represent the loss balance hyperparameters, A represents the mask of the original image, and & represents bitwise multiplication.

[0113] The mean square error loss is calculated using equation (8):

[0114]

[0115] In the formula, L MSE (I,I r ) represents image I and image I r Mean squared error loss, I (m,n) I represents the pixel value at position (m,n) in image I. r(m,n) Image I r The pixel value at position (m,n), where · indicates multiplication, and E and F represent the resolution of the original image.

[0116] First, the structural similarity graph is calculated using equation (9):

[0117]

[0118] In the formula, SSIM(I,I) r ) represents image I and image I r Structural similarity diagram, η I This represents the average gray value of image I. Image I r The average gray value, σ I This represents the standard deviation of the gray levels in image I. Image I r The standard deviation of gray level Representing image I and image I rThe covariance between them, c1 and c2 represent constants used to maintain stability.

[0119] Reuse equation (10) to calculate the structural similarity loss:

[0120]

[0121] In the formula, L SSIM (I,I r ) represents image I and image I r Structural similarity loss, SSIM(I,I) r ) (m,n) Representing image I and image I r The structural similarity value at pixel (m,n) is represented by ·, where · indicates multiplication, and E and F represent the resolution of the original image.

[0122] The multi-scale gradient magnitude similarity loss is obtained through the following steps:

[0123] Using equation (11), calculate image I and image I at a single scale. r Gradient magnitude similarity plot between them:

[0124]

[0125] In the formula, g(I) represents the gradient magnitude map of image I, h x and h y This represents a 3×3 Prewitt filter along the x and y dimensions, * represents a convolution operation, and c represents a constant to ensure numerical stability.

[0126] Image I and image I at multiple scales obtained through calculation r The gradient magnitude similarity maps between the two images are obtained by upsampling the gradient magnitude similarity maps to the original image resolution and averaging them to obtain image I and image I2. r Multiscale gradient magnitude similarity graph MSGMS(I,I) r Multiple scales are included, encompassing the original image size as well as 1 / 2, 1 / 4, and 1 / 8 of the original image size. Downsampled images at multiple scales are generated by averaging and merging each image multiple times using a 2×2 sliding window with a stride of 2. The Prewitt filter is a commonly used filter for detecting image edges; it consists of horizontal and vertical operators, used to detect vertical and horizontal edges of the image, respectively.

[0127] Reuse equation (12) to calculate the multi-scale gradient magnitude similarity loss:

[0128]

[0129] In the formula, LMSGMS (I,I r ) represents image I and image I r Multi-scale gradient magnitude similarity loss, MSGMS(I,I) r ) (m,n) Representing image I and image I r The multi-scale gradient magnitude similarity value at pixel (m,n), where · indicates multiplication, and E and F are the resolutions of the original image.

[0130] Step 032) The training image is masked using a randomly generated mask, and the masked training image is input into the image reconstruction model for training. The loss function shown above is used as the optimization objective, and the Adam optimizer is used to train the context attention-based image reconstruction model to obtain the trained image reconstruction model.

[0131] The image reconstruction model in this embodiment is designed with a coarse-to-fine network structure. The first-stage sub-network model makes a coarse prediction of the image, and the second-stage sub-network model refines the coarse prediction and fills in the details. The image reconstruction model can achieve highly realistic reconstruction of images, and the context-focused branch introduced in the model makes the model pay more attention to learning contextual features, improving the model's ability to reconstruct the original image flawlessly using information from uncovered areas.

[0132] Preferably, in this embodiment, step 10) specifically includes:

[0133] Step 101) Use the pre-trained network obtained in step 011) to extract the feature vector of the image to be tested, and concatenate the feature vectors extracted from different levels to obtain the embedding vector of each image block of the image to be tested.

[0134] Step 102) Use equation (1) to obtain the Mahalanobis distance of each image patch;

[0135]

[0136] In the formula, M(W) ij W represents the Mahalanobis distance of the image patch located at position (i,j) in the image to be tested. ij μ represents the embedding vector of the image patch located at position (i,j) in the image to be tested. ij The mean of the embedding vector in the multivariate Gaussian distribution that the image patch at position (i,j) obtained in step 013) is embedded.

[0137] Step 103) Compare the Mahalanobis distance of each image block with the anomaly threshold. Set the image blocks with a Mahalanobis distance greater than the anomaly threshold to 1 and the other image blocks to 0 to obtain the mask.

[0138] Step 104) Use a mask to mask the image to be tested to obtain a masked image.

[0139] Specifically, using equation (2), a mask is used to mask the image to be tested, resulting in a masked image.

[0140] W′=W*(1-M) Equation (2)

[0141] In the formula, W' represents the masked image, W represents the image to be tested, and M represents the mask.

[0142] Preferably, the abnormal threshold in the method of this embodiment is obtained by the following steps:

[0143] Step 021) Use the pre-trained network obtained in step 011) to extract the feature vector of the verification image, and concatenate the feature vectors extracted from different levels to obtain the embedding vector of each image block of the verification image.

[0144] Step 022) Calculate the Mahalanobis distance for each image block of the verification image, which is the distance between the embedding vector of each image block and the multivariate Gaussian distribution of the corresponding image block obtained in step 013).

[0145] Step 023) Calculate the average Mahalanobis distance of pixel blocks at the same position in all verification images, and use the maximum value of the average Mahalanobis distance as the anomaly threshold.

[0146] Preferably, step 30) in the method of this embodiment specifically includes:

[0147] Step 301) Obtain the multi-scale gradient magnitude similarity map between the image to be tested and the reconstructed image. The specific calculation steps are the same as those in step 0313) above for obtaining the multi-scale gradient magnitude similarity map.

[0148] Step 302) Obtain the anomaly maps of the image to be tested and the reconstructed image using equation (10):

[0149]

[0150] In the formula, f sf×sf This indicates a mean filter, * indicates a convolution operation, 1 P×Q This represents a filter matrix of size P×Q. Represents the image to be tested W and the reconstructed image Multiscale gradient magnitude similarity map.

[0151] Step 303) The pixel-level anomaly score of the image to be tested is the value of the corresponding pixel in the anomaly map, and the image-level anomaly score of the image to be tested is the maximum value of all pixels in the anomaly map.

[0152] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the specific embodiments described above. The specific embodiments and descriptions in the specification are merely for further illustrating the principles of the present invention. Various changes and modifications can be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed.

Claims

1. An industrial image anomaly detection method based on prior knowledge and image reconstruction, characterized in that, Includes the following steps: Step 10) Input the image to be tested into the anomaly detection model based on multivariate Gaussian distribution to obtain the mask of the image to be tested, and use the mask to occlude the image to be tested to obtain the occluded image. Step 20) Input the occluded image into the image reconstruction model for reconstruction to obtain the reconstructed image; Step 30) Calculate the multi-scale gradient magnitude similarity map between the image to be tested and the reconstructed image to obtain the anomaly detection result; The image reconstruction model is obtained through the following steps: Step 031) Construct the image reconstruction model; Step 032) Use a randomly generated mask to mask the training image, and input the masked training image into the image reconstruction model for training to obtain the trained image reconstruction model; Step 031) specifically includes: Step 0311) Construct the first-stage sub-network model, which is represented by equation (3): Equation (3) In the formula, This indicates the obtained rough reconstruction result. Represents a regular convolutional block. Indicates a dilated convolution block. This indicates an interpolation operation. Represents the input masked image; Step 0312) Construct the second-stage sub-network model, which is represented by equation (4): Equation (4) In the formula, This represents the final reconstruction result. Indicates a dilated convolution branch. Indicates the context attention branch; Step 0313) Construct the loss function shown in equation (5) using mean squared error loss, structural similarity loss, and multi-scale gradient magnitude similarity loss: Equation (5) In the formula, Indicates the overall reconstruction loss. This represents the reconstruction loss in the unshaded area. Represents the reconstruction loss of the shading area. This represents the loss balance hyperparameter; Equation (6) Equation (7) In the formula, Represents the original image. This represents the reconstructed image obtained through the image reconstruction model. and This represents the loss balance hyperparameter. A mask representing the original image. This indicates digit-wise multiplication. Representing an image and images The mean square error loss, Representing an image and images Structural similarity loss, Representing an image and images Multi-scale gradient magnitude similarity loss.

2. The industrial image anomaly detection method based on prior knowledge and image reconstruction according to claim 1, characterized in that, The anomaly detection model based on multivariate Gaussian distribution is obtained through the following steps: Step 011) Pre-train the neural network on a classic dataset to obtain a pre-trained network; Step 012) Use a pre-trained network to extract feature vectors from the training images, and concatenate the feature vectors extracted from different levels to obtain the embedding vector for each image patch of the training image; Step 013) Model the embedding vectors of image patches at the same location in all training images using a multivariate Gaussian distribution to obtain the multivariate Gaussian distribution that the embedding vector of each image patch follows.

3. The industrial image anomaly detection method based on prior knowledge and image reconstruction according to claim 2, characterized in that, Step 10) specifically includes: Step 101) Use the pre-trained network obtained in step 011) to extract the feature vector of the image to be tested, and concatenate the feature vectors extracted from different levels to obtain the embedding vector of each image block of the image to be tested; Step 102) Use equation (1) to obtain the Mahalanobis distance of each image patch; Equation (1) In the formula, Indicates the location in the image under test Mahalanobis distance of the image patch Indicates the location in the image under test The embedding vector of the image patch, This indicates that the location obtained in step 013) is... The sample mean of the embedding vector in the multivariate Gaussian distribution that the image patch follows; Step 103) Compare the Mahalanobis distance of each image block with the anomaly threshold, set the image blocks with a Mahalanobis distance greater than the anomaly threshold to 1, and set the other image blocks to 0, to obtain the mask; Step 104) Use a mask to mask the image to be tested to obtain a masked image.

4. The industrial image anomaly detection method based on prior knowledge and image reconstruction according to claim 3, characterized in that, Step 104) specifically includes: Using equation (2), a mask is used to mask the image to be tested, resulting in a masked image. Equation (2) In the formula, Indicates an image being masked. Represents the image to be tested. It represents a mask.

5. The industrial image anomaly detection method based on prior knowledge and image reconstruction according to claim 3, characterized in that, The abnormal threshold is obtained using the following steps: Step 021) Use the pre-trained network obtained in Step 011) to extract the feature vector of the verification image, and concatenate the feature vectors extracted from different levels to obtain the embedding vector of each image block of the verification image; Step 022) Calculate the Mahalanobis distance for each image patch of the verification image; Step 023) Calculate the average Mahalanobis distance of pixel blocks at the same position in all verification images, and obtain the anomaly threshold based on the maximum value of the average Mahalanobis distance.

6. The industrial image anomaly detection method based on prior knowledge and image reconstruction according to claim 1, characterized in that, image and images Structural similarity loss Obtained through the following steps: First, use equation (9) to calculate the image. and images Structural similarity diagram : Equation (9) In the formula, Representing an image Average gray value, Representing an image Average gray value, Representing an image The standard deviation of gray level Representing an image The standard deviation of gray level Representing an image and images Covariance between and This represents a constant used to maintain stability; Reuse equation (10) to calculate the image and images Structural similarity loss : Equation (10) In the formula, Representing an image and images Medium pixel Structural similarity value at the location, The expression indicates multiplication, and E and F represent the resolution of the original image.

7. The industrial image anomaly detection method based on prior knowledge and image reconstruction according to claim 1, characterized in that, image and images Multi-scale gradient magnitude similarity loss Obtained through the following steps: Calculate the image at a single scale using equation (11) and images Gradient magnitude similarity plot between them: Equation (11) In the formula, , This represents the gradient magnitude map of the image. and Indicates along and Dimensional filter, This represents the convolution operation. This represents a constant that ensures numerical stability; Images obtained through calculation at multiple scales and images The gradient magnitude similarity map between the two images is obtained by upsampling the gradient magnitude similarity map to the original image resolution and then averaging the results. and images Multiscale gradient magnitude similarity map ; Calculate the image using equation (12) and images Multi-scale gradient magnitude similarity loss : Equation (12) In the formula, Representing an image and images Pixels Multi-scale gradient magnitude similarity value at the location, This indicates multiplication, where E and F are the resolutions of the original images.

8. The industrial image anomaly detection method based on prior knowledge and image reconstruction according to claim 1, characterized in that, Step 30) specifically includes: Step 301) Obtain the multi-scale gradient magnitude similarity map between the image to be tested and the reconstructed image; Step 302) Obtain the anomaly maps of the image to be tested and the reconstructed image using equation (10): Equation (10) In the formula, This represents a mean filter. This represents the convolution operation. Indicates size is The filter matrix, Represents the image to be tested and reconstructed image Multi-scale gradient magnitude similarity map; Step 303) The pixel-level anomaly score of the image under test is the value of the corresponding pixel in the anomaly map, and the image-level anomaly score of the image under test is the maximum value of all pixels in the anomaly map.

Images