Image anomaly detection method fusing local-global feature reconstruction

Abstract

The invention discloses an image anomaly detection method fusing local-global feature reconstruction. According to the method, a local-global feature reconstruction framework is fused, and two feature reconstruction branches are utilized to detect local and global anomalies respectively, so that the method has the capability of simultaneously detecting structure and logic anomalies; different types of normal features are reserved by using different bottleneck embedding modules, so that local feature reconstruction is accurately realized by a local feature reconstruction branch, and global feature reconstruction is realized by paying attention to global context information by a global feature reconstruction branch. Meanwhile, the abnormal information is effectively filtered through the filter module based on abnormal simulation, and the detection capability is improved.

Description

Technical Field

[0001] The present invention relates to the fields of visual detection technology and image anomaly detection technology, and in particular to an image anomaly detection method that integrates local and global feature reconstruction. Background Art

[0002] In recent years, technologies such as computer vision and deep learning have developed rapidly. Visual inspection technology based on deep learning is of great significance in intelligent manufacturing and has achieved great results. However, in actual application scenarios, there are problems such as insufficient defect samples and high annotation costs, which make it difficult to apply visual defect detection methods based on supervised learning. The current research work introduces deep image anomaly detection methods into visual inspection tasks, and builds detection models through normal images to improve defect detection and positioning effects. It has great potential in solving the above shortcomings of visual inspection.

[0003] Existing research on image anomaly detection mainly focuses on local structural anomalies, such as cracks, color contamination, scratches or contamination. Zavrtanik et al. proposed DRAEM in 2021 (Zavrtanik V, Kristan M, D.Draem-adiscriminatively trained reconstruction embedding for surface anomaly detection[C] / / Proceedings of the IEEE / CVF International Conference onComputer Vision.2021:8330-8339.), selected various texture images as defective backgrounds, used noise generation methods to create anomalies and superimposed them on normal images, learned the joint representation of normal images and their synthetic anomalies to detect anomalies. Roth et al. (Roth K, Pemula L, Zepeda J, et al. Towards total recall in industrial anomaly detection[C] / / Proceedings of the IEEE / CVF Conference onComputer Vision and Pattern Recognition.2022:14318-14328.) proposed PatchCore, which uses nominal patch-level feature representation extracted from the ImageNet pre-trained network and achieves minimal running time through core set subsampling, thereby improving anomaly detection accuracy. In 2022, Deng et al. proposed a feature reconstruction network RD4AD (Deng H, Li X. Anomaly detection via reverse distillation from one-class embedding [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2022: 9737-9746.), which uses a decoder to reconstruct multi-level pre-trained features to achieve anomaly detection. However, while existing methods achieve high detection performance for local structural anomalies, some complex scenes have logical anomalies that violate long-range dependencies, such as normal objects placed in the wrong position or missing objects. The existing technology mainly focuses on structural anomalies, but faces difficulties in detecting logical anomalies, and often ignores the detection of logical anomalies. At the same time, logical anomalies have complex normal patterns due to their complex background and global constraints, which brings severe challenges to training the normal pattern of logical anomalies.

[0004] In summary, how to solve the challenges associated with complex normal patterns of logical anomalies and achieve accurate positioning, and improve the usability of anomaly detection methods in various practical scenarios, is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] The purpose of the present invention is to provide an image anomaly detection method that integrates local and global feature reconstruction to solve the problems raised in the above background technology.

[0006] The purpose of the present invention is achieved by at least one of the following technical solutions.

[0007] An image anomaly detection method by integrating local and global feature reconstruction comprises the following steps:

[0008] S1. Obtaining image data, including obtaining public data set images and simulated abnormal sample images, where the public data set images correspond to the simulated abnormal sample images one by one;

[0009] S2. In the training phase, the public dataset images and the simulated abnormal sample images are input into the feature extractor pre-trained on ImageNet to obtain multi-level embedded features of the public dataset images and their corresponding simulated abnormal sample images respectively;

[0010] S3, in the training phase, the multi-level embedded features of the acquired public dataset images and simulated abnormal sample images are input into the filter module to obtain the features of each level after filtering of the public dataset images and simulated abnormal sample images;

[0011] S4, input the features of each level of the public dataset image into the local feature reconstruction branch, and obtain the reconstructed features through the local bottleneck embedding module and decoder;

[0012] S5, input the highest level features among the features of each level of the public dataset image into the global feature reconstruction branch, and obtain the reconstructed features through the global bottleneck embedding module and decoder;

[0013] S6. Construct an overall loss, and train the filter module and the global feature reconstruction branch and the local feature reconstruction branch according to the overall loss;

[0014] S7. In the inference stage, only the test set samples in the public dataset images are used to input the feature extractor, the trained filter module, and the global feature reconstruction branch and the local feature reconstruction branch. The anomaly detection and positioning results are calculated by using the filtering features of each level of the samples in the test set of the public dataset images and the global reconstruction features obtained in the global feature reconstruction branch and the local reconstruction features obtained in the local feature reconstruction branch.

[0015] Furthermore, in step S1, a public data set for logical anomaly and structural anomaly detection is obtained, and a simulated anomaly sample image is obtained through the public data set image using an anomaly simulation strategy, which specifically includes the following steps:

[0016] S1.1, generate noise, obtain the noise binarization mask through the threshold; at the same time, binarize the input image to obtain the corresponding mask; then, obtain the final mask image by performing element-wise product of the two masks obtained;

[0017] S1.2. Randomly select a texture mask image from the public texture dataset DTD, and perform dot multiplication to obtain defects containing texture information. For structural anomalies, randomly adjust the public dataset image first, and then divide it into multiple identical grids, and then randomly arrange them to obtain simulated structural anomalies. Randomly use these two methods to obtain noise images corresponding to texture information defects or structural anomalies.

[0018] S1.3, performing element-wise product of the mask image and the noise image to obtain a noise foreground image;

[0019] S1.4, invert the mask image obtained in step S1.1, and then multiply it element by element with the public dataset image, and then superimpose it with the noise foreground image to obtain the final abnormal simulation image.

[0020] Furthermore, in step S2, a feature extractor pre-trained on ImageNet is used as an encoder, positive samples in the public dataset image are input into the encoder, the first K layers of the encoder network are fixed, and K output features of different sizes are obtained, i.e., multi-level embedded features of the public dataset image;

[0021] The simulated abnormal images corresponding to the public dataset images are input into the encoder, the first K layers of the encoder network are fixed, and K output features of different sizes are obtained, namely, the K-level embedded features of the simulated abnormal images.

[0022] Furthermore, in step S3, the multi-level embedded features of the public dataset images and the simulated abnormal sample images are input into the filter module to obtain the filtered K-level features; each layer of the filter module is structured as L convolution blocks stacked in sequence.

[0023] Furthermore, based on the K-level features after filtering the public dataset images and the simulated abnormal sample images, the consistency loss is calculated as follows:

[0024]

[0025] Among them, fn i,k ,fa i,k Respectively represent the i-th public dataset image x i and the corresponding ith simulated abnormal sample image ξ(xi ) Feature output of the k-th layer encoder: F k represents the kth layer of the filter module;

[0026] The contrast loss is calculated by simulating the multi-level features obtained by the encoder of abnormal images and the multi-level features obtained after filtering the public dataset images:

[0027]

[0028] fn i,k ,fa i,k Respectively represent the i-th public dataset image x i and the corresponding ith simulated abnormal sample image ξ(x i ) Feature output of the k-th layer encoder: F k Represents the k-th layer of the filter module.

[0029] Furthermore, in step S4, the features of each level obtained after filtering the public dataset image input in step S1 through the filter module are input into the local feature reconstruction branch, and the reconstructed features are obtained through the local bottleneck embedding module and the local branch decoder, as follows:

[0030] The multi-layer features obtained after filtering the public dataset image by the filter module are input into the local bottleneck embedding module. First, the first K-1 layers of features of the K-level features are downsampled through the convolutional layer in the local bottleneck embedding module. Then, the obtained multi-level representations are connected and fused to finally obtain the compressed features.

[0031] The local branch decoder in the local feature reconstruction branch reconstructs the compressed features back into filtered features of different levels to obtain local reconstructed features; the local branch decoder and the feature extractor in step S2 have a symmetrical structure. In the local branch decoder, the downsampling operation used in the feature extractor is replaced by upsampling.

[0032] Furthermore, in step S5, the highest level feature among the multi-layer features obtained after filtering the input image by the filter module is input into the global feature reconstruction branch, and the reconstructed feature is obtained through the global bottleneck embedding module and the global branch decoder, including:

[0033] The highest-level features obtained after the input image features are filtered by the filter module are input into the global bottleneck embedding module. The highest-level filtered features are first reconstructed through the residual block, and then the features are compressed through the global maximum pooling operation to obtain a more compact feature vector; finally, the obtained feature vector is upsampled to 8 times using deconvolution to obtain compressed features;

[0034] The global branch decoder of the global feature reconstruction branch then reconstructs the compressed features back into filtered features of different levels to obtain global reconstructed features; the global branch decoder in the global feature reconstruction branch has the same architecture as the local branch decoder in the local reconstruction branch, but its input is only the highest level of abstract feature representation.

[0035] Furthermore, the cosine distance is calculated by using the local reconstruction features and the multi-layer features obtained after filtering the public dataset image input in step S1 to generate the corresponding feature map and calculate the reconstruction loss, as follows:

[0036]

[0037] in, represents the filtering features output by the k-th filter module, Represents the reconstructed features of the first decoder at the kth layer of the local feature reconstruction branch, represents the feature map output by the kth layer of the local branch decoder in the local reconstruction branch, L 局部 represents the reconstruction loss of the local feature reconstruction branch, h k represents the height of the feature output of the kth layer of the local branch decoder in the local reconstruction branch, w k Represents the width of the feature output by the k-th layer of the local branch decoder in the local reconstruction branch;

[0038] The cosine distance is calculated by filtering the global reconstruction features and the multi-layer features obtained after filtering the public dataset image input in step S1 to generate the corresponding feature map and calculate the reconstruction loss, as follows:

[0039]

[0040] in, represents the filtering features output by the k-th filter module, The global branch decoder represents the reconstructed features of the global feature reconstruction branch at the kth layer, represents the feature map output by the kth layer of the global branch decoder in the global reconstruction branch, L 全局 represents the reconstruction loss of the global feature reconstruction branch, h k represents the height of the feature output of the kth layer of the global branch decoder in the global reconstruction branch, w k Represents the width of the feature output by the k-th layer of the global branch decoder in the global reconstruction branch.

[0041] Furthermore, in step S6, the overall loss L is specifically as follows:

[0042] L = L - 致 +L 对比 +L 重建 ;

[0043] L 重建 =L 局部 +L 全局 .

[0044] Furthermore, in step S7, in the inference stage, only the test set samples in the public dataset images are used to input the reconstruction model after training, and the anomaly detection and positioning results are calculated by using the filtering features of the samples in the test set of the public dataset images at all levels and the global reconstruction features obtained in the global feature reconstruction branch and the local reconstruction features obtained in the local feature reconstruction branch, including:

[0045] Input the samples of the test set in the public dataset image to the pre-trained feature extractor to obtain K-level embedded features;

[0046] Input the K-level embedding features into the trained filter module to obtain the filtered K-level embedding features;

[0047] The filtered K-level embedded features are input into the trained local feature reconstruction branch to obtain the local anomaly detection results, each score map is upsampled to the image resolution, and each score map is element-wise added as follows:

[0048]

[0049] Among them, U k (·) represents the upsampling operation of the kth anomaly map, and Mlocal represents the anomaly score map obtained by the local reconstruction branch;

[0050] The filtered K-level embedded features are input into the trained global feature reconstruction branch to obtain the global anomaly detection results, each score map is upsampled to the image resolution, and each score map is element-wise added as follows:

[0051]

[0052] Among them, M 全局 Represents the anomaly score map obtained by the global reconstruction branch;

[0053] The local anomaly detection results and the global anomaly detection results are combined to obtain the final anomaly detection result M, as follows:

[0054] M=M 局部 +M 全局 .

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] The present invention integrates the local-global feature reconstruction framework and uses two feature reconstruction branches to detect local and global anomalies respectively, so that it has the ability to detect structural and logical anomalies at the same time; different bottleneck embedding modules are used to retain different types of normal features, so that the local feature reconstruction branch accurately realizes local feature reconstruction, while the global feature reconstruction branch focuses on global context information to realize global feature reconstruction. At the same time, through the filter module based on anomaly simulation, abnormal information is effectively filtered to improve detection capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 It is an implementation flow chart of an image anomaly detection method integrating local and global feature reconstruction in an embodiment of the present invention;

[0059] Figure 2 A schematic diagram of the network structure of an image anomaly detection method integrating local and global feature reconstruction in an embodiment of the present invention;

[0060] Figure 3 Schematic diagram of the structure of the feature extractor in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] Example:

[0063] An image anomaly detection method that integrates local and global feature reconstruction, such as Figure 1 , Figure 2 As shown, the following steps are included:

[0064] S1. Obtaining image data, including obtaining public data set images and simulated abnormal sample images, where the public data set images correspond to the simulated abnormal sample images one by one;

[0065] Obtain a public dataset for logical anomaly and structural anomaly detection, and use anomaly simulation strategies to obtain simulated anomaly sample images from public dataset images, specifically including the following steps:

[0066] S1.1, generate noise, obtain the noise binarization mask through the threshold; at the same time, binarize the input image to obtain the corresponding mask; then, obtain the final mask image by performing element-wise product of the two masks obtained;

[0067] S1.2. Randomly select a texture mask image from the public texture dataset DTD, and perform dot multiplication to obtain defects containing texture information. For structural anomalies, randomly adjust the public dataset image first, and then divide it into multiple identical grids, and then randomly arrange them to obtain simulated structural anomalies. Randomly use these two methods to obtain noise images corresponding to texture information defects or structural anomalies.

[0068] S1.3, performing element-wise product of the mask image and the noise image to obtain a noise foreground image;

[0069] S1.4, invert the mask image obtained in step S1.1, and then multiply it element by element with the public dataset image, and then superimpose it with the noise foreground image to obtain the final abnormal simulation image.

[0070] In one embodiment, a dataset MVTec LOCO AD for logical anomaly and structural anomaly detection is obtained from a public website;

[0071] In one embodiment, a two-dimensional Perlin noise P is generated, and a binary mask M is obtained by a threshold T. P . Binarize the input image I to obtain a mask. Then, by performing an element-wise product of the two masks obtained, the final mask image M is obtained;

[0072] Randomly select texture images R and M from the public texture dataset DTD P The dot product obtains the defects containing texture information; for structural anomalies, the input image is first randomly adjusted, and then it is evenly divided into grids, and the simulated structural anomalies are obtained after random arrangement. These two methods are randomly used to obtain the noise image I n , mask image M and noise image I n Perform element-wise product to obtain the noisy foreground image I′ n In this process, a transparency factor δ is introduced to balance the fusion of the original image and the noise image, as shown in the following formula:

[0073] I′ n =δ(M⊙I n )+(1-δ)(M⊙I)

[0074] The inverted M is multiplied element by element by the original input I to obtain I′. n The final abnormal simulation image I is obtained by superposition. A, add it and its mask M to the training set for supervised learning.

[0075] S2. In the training phase, the public dataset images and the simulated abnormal sample images are input into the feature extractor pre-trained on ImageNet to obtain multi-level embedded features of the public dataset images and their corresponding simulated abnormal sample images respectively;

[0076] In one embodiment, Figure 3 As shown in the figure, the WideResNet50 model (WideResNet50: Li G, Chen W, Mu C. Residual-wider convolutional neural network for image recognition [J]. IET Image Processing 14.16 (2020): 4385-4391.) pre-trained on ImageNet is used as the encoder, the positive samples in the public dataset images are input into the encoder, the first three layers of the encoder network are fixed, and three output features of different sizes are obtained, namely the multi-level embedded features of the public dataset images;

[0077] The simulated abnormal images corresponding to the public dataset images are input into the encoder, the first three layers of the encoder network are fixed, and three output features of different sizes are obtained, namely the three-level embedded features of the simulated abnormal images.

[0078] In other embodiments, other resnet series models (He K, Zhang X, Ren S, et al. Deepest learning for image recognition [C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778.) are used as encoders.

[0079] S3, in the training phase, the multi-level embedded features of the acquired public dataset images and simulated abnormal sample images are input into the filter module to obtain the features of each level after filtering of the public dataset images and simulated abnormal sample images;

[0080] The multi-level embedded features of the public dataset images and the simulated abnormal sample images are input into the filter module to obtain the filtered three-level features; each layer of the filter module is composed of L convolution blocks stacked in sequence.

[0081] In the embodiment, L=4 is set.

[0082] Based on the K-level features after filtering the public dataset images and the simulated abnormal sample images, the consistency loss is calculated as follows:

[0083]

[0084] Among them, fn i,k ,fa i,k Respectively represent the i-th public dataset image x i and the corresponding ith simulated abnormal sample image ξ(x i ) Feature output of the k-th layer encoder: F k represents the kth layer of the filter module;

[0085] The contrast loss is calculated by simulating the multi-level features obtained by the encoder of abnormal images and the multi-level features obtained after filtering the public dataset images:

[0086]

[0087] fn i,k ,fa i,k Respectively represent the i-th public dataset image x i and the corresponding ith simulated abnormal sample image ξ(x i ) Feature output of the k-th layer encoder: F k Represents the k-th layer of the filter module.

[0088] S4, input the features of each level of the public dataset image into the local feature reconstruction branch, and obtain the reconstructed features through the local bottleneck embedding module and decoder;

[0089] The local feature reconstruction branch receives densely encoded features and decodes the features from high semantic level to low semantic level. The dense feature space may abandon unseen abnormal feature representations during inference to encourage abnormal feature differences. This leads to accurate feature reconstruction and structural anomaly detection.

[0090] The public dataset image input in step S1 is filtered by the filter module to obtain the features of each level, which are input into the local feature reconstruction branch, and the reconstructed features are obtained through the local bottleneck embedding module and the local branch decoder, as follows:

[0091] The multi-layer features obtained after filtering the public dataset image by the filter module are input into the local bottleneck embedding module. First, the first K-1 layers of features of the K-level features are downsampled through the convolutional layer in the local bottleneck embedding module. Then, the obtained multi-level representations are connected and fused to finally obtain the compressed features.

[0092] The local branch decoder in the local feature reconstruction branch reconstructs the compressed features back into filtered features of different levels to obtain local reconstructed features; the local branch decoder and the feature extractor in step S2 have a symmetrical structure. In the local branch decoder, the downsampling operation used in the feature extractor is replaced by upsampling.

[0093] S5, input the highest level features among the features of each level of the public dataset image into the global feature reconstruction branch, and obtain the reconstructed features through the global bottleneck embedding module and decoder;

[0094] Logical anomalies are caused by the absence or misalignment of normal components, which may be globally abnormal but locally normal. This situation brings difficulties to existing anomaly detection methods. Although the encoder-decoder architecture can naturally retain the most important information of the image, and the multi-scale feature extraction paradigm can consider both low-level and high-level information. However, if the model only uses the local feature reconstruction branch, it is still impossible to perform global learning to detect logical anomalies. In addition, the local bottleneck embedding module will fuse low-level features into the final embedding space, which is conducive to accurate low-level feature reconstruction, but it also reduces the global context learning ability to a certain extent. Therefore, a global feature reconstruction branch is designed to focus on global context information.

[0095] The highest level features among the multi-layer features obtained after the input image is filtered by the filter module are input into the global feature reconstruction branch, and the reconstructed features are obtained through the global bottleneck embedding module and the global branch decoder, including:

[0096] The highest level features obtained after the input image features are filtered by the filter module are input into the global bottleneck embedding module, and the highest level filtered features are first reconstructed by the residual block, and then the features are compressed by the global maximum pooling operation to obtain a more compact feature vector; finally, in one embodiment, the obtained feature vector is upsampled to 8 times using deconvolution to obtain compressed features;

[0097] The global branch decoder of the global feature reconstruction branch then reconstructs the compressed features back into filtered features of different levels to obtain global reconstructed features; the global branch decoder in the global feature reconstruction branch has the same architecture as the local branch decoder in the local reconstruction branch, but its input is only the highest level of abstract feature representation.

[0098] The cosine distance is calculated by filtering the local reconstruction features and the multi-layer features obtained after filtering the public dataset image input in step S1 to generate the corresponding feature map and calculate the reconstruction loss, as follows:

[0099]

[0100] in, represents the filtering features output by the k-th filter module, Represents the reconstructed features of the first decoder at the kth layer of the local feature reconstruction branch, represents the feature map output by the kth layer of the local branch decoder in the local reconstruction branch, L 局部 represents the reconstruction loss of the local feature reconstruction branch, h k represents the height of the feature output of the kth layer of the local branch decoder in the local reconstruction branch, w k Represents the width of the feature output by the k-th layer of the local branch decoder in the local reconstruction branch;

[0101] The cosine distance is calculated by filtering the global reconstruction features and the multi-layer features obtained after filtering the public dataset image input in step S1 to generate the corresponding feature map and calculate the reconstruction loss, as follows:

[0102]

[0103]

[0104] in, represents the filtering features output by the k-th filter module, The global branch decoder represents the reconstructed features of the global feature reconstruction branch at the kth layer, represents the feature map output by the kth layer of the global branch decoder in the global reconstruction branch, L 全局 represents the reconstruction loss of the global feature reconstruction branch, h k represents the height of the feature output of the kth layer of the global branch decoder in the global reconstruction branch, w k Represents the width of the feature output by the k-th layer of the global branch decoder in the global reconstruction branch.

[0105] S6, such as Figure 2 As shown, an overall loss is constructed, and the filter module as well as the global feature reconstruction branch and the local feature reconstruction branch are trained according to the overall loss;

[0106] The overall loss L is as follows:

[0107] L = L - 致 +L 对比 +L 重建 ;

[0108] L 重建 =L 局部 +L 全局 .

[0109] S7. In the inference stage, only the test set samples in the public dataset images are used to input the feature extractor, the trained filter module, and the global feature reconstruction branch and the local feature reconstruction branch. The anomaly detection and positioning results are calculated by the filtering features of the samples in the test set of the public dataset images at all levels and the global reconstruction features obtained in the global feature reconstruction branch and the local reconstruction features obtained in the local feature reconstruction branch.

[0110] In the inference phase, only the test set samples in the public dataset images are used to input the reconstruction model after training. The anomaly detection and positioning results are calculated through the filtering features of the samples in the test set of the public dataset images at all levels and the global reconstruction features obtained in the global feature reconstruction branch and the local reconstruction features obtained in the local feature reconstruction branch, including:

[0111] Input the samples of the test set in the public dataset image to the pre-trained feature extractor to obtain K-level embedded features;

[0112] Input the K-level embedding features into the trained filter module to obtain the filtered K-level embedding features;

[0113] The filtered K-level embedded features are input into the trained local feature reconstruction branch to obtain the local anomaly detection results, each score map is upsampled to the image resolution, and each score map is element-wise added as follows:

[0114]

[0115] Among them, U k (·) represents the upsampling operation of the kth anomaly map, and Mlocal represents the anomaly score map obtained by the local reconstruction branch;

[0116] The filtered K-level embedded features are input into the trained global feature reconstruction branch to obtain the global anomaly detection results, each score map is upsampled to the image resolution, and each score map is element-wise added as follows:

[0117]

[0118] Among them, M 全局 Represents the anomaly score map obtained by the global reconstruction branch;

[0119] The local anomaly detection results and the global anomaly detection results are combined to obtain the final anomaly detection result M, as follows:

[0120] M=M 局部 +M 全局 .

[0121] In one embodiment, in order to remove noise in the graph, a Gaussian filter is used to smooth the abnormal graph M.

[0122] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. An image anomaly detection method integrating local and global feature reconstruction, characterized in that: The following steps are involved: S1. Obtaining image data, including obtaining public data set images and simulated abnormal sample images, where the public data set images correspond to the simulated abnormal sample images one by one; S2. In the training phase, the public dataset images and the simulated abnormal sample images are input into the feature extractor pre-trained on ImageNet to obtain multi-level embedded features of the public dataset images and their corresponding simulated abnormal sample images respectively; S3, in the training phase, the multi-level embedded features of the acquired public dataset images and simulated abnormal sample images are input into the filter module to obtain the features of each level after filtering of the public dataset images and simulated abnormal sample images; S4, input the features of each level of the public dataset image into the local feature reconstruction branch, and obtain the reconstructed features through the local bottleneck embedding module and decoder; S5, input the highest level features among the features of each level of the public dataset image into the global feature reconstruction branch, and obtain the reconstructed features through the global bottleneck embedding module and decoder; S6. Construct an overall loss, and train the filter module and the global feature reconstruction branch and the local feature reconstruction branch according to the overall loss; S7. In the inference stage, only the test set samples in the public dataset images are used to input the feature extractor, the trained filter module, and the global feature reconstruction branch and the local feature reconstruction branch. The anomaly detection and positioning results are calculated by using the filtering features of each level of the samples in the test set of the public dataset images and the global reconstruction features obtained in the global feature reconstruction branch and the local reconstruction features obtained in the local feature reconstruction branch.

2. The image anomaly detection method based on local-global feature fusion reconstruction according to claim 1, characterized in that: In step S1, a public data set for logical anomaly and structural anomaly detection is obtained, and a simulated anomaly sample image is obtained through the public data set image using an anomaly simulation strategy, which specifically includes the following steps: S1.1, generate noise, obtain the noise binarization mask through the threshold; at the same time, binarize the input image to obtain the corresponding mask; then, obtain the final mask image by performing element-wise product of the two masks obtained; S1.

2. Randomly select a texture mask image from the public texture dataset DTD, and perform dot multiplication to obtain defects containing texture information. For structural anomalies, randomly adjust the public dataset image first, and then divide it into multiple identical grids, and then randomly arrange them to obtain simulated structural anomalies. Randomly use these two methods to obtain noise images corresponding to texture information defects or structural anomalies. S1.3, performing element-wise product of the mask image and the noise image to obtain a noise foreground image; S1.4, invert the mask image obtained in step S1.1, and then multiply it element by element with the public dataset image, and then superimpose it with the noise foreground image to obtain the final abnormal simulation image.

3. The image anomaly detection method based on local-global feature reconstruction fusion according to claim 1, characterized in that: In step S2, a feature extractor pre-trained on ImageNet is used as an encoder, positive samples in the public dataset images are input into the encoder, the first K layers of the encoder network are fixed, and K output features of different sizes are obtained, i.e., multi-level embedded features of the public dataset images; The simulated abnormal images corresponding to the public dataset images are input into the encoder, the first K layers of the encoder network are fixed, and K output features of different sizes are obtained, namely, the K-level embedded features of the simulated abnormal images.

4. The image anomaly detection method based on local-global feature fusion reconstruction according to claim 3, characterized in that: In step S3, the multi-level embedded features of the public dataset images and the simulated abnormal sample images are input into the filter module to obtain the filtered K-level features; each layer of the filter module is structured as L convolution blocks stacked in sequence.

5. The image anomaly detection method of claim 4, characterized in that: Based on the K-level features after filtering the public dataset images and the simulated abnormal sample images, the consistency loss is calculated as follows: Among them, fn i,k ,fa i,k Respectively represent the i-th public dataset image x i and the corresponding ith simulated abnormal sample image ξ(x i ) Feature output of the k-th layer encoder: F k represents the kth layer of the filter module; The contrast loss is calculated by simulating the multi-level features obtained by the encoder of abnormal images and the multi-level features obtained after filtering the public dataset images: fn i,k ,fa i,k Respectively represent the i-th public dataset image x i and the corresponding ith simulated abnormal sample image ξ(x i ) Feature output of the k-th layer encoder: F k Represents the k-th layer of the filter module.

6. The image anomaly detection method of claim 4, characterized in that: In step S4, the public dataset image input in step S1 is filtered by the filter module to obtain the features of each level, which are input into the local feature reconstruction branch, and the reconstructed features are obtained through the local bottleneck embedding module and the local branch decoder, as follows: The multi-layer features obtained after filtering the public dataset image by the filter module are input into the local bottleneck embedding module. First, the first K-1 layers of features of the K-level features are downsampled through the convolutional layer in the local bottleneck embedding module. Then, the obtained multi-level representations are connected and fused to finally obtain the compressed features. The local branch decoder in the local feature reconstruction branch reconstructs the compressed features back into filtered features of different levels to obtain local reconstructed features; the local branch decoder and the feature extractor in step S2 have a symmetrical structure. In the local branch decoder, the downsampling operation used in the feature extractor is replaced by upsampling.

7. The image anomaly detection method of claim 6, characterized in that: In step S5, the highest level feature among the multi-layer features obtained after the input image is filtered by the filter module is input into the global feature reconstruction branch, and the reconstructed feature is obtained through the global bottleneck embedding module and the global branch decoder, including: The highest-level features obtained after the input image features are filtered by the filter module are input into the global bottleneck embedding module. The highest-level filtered features are first reconstructed through the residual block, and then the features are compressed through the global maximum pooling operation to obtain a more compact feature vector; finally, the obtained feature vector is upsampled to 8 times using deconvolution to obtain compressed features; The global branch decoder of the global feature reconstruction branch then reconstructs the compressed features back into filtered features of different levels to obtain global reconstructed features; the global branch decoder in the global feature reconstruction branch has the same architecture as the local branch decoder in the local reconstruction branch, but its input is only the highest level of abstract feature representation.

8. The image anomaly detection method of claim 7, characterized in that: The cosine distance is calculated by filtering the local reconstruction features and the multi-layer features obtained after filtering the public dataset image input in step S1 to generate the corresponding feature map and calculate the reconstruction loss, as follows: in, represents the filtering features output by the k-th filter module, Represents the reconstructed features of the first decoder at the kth layer of the local feature reconstruction branch, represents the feature map output by the kth layer of the local branch decoder in the local reconstruction branch, L 局部 represents the reconstruction loss of the local feature reconstruction branch, h k represents the height of the feature output of the kth layer of the local branch decoder in the local reconstruction branch, w k Represents the width of the feature output by the k-th layer of the local branch decoder in the local reconstruction branch; The cosine distance is calculated by filtering the global reconstruction features and the multi-layer features obtained after filtering the public dataset image input in step S1 to generate the corresponding feature map and calculate the reconstruction loss, as follows: in, represents the filtering features output by the k-th filter module, The global branch decoder represents the reconstructed features of the global feature reconstruction branch at the kth layer, represents the feature map output by the kth layer of the global branch decoder in the global reconstruction branch, L 全局 represents the reconstruction loss of the global feature reconstruction branch, h k represents the height of the feature output of the kth layer of the global branch decoder in the global reconstruction branch, w k Represents the width of the feature output by the k-th layer of the global branch decoder in the global reconstruction branch.

9. The image anomaly detection method based on local-global feature fusion reconstruction according to any one of claims 5 and 8, characterized in that: In step S6, the overall loss L is specifically as follows: L = L - 致 +L 对比 +L 重建 ; L 重建 =L 局部 +L 全局 .

10. The image anomaly detection method of any one of claims 1 to 9, characterized in that: In step S7, in the inference stage, only the test set samples in the public dataset images are used to input the reconstruction model after training, and the anomaly detection and positioning results are calculated by using the filtering features of the samples in the test set of the public dataset images at all levels and the global reconstruction features obtained in the global feature reconstruction branch and the local reconstruction features obtained in the local feature reconstruction branch, including: Input the samples of the test set in the public dataset image to the pre-trained feature extractor to obtain K-level embedded features; Input the K-level embedding features into the trained filter module to obtain the filtered K-level embedding features; The filtered K-level embedded features are input into the trained local feature reconstruction branch to obtain the local anomaly detection results, each score map is upsampled to the image resolution, and each score map is element-wise added as follows: Among them, U k (·) represents the upsampling operation of the kth anomaly map, and Mlocal represents the anomaly score map obtained by the local reconstruction branch; The filtered K-level embedded features are input into the trained global feature reconstruction branch to obtain the global anomaly detection results, each score map is upsampled to the image resolution, and each score map is element-wise added as follows: Among them, M 全局 Represents the anomaly score map obtained by the global reconstruction branch; The local anomaly detection results and the global anomaly detection results are combined to obtain the final anomaly detection result M, as follows: M=M 局部 +M 全局 。

Images