Image anomaly detection method and device based on hierarchical vector quantization of conditional transmission

Abstract

The invention discloses an image anomaly detection method and device based on hierarchical vector quantization of conditional transmission, and relates to the technical field of image processing, and the method comprises the steps: inputting a to-be-processed image into a trained extraction model, and obtaining an image visual feature; performing non-local multi-level feature aggregation on the image visual features based on a cascade encoder, and inputting the aggregated features of each layer to a corresponding layer based on vector quantization to obtain multi-layer quantized visual features; the image visual features and the multi-layer quantitative visual features are fused successively based on a cascaded decoder, and then reconstruction is carried out through a preset hybrid expert model; and on the basis of the reconstructed image visual features, abnormal scores are obtained by measuring condition-based transmission layered feature differences, and abnormal detection and positioning are realized. The detection precision can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an image anomaly detection method and device based on conditional transmission layered vector quantization. Background Art

[0002] Anomaly detection is an important task that is increasingly used in various fields such as video surveillance, industrial inspection, and medical image analysis. Due to the scarcity of abnormal samples, unsupervised anomaly detection methods have received widespread attention. This method only models the normal data distribution and then identifies samples that deviate from the normal distribution as abnormal samples. Common methods adopt a one-to-one correspondence scheme to train models for different categories of objects separately. This method consumes a lot of time and memory in practical applications and is not conducive to object categories with large intra-category diversity. In addition, many reconstruction-based models assume that the latent representation is continuous, which may cause both normal and abnormal samples to be reconstructed equally well and difficult to distinguish, the so-called "same shortcut" problem; discrete variational autoencoders (VAEs) using vector quantization may encounter the codebook collapse problem, that is, part of the latent code may no longer work during training, limiting the representation ability of the model; most anomaly scoring methods are limited to the observation space and may not be robust and discriminative enough for complex data distributions.

[0003] In the prior art, Karsten RCTh, Latha Pemula, Joaquin Zepeda, Bernhard Thomas Brox, Peter Gehler and others published an algorithm PatchCore for industrial anomaly detection based on deep learning and nearest neighbor search in their paper "Towards TCTal Recall Industrial Anomaly Detection" (2022 IEEE / CVF Conference on Computer Vision and Pattern Recognition). This method uses a pre-trained convolutional neural network to extract features from normal samples, and selects the most representative feature points through the KNNGreedy CoreSet sampling method to build a core set, and then detects whether the new sample is abnormal based on whether the distance between the feature of the new test sample and the feature in the core set exceeds a certain threshold. This method can reduce the amount of computation and storage requirements without sacrificing detection performance through the core set, adapt to different industrial scenarios and different types of anomalies, and has good versatility. However, the performance of the algorithm depends to a large extent on the quality and diversity of normal samples. The size of the core set and the selection of the anomaly threshold may affect the performance of the algorithm, and need to be adjusted according to the specific application scenario, which restricts the performance of the algorithm. Zhiyuan You, Lei Cui, Yujun Shen, Kai Yang, XinLu, Yu Zheng, Xinyi Le and others published a reconstruction-based anomaly detection method UniAD in their paper "A Unified Model for Multi-class Anomaly Detection" (2022 Conference and Workshop on Neural Information Processing Systems). This method is designed to use a single model to handle all categories of anomaly detection tasks. A reconstruction model is trained on normal samples. If the reconstruction can only be successful on normal samples, there will be large errors for abnormal samples.UniAD adopts the Layer-wise QueryEmbedding method, that is, query embedding is added to each layer of the decoder to enhance the model's sensitivity to anomalies. The model also introduces the Neighbor Masked Attention mechanism, which restricts the spatial vector from using its own and neighboring information, forcing the model to understand the information of points through farther spatial vectors, so as to better fit the distribution of normal samples; UniAD also uses the Feature Jittering strategy, that is, adding noise to the input features and using the unnoised features as the reconstruction target, which helps the model remove noise and understand normal samples; however, the multiple innovative technologies adopted by UniAD may increase the complexity and training difficulty of the model. Although UniAD performs well on specific datasets, its generalization ability on different types of datasets remains to be verified.

[0004] Existing technologies use a one-to-one correspondence scheme to train models for different categories of objects separately. This method consumes a lot of time and memory in practical applications, and is not conducive to object categories with large intra-category diversity. In addition, many reconstruction-based networks assume that the latent representation is continuous, which may cause normal and abnormal samples to be reconstructed equally well and difficult to distinguish, the so-called "same shortcut" problem, which restricts the performance of the model. Therefore, it is urgent to provide a new image anomaly detection method to overcome the above technical problems. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides an image anomaly detection method and device based on conditional transmission layered vector quantization. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0006] In a first aspect, the present invention provides an image anomaly detection method based on conditional transmission layered vector quantization, comprising:

[0007] Get the image to be processed;

[0008] Input the image to be processed into the trained extraction model to obtain the image visual features;

[0009] The cascade-based encoder performs non-local multi-level feature aggregation on the image visual features, and inputs the aggregated features of each layer into the corresponding vector quantization-based layer to obtain multi-layer quantized visual features;

[0010] The cascade-based decoder successively fuses the image visual features with multi-layer quantized visual features, and then reconstructs them through a preset hybrid expert model;

[0011] Based on the reconstructed image visual features, the anomaly score is obtained by measuring the difference of hierarchical features based on conditional transmission, thus achieving anomaly detection and positioning.

[0012] In a second aspect, the present invention further provides an image anomaly detection device based on conditional transmission layered vector quantization, comprising:

[0013] An image acquisition module, used for acquiring an image to be processed;

[0014] An image preprocessing module is used to input the image to be processed into a trained extraction model to obtain image visual features;

[0015] A first image processing module, configured to perform non-local multi-level feature aggregation on the image visual features based on a cascaded encoder, and input each layer of aggregated features into a corresponding layer based on vector quantization to obtain multi-layer quantized visual features;

[0016] A second image processing module, configured to successively fuse the image visual features with the multi-layer quantized visual features based on a cascaded decoder, and then reconstruct them through a preset hybrid expert model;

[0017] The anomaly detection and positioning device is used to obtain anomaly scores based on reconstructed image visual features by measuring the difference of conditionally transmitted hierarchical features, thereby realizing anomaly detection and positioning.

[0018] Beneficial effects of the present invention:

[0019] The present invention provides an image anomaly detection method and device based on conditional transmission and hierarchical vector quantization. The method and device obtain image visual features by inputting an image to be processed into a trained extraction model; a cascade-based encoder performs non-local multi-level feature aggregation on the image visual features, and inputs the aggregated features of each layer into a corresponding vector quantization-based layer to obtain multi-layer quantized visual features; a cascade-based decoder successively fuses the image visual features with the multi-layer quantized visual features, and then reconstructs them through a preset hybrid expert model; based on the reconstructed image visual features, anomaly scores are obtained by measuring the differences in hierarchical features based on conditional transmission, thereby realizing anomaly detection and positioning; in this way, the present invention can improve detection accuracy.

[0020] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flow chart of an image anomaly detection method based on conditional transmission and hierarchical vector quantization provided by an embodiment of the present invention;

[0022] Figure 2 is a schematic diagram of a transformer model of layered vector quantization provided by an embodiment of the present invention;

[0023] Figure 3 It is a schematic diagram of a standard multi-head self-attention mechanism and a multi-head cross-attention mechanism in an encoder provided by an embodiment of the present invention;

[0024] Figure 4 It is a qualitative result display diagram of anomaly positioning on the MVTec-AD data set obtained by using the method provided by the embodiment of the present invention;

[0025] Figure 5 It is a quantitative result display diagram of anomaly detection and positioning on the MVTec-AD data set obtained by using the method provided by the embodiment of the present invention;

[0026] Figure 6 It is a quantitative result display diagram of anomaly detection and positioning on the VisA data set obtained by using the method provided by the embodiment of the present invention;

[0027] Figure 7 It is a qualitative result display diagram of abnormality positioning on the VisA data set obtained by using the method provided by the embodiment of the present invention;

[0028] Figure 8 This is a graph showing the results of anomaly detection on the CIFAR-10 dataset in a “one-to-many” setting obtained using the method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0030] In view of the defects existing in the prior art, the present invention provides an image anomaly detection method and device based on conditional transmission of hierarchical vector quantization, which adopts a variable automatic encoding framework for unsupervised anomaly detection, called a hierarchical vector quantization transformer model, to achieve the purpose of the present invention, that is, to establish a unified framework for multiple categories of detection objects to enhance versatility and save costs; and to alleviate the code set collapse problem by constructing a hierarchical framework; to use a vector quantization method to avoid the "same shortcut" problem; and to introduce a prototype-oriented hierarchical conditional transmission method to better adjust the prototype and evaluate the anomaly score in a hierarchical manner.

[0031] See also Figure 1 , Figure 1 is a flow chart of an image anomaly detection method based on conditional transmission and hierarchical vector quantization provided by an embodiment of the present invention. Figure 2 1 is a schematic diagram of a transformer model of layered vector quantization provided by an embodiment of the present invention. The image anomaly detection method of layered vector quantization based on conditional transmission provided by the present invention includes:

[0032] S101: Obtain an image to be processed.

[0033] Specifically, in this embodiment, the image to be processed is represented as Represents an n-dimensional vector space, H represents the height of the image, and W represents the width of the image.

[0034] S102: Input the image to be processed into the trained extraction model to obtain the image visual features.

[0035] Specifically, in this embodiment, a pre-trained extraction model is used to obtain N image visual features. C represents the number of image channels, φ 0 represents the parameters of the encoder, and f represents the encoder.

[0036] The image visual features are input into the hierarchical vector quantization transformer model, that is, the hierarchical vector quantization Transformer model, please continue to refer to Figure 2 , Input represents the image to be processed, EfficientNet is the trained extraction model, and the layered vector quantization Transformer model includes a cascaded vanilla Transformer encoder. The encoder parameters are represented by φ l , Position Embedding represents the position encoding in the encoder, Figure 2 Four cascaded encoders are set up, namely Vanilla Trans Enc Layer1, Vanilla Trans Enc Layer2, Vanilla Trans Enc Layer3 and Vanilla Trans Enc Layer4. Figure 2 There are 5 layers based on vector quantization, namely VQLayer0, VQ Layer1, VQ Layer2, VQ Layer3 and VQ Layer4. Figure 2 Four cascaded decoders are set up, namely VQTrans Dec Layer1, VQTrans Dec Layer2, VQTrans Dec Layer3, and VQTrans DecLayer4; the output of each level of encoder is connected to the input of its corresponding vector quantization-based layer, and the output of each layer of vector quantization-based layer is connected to the input of its corresponding decoder.

[0037] S103, performing non-local multi-level feature aggregation on the image visual features based on the cascade encoder, inputting the aggregated features of each layer into the corresponding vector quantization-based layer to obtain multi-layer quantized visual features.

[0038] Specifically, in this embodiment, each level of encoder embeds the multi-layer region into the following code:

[0039]

[0040] Among them, h l represents the features aggregated by the l-th level encoder, l represents the index of the encoder, l∈[1,...,L], φ l represents the parameters of the encoder, h l-1 represents the features aggregated by the l-1th level encoder, f represents the encoder, represents the feature size, N represents the number of features, and C represents the number of channels.

[0041] In this embodiment, based on the vector quantization layer, the aggregated features of each layer are input to the corresponding vector quantization layer to obtain multi-layer quantized visual features, that is, the aggregated features Hierarchical Quantization as a Learnable Codebase Prototype in That is, this embodiment hierarchically aggregates the features h l Replace with codebook E l The most similar prototype in , i.e., the quantized visual feature z l , whose expression is:

[0042]

[0043] Among them, γ l (·) represents the hierarchical embedding function, [·] represents the concatenation operation, θ represents the global quantization vector, represents the corresponding prototype in the learnable code base of the first layer, r represents the prototype identifier in the learnable code base of the first layer, represents the corresponding prototype in the last layer L of the learnable code base, j represents the prototype identifier in the last layer L of the learnable code base, and h L represents the features aggregated by the last level encoder, and k represents the identity of the most similar prototype found from the learnable code base at the corresponding level.

[0044] like Figure 2 As shown, the feature input to Vanilla Trans Enc Layer1 is h 0 , the feature input to VQ Layer0 is h 0 ; The operations of the first L-1 level encoders are completely different from those of the Lth level encoder, and the two need to be represented separately.

[0045] S104 , the cascade-based decoder successively fuses the image visual features with the multi-layer quantized visual features, and then reconstructs them through a preset hybrid expert model.

[0046] Specifically, in this embodiment, given the quantized visual feature z of the lth layer l and the corresponding output d of the previous decoder l+1 , the operation of each decoding layer can be expressed as:

[0047]

[0048] q l =MSA(query=W q d l+1 ,key=W k d l+1 ,value=W v d l+1 )+d l+1 ;

[0049] in, represents a custom variable, MCA(·) represents a multi-head cross attention mechanism, query represents a query vector, and W q ' represents the weight matrix of the query, key represents the key vector, W k ' represents the weight matrix of key, value represents the value vector, W v 'Represents the weight matrix of value, d l represents the output feature of the l+1th level decoder, FFN(·) represents the feedforward neural network, q l represents the query vector of the l-th level decoder, MSA(·) represents the multi-head self-attention mechanism, W q Another weight matrix representing the query, W k Another weight matrix representing the key, W v Another weight matrix representing value.

[0050] It should be noted that MSA(·) and MCA(·) have the same structure as the standard multi-head self-attention mechanism and multi-head cross-attention mechanism in the Vanilla Transformer encoder, see Figure 3 .

[0051] In this embodiment, the decoded features are reconstructed by a hybrid expert model. The preset hybrid expert model includes a clusterer, M codebooks, and M reconstruction experts. The preset hybrid expert model distributes the features to M clusters. The reconstructed image visual features are expressed as:

[0052]

[0053] Among them, centroid j represents the centroid of the jth cluster, |d 0-centroid j | 2 Represents the input feature d 0 The Euclidean distance to the cluster centroid, Ψ m represents the reconstruction expert corresponding to the closest cluster, represents the reconstructed image visual features output by the decoder, and m represents the minimum Euclidean distance between the features input to the decoder and the cluster centroid.

[0054] S105. Based on the reconstructed image visual features, anomaly scores are obtained by measuring the differences in conditionally transmitted hierarchical features to achieve anomaly detection and positioning.

[0055] Specifically, in this embodiment, by measuring the hierarchical feature differences based on conditional transmission, a calibration anomaly score map is perfected by the prototype-oriented module to achieve anomaly detection and location.

[0056] In this embodiment, at the lth level, the codebook of each category contains a set of prototypes The index l is omitted below; the N regions of image visual features are represented as empirical distributions:

[0057]

[0058] in, represents the features sampled from the latent variable.

[0059] In this embodiment, the distribution of normal prototypes can also be expressed as an empirical distribution:

[0060]

[0061] make and Represents the empirical distribution and Two sets of discrete objects sampled in, where the forward CT distance It is expressed as:

[0062]

[0063] Forward transmission probability π n (z j |y i ; φ) is expressed as:

[0064]

[0065] Where c(·,·) represents the Euclidean distance, d φ (·,·) represents a nonlinear measurement function, such as a multilayer perceptron (MLP).

[0066] Backward CT distance C φ The definition of (z→y) is similar to the forward CT distance and is expressed as:

[0067]

[0068] Backward transmission probability π m (y i |z j ; φ) is expressed as:

[0069]

[0070] In this embodiment, the average loss of prototype-oriented hierarchical conditional transmission is expressed as:

[0071]

[0072] Among them, φ represents the optimizable parameter and ρ represents the trade-off factor that controls the strength of the forward and backward directions.

[0073] The anomaly scores calculated by Transformer-based methods are always affected by suboptimal distance measurements, which are usually calculated by point-wise L2 normalization of the reconstructed differences, i.e. In this embodiment, the multi-level anomaly score is calibrated as To achieve better anomaly detection effect.

[0074] This embodiment also includes optimizing the loss function of the layered vector quantization Transformer model. The optimized loss function of the layered vector quantization Transformer model is expressed as:

[0075]

[0076] Among them, sg(·) indicates the stop gradient operation, P x represents the category label, β l and α l All of them represent hyperparameters. The first term refers to the reconstruction loss. The second term in the L-layer summation range is the hierarchical prototype loss. l Push visual features h l , and the third term is the hierarchical commitment loss, which encourages the encoder h l The output of is kept close to the selected prototype, the fourth term is the prototype-oriented hierarchical conditional transfer CT loss, and the last term is the cross entropy loss, which is used to train the classifier to adaptively switch the appropriate reconstruction expert and codebook. The present invention uses the exponential moving average method to update the codebook.

[0077] In summary, the present invention provides an image anomaly detection method based on conditional transmission and hierarchical vector quantization, which obtains image visual features by inputting the image to be processed into a trained extraction model; a cascade-based encoder performs non-local multi-level feature aggregation on the image visual features, and inputs the aggregated features of each layer into the corresponding vector quantization-based layer to obtain multi-layer quantized visual features; a cascade-based decoder successively fuses the image visual features with the multi-layer quantized visual features, and then reconstructs them through a preset hybrid expert model; based on the reconstructed image visual features, anomaly scores are obtained by measuring the differences in hierarchical features based on conditional transmission, thereby realizing anomaly detection and positioning; in this way, the detection accuracy can be improved.

[0078] Based on the same inventive concept, the present invention further provides an image anomaly detection device based on conditional transmission layered vector quantization, which is used to implement an image anomaly detection method based on conditional transmission layered vector quantization provided in the above embodiment of the present invention. The embodiment of the method is referred to above and will not be described in detail here; the device comprises:

[0079] An image acquisition module, used for acquiring an image to be processed;

[0080] An image preprocessing module is used to input the image to be processed into a trained extraction model to obtain image visual features;

[0081] A first image processing module, configured to perform non-local multi-level feature aggregation on the image visual features based on a cascaded encoder, and input each layer of aggregated features into a corresponding layer based on vector quantization to obtain multi-layer quantized visual features;

[0082] A second image processing module, configured to successively fuse the image visual features with the multi-layer quantized visual features based on a cascaded decoder, and then reconstruct them through a preset hybrid expert model;

[0083] The anomaly detection and positioning device is used to obtain anomaly scores based on reconstructed image visual features by measuring the difference of conditionally transmitted hierarchical features, thereby realizing anomaly detection and positioning.

[0084] In an optional embodiment of the present invention, the effectiveness of the method and device provided in the above embodiment of the present invention is verified by simulation experiments, specifically:

[0085] The simulation experiment of this embodiment uses an NVIDIA GeForce RTX 3080 10GB operating system and Python as the simulation language. The experimental data used is a widely used industrial anomaly detection dataset "MVTec-AD", which contains 15 categories and covers more than 5k high-resolution images, including objects and textures; a recently released large-scale dataset VisA, consisting of 9621 normal images and 1200 abnormal high-resolution images, including images with complex structures, sparsely placed objects, and various types of objects, among which anomalies include scratches, dents, color spots, cracks, and structural defects; a classic image classification dataset CIFAR-10 containing 10 categories, of which the present invention regards 5 categories as normal categories and the remaining 5 categories as abnormal categories.

[0086] Simulation experiment 1

[0087] The input image size of the MVTec-AD dataset is 224×224×3. After inputting the pre-trained EfficientNet model, the feature map becomes 14×14×272, that is, the region size is 16; then, the present invention reduces the channel dimension of each region to 256, and then feeds it into 4 layers of enc, and then into the corresponding 4 layers of dec. The present invention uses the AdamW optimization algorithm with a weight decay of 0.0001 for optimization. The model of the present invention was trained 1000 times on 2 GPUs (NVIDIA GeForce RTX 3080 10GB) with a batch size of 16. The learning rate is initialized to 1×10 -4 , which drops by 0.1 after 800 epochs. In addition to the pre-trained EfficientNet model, the model of the present invention is trained from scratch.

[0088] See also Figure 4 , Figure 4 This is a qualitative result display diagram of anomaly localization on the MVTec-AD dataset obtained by using the method provided by the embodiment of the present invention. For object anomalies (left picture) and texture damage (right picture), the method of the present invention can successfully restore the abnormal area and its corresponding normal mode. It can be seen that the model with hierarchical vector quantization layer can better generate normal mode in the abnormal area, thereby achieving more accurate anomaly localization. Please refer to Figure 5 , Figure 5 is a quantitative result display diagram of anomaly detection and positioning on the MVTec-AD dataset obtained by using the method provided in the embodiment of the present invention, such as Figure 5 As shown, the performance of anomaly detection and localization of the model of the present invention is generally better than all competitors.

[0089] Simulation experiment 2

[0090] See also Figure 6 , Figure 6 is a quantitative result display diagram of anomaly detection and positioning on the VisA dataset obtained by using the method provided in the embodiment of the present invention. Figure 6 We demonstrate the superior performance of our model compared to three other reconstruction-based methods under a unified setting. Figure 7 , Figure 7 It is a qualitative result display diagram of abnormality positioning on the VisA data set obtained by using the method provided by the embodiment of the present invention;

[0091] Figure 7 The excellent performance of the reconstruction and positioning technology of the present invention in different categories is demonstrated. Even in multi-instance scenarios, the model of the present invention can effectively restore abnormal areas to normal states. The reconstructed images have high fidelity and are very consistent with the appearance of normal areas, and the expected effect is achieved in restoring abnormal areas.

[0092] Simulation experiment three

[0093] See also Figure 8 , Figure 8 This is a graph showing the results of anomaly detection on the CIFAR-10 dataset under a "one-to-many" setting obtained using the method provided by an embodiment of the present invention. 01234 indicates that normal samples include images of categories 0, 1, 2, 3, and 4, while images of categories 5, 6, 7, 8, and 9 are abnormal images. In order to improve statistical robustness, the present invention repeats the splitting to obtain four combinations, see Figure 8 , our model outperforms all other competitors in the many-to-many setting for each dataset split.

[0094] This embodiment realizes the learning of discrete normal representation by extracting prototypes, and proposes a vector quantization-based Transformer model to solve the "same shortcut" problem by inducing large feature differences in abnormal points; a vector quantization-based hierarchical method with a hybrid expert model is developed to overcome the "prototype collapse" problem and effectively utilize multi-level feature representation to maximize the available nominal information;

[0095] This embodiment develops a prototype-oriented hierarchical learning and anomaly scoring method to guide prototype learning and cleverly measure feature-level anomaly scores, thereby robustly and accurately identifying anomalies. Extensive experiments have demonstrated that this embodiment achieves state-of-the-art performance in anomaly detection and localization, and enhances interpretability through prototype visualization.

[0096] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant is intended to cover non-exclusive inclusion, so that the article or device including a series of elements includes not only those elements, but also other elements that are not explicitly listed. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the article or device including the elements. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The orientation or position relationship indicated by "up", "down", "left", "right", etc. is based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention.

[0097] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification.

[0098] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for image anomaly detection based on conditional transmission layered vector quantization, characterized in that: include: Get the image to be processed; Inputting the image to be processed into a trained extraction model to obtain image visual features; A cascade-based encoder performs non-local multi-level feature aggregation on the image visual features, and inputs the aggregated features of each layer into a corresponding vector quantization-based layer to obtain multi-layer quantized visual features; The cascade-based decoder successively fuses the image visual features with the multi-layer quantized visual features, and then reconstructs them through a preset hybrid expert model; Based on the reconstructed image visual features, the anomaly score is obtained by measuring the difference of hierarchical features based on conditional transmission, thus achieving anomaly detection and positioning.

2. The image anomaly detection method based on conditional transmission layered vector quantization according to claim 1 is characterized in that: Also includes: Construct a transformer model for hierarchical vector quantization; Among them, the hierarchical vector quantization transformer model includes the cascaded encoder, the vector quantization-based layer and the cascaded decoder, the output of each level of the encoder is connected to the input of its corresponding vector quantization-based layer, and the output of each layer of the vector quantization-based layer is connected to the input of its corresponding decoder.

3. The image anomaly detection method based on conditional transmission layered vector quantization according to claim 1, characterized in that: The features aggregated by the encoder are represented as: Among them, h l represents the features aggregated by the l-th level encoder, l represents the index of the encoder, l∈[1,...,L], φ l represents the parameters of the encoder, h l-1 represents the features aggregated by the encoder at level l-1, f represents the encoder, represents the feature size, N represents the number of features, and C represents the number of channels.

4. The image anomaly detection method based on conditional transmission layered vector quantization according to claim 1, characterized in that: The aggregated features of each layer are input into the corresponding vector quantization-based layer to obtain multi-layer quantized visual features, including: Vector quantization-based layers quantize aggregated features hierarchically into learnable code bases Prototype in That is, quantify the visual feature z l , whose expression is: Among them, γ l (·) represents the hierarchical embedding function, [·] represents the concatenation operation, θ represents the global quantization vector, represents the corresponding prototype in the learnable code base of the first layer, r represents the prototype identifier in the learnable code base of the first layer, represents the prototype corresponding to the last layer L of the learnable code base, j represents the prototype identifier of the last layer L of the learnable code base, h L represents the features aggregated by the last level encoder, and k represents the identity of the most similar prototype found from the learnable code base at the corresponding level.

5. The image anomaly detection method based on conditional transmission layered vector quantization according to claim 1, characterized in that: The cascade-based decoder successively fuses the image visual features with the multi-layer quantized visual features, comprising: According to the quantized visual features z at the lth layer l And the feature d output by the previous decoder l+1 , the operation of each decoder is expressed as: in, represents a custom variable, MCA(·) represents a multi-head cross attention mechanism, query represents a query vector, and W q ' represents the weight matrix of the query, key represents the key vector, W k ' represents the weight matrix of key, value represents the value vector, W v 'Represents the weight matrix of value, d l represents the output feature of the l+1th level decoder, FFN(·) represents the feedforward neural network, q l represents the query vector of the l-th level decoder, MSA(·) represents the multi-head self-attention mechanism, W q Another weight matrix representing the query, W k Another weight matrix representing the key, W v Another weight matrix representing value.

6. The image anomaly detection method based on conditional transmission and layered vector quantization according to claim 1 is characterized in that: The preset hybrid expert model includes a clusterer, M codebooks and M reconstruction experts. The preset hybrid expert model distributes features into M clusters. The reconstructed image visual features are expressed as: Among them, centroid j represents the centroid of the jth cluster, |d 0 -centroid j |2 represents the input feature d 0 The Euclidean distance to the cluster centroid, Ψ m represents the reconstruction expert corresponding to the closest cluster, represents the reconstructed image visual features output by the decoder, and m represents the minimum Euclidean distance between the features input to the decoder and the cluster centroid.

7. The image anomaly detection method based on conditional transmission layered vector quantization according to claim 1, characterized in that: The average loss function based on conditional transmission for: Among them, L φ,ρ represents the loss function of the conditional transmission distance, C φ (y→z) represents the forward CT distance, C φ (z→y) represents the backward CT distance, Represents the empirical distribution Discrete objects sampled in Representing distributed experience The discrete object sampled from , y represents the empirical distribution Any discrete object in , z represents the empirical distribution is any discrete object in , and ρ represents a trade-off factor that controls the strength of the forward and backward directions.

8. The image anomaly detection method based on conditional transmission layered vector quantization according to claim 1, characterized in that: It also includes: calibrating the anomaly score, and the calibration expression is: Among them, s cab represents the total abnormality score, s org represents the reconstruction loss between the image visual features and the reconstructed image visual features, λ represents the hyperparameter, Represents the transmission distance of two distributions at layer l.

9. The image anomaly detection method based on conditional transmission layered vector quantization according to claim 2, characterized in that: Also includes: The loss function of the layered vector quantization Transformer model is optimized, and the loss function of the optimized layered vector quantization Transformer model is expressed as: in, represents the loss function, h 0 Represents the visual features of the image, represents the reconstructed image visual features output by the decoder, h l represents the features aggregated by the encoder at the first level, sg(·) indicates the stop gradient operation, and e l represents the prototype in the learnable code base, β l represents the hyperparameter, α l represents the hyperparameter, represents the average loss function based on conditional transmission, represents the cross entropy loss, P x Represents the category label.

10. An image anomaly detection device based on conditional transmission layered vector quantization, characterized in that: include: An image acquisition module, used for acquiring an image to be processed; An image preprocessing module is used to input the image to be processed into a trained extraction model to obtain image visual features; A first image processing module, configured to perform non-local multi-level feature aggregation on the image visual features based on a cascaded encoder, and input each layer of aggregated features into a corresponding layer based on vector quantization to obtain multi-layer quantized visual features; A second image processing module, configured to successively fuse the image visual features with the multi-layer quantized visual features based on a cascaded decoder, and then reconstruct them through a preset hybrid expert model; The anomaly detection and positioning device is used to obtain anomaly scores based on reconstructed image visual features by measuring the difference of conditionally transmitted hierarchical features, thereby realizing anomaly detection and positioning.

Images