Zero sample anomaly detection method based on multilevel feature fusion and residual de-confusion

Abstract

The invention provides a zero sample anomaly detection method based on multilevel feature fusion and residual de-confusion, and relates to the field of anomaly detection, the method comprises the following steps: obtaining a to-be-detected image and corresponding text descriptions of a normal category and an anomaly category, respectively extracting initial image features and initial text features, and calculating to obtain an initial zero sample; processing the initial image features through a multi-level feature fusion module to obtain enhanced visual features; performing de-obfuscation processing on the initial zero sample through a residual error correction mechanism to obtain a corrected logic value; and based on the corrected logic value, calculating an image level anomaly detection score, and completing image level anomaly discrimination. And based on the enhanced visual features and the initial text features, calculating a pixel-level anomaly detection score and generating a pixel-level anomaly graph to complete pixel-level anomaly positioning. According to the method and the device, confusion caused by similar categories or fuzzy boundaries is eliminated, so that the accuracy and the robustness of anomaly detection are improved.

Description

Technical Field

[0001] This application relates to the field of anomaly detection, and in particular to a zero-sample anomaly detection method based on multi-level feature fusion and residual de-obfuscation. Background Technology

[0002] Existing anomaly detection methods primarily rely on modeling the distribution of normal data to determine whether a sample is an anomaly. Early methods included reconstruction-based schemes (such as autoencoders or generative adversarial networks), which characterize the degree of anomaly through reconstruction error; memory-based methods, which utilize the proximity of features of normal samples to measure anomaly scores; and discriminative schemes, which train classifiers by faking anomalous samples. These methods can achieve good results with sufficient labeled data or a large number of normal samples, but they generally rely on accurate modeling of the normal distribution and lack adaptability to novel categories and rare anomalies, making it difficult to meet the complex and ever-changing needs of real-world industrial scenarios.

[0003] To reduce reliance on large-scale labeled data, researchers have proposed zero-shot or few-shot anomaly detection frameworks. Typical methods, such as WinCLIP, achieve detection by constructing normal / abnormal text prompts and aligning them with image features, while AnomalyCLIP proposes object-independent prompt learning to enhance the model's generalization across different categories. Subsequent work has explored strategies such as dynamic prompts, anomaly-aware prompts, and learnable class names, leading to significant progress in image-level discrimination and pixel-level localization tasks using CLIP-based anomaly detection methods. However, directly applying CLIP to anomaly detection still has two drawbacks: Inter-class confusion: Existing CLIP-based anomaly detection methods primarily focus on the overall semantic consistency between images and text during pre-training, rather than optimizing for the anomaly detection task. In real-world industrial scenarios, normal and anomalous samples are often highly similar in global semantics. Directly using the zero-sample discrimination results generated by CLIP can easily lead to confusion when similar categories or normal / anomaly boundaries are blurred, resulting in false positives or false negatives and reducing the reliability and stability of the detection results.

[0004] Lack of fine-grained modeling: Anomalies in industrial scenarios often manifest as minor defects in localized areas, requiring models with strong local feature modeling capabilities. However, CLIP's feature extraction focuses on global semantic representation, making it difficult to effectively capture these fine-grained texture and edge features. Consequently, it performs poorly in pixel-level localization tasks, often resulting in inaccurate localization, blurred boundaries, or even complete omission of small-area defects when segmenting abnormal regions, failing to meet the actual needs of industrial quality inspection for high-precision localization. Summary of the Invention

[0005] The purpose of this invention is to address the problems of inter-class confusion and insufficient perception of local defects in CLIP-based anomaly detection, and to provide a zero-sample anomaly detection method based on multi-level feature fusion and residual de-confusion.

[0006] The above-mentioned objective of this application is achieved through the following technical solution: S1: Obtain the image to be detected and the corresponding text descriptions of the normal and abnormal categories; S2: Input the image to be detected and the text description into the visual encoder and the text encoder respectively to extract initial image features and initial text features, and calculate the initial zero sample based on the initial image features and initial text features; S3: The initial image features are processed by a multi-level feature fusion module to fuse visual features at different levels and obtain enhanced visual features; S4: Input the initial zero samples and enhanced visual features into the abnormal residual deobfuscation module, and perform deobfuscation processing on the initial zero samples through the residual correction mechanism to obtain the corrected logical value. ; S5: Based on the corrected logical value, calculate the image-level anomaly detection score to complete the image-level anomaly discrimination; based on the enhanced visual features and the initial text features, calculate the pixel-level anomaly detection score and generate a pixel-level anomaly map to complete the pixel-level anomaly localization.

[0007] Optionally, step S2 includes: The visual encoder and text encoder are encoders based on the pre-trained visual-language model CLIP; The input to the visual encoder includes a basic block token obtained from image segmentation, a static cue token for providing a general prior for the anomaly detection task, and a dynamic image cue token adaptively generated based on the features of the input image. The input to the text encoder includes the text description, a static cue token for providing a general prior for the anomaly detection task, and a dynamic text cue token.

[0008] Optionally, step S3 includes: The multi-level feature fusion module performs the following operations: From the first of the visual encoder Extract multi-scale block representations from a specified level to form a multi-level feature set. ,in , Indicates the first The number of blocks in a layer Indicates feature dimension; Each layer of features The features are input into a lightweight adapter, which maps features from different levels to a unified semantic space through operations including linear transformations, nonlinear activation functions, and normalization layers, resulting in an adapted multi-layer feature set. ; The adapted feature set is weighted and fused to obtain the fused features. The calculation formula is as follows:

[0009] in, For the first Weight coefficients of layer features; The fusion features The final enhanced visual features are obtained through a projection layer. .

[0010] Optionally, step S3 may also include: When the visual encoder is a Vision Transformer, the projection layer is a linear projection layer; when the visual encoder is a ResNet, the projection layer is an attention pooling layer. The different levels of visual features include: low-level features containing fine-grained texture information and high-level features containing global semantic information.

[0011] Optionally, step S4 includes: The abnormal residual deobfuscation module includes: a first adapter, a second adapter, a third adapter, and a residual correction unit; The initial zero sample is processed through the first adapter. The process is performed to learn the underlying obfuscation patterns and obtain the first obfuscation representation. ; The enhanced visual features are applied via a second adapter. The image is processed to extract confusion priors related to its content, resulting in a second confusion representation. ; The first and second confusion representations are jointly modeled using a third adapter to generate confusion estimates. ; Through the residual correction mechanism, the initial zero samples are... Subtracting the confusion estimate yields the corrected logic value. .

[0012] Optionally, step S5 includes: The image-level anomaly detection score The calculation formula is:

[0013] in, and These are the components in the corrected logical value corresponding to the normal category and the abnormal category, respectively; The pixel-level anomaly detection score is calculated as follows: For each block feature in the enhanced visual features Calculate its features compared to normal text. and abnormal text features The cosine similarity is used to obtain the anomaly score of the segment through normalization. The calculation formula is as follows:

[0014] The anomaly scores of all blocks are rearranged into a grid and upsampled and interpolated to the original image resolution to obtain the pixel-level anomaly map. .

[0015] Optionally, step S6 includes: The method learns the objective function during the training phase by jointly optimizing the objective function. Image-level loss Pixel-level loss and similarity regularization loss Weighted sum:

[0016] in, This is the balance coefficient; The image-level loss A focus loss function is used to constrain the corrected logic value. With image-level real labels Consistency; The pixel-level loss The sum of the Dice loss and the focus loss is used to constrain the predicted pixel-level anomaly map. Compared with pixel-level true labeled mask Consistency; The similarity regularization loss The L1 norm loss is used to constrain the corrected logic value. With the initial zero sample Consistency between them.

[0017] An electronic device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to enable the electronic device to perform a zero-sample anomaly detection method based on multi-level feature fusion and residual de-obfuscation.

[0018] A computer-readable storage medium storing instructions that, when executed, perform a zero-sample anomaly detection method based on multi-level feature fusion and residual de-obfuscation.

[0019] The beneficial effects of the technical solution provided in this application are: A multi-level feature fusion module and an anomaly residual de-obfuscation module were designed. The multi-level feature fusion module effectively integrates low-level texture and high-level semantic information, significantly enhancing the representation ability of local fine-grained features. At the same time, the anomaly residual de-obfuscation module performs residual correction on the initial results at the discriminant layer, explicitly eliminating the confusion caused by class similarity or boundary ambiguity, thereby improving the model's discriminative clarity and localization accuracy. Attached Figure Description

[0020] The present application will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is an overall framework diagram of the embodiments of this application; Figure 2 This is a diagram of the prompt layer structure in the embodiments of this application; Figure 3 This is a diagram of the Adapter structure in the embodiments of this application; Figure 4 This is a qualitative result diagram of the method in this application tested on the MVTec-AD dataset in the embodiments of this application; Figure 5 This is a schematic diagram of the electronic device structure in the embodiments of this application. Detailed Implementation

[0021] To provide a clearer understanding of the technical features, objectives, and effects of this application, the specific embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0022] The embodiments of this application provide a zero-sample anomaly detection method based on multi-level feature fusion and residual de-obfuscation.

[0023] Please refer to Figure 1 , Figure 1 This is an overall framework diagram of a zero-shot anomaly detection method based on multi-level feature fusion and residual de-obfuscation, as described in this application, including: S1: Obtain the image to be detected and the corresponding text descriptions of the normal and abnormal categories; S2: Input the image to be detected and the text description into the visual encoder and the text encoder respectively to extract initial image features and initial text features, and calculate the initial zero sample based on the initial image features and initial text features; S3: The initial image features are processed by a multi-level feature fusion module to fuse visual features at different levels and obtain enhanced visual features; S4: Input the initial zero samples and enhanced visual features into the abnormal residual deobfuscation module, and perform deobfuscation processing on the initial zero samples through the residual correction mechanism to obtain the corrected logical value. ; S5: Based on the corrected logical value, calculate the image-level anomaly detection score to complete the image-level anomaly discrimination; based on the enhanced visual features and the initial text features, calculate the pixel-level anomaly detection score and generate a pixel-level anomaly map to complete the pixel-level anomaly localization.

[0024] As one embodiment, this invention proposes a zero-sample anomaly detection method based on multi-level feature fusion and residual de-obfuscation, the overall structure of which is as follows: Figure 1 As shown, it mainly consists of three parts: a zero-sample CLIP with mixed cues, a multi-level feature fusion (MFF) module, and an anomaly residual deconfusion (ARD) module. The input consists of the image to be detected and the corresponding normal / abnormal text description. First, an initial image-text aligned representation is obtained through a visual encoder and a text encoder. Then, the MFF module adapts and fuses image features at different levels to obtain enhanced features that simultaneously contain low-level texture and high-level semantics. Finally, the ARD module, based on the initial zero-sample discrimination results, introduces a residual correction mechanism to eliminate confusion caused by category similarity or boundary blurring, thereby improving the accuracy and robustness of anomaly detection.

[0025] Step S2 includes: The visual encoder and text encoder are encoders based on the pre-trained visual-language model CLIP; The input to the visual encoder includes a basic block token obtained from image segmentation, a static cue token for providing a general prior for the anomaly detection task, and a dynamic image cue token adaptively generated based on the features of the input image. The input to the text encoder includes the text description, a static cue token for providing a general prior for the anomaly detection task, and a dynamic text cue token.

[0026] Step S2 includes: The visual encoder and text encoder are encoders based on the pre-trained visual-language model CLIP; The input to the visual encoder includes a basic block token obtained from image segmentation, a static cue token for providing a general prior for the anomaly detection task, and a dynamic image cue token adaptively generated based on the features of the input image. The input to the text encoder includes the text description, a static cue token for providing a general prior for the anomaly detection task, and a dynamic text cue token.

[0027] Step S3 also includes: When the visual encoder is a Vision Transformer, the projection layer is a linear projection layer; when the visual encoder is a ResNet, the projection layer is an attention pooling layer. The different levels of visual features include: low-level features containing fine-grained texture information and high-level features containing global semantic information.

[0028] This application provides an embodiment as follows, where the initial image features are input. First, the image encoder with hybrid cues takes the input image as input, consisting of basic block tokens derived from image segmentation, and concatenates static and dynamic image cues. The static cues provide general priors relevant to the anomaly detection task, while the dynamic image cues are adaptively generated based on the features of the input image, dynamically adjusting feature representations for different image content. Similarly, the text encoder incorporates static and dynamic cues when inputting text descriptions of normal and anomaly categories. The cue layer structure is as follows: Figure 2 As shown.

[0029] After embedding, the image is divided into block tokens, which are then concatenated with static cues and dynamic image-related cues and fed into an image encoder composed of multiple Transformers. Under the action of the self-attention mechanism, the cue vectors can interact with the image block tokens, thereby improving the image features' ability to perceive anomalies. Through this process, this invention obtains multi-scale block representations from different levels:

[0030] in , Indicates the first The number of blocks in a layer Represents feature dimension. Low-level features. It mainly contains fine-grained information such as edges and textures, while high-level features... It places greater emphasis on semantics and global structure.

[0031] To map features from different levels to a unified semantic space, this invention introduces a lightweight Adapter structure after the output of each layer, such as... Figure 3 As shown. For any layer feature It is converted to:

[0032] The Adapter includes linear transformations, non-linear activation functions (such as GELU), and normalization operations (such as LayerNorm). The resulting multi-layer feature set is then adapted. It will be integrated through a fusion mechanism, the specific formula of which is:

[0033] Finally, the features are fused. It will be further mapped through a frozen projection layer to obtain an enhanced feature representation. When the image encoder is ResNet, the Projector is implemented as an attention pooling layer; when it is ViT, a linear projection layer is used to maintain the consistency of the block structure.

[0034] Step S4 includes: The abnormal residual deobfuscation module includes: a first adapter, a second adapter, a third adapter, and a residual correction unit; The initial zero sample is processed through the first adapter. The process is performed to learn the underlying obfuscation patterns and obtain the first obfuscation representation. ; The enhanced visual features are applied via a second adapter. The image is processed to extract confusion priors related to its content, resulting in a second confusion representation. ; The first and second confusion representations are jointly modeled using a third adapter to generate confusion estimates. ; Through the residual correction mechanism, the initial zero samples are... Subtracting the confusion estimate yields the corrected logic value. .

[0035] This application provides an embodiment as follows. Specifically, the input of the ARD module includes two parts: one is the initial zero-sample logits calculated by the image encoder and the text encoder. These correspond to image and normal text embedding, respectively. and abnormal text embedding The similarity; and the enhanced visual features output by the multi-level feature fusion module (MFF). This feature contains both low-level texture details and high-level semantic information.

[0036] Structurally, the ARD module consists of three adapters and one residual correction unit. First, Receive zero-sample logits Learn potential confusion patterns from them to obtain confusion representations. .at the same time, Acting on enhanced features Extract the confusion priors related to the image content, denoted as Subsequently, By combining the two, the interaction between logits and visual priors is comprehensively modeled in the joint space to generate the final confusion estimate. Finally, the ARD module adjusts the original logits using a residual correction method. The specific formula is:

[0037] This residual deconfusion mechanism corrects the ambiguous boundary between normal and abnormal data, reducing the risk of false positives and false negatives. Furthermore, it supplements local and global information using enhanced features, ultimately yielding... It is cleaner, thereby improving the robustness and accuracy of the test.

[0038] Step S5 includes: The image-level anomaly detection score The calculation formula is:

[0039] in, and These are the components in the corrected logical value corresponding to the normal category and the abnormal category, respectively; The pixel-level anomaly detection score is calculated as follows: For each block feature in the enhanced visual features Calculate its features compared to normal text. and abnormal text features The cosine similarity is used to obtain the anomaly score of the segment through normalization. The calculation formula is as follows:

[0040] The anomaly scores of all blocks are rearranged into a grid and upsampled and interpolated to the original image resolution to obtain the pixel-level anomaly map. .

[0041] This application provides an embodiment as follows: image-level anomaly detection score: input image Features of normal / abnormal text The initial zero sample logits were calculated. To alleviate boundary ambiguity, ARD is introduced and combined with enhanced priors generated by MFF. The corrected logits are obtained. The final abnormal score is:

[0042] This application provides an embodiment as follows: pixel-level anomaly detection score: the input image is processed by an image encoder with mixing cue and a multi-level feature fusion module (MFF) to obtain a fused block representation set. Correspondingly, the normal / abnormal features output by the text encoder are as follows: For each segment, an anomaly score is obtained by calculating and normalizing the cosine similarity between it and the features of normal / abnormal text. The specific formula is as follows:

[0043] After rearranging all the block scores into a grid and interpolating them to the original image resolution, a pixel-level anomaly map is obtained. This anomaly map provides pixel-by-pixel anomaly probabilities and can be used for localization and segmentation tasks.

[0044] Step S6 includes: The method learns the objective function during the training phase by jointly optimizing the objective function. Image-level loss Pixel-level loss and similarity regularization loss Weighted sum:

[0045] in, This is the balance coefficient; The image-level loss A focus loss function is used to constrain the corrected logic value. With image-level real labels Consistency; The pixel-level loss The sum of the Dice loss and the focus loss is used to constrain the predicted pixel-level anomaly map. Compared with pixel-level true labeled mask Consistency; The similarity regularization loss The L1 norm loss is used to constrain the corrected logic value. With the initial zero sample Consistency between them.

[0046] In one embodiment, the experimental setup is as follows: (1) Data set and evaluation metrics This invention was tested on several publicly available benchmark datasets in the field of industrial inspection. Specifically, four commonly used datasets were employed: MVTecAD (Bergman et al., 2019), VisA (Zou et al., 2022), MPDD (Jezeket et al., 2021), and BTAD (Mishra et al., 2021). These datasets cover common object and defect types in various industrial scenarios and can effectively evaluate the model's generalization ability in actual industrial quality inspection. This invention uses Area Under the Receiver Operating Characteristic Curve (AUROC) as the primary evaluation metric. Specifically, in image-level tasks, AUROC measures the model's ability to distinguish between normal and abnormal samples; in pixel-level tasks, AUROC is used to evaluate the model's discriminative performance when locating abnormal regions.

[0047] (2) Implementation details This invention uses a pre-trained CLIP (ViT-L / 14@336px) as the default backbone network for the vision-language model. Images are scaled to... The resolution was optimized by extracting multi-scale block representations from layers 6, 12, 18, and 24 of the visual encoder. In the cue learning part, the cue depth was set to 4 and the cue length to 5. The optimizer used was AdamW with an initial learning rate of 0.001, a batch size of 64, and was trained for 50 epochs in all experiments. The weight parameters in the weighted fusion mechanism were... The values ​​were set to 0.1, 0.2, 0.3, and 0.4 respectively to ensure the balance of multi-scale features during fusion. These are the regularization weight parameters in the loss function. Set to version 1.0. All experiments were implemented using PyTorch and were trained and tested on a single NVIDIA RTX 4090 GPU.

[0048] The core objective of this invention is to design a zero-shot anomaly detection method based on multi-level feature fusion and residual de-obfuscation. This method can simultaneously achieve image-level discrimination and pixel-level localization without requiring additional labeled data, significantly improving the accuracy and robustness of anomaly detection. The key innovation of this invention lies in the collaborative design of the following two core modules, which jointly solve the bottleneck problems in the prior art, as follows: (1) Fine-grained modeling of multi-level feature fusion: Unlike traditional methods that rely on a single global representation for discrimination, this invention uses a multi-level feature fusion module to uniformly map and weight the features of different levels of the visual encoder, taking into account both low-level texture details and high-level semantic information, thereby significantly enhancing the model's ability to perceive local minor defects and effectively solving the problem that existing methods are unable to capture fine-grained anomalies.

[0049] (2) Inter-class de-confusion mechanism based on residual correction: This invention breaks through the limitation that the existing CLIP zero-sample discrimination results are prone to confusion when the categories are similar or the boundaries are blurred. The invention proposes an abnormal residual de-confusion module, which uses enhanced features as priors to perform residual correction on the initial discrimination results, removes confusion noise, and purifies the abnormal score distribution, making the boundary between normal and abnormal categories clearer, and fundamentally reducing the risk of false alarms and false negatives.

[0050] To verify the effectiveness of this invention, experiments were conducted on datasets from multiple industrial inspection fields, and quantitative and qualitative comparisons were made with CLIP-based methods and several recent state-of-the-art methods. To ensure fair comparison, the original settings of each method were followed in the experiments, and the models were reproduced or retrained as necessary to match the evaluation protocol of this invention. Specific results will be described in the following sections.

[0051] The main experimental results are as follows: (1) Image-level zero-shot anomaly detection results Table 1 presents the image-level detection results on industrial benchmarks (MVTecAD, VisA, MPDD, and BTAD). The overall trend indicates that zero-shot anomaly detection methods based on assisted training (through adaptation of pre-trained VLMs) significantly outperform unadapted baselines (such as CLIP or WinCLIP using only human prompts). This demonstrates that while visual-language models possess the potential knowledge for anomaly detection, they require effective adaptation mechanisms to guide them towards discriminative tasks. Numerically, the proposed method achieves leading performance on all four industrial datasets. The AUROCs on MVTecAD, VisA, MPDD, and BTAD are 87.9, 80.8, 73.9, 75.2, and 79.5, respectively, with an average of 79.5, representing a 4.1 improvement over the second-place WinCLIP. The advantages of this method are most pronounced on the VisA and BTAD datasets, fully demonstrating its robust adaptability to complex semantics and material defects.

[0052] Table 1: Comparison of image-level zero-shot anomaly detection methods in industrial applications.

[0053]

[0054] (2) Pixel-level zero-sample anomaly detection results Table 2 shows the pixel-level detection results on the same dataset. It can be seen that the unadapted CLIP and CLIP-AC methods are almost ineffective at segmenting anomalous regions at the pixel level (average AUROC of only 44.4), while methods introducing multi-level features or discriminative enhancement (such as WinCLIP) significantly improve performance. Based on this, the method of this invention achieves the highest overall performance, with AUROCs of 88.1, 82.9, 79.0, and 74.1 on the four datasets, averaging 81.0, an improvement of 2.5 compared to WinCLIP. In particular, on VisA and MPDD, this method achieves the best results of 82.9 and 79.0 respectively, indicating that enhancing local representation through MFF and removing obfuscation in ARD using residuals can significantly improve the sensitivity and discriminative power for fine-grained defects.

[0055] Table 2: Comparison of pixel-level zero-sample anomaly detection methods in the industrial field.

[0056]

[0057] Qualitative results analysis: This invention demonstrates the anomaly segmentation maps generated by the proposed method on the MVTec-AD dataset, including various categories such as industrial products (e.g., bottles, capsules, pills, screws, transistors) and industrial materials (e.g., carpets, leather, tiles, wood, zippers).

[0058] Figure 4 The image sequentially displays the original image, pixel-level annotations, and the detection results of the proposed method. It can be seen that the proposed method accurately locates anomalous regions in most categories and significantly reduces false positives and false negatives. For example, in hazelnuts and metal nuts, the proposed method effectively captures minute cracks and defects; in tiles and wood, the model can distinguish between normal textures and abnormal scratches; and even in complex structures such as toothbrushes, the model still demonstrates good localization capabilities. These visualizations intuitively illustrate the strong generalization and pixel-level accuracy of the proposed method in cross-class zero-shot anomaly detection.

[0059] This application also discloses an electronic device. (See reference...) Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. The electronic device 500 may include: at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.

[0060] The communication bus 502 is used to enable communication between these components.

[0061] The user interface 503 may include a display screen, and optionally, the user interface 503 may also include a standard wired interface or a wireless interface.

[0062] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0063] This application also discloses a computer-readable storage medium storing multiple instructions adapted for loading by a processor to execute the aforementioned zero-sample anomaly detection method based on multi-level feature fusion and residual de-obfuscation.

[0064] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure.

[0065] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A zero-sample anomaly detection method based on multi-level feature fusion and residual de-obfuscation, characterized in that, The method includes the following steps: S1: Obtain the image to be detected and the corresponding text descriptions of the normal and abnormal categories; S2: Input the image to be detected and the text description into the visual encoder and the text encoder respectively to extract initial image features and initial text features, and calculate the initial zero sample based on the initial image features and initial text features; S3: The initial image features are processed by a multi-level feature fusion module to fuse visual features at different levels and obtain enhanced visual features; S4: Input the initial zero samples and enhanced visual features into the abnormal residual deobfuscation module, and perform deobfuscation processing on the initial zero samples through the residual correction mechanism to obtain the corrected logical value. ; S5: Based on the corrected logical value, calculate the image-level anomaly detection score to complete the image-level anomaly discrimination; based on the enhanced visual features and the initial text features, calculate the pixel-level anomaly detection score and generate a pixel-level anomaly map to complete the pixel-level anomaly localization.

2. The zero-sample anomaly detection method based on multi-level feature fusion and residual de-obfuscation as described in claim 1, characterized in that, Step S2 includes: The visual encoder and text encoder are encoders based on the pre-trained visual-language model CLIP; The input to the visual encoder includes a basic block token obtained from image segmentation, a static cue token for providing a general prior for the anomaly detection task, and a dynamic image cue token adaptively generated based on the features of the input image. The input to the text encoder includes the text description, a static cue token for providing a general prior for the anomaly detection task, and a dynamic text cue token.

3. The zero-sample anomaly detection method based on multi-level feature fusion and residual de-obfuscation as described in claim 1, characterized in that, Step S3 includes: The multi-level feature fusion module performs the following operations: From the first of the visual encoder Extract multi-scale block representations from a specified level to form a multi-level feature set. ,in , Indicates the first The number of blocks in a layer Indicates feature dimension; Each layer of features The features are input into a lightweight adapter, which maps features from different levels to a unified semantic space through operations including linear transformations, nonlinear activation functions, and normalization layers, resulting in an adapted multi-layer feature set. ; The adapted feature set is weighted and fused to obtain the fused features. The calculation formula is as follows: in, For the first Weight coefficients of layer features; The fusion features The final enhanced visual features are obtained through a projection layer. .

4. The zero-sample anomaly detection method based on multi-level feature fusion and residual de-obfuscation as described in claim 3, characterized in that, Step S3 also includes: When the visual encoder is a Vision Transformer, the projection layer is a linear projection layer; when the visual encoder is a ResNet, the projection layer is an attention pooling layer. The different levels of visual features include: low-level features containing fine-grained texture information and high-level features containing global semantic information.

5. The zero-sample anomaly detection method based on multi-level feature fusion and residual de-obfuscation as described in claim 1, characterized in that, Step S4 includes: The abnormal residual deobfuscation module includes: a first adapter, a second adapter, a third adapter, and a residual correction unit; The initial zero sample is processed through the first adapter. The process is performed to learn the underlying obfuscation patterns and obtain the first obfuscation representation. ; The enhanced visual features are applied via a second adapter. The image is processed to extract confusion priors related to its content, resulting in a second confusion representation. ; The first and second confusion representations are jointly modeled using a third adapter to generate confusion estimates. ; Through the residual correction mechanism, the initial zero samples are... Subtracting the confusion estimate yields the corrected logic value. .

6. The zero-sample anomaly detection method based on multi-level feature fusion and residual de-obfuscation as described in claim 5, characterized in that, Step S5 includes: The image-level anomaly detection score The calculation formula is: in, and These are the components in the corrected logical value corresponding to the normal category and the abnormal category, respectively; The pixel-level anomaly detection score is calculated as follows: For each block feature in the enhanced visual features Calculate its features compared to normal text. and abnormal text features The cosine similarity is used to obtain the anomaly score of the segment through normalization. The calculation formula is as follows: The anomaly scores of all blocks are rearranged into a grid and upsampled and interpolated to the original image resolution to obtain the pixel-level anomaly map. .

7. The zero-sample anomaly detection method based on multi-level feature fusion and residual de-obfuscation as described in claim 6, characterized in that, Step S6 includes: The method learns the objective function during the training phase by jointly optimizing the objective function. Image-level loss Pixel-level loss and similarity regularization loss Weighted sum: in, This is the balance coefficient; The image-level loss A focus loss function is used to constrain the corrected logic value. With image-level real labels Consistency; The pixel-level loss The sum of the Dice loss and the focus loss is used to constrain the predicted pixel-level anomaly map. Compared with pixel-level true labeled mask Consistency; The similarity regularization loss The L1 norm loss is used to constrain the corrected logic value. With the initial zero sample Consistency between them.

8. An electronic device, characterized in that, The device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to enable the electronic device to perform the zero-sample anomaly detection method based on multi-level feature fusion and residual de-obfuscation as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed by a computer, perform the zero-sample anomaly detection method based on multi-level feature fusion and residual de-obfuscation as described in any one of claims 1-7.

Images