A Zero-Shot Anomaly Detection Method and System Based on Bi-directional Alignment Enhancement

Abstract

This invention provides a zero-shot anomaly detection method and system based on bidirectional alignment enhancement. The method includes: achieving direct interaction between global and local features through the cross-attention module CAGL; extracting fine-grained local features through the region token alignment module RegionAlign; and projecting global visual semantics onto the text embedding space through the text embedding enhancement module TextAug to generate domain-aware cue words. This invention solves the technical problems of poor generalization ability, neglect of local spatial details, and difficulty in hierarchical modeling of multi-scale visual anomalies.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, specifically to a zero-shot anomaly detection method and system based on bidirectional alignment enhancement. Background Technology

[0002] Industrial anomaly detection is a crucial aspect of quality control in manufacturing, aiming to identify defective products by analyzing subtle deviations in images. Traditional methods rely on supervised learning, requiring a large number of labeled anomaly samples. However, in real-world industrial scenarios, anomaly samples are scarce, diverse, and domain-specific, resulting in poor model generalization ability.

[0003] In recent years, visual-language models such as CLIP have demonstrated strong zero-shot generalization capabilities through large-scale cross-modal pre-training. However, CLIP primarily focuses on global semantic alignment, neglecting the modeling of local details, making it difficult to detect subtle or texture-related anomalies. Existing zero-shot anomaly detection methods mostly concentrate on text cue optimization, such as manual templates or cue learning, but still suffer from the following problems:

[0004] 1. Manual templates rely on domain expert knowledge and have limited generalization ability; 2. It suggests that learning may overfit to specific anomaly types and neglect local spatial details; 3. Lack of hierarchical modeling capabilities for multi-scale visual anomalies.

[0005] The existing invention patent application document CN117725230A, entitled "Entity Alignment Method and System Based on Image Generation Algorithm and Multimodal Large Model," describes a method that includes: preprocessing two pre-acquired multimodal knowledge graph data using a multimodal pre-trained model CLIP; extracting features from the graph data to obtain embeddings for different modalities, considering modal heterogeneity; generating pseudo-visual modalities using a diffusion model to fill in the visual modal gaps of entities in the multimodal knowledge graph, and combining them with embeddings extracted from other modalities to form a comprehensive entity embedding; assigning weights to different modal embeddings to obtain enhanced modal data, and then splicing and fusing the modal information with different weights; and training with a specific contrastive loss function to minimize the loss function and obtain the final entity alignment result. The drawback of this existing method is that its technical goal is to determine whether entities in different knowledge graphs are "equivalent" or "point to the same object" (i.e., aligned), which is a relational judgment task rather than a "detection" task of identifying abnormal regions within an image. Therefore, its model architecture aims to generate a unified global feature representation for entities for equivalence comparison, rather than to locate pixel-level defects.

[0006] The existing invention patent application document CN120105353A, entitled "A Multi-Image Processing Method Based on Multimodal Entity Alignment," describes a method that includes: acquiring multiple images; retrieving rich semantic information of entities from an external knowledge base; using the semantic information to select the most representative image for each entity in a multi-image scene; encoding the original input for each modality; applying cross-diffusion attention to perform a first-layer fusion between the visual and textual modalities, followed by a second-layer interaction with a structured modality, and finally aligning the entity representations of the images using contrastive loss; and outputting a multi-image fusion of the textual and visual modalities. The drawback of this method is that its semantic information enhancement module heavily relies on external structured knowledge bases (such as DBpedia). In specific fields such as specialized industrial components or scenarios with high data privacy requirements, the lack of relevant knowledge leads to method failure or reduced effectiveness.

[0007] The existing invention patent application document CN119206563A, entitled "A Weakly Supervised Video Anomaly Detection Method Based on Multiple Textual Hints," includes the following steps: 1) extracting original features from video data using a video feature extractor; 2) inputting the original features into a global-local temporal dependency network to obtain temporal enhancement features; 3) obtaining two different hints from the labeled text using a knowledge graph and learning hints respectively; 4) fusing the two hints with the temporal enhancement features across modalities; and 5) obtaining segment anomaly scores from the temporal enhancement features using a segment classifier, and optimizing the network by designing a loss function to achieve the video anomaly detection task. Its shortcomings are: first, it belongs to "weakly supervised" learning, still requiring a large amount of video-level labeled data, and cannot achieve "zero-shot" detection; second, its model is designed for video temporal modeling and does not optimize the perception of spatial local details in static images.

[0008] In summary, existing technologies suffer from poor generalization ability, neglect of local spatial details, and difficulty in performing hierarchical modeling of multi-scale visual anomalies. Summary of the Invention

[0009] The technical problem to be solved by this invention is: how to solve the technical problems of poor generalization ability, neglect of local spatial details and difficulty in hierarchical modeling of multi-scale visual anomalies in the prior art.

[0010] This invention solves the above-mentioned technical problems by employing the following technical solution: a zero-sample anomaly detection method based on bidirectional alignment enhancement includes: S1. Extract global and local features from the input image; S2. Using the cross-attention module CAGL and a multi-head attention mechanism, the global and local features are directly interacted to obtain the first anomaly map. S3. Through the RegionAlign module, fine-grained local features are extracted from local features to obtain enhanced local features; S4. Perform text embedding enhancement operation through the text embedding enhancement module TextAug, project the global visual semantics in the enhanced global feature Featureaug onto the text embedding space, generate domain-aware prompt words, and obtain enhanced text features; S5. Calculate the cosine similarity between enhanced local features and enhanced text features to generate a second anomaly map, thereby realizing the indirect interaction between image and text features and generating an indirect anomaly map. S6. Perform a fusion operation on the first anomaly map and the second anomaly map to obtain the final anomaly segmentation map; S7. Generate anomaly classification scores based on enhanced global features, original global features, and text features; S8. Convert the trained PyTorch model to ONNX format to enable cross-platform deployment of the model.

[0011] This invention achieves state-of-the-art performance in classification and segmentation tasks on multiple industrial anomaly detection datasets, significantly improving the model's generalization ability and detection accuracy in zero-shot scenarios. Furthermore, the method described in this invention exhibits excellent computational efficiency during the deployment phase, and after optimization, it can achieve real-time or near-real-time inference on embedded edge computing devices, including Jetson Orin NX, meeting the practical requirements of low latency and low power consumption in industrial settings.

[0012] In a more specific technical solution, in S1, the CLIP image encoder is used to extract global and local features of the input image; among them, the global feature is the class token in the CLIP image encoder, which is used to represent the overall semantic information of the image; and the patch embeddings extracted from the intermediate layer of the network are used as local features to capture the fine-grained spatial details of the image.

[0013] This invention presents a bidirectional alignment-enhanced CLIP framework, which achieves efficient anomaly detection and localization in zero-shot scenarios through direct and indirect interaction of global and local features. Addressing the shortcomings of existing zero-shot anomaly detection methods in local detail modeling and cross-modal alignment, this invention proposes a bidirectional alignment-enhanced CLIP framework (BAE-CLIP) that achieves bidirectional enhancement of global and local features through multi-module collaboration.

[0014] In a more specific technical solution, in S2, the global feature is used as the query, and the local features are used as the key and value. The following logic is used to process and obtain the enhanced global feature Featureaug and the first anomaly map based on feature similarity: Qg=FgWq g,Kl=FlWk l,Vl=FlWv l A(i) g=SoftMax(Q(i) g(K(i) l) / √dk)V(i) l Featureaug=Concat(Al g,…,AJ g)Wproj In the formula, Wq g, Wk l, Wv l, Wproj are learnable linear projection matrices, J is the number of attention heads, and dk is the dimension of the key vector.

[0015] This invention provides a zero-shot detection framework based on bidirectional alignment enhancement. Through the synergistic effect of cross-attention mechanism and region token alignment module, it enhances the ability to discriminate local abnormal patterns while maintaining global semantic consistency, significantly improving the accuracy and robustness of anomaly detection in zero-shot scenarios.

[0016] Addressing the shortcomings of existing technologies, such as those in CN117725230A, this application fundamentally changes the task objective and output format. The proposed Cross-Attention Module (CAGL) and Region Align Module (RegionAlign) work collaboratively to generate high-resolution spatial anomaly maps, directly outputting the pixel-level location of defects. This represents a fundamental leap from "global equivalence judgment" to "local anomaly localization," meeting the core requirement of industrial quality inspection for precise defect location.

[0017] In a more specific technical solution, in S3, the RegionAlign module enhances the representation capability of local features by connecting convolutional layers with linear residuals; using the following logic, the local features Fln at each level are processed to obtain the enhanced local features F′ln: Yl n=Conv(Fl n) F′ln=Yl n+knFl n+bn In the formula, n=1,2,3,4 represents four feature levels, Conv represents the convolution operation, and kn and bn are the weights and bias parameters of the linear layer, respectively.

[0018] In a more specific technical solution, in S4, the text embedding enhancement module TextAug aligns the enhanced global feature Featureaug with the text feature space.

[0019] To address the deficiencies of existing technologies, such as those described in CN119206563A, the solutions proposed in this application include: First, the entire framework is built on a pre-trained CLIP model. During testing, only a text description of a normal state (such as "a picture of a defect-free circuit board") needs to be input. No abnormal samples are required for training, thus achieving true zero-shot reasoning capability. Second, this application pioneers a "two-way alignment" enhancement mechanism. It not only achieves direct "global-local" interaction through CAGL, but also enhances local features through RegionAlign and then performs "local-text" cosine similarity calculation with the text features enhanced by TextAug. This two-way interaction mechanism, specifically designed for spatial details, significantly enhances the accuracy of identifying and locating local anomalies such as subtle textures and scratches in images, overcoming the core problem of insufficient local detail modeling in existing methods.

[0020] In a more specific technical solution, in S4, text embedding enhancement is performed through the following logic to obtain enhanced text features F′ t: F′ t=MLP(Concat(Fg+Fr,Ft)) In the formula, Fr is a randomly initialized feature vector, Ft is the original text feature, and MLP is a multilayer perceptron. The output is the enhanced text feature F′ t.

[0021] To address the shortcomings of existing technologies, such as those described in CN120105353A, this application designs a text embedding enhancement module (TextAug). This module does not rely on any external knowledge base. Instead, it uses a learnable multilayer perceptron (MLP) to project visual features and fuse them with original text features, dynamically generating "domain-aware cue words" closely related to the current visual content. This internal generation mechanism eliminates dependence on external data, improving the model's generalization ability and deployment convenience in unfamiliar industrial scenarios.

[0022] In a more specific technical solution, in S5, the fusion operation includes: element-level addition or maximum value operation.

[0023] This invention proposes a multi-granularity feature interaction and fusion mechanism. Through the cascaded design of CAGL, RegionAlign and TextAug modules, it achieves bidirectional semantic alignment from global to local, effectively solving the problem of insufficient adaptability of traditional methods in detecting subtle defects and complex structural anomalies.

[0024] In a more specific technical solution, in S7, the similarity score between the enhanced global features and the original global features and text features is calculated, the maximum value is taken, and it is added to the maximum outlier value in the final outlier segmentation map to obtain the image-level outlier classification score, thus completing the outlier classification task.

[0025] The method of this invention has excellent versatility and deployment flexibility under zero-shot settings. It can achieve efficient classification and segmentation on multiple industrial datasets by relying only on pre-trained visual-language models and semantic prompts. After optimization, it can achieve real-time inference on edge devices.

[0026] In a more specific technical solution, hardware deployment operations are carried out using a standardized model in the ONNX format.

[0027] In more specific technical solutions, zero-shot anomaly detection systems based on bidirectional alignment enhancement include: The feature extraction module is used to extract global and local features of the input image; The Cross Attention Module (CAGL) is used to employ a multi-head attention mechanism to directly interact with global and local features to obtain the first anomaly map. The Cross Attention Module (CAGL) is connected to the feature extraction module. The RegionAlign module is used to extract fine-grained local features from local features to obtain enhanced local features. The RegionAlign module is connected to the Cross Attention module CAGL. The TextAug module is used to perform text embedding enhancement operations. It projects the global visual semantics in the enhanced global feature Featureaug onto the text embedding space to generate domain-aware cue words and obtain enhanced text features. The TextAug module is connected to the cross-attention module CAGL. The indirect anomaly map generation module is used to calculate the cosine similarity between enhanced local features and enhanced text features, generate a second anomaly map, realize the indirect interaction between image and text features, and generate an indirect anomaly map. The indirect anomaly map generation module is connected to the region token alignment module RegionAlign and the text embedding enhancement module TextAug. The anomaly graph fusion module is used to fuse the first anomaly graph and the second anomaly graph to obtain the final anomaly segmentation graph. The anomaly graph fusion module is connected to the cross attention module CAGL and the indirect anomaly graph generation module. The anomaly classification module generates anomaly classification scores based on enhanced global features, original global features, and text features. The anomaly classification module is connected to the anomaly graph fusion module.

[0028] The present invention has the following advantages over the prior art: This invention achieves state-of-the-art performance in classification and segmentation tasks on multiple industrial anomaly detection datasets, significantly improving the model's generalization ability and detection accuracy in zero-shot scenarios. Furthermore, the method described in this invention exhibits excellent computational efficiency during the deployment phase, and after optimization, it can achieve real-time or near-real-time inference on embedded edge computing devices, including Jetson Orin NX, meeting the practical requirements of low latency and low power consumption in industrial settings.

[0029] This invention presents a bidirectional alignment-enhanced CLIP framework, which achieves efficient anomaly detection and localization in zero-shot scenarios through direct and indirect interaction of global and local features. Addressing the shortcomings of existing zero-shot anomaly detection methods in local detail modeling and cross-modal alignment, this invention proposes a bidirectional alignment-enhanced CLIP framework (BAE-CLIP) that achieves bidirectional enhancement of global and local features through multi-module collaboration.

[0030] This invention provides a zero-shot detection framework based on bidirectional alignment enhancement. Through the synergistic effect of cross-attention mechanism and region token alignment module, it enhances the ability to discriminate local abnormal patterns while maintaining global semantic consistency, significantly improving the accuracy and robustness of anomaly detection in zero-shot scenarios.

[0031] This invention proposes a multi-granularity feature interaction and fusion mechanism. Through the cascaded design of CAGL, RegionAlign and TextAug modules, it achieves bidirectional semantic alignment from global to local, effectively solving the problem of insufficient adaptability of traditional methods in detecting subtle defects and complex structural anomalies.

[0032] The method of this invention has excellent versatility and deployment flexibility under zero-shot settings. It can achieve efficient classification and segmentation on multiple industrial datasets by relying only on pre-trained visual-language models and semantic prompts. After optimization, it can achieve real-time inference on edge devices.

[0033] This invention solves the technical problems of poor generalization ability, neglect of local spatial details, and difficulty in hierarchical modeling of multi-scale visual anomalies in the prior art. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the basic steps of the zero-sample anomaly detection method based on bidirectional alignment enhancement according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of data interaction in the zero-sample anomaly detection system based on bidirectional alignment enhancement according to Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of data flow processing for the zero-sample anomaly detection method based on bidirectional alignment enhancement in Embodiment 1 of the present invention; Figure 4 This is a hardware deployment diagram of the zero-sample anomaly detection system based on bidirectional alignment enhancement according to Embodiment 1 of the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] Example 1 like Figure 1 and Figure 2 As shown, the zero-sample anomaly detection method based on bidirectional alignment enhancement provided by this invention includes the following basic steps: S1. Extract global and local features from the input image; In this embodiment, the CLIP image encoder is used to extract visual features from the input image. Global features are the class tokens in CLIP, used to represent the overall semantic information of the image; local features are patch embeddings extracted from the intermediate layers of the network (layers 6, 12, 18, and 24), used to capture fine-grained spatial details of the image. Let the global feature be Fg∈Rd, and the local feature be Fl∈RN×d, where N is the number of patches in the image partitioning, and d is the feature dimension.

[0037] S2. Direct interaction between global and local features is achieved through the CAGL module; See Figure 3 In this embodiment, the CAGL module employs a multi-head attention mechanism to achieve direct interaction between global and local features. Global features serve as the query, while local features act as the key and value. The specific calculation process is as follows: Qg=FgWq g,Kl=FlWk l,Vl=FlWv l A(i) g=SoftMax(Q(i) g(K(i) l) / √dk)V(i) l Featureaug=Concat(Al g,…,AJ g)Wproj Where Wq g, Wk l, Wv l, and Wproj are learnable linear projection matrices, J is the number of attention heads, and dk is the dimension of the key vector. This module outputs the enhanced global feature Featureaug and the first anomaly map based on feature similarity.

[0038] S3. Enhance local features using the RegionAlign module; In this embodiment, the RegionAlign module enhances the representation capability of local features through convolutional layers and linear residual connections. The local features Fl n at each level are processed as follows: Yl n=Conv(Fl n) F′ln=Yl n+knFl n+bn Here, n=1,2,3,4 represents four feature layers, Conv represents the convolution operation, and kn and bn are the weights and bias parameters of the linear layer, respectively. This design effectively enhances the model's ability to discriminate local anomalies such as edges and textures while preserving the original feature information.

[0039] S4. Enhance text embedding through the TextAug module; In this embodiment, the TextAug module aligns global visual features with the text feature space, enhancing the text embedding representation through the following calculations: F′ t=MLP(Concat(Fg+Fr,Ft)) Where Fr is a randomly initialized feature vector, Ft is the original text feature, and MLP is a multilayer perceptron. The output is the enhanced text feature F′ t, whose semantic expression matches the image content better.

[0040] S5. Generate an indirect anomaly graph; In this embodiment, the cosine similarity between the local feature F′ln enhanced by the RegionAlign module and the text feature F′t enhanced by the TextAug module is calculated to generate a second anomaly map, thereby realizing indirect interaction between image and text features.

[0041] S6, Fusion Anomaly Map; In this embodiment, the first anomaly map generated by the CAGL module and the second anomaly map generated in step 5 are fused to obtain the final anomaly segmentation map. The fusion method can be element-wise addition or taking the maximum value operation to comprehensively utilize the detection results of direct and indirect interaction paths.

[0042] S7. Generate anomaly classification scores; In this embodiment, the similarity scores between the CAGL enhanced global features, the original global features, and the text features are calculated. The maximum value is then added to the maximum outlier value in the final anomaly segmentation map to obtain the image-level anomaly classification score, thus completing the anomaly classification task.

[0043] S8, edge deployment; like Figure 4 As shown, in this embodiment, the trained PyTorch model is converted to ONNX format using official tools. The specific operations include: Export the model using the torch.onnx.export interface; Set appropriate input and output node names; Verify the consistency of the model output before and after the transformation; The standardized models in the ONNX format can be efficiently deployed on various inference engines and hardware platforms, significantly enhancing the practical application value of the models.

[0044] In summary, this invention achieves state-of-the-art performance in classification and segmentation tasks on multiple industrial anomaly detection datasets, significantly improving the model's generalization ability and detection accuracy in zero-shot scenarios. Furthermore, the method described in this invention exhibits excellent computational efficiency during deployment, and after optimization, it can achieve real-time or near-real-time inference on embedded edge computing devices, including Jetson Orin NX, meeting the practical requirements of low latency and low power consumption in industrial settings.

[0045] This invention presents a bidirectional alignment-enhanced CLIP framework, which achieves efficient anomaly detection and localization in zero-shot scenarios through direct and indirect interaction of global and local features. Addressing the shortcomings of existing zero-shot anomaly detection methods in local detail modeling and cross-modal alignment, this invention proposes a bidirectional alignment-enhanced CLIP framework (BAE-CLIP) that achieves bidirectional enhancement of global and local features through multi-module collaboration.

[0046] This invention provides a zero-shot detection framework based on bidirectional alignment enhancement. Through the synergistic effect of cross-attention mechanism and region token alignment module, it enhances the ability to discriminate local abnormal patterns while maintaining global semantic consistency, significantly improving the accuracy and robustness of anomaly detection in zero-shot scenarios.

[0047] This invention proposes a multi-granularity feature interaction and fusion mechanism. Through the cascaded design of CAGL, RegionAlign and TextAug modules, it achieves bidirectional semantic alignment from global to local, effectively solving the problem of insufficient adaptability of traditional methods in detecting subtle defects and complex structural anomalies.

[0048] The method of this invention has excellent versatility and deployment flexibility under zero-shot settings. It can achieve efficient classification and segmentation on multiple industrial datasets by relying only on pre-trained visual-language models and semantic prompts. After optimization, it can achieve real-time inference on edge devices.

[0049] This invention solves the technical problems of poor generalization ability, neglect of local spatial details, and difficulty in hierarchical modeling of multi-scale visual anomalies in the prior art.

[0050] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A zero-shot anomaly detection method based on bidirectional alignment enhancement, characterized in that, The method includes: S1. Extract global and local features from the input image; S2. Using the cross attention module CAGL and a multi-head attention mechanism, the global features and the local features are directly interacted to obtain the first anomaly map. S3. Through the RegionAlign module, fine-grained local features are extracted from the local features to obtain enhanced local features; S4. Perform text embedding enhancement operation through the text embedding enhancement module TextAug, project the global visual semantics in the enhanced global feature Featureaug onto the text embedding space, generate domain-aware prompt words, and obtain enhanced text features; S5. Calculate the cosine similarity between the enhanced local features and the enhanced text features, generate a second anomaly map, realize the indirect interaction between image and text features, and generate an indirect anomaly map. S6. Perform a fusion operation on the first anomaly map and the second anomaly map to obtain the final anomaly segmentation map; S7. Generate anomaly classification scores based on the enhanced global features, the original global features, and the text features.

2. The zero-sample anomaly detection method based on bidirectional alignment enhancement according to claim 1, characterized in that, In step S1, the CLIP image encoder is used to extract the global features and the local features of the input image; wherein, the global features are the class tokens in the CLIP image encoder, which are used to characterize the overall semantic information of the image; and the patch embeddings extracted from the intermediate layers of the network are used as local features to capture the fine-grained spatial details of the image.

3. The zero-sample anomaly detection method based on bidirectional alignment enhancement according to claim 1, characterized in that, In step S2, the global feature is used as the query, and the local feature is used as the key and value. The enhanced global feature Featureaug and the first anomaly map based on feature similarity are obtained using the following logic: Qg=FgWq g,Kl=FlWk l,Vl=FlWv l A(i) g=SoftMax(Q(i) g(K(i) l) / √dk)V(i) l Featureaug=Concat(Al g,…,AJ g)Wproj In the formula, Wq g, Wk l, Wv l, Wproj are learnable linear projection matrices, J is the number of attention heads, and dk is the dimension of the key vector.

4. The zero-sample anomaly detection method based on bidirectional alignment enhancement according to claim 1, characterized in that, In step S3, the region token alignment module RegionAlign enhances the representation capability of the local features by connecting convolutional layers with linear residuals; the local features Fln at each level are processed using the following logic to obtain the enhanced local features F′ln: Yl n=Conv(Fl n) F′ln=Yl n+knFl n+bn In the formula, n=1,2,3,4 represents four feature levels, Conv represents the convolution operation, and kn and bn are the weights and bias parameters of the linear layer, respectively.

5. The zero-sample anomaly detection method based on bidirectional alignment enhancement according to claim 1, characterized in that, In step S4, the text embedding enhancement module TextAug aligns the enhanced global feature Featureaug with the text feature space.

6. The zero-sample anomaly detection method based on bidirectional alignment enhancement according to claim 1, characterized in that, In step S4, the text embedding enhancement operation is performed through the following logic to obtain the enhanced text feature F′t: F′ t=MLP(Concat(Fg+Fr,Ft)) In the formula, Fr is a randomly initialized feature vector, Ft is the original text feature, and MLP is a multilayer perceptron. The output is the enhanced text feature F′ t.

7. The zero-sample anomaly detection method based on bidirectional alignment enhancement according to claim 1, characterized in that, In S5, the fusion operation includes element-level addition or maximum value operation.

8. The zero-sample anomaly detection method based on bidirectional alignment enhancement according to claim 1, characterized in that, In step S7, the similarity score between the enhanced global feature and the original global feature and text feature is calculated, the maximum value is taken, and it is added to the maximum anomaly value in the final anomaly segmentation map to obtain the image-level anomaly classification score, thus completing the anomaly classification task.

9. The zero-sample anomaly detection method based on bidirectional alignment enhancement according to claim 1, characterized in that, Hardware deployment operations are performed using standardized models in the ONNX format.

10. A zero-shot anomaly detection system based on bidirectional alignment enhancement, characterized in that, The system includes: The feature extraction module is used to extract global and local features of the input image; The cross-attention module CAGL is used to employ a multi-head attention mechanism to directly interact with the global features and the local features to obtain a first anomaly map. The cross-attention module CAGL is connected to the feature extraction module. The RegionAlign module is used to extract fine-grained local features from the local features to obtain enhanced local features. The RegionAlign module is connected to the Cross Attention module CAGL. The TextAug module is used to perform text embedding enhancement operations. It projects the global visual semantics in the enhanced global feature Featureaug onto the text embedding space to generate domain-aware cue words and obtain enhanced text features. The TextAug module is connected to the cross-attention module CAGL. An indirect anomaly graph generation module is used to calculate the cosine similarity between the enhanced local features and the enhanced text features, generate a second anomaly graph, realize the indirect interaction between image and text features, and generate an indirect anomaly graph. The indirect anomaly graph generation module is connected to the region token alignment module RegionAlign and the text embedding enhancement module TextAug. An anomaly graph fusion module is used to perform a fusion operation on the first anomaly graph and the second anomaly graph to obtain the final anomaly segmentation graph. The anomaly graph fusion module is connected to the cross attention module CAGL and the indirect anomaly graph generation module. An anomaly classification module is used to generate anomaly classification scores based on the enhanced global features, the original global features, and the text features. The anomaly classification module is connected to the anomaly graph fusion module.

Images