Zero-sample industrial anomaly detection method, system, equipment and medium
By employing a sparse hybrid expert system and a dynamic caching mechanism, the problems of feature projection and normal reference updating in zero-sample industrial anomaly detection are solved, achieving efficient and accurate industrial anomaly detection that is adaptable to the detection of different types of industrial products.
Patent Information
- Application Number
- CN202511177771.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-21
- Publication Date
- 2025-11-28
AI Technical Summary
Existing zero-sample industrial anomaly detection methods lack accuracy in multi-level feature projection adaptation and dynamic normal reference construction, making them difficult to apply effectively in industrial scenarios.
A sparse hybrid expert system is introduced for image feature projection, and pseudo-normal features are updated through a dynamic caching mechanism. By combining fine-grained anomaly response maps and preliminary anomaly heatmaps, cross-modal feature alignment and dynamic feature updating are achieved.
It improves the resolution and accuracy of detection, optimizes computational efficiency, enhances the model's generalization ability and real-time performance, and ensures high accuracy and reliability of anomaly detection.
Smart Images
Figure CN121032985A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of industrial anomaly detection, and particularly relates to a zero-shot industrial anomaly detection method, system, device and medium. BACKGROUND
[0002] Industrial anomaly detection is a key link to ensure product quality and production safety. However, existing methods mostly rely on a large number of normal samples or abnormal samples for training. In the industrial scene, abnormal data is extremely scarce and the labeling cost is high, making it difficult for supervised learning methods to be popularized in actual deployment. Zero-shot learning provides a new possibility for this, that is, to realize anomaly detection without abnormal samples.
[0003] Current zero-shot industrial anomaly detection methods usually use pre-trained visual language models (such as CLIP) to extract image and text features and perform cross-modal alignment. Although these methods have certain generalization ability, they still face two core problems in actual application. First, it is difficult to align multi-semantic level image features across modalities. In industrial images, shallow features often capture detailed information such as texture and edge, while deep features express more abstract semantics. Aligning these features from different stages to the text semantic space is a highly nonlinear mapping problem. Existing methods mostly use uniform projection or shared weights, which cannot take into account the differences between multi-level semantic features, resulting in a decrease in image-text alignment accuracy, which in turn affects the anomaly localization performance. Therefore, a module with selective expert ability needs to be introduced, which can automatically match the appropriate projection path according to the semantic level of the input feature. Sparse expert system is an ideal solution to this problem. Sparse expert system realizes the "customized mapping" of different feature semantics by introducing multiple parallel expert networks and using a gating function to select a small number of experts to participate in feature transformation. This not only improves the expression ability of image features at each stage during projection, but also significantly enhances the alignment quality with text features. Second, it is difficult to dynamically update normal reference features. Most methods rely on fixed normal image mean or predefined templates as a comparison benchmark, which cannot adapt to potential distribution changes or concept drift in input data. In the zero-shot or unsupervised scene, the lack of label information of abnormal samples further limits the expression ability of static normal benchmarks. Therefore, the present application proposes a dynamic caching mechanism: the mechanism uses the similarity between images and texts as a confidence indicator to automatically select the "most likely normal" images from the test images and store their features in the cache as pseudo-normal features. This dynamic updating strategy not only does not require label supervision, but also has online adaptation ability, which can enhance the robustness and generalization ability of the model in new scenes.
[0004] In summary, existing zero-shot anomaly detection technologies still have significant shortcomings in terms of multi-level feature projection adaptation and dynamic normal reference construction, which urgently need to be addressed through sparse hybrid expert systems and dynamic caching mechanisms. Summary of the Invention
[0005] This invention provides a zero-sample industrial anomaly detection method, system, equipment, and medium to at least solve the problem of insufficient accuracy in locating anomaly areas in existing detection methods.
[0006] In a first aspect, embodiments of this application provide a zero-sample industrial anomaly detection method, the method comprising: A visual encoder is used to extract multi-stage image features and global image features. The semantic embeddings of the corresponding categories are extracted by a text encoder to generate global text features; A sparse hybrid expert system is introduced. For the image features at each stage, the weight of each expert is calculated. The K different experts with the largest weights are selected and used to project and fuse the original image features at each stage, so as to obtain the image features of each stage after projection. Calculate the cosine similarity between global text features and image features at each stage after projection, and generate preliminary anomaly heatmaps for each stage. Calculate the cosine similarity between global image features and global text features to obtain the confidence benchmark; The cache is dynamically constructed and updated based on the confidence benchmark. The image with the highest confidence benchmark is selected as the pseudo-normal feature, and the image features of each stage after the pseudo-normal feature is projected are stored in the cache as normal features to obtain the cached features. The cosine similarity between the cached features and the projected image features is calculated stage by stage to generate fine-grained anomaly heatmaps for each stage. The preliminary anomaly heatmaps and fine-grained anomaly heatmaps from each stage are weighted and fused to obtain a text-based fused heatmap and a cache-based fused heatmap. The text-based fusion heatmap and the cache-based fusion heatmap are weighted and calculated to obtain the final fusion heatmap. The maximum value is extracted from the final fused heatmap as the pixel-level anomaly score. The pixel-level anomaly score is then weighted and fused with the confidence benchmark to obtain the final image-level anomaly score.
[0007] By employing cross-modal feature alignment, dynamic caching mechanisms, and the generation of fine-grained anomaly response maps, efficient and accurate zero-shot anomaly detection is achieved.
[0008] Furthermore, the visual encoder is a visual Transformer, whose multi-level features are divided into four stages, corresponding to feature extraction from shallow to deep layers respectively. A visual encoder is used to extract multi-stage image features and global image features, and its expression is as follows: , ; In the formula, Indicates the first Image features at each stage Represents global image features. This represents the test image. This represents a visual encoder.
[0009] Furthermore, semantic embeddings of the corresponding categories are extracted through a text encoder to generate global text features, specifically including: The natural language description of category label c is encoded using a text encoder to obtain global text features, the expression of which is as follows: ; In the formula, Represents global text features. This indicates that the natural language description of the category labels is encoded using a text encoder. Indicates category label, Let D represent a real vector space of dimension D.
[0010] Furthermore, a sparse hybrid expert system is introduced. For the image features at each stage, the weight of each expert is calculated, and the K different experts with the largest weights are selected. These experts are then used to project and fuse the original image features at each stage, thereby obtaining the projected image features for each stage. Specifically, these include: For the For each stage of image features, calculate the weight of each expert, and select the K different experts with the highest weights. The expression is as follows: , ; In the formula, Indicates the first Image features at each stage, , Indicates the parameters of the gating network. This represents the normalized weights of N experts. This represents the K experts with the highest weights; Using the selected K experts respectively Project the image and then weight and fuse the results to obtain the projected image. The image features at each stage are expressed as follows: , ; In the formula, Experts The output, Experts Network parameters, For activation function, Indicates the projected number of... Image features at each stage.
[0011] Furthermore, the cosine similarity between the global text features and the image features at each stage after projection is calculated to generate preliminary anomaly heatmaps corresponding to each stage, specifically including: Calculate the cosine similarity between the global text features and the image features at each stage after projection, and obtain the similarity matrix for each stage. The expression is as follows: ; In the formula, Indicates the first Similarity matrix for each stage, Represents global text features; Based on the similarity matrices at each stage, a preliminary anomaly heatmap is generated, the expression of which is: ; In the formula, Indicates the first Preliminary anomaly heatmaps for each stage.
[0012] Furthermore, the cosine similarity between global image features and global text features is calculated to obtain the confidence benchmark, which is expressed as follows: ; In the formula, Indicates the confidence level benchmark. Represents global text features; The cache is dynamically constructed and updated based on the confidence benchmark. The image with the highest confidence benchmark is selected as the pseudo-normal feature, and the image features at each stage after projection of the pseudo-normal feature are stored in the cache as normal features, resulting in the cached feature, the expression of which is: ; In the formula, This represents normal characteristics in the cache, i.e., cache characteristics. This means selecting the four images with the highest confidence baseline values, projecting them, and storing the image features extracted from each stage from shallow to deep layers as normal features in the cache; The cosine similarity between the cached features and the projected image features is calculated stage by stage to generate fine-grained anomaly heatmaps for each stage. The expression is as follows: ; In the formula, This represents the fine-grained anomaly response diagram for the i-th stage. This indicates normal characteristics in the cache.
[0013] Furthermore, the preliminary anomaly heatmaps and fine-grained anomaly heatmaps from each stage are weighted and fused to obtain a text-based fused heatmap and a cache-based fused heatmap, the expressions of which are: , ; In the formula, This represents a text-based fusion heatmap. Indicates the first Weighting coefficients for each stage This represents a text-based fusion heatmap. Indicates the first Weighting coefficients for each stage; The text-based fusion heatmap and the cache-based fusion heatmap are weighted and calculated to obtain the final fusion heatmap, which is expressed as follows:
[0014] In the formula, This represents the final fusion heatmap; The weights represent the weights in the text fusion heatmap; The maximum value is extracted from the final fused heatmap as the pixel-level anomaly score, and its expression is: ; In the formula, Indicates pixel-level anomaly scores; The pixel-level anomaly score is weighted and fused with the confidence benchmark to obtain the final image-level anomaly score, which is expressed as follows:
[0015] In the formula, This indicates the final abnormal score. This indicates that the weights are dynamically adjusted.
[0016] Secondly, embodiments of this application also provide a system for the zero-sample industrial anomaly detection method as described in the above aspects, the system comprising: The encoding module is used to extract multi-stage image features and global image features from the input image, and generate global text features for the corresponding category through the text encoder; The projection module is used to calculate the weight of each expert for the image features at each stage through a sparse hybrid expert system, select the K different experts with the largest weights, and use them to project and fuse the original image features at each stage to obtain the image features of each stage after projection. The cross-modal alignment module is used to perform cross-modal alignment between text features and projected image features; The preliminary anomaly heatmap generation module is used to calculate the cosine similarity between global text features and image features at each stage after projection, and generate preliminary anomaly heatmaps corresponding to each stage. The confidence benchmark acquisition module is used to calculate the cosine similarity between global image features and global text features to obtain the confidence benchmark. The dynamic caching module is used to dynamically build and update the cache. It dynamically builds and updates the cache based on the confidence benchmark, selects the image with the highest confidence benchmark as the pseudo-normal feature, and saves the image features of each stage after the pseudo-normal feature is projected as normal features in the cache to obtain cache features. The fine-grained anomaly heatmap generation module is used to calculate the cosine similarity between cached features and projected image features in stages, and generate fine-grained anomaly heatmaps corresponding to each stage. The fusion module is used to perform weighted fusion of the preliminary anomaly heatmaps and fine-grained anomaly heatmaps from each stage to obtain a text-based fusion heatmap and a cache-based fusion heatmap; then, the text-based fusion heatmap and the cache-based fusion heatmap are weighted and calculated to obtain the final fusion heatmap. The pixel-level anomaly score extraction module is used to extract the maximum value from the final fused heatmap as the pixel-level anomaly score. The final anomaly score generation module is used to weight and fuse pixel-level anomaly scores with confidence benchmarks to obtain the final image-level anomaly score.
[0017] Thirdly, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of the zero-sample industrial anomaly detection method as described in the preceding aspects.
[0018] Fourthly, a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the zero-sample industrial anomaly detection method as described in the preceding aspects.
[0019] As can be seen from the above technical solutions, the present invention has the following advantages: This application provides a zero-sample industrial anomaly detection method, system, equipment, and medium that utilizes the fusion of fine-grained anomaly response maps and preliminary anomaly heatmaps to improve detection resolution and accuracy while optimizing computational efficiency, thus meeting the real-time requirements of zero-sample industrial detection.
[0020] By constructing a dynamic cache, the cached features are always the most representative normal features, which improves the model's generalization ability and enables it to adapt to the inspection of different types of industrial products.
[0021] By combining confidence benchmarks with pixel-level anomaly scores, the reliability of anomaly scores is improved, enabling high-precision industrial anomaly detection. Attached Figure Description
[0022] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 This is a flowchart illustrating the zero-sample industrial anomaly detection method provided in this embodiment of the invention.
[0024] Figure 2 This is an architecture diagram of the zero-sample industrial anomaly detection method provided in this embodiment of the invention.
[0025] Figure 3 This is a schematic diagram of cross-modal feature alignment provided in an embodiment of the present invention.
[0026] Figure 4 This is an experimental result diagram of using the detection method of the present invention to detect anomalies in industrial products. Detailed Implementation
[0027] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this patent, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this patent.
[0028] This application provides a zero-sample industrial anomaly detection method, system, device, and medium, which solves the technical problem that there is an urgent need for a method to achieve the insufficient accuracy of existing detection methods in locating anomaly areas.
[0029] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.
[0030] Figure 1 This is a flowchart illustrating a zero-sample industrial anomaly detection method provided in an embodiment of this application. Figure 1 As shown in the figure, the zero-sample industrial anomaly detection method provided in this application specifically includes the following steps: Step S1: Extract the global image feature representation and multi-stage image feature representation of the input image, and generate the global text features of the corresponding category through the text encoder; It should be noted that the visual encoder is a Visual Transformer (VIT), whose multi-level features are divided into four stages, corresponding to feature extraction from shallow to deep layers respectively. Multi-level features of the test image are extracted by a visual encoder, resulting in image features at four stages from shallow to deep.
[0031] In one exemplary embodiment, a visual encoder extracts multi-level features from the test image to obtain image features in four stages from shallow to deep, specifically including: The test image is input into the visual encoder. After processing by the visual encoder, the image features in four stages are obtained, and their expressions are as follows: , ; In the formula, Indicates the first Image features at each stage Represents global image features. This represents the test image. This represents a visual encoder.
[0032] According to another embodiment of the present invention, the natural language description of category label c is encoded by a text encoder to obtain global text features, the expression of which is as follows: ; Step S2: Using a sparse hybrid expert system, calculate the weight of each expert for the image features of each stage, select the K different experts with the largest weights, and use them to project and fuse the original image features of each stage to obtain the projected image features of each stage.
[0033] According to another embodiment of the present invention, in step S2, a sparse hybrid expert system is used to calculate the weight of each expert for the image features of each stage, select the K different experts with the largest weights, and use them to project and fuse the original image features of each stage to obtain the projected image features of each stage, specifically including: Step S21: For the first For each stage of image features, calculate the weight of each expert, and select the K different experts with the highest weights. The expression is as follows: , ; In the formula, Indicates the first Image features at each stage, , Indicates the parameters of the gating network. This represents the normalized weights of N experts. This represents the K experts with the highest weights; Step S22: Using the selected K experts respectively... Project the image and then weight and fuse the results to obtain the projected image. The image features at each stage are expressed as follows: , ; In the formula, Experts The output, Experts Network parameters, For activation function, Indicates the projected number of... Image features at each stage.
[0034] Step S3: Align the text features with the projected image features across modalities.
[0035] This invention projects the original image features from each stage into a unified embedding space using a sparse hybrid expert system and aligns them with text features across modalities. By leveraging the sparse projection mechanism and cross-modal alignment technology of the sparse hybrid expert system, it solves the computational redundancy problem caused by complex coding networks in traditional methods, achieving dimensionality reduction and unified representation of features, reducing computational load, and significantly improving real-time detection efficiency in industrial scenarios.
[0036] Step S4: Calculate the cosine similarity between the global text features and the image features at each stage after projection, and generate preliminary anomaly heatmaps corresponding to each stage.
[0037] According to an embodiment of this application, in step S4, the cosine similarity between the global text features and the image features at each stage after projection is calculated to generate preliminary anomaly heatmaps corresponding to each stage, specifically including: Step S41: Calculate the cosine similarity between the global text features and the image features at each stage after projection, and obtain the similarity matrix for each stage. The expression is as follows: ; In the formula, Indicates the first Similarity matrix for each stage, Represents global text features.
[0038] Step S42: Based on the similarity matrix of each stage, generate a preliminary anomaly heatmap, the expression of which is: ; In the formula, Indicates the first Preliminary anomaly heatmaps for each stage.
[0039] Step S5: Calculate the cosine similarity between global image features and global text features to obtain the confidence benchmark.
[0040] In one embodiment, in step S5, the cosine similarity between global image features and global text features is calculated to obtain the confidence benchmark, the expression of which is: ; In the formula, Indicates the confidence level benchmark. Represents global text features; Step S6: Dynamically construct and update the cache based on the confidence benchmark, select the image with the highest confidence benchmark as the pseudo-normal feature, and save the image features of each stage after the pseudo-normal feature is projected as normal features in the cache to obtain the cached features.
[0041] In one embodiment, in step S6, the cache is dynamically constructed and updated based on the confidence benchmark. The image with the highest confidence benchmark is selected as the pseudo-normal feature, and the image features of each stage after its projection are stored in the cache as normal features. The expression is as follows: ; In the formula, This indicates normal characteristics in the cache. This means selecting the four images with the highest confidence baseline values, and then storing the image features extracted from each stage from shallow to deep layers as normal features in the cache after projecting them.
[0042] This invention dynamically constructs and updates a cache, retaining the top-4 images after sorting them by confidence benchmark, and storing the image features extracted from each stage from shallow to deep layers as normal features in the cache. By continuously updating the most representative normal features through a dynamic caching mechanism, this invention solves the problem of insufficient generalization ability caused by fixed feature libraries in existing methods. This significantly improves the detection accuracy of the model for new categories of industrial products and effectively adapts to different detection scenarios.
[0043] Step S7: Calculate the cosine similarity between the cached features and the projected image features in each stage to generate fine-grained anomaly heatmaps for each stage. In step S7, the cosine similarity between the cached features and the projected image features is calculated stage by stage to generate fine-grained anomaly heatmaps corresponding to each stage. The expression for this heatmap is as follows: ; In the formula, Indicates the first Fine-grained anomaly response plots for each stage. This indicates normal characteristics in the cache.
[0044] Step S8: The preliminary anomaly heatmaps and fine-grained anomaly heatmaps from each stage are weighted and fused to obtain a text-based fused heatmap and a cache-based fused heatmap.
[0045] As an example, in step S8, the preliminary anomaly heatmaps and fine-grained anomaly heatmaps from each stage are weighted and fused to obtain a text-based fused heatmap and a cache-based fused heatmap, the expression of which is: , ; In the formula, This represents a text-based fusion heatmap. Indicates the first Weighting coefficients for each stage This represents a text-based fusion heatmap. Indicates the first The weighting coefficients for each stage.
[0046] The text-based fusion heatmap and the cache-based fusion heatmap are weighted and calculated to obtain the final fusion heatmap, which is expressed as follows:
[0047] This invention calculates the similarity between cached normal features and projected image features in stages to generate a fine-grained anomaly response map, which is then weighted and fused with a preliminary anomaly heatmap. By using this fine-grained response map and heatmap fusion technique, the invention solves the problem of insufficient accuracy in anomaly localization using traditional methods, achieving pixel-level anomaly score extraction. This reduces the anomaly region localization error to the pixel level, significantly improving detection resolution and meeting the high-precision requirements of industrial scenarios.
[0048] Step S9: Extract the maximum value from the final fused heatmap as the pixel-level anomaly score.
[0049] In step S9, the maximum value is extracted from the final fused heatmap as the pixel-level anomaly score, and its expression is: ; In the formula, This indicates pixel-level anomaly scores.
[0050] Step S10: The pixel-level anomaly score is weighted and fused with the confidence benchmark to obtain the final image-level anomaly score.
[0051] In step S10, the pixel-level anomaly score is weighted and fused with the confidence benchmark to obtain the final image-level anomaly score, which is expressed as:
[0052] In the formula, This indicates the final abnormal score. This indicates that the weights are dynamically adjusted.
[0053] This invention extracts multi-level features (four stages from shallow to deep) from test images and combines pixel-level anomaly scores with confidence benchmarks to generate a final anomaly score. By fusing multi-level features and calculating dual-dimensional anomaly scores, it solves the scoring bias problem caused by insufficient feature utilization in existing methods, significantly improving the reliability of anomaly scores and effectively reducing false positive and false negative rates.
[0054] The present invention also provides a system applied to the zero-sample industrial anomaly detection method as described in the above embodiments. The following are embodiments of the zero-sample industrial anomaly detection system provided in this disclosure. This zero-sample industrial anomaly detection system and the zero-sample industrial anomaly detection method of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the zero-sample industrial anomaly detection system, please refer to the embodiments of the zero-sample industrial anomaly detection method described above.
[0055] The system includes: The encoding module is used to extract global feature representations and feature representations at multiple stages from the input image, and to generate global text features for the corresponding category through a text encoder; The projection module is used to calculate the weight of each expert for the image features at each stage through a sparse hybrid expert system, select the K different experts with the largest weights, and use them to project and fuse the original image features at each stage to obtain the image features of each stage after projection. The cross-modal alignment module is used to perform cross-modal alignment between text features and projected image features; The preliminary anomaly heatmap generation module is used to calculate the cosine similarity between global text features and image features at each stage after projection, and generate preliminary anomaly heatmaps corresponding to each stage. The confidence benchmark acquisition module is used to calculate the cosine similarity between global image features and global text features to obtain the confidence benchmark. The dynamic caching module is used to dynamically build and update the cache. It dynamically builds and updates the cache based on the confidence benchmark, selects the image with the highest confidence benchmark as the pseudo-normal feature, and saves the image features of each stage after its projection as normal features in the cache. The fine-grained anomaly heatmap generation module is used to calculate the cosine similarity between cached features and projected image features in stages, and generate fine-grained anomaly heatmaps corresponding to each stage. The fusion module is used to perform weighted fusion of the preliminary anomaly heatmaps and fine-grained anomaly heatmaps from each stage to obtain a text-based fusion heatmap and a cache-based fusion heatmap; then, the text-based fusion heatmap and the cache-based fusion heatmap are weighted and calculated to obtain the final fusion heatmap. The pixel-level anomaly score extraction module is used to extract the maximum value from the final fused heatmap as the pixel-level anomaly score. The final anomaly score generation module is used to weight and fuse pixel-level anomaly scores with confidence benchmarks to obtain the final image-level anomaly score.
[0056] The zero-sample industrial anomaly detection method provided in this application can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of this invention does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, electronic devices include, but are not limited to, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.
[0057] Electronic devices may include processors, external memory interfaces, internal memory, universal serial bus (USB) interfaces, charging management modules, power management modules, batteries, wireless communication modules, audio modules, speakers, microphones, sensor modules, buttons, cameras, displays, and SIM card interfaces, etc.
[0058] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
[0059] A processor may include one or more processing units, such as: a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.
[0060] The processor can serve as the nerve center and command center of an electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.
[0061] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.
[0062] An external storage interface (ESI) can be used to connect external memory cards, such as microSD cards, to expand the storage capacity of electronic devices. The external memory card communicates with the processor through the ESI to perform data storage functions, such as saving music and video files on the external memory card.
[0063] Internal memory can be used to store computer executable program code, which includes instructions. The processor executes various functional applications and data processing of electronic devices by running the instructions stored in internal memory. Internal memory can include a program storage area and a data storage area. Internal memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.
[0064] Wireless communication functionality in electronic devices can be achieved through antennas, wireless communication modules, modem processors, and baseband processors.
[0065] Wireless communication modules can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.
[0066] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.
[0067] Electronic devices can achieve shooting functions through ISPs, cameras, video codecs, GPUs, displays, and application processors.
[0068] Electronic devices can achieve display functions through GPUs, displays, and application processors.
[0069] A GPU is a microprocessor for image processing, connected to the display screen and application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.
[0070] A display screen is used to display images, videos, etc. A display screen includes a display panel.
[0071] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0072] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0073] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, apparatuses, or units, or they may be electrical, mechanical, or other forms of connection.
[0074] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of the invention. However, those skilled in the art will recognize that the technical solutions of the invention can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of the invention.
[0075] The aforementioned electronic device realizes the zero-sample industrial anomaly detection method of this application, which utilizes a visual encoder to extract multi-stage image features and global image features; extracts semantic embeddings of corresponding categories through a text encoder to generate global text features; introduces a sparse hybrid expert system, calculates the weight of each expert for the image features of each stage, selects the K different experts with the largest weights, and uses them to project and fuse the original image features of each stage to obtain the projected image features of each stage; calculates the cosine similarity between the global text features and the projected image features of each stage to generate preliminary anomaly heatmaps corresponding to each stage; calculates the cosine similarity between the global image features and the global text features to obtain a confidence benchmark; and dynamically constructs anomaly heatmaps based on the confidence benchmark. The algorithm updates the cache, selects the image with the highest confidence benchmark as the pseudo-normal feature, and saves the image features of each stage after projection as normal features in the cache, thus obtaining cached features. The cosine similarity between the cached features and the projected image features is calculated stage by stage to generate fine-grained anomaly heatmaps for each stage. The preliminary anomaly heatmaps and fine-grained anomaly heatmaps of each stage are then weighted and fused to obtain a text-based fused heatmap and a cache-based fused heatmap. The text-based fused heatmap and the cache-based fused heatmap are then weighted to obtain the final fused heatmap. The maximum value is extracted from the final fused heatmap as the pixel-level anomaly score, and the pixel-level anomaly score is weighted and fused with the confidence benchmark to obtain the final image-level anomaly score. This achieves efficient and accurate zero-shot anomaly detection.
[0076] The storage medium provided in this application stores a program product capable of implementing a zero-sample industrial anomaly detection method.
[0077] A visual encoder is used to extract multi-stage image features and global image features; a text encoder is used to extract semantic embeddings of corresponding categories to generate global text features; a sparse hybrid expert system is introduced, and for the image features of each stage, the weight of each expert is calculated. The K experts with the largest weights are selected, and they are used to project and fuse the original image features of each stage to obtain the projected image features of each stage; the cosine similarity between the global text features and the projected image features of each stage is calculated to generate preliminary anomaly heatmaps for each stage; the cosine similarity between the global image features and the global text features is calculated to obtain the confidence benchmark; a cache is dynamically constructed and updated based on the confidence benchmark, and the confidence base is selected. The image with the highest accuracy is used as the pseudo-normal feature, and the image features of each stage after projection are stored in the cache as normal features to obtain cached features. The cosine similarity between the cached features and the projected image features is calculated stage by stage to generate fine-grained anomaly heatmaps corresponding to each stage. The preliminary anomaly heatmaps and fine-grained anomaly heatmaps of each stage are weighted and fused to obtain a text-based fused heatmap and a cache-based fused heatmap. The text-based fused heatmap and the cache-based fused heatmap are weighted and calculated to obtain the final fused heatmap. The maximum value is extracted from the final fused heatmap as the pixel-level anomaly score, and the pixel-level anomaly score is weighted and fused with the confidence benchmark to obtain the final image-level anomaly score.
[0078] By constructing a dynamic cache, the cached features are ensured to always be the most representative normal features, improving the model's generalization ability and enabling it to adapt to the inspection of different categories of industrial products. The fusion of fine-grained anomaly response maps and preliminary anomaly heatmaps improves detection resolution and accuracy while optimizing computational efficiency, meeting the real-time requirements of zero-shot industrial inspection. Combining confidence benchmarks with pixel-level anomaly scores enhances the reliability of anomaly scores, achieving high-precision industrial anomaly detection.
[0079] In some possible implementations, the zero-sample industrial anomaly detection method of this disclosure can be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.
[0080] The storage medium disclosed herein can take the form of any combination of one or more readable media. A readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.
[0081] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
[0082] Any changes, modifications, substitutions, and variations made to the embodiments without departing from the principles and spirit of the present invention still fall within the protection scope of the present invention.
Claims
1. A zero-sample industrial anomaly detection method, characterized in that, The method includes: A visual encoder is used to extract multi-stage image features and global image features. The semantic embeddings of the corresponding categories are extracted by a text encoder to generate global text features; A sparse hybrid expert system is introduced. For the image features at each stage, the weight of each expert is calculated. The K different experts with the largest weights are selected and used to project and fuse the original image features at each stage, so as to obtain the image features of each stage after projection. Calculate the cosine similarity between global text features and image features at each stage after projection, and generate preliminary anomaly heatmaps for each stage. Calculate the cosine similarity between global image features and global text features to obtain the confidence benchmark; The cache is dynamically constructed and updated based on the confidence benchmark. The image with the highest confidence benchmark is selected as the pseudo-normal feature, and the image features of each stage after the pseudo-normal feature is projected are stored in the cache as normal features to obtain the cached features. The cosine similarity between the cached features and the projected image features is calculated stage by stage to generate fine-grained anomaly heatmaps for each stage. The preliminary anomaly heatmaps and fine-grained anomaly heatmaps from each stage are weighted and fused to obtain a text-based fused heatmap and a cache-based fused heatmap. The text-based fusion heatmap and the cache-based fusion heatmap are weighted and calculated to obtain the final fusion heatmap. The maximum value is extracted from the final fused heatmap as the pixel-level anomaly score. The pixel-level anomaly score is then weighted and fused with the confidence benchmark to obtain the final image-level anomaly score.
2. The zero-sample industrial anomaly detection method as described in claim 1, characterized in that, The visual encoder is a visual Transformer, whose multi-level features are divided into four stages, corresponding to feature extraction from shallow to deep layers. A visual encoder is used to extract multi-stage image features and global image features, and its expression is as follows: ， ； In the formula, Indicates the first Image features at each stage Represents global features of the image. This represents the test image. This represents a visual encoder.
3. The method as described in claim 1, characterized in that, The semantic embeddings of the corresponding categories are extracted through a text encoder to generate global text features, specifically including: The natural language description of category label c is encoded using a text encoder to obtain global text features, the expression of which is as follows: ; In the formula, Represents global text features. This indicates that the natural language description of the category labels is encoded using a text encoder. Indicates category label, Let D represent a real vector space of dimension D.
4. The zero-sample industrial anomaly detection method as described in claim 3, characterized in that, A sparse hybrid expert system is introduced. For the image features at each stage, the weight of each expert is calculated. The K experts with the largest weights are selected, and their weights are used to project and fuse the original image features at each stage, thereby obtaining the projected image features at each stage. Specifically, these include: For the For each stage of image features, calculate the weight of each expert, and select the K different experts with the highest weights. The expression is as follows: , ; In the formula, Indicates the first Image features at each stage, , Indicates the parameters of the gating network. This represents the normalized weights of N experts. This represents the K experts with the highest weights; Using the selected K experts respectively Project the image and then weight and fuse the results to obtain the projected image. The image features at each stage are expressed as follows: , ; In the formula, Experts The output, Experts Network parameters, For activation function, Indicates the projected first... Image features at each stage.
5. The zero-sample industrial anomaly detection method as described in claim 4, characterized in that, Calculate the cosine similarity between the global text features and the image features at each stage after projection, and generate preliminary anomaly heatmaps corresponding to each stage, specifically including: Calculate the cosine similarity between the global text features and the image features at each stage after projection, and obtain the similarity matrix for each stage. The expression is as follows: ； In the formula, Indicates the first Similarity matrix for each stage, Represents global text features; Based on the similarity matrices at each stage, a preliminary anomaly heatmap is generated, the expression of which is: ； In the formula, Indicates the first Preliminary anomaly heatmaps for each stage.
6. The zero-sample industrial anomaly detection method as described in claim 5, characterized in that, The cosine similarity between global image features and global text features is calculated to obtain the confidence benchmark, which is expressed as follows: ； In the formula, Indicates the confidence level benchmark. Represents global text features; The cache is dynamically constructed and updated based on the confidence benchmark. The image with the highest confidence benchmark is selected as the pseudo-normal feature, and the image features at each stage after projection of the pseudo-normal feature are stored in the cache as normal features, resulting in the cached features, the expression of which is: ； In the formula, This represents normal characteristics in the cache, i.e., cache characteristics. This means selecting the four images with the highest confidence baseline values, projecting them, and storing the image features extracted from each stage from shallow to deep layers as normal features in the cache; The cosine similarity between the cached features and the projected image features is calculated stage by stage to generate fine-grained anomaly heatmaps for each stage. The expression is as follows: ； In the formula, This represents the fine-grained anomaly response diagram for the i-th stage. This indicates normal characteristics in the cache.
7. The zero-sample industrial anomaly detection method as described in claim 6, characterized in that, The preliminary and fine-grained anomaly heatmaps from each stage are weighted and fused to obtain a text-based fused heatmap and a cache-based fused heatmap, expressed as follows: ， ； In the formula, This represents a text-based fusion heatmap. Indicates the first Weighting coefficients for each stage, This represents a text-based fusion heatmap. Indicates the first Weighting coefficients for each stage; The text-based fusion heatmap and the cache-based fusion heatmap are weighted and calculated to obtain the final fusion heatmap, which is expressed as follows: In the formula, This represents the final fusion heatmap; The weights represent the weights in the text fusion heatmap; The maximum value is extracted from the final fused heatmap as the pixel-level anomaly score, and its expression is: ； In the formula, Indicates pixel-level anomaly scores; The pixel-level anomaly score is weighted and fused with the confidence benchmark to obtain the final image-level anomaly score, which is expressed as follows: In the formula, This indicates the final abnormal score. This indicates that the weights are dynamically adjusted.
8. A system applied to the zero-sample industrial anomaly detection method as described in any one of claims 1-7, characterized in that, The system includes: The encoding module is used to extract multi-stage image features and global image features from the input image, and generate global text features for the corresponding category through the text encoder; The projection module is used to calculate the weight of each expert for the image features at each stage through a sparse hybrid expert system, select the K different experts with the largest weights, and use them to project and fuse the original image features at each stage to obtain the image features of each stage after projection. The cross-modal alignment module is used to perform cross-modal alignment between text features and projected image features; The preliminary anomaly heatmap generation module is used to calculate the cosine similarity between global text features and image features at each stage after projection, and generate preliminary anomaly heatmaps corresponding to each stage. The confidence benchmark acquisition module is used to calculate the cosine similarity between global image features and global text features to obtain the confidence benchmark. The dynamic caching module is used to dynamically build and update the cache. It dynamically builds and updates the cache based on the confidence benchmark, selects the image with the highest confidence benchmark as the pseudo-normal feature, and saves the image features of each stage after the pseudo-normal feature is projected as normal features in the cache to obtain cache features. The fine-grained anomaly heatmap generation module is used to calculate the cosine similarity between cached features and projected image features in stages, and generate fine-grained anomaly heatmaps corresponding to each stage. The fusion module is used to perform weighted fusion of the preliminary anomaly heatmaps and fine-grained anomaly heatmaps from each stage to obtain a text-based fusion heatmap and a cache-based fusion heatmap; then, the text-based fusion heatmap and the cache-based fusion heatmap are weighted and calculated to obtain the final fusion heatmap. The pixel-level anomaly score extraction module is used to extract the maximum value from the final fused heatmap as the pixel-level anomaly score. The final anomaly score generation module is used to weight and fuse pixel-level anomaly scores with confidence benchmarks to obtain the final image-level anomaly score.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the zero-sample industrial anomaly detection method as described in any one of claims 1-7.
10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the zero-sample industrial anomaly detection method as described in any one of claims 1-7.