Industrial image anomaly detection method, device and equipment and storage medium
By extracting, enhancing, and fusing features from the target anomaly detection model, the global semantic bias problem in industrial image anomaly detection is solved, the detection accuracy is improved, and the model's transfer and generalization capabilities are enhanced.
Patent Information
- Application Number
- CN202511737761.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-25
- Publication Date
- 2026-02-24
AI Technical Summary
Existing industrial image anomaly detection methods suffer from global semantic bias in cross-dataset transfer and few-shot learning scenarios, which limits their detection accuracy.
The target anomaly detection model performs feature extraction, enhancement, and fusion, including a feature extraction layer, a feature enhancement layer, and a feature fusion layer. It combines textual and visual features to achieve feature calibration and multimodal feature fusion, generating an anomaly probability map.
It improves the accuracy of anomaly detection in complex scenarios, enhances the model's transfer and generalization capabilities across different datasets, and has extremely high adaptability to few samples.
Smart Images
Figure CN121564408A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence, and in particular to a method, apparatus, device, and storage medium for detecting anomalies in industrial images. Background Technology
[0002] Existing anomaly detection methods mainly include rule-based methods, traditional machine learning methods, and deep learning methods. With the development of deep learning technology, anomaly detection methods based on pre-trained models have attracted widespread attention due to their ability to generalize well without requiring large amounts of labeled data. However, existing methods still suffer from feature bias in cross-dataset transfer and few-shot learning scenarios. Taking the ViT model as an example, its feature extraction exhibits significant hierarchical limitations. When extracting shallow features, it typically focuses on low-level details such as local textures and edges, excelling only at capturing "minor anomalies" (such as small scratches). When extracting deep features, it usually emphasizes global semantic information, excelling only at judging the "match between anomalies and global context" (such as whether a scratch is located in a critical structure), but losing local details. This single-level feature bias limits the effectiveness of existing methods in cross-dataset transfer and few-shot learning scenarios.
[0003] In summary, how to solve the problem of global semantic bias in anomaly detection to improve the accuracy of anomaly detection for complex industrial product images is an urgent problem to be solved. Summary of the Invention
[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and storage medium for anomaly detection in industrial images, which can solve the problem of global semantic bias in anomaly detection, thereby improving the accuracy of anomaly detection for complex industrial product images. The specific solution is as follows:
[0005] In a first aspect, this application discloses an industrial image anomaly detection method, applied to a terminal device equipped with a target anomaly detection model, comprising:
[0006] A target detection image is acquired, and features are extracted from the target detection image using the feature extraction layer of a target anomaly detection model to obtain the target features corresponding to the target detection image; the target features include target text features and target visual features; the target visual features include visual features at least one feature level; the target detection image is an image of an industrial product with defects to be detected;
[0007] The target visual features are enhanced by using the feature enhancement layer of the target anomaly detection model, based on the target feature enhancement function and the feature level corresponding to the target visual features, so as to obtain the corresponding enhanced visual features.
[0008] The enhanced visual features and target text features at each feature level are fused using the feature fusion layer of the target anomaly detection model to determine the anomaly probability map corresponding to each feature level. Based on all the anomaly probability maps, the target anomaly probability map corresponding to the target detection image is determined, and the anomaly detection result corresponding to the target detection image is generated based on the target anomaly probability map.
[0009] Optionally, the step of using the feature extraction layer of the target anomaly detection model to extract features from the target detection image to obtain the target features corresponding to the target detection image includes:
[0010] A target anomaly detection model is used to construct state-level and template-level prompts for the target detection image, and a combination of text descriptions is determined based on all the state-level and template-level prompts;
[0011] The text encoder of the target anomaly detection model determines the target text features corresponding to the target detection image based on all the combinations of the text descriptions.
[0012] The image encoder of the target anomaly detection model is used to process the target detection image to obtain the target visual features corresponding to the target detection image.
[0013] Optionally, the step of using the feature enhancement layer of the target anomaly detection model to enhance the target visual features based on the target feature enhancement function and the feature level corresponding to the target visual features to obtain the corresponding enhanced visual features includes:
[0014] The target visual features are enhanced by the feature enhancement layer of the target anomaly detection model based on the target feature enhancement function and the feature hierarchy corresponding to the target visual features. The enhanced target visual features are then mapped to the target joint embedding space by the linear mapping layer corresponding to the target visual features to obtain the corresponding enhanced visual features.
[0015] The target joint embedding space is the vector space in which the target text features are located.
[0016] Optionally, the loss function of the linear mapping layer is:
[0017] ;
[0018] in, The focus loss is the weighting coefficient used to balance the two types of losses. Used to solve the class imbalance problem, dice loss. Used to optimize boundary matching accuracy in segmentation tasks.
[0019] Optionally, fusing the enhanced visual features and the target text features at each feature level to determine the anomaly probability map corresponding to each feature level includes:
[0020] The target similarity between the enhanced visual features and the target text features at each feature level is determined, and the enhanced visual features and the target text features at each feature level are fused based on the target similarity, and the anomaly probability map corresponding to each feature level is determined.
[0021] Optionally, generating the anomaly detection result corresponding to the target detection image based on the target anomaly probability map includes:
[0022] Obtain normal reference samples; the number of normal reference samples is less than the target number of samples;
[0023] The image encoder of the target anomaly detection model is used to extract the image features of the normal reference sample, and the image features are saved to the target feature database;
[0024] Anomaly detection results corresponding to the target detection image are generated based on the target anomaly probability map and the image features in the target feature database.
[0025] Optionally, the update process of the target anomaly detection model includes:
[0026] Obtain a training sample set and a corresponding label dataset; the label dataset includes the actual anomaly maps corresponding to the training sample set.
[0027] The anomaly detection model is used to obtain the anomaly detection results corresponding to the training sample set, and the target loss gradient corresponding to the training sample set is determined based on the anomaly detection results and the label dataset.
[0028] The target parameters of the target anomaly detection model are updated based on the target loss gradient to obtain the updated target anomaly detection model.
[0029] Secondly, this application discloses an industrial image anomaly detection device, applied to a terminal device equipped with a target anomaly detection model, comprising:
[0030] The feature extraction module is used to acquire a target detection image and extract features from the target detection image using the feature extraction layer of the target anomaly detection model to obtain the target features corresponding to the target detection image; the target features include target text features and target visual features; the target visual features include visual features at least one feature level; the target detection image is an image of an industrial product with defects to be detected;
[0031] The feature enhancement module is used to enhance the target visual features based on the target feature enhancement function and the feature level corresponding to the target visual features using the feature enhancement layer of the target anomaly detection model, so as to obtain the corresponding enhanced visual features.
[0032] An anomaly detection module is used to fuse the enhanced visual features and the target text features at each feature level using the feature fusion layer of the target anomaly detection model to determine the anomaly probability map corresponding to each feature level, and to determine the target anomaly probability map corresponding to the target detection image based on all the anomaly probability maps, so as to generate the anomaly detection result corresponding to the target detection image based on the target anomaly probability map.
[0033] Thirdly, this application discloses an electronic device, including:
[0034] Memory, used to store computer programs;
[0035] A processor is used to execute the computer program to implement the aforementioned industrial image anomaly detection method.
[0036] Fourthly, this application discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned industrial image anomaly detection method.
[0037] In this application, a terminal device equipped with a target anomaly detection model acquires a target detection image when performing industrial image anomaly detection. The device then uses the feature extraction layer of the target anomaly detection model to extract features from the target detection image to obtain target features corresponding to the target detection image. The target features include target text features and target visual features. The target visual features include visual features at least one feature level. The target detection image is an image of an industrial product with defects to be detected. The target visual features are enhanced based on the target feature enhancement function and the feature level corresponding to the target visual features using the feature enhancement layer of the target anomaly detection model to obtain corresponding enhanced visual features. The enhanced visual features and target text features at each feature level are fused using the feature fusion layer of the target anomaly detection model to determine the anomaly probability map corresponding to each feature level. Based on all the anomaly probability maps, a target anomaly probability map corresponding to the target detection image is determined, and an anomaly detection result corresponding to the target detection image is generated based on the target anomaly probability map. As can be seen, this application utilizes a target anomaly detection model to extract target visual and textual features from the target detection image. Based on the feature hierarchy corresponding to the target visual features, it enhances the target visual features. Then, it performs hierarchical feature fusion of the enhanced visual and textual features, achieving feature calibration and multimodal feature fusion for the target detection image. This solves the problem of global semantic bias in anomaly detection and improves the accuracy of anomaly detection in complex scenes. Furthermore, it obtains corresponding anomaly probability maps based on feature hierarchy, then determines the target anomaly probability map based on all anomaly probability maps, and finally determines the anomaly detection result corresponding to the target detection image based on the target anomaly probability map. This gives the target anomaly detection model in this application extremely high few-sample adaptability. Feature calibration enhances the transferability of the target anomaly detection model across different datasets, resulting in strong generalization ability. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0039] Figure 1 This is a flowchart of an industrial image anomaly detection method disclosed in this application;
[0040] Figure 2 This is a schematic diagram of the structure of an industrial image anomaly detection device disclosed in this application;
[0041] Figure 3This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation
[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0043] Existing anomaly detection methods mainly include rule-based methods, traditional machine learning methods, and deep learning methods. With the development of deep learning technology, anomaly detection methods based on pre-trained models have attracted widespread attention due to their ability to generalize well without requiring large amounts of labeled data. However, existing methods still suffer from feature bias in cross-dataset transfer and few-shot learning scenarios. Taking the ViT model as an example, its feature extraction has obvious hierarchical limitations. When extracting shallow features, it usually focuses on low-level details such as local textures and edges, and is only good at capturing "minor anomalies" (such as small scratches). When extracting deep features, it usually focuses on global semantic information and is only good at judging the "match between anomalies and global context" (such as whether a scratch is located in a key structure), but loses local details. This single-level feature bias limits the effectiveness of existing methods in cross-dataset transfer and few-shot learning scenarios. To solve the above technical problems, this application discloses an industrial image anomaly detection method that can solve the problem of global semantic bias in anomaly detection, thereby improving the accuracy of anomaly detection for complex industrial product images.
[0044] See Figure 1 As shown, this embodiment of the invention discloses an industrial image anomaly detection method, applied to a terminal device equipped with a target anomaly detection model, including:
[0045] Step S11: Obtain the target detection image and use the feature extraction layer of the target anomaly detection model to extract features from the target detection image to obtain the target features corresponding to the target detection image; the target features include target text features and target visual features; the target visual features include visual features at least one feature level; the target detection image is an image of an industrial product with defects to be detected.
[0046] In this embodiment, the target detection image is an image of an industrial product with defects to be detected. After acquiring the target detection image, the feature extraction layer of the target anomaly detection model is used to extract features from the target detection image to determine the corresponding target features. The specific process may include: using the target anomaly detection model to construct state-level prompts and template-level prompts for the target detection image, and determining text description combinations based on all state-level prompts and all template-level prompts; using the text encoder of the target anomaly detection model to determine the target text features corresponding to the target detection image based on all text description combinations; and using the image encoder of the target anomaly detection model to process the target detection image to obtain the target visual features corresponding to the target detection image.
[0047] In one specific implementation, the target anomaly detection model is a pre-trained multimodal model, such as CLIP (Contrastive Language-Image Pre-training, a multimodal pre-trained model that achieves cross-modal understanding of text and images through contrastive learning). CLIP is used as the target anomaly detection model to perform few-shot detection on target images. The specific process of target feature extraction may include: constructing a two-level "state-template" text prompt set. The state-level prompt uses general descriptive terms to represent the normal / abnormal state of the target, such as "flawless [target category]" or "damaged [target category]", avoiding overly detailed local descriptions (such as "[target category] with edge gaps"); the template-level prompt selects templates suitable for anomaly detection from the 85 native ImageNet task templates of the CLIP model, removing incompatible templates such as "a strange [target category] photo"; finally, the state-level prompt and the template-level prompt are combined to generate multiple sets of text description combinations, which are input into the CLIP text encoder to extract text features. The average of all text features is then used to obtain the final target text features. (Where 2 corresponds to the normal / abnormal categories, and C is the number of feature channels,) (For the set of real numbers). Simultaneously, image features for classification are extracted from the CLIP image encoder. .
[0048] Step S12: Using the feature enhancement layer of the target anomaly detection model, enhance the target visual features based on the target feature enhancement function and the feature level corresponding to the target visual features to obtain the corresponding enhanced visual features.
[0049] In this embodiment, the feature enhancement layer of the target anomaly detection model is used to enhance the target visual features based on the target feature enhancement function and the feature hierarchy corresponding to the target visual features, thereby obtaining the corresponding enhanced visual features. This includes: enhancing the target visual features using the feature enhancement layer of the target anomaly detection model based on the target feature enhancement function and the feature hierarchy corresponding to the target visual features, and mapping the enhanced target visual features to the target joint embedding space using the linear mapping layer corresponding to the target visual features, thereby obtaining the corresponding enhanced visual features; wherein, the target joint embedding space is the vector space where the target text features reside. The loss function of the linear mapping layer is:
[0050] ;
[0051] in, The focus loss is the weighting coefficient used to balance the two types of losses. Used to solve the class imbalance problem, dice loss. This is used to optimize boundary matching accuracy in segmentation tasks. It is understood that, in this embodiment, the training process of all newly added linear layers in the target anomaly detection model can rely on this form of loss function.
[0052] In one specific implementation, if the image encoder of the target anomaly detection model is a ViT architecture, all its feature levels are divided into four feature stages, and the patch tokens features of each feature stage are extracted. (n is the stage number, H and W are the height and width of the feature map, respectively, and Cs is the number of feature channels in that stage); during feature calibration and enhancement, a linear layer is added to each feature stage. Feature enhancement is performed to strengthen the local expressive power of shallow features and the shallow expressive power of deep features, resulting in enhanced visual features. as follows:
[0053] ;
[0054] in, The weights of the linear layer in the nth stage are... For bias terms, .
[0055] Furthermore, when mapping the enhanced target visual features to the target joint embedding space using the linear mapping layer corresponding to the target visual features, a new linear layer is added for each feature stage to... Joint embedding space mapped to CLIP (with text features) (Spatial consistency), the enhanced visual features obtained after mapping as follows:
[0056] ;
[0057] in, The weights of the linear layer in the nth stage are... For bias terms, .
[0058] In one specific implementation, when setting the semantic alignment linear layer of the target anomaly detection model (taking CLIP as an example), a new linear layer is added to each of the four feature stages of ViT (feature levels 1-4, 5-8, 9-12, and 13-16) as a semantic alignment linear layer, with the input dimension being the number of feature channels in each stage and the output dimension being 512. Furthermore, a new linear layer is added to each of the four feature stages of ViT (feature levels 1-4, 5-8, 9-12, and 13-16) as a feature enhancement layer, with the input dimension being the number of feature channels in each stage and the output dimension being 512. Compared to complex end-to-end models, the target anomaly detection model proposed in this embodiment has high computational efficiency, a simple model structure, and is more suitable for deployment in real-world industrial environments.
[0059] Step S13: The enhanced visual features and target text features of each feature level are fused using the feature fusion layer of the target anomaly detection model to determine the anomaly probability map corresponding to each feature level, and the target anomaly probability map corresponding to the target detection image is determined based on all the anomaly probability maps, so as to generate the anomaly detection result corresponding to the target detection image based on the target anomaly probability map.
[0060] In this embodiment, when using the feature fusion layer of the target anomaly detection model to fuse the enhanced visual features and target text features at each feature level to determine the anomaly probability map corresponding to each feature level, it may specifically include: determining the target similarity between the enhanced visual features and target text features at each feature level, fusing the enhanced visual features and target text features at each feature level based on the target similarity, and determining the anomaly probability map corresponding to each feature level. That is, calculating... and The similarity is converted into a probability distribution s using the softmax function, as shown in the following formula:
[0061] ;
[0062] The probability value corresponding to the "abnormal" category in the probability distribution is taken as the abnormal classification score of the detected image of the target, thereby obtaining the corresponding target similarity.
[0063] In one specific implementation, calculation is performed hierarchically, feature by feature. and The similarity is used to obtain the anomaly probability map corresponding to the feature level through the softmax function; then, the anomaly probability maps corresponding to all feature levels are summed to obtain the final anomaly segmentation map M as the anomaly detection result corresponding to the target detection image. M can be expressed as:
[0064] .
[0065] In this embodiment, generating anomaly detection results corresponding to the target detection image based on the target anomaly probability map includes: obtaining normal reference samples; the number of normal reference samples is less than the number of target samples; extracting image features of the normal reference samples using the image encoder of the target anomaly detection model, and saving the image features to the target feature database; generating anomaly detection results corresponding to the target detection image based on the target anomaly probability map and the image features in the target feature database. That is, in this embodiment, a small number of labeled normal reference samples (less than 4 samples) can be input, and the features of the reference samples can be extracted by the CLIP image encoder, stored in the target feature database, and used as a "memory" to store the reference sample features. Then, when performing industrial image anomaly detection, the linear layers of the target anomaly detection model trained in the zero-sample detection stage (i.e., retaining...) are reused. and The weights are adjusted without retraining, and features of the target detection image (including classification features and segmentation features at each stage) are extracted. On the one hand, anomaly scores and segmentation maps are generated through a zero-shot classification / segmentation process. On the other hand, the similarity between the test sample features and the reference sample features in the "memory bank" is compared to help optimize the anomaly detection results.
[0066] In this embodiment, the update process of the target anomaly detection model includes: obtaining a training sample set and a corresponding label dataset; the label dataset includes the actual anomaly map corresponding to the training sample set; using the anomaly detection model to obtain the anomaly detection result corresponding to the training sample set, and determining the target loss gradient corresponding to the training sample set based on the anomaly detection result and the label dataset; updating the target parameters of the target anomaly detection model based on the target loss gradient to obtain the updated target anomaly detection model.
[0067] In one specific implementation, after the input target detection image undergoes multi-stage feature extraction via CLIP, each Patchtoken is processed by its respective linear layer, while a text branch generates cue embeddings. On one hand, the processed Patchtokens are compared with the text embeddings to calculate similarity, generating a predicted anomaly map (used for...). Loss 1); On the other hand, after processing, the deep patch tokens are compared with the shallower patch tokens, and the shallower patch tokens are compared with the deeper patch tokens to generate an attention distribution (used for...). Loss 2). Calculate simultaneously. Loss 1 and Loss 2, and then weighted and merged into total loss. :
[0068] ;
[0069] The weights can be set as follows: .
[0070] The target loss gradient obtained in the above process applies to both types of linear layer parameters simultaneously. Loss 1 primarily updates the linear layer parameters of the visual branch Patch tokens. Loss 2 only updates the parameters of the visual branch calibration; the gradients of both are updated synchronously in the same optimizer (Adam), achieving coordinated parameter adjustment.
[0071] During training, the Adam optimizer can be used, with training parameters set to a batch size of 16, a learning rate of 1e-4, and 50 training epochs, thus enabling the training and updating of the target anomaly detection model. When using the target anomaly detection model to detect anomalies in target images, the model can determine the target anomaly probability map of the image, then use a threshold of 0.5 to determine whether an anomaly is present, and finally scale the final anomaly map to match the original. Figure 1 The size is determined, and the abnormal region is divided with a threshold of 0.3 to obtain the final abnormal detection result.
[0072] As can be seen, this application utilizes a target anomaly detection model to extract target visual and textual features from the target detection image. Based on the feature hierarchy corresponding to the target visual features, it enhances the target visual features. Then, it performs hierarchical feature fusion of the enhanced visual and textual features, achieving feature calibration and multimodal feature fusion for the target detection image. This solves the problem of global semantic bias in anomaly detection and improves the accuracy of anomaly detection in complex scenes. Furthermore, it obtains corresponding anomaly probability maps based on feature hierarchy, then determines the target anomaly probability map based on all anomaly probability maps, and finally determines the anomaly detection result corresponding to the target detection image based on the target anomaly probability map. This gives the target anomaly detection model in this application extremely high few-sample adaptability. Feature calibration enhances the transferability of the target anomaly detection model across different datasets, resulting in strong generalization ability.
[0073] See Figure 2 As shown, this application discloses an industrial image anomaly detection device, applied to a terminal device equipped with a target anomaly detection model, comprising:
[0074] The feature extraction module 11 is used to acquire a target detection image and extract features from the target detection image using the feature extraction layer of the target anomaly detection model to obtain the target features corresponding to the target detection image; the target features include target text features and target visual features; the target visual features include visual features at least one feature level; the target detection image is an image of an industrial product with defects to be detected;
[0075] Feature enhancement module 12 is used to enhance the target visual features based on the target feature enhancement function and the feature level corresponding to the target visual features using the feature enhancement layer of the target anomaly detection model, so as to obtain the corresponding enhanced visual features.
[0076] The anomaly detection module 13 is used to fuse the enhanced visual features and the target text features at each feature level using the feature fusion layer of the target anomaly detection model to determine the anomaly probability map corresponding to each feature level, and to determine the target anomaly probability map corresponding to the target detection image based on all the anomaly probability maps, so as to generate the anomaly detection result corresponding to the target detection image based on the target anomaly probability map.
[0077] As can be seen, this application utilizes a target anomaly detection model to extract target visual and textual features from the target detection image. Based on the feature hierarchy corresponding to the target visual features, it enhances the target visual features. Then, it performs hierarchical feature fusion of the enhanced visual and textual features, achieving feature calibration and multimodal feature fusion for the target detection image. This solves the problem of global semantic bias in anomaly detection and improves the accuracy of anomaly detection in complex scenes. Furthermore, it obtains corresponding anomaly probability maps based on feature hierarchy, then determines the target anomaly probability map based on all anomaly probability maps, and finally determines the anomaly detection result corresponding to the target detection image based on the target anomaly probability map. This gives the target anomaly detection model in this application extremely high few-sample adaptability. Feature calibration enhances the transferability of the target anomaly detection model across different datasets, resulting in strong generalization ability.
[0078] In one specific embodiment, the feature extraction module 11 may include:
[0079] The prompt combination unit is used to construct state-level prompts and template-level prompts for the target detection image using a target anomaly detection model, and to determine a text description combination based on all the state-level prompts and all the template-level prompts;
[0080] A text feature determination unit is used to determine the target text features corresponding to the target detection image based on all the text description combinations using the text encoder of the target anomaly detection model.
[0081] The visual feature determination unit is used to process the target detection image using the image encoder of the target anomaly detection model to obtain the target visual features corresponding to the target detection image.
[0082] In one specific embodiment, the feature enhancement module 12 may include:
[0083] The visual feature enhancement unit is used to enhance the target visual features based on the target feature enhancement function and the feature level corresponding to the target visual features using the feature enhancement layer of the target anomaly detection model, and to map the enhanced target visual features to the target joint embedding space using the linear mapping layer corresponding to the target visual features to obtain the corresponding enhanced visual features.
[0084] The target joint embedding space is the vector space in which the target text features are located.
[0085] In one specific embodiment, the anomaly detection module 13 may include:
[0086] An anomaly probability map acquisition unit is used to determine the target similarity between the enhanced visual features and the target text features at each feature level, so as to fuse the enhanced visual features and the target text features at each feature level based on the target similarity, and determine the anomaly probability map corresponding to each feature level.
[0087] In one specific embodiment, the anomaly detection module 13 may include:
[0088] A reference sample acquisition unit is used to acquire normal reference samples; the number of normal reference samples is less than the number of target samples.
[0089] The image feature storage unit is used to extract the image features of the normal reference sample using the image encoder of the target anomaly detection model, and save the image features to the target feature database;
[0090] The detection result acquisition unit is used to generate anomaly detection results corresponding to the target detection image based on the target anomaly probability map and the image features in the target feature database.
[0091] In one specific embodiment, the device may further include:
[0092] The training set acquisition unit is used to acquire a training sample set and a corresponding label dataset; the label dataset includes the actual anomaly map corresponding to the training sample set.
[0093] The loss extraction and acquisition unit is used to obtain the anomaly detection results corresponding to the training sample set using the anomaly detection model, and to determine the target loss gradient corresponding to the training sample set based on the anomaly detection results and the label dataset.
[0094] The model update unit is used to update the target parameters of the target anomaly detection model based on the target loss gradient, so as to obtain the updated target anomaly detection model.
[0095] Furthermore, embodiments of this application also disclose an electronic device, Figure 3 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.
[0096] Figure 3 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the industrial image anomaly detection method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.
[0097] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0098] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.
[0099] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the industrial image anomaly detection method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.
[0100] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed industrial image anomaly detection method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.
[0101] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.
[0102] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0103] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0104] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0105] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for detecting anomalies in industrial images, characterized in that, Applied to terminal devices equipped with target anomaly detection models, including: A target detection image is acquired, and features are extracted from the target detection image using the feature extraction layer of a target anomaly detection model to obtain the target features corresponding to the target detection image; the target features include target text features and target visual features; the target visual features include visual features at least one feature level; the target detection image is an image of an industrial product with defects to be detected; The target visual features are enhanced by using the feature enhancement layer of the target anomaly detection model, based on the target feature enhancement function and the feature level corresponding to the target visual features, so as to obtain the corresponding enhanced visual features. The enhanced visual features and target text features at each feature level are fused using the feature fusion layer of the target anomaly detection model to determine the anomaly probability map corresponding to each feature level. Based on all the anomaly probability maps, the target anomaly probability map corresponding to the target detection image is determined, and the anomaly detection result corresponding to the target detection image is generated based on the target anomaly probability map.
2. The industrial image anomaly detection method according to claim 1, characterized in that, The step of extracting features from the target detection image using the feature extraction layer of the target anomaly detection model to obtain the target features corresponding to the target detection image includes: A target anomaly detection model is used to construct state-level and template-level prompts for the target detection image, and a combination of text descriptions is determined based on all the state-level and template-level prompts; The text encoder of the target anomaly detection model determines the target text features corresponding to the target detection image based on all the combinations of the text descriptions. The image encoder of the target anomaly detection model is used to process the target detection image to obtain the target visual features corresponding to the target detection image.
3. The industrial image anomaly detection method according to claim 1, characterized in that, The feature enhancement layer of the target anomaly detection model enhances the target visual features based on the target feature enhancement function and the feature hierarchy corresponding to the target visual features to obtain the corresponding enhanced visual features, including: The target visual features are enhanced by the feature enhancement layer of the target anomaly detection model based on the target feature enhancement function and the feature hierarchy corresponding to the target visual features. The enhanced target visual features are then mapped to the target joint embedding space by the linear mapping layer corresponding to the target visual features to obtain the corresponding enhanced visual features. The target joint embedding space is the vector space in which the target text features are located.
4. The industrial image anomaly detection method according to claim 3, characterized in that, The loss function of the linear mapping layer is: ； in, The focus loss is the weighting coefficient used to balance the two types of losses. Used to solve the class imbalance problem, dice loss. Used to optimize boundary matching accuracy in segmentation tasks.
5. The industrial image anomaly detection method according to claim 1, characterized in that, The process of fusing the enhanced visual features and the target text features at each feature level to determine the anomaly probability map corresponding to each feature level includes: The target similarity between the enhanced visual features and the target text features at each feature level is determined, and the enhanced visual features and the target text features at each feature level are fused based on the target similarity, and the anomaly probability map corresponding to each feature level is determined.
6. The industrial image anomaly detection method according to claim 1, characterized in that, The step of generating the anomaly detection result corresponding to the target detection image based on the target anomaly probability map includes: Obtain normal reference samples; the number of normal reference samples is less than the target number of samples; The image encoder of the target anomaly detection model is used to extract the image features of the normal reference sample, and the image features are saved to the target feature database; Anomaly detection results corresponding to the target detection image are generated based on the target anomaly probability map and the image features in the target feature database.
7. The industrial image anomaly detection method according to any one of claims 1 to 6, characterized in that, The update process of the target anomaly detection model includes: Obtain a training sample set and a corresponding label dataset; the label dataset includes the actual anomaly maps corresponding to the training sample set. The anomaly detection model is used to obtain the anomaly detection results corresponding to the training sample set, and the target loss gradient corresponding to the training sample set is determined based on the anomaly detection results and the label dataset. The target parameters of the target anomaly detection model are updated based on the target loss gradient to obtain the updated target anomaly detection model.
8. An industrial image anomaly detection device, characterized in that, Applied to terminal devices equipped with target anomaly detection models, including: The feature extraction module is used to acquire a target detection image and extract features from the target detection image using the feature extraction layer of the target anomaly detection model to obtain the target features corresponding to the target detection image; the target features include target text features and target visual features; the target visual features include visual features at least one feature level; the target detection image is an image of an industrial product with defects to be detected; The feature enhancement module is used to enhance the target visual features based on the target feature enhancement function and the feature level corresponding to the target visual features using the feature enhancement layer of the target anomaly detection model, so as to obtain the corresponding enhanced visual features. An anomaly detection module is used to fuse the enhanced visual features and the target text features at each feature level using the feature fusion layer of the target anomaly detection model to determine the anomaly probability map corresponding to each feature level, and to determine the target anomaly probability map corresponding to the target detection image based on all the anomaly probability maps, so as to generate the anomaly detection result corresponding to the target detection image based on the target anomaly probability map.
9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the industrial image anomaly detection method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the industrial image anomaly detection method as described in any one of claims 1 to 7.