An image anomaly detection method, device, equipment and storage medium

Abstract

The application discloses an image anomaly detection method, device and equipment and a storage medium. The method comprises the following steps: acquiring an input image, performing feature coding on the input image, and obtaining an intermediate feature set of the input image; performing weighted fusion on the intermediate feature set, performing constraint training and feature reconstruction processing on the weighted fusion result, and obtaining a reconstructed feature; performing grouping fusion on the intermediate feature set, calculating the difference between the grouping fusion result and the reconstructed feature, and obtaining an anomaly detection result. The application can effectively inhibit the direct reconstruction of abnormal details in the target image by performing constraint training and reconstruction processing on the weighted fusion feature and calculating the difference between the grouping fusion feature and the reconstructed feature, improve the stability and positioning accuracy of abnormal recognition, reduce the calculation and maintenance cost caused by multi-model deployment, and has good generalization ability and engineering application value.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image analysis technology, and specifically relates to an image anomaly detection method, device, equipment and storage medium. Background Technology

[0002] Most existing image anomaly detection methods rely on single-layer features or final features for anomaly discrimination. This approach is prone to overfitting to local anomaly details, leading to incorrect anomaly reconstruction and reduced detection accuracy. Furthermore, in typical industrial scenarios, image anomaly detection often faces complex situations such as uniform backgrounds, textured backgrounds (e.g., periodic fabrics), or structural backgrounds, making it difficult for single-layer features to handle all types of anomalies. In addition, in multi-class anomaly detection scenarios, building models separately for each class significantly increases training and deployment costs, limiting the engineering application capabilities of detection methods.

[0003] Therefore, it is necessary to provide improved technical solutions to overcome the above-mentioned technical problems existing in the prior art. Summary of the Invention

[0004] The purpose of this application is to provide an image anomaly detection method, apparatus, device and storage medium, which can effectively suppress the direct reconstruction of abnormal details in the target image by constraining and training the weighted fusion features and performing reconstruction processing, and calculating the difference between the grouped fusion features and the reconstructed features.

[0005] To achieve the above objectives: In a first aspect, embodiments of this application provide an image anomaly detection method, including: The input image is acquired, and its features are encoded to obtain an intermediate feature set of the input image. The intermediate feature set is weighted and fused, and the weighted fusion result is subjected to constrained training and feature reconstruction processing to obtain the reconstructed features; The intermediate feature set is grouped and fused, and the difference between the grouped and fused result and the reconstructed features is calculated to obtain the anomaly detection result.

[0006] In one embodiment, prior to acquiring the input image, the method further includes: Initialize the feature level index list for feature encoding, and the fusion grouping parameters for weighted fusion.

[0007] In one embodiment, the step of performing feature encoding on the input image to obtain an intermediate feature set of the input image includes: The input image is subjected to layer-by-layer feature encoding, and the result of the layer-by-layer feature encoding is subjected to dimension alignment to obtain the intermediate feature set of the input image.

[0008] In one embodiment, the weighted fusion of the intermediate feature set includes: Linear attention is calculated for each feature in the intermediate feature set to obtain the fusion weight of each feature, and then the fusion weight is weighted and fused.

[0009] In one embodiment, the process of constraining and reconstructing the weighted fusion result to obtain reconstructed features includes: The weighted fusion results are trained under constraints using a pre-defined network model. The fused features after constrained training are reconstructed to obtain the reconstructed features.

[0010] In one embodiment, the grouping and fusion of the intermediate feature set includes: According to the preset fusion grouping strategy, the features in the intermediate feature set and the reconstructed features are grouped and fused.

[0011] In one embodiment, the difference between the calculated group fusion result and the reconstructed features yields an anomaly detection result, including: The difference between the group fusion result and the reconstructed features is calculated by cosine similarity to obtain the feature difference value; If the feature difference value exceeds a preset difference threshold, the image region corresponding to the feature difference value is determined to be an abnormal region.

[0012] Secondly, embodiments of this application provide an image anomaly detection device, comprising: The feature encoding module is used to acquire the input image and perform feature encoding on the input image to obtain an intermediate feature set of the input image; The feature reconstruction module is used to perform weighted fusion on the intermediate feature set, and to perform constrained training and feature reconstruction processing on the weighted fusion result to obtain reconstructed features; An anomaly detection module is used to group and fuse the intermediate feature set, and calculate the difference between the group fusion result and the reconstructed features to obtain the anomaly detection result.

[0013] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor, wherein the memory stores executable program code, and when the executable program code is executed by the processor, it implements the steps of the image anomaly detection method as described in the first aspect.

[0014] Fourthly, embodiments of this application provide a readable storage medium, characterized in that the readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the image anomaly detection method as described in the first aspect.

[0015] The image anomaly detection method, apparatus, device, and storage medium provided in this application include: acquiring an input image and performing feature encoding on the input image to obtain an intermediate feature set; performing weighted fusion on the intermediate feature set, and performing constraint training and feature reconstruction processing on the weighted fusion result to obtain reconstructed features; performing group fusion on the intermediate feature set, and calculating the difference between the group fusion result and the reconstructed features to obtain an anomaly detection result. This application, by performing constraint training and reconstruction processing on the weighted fusion features, and calculating the difference between the group fusion features and the reconstructed features, can effectively suppress the direct reconstruction of abnormal details in the target image, improve the stability and localization accuracy of anomaly recognition, reduce the computational and maintenance costs caused by multi-model deployment, and has good generalization ability and engineering application value. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the image anomaly detection method provided in an embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram of the structure for image anomaly detection provided in an embodiment of the present invention.

[0019] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0021] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0022] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if," as used herein, can be interpreted as "when," "when," or "in response to determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising," "including," indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" as used herein are to be interpreted as inclusive, or mean any one or any combination thereof. Therefore, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A, B, and C". Exceptions to this definition will only occur if the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0023] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0024] It should be noted that step designations such as S110 and S120 are used in this document for the purpose of more clearly and concisely describing the corresponding content, and do not constitute a substantial limitation on the order. In specific implementation, those skilled in the art may execute S120 first and then S110, etc., but these should all be within the protection scope of this application.

[0025] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0026] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0027] See Figure 1 This application provides an image anomaly detection method, which can be executed by an image anomaly detection device provided in this application. The device can be implemented in software and / or hardware. In this embodiment, the image anomaly detection device is taken as the executing entity of the method. The image anomaly detection method provided in this embodiment includes the following steps: Step S110: Obtain the input image and perform feature encoding on the input image to obtain the intermediate feature set of the input image.

[0028] It is understandable that by encoding the input image layer by layer, intermediate features are extracted from each layer. These intermediate features can capture different semantic information and details of the image at different levels, thus providing rich feature support for subsequent processing.

[0029] Specifically, an expression for an intermediate feature set provided in this application embodiment is as follows:

[0030] in, For the input image, For the first Hierarchical feature extraction mapping For the first Intermediate features of the layer output.

[0031] In one embodiment, before acquiring the input image, the method further includes: Initialize the feature level index list for feature encoding, and the fusion grouping parameters for weighted fusion.

[0032] It is understandable that during feature encoding, it is necessary to specify which layers' features will participate in subsequent fusion operations. This embodiment specifies these layers by initializing a layer index set, the calculation formula for which is as follows:

[0033] in, For hierarchical index sets, For the first The selected hierarchical index, The number of layers participating in subsequent integration operations.

[0034] Furthermore, in order to more effectively fuse features from different levels, embodiments of this application divide these layers into several feature groups, and flexibly assign levels to different groups by initializing fusion grouping parameters. The calculation formula for this process is as follows:

[0035] in, For grouped sets, For the first Each feature group contains a subset of hierarchical indexes. This represents the number of feature groups.

[0036] Preferably, embodiments of this application further pre-configure the feature dimensions of the input image to ensure that the output features of each layer are aligned and compatible during fusion. This configuration ensures that features from different levels can form an efficient comprehensive feature representation after weighted fusion.

[0037] In one embodiment, feature encoding is performed on the input image to obtain an intermediate feature set of the input image, including: The input image is subjected to layer-by-layer feature encoding, and the result of the layer-by-layer feature encoding is subjected to dimension alignment to obtain the intermediate feature set of the input image.

[0038] It is understandable that by dimensionally aligning the intermediate features of each layer, we can ensure that the features at different levels have consistent dimensions and structure in the subsequent fusion process, so that these features can be successfully fused.

[0039] Specifically, the expression for a dimension alignment process provided in the application embodiment is as follows:

[0040] in, To align features to a uniform number of positions With feature dimension Processing operators, The aligned intermediate features are the output of the layer.

[0041] Step S120: Perform weighted fusion on the intermediate feature set, and perform constrained training and feature reconstruction processing on the weighted fusion result to obtain the reconstructed features.

[0042] It is understandable that by imposing information constraints on the intermediate feature set after weighted fusion and performing reconstruction processing, a basis is provided for subsequent calculation of the difference between fused features and reconstructed features; at the same time, constrained training can suppress the direct reconstruction of abnormal details and reduce the risk of the model undergoing identity mapping.

[0043] In one embodiment, weighted fusion of the intermediate feature set includes: Linear attention is calculated for each feature in the intermediate feature set to obtain the fusion weight of each feature, and then the fusion weight is weighted and fused.

[0044] It can be understood that linear attention is calculated on all features in the intermediate feature set to generate fusion weights for each feature. Then, through linear transformation and Softmax (a normalized exponential function), the weight of each feature is calculated. The purpose of calculating linear attention is to generate weights suitable for weighted fusion based on the importance of the features.

[0045] Furthermore, the calculated fusion weights are weighted and fused to obtain the final fusion feature, which is calculated as follows:

[0046] in, For the final fusion feature, For the first Fusion weights of layer features ( satisfy ).

[0047] In one embodiment, the weighted fusion result is subjected to constrained training and feature reconstruction processing to obtain reconstructed features, including: The weighted fusion results are trained under constraints using a pre-defined network model; the fused features after constraint training are then reconstructed to obtain the reconstructed features.

[0048] It is understandable that by introducing a noise bottleneck mechanism, the fused features are constrained during training to prevent the model from becoming overly reliant on certain specific feature information, thereby enhancing the model's robustness in the face of noise and anomalous data.

[0049] Furthermore, the constrained fusion features are reconstructed using the following formula: The input feature reconstruction module generates reconstructed features, which can be represented as:

[0050] in, For the first Group reconstruction features, To rebuild module mappings.

[0051] Step S130: Group and fuse the intermediate feature set, and calculate the difference between the group fusion result and the reconstructed features to obtain the anomaly detection result.

[0052] In one embodiment, the grouping and fusion of the intermediate feature set includes: According to the preset fusion grouping strategy, the features in the intermediate feature set and the reconstructed features are grouped and fused.

[0053] It is understood that, according to the preset fusion grouping strategy, the encoded features and reconstructed features are divided into multiple feature groups, and the encoded features and the corresponding reconstructed features of each feature group are averaged and fused to obtain parameters for subsequent calculation of the difference between the group fusion result and the reconstructed features.

[0054] In one embodiment, the difference between the group fusion result and the reconstructed features is calculated to obtain the anomaly detection result, including: The difference between the group fusion result and the reconstructed features is calculated by cosine similarity to obtain the feature difference value; if the feature difference value exceeds the preset difference threshold, the image region corresponding to the feature difference value is determined to be an abnormal region.

[0055] It can be understood that by calculating the difference between the fused features and the corresponding reconstructed features of each feature group, and combining this difference with cosine similarity to measure the difference, the difference value of each group can be obtained. The calculation formula for this process is as follows:

[0056] in, For the first Group difference measurement For the first Group fusion characteristics For the first Reconstruction characteristics of the group.

[0057] Furthermore, the difference metric calculated based on cosine similarity is used as the anomaly score, and anomaly score thresholds are set to determine abnormal regions in the input image.

[0058] In summary, the implementation method of this application, by constraining the training and reconstruction of weighted fusion features and calculating the difference between grouped fusion features and reconstructed features, can effectively suppress the direct reconstruction of abnormal details in the target image, improve the stability and positioning accuracy of anomaly recognition, reduce the computation and maintenance costs caused by multi-model deployment, and has good generalization ability and engineering application value.

[0059] Based on the first embodiment of this application, an image anomaly detection device is provided in this embodiment, see reference. Figure 2 The device includes: The feature encoding module 21 is used to acquire the input image and perform feature encoding on the input image to obtain the intermediate feature set of the input image; The feature reconstruction module 22 is used to perform weighted fusion on the intermediate feature set, and to perform constrained training and feature reconstruction processing on the weighted fusion result to obtain the reconstructed features; The anomaly detection module 23 is used to group and fuse the intermediate feature set, and calculate the difference between the group fusion result and the reconstructed features to obtain the anomaly detection result.

[0060] In one embodiment, the feature encoding module 21 is further configured to: The input image is subjected to layer-by-layer feature encoding, and the result of the layer-by-layer feature encoding is subjected to dimension alignment to obtain the intermediate feature set of the input image.

[0061] In one embodiment, the feature reconstruction module 22 is further configured to: Linear attention is calculated for each feature in the intermediate feature set to obtain the fusion weight of each feature, and then the fusion weight is weighted and fused.

[0062] In one embodiment, the feature reconstruction module 22 is further configured to: The weighted fusion results are trained under constraints using a pre-defined network model; the fused features after constraint training are then reconstructed to obtain the reconstructed features.

[0063] In one embodiment, the anomaly detection module 23 is further configured to: According to the preset fusion grouping strategy, the features in the intermediate feature set and the reconstructed features are grouped and fused.

[0064] In one embodiment, the anomaly detection module 23 is further configured to: The difference between the group fusion result and the reconstructed features is calculated by cosine similarity to obtain the feature difference value; if the feature difference value exceeds the preset difference threshold, the image region corresponding to the feature difference value is determined to be an abnormal region.

[0065] It should be noted that the description of the image anomaly detection device above is similar to the description of the image anomaly detection method above, and the beneficial effects of the same method will not be repeated. For technical details not disclosed in the embodiments of the image anomaly detection device of this invention, please refer to the description of the embodiments of the image anomaly detection method of this invention.

[0066] Based on the same inventive concept as the foregoing embodiments, this application provides an electronic device, such as... Figure 3 As shown, the device includes: a processor 301 and a memory 302 storing a computer program; wherein, Figure 3 The processor 301 shown in the diagram does not indicate that there is only one processor 301, but only indicates the positional relationship of processor 301 relative to other devices. In practical applications, there can be one or more processors 301; similarly, Figure 3 The memory 302 shown in the diagram has the same meaning, that is, it is only used to indicate the positional relationship of memory 302 relative to other devices. In practical applications, there can be one or more memories 302. When the processor 301 runs the computer program, it implements the image anomaly detection method described above.

[0067] The device may also include at least one network interface 303. The various components of the device are coupled together via a bus system 304. It is understood that the bus system 304 is used to implement communication between these components. In addition to a data bus, the bus system 304 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 3 The general designated all buses as Bus System 304.

[0068] The memory 302 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 302 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0069] Fourth embodiment Based on the same inventive concept as the foregoing embodiments, this embodiment also provides a computer-readable storage medium storing a computer program. The computer-readable storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it can also be various devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc. When the computer program stored in the computer-readable storage medium is executed by a processor, it implements the above-described image anomaly detection method. For the specific steps implemented when the computer program is executed by the processor, please refer to [link to relevant documentation]. Figure 2 The description of the illustrated embodiments will not be repeated here.

[0070] In this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions are generally described in detail only when they appear for the first time. When they appear again, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions that are not described in detail later can be referred to their previous relevant detailed descriptions.

[0071] In this application, the descriptions of the various embodiments have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0072] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0073] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. For those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An image anomaly detection method, characterized in that, include: The input image is acquired, and its features are encoded to obtain an intermediate feature set of the input image. The intermediate feature set is weighted and fused, and the weighted fusion result is subjected to constrained training and feature reconstruction processing to obtain the reconstructed features; The intermediate feature set is grouped and fused, and the difference between the grouped and fused result and the reconstructed features is calculated to obtain the anomaly detection result.

2. The image anomaly detection method according to claim 1, characterized in that, Before acquiring the input image, the following is also included: Initialize the feature level index list for feature encoding, and the fusion grouping parameters for weighted fusion.

3. The image anomaly detection method according to claim 1, characterized in that, The step of performing feature encoding on the input image to obtain an intermediate feature set of the input image includes: The input image is subjected to layer-by-layer feature encoding, and the result of the layer-by-layer feature encoding is subjected to dimension alignment to obtain the intermediate feature set of the input image.

4. The image anomaly detection method according to claim 1, characterized in that, The weighted fusion of the intermediate feature set includes: Linear attention is calculated for each feature in the intermediate feature set to obtain the fusion weight of each feature, and then the fusion weight is weighted and fused.

5. The image anomaly detection method according to claim 1, characterized in that, The weighted fusion result is subjected to constrained training and feature reconstruction processing to obtain reconstructed features, including: The weighted fusion results are trained under constraints using a pre-defined network model. The fused features after constrained training are reconstructed to obtain the reconstructed features.

6. The image anomaly detection method according to claim 1, characterized in that, The step of grouping and fusing the intermediate feature set includes: According to the preset fusion grouping strategy, the features in the intermediate feature set and the reconstructed features are grouped and fused.

7. The image anomaly detection method according to claim 1, characterized in that, The difference between the calculated group fusion result and the reconstructed features yields the anomaly detection result, including: The difference between the group fusion result and the reconstructed features is calculated by cosine similarity to obtain the feature difference value; If the feature difference value exceeds a preset difference threshold, the image region corresponding to the feature difference value is determined to be an abnormal region.

8. An image anomaly detection device, characterized in that, The device includes: The feature encoding module is used to acquire the input image and perform feature encoding on the input image to obtain an intermediate feature set of the input image; The feature reconstruction module is used to perform weighted fusion on the intermediate feature set, and to perform constrained training and feature reconstruction processing on the weighted fusion result to obtain reconstructed features; An anomaly detection module is used to group and fuse the intermediate feature set, and calculate the difference between the group fusion result and the reconstructed features to obtain the anomaly detection result.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, wherein the memory stores executable program code, which, when executed by the processor, implements the steps of the image anomaly detection method as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements the steps of the image anomaly detection method as described in any one of claims 1 to 7.

Images