Image anomaly detection method and device, computer device and storage medium

Abstract

The application relates to an image anomaly detection method and device, computer equipment, a storage medium and a computer program product. The method relates to artificial intelligence technology and comprises the following steps: encoding original image features of a to-be-detected image to obtain detection coding features; encoding prompt original features associated with the to-be-detected image to obtain prompt coding features; performing at least one level of combined decoding processing based on the detection coding features and the prompt coding features to obtain combined decoding features; in each level, performing attention feature extraction on input target detection coding features and target prompt coding features, and performing attention feature extraction on output prompt coding features in combination with target detection coding features to obtain output detection coding features; and obtaining an anomaly detection result based on a reconstruction difference between image reconstruction features obtained according to the combined decoding features and the original image features. The method can improve the accuracy of image anomaly detection.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image anomaly detection method, apparatus, computer equipment, storage medium, and computer program product, as well as an image anomaly detection model processing method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] With the development of computer technology, computer vision is being applied more and more widely in various fields, including autonomous driving, medical image analysis, retail, security, and gaming. It enables diverse tasks such as image classification, object detection, semantic segmentation, face recognition, and image anomaly detection, greatly improving image processing efficiency. Among these, image anomaly detection aims to identify abnormal images and has significant research and application value in areas such as industrial appearance defect detection, medical image analysis, and hyperspectral image processing.

[0003] However, in current image anomaly detection processing, the high consistency between abnormal regions and neighboring regions in an image makes it difficult to effectively distinguish between normal and abnormal regions, resulting in low accuracy of image anomaly detection. Summary of the Invention

[0004] Therefore, it is necessary to provide an image anomaly detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the accuracy of image anomaly detection, as well as an image anomaly detection model processing method, apparatus, computer equipment, storage medium, and computer program product, to address the above-mentioned technical problems.

[0005] Firstly, this application provides an image anomaly detection method. The method includes:

[0006] The image to be detected is acquired, and the original features of the image to be detected are encoded to obtain the detection encoding features of the image to be detected.

[0007] The original cue features associated with the image to be detected are determined, and the original cue features are encoded to obtain the cue encoded features; the original cue features are extracted from the reference image corresponding to the image to be detected, which does not contain anomalies.

[0008] Based on the detection coding features and the cue coding features, perform at least one level of combined decoding processing to obtain the combined decoding features output by at least one level of combined decoding processing. In each level of combined decoding processing, perform attention feature extraction on the target detection coding features and target cue coding features input at the level to obtain the cue coding features output at the level. Perform attention feature extraction on the cue coding features and target detection coding features output at the level to obtain the detection coding features output at the level.

[0009] Image reconstruction features for the image to be detected are obtained based on the combined decoding features;

[0010] Based on the reconstruction differences between the image reconstruction features and the original image features, anomaly detection results are obtained for the image to be detected.

[0011] Secondly, this application also provides an image anomaly detection device. The device includes:

[0012] The image processing module is used to acquire the image to be detected and encode the original features of the image to be detected to obtain the detection coding features of the image to be detected.

[0013] The cue feature acquisition module is used to determine the original cue features associated with the image to be detected, and to encode the original cue features to obtain the cue encoded features; the original cue features are extracted from the reference image corresponding to the image to be detected, which does not contain anomalies.

[0014] The combined decoding processing module is used to perform at least one level of combined decoding processing based on detection coding features and prompt coding features to obtain combined decoding features output by at least one level of combined decoding processing. In each level of combined decoding processing, attention features are extracted for the target detection coding features and target prompt coding features input at the level to obtain the prompt coding features output at the level. Attention features are also extracted for the prompt coding features and target detection coding features output at the level to obtain the detection coding features output at the level.

[0015] The reconstruction feature acquisition module is used to obtain image reconstruction features for the image to be detected based on the combined decoding features;

[0016] The reconstruction difference processing module is used to obtain anomaly detection results for the image to be detected based on the reconstruction differences between the image reconstruction features and the original image features.

[0017] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-described image anomaly detection method.

[0018] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described image anomaly detection method.

[0019] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the above-described image anomaly detection method.

[0020] The aforementioned image anomaly detection method, apparatus, computer equipment, storage medium, and computer program product encode the original image features of the image to be detected to obtain detection coding features, and encode the original cue features extracted from a reference image (excluding anomalies) associated with the image to be detected to obtain cue coding features. Based on the detection coding features and cue coding features, at least one level of combined decoding processing is performed to obtain combined decoding features. In each level of combined decoding processing, attention features are extracted from the target detection coding features and target cue coding features input at that level to obtain cue coding features output at that level. Attention features are also extracted from the cue coding features and target detection coding features output at that level to obtain detection coding features output at that level. Based on the combined decoding features, image reconstruction features for the image to be detected are obtained, and based on the reconstruction difference between the image reconstruction features and the original image features, the anomaly detection result for the image to be detected is obtained. In image anomaly detection processing, in the combined decoding process at each level, attention features are extracted from the target detection coding features and target cue coding features input at the level to obtain the cue coding features output at the level. Attention features are then extracted from the cue coding features and target detection coding features output at the level to obtain the detection coding features output at the level. This approach can comprehensively utilize the image context information of the reference image excluding anomalies and the corresponding dynamic cue coding features at each level to construct combined decoding features, thereby enhancing the feature representation capability of the combined decoding features and improving the accuracy of image anomaly detection based on the reconstructed features obtained from the combined decoding features.

[0021] Sixthly, this application provides an image anomaly detection model processing method. The method includes:

[0022] Acquire normal sample images, reference sample images associated with normal sample images, and abnormal sample images;

[0023] Feature extraction is performed on normal sample images, reference sample images, and abnormal sample images respectively to obtain the original features of normal sample images, the original features of reference sample images, and the original features of abnormal sample images.

[0024] The image anomaly detection model to be trained encodes the original features of normal images, original features of reference images, and original features of abnormal images, respectively, to obtain the normal sample encoding features of normal sample images, the prompt sample encoding features of reference sample images, and the abnormal sample encoding features of abnormal sample images.

[0025] The image anomaly detection model to be trained performs at least one level of combined decoding processing based on normal sample coding features and cue sample coding features to obtain the first combined decoding feature output by at least one level of combined decoding processing, and obtains normal sample reconstruction features for normal sample images based on the first combined decoding feature.

[0026] The image anomaly detection model to be trained performs at least one level of combined decoding processing based on the anomaly sample encoding features and the cue sample encoding features to obtain the second combined decoding features output by the at least one level of combined decoding processing, and obtains the anomaly sample recovery features for the anomaly sample image based on the second combined decoding features.

[0027] The image anomaly detection model to be trained is updated based on the reconstruction features of normal samples, the recovery features of abnormal samples, and the original features of normal images. The training continues until the training is completed, and the trained image anomaly detection model is obtained. The trained image anomaly detection model is used to perform image anomaly detection on the input image.

[0028] In one embodiment, the image anomaly detection model to be trained is updated based on normal sample reconstruction features, abnormal sample recovery features, and normal image original features, and then trained again until training is completed to obtain a trained image anomaly detection model. This includes: determining a sample reconstruction loss based on the sample reconstruction difference between normal sample reconstruction features and normal image original features; determining a sample recovery loss based on the sample recovery difference between abnormal sample recovery features and normal image original features; and updating the image anomaly detection model to be trained based on the sample reconstruction loss and sample recovery loss, and then training again until training is completed to obtain a trained image anomaly detection model.

[0029] In one embodiment, the image anomaly detection model processing method further includes: determining a first feature difference between the reconstructed features of normal samples and the original features of normal images, and a second feature difference between the restored features of abnormal samples and the original features of abnormal images; using the image anomaly detection model to be trained, performing at least one level of pixel feature extraction for the first feature difference and the second feature difference respectively, to obtain a first pixel anomaly result and a second pixel anomaly result; determining a pixel anomaly detection loss based on the abnormal region label carried by the abnormal sample image, the first pixel anomaly result, and the second pixel anomaly result; updating the image anomaly detection model to be trained based on the sample reconstruction loss and the sample restoration loss, and continuing training until training is completed, to obtain a trained image anomaly detection model, including: fusing the sample reconstruction loss, the sample restoration loss, and the pixel anomaly detection loss to obtain a model target loss; updating the image anomaly detection model to be trained based on the model target loss, and continuing training until training is completed, to obtain a trained image anomaly detection model.

[0030] Seventhly, this application also provides an image anomaly detection model processing apparatus. The apparatus includes:

[0031] The sample image acquisition module is used to acquire normal sample images, reference sample images associated with normal sample images, and abnormal sample images;

[0032] The sample feature extraction module is used to extract features from normal sample images, reference sample images, and abnormal sample images respectively, to obtain the original features of normal sample images, the original features of reference sample images, and the original features of abnormal sample images.

[0033] The sample feature encoding module is used to encode the original features of normal images, original features of reference images, and original features of abnormal images using the image anomaly detection model to be trained, so as to obtain the normal sample encoding features of normal sample images, the prompt sample encoding features of reference sample images, and the abnormal sample encoding features of abnormal sample images.

[0034] The normal sample reconstruction module is used to perform at least one level of combined decoding processing based on normal sample encoding features and cue sample encoding features through the image anomaly detection model to be trained, to obtain the first combined decoding feature output by the at least one level of combined decoding processing, and to obtain normal sample reconstruction features for normal sample images based on the first combined decoding feature.

[0035] An abnormal sample recovery module is used to perform at least one level of combined decoding processing based on the abnormal sample encoding features and the cue sample encoding features through the image anomaly detection model to be trained, to obtain the second combined decoding features output by the at least one level of combined decoding processing, and to obtain the abnormal sample recovery features for the abnormal sample image based on the second combined decoding features.

[0036] The model update module is used to update the image anomaly detection model to be trained based on the reconstruction features of normal samples, the recovery features of abnormal samples, and the original features of normal images, and then continue training until training is completed, thus obtaining the trained image anomaly detection model; the trained image anomaly detection model is used to perform image anomaly detection on the input image.

[0037] Eighthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-described image anomaly detection model processing method.

[0038] Ninthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described image anomaly detection model processing method.

[0039] Tenthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the above-described image anomaly detection model processing method.

[0040] The aforementioned image anomaly detection model processing method, apparatus, computer equipment, storage medium, and computer program product, for normal sample images, reference sample images associated with normal sample images, and abnormal sample images, encodes the original features of each image using the image anomaly detection model to be trained, obtaining normal sample encoded features, prompt sample encoded features, and abnormal sample encoded features. The image anomaly detection model to be trained then performs at least one level of combined decoding processing based on the normal sample encoded features and prompt sample encoded features to construct normal sample reconstruction features for normal sample images. The image anomaly detection model to be trained then performs at least one level of combined decoding processing based on the abnormal sample encoded features and prompt sample encoded features to construct abnormal sample recovery features for abnormal sample images. Finally, the image anomaly detection model to be trained is updated based on the normal sample reconstruction features, abnormal sample recovery features, and original features of the normal image, and then trained again to obtain a trained image anomaly detection model for detecting image anomalies in input images. By introducing cue sample coding features and combining them with normal sample coding features for at least one level of combined decoding processing to reconstruct features for normal sample images, and introducing cue sample coding features and combining them with abnormal sample coding features for at least one level of combined decoding processing to restore features for abnormal sample images, the image anomaly detection capability of the image anomaly detection model can be enhanced by utilizing the image context information of the reference sample image and the corresponding dynamic cue coding features at each level. This improves the accuracy of the trained image anomaly detection model in detecting image anomalies in input images. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is an application environment diagram of an image anomaly detection method in one embodiment;

[0043] Figure 2 This is a flowchart illustrating an image anomaly detection method in one embodiment;

[0044] Figure 3 This is a flowchart illustrating the image anomaly detection method in yet another embodiment;

[0045] Figure 4 This is a flowchart illustrating the feature extraction and encoding process in one embodiment;

[0046] Figure 5 This is a schematic diagram of the combined decoding process in one embodiment;

[0047] Figure 6 This is a schematic diagram of the combined decoding process within a certain layer in one embodiment;

[0048] Figure 7 This is a schematic diagram of the process for obtaining the prompt encoding features in one embodiment;

[0049] Figure 8 This is a schematic diagram of the process for obtaining the detected encoded features in one embodiment;

[0050] Figure 9 This is a schematic diagram of the model update process in one embodiment;

[0051] Figure 10 This is a flowchart illustrating an image anomaly detection model processing method in one embodiment;

[0052] Figure 11 This is a framework diagram for image anomaly detection in one embodiment;

[0053] Figure 12 This is a comparison table of detection results of different image anomaly detection methods in one embodiment;

[0054] Figure 13 This is a structural block diagram of an image anomaly detection device in one embodiment;

[0055] Figure 14 This is a structural block diagram of an image anomaly detection model processing device in one embodiment;

[0056] Figure 15 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] The image anomaly detection method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server.

[0059] Terminal 102 can acquire images, such as industrial product images or medical images. Terminal 102 can send the acquired images to be detected to server 104 via the network. Server 104 receives the images to be detected sent by terminal 102 and encodes the original image features of the images to obtain detection coding features. Server 104 determines the original prompt features extracted from a reference image (excluding anomalies) associated with the images to be detected and encodes these original prompt features to obtain prompt coding features. Server 104 performs at least one level of combined decoding processing based on the detection coding features and prompt coding features to obtain combined decoding features output from at least one level of combined decoding processing. In each level of combined decoding processing, attention features are extracted from the target detection coding features and target prompt coding features input at that level to obtain prompt coding features output at that level. Attention features are also extracted from the prompt coding features and target detection coding features output at that level to obtain detection coding features output at that level. Server 104 obtains image reconstruction features for the image to be detected based on the combined decoding features, and obtains anomaly detection results for the image to be detected based on the reconstruction differences between the image reconstruction features and the original image features. Server 104 can also feed back the anomaly detection results to terminal 102. In some embodiments, the image anomaly detection method can also be implemented by server 104 or terminal 102 separately. For example, server 104 can directly obtain the image to be detected from the data storage system and perform image anomaly detection processing on the image to be detected; or, after terminal 102 acquires the image to be detected, it can directly perform image anomaly detection processing on the image to be detected through terminal 102.

[0060] The image anomaly detection model processing method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, server 104 can acquire normal sample images, reference sample images associated with normal sample images, and abnormal sample images. These images can be sent from terminal 102 to server 104. Server 104 can extract features from each of the normal, reference, and abnormal sample images to obtain their respective original image features. Server 104 then encodes these original features using a training image anomaly detection model to obtain normal sample encoded features, prompt sample encoded features, and abnormal sample encoded features. Server 104 performs at least one level of combined decoding based on the normal and prompt sample encoded features using the training image anomaly detection model to construct normal sample reconstruction features for normal sample images. Similarly, server 104 performs at least one level of combined decoding based on the abnormal sample encoded features and prompt sample encoded features using the training image anomaly detection model to construct abnormal sample recovery features for abnormal sample images. Server 104 then updates the training image anomaly detection model based on the normal sample reconstruction features, abnormal sample recovery features, and normal image original features, and continues training to obtain a trained image anomaly detection model for detecting image anomalies in input images. After obtaining the trained image anomaly detection model, the user can send the image to be detected to the server 104 through the terminal 102. The server 104 can input the image to be detected sent by the terminal 102 into the trained image anomaly detection model to perform image anomaly detection, and feed back the anomaly detection result output by the image anomaly detection model to the terminal 102.

[0061] Terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Terminal 102 and server 104 can be connected directly or indirectly through wired or wireless communication, without limitation.

[0062] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Based on the cloud computing business model, cloud technology encompasses network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0063] Cloud computing refers to the delivery and usage model of IT infrastructure, meaning obtaining necessary resources through a network in an on-demand and easily scalable manner. In a broader sense, cloud computing also refers to the delivery and usage model of services, meaning obtaining necessary services through a network in an on-demand and easily scalable manner. These services can be IT and software related, internet-related, or other services. Cloud computing is a product of the convergence and development of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing. Driven by the development of the internet, real-time data streams, the diversification of connected devices, and the demands of search services, social networks, mobile commerce, and open collaboration, cloud computing has developed rapidly. Unlike previous parallel and distributed computing, the emergence of cloud computing will fundamentally revolutionize the entire internet model and enterprise management model.

[0064] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. AI technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained model technology, operating / interactive systems, and mechatronics. Pre-trained models, also known as large models or foundational models, can be widely applied to downstream tasks in various areas of AI after fine-tuning. AI software technologies mainly include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0065] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing and measuring targets, and further processes images to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Large model technology has brought significant changes to the development of computer vision technology. Pre-trained models in the vision field, such as Swin-transformer, ViT, V-MOE, and MAE, can be quickly and widely applied to downstream tasks after fine-tuning. Computer vision technology typically includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and other technologies, as well as common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0066] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and pre-trained learning. Pre-trained models are the latest development in deep learning, integrating all of these techniques.

[0067] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, digital twins, virtual humans, robots, AI-generated content (AIGC), conversational interaction, smart healthcare, smart customer service, and game AI. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0068] Artificial intelligence cloud services are generally also known as AIaaS (AI as a Service). This is currently a mainstream service model for artificial intelligence platforms. Specifically, AIaaS platforms break down several common AI services and provide them as independent or packaged services in the cloud. This service model is similar to opening an AI-themed marketplace: all developers can access and use one or more AI services provided by the platform through API interfaces. Some experienced developers can also use the AI ​​framework and AI infrastructure provided by the platform to deploy and maintain their own dedicated cloud AI services.

[0069] The solutions provided in this application involve technologies such as computer vision, machine learning, and deep learning in artificial intelligence, which are specifically illustrated through the following embodiments.

[0070] In one exemplary embodiment, such as Figure 2 As shown, an image anomaly detection method is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking the server in the example, the explanation includes the following steps 202 to 210. Wherein:

[0071] Step 202: Obtain the image to be detected and encode the original features of the image to be detected to obtain the detection coding features of the image to be detected.

[0072] Image anomaly detection technology has wide applications in multiple fields. For example, in the medical field, image anomaly detection can be used for disease diagnosis; in the field of autonomous driving, this technology can help vehicles identify abnormal objects on the road; in the retail industry, image anomaly detection can be used to identify abnormal behavior; and in the agricultural field, it can be used for crop health monitoring and pest and disease detection. The image to be detected is the image for which anomaly detection is required, and it can be acquired by an image acquisition device. In different image anomaly detection application scenarios, the image to be detected can include different image content. The original image features can be preliminary image features extracted from the image to be detected, such as the original image features extracted from the image to be detected through a pre-trained feature extraction network. The feature extraction network can be trained based on various neural network algorithms, specifically including but not limited to at least one of Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Transformer, Feedforward Neural Networks, Generative Adversarial Networks (GANs), or Autoencoders. The detection coding features are obtained by encoding the original features of the image. Specifically, an encoding network can be used to encode the original features of the image to obtain the detection coding features of the image to be detected. In practice, the encoding network can be based on various neural network model architectures, such as those built based on attention mechanisms.

[0073] For example, the server can acquire the image to be detected for anomaly detection. The server determines the original features of the image to be detected. Specifically, the server can call a pre-trained feature extraction network to extract features from the image to obtain the original image features. The server can encode the original image features. Specifically, it can call an encoding network to perform at least one level of encoding processing on the original image features to obtain the detection encoded features of the image to be detected. By decoding the detection encoded features of the image to be detected, feature reconstruction of the image to be detected can be achieved, thereby enabling anomaly detection processing using the obtained reconstructed features.

[0074] Step 204: Determine the original cue features associated with the image to be detected, and encode the original cue features to obtain the cue encoded features; the original cue features are extracted from the reference image corresponding to the image to be detected, which does not contain anomalies.

[0075] The prompt indicates that the original features are associated with the image to be detected. Specifically, these can be preliminary image features extracted from a reference image corresponding to the image to be detected. The reference image does not contain anomalies; it is a normal image relative to the image to be detected, meaning it does not contain any anomalies. For example, in anomaly detection processing for medical images, the image to be detected can be a acquired medical image that needs to be detected, while the reference image can be a medical image that has been pre-confirmed to be normal, i.e., it does not contain any anomalies. The higher the similarity between the image to be detected and the reference image, such as a higher feature matching degree between the image features of the image to be detected and the image features of the reference image, the more likely the image to be detected is a normal image without anomalies. Conversely, the less similar the image to be detected and the reference image, the more likely the image to be an anomaly image containing abnormal content. For example, the image to be detected may contain various anomalies such as foreign objects, scratches, or missing parts.

[0076] In practical applications, the server can pre-acquire various reference images that do not contain anomalies and perform feature extraction on each reference image separately. For example, feature extraction can be performed separately using a pre-trained feature extraction network to obtain the original cue features for each reference image. After obtaining the image to be detected, the server can determine the original cue features associated with the image to be detected from the original cue features of each reference image. The cue encoding features are obtained by encoding the original cue features. Specifically, an encoding network can be used to encode the original cue features to obtain the cue encoding features.

[0077] Optionally, the server can determine the original cue features associated with the image to be detected. In a specific implementation, the server can pre-store the original cue features of each reference image. The original cue features of each reference image can be obtained by the server in advance through feature extraction network for each reference image. The server can directly determine the original cue features associated with the image to be detected from the stored original cue features. For example, the server can match the original image features of the image to be detected with each stored original cue feature, and determine the original cue features associated with the image to be detected based on the matching results. Specifically, the original cue feature with the highest matching degree can be used as the original cue feature associated with the image to be detected. In some embodiments, the server can also first determine the reference image corresponding to the image to be detected, and perform feature extraction on the reference image corresponding to the image to be detected to obtain the original cue features associated with the image to be detected. The server can encode the original cue features, specifically by calling an encoding network to perform at least one level of encoding processing on the original cue features to obtain the cue encoded features. In some embodiments, the original image features and the original cue features of the image to be detected can be extracted separately using the same feature extraction network, and the encoding of the original image features and the original cue features can also be implemented based on the same encoding network.

[0078] Step 206: Perform at least one level of combined decoding processing based on detection coding features and cue coding features to obtain combined decoding features output by at least one level of combined decoding processing. In each level of combined decoding processing, perform attention feature extraction on the target detection coding features and target cue coding features input at the level to obtain cue coding features output at the level. Perform attention feature extraction on the cue coding features and target detection coding features output at the level to obtain detection coding features output at the level.

[0079] The input to the combined decoding process consists of the detection coding features of the image to be detected and the prompt coding features obtained by encoding the original prompt features. The output of the combined decoding process can also include the corresponding detection coding features and prompt coding features. That is, the combined decoding features output by at least one level of combined decoding process can include the detection coding features and prompt coding features output by the last level. In multi-level combined decoding processes, the first level of combined decoding process can be directly implemented based on the detection coding features of the image to be detected and the prompt coding features obtained by encoding the original prompt features. Starting from the second level of combined decoding process, the detection coding features and prompt coding features output by the previous level of combined decoding process can be used as the input of this level, thereby performing multiple iterations of combined decoding process until the last level of combined decoding process is completed and the combined decoding features are output.

[0080] In each level of combined decoding, the object detection encoded features and the target cue encoded features are the inputs to the current level's combined decoding process, and can specifically be the outputs of the previous level's combined decoding process. Attention feature extraction can be performed based on an attention mechanism. There is a certain relationship between the object detection encoded features and the target cue encoded features. The attention mechanism can establish the correlation between these two features, thereby obtaining features with stronger representational power. In other words, the attention mechanism can enhance the expression of the correlation between the object detection encoded features and the target cue encoded features. The cue encoded features output at a given level are obtained through attention feature extraction based on the object detection encoded features and target cue encoded features input at that level; similarly, the object detection encoded features output at a given level are obtained through attention feature extraction based on the cue encoded features and object detection encoded features output at that level.

[0081] Specifically, the server can perform at least one level of combined decoding processing based on the detection coding features and prompt coding features obtained from the encoding, to obtain combined decoding features. In each level of combined decoding processing, the server can determine the target detection coding features and target prompt coding features input at the current level, and perform feature extraction on the target detection coding features and target prompt coding features based on an attention mechanism to obtain the prompt coding features output at the current level. Further, the server performs feature extraction on the prompt coding features and target detection coding features output at the current level based on an attention mechanism to obtain the detection coding features output at the current level. The server can use the output of the combined decoding processing at this level as the input of the combined decoding processing at the next level, that is, the server can input the prompt coding features and detection coding features output by the combined decoding processing at this level into the next level to perform the combined decoding processing at the next level. In some embodiments, at least one level of combined decoding processing can be implemented based on a combined decoding processing layer structure, where each layer structure can implement one level of combined decoding processing, thereby implementing at least one level of combined decoding processing by setting at least one layer structure.

[0082] Step 208: Obtain image reconstruction features for the image to be detected based on the combined decoding features.

[0083] Image reconstruction features are features obtained by reconstructing features from the image to be detected. Specifically, they can be obtained by feature mapping of combined decoded features. For example, feature mapping can be performed on combined decoded features based on an attention mechanism to obtain the image reconstruction features of the image to be detected. Specifically, the server can obtain the image reconstruction features of the image to be detected based on the combined decoded features. For example, the server can perform feature mapping on the combined decoded features through a feature mapping layer to obtain the image reconstruction features of the image to be detected. In specific implementations, the feature mapping of combined decoded features can be implemented according to actual needs. For example, further attention feature extraction can be performed on the combined decoded features to obtain the image reconstruction features of the image to be detected.

[0084] Step 210: Based on the reconstruction difference between the image reconstruction features and the original image features, obtain the anomaly detection results for the image to be detected.

[0085] The reconstruction difference characterizes the feature differences between the reconstructed image features and the original image features. Specifically, it can be obtained by comparing the reconstructed and original image features. Based on the reconstruction difference, the anomaly detection result for the image to be detected can be determined. For example, the server can determine the reconstruction difference between the reconstructed and original image features. Specifically, the server can compare the reconstructed and original image features to obtain the reconstruction difference, and the server determines the anomaly detection result for the image to be detected based on the reconstruction difference. For instance, the server can map the reconstruction difference to obtain anomaly detection parameters for the image to be detected, and the anomaly detection result can be obtained based on the anomaly detection parameters. In some embodiments, the anomaly detection parameters may include an anomaly detection score. Based on the anomaly detection score and a score threshold, the anomaly detection result can be determined. If the anomaly detection score exceeds the score threshold, it can be considered that there is an anomaly in the image to be detected, i.e., the image to be detected is an anomalous image.

[0086] In some embodiments, such as Figure 3 As shown, after acquiring the image to be detected, the server extracts the original image features of the image and encodes these original features to obtain the detection coding features of the image to be detected. The server also acquires the original prompt features associated with the image to be detected. These original prompt features are obtained by feature extraction from the corresponding reference image (excluding anomalies) of the image to be detected. The server encodes these original prompt features to obtain the prompt coding features. The server performs at least one level of combined decoding processing based on the detection coding features and the prompt coding features to obtain combined decoded features. The server determines the image reconstruction features based on the combined decoded features and obtains the reconstruction difference based on the image reconstruction features and the original image features. The server determines the anomaly detection result of the image to be detected based on the reconstruction difference.

[0087] In some embodiments, such as Figure 4As shown, features can be extracted from both the image to be detected and the reference image using the same feature extraction network. Specifically, the server can input the image to be detected and the reference image into the same feature extraction network, which then outputs the original image features and the original cue features, respectively. The encoding of the original image features and the original cue features can also be achieved using the same encoding network. Specifically, the server can input the original image features and the original cue features into the same encoding network, which then outputs the detection encoded features and the cue encoded features, respectively.

[0088] In some embodiments, such as Figure 5 As shown, at least one level of combined decoding processing can include N levels of combined decoding processing. For the first level, the detection coding features and prompt coding features of the image to be detected can be directly used for the first-level combined decoding processing. That is, in the first level, the detection coding features of the image to be detected are directly used as the target detection coding features of this level, and the prompt coding features obtained by encoding the original prompt features are used as the target prompt coding features of this level, resulting in the detection coding features and prompt coding features output by the first level. The detection coding features and prompt coding features output by the first level can be input into the second level for the second-level combined decoding processing. The detection coding features and prompt coding features output by the second level can be input into the next level for the next level of combined decoding processing. Until the last level, that is, for the Nth level, the detection coding features and prompt coding features output by the (N-1)th level can be used as input for the Nth level of combined decoding processing, outputting the combined decoded features.

[0089] In some embodiments, such as Figure 6 As shown, in the combined decoding process at each level, such as in the combined decoding process at level K, the server can use the cue coding features and detection coding features output from level K-1 as input to level K. That is, the cue coding features output from level K-1 are used as the target cue coding features of level K, and the detection coding features output from level K-1 are used as the target detection coding features of level K. The server performs attention feature extraction based on the target detection coding features and target cue coding features to obtain the cue coding features output at the current level; the server also performs attention feature extraction based on the cue coding features and target detection coding features output at the current level to obtain the detection coding features output at the current level. Using the cue coding features and detection coding features output from level K, the server can use the cue coding features and detection coding features output from level K as input to level K+1 to perform combined decoding processing at level K+1.

[0090] In the above-mentioned image anomaly detection method, detection coding features are obtained by encoding the original image features of the image to be detected, and prompt coding features are obtained by encoding the original prompt features of the image to be detected, which are extracted from a reference image that does not contain anomalies. At least one level of combined decoding processing is performed based on the detection coding features and the prompt coding features to obtain combined decoding features. In each level of combined decoding processing, attention features are extracted for the target detection coding features and target prompt coding features input at the current level to obtain prompt coding features output at the current level, and attention features are extracted for the prompt coding features and target detection coding features output at the current level to obtain detection coding features output at the current level. Image reconstruction features for the image to be detected are obtained based on the combined decoding features, and the anomaly detection result of the image to be detected is obtained based on the reconstruction difference between the image reconstruction features and the original image features. In image anomaly detection processing, in the combined decoding process at each level, attention features are extracted from the target detection coding features and target cue coding features input at the level to obtain the cue coding features output at the level. Attention features are then extracted from the cue coding features and target detection coding features output at the level to obtain the detection coding features output at the level. This approach can comprehensively utilize the image context information of the reference image excluding anomalies and the corresponding dynamic cue coding features at each level to construct combined decoding features, thereby enhancing the feature representation capability of the combined decoding features and improving the accuracy of image anomaly detection based on the reconstructed features obtained from the combined decoding features.

[0091] In an exemplary embodiment, attention feature extraction is performed on the target detection encoding features and target prompt encoding features input at the current level to obtain the prompt encoding features output at the current level. This includes: obtaining a first key feature and a first value feature based on the target detection encoding features input at the current level, and obtaining a first query feature based on the target prompt encoding features input at the current level; and performing cross-attention feature extraction on the first key feature, the first value feature, and the first query feature to obtain the prompt encoding features output at the current level.

[0092] In the attention mechanism, Q (Query), K (Key), and V (Value) are three core concepts. The basic idea of ​​Q, K, and V is to calculate the relevance between the Query and each Key, obtain the weight of the Value corresponding to each Key, and then perform a weighted sum of the Values ​​to obtain the final attention value. This mechanism can adaptively select and focus on the Key and Value most relevant to the Query when processing information, based on the needs of the Query. Key features can serve as the Key vector in the attention mechanism, value features as the Value vector, and query features as the Query vector.

[0093] Cross-attention is a variant of neural network attention mechanisms, primarily used to handle the correlation information between multiple input sequences or features. In traditional self-attention mechanisms, attention is calculated between the input sequences or features themselves. However, in cross-attention mechanisms, in addition to calculating self-attention within the input sequences or features, attention is also calculated between different input sequences or features. Cross-attention mechanisms can selectively focus on specific regions or features in an image, thereby generating more discriminative feature representations and enabling more accurate recognition and understanding of image content. Furthermore, different levels of attention modules can capture different levels of features, achieving incremental performance improvements. This mechanism makes the model more flexible and efficient when handling complex image tasks.

[0094] Specifically, in the combined decoding process at each level, the server can determine the input of its current level, including object detection encoding features and object prompt encoding features. The input of the current level can be the output of the previous level. The server constructs parameters under the attention mechanism based on the input of each level, including key features, value features, and query features. In specific applications, the server can obtain the first key feature and the first value feature based on the object detection encoding features; for example, the object detection encoding features can be directly used as the first key feature and the first value feature. The server can obtain the first query feature based on the object prompt encoding features input at its current level; for example, the object prompt encoding features can be directly used as the first query feature. The server performs cross-attention feature extraction based on the obtained first key feature, first value feature, and first query feature. Specifically, it can perform feature extraction based on the cross-attention mechanism for the first key feature, first value feature, and first query feature to obtain the prompt encoding features output at the current level.

[0095] In one embodiment, such as Figure 7As shown, in the combined decoding process of a certain level, the target detection encoding features input at the current level are used as key vector (K) and value vector (V), respectively, and the target prompt encoding features input at the current level are used as query vector (Q). Cross-attention feature extraction is performed through key vector (K), value vector (V) and query vector (Q) to obtain the prompt encoding features output at the current level.

[0096] In this embodiment, in the combined decoding process of each level, the server obtains the first key feature and the first value feature based on the target detection coding features input at the level it is in, and obtains the first query feature based on the target prompt coding features input at the level it is in. Based on the determined first key feature, first value feature and first query feature, cross-attention feature extraction is performed, which can realize cross-attention processing from target prompt coding features to target detection coding features, enhance the feature expression capability of the prompt coding features output at the level it is in, and thus help improve the accuracy of image anomaly detection.

[0097] In an exemplary embodiment, cross-attention feature extraction is performed on the first key feature, the first value feature, and the first query feature to obtain the prompt encoding feature of the output at the current level. This includes: extracting cross-attention features based on the first key feature, the first value feature, and the first query feature using an attention mechanism to obtain the first cross-attention feature; performing residual fusion of the first cross-attention feature and the first query feature to obtain the fused feature; performing feature mapping on the fused feature at least once to obtain the mapped feature; and performing residual fusion of the mapped feature and the fused feature to obtain the prompt encoding feature of the output at the current level.

[0098] Among them, cross-attention features are features directly obtained through cross-attention feature extraction based on the attention mechanism, derived from corresponding value features, key features, and query features. Fusion features are obtained by fusing the first cross-attention feature with the first query feature; in specific implementations, the first cross-attention feature and the first query feature can be weighted and fused. Mapping features are obtained by mapping the fusion features; specifically, mapping can be performed through a mapping layer structure, such as using an MLP (Multilayer Perceptron) layer structure to map the fusion features and obtain the mapped features.

[0099] For example, the server can directly extract cross-attention features from the first key feature, the first value feature, and the first query feature based on an attention mechanism to obtain the first cross-attention feature. The server can then perform residual fusion with the first query feature to obtain a fused feature. Residual fusion combines feature information from different layers or different modalities to obtain a richer feature representation. The server can perform at least one feature mapping on the fused feature, such as mapping the fused feature through at least one mapping layer structure to obtain a mapped feature. The server further performs residual fusion on the mapped feature and the fused feature; specifically, it can perform a weighted fusion of the mapped feature and the fused feature to obtain the prompt encoding feature output at the current layer.

[0100] In this embodiment, the server performs residual fusion on the obtained first cross-attention feature and the first query feature, and performs feature mapping on the fused feature at least once. Then, it performs residual fusion on the mapped feature and the fused feature. This can fuse image feature information at different levels, which can enhance the feature expression capability of the prompt coding feature output at the current level, thereby improving the accuracy of image anomaly detection.

[0101] In an exemplary embodiment, attention feature extraction is performed on the prompt encoding features and target detection encoding features output at the current level to obtain the detection encoding features output at the current level. This includes: obtaining a second key feature and a second value feature based on the prompt encoding features output at the current level, and obtaining a second query feature based on the target detection encoding features input at the current level; and performing cross-attention feature extraction on the second key feature, the second value feature, and the second query feature to obtain the detection encoding features output at the current level.

[0102] For example, in the combined decoding process at each level, the server can construct parameters under the attention mechanism based on the target detection coding features input at the current level and the cue coding features output at the current level, including key features, value features, and query features. In specific applications, the server can obtain the second key feature and the second value feature based on the cue coding features output at the current level, such as directly using the cue coding features output at the current level as the second key feature and the second value feature respectively. The server can obtain the second query feature based on the target detection coding features input at the current level, such as directly using the target detection coding features as the second query feature. The server performs cross-attention feature extraction based on the obtained second key feature, second value feature, and second query feature. Specifically, feature extraction can be performed on the second key feature, second value feature, and second query feature based on the cross-attention mechanism to obtain the detection coding features output at the current level.

[0103] In one embodiment, such as Figure 8As shown, in the combined decoding process of a certain level, the prompt encoding features output by the level are used as key vector (K) and value vector (V), respectively, and the target detection encoding features input by the level are used as query vector (Q). Cross-attention feature extraction is performed through key vector (K), value vector (V) and query vector (Q) to obtain the detection encoding features output by the level.

[0104] In this embodiment, in the combined decoding process of each level, the server obtains the second key feature and the second value feature based on the prompt coding features output by the level, and obtains the second query feature based on the target detection coding features input by the level. Based on the determined second key feature, second value feature and second query feature, cross-attention feature extraction is performed, which can realize cross-attention processing from target detection coding features to target prompt coding features, enhance the feature expression capability of the prompt coding features output by the level, and thus help improve the accuracy of image anomaly detection.

[0105] In an exemplary embodiment, cross-attention feature extraction is performed on the second key feature, the second value feature, and the second query feature to obtain the detection coding feature output at the current level. This includes: performing cross-attention feature extraction based on the second key feature, the second value feature, and the second query feature using an attention mechanism to obtain the second cross-attention feature; and performing residual fusion between the second cross-attention feature and the second query feature to obtain the detection coding feature output at the current level.

[0106] Optionally, the server can directly extract cross-attention features from the second key feature, the second value feature, and the second query feature based on an attention mechanism to obtain the second cross-attention feature. The server can then perform residual fusion between the second cross-attention feature and the second query feature to obtain the detection coding feature output at the current level. In this embodiment, the server performs residual fusion between the obtained second cross-attention feature and the second query feature, which can fuse image feature information from different levels, enhance the feature representation capability of the detection coding feature output at the current level, and thus improve the accuracy of image anomaly detection.

[0107] In an exemplary embodiment, at least one level includes multiple levels; performing at least one level of combined decoding processing based on detection coding features and cue coding features to obtain combined decoding features output by at least one level of combined decoding processing includes: in the combined decoding processing of the first level among multiple levels, performing attention feature extraction on the detection coding features and cue coding features input to the first level to obtain cue coding features output by the first level, and performing attention feature extraction on the cue coding features and detection features output by the first level to obtain detection coding features output by the first level; for each level starting from the second level among multiple levels, performing combined decoding processing based on the cue coding features and detection coding features output by the previous level to obtain cue coding features and detection coding features output by the current level, until the combined decoding processing of the last level among multiple levels outputs combined decoding features of multiple levels.

[0108] In the case of multi-level combined decoding processing, for the first level, the detection coding features of the image to be detected and the prompt coding features obtained from the original prompt features are directly used as input for combined decoding. Starting from the second level, the output of the previous level is used as the input for the current level for combined decoding.

[0109] For example, for the combined decoding process of the first level in a multi-level architecture, the server can directly perform attention feature extraction on the detection coding features and cue coding features input to the first level to obtain the cue coding features output to the first level. The server then performs attention feature extraction on the cue coding features and detection features output to the first level to obtain the detection coding features output to the first level. From the second level onwards, for each level, the server uses the cue coding features and detection coding features output to the previous level as the target detection coding features and target cue coding features input to the current level. In a specific implementation, the server can determine the cue coding features and detection coding features output to the previous level, and perform combined decoding processing on the current level based on these features to obtain the cue coding features and detection coding features output to the current level, until the last level completes the combined decoding process. Based on the output of the last level's combined decoding process, the combined decoding features of multiple levels are obtained.

[0110] In this embodiment, the server performs multi-level combined decoding processing based on detection coding features and prompt coding features. This allows for deep attention feature extraction based on detection coding features and prompt coding features, which enhances the feature representation capability of the combined decoding features and thus improves the accuracy of image anomaly detection based on combined decoding features.

[0111] In an exemplary embodiment, obtaining image reconstruction features for the image to be detected based on the combined decoding features includes: performing cross-attention feature extraction on the detection coding features and cue coding features included in the combined decoding features to obtain image reconstruction features for the image to be detected.

[0112] The combined decoding features are obtained from the output of the last level of the combined decoding process in at least one level of combined decoding process. The combined decoding features may include the prompt coding features and detection coding features output by the last level of combined decoding process.

[0113] Specifically, for the combined decoding features output by at least one level of combined decoding processing, the server can perform cross-attention feature extraction based on the detection coding features and cue coding features included in the combined decoding features. For example, cross-attention feature extraction can be performed on the detection coding features and cue coding features included in the combined decoding features based on an attention mechanism to obtain the image reconstruction features of the image to be detected.

[0114] In this embodiment, the server performs cross-attention feature extraction on the detection coding features and prompt coding features included in the combined decoding features. Based on the attention mechanism, the combined decoding features are used to reconstruct the image reconstruction features of the image to be detected. This can further integrate the information carried by the detection coding features and prompt coding features, which can enhance the feature expression ability of the image reconstruction features, thereby improving the accuracy of image anomaly detection.

[0115] In an exemplary embodiment, an anomaly detection result for the image to be detected is obtained based on the reconstruction difference between the image reconstruction features and the original image features, including: determining the reconstruction difference between the image reconstruction features and the original image features; mapping the reconstruction difference to obtain anomaly mapping parameters; and determining the anomaly detection result for the image to be detected based on the anomaly mapping parameters.

[0116] The reconstruction difference is used to represent the similarity between the reconstructed features and the original features of the image, reflecting the error generated during reconstruction of the image to be detected. The anomaly mapping parameters are obtained based on the reconstruction difference mapping, which can be implemented by mapping the reconstruction differences using a preset mapping relationship.

[0117] For example, the server can determine the reconstruction difference between the reconstructed image features and the original image features. Specifically, it can compare the reconstructed image features and the original image features to obtain the reconstruction difference. For instance, the server can determine the reconstruction difference by absolute element-wise subtraction between the reconstructed image features and the original image features. The server can map the reconstruction difference, specifically according to a preset mapping relationship, to obtain anomaly mapping parameters. For instance, the server can perform L2 norm mapping on the reconstruction difference to obtain the L2 norm of the reconstruction difference, and use the L2 norm as the anomaly mapping parameter. The server obtains the anomaly detection result of the image to be detected based on the anomaly mapping parameters. The server can obtain the anomaly detection result of the image to be detected directly from the anomaly mapping parameters, or it can obtain the anomaly detection result of the image to be detected after performing anomaly judgment on the anomaly mapping parameters. For instance, the server can compare the anomaly mapping parameters with a preset threshold. When the anomaly mapping parameters exceed the preset threshold, the image to be detected is considered an anomaly image, thus obtaining an anomaly detection result indicating that the image to be detected is an anomaly image.

[0118] In this embodiment, the server maps the reconstruction differences between the image reconstruction features and the original image features, and determines the anomaly detection result based on the anomaly mapping parameters obtained from the mapping. This enables quantitative anomaly judgment for the image to be detected, which helps to ensure the accuracy of image anomaly detection.

[0119] In one exemplary embodiment, the image anomaly detection method further includes: extracting pixel features at least one level for the reconstruction differences to obtain pixel anomaly results.

[0120] The pixel anomaly result is obtained by extracting pixel features from the reconstructed differences. Pixel feature extraction refers to extracting features at the pixel level based on the reconstructed differences, which can include various feature extraction processes such as convolution, normalization, and linear rectification. For example, the server can further extract pixel features based on the reconstructed differences. Specifically, the server can perform at least one level of pixel feature extraction on the reconstructed differences, such as performing at least one level of convolution, normalization, linear rectification, deconvolution, and other pixel feature extraction operations to obtain the pixel anomaly result. The pixel anomaly result reflects the pixel-level anomaly detection result of the image to be detected, and can specifically include an anomaly map, which can be the image region in the image to be detected where anomalies are detected.

[0121] Furthermore, determining the anomaly detection result for the image to be detected based on the anomaly mapping parameters includes: weighted fusion of the anomaly mapping parameters and the pixel anomaly results to obtain an anomaly fusion result; and determining the anomaly detection result for the image to be detected based on the anomaly fusion result.

[0122] The anomaly fusion result is obtained by weighted fusion of anomaly mapping parameters and pixel anomaly results. Specifically, the server can fuse the anomaly mapping parameters and pixel anomaly results according to a preset weighting, resulting in the anomaly fusion result. The server then uses this anomaly fusion result to obtain the anomaly detection result for the image to be detected. In practical applications, the anomaly fusion result may include quantization parameters. The server can then perform a quality comparison between the anomaly fusion result and a threshold parameter to determine the anomaly detection result for the image to be detected based on the numerical comparison result.

[0123] In this embodiment, the server performs at least one level of pixel feature extraction on the reconstruction differences to obtain pixel anomaly results, and determines the anomaly detection result of the image to be detected by weighted fusion of anomaly mapping parameters and pixel anomaly results. Anomaly detection can be performed on the image to be detected from different dimensions, which can improve the accuracy of image anomaly detection.

[0124] In an exemplary embodiment, encoding the original image features of the image to be detected to obtain the detection encoding features of the image to be detected includes: extracting the original image features of the image to be detected, and performing at least one level of attention feature encoding on the original image features to obtain the detection encoding features of the image to be detected.

[0125] In this process, the original image features can be extracted from the image to be detected through a pre-trained feature extraction network, serving as the initial image features extracted from the image to be detected. Feature encoding based on an attention mechanism is then performed on the original image features to obtain the detection encoded features. To encode deeper features, multiple attention feature encodings can be performed on the original image features to obtain the detection encoded features.

[0126] Specifically, the server can perform feature extraction on the acquired image to be detected. For example, it can use a pre-trained feature extraction network to extract features from the image to obtain the original image features. The server then performs attention feature encoding on the original image features. Specifically, it can use an encoding network to perform at least one level of attention feature encoding on the original image features to obtain the detection encoding features of the image to be detected. In specific applications, the server can perform at least one level of attention feature encoding on the original image features based on a self-attention mechanism, that is, perform at least one level of self-attention feature extraction on the original image features to obtain the detection encoding features of the image to be detected. In some embodiments, various processing methods such as residual fusion and feature mapping can also be introduced for attention feature encoding to enrich the information carried by the detection encoding features and ensure the feature representation capability of the detection encoding features.

[0127] Further, the original cue features associated with the image to be detected are determined, and the original cue features are encoded to obtain cue encoded features, including: determining the original cue features associated with the image to be detected from the cue feature pool, the cue feature pool including at least one candidate original cue feature extracted based on the reference image; and performing at least one level of attention feature encoding on the original cue features to obtain cue encoded features.

[0128] The cue feature pool stores the original features of candidate cues, which are image features extracted from a reference image. The reference image can be a pre-determined normal image, i.e., one that does not contain anomalies. The original cue features can be encoded using an attention mechanism to obtain the cue encoded features.

[0129] Specifically, the server can determine the original cue features associated with the image to be detected from the cue feature pool. The cue feature pool includes at least one candidate original cue features extracted based on a reference image, which are obtained by pre-extracting features from the reference image. In some embodiments, the server can query a preset cue feature pool and match it with the original image features to obtain the original cue features associated with the image to be detected. The server can call an encoding network to perform at least one level of attention feature encoding on the original cue features to obtain cue encoded features. In specific applications, the server can perform at least one level of attention feature encoding on the original cue features based on a self-attention mechanism, that is, perform at least one level of self-attention feature extraction on the original cue features to obtain cue encoded features. In some embodiments, various processing methods such as residual fusion and feature mapping can also be introduced for attention feature encoding to enrich the information carried by the cue encoded features and ensure the feature expressive power of the cue encoded features.

[0130] In this embodiment, the server performs at least one level of attention feature encoding on the original image features and the original prompt features. Based on the attention mechanism, the information of the detection encoding features and prompt encoding features can be enriched, thereby ensuring the feature representation ability of the detection encoding features and prompt encoding features, which is beneficial to improving the accuracy of image anomaly detection.

[0131] In an exemplary embodiment, determining the original cue features associated with the image to be detected from the cue feature pool includes: determining the cue feature pool; performing feature matching between the original image features and each candidate original cue features in the cue feature pool to obtain feature matching results; and determining the candidate original cue features corresponding to the feature matching results that indicate feature matching as the original cue features associated with the image to be detected.

[0132] The cue feature pool stores the original features of candidate cues extracted from each reference image. The feature matching result characterizes the degree of feature matching between the original features of the image and the original features of the corresponding candidate cues.

[0133] Specifically, the server can query a preset cue feature pool and perform feature matching between the original image features and each candidate cue feature in the pool. For example, the server can calculate the feature similarity between the original image features and each candidate cue feature, thereby obtaining the feature matching result. The feature matching result can be used to characterize the degree of feature similarity between the original image features and the candidate cue features, thus reflecting whether the original image features and the candidate cue features match. The server can determine the feature matching result indicating a feature match and use the candidate cue feature corresponding to this feature matching result to determine the cue feature associated with the image to be detected.

[0134] In this embodiment, the server performs feature matching between the original features of the image and each candidate original feature of the prompt feature pool, and determines the original feature of the prompt based on the feature matching results. This allows for the rapid and accurate acquisition of the original feature of the prompt associated with the image to be detected for anomaly detection processing, which is beneficial to improving the processing efficiency and accuracy of image anomaly detection.

[0135] In an exemplary embodiment, the image anomaly detection method is implemented through an image anomaly detection model. The training steps of the image anomaly detection model include: acquiring normal sample images, reference sample images associated with normal sample images, and abnormal sample images; extracting features from the normal sample images, reference sample images, and abnormal sample images respectively to obtain the original normal image features of the normal sample images, the original reference image features of the reference sample images, and the original abnormal image features of the abnormal sample images; encoding the original normal image features, the original reference image features, and the original abnormal image features using the image anomaly detection model to be trained, to obtain the normal sample encoded features of the normal sample images, the cue sample encoded features of the reference sample images, and the abnormal sample encoded features of the abnormal sample images; and training the image anomaly detection model to be trained. The detection model performs at least one level of combined decoding processing based on normal sample encoding features and cue sample encoding features to obtain a first combined decoding feature output by at least one level of combined decoding processing, and obtains normal sample reconstruction features for normal sample images based on the first combined decoding feature. The image anomaly detection model to be trained performs at least one level of combined decoding processing based on abnormal sample encoding features and cue sample encoding features to obtain a second combined decoding feature output by at least one level of combined decoding processing, and obtains abnormal sample recovery features for abnormal sample images based on the second combined decoding feature. The image anomaly detection model to be trained is updated based on the normal sample reconstruction features, abnormal sample recovery features, and original normal image features, and then trained again until training is complete, resulting in a trained image anomaly detection model.

[0136] Image anomaly detection models can be built based on various artificial neural network algorithms and are used to detect anomalies in input images. Normal sample images are those that do not contain image anomalies. Reference sample images are associated with normal sample images and serve as a reference; the image features carried by the reference sample images are used as cue features to guide the training of the image anomaly detection model. Anomaly sample images are those that contain image anomalies. Normal sample images, reference sample images, and anomaly sample images all belong to the same application scenario; the only difference lies in whether there are anomalies in the image content, such as missing parts, scratches, decay, or other abnormal content.

[0137] Specifically, the server can acquire sample images for the image anomaly detection model, including normal sample images, reference sample images, and anomaly sample images. For each of these images, the server can perform feature extraction. For example, a pre-trained feature extraction network can be used to extract features from each image, obtaining their respective original image features. These features include the original features of normal images, the original features of reference images, and the original features of anomaly images. The server can construct an image anomaly detection model to be trained. This model can be built based on at least one of convolutional neural networks, recurrent neural networks, transformers, feedforward neural networks, generative adversarial networks, or autoencoders. The server can encode the sample images using the model to obtain their respective encoded features. Specifically, the server can acquire the original features of normal images, the original features of reference images, and the original features of anomaly images. The original features of each sample image can be extracted using the same pre-trained feature extraction network. The server can encode the original features of normal images, reference images, and abnormal images separately using the image anomaly detection model to be trained, resulting in normal sample encoding features for normal sample images, cue sample encoding features for reference sample images, and abnormal sample encoding features for abnormal sample images. In a specific implementation, the image anomaly detection model to be trained may include an encoder network, which encodes the input image's original features to output corresponding encoded features.

[0138] The server can use the image anomaly detection model to be trained to perform at least one level of combined decoding processing based on normal sample encoding features and cue sample encoding features to obtain the corresponding first combined decoding features, and obtain normal sample reconstruction features based on the first combined decoding features. The server can also use the image anomaly detection model to be trained to perform at least one level of combined decoding processing based on anomaly sample encoding features and cue sample encoding features to obtain the corresponding second combined decoding features, and obtain anomaly sample recovery features based on the second combined decoding features. In specific implementations, the server can use the image anomaly detection model to be trained to perform combined decoding processing using feature combinations of normal sample encoding features + cue sample encoding features and anomaly sample encoding features + cue sample encoding features respectively, so as to obtain the corresponding normal sample reconstruction features and anomaly sample recovery features based on their respective combined decoding features. In each level of combined decoding processing, the image anomaly detection model to be trained performs attention feature extraction on the target detection encoding features and target cue encoding features input at the level, obtaining the cue encoding features output at the level, and performs attention feature extraction on the cue encoding features and target detection encoding features output at the level, obtaining the detection encoding features output at the level.

[0139] The server updates the image anomaly detection model under training based on features reconstructed from normal samples, features restored from abnormal samples, and original features of normal images. This eliminates the need for manually pre-labeled tags, enabling unsupervised training of the image anomaly detection model. For example, the model's target loss can be constructed using these features, and the model parameters are updated based on this target loss. Training continues until completion, yielding a fully trained image anomaly detection model. Training termination conditions can be set according to actual needs, such as reaching a preset number of iterations, meeting the accuracy requirements of the image anomaly detection model, or completing training on all sample images.

[0140] In this embodiment, the server uses the image anomaly detection model to be trained to reconstruct features based on the combination of normal sample coding features and prompt sample coding features to obtain normal sample reconstructed features. It then performs feature recovery based on the combination of abnormal sample coding features and prompt sample coding features to obtain abnormal sample recovered features. Finally, the image anomaly detection model is updated based on the normal sample reconstructed features, abnormal sample recovered features, and normal image original features. By introducing abnormal sample images and reference sample images for model training, the generalization ability of the image anomaly detection model can be enhanced, and the accuracy of image anomaly detection based on the image anomaly detection model can be improved.

[0141] In one exemplary embodiment, such as Figure 9As shown, the model update process involves updating the image anomaly detection model under training based on the reconstructed features of normal samples, the restored features of abnormal samples, and the original features of normal images, and then continuing training until training is complete, resulting in a fully trained image anomaly detection model. This includes:

[0142] Step 902: Determine the sample reconstruction loss based on the sample reconstruction difference between the reconstructed features of normal samples and the original features of normal images.

[0143] Here, the sample reconstruction difference characterizes the feature difference between the reconstructed features of normal samples and the original features of the normal image. The sample reconstruction loss is obtained based on the sample reconstruction difference and is used to adjust the degree of influence of the reconstructed features of normal samples on the model update.

[0144] Specifically, the server can determine the sample reconstruction loss based on the sample reconstruction difference between the reconstructed features of normal samples and the original features of the normal image. Specifically, the server can determine the sample reconstruction difference between the reconstructed features of normal samples and the original features of the normal image, such as by comparing the reconstructed features of normal samples with the original features of the normal image to obtain the sample reconstruction difference, and then construct the sample reconstruction loss based on this difference. The form of the sample reconstruction loss can be set according to actual needs, and may include, but is not limited to, at least one or a combination of Mean Squared Error (MSE), Cross-Entropy Loss (CE), Logarithmic Loss (LogLoss), or L1 Loss (Mean Absolute Error, MAE).

[0145] Step 904: Determine the sample recovery loss based on the sample recovery difference between the abnormal sample recovery features and the original features of the normal image.

[0146] The sample recovery difference characterizes the feature differences between the recovered features of abnormal samples and the original features of normal images. The sample recovery loss is derived from the sample recovery difference and is used to adjust the degree of influence of the recovered features of abnormal samples on the model update.

[0147] Specifically, the server can determine the sample recovery loss based on the sample recovery difference between the abnormal sample recovery features and the original features of the normal image. Specifically, the server can determine the sample recovery difference between the abnormal sample recovery features and the original features of the normal image. For example, it can compare the abnormal sample recovery features with the original features of the normal image to obtain the sample recovery difference, and construct the sample recovery loss based on the sample recovery difference. The form of the sample recovery loss can be set according to actual needs.

[0148] Step 906: Update the image anomaly detection model to be trained based on the sample reconstruction loss and sample recovery loss, and continue training until training is complete to obtain the trained image anomaly detection model.

[0149] Specifically, the server updates the model parameters of the image anomaly detection model under training based on the sample reconstruction loss and sample recovery loss, and continues training using the updated model until training is complete, thus obtaining the trained image anomaly detection model. In practical applications, the server can obtain the model's target loss based on the sum or weighted sum of the sample reconstruction loss and sample recovery loss, and update the image anomaly detection model under training based on the target loss. The sample reconstruction loss and sample recovery loss are directly calculated based on the normal sample reconstruction features, anomaly sample recovery features, and normal image original features obtained during training, without the need for manually pre-labeled tags, enabling unsupervised training of the image anomaly detection model.

[0150] In this embodiment, the server determines the sample reconstruction loss based on the sample reconstruction difference between the reconstructed features of normal samples and the original features of normal images, and determines the sample recovery loss based on the sample recovery difference between the recovered features of abnormal samples and the original features of normal images. The model is updated by combining the sample reconstruction loss and the sample recovery loss. The model can be trained from two dimensions: feature reconstruction and feature recovery. This can ensure the generalization ability of the image anomaly detection model for image anomaly detection, thereby improving the accuracy of image anomaly detection.

[0151] In an exemplary embodiment, the image anomaly detection method further includes: determining a first feature difference between the reconstructed features of a normal sample and the original features of a normal image, and determining a second feature difference between the restored features of an abnormal sample and the original features of an abnormal image; using the image anomaly detection model to be trained, performing at least one level of pixel feature extraction for the first feature difference and the second feature difference respectively, to obtain a first pixel anomaly result and a second pixel anomaly result; and determining a pixel anomaly detection loss based on the anomaly region label carried by the abnormal sample image, the first pixel anomaly result, and the second pixel anomaly result.

[0152] The first feature difference characterizes the feature difference between the reconstructed features of the normal sample and the original features of the normal image. Specifically, it can be obtained by subtracting the reconstructed features of the normal sample from the original features of the normal image element-wise. The second feature difference characterizes the feature difference between the restored features of the abnormal sample and the original features of the abnormal image. Specifically, it can be obtained by subtracting the restored features of the abnormal sample from the original features of the abnormal image element-wise. Pixel anomaly results are obtained by extracting pixel features from the feature differences. Pixel feature extraction refers to extracting features at the pixel level based on the feature differences, which can include various feature extraction processes such as convolution, normalization, and linear rectification. Pixel anomaly results can include first pixel anomaly results and second pixel anomaly results. The first pixel anomaly result is obtained by extracting pixel features at least one level based on the first feature difference, and the second pixel anomaly result is obtained by extracting pixel features at least one level based on the second feature difference. Anomaly region labels can be pre-annotated on the abnormal sample images to characterize the abnormal regions included in the abnormal sample images.

[0153] Specifically, the server can determine the first feature difference based on the reconstructed features of normal samples and the original features of normal images, and determine the second feature difference based on the restored features of abnormal samples and the original features of abnormal images. For example, the server can perform absolute element-wise subtraction between the reconstructed features of normal samples and the original features of normal images, and vice versa, to obtain the first and second feature differences. The server then uses the image anomaly detection model to be trained to further extract pixel features based on the first and second feature differences. Specifically, at least one level of pixel feature extraction can be performed, such as at least one level of convolution, normalization, linear rectification, deconvolution, and other pixel feature extraction operations, to obtain the first and second pixel anomaly results, respectively. The server can obtain the anomaly region labels carried by the abnormal sample images. These labels can be pre-annotated manually to characterize the abnormal regions in the abnormal sample images. The server then constructs a pixel anomaly detection loss by combining the anomaly region labels, the first pixel anomaly result, and the second pixel anomaly result. In some embodiments, the server can construct a pixel anomaly detection loss based on the anomaly region label, the first pixel anomaly result, and the second pixel anomaly result, according to a selected loss function. The form of the loss function can be flexibly set according to actual needs. The pixel anomaly detection loss is obtained by combining the anomaly region label, the first pixel anomaly result, and the second pixel anomaly result; that is, it requires the introduction of pre-annotated anomaly region labels for the anomaly sample images. The optimization of the pixel anomaly detection loss calculation belongs to supervised training.

[0154] In some embodiments, the first pixel anomaly result is obtained based on the first feature difference between the reconstructed features of the normal sample and the original features of the normal image. Since there are no abnormal regions in the normal sample image corresponding to the original features of the normal image, the abnormal region represented by the abnormal region label of the normal sample image can be considered empty. The server can directly determine the first pixel anomaly detection loss for the normal sample image based on the first pixel anomaly result. For the second pixel anomaly result, it is obtained based on the second feature difference between the recovered features of the abnormal sample and the original features of the abnormal image. The abnormal region in the abnormal sample image corresponding to the original features of the abnormal image is represented by the carried abnormal region label. The server can compare the abnormal regions based on the carried abnormal region label of the abnormal sample image and the second pixel anomaly result, and obtain the second pixel anomaly detection loss for the abnormal sample image based on the comparison result. The server can combine the first pixel anomaly detection loss and the second pixel anomaly detection loss to obtain the pixel anomaly detection loss, such as by summing the first pixel anomaly detection loss and the second pixel anomaly detection loss.

[0155] Furthermore, the image anomaly detection model to be trained is updated based on the sample reconstruction loss and sample recovery loss, and then trained again until training is complete, to obtain a trained image anomaly detection model, including: fusing the sample reconstruction loss, sample recovery loss and pixel anomaly detection loss to obtain the model target loss; the image anomaly detection model to be trained is updated based on the model target loss, and then trained again until training is complete, to obtain a trained image anomaly detection model.

[0156] The pixel anomaly detection loss reflects the pixel-level anomaly detection results, while the model target loss is obtained by combining the sample reconstruction loss, sample restoration loss, and pixel anomaly detection loss. The model target loss is used to update the image anomaly detection model. Specifically, the server can fuse the sample reconstruction loss, sample restoration loss, and pixel anomaly detection loss, for example, by weighting them according to set weights to obtain the model target loss. The server updates the model parameters of the image anomaly detection model to be trained based on the model target loss, and continues training based on the updated model until training is complete, obtaining the trained image anomaly detection model.

[0157] In this embodiment, the server further constructs a pixel-level pixel anomaly detection loss based on the first feature difference and the second feature difference. The model target loss is obtained by combining the abnormal region label carried by the abnormal sample image, the sample reconstruction loss, the sample recovery loss and the pixel anomaly detection loss. The image anomaly detection model is updated, and the model can be trained from three dimensions: feature reconstruction, feature recovery and pixel-level anomaly detection. This can ensure the generalization ability of the image anomaly detection model for image anomaly detection, thereby improving the accuracy of image anomaly detection.

[0158] In one exemplary embodiment, such as Figure 10 As shown, an image anomaly detection model processing method is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking the server in the example, the explanation includes the following steps 1002 to 1012. Wherein:

[0159] Step 1002: Obtain normal sample images, reference sample images associated with normal sample images, and abnormal sample images.

[0160] The image anomaly detection model can be constructed based on various artificial neural network algorithms. Normal sample images are those that do not contain image anomalies. Reference sample images are associated with normal sample images and serve as a reference. The image features carried by the reference sample images are used as cue features to guide the training of the image anomaly detection model. Anomaly sample images are those that contain image anomalies. Normal sample images, reference sample images, and anomaly sample images all belong to the same application scenario, differing only in whether the image content contains anomalies, such as missing parts, scratches, or decay. Specifically, the server can obtain sample images for the image anomaly detection model, including normal sample images, reference sample images, and anomaly sample images.

[0161] Step 1004: Perform feature extraction on normal sample images, reference sample images and abnormal sample images respectively to obtain the original features of normal images, the original features of reference images, and the original features of abnormal images.

[0162] For example, for normal sample images, reference sample images, and abnormal sample images, the server can perform feature extraction separately. For instance, a pre-trained feature extraction network can extract features for normal sample images, reference sample images, and abnormal sample images respectively to obtain their respective original image features, specifically including the original features of normal sample images, the original features of reference sample images, and the original features of abnormal sample images.

[0163] Step 1006: Using the image anomaly detection model to be trained, the original features of the normal image, the original features of the reference image, and the original features of the abnormal image are encoded respectively to obtain the normal sample encoding features of the normal sample image, the prompt sample encoding features of the reference sample image, and the abnormal sample encoding features of the abnormal sample image.

[0164] Specifically, the server can encode sample images using the image anomaly detection model to be trained, and obtain their respective encoding features, including normal sample encoding features of normal sample images, prompt sample encoding features of reference sample images, and abnormal sample encoding features of abnormal sample images.

[0165] Step 1008: Using the image anomaly detection model to be trained, perform at least one level of combined decoding processing based on normal sample coding features and cue sample coding features to obtain the first combined decoding feature output by at least one level of combined decoding processing, and obtain normal sample reconstruction features for normal sample images based on the first combined decoding feature.

[0166] Optionally, the server can use the image anomaly detection model to be trained to perform combined decoding processing using the feature combination of normal sample encoding features and cue sample encoding features to obtain the corresponding normal sample reconstruction features. In the combined decoding processing at each level, the image anomaly detection model to be trained performs attention feature extraction on the target detection encoding features and target cue encoding features input at the level to obtain the cue encoding features output at the level, and performs attention feature extraction on the cue encoding features and target detection encoding features output at the level to obtain the detection encoding features output at the level.

[0167] Step 1010: Using the image anomaly detection model to be trained, perform at least one level of combined decoding processing based on the anomaly sample encoding features and the cue sample encoding features to obtain the second combined decoding features output by the at least one level of combined decoding processing, and obtain the anomaly sample recovery features for the anomaly sample image based on the second combined decoding features.

[0168] Optionally, the server can use the image anomaly detection model to be trained to perform combined decoding processing using the feature combination of anomaly sample encoding features and cue sample encoding features to obtain the corresponding anomaly sample recovery features. In the combined decoding processing at each level, the image anomaly detection model to be trained performs attention feature extraction on the target detection encoding features and target cue encoding features input at the level to obtain the cue encoding features output at the level, and performs attention feature extraction on the cue encoding features and target detection encoding features output at the level to obtain the detection encoding features output at the level.

[0169] Step 1012: After updating the image anomaly detection model to be trained based on the reconstruction features of normal samples, the recovery features of abnormal samples, and the original features of normal images, continue training until training is completed to obtain the trained image anomaly detection model; the trained image anomaly detection model is used to perform image anomaly detection on the input image.

[0170] Specifically, the server can construct the model's target loss using features reconstructed from normal samples, features recovered from abnormal samples, and original features from normal images. The model parameters of the image anomaly detection model to be trained are then updated based on this target loss, and training continues until completion, resulting in a fully trained image anomaly detection model. The training termination condition can be set according to actual needs, such as reaching a preset number of training iterations, meeting the accuracy requirements of the image anomaly detection model, or completing training on all sample images.

[0171] The above-described image anomaly detection model processing method involves encoding the original features of normal sample images, reference sample images associated with normal sample images, and abnormal sample images using the image anomaly detection model to be trained. This yields normal sample encoding features, cue sample encoding features, and abnormal sample encoding features. The image anomaly detection model to be trained then performs at least one level of combined decoding based on the normal sample encoding features and cue sample encoding features to construct normal sample reconstruction features for normal sample images. The image anomaly detection model to be trained then performs at least one level of combined decoding based on the abnormal sample encoding features and cue sample encoding features to construct abnormal sample recovery features for abnormal sample images. Finally, the image anomaly detection model to be trained is updated based on the normal sample reconstruction features, abnormal sample recovery features, and original features of the normal image, and then trained again to obtain a trained image anomaly detection model for detecting image anomalies in input images. By introducing cue sample coding features and combining them with normal sample coding features for at least one level of combined decoding processing to reconstruct features for normal sample images, and introducing cue sample coding features and combining them with abnormal sample coding features for at least one level of combined decoding processing to restore features for abnormal sample images, the image anomaly detection capability of the image anomaly detection model can be enhanced by utilizing the image context information of the reference sample image and the corresponding dynamic cue coding features at each level. This improves the accuracy of the trained image anomaly detection model in detecting image anomalies in input images.

[0172] In an exemplary embodiment, the image anomaly detection model to be trained is updated based on normal sample reconstruction features, abnormal sample recovery features, and normal image original features, and then trained again until training is completed to obtain a trained image anomaly detection model. This includes: determining a sample reconstruction loss based on the sample reconstruction difference between normal sample reconstruction features and normal image original features; determining a sample recovery loss based on the sample recovery difference between abnormal sample recovery features and normal image original features; updating the image anomaly detection model to be trained based on the sample reconstruction loss and sample recovery loss, and then training again until training is completed to obtain a trained image anomaly detection model.

[0173] Specifically, the server can determine the sample reconstruction differences between the reconstructed features of normal samples and the original features of normal images. For example, it can compare the reconstructed features of normal samples with the original features of normal images to obtain the sample reconstruction differences, and construct a sample reconstruction loss based on these differences. Similarly, the server can compare the restored features of abnormal samples with the original features of normal images to obtain the sample restoration differences, and construct a sample restoration loss based on these differences. The forms of the sample reconstruction loss and the sample restoration loss can be set according to actual needs. The server can obtain the model's target loss based on the sum or weighted sum of the sample reconstruction loss and the sample restoration loss, and update the image anomaly detection model to be trained based on the model's target loss.

[0174] In this embodiment, the server determines the sample reconstruction loss based on the sample reconstruction difference between the reconstructed features of normal samples and the original features of normal images, and determines the sample recovery loss based on the sample recovery difference between the recovered features of abnormal samples and the original features of normal images. The model is updated by combining the sample reconstruction loss and the sample recovery loss. The model can be trained from two dimensions: feature reconstruction and feature recovery. This can ensure the generalization ability of the image anomaly detection model for image anomaly detection, thereby improving the accuracy of image anomaly detection.

[0175] In an exemplary embodiment, the image anomaly detection model processing method further includes: determining a first feature difference between the reconstructed features of normal samples and the original features of normal images, and a second feature difference between the restored features of abnormal samples and the original features of abnormal images; using the image anomaly detection model to be trained, performing at least one level of pixel feature extraction for the first feature difference and the second feature difference respectively, to obtain a first pixel anomaly result and a second pixel anomaly result; determining a pixel anomaly detection loss based on the abnormal region label carried by the abnormal sample image, the first pixel anomaly result, and the second pixel anomaly result; updating the image anomaly detection model to be trained based on the sample reconstruction loss and the sample restoration loss, and continuing training until training is completed, to obtain a trained image anomaly detection model, including: fusing the sample reconstruction loss, the sample restoration loss, and the pixel anomaly detection loss to obtain a model target loss; updating the image anomaly detection model to be trained based on the model target loss, and continuing training until training is completed, to obtain a trained image anomaly detection model.

[0176] For example, the server can perform absolute element-wise subtraction between the reconstructed features of normal samples and the original features of normal images, and perform absolute element-wise subtraction between the recovered features of abnormal samples and the original features of abnormal images to obtain the first feature difference and the second feature difference. The server then uses the image anomaly detection model to be trained to perform at least one level of convolution operations, normalization, linear rectification, deconvolution, and other pixel feature extractions on the first and second feature differences respectively, to obtain the first pixel anomaly result and the second pixel anomaly result. The server constructs a pixel anomaly detection loss by combining the anomaly region label, the first pixel anomaly result, and the second pixel anomaly result. In some embodiments, the server can construct the pixel anomaly detection loss according to a selected loss function form based on the anomaly region label, the first pixel anomaly result, and the second pixel anomaly result. The form of the loss function can be flexibly set according to actual needs.

[0177] Furthermore, the server can perform weighted fusion of sample reconstruction loss, sample recovery loss and pixel anomaly detection loss according to the set weights to obtain the model target loss. Based on the model target loss, the model parameters of the image anomaly detection model to be trained are updated, and the training continues based on the updated image anomaly detection model until training is completed, thus obtaining the trained image anomaly detection model.

[0178] In this embodiment, the server further constructs a pixel-level pixel anomaly detection loss based on the first feature difference and the second feature difference. The model's target loss is obtained by integrating the anomaly region label carried by the anomaly sample image, the sample reconstruction loss, the sample recovery loss, and the pixel anomaly detection loss. This update the image anomaly detection model allows for model training from three dimensions: feature reconstruction, feature recovery, and pixel-level anomaly detection. This ensures the generalization ability of the image anomaly detection model for image anomaly detection, thereby improving the accuracy of image anomaly detection. This application also provides an application scenario where the above-described image anomaly detection method is applied. Specifically, the image anomaly detection method is applied in this scenario as follows:

[0179] For image anomaly detection, there are currently two main approaches: reconstruction-based unsupervised anomaly detection methods and text-based anomaly detection methods. Reconstruction-based unsupervised anomaly detection methods assume that anomalous image regions or features should not be correctly reconstructed because they do not exist in normal training samples. Some methods use generative models, such as autoencoders and generative adversarial networks (GANs), to reconstruct normal images. RGI (Robust GAN-Inversion) proposes a robust GAN inversion that can restore any input image (even severely damaged ones) to a clean image and identifies damaged region masks by solving its optimization problem. Some methods treat anomaly detection as an image inpainting problem, where patches in the image are partially masked. RIAD (Region-based Image Anomaly Detection) randomly removes parts of the image and uses a convolutional neural network to reconstruct the image of the removed regions using the unremoved parts. SSPCAB (Spatial and Channel-wise Pixel-wise Anomaly Block) learns to reconstruct masked regions using contextual information with masked convolutional kernels. To enhance the diversity of reconstructions while avoiding unwanted anomaly generalization, pyramid deformation modules have been proposed to model different normals and measure the severity of anomalies. However, these methods tend to be computationally expensive because they involve reconstruction in the original image space. UniAD (Unified model for Anomaly Detection) extracts features from a pre-trained model and reconstructs those features, achieving unified anomaly detection and balancing efficiency and performance. However, UniAD's pixel-level anomaly segmentation remains unsatisfactory.

[0180] For text-based anomaly detection methods, the use of large pre-trained visual language models (such as CLIP, Contrastive Language–Image Pre-training) has shown unprecedented versatility and achieved impressive performance on various tasks, such as few-shot image classification, open-word object detection, and text-to-image generation. Utilizing multiple text cues on the powerful CLIP model can yield excellent performance in zero-shot and few-shot anomaly detection. Furthermore, applying multi-turn dialogue AnomalyGPT (Anomaly Generative Pre-Trained Transformer) not only indicates the presence and location of anomalies but also provides a detailed description of the anomalies in the test image. However, these methods primarily rely on text cues to identify anomalies, and the workload of constructing text cues for images is substantial, impacting the processing efficiency of image anomaly detection. The image anomaly detection method provided in this embodiment detects anomalies by using normal images as visual cues, which is more intuitive and helps improve the accuracy and processing efficiency of image anomaly detection.

[0181] For image anomaly detection, unsupervised reconstruction methods based on the Transformer with self-attention mechanism have achieved state-of-the-art results in unified (single model, multi-class) anomaly detection. However, these self-attention-based reconstruction models primarily operate on the target features themselves, while the anomaly itself and its context may exhibit high consistency, leading to perfect reconstruction of both normal and anomalous features, thus causing true anomaly detection to fail. To ensure efficiency in the reconstruction process, the reconstruction network typically operates in a low-resolution latent space, but this generally results in poor anomaly localization performance. To enhance the generalization ability of the reconstruction model for anomaly detection while maintaining efficiency, the image anomaly detection method provided in this embodiment requires only a single normal image cue to reconstruct normal features for normal samples and restore anomalous features to normal for anomalous samples. The image anomaly detection method provided in this embodiment allows for the reconstruction or restoration of anomalies using only a single normal image cue, effectively improving anomaly detection performance. Furthermore, the image anomaly detection method provided in this embodiment also includes a supervised fine-tuning module, which regresses reconstruction errors using real normal images and synthetic anomalous images, significantly improving the performance of pixel-level anomaly segmentation. The image anomaly detection method provided in this embodiment outperforms previous methods on three anomaly detection benchmark datasets: MVTec, BTAD, and ViSA.

[0182] Specifically, such as Figure 11The diagram shown illustrates the unified unsupervised anomaly detection framework in the image anomaly detection method provided in this embodiment. The input includes normal images. ), prompt image, ) and pseudo anomaly images There are three types of images. These three images are processed through a feature extraction network (Backbone) to obtain their respective original image features, including the image features corresponding to the normal image. , prompting the image features corresponding to the image and the image features corresponding to the abnormal images In this diagram, solid lines represent unsupervised reconstruction, dashed lines represent unsupervised restoration, and dashed boxes represent supervised refinement. Image features corresponding to normal images, cue images, and abnormal images are encoded using N encoders. Each encoder includes a self-attention layer (self-attn) and a multilayer perceptron (MLP) layer connected sequentially to obtain the encoded features corresponding to normal images. Encoding features of the prompt image and the coding features corresponding to the abnormal images .

[0183] For unsupervised reconstruction, the encoded features of the normal image and the encoded features of the prompt image are decoded by N decoders. Each decoder includes a cross-attention layer 1 (cross-attn), a multilayer perceptron (MLP) layer, and a cross-attention layer 2 (cross-attn) connected in sequence. For cross-attention layer 1, the prompt encoded features output by the previous decoder are processed... As the query vector (q), the normal encoded features output by the previous decoder are used. As the key vector (k) and value vector (v), the output of cross-attention layer 1 is input into the multilayer perceptron layer; for cross-attention layer 2, the normal encoded features output from the previous decoder are used. As the query vector, the output of the multilayer perceptron layer is used as the key vector and value vector, and the output of the cross-attention layer 2 is used as the cue encoding feature of the decoder output of this layer. The output of the multilayer perceptron layer serves as the normal encoded feature output of the decoder at that layer. The output of the last decoder is passed through a cross-attention layer (cross-attn) to obtain the reconstructed features. Reconstruction features Image features corresponding to normal images The reconstruction error between the two parts is processed by a refiner to obtain the final output.

[0184] For unsupervised recovery processing, the input to the decoder consists of the encoded features of the cue image and the encoded features of the anomalous image. The output of the last decoder is then passed through a cross-attention layer to obtain the recovered features. The restored features correspond to the image features of the normal image. The recovery error between the two is fine-tuned to obtain the final output.

[0185] Furthermore, during the training phase, both normal and synthetic images are input into the pre-trained backbone network to extract multi-level representations. Guided by cues from normal images, the unsupervised reconstruction stream reconstructs normal features for normal samples, while the unsupervised recovery stream recovers features from synthetic anomalous samples to normal features. Additionally, a supervised fine-tuning module is used to regress the reconstruction errors of both normal and synthetic anomalous images. Notably, the unsupervised recovery stream is purged during inference, thus incurring no additional computational overhead.

[0186] Furthermore, regarding the processing of reconstruction based on normal image cues, the image anomaly detection method provided in this embodiment focuses on unified anomaly detection, which means that a single model is needed to handle data from multiple categories. Compared to anomaly detection methods that use one model per category, unified anomaly detection deals with data with more complex distributions, and is therefore more challenging. The UniAD encoder-decoder architecture is a powerful and state-of-the-art unified unsupervised anomaly detection solution. UniAD consists of a Neighborhood Masked Attention (NMA) and a layered query decoder (LQD). NMA restricts a feature from seeing itself and its neighborhood, thus employing its contextual features (long dependencies) for reconstruction. On the other hand, learnable query embeddings... First, with encoder embedding Merge, then with the output Integration of the first There are several decoder layers. Experiments show that the learned query embeddings can alleviate overfitting.

[0187] For simplicity, the MLP (Multilayer Perceptron), residual connections, layer normalization, and dropout in LQD are omitted. Then, in the LQD... In each module, the two important steps are processed as shown in equations (1) and (2) below.

[0188] (1)

[0189] (2)

[0190] in, It is attention feature fusion processing. for The transpose of the matrix, yes Dimensions From the first block in LQD Initialization yielded, For the first The output of each module. The LQD described above can be implemented using multi-head self-attention. It is important to note that within each sub-block of the LQD, It is parameterized and learnable, and Maintaining a fixed and invariant approach, where LQD only reconstructs features on its own, can fail when faced with challenging scenarios such as masquerading anomalies.

[0191] The image anomaly detection method provided in this embodiment aims to reduce reconstruction difficulty and improve anomaly detection performance by relying not only on the image's own structure and features but also by using normal image cues as guidance. A simple approach is to directly use ordinary image cues. encoder output Replace the query embedded in LQD This enables interaction between prompts and target features. Therefore, LQD is transformed as shown in equations (3) and (4) below.

[0192] (3)

[0193] (4)

[0194] This change is simple, but it has entirely different implications and can significantly improve the performance of anomaly detection. In the modified version described above, the cue encoding interacts statically with the target features in a one-way manner, hence it is called a one-way decoder with static cues. However, this one-way mode may not be flexible enough, especially when the target features are constantly updated, and may fail to align with the target features.

[0195] Unlike the static cues in a one-way decoder, the image anomaly detection method provided in this embodiment uses a pair of bidirectional cross-attention pairs to dynamically update cues and target features, as shown in equations (5) and (6) below.

[0196] (5)

[0197] (6)

[0198] in, and From the encoder module and The initialization is as follows. The bidirectional decoder with dynamic cues described above simulates bidirectional feature interactions, including cue-to-feature and feature-to-cue interactions. The first interaction performs a cross-attention from the cue (as a query) to the target feature, and the second interaction performs another cross-attention from the target feature (as a query) to the cue. The next decoder block obtains updated cues and target features from the previous block. This allows target feature reconstruction to utilize not only its contextual information but also dynamically leverage corresponding normal cues. Notably, this bidirectional modeling approach also enhances the flexibility of cue features, enabling it to adapt to variations in the distribution of target features to some extent. Finally, a cross-attention is used to update the cue on the bidirectional decoder output, which is treated as the original feature. Reconstruction features ( The reconstruction loss function calculates the mean squared error (MSE) between the reconstructed features and the original features, as shown in equation (7) below.

[0199] (7)

[0200] in, To rebuild the losses, For image channels, Image height, This represents the image width.

[0201] Furthermore, regarding the processing of reconstruction based on normal image cues, unsupervised reconstruction learning is performed only on normal training images, which may cause the model to rely more on its own contextual information and weaken the participation of image cues in the reconstruction process. To further enhance the guiding nature of image cues, one anticipated approach is to increase the difficulty of the reconstruction task, forcing the network to rely not only on its own contextual information but also on the cues provided by normal image cues. To achieve this goal, the image anomaly detection method provided in this embodiment introduces artificially synthesized pseudo-anomaly images. It can be achieved through normal training images Adding damage or destruction to the image can easily generate it, such as using CutPaste and DRAEM (Discriminatively trained Reconstruction Anomaly Embedding Model). Based on the pseudo-anomaly image, the previous reconstruction can be transformed into a restoration problem, hoping to provide clues in the normal image. Under the guidance of abnormal features Restored to normal characteristics This recovery method is consistent with the expectations of the reconstruction model during the testing phase.

[0202] Similar to the reconstruction process, the image anomaly detection method provided in this embodiment first divides a pair of images ( and The input is fed into a pre-trained backbone to extract offline features. and Then obtain the final repair features. The offline paired features are applied sequentially to the self-attention encoder and the bidirectional cross-attention decoder. Specifically, the self-attention encoder applies the offline paired features in parallel to the self-attention encoder. and As input, the output is and Next, the bidirectional decoder is... and Initialize and update dynamically. In the bidirectional decoder... Within the block, dynamic features and prompts are represented as and It can be easily obtained by simply using replace Unlike the objective function in unsupervised reconstruction, the recovery loss function calculates the recovered features. ) and the corresponding original normal features ( ), specifically as shown in equation (8),

[0203] (8)

[0204] in, To recover from the losses.

[0205] For the supervised fine-tuning module, given normal training images and corresponding anomaly mask (With all elements being zero), abnormal images can be synthesized. and its anomaly mask is represented as Then, the normal image and the composite image are compared. They are fed into a pre-trained backbone network, and their offline representations are derived as follows: Then, the proposed reconstruction stream will be used to restore normal operation. Reconstructed And use the recommended recovery flow to resolve the exception. Restore to The difference between the original features and the reconstructed (restored) features is measured by the absolute element-wise subtraction, as shown in equation (9) below.

[0206] (9)

[0207] In fact, the differences in the equation of Norms can be used to roughly locate anomalous regions. However, in this way, it is difficult to accurately locate anomalous regions because feature reconstruction or recovery is performed in a low-resolution latent space (i.e., 1 / 16 of the original input).

[0208] Note the synthesized anomalous image. Naturally includes pixel-level anomaly masking The image anomaly detection method provided in this embodiment aims to fully utilize... To further refine the reconstruction error from low resolution to high resolution, the image anomaly detection method provided in this embodiment designs a lightweight pixel-level fine-tuning module based on the reconstruction error for anomaly segmentation. This module consists of several transposed convolutional blocks, followed by a 1x1 convolutional layer. Here, each transposed convolutional block incorporates the reconstruction error... The upsampling is 2x, and it consists of 3x3 convolution, BatchNorm (Batch Normalization), ReLU (Rectified Linear Unit), and 2x2 deconvolution. In the experiments, the image anomaly detection method provided in this embodiment uses two transposed convolutional blocks, thereby reducing the reconstruction error relative to the input image from 1 / 16 to 1 / 4. Finally, a 1x1 convolutional layer transforms the number of channels of the amplified reconstruction error to 1, and obtains the estimated anomaly map. In order to calculate and the true value The losses between them will further Adjusted to The resolution size. Considering that anomalous pixels usually account for a minority in anomaly detection, the model is optimized using the Dice loss function, which is very effective for learning from extremely imbalanced data, as shown in equation (10) below.

[0209] (10)

[0210] in, express or Spatial location within; To fine-tune the loss, for or height, for or The width.

[0211] For training and inference processing, considering that the proposed scheme involves three objectives, including unsupervised reconstruction, unsupervised recovery, and supervised fine-tuning, the overall training loss function is shown in equation (11) below.

[0212] (11)

[0213] in, It is a balance between two types of loss functions and Importance weights. During inference, offline cue features for all categories are pre-extracted and stored in a cue pool. Given a test image. and its characteristics The optimal cue features are selected using the cosine similarity between the features and the cue pool. For unsupervised reconstruction, the anomaly score map is calculated as a fraction of the reconstruction error. The norm is shown in equation (12) below.

[0214] (12)

[0215] in, This is an anomaly score map for unsupervised reconstruction.

[0216] For the supervised refiner, the anomaly score map was predicted as follows: Finally, (Adjusted to original resolution) and Combined, they form the final anomaly segmentation map, as shown in equation (13) below.

[0217] (13)

[0218] in, These are weights; anomaly classification aims to detect whether an image contains abnormal regions. The image anomaly detection method provided in this embodiment will... The maximum value is used as the image-level anomaly score.

[0219] In a specific application, the ROC (Receiver Operating Characteristic) / PR (Precision-Recall) metrics for image-level anomaly classification and pixel-level anomaly segmentation were compared on datasets such as MVTec, BTAD, and VisA. All methods were evaluated under uniform settings, where the results are averages across multiple classes.

[0220] like Figure 12 The table shows a comparison of image anomaly detection results on three different datasets. Specifically, it evaluates representative methods for image-level classification and pixel-level segmentation on three anomaly detection datasets (MVTec, BTAD, and VisA). The metrics include image-level and pixel-level metrics. Image-level metrics can include the ROC / PR ratio for image-level anomaly classification (I-ROC / PR), and pixel-level metrics can include the ROC / PR ratio for pixel-level anomaly segmentation (P-ROC / PR). The referenced image anomaly detection methods include embedding-based, discriminator-based, and reconstruction-based methods. Embedding-based methods include Reference 1 (CS-Flow, Cross-Scale Flow), Reference 2 (PaDiM, Patch Distribution Modeling), and Reference 3 (DFM, Distribution-based Feature). Image anomaly detection methods based on discriminators include references 6 (DRAEM) and 7 (SimpleNet), while reconstruction-based methods include reference 8 (UniAD) and the image anomaly detection method in this scheme. Specifically, some important observations are summarized below:

[0221] When extending single-model, single-class methods to single-model, multi-class methods, most state-of-the-art approaches suffer significant performance degradation in both image-level classification and pixel-level segmentation, consistent with UniAD's observations. For example, the state-of-the-art SimpleNet experienced a drop of approximately 21% in I-ROC (from 99.6% to 78.2%) and approximately 17% in P-ROC (from 98.1% to 81.0%).

[0222] The image anomaly detection method provided in this embodiment outperforms all competitors and significantly surpasses the state-of-the-art UniAD in pixel-level anomaly segmentation on all three datasets, for example, from 44.7% to 63.7% on MVTec, from 50.9% to 56.8% on BTAD, and from 33.6% to 43.3% on VisA.

[0223] Some methods are not robust to different application scenarios, while the image anomaly detection method provided in this embodiment consistently outperforms the best current methods. For example, DRAEM achieves a P-PR of 49.6% on MVTec, but only 12.3% on BTAD and 15.1% on VisA.

[0224] For image-level anomaly classification, the image anomaly detection method provided in this embodiment also surpasses UniAD in most cases. For example, it improves the I-ROC performance on MVTec from 96.5% to 97.9% and the I-ROC performance on VisA from 90.8% to 92.5%.

[0225] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0226] Based on the same inventive concept, this application also provides an image anomaly detection apparatus for implementing the image anomaly detection method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image anomaly detection apparatus embodiments provided below can be found in the limitations of the image anomaly detection method described above, and will not be repeated here.

[0227] In one exemplary embodiment, such as Figure 13 As shown, an image anomaly detection device 1300 is provided, including: an image processing module 1302, a cue feature acquisition module 1304, a combined decoding processing module 1306, a reconstruction feature acquisition module 1308, and a reconstruction difference processing module 1310, wherein:

[0228] The image processing module 1302 is used to acquire the image to be detected and encode the original features of the image to be detected to obtain the detection coding features of the image to be detected.

[0229] The prompt feature acquisition module 1304 is used to determine the original prompt features associated with the image to be detected, and to encode the original prompt features to obtain the prompt encoded features;

[0230] The combined decoding processing module 1306 is used to perform at least one level of combined decoding processing based on detection coding features and prompt coding features to obtain combined decoding features output by at least one level of combined decoding processing. In each level of combined decoding processing, attention features are extracted for the target detection coding features and target prompt coding features input at the level to obtain prompt coding features output at the level. Attention features are also extracted for the prompt coding features and target detection coding features output at the level to obtain detection coding features output at the level.

[0231] The reconstruction feature acquisition module 1308 is used to obtain image reconstruction features for the image to be detected based on the combined decoding features;

[0232] The reconstruction difference processing module 1310 is used to obtain anomaly detection results for the image to be detected based on the reconstruction difference between the image reconstruction features and the original image features.

[0233] In one embodiment, the combined decoding processing module 1306 is further configured to obtain a first key feature and a first value feature based on the target detection encoding features input at the current level, and to obtain a first query feature based on the target prompt encoding features input at the current level; and to perform cross-attention feature extraction on the first key feature, the first value feature and the first query feature to obtain the prompt encoding features output at the current level.

[0234] In one embodiment, the combined decoding processing module 1306 is further configured to extract cross-attention features based on the first key feature, the first value feature, and the first query feature, thereby obtaining a first cross-attention feature; perform residual fusion of the first cross-attention feature and the first query feature to obtain a fused feature; perform feature mapping on the fused feature at least once to obtain a mapped feature; and perform residual fusion of the mapped feature and the fused feature to obtain the prompt encoding feature of the output at the current level.

[0235] In one embodiment, the combined decoding processing module 1306 is further configured to obtain a second key feature and a second value feature based on the prompt encoding features output by the current level, and to obtain a second query feature based on the target detection encoding features input by the current level; and to perform cross-attention feature extraction on the second key feature, the second value feature and the second query feature to obtain the detection encoding features output by the current level.

[0236] In one embodiment, the combined decoding processing module 1306 is further configured to extract cross-attention features based on the second key feature, the second value feature, and the second query feature, thereby obtaining the second cross-attention feature; and to perform residual fusion of the second cross-attention feature and the second query feature to obtain the detection coding feature output at the current level.

[0237] In one embodiment, at least one level includes multiple levels; the combined decoding processing module 1306 is further configured to, in the combined decoding processing of the first level among multiple levels, perform attention feature extraction on the detection coding features and prompt coding features input to the first level to obtain the prompt coding features output to the first level, and perform attention feature extraction on the prompt coding features and detection features output to the first level to obtain the detection coding features output to the first level; for each level starting from the second level among multiple levels, perform combined decoding processing on the prompt coding features and detection coding features output by the previous level to obtain the prompt coding features and detection coding features output by the level, until the combined decoding processing of the last level among multiple levels outputs the combined decoding features of multiple levels.

[0238] In one embodiment, the reconstruction feature acquisition module 1308 is further configured to perform cross-attention feature extraction on the detection coding features and cue coding features included in the combined decoding features based on an attention mechanism, so as to obtain image reconstruction features for the image to be detected.

[0239] In one embodiment, the reconstruction difference processing module 1310 is further configured to determine the reconstruction difference between the image reconstruction features and the original image features; map the reconstruction difference to obtain anomaly mapping parameters; and determine the anomaly detection result for the image to be detected based on the anomaly mapping parameters.

[0240] In one embodiment, the system further includes a pixel feature processing module for extracting pixel features at least one level for the reconstruction differences to obtain pixel anomaly results; the reconstruction difference processing module 1310 is also used to weightedly fuse the anomaly mapping parameters with the pixel anomaly results to obtain an anomaly fusion result; and to determine the anomaly detection result for the image to be detected based on the anomaly fusion result.

[0241] In one embodiment, the image processing module 1302 is further configured to extract the original image features of the image to be detected and perform at least one level of attention feature encoding on the original image features to obtain the detection encoding features of the image to be detected; the prompt feature acquisition module 1304 is further configured to determine the prompt original features associated with the image to be detected from the prompt feature pool, the prompt feature pool including at least one candidate prompt original feature extracted based on the reference image; and perform at least one level of attention feature encoding on the prompt original features to obtain the prompt encoding features.

[0242] In one embodiment, the prompt feature acquisition module 1304 is further configured to determine a prompt feature pool; perform feature matching between the original image features and each candidate prompt original feature in the prompt feature pool to obtain a feature matching result; and determine the candidate prompt original feature corresponding to the feature matching result that indicates feature matching as the prompt original feature associated with the image to be detected.

[0243] In one embodiment, the method is implemented using an image anomaly detection model; it further includes a model training module for acquiring normal sample images, reference sample images associated with normal sample images, and anomaly sample images; using the image anomaly detection model to be trained, encoding the original normal image features of the normal sample images, the original reference image features of the reference sample images, and the original anomaly image features of the anomaly sample images respectively, to obtain normal sample encoding features of the normal sample images, cue sample encoding features of the reference sample images, and anomaly sample encoding features of the anomaly sample images; using the image anomaly detection model to be trained, performing at least one level of combined decoding based on the normal sample encoding features and the cue sample encoding features. The process involves obtaining at least one level of combined decoding output first combined decoding features, and obtaining normal sample reconstruction features for normal sample images based on the first combined decoding features; using the image anomaly detection model to be trained, performing at least one level of combined decoding based on the anomaly sample encoding features and the cue sample encoding features to obtain at least one level of combined decoding output second combined decoding features, and obtaining anomaly sample recovery features for anomaly sample images based on the second combined decoding features; updating the image anomaly detection model to be trained based on the normal sample reconstruction features, the anomaly sample recovery features, and the original features of the normal image, and continuing training until training is complete to obtain a trained image anomaly detection model.

[0244] In one embodiment, the model training module is further configured to determine the sample reconstruction loss based on the sample reconstruction difference between the reconstructed features of normal samples and the original features of normal images; determine the sample recovery loss based on the sample recovery difference between the recovered features of abnormal samples and the original features of normal images; and continue training after updating the image anomaly detection model to be trained based on the sample reconstruction loss and the sample recovery loss, until training is completed, thereby obtaining the trained image anomaly detection model.

[0245] In one embodiment, the model training module is further configured to extract pixel features at least one level for sample reconstruction differences and sample recovery differences using the image anomaly detection model to be trained, to obtain a first pixel anomaly result and a second pixel anomaly result; determine a pixel anomaly detection loss based on the first pixel anomaly result and the second pixel anomaly result; fuse the sample reconstruction loss, the sample recovery loss and the pixel anomaly detection loss to obtain the model target loss; update the image anomaly detection model to be trained based on the model target loss and continue training until training is completed, to obtain a trained image anomaly detection model.

[0246] Each module in the aforementioned image anomaly detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0247] Based on the same inventive concept, this application also provides an image anomaly detection model processing apparatus for implementing the image anomaly detection model processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more embodiments of the image anomaly detection model processing apparatus provided below can be found in the limitations of the image anomaly detection model processing method described above, and will not be repeated here.

[0248] In one exemplary embodiment, such as Figure 14 As shown, an image anomaly detection model processing device 1400 is provided, including: a sample image acquisition module 1402, a sample feature extraction module 1404, a sample feature encoding module 1406, a normal sample reconstruction module 1408, an abnormal sample recovery module 1410, and a model update module 1412, wherein:

[0249] The sample image acquisition module 1402 is used to acquire normal sample images, reference sample images associated with normal sample images, and abnormal sample images;

[0250] The sample feature extraction module 1404 is used to extract features from normal sample images, reference sample images and abnormal sample images respectively, to obtain the original features of normal images, the original features of reference images, and the original features of abnormal images.

[0251] The sample feature encoding module 1406 is used to encode the original features of the normal image, the original features of the reference image, and the original features of the abnormal image through the image anomaly detection model to be trained, so as to obtain the normal sample encoding features of the normal sample image, the prompt sample encoding features of the reference sample image, and the abnormal sample encoding features of the abnormal sample image.

[0252] The normal sample reconstruction module 1408 is used to perform at least one level of combined decoding processing based on normal sample encoding features and prompt sample encoding features through the image anomaly detection model to be trained, to obtain the first combined decoding feature output by the at least one level of combined decoding processing, and to obtain normal sample reconstruction features for normal sample images based on the first combined decoding feature.

[0253] The abnormal sample recovery module 1410 is used to perform at least one level of combined decoding processing based on the abnormal sample encoding features and the prompt sample encoding features through the image anomaly detection model to be trained, to obtain the second combined decoding features output by the at least one level of combined decoding processing, and to obtain the abnormal sample recovery features for the abnormal sample image based on the second combined decoding features.

[0254] The model update module 1412 is used to update the image anomaly detection model to be trained based on the reconstruction features of normal samples, the recovery features of abnormal samples, and the original features of normal images, and then continue training until training is completed to obtain the trained image anomaly detection model; the trained image anomaly detection model is used to perform image anomaly detection on the input image.

[0255] In one embodiment, the model update module 1412 is further configured to determine the sample reconstruction loss based on the sample reconstruction difference between the normal sample reconstruction features and the original features of the normal image; determine the sample recovery loss based on the sample recovery difference between the abnormal sample recovery features and the original features of the normal image; update the image anomaly detection model to be trained based on the sample reconstruction loss and the sample recovery loss and continue training until training is completed, thereby obtaining the trained image anomaly detection model.

[0256] In one embodiment, the model update module 1412 is further configured to determine a first feature difference between the reconstructed features of normal samples and the original features of normal images, and a second feature difference between the restored features of abnormal samples and the original features of abnormal images; extract pixel features at least one level for the first feature difference and the second feature difference using the image anomaly detection model to be trained, to obtain a first pixel anomaly result and a second pixel anomaly result; determine a pixel anomaly detection loss based on the abnormal region label carried by the abnormal sample image, the first pixel anomaly result, and the second pixel anomaly result; fuse the sample reconstruction loss, the sample restoration loss, and the pixel anomaly detection loss to obtain the model target loss; update the image anomaly detection model to be trained based on the model target loss and continue training until training is completed, to obtain the trained image anomaly detection model.

[0257] Each module in the aforementioned image anomaly detection model processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0258] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 15 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores image anomaly detection data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements an image anomaly detection method or an image anomaly detection model processing method.

[0259] Those skilled in the art will understand that Figure 15 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0260] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0261] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0262] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0263] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0264] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0265] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0266] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image anomaly detection method, characterized in that, The method includes: The image to be detected is acquired, and the original features of the image to be detected are encoded to obtain the detection encoding features of the image to be detected; The original features of the prompts associated with the image to be detected are determined, and the original features of the prompts are encoded to obtain the prompt encoded features; the original features of the prompts are extracted from the reference image corresponding to the image to be detected, which does not contain anomalies. Based on the detection coding features and the prompt coding features, at least one level of combined decoding processing is performed to obtain the combined decoding features output by the at least one level of combined decoding processing. In each level of combined decoding processing, attention features are extracted for the target detection coding features and target prompt coding features input at the level to obtain the prompt coding features output at the level. Attention features are also extracted for the prompt coding features output at the level and the target detection coding features to obtain the detection coding features output at the level. Based on the combined decoding features, image reconstruction features for the image to be detected are obtained; Based on the reconstruction difference between the image reconstruction features and the original image features, anomaly detection results are obtained for the image to be detected.

2. The method according to claim 1, characterized in that, The attention feature extraction is performed on the target detection coding features and target cue coding features input at the current level to obtain the cue coding features output at the current level, including: The first key feature and the first value feature are obtained based on the target detection encoding features input at the current level, and the first query feature is obtained based on the target prompt encoding features input at the current level. Cross-attention feature extraction is performed on the first key feature, the first value feature, and the first query feature to obtain the prompt encoding feature of the output at the current level.

3. The method according to claim 2, characterized in that, The step of performing cross-attention feature extraction on the first key feature, the first value feature, and the first query feature to obtain the prompt encoding feature of the output at the current level includes: Based on the first key feature, the first value feature, and the first query feature, cross-attention feature extraction is performed using an attention mechanism to obtain the first cross-attention feature; The first cross-attention feature and the first query feature are residually fused to obtain the fused feature; Perform feature mapping at least once on the fused features to obtain the mapped features; The mapping features and the fusion features are residually fused to obtain the prompt encoding features of the output at the current level.

4. The method according to claim 1, characterized in that, The attention feature extraction is performed on the cue coding features of the output at the current level and the target detection coding features to obtain the detection coding features of the output at the current level, including: The second key feature and the second value feature are obtained based on the prompt encoding features output by the level, and the second query feature is obtained based on the target detection encoding features input by the level. Cross-attention feature extraction is performed on the second key feature, the second value feature, and the second query feature to obtain the detection coding feature of the output at the current level.

5. The method according to claim 4, characterized in that, The step of performing cross-attention feature extraction on the second key feature, the second value feature, and the second query feature to obtain the detection coding features output at the current level includes: Based on the second key feature, the second value feature, and the second query feature, cross-attention feature extraction is performed using an attention mechanism to obtain the second cross-attention feature; The second cross-attention feature and the second query feature are residually fused to obtain the detection coding feature output at the current level.

6. The method according to claim 1, characterized in that, The at least one level includes multiple levels; the at least one level of combined decoding processing based on the detection coding features and the cue coding features to obtain the combined decoding features output by the at least one level of combined decoding processing includes: In the combined decoding process of the first level among the multiple levels, attention feature extraction is performed on the detection coding feature and the cue coding feature input to the first level to obtain the cue coding feature output to the first level, and attention feature extraction is performed on the cue coding feature output to the first level and the detection coding feature to obtain the detection coding feature output to the first level. Starting from the second level among the multiple levels, each level performs combined decoding processing based on the prompt coding features and detection coding features output by the previous level to obtain the prompt coding features and detection coding features output by the current level, until the combined decoding processing of the last level among the multiple levels outputs the combined decoding features of the multiple levels.

7. The method according to claim 1, characterized in that, The step of obtaining image reconstruction features for the image to be detected based on the combined decoding features includes: Cross-attention feature extraction is performed on the detection coding features and cue coding features included in the combined decoding features to obtain image reconstruction features for the image to be detected.

8. The method according to claim 1, characterized in that, The method of obtaining anomaly detection results for the image to be detected based on the reconstruction difference between the image reconstruction features and the original image features includes: Determine the reconstruction differences between the reconstructed image features and the original image features; Mapping the reconstructed differences yields anomaly mapping parameters; The anomaly detection result for the image to be detected is determined based on the anomaly mapping parameters.

9. The method according to claim 8, characterized in that, The method further includes: At least one level of pixel feature extraction is performed on the reconstructed differences to obtain pixel anomaly results; Determining the anomaly detection result for the image to be detected based on the anomaly mapping parameters includes: The anomaly mapping parameters and the pixel anomaly results are weighted and fused to obtain the anomaly fusion result; The anomaly detection result for the image to be detected is determined based on the anomaly fusion result.

10. The method according to claim 1, characterized in that, The process of encoding the original image features of the image to be detected to obtain the detection encoding features of the image to be detected includes: Extract the original image features of the image to be detected, and perform at least one level of attention feature encoding on the original image features to obtain the detection encoding features of the image to be detected; The process of determining the original features of the prompt associated with the image to be detected, and encoding the original features of the prompt to obtain the prompt encoded features, includes: The original cue features associated with the image to be detected are determined from the cue feature pool, which includes at least one candidate original cue feature extracted based on the reference image; Attention feature encoding at least one level is performed on the original cue features to obtain cue encoded features.

11. The method according to claim 10, characterized in that, The step of determining the original cue features associated with the image to be detected from the cue feature pool includes: Determine the feature pool for prompts; The original features of the image are matched with the original features of each candidate prompt in the prompt feature pool to obtain the feature matching results. The candidate prompt original features corresponding to the feature matching results that represent matching features are determined as the prompt original features associated with the image to be detected.

12. The method according to any one of claims 1 to 11, characterized in that, The method is implemented using an image anomaly detection model; the training steps of the image anomaly detection model include: Acquire normal sample images, reference sample images associated with the normal sample images, and abnormal sample images; Feature extraction is performed on the normal sample image, the reference sample image, and the abnormal sample image respectively to obtain the original normal image features of the normal sample image, the original reference image features of the reference sample image, and the original abnormal image features of the abnormal sample image. The image anomaly detection model to be trained encodes the original features of the normal image, the original features of the reference image, and the original features of the abnormal image, respectively, to obtain the normal sample encoding features of the normal sample image, the prompt sample encoding features of the reference sample image, and the abnormal sample encoding features of the abnormal sample image. The image anomaly detection model to be trained performs at least one level of combined decoding processing based on the normal sample encoding features and the prompt sample encoding features to obtain the first combined decoding feature output by the at least one level of combined decoding processing, and obtains the normal sample reconstruction features for the normal sample image based on the first combined decoding feature. The image anomaly detection model to be trained performs at least one level of combined decoding processing based on the anomaly sample encoding features and the cue sample encoding features to obtain a second combined decoding feature output by at least one level of combined decoding processing, and obtains anomaly sample recovery features for the anomaly sample image based on the second combined decoding features. The image anomaly detection model to be trained is updated based on the normal sample reconstruction features, the abnormal sample recovery features, and the normal image original features, and then trained until the training is completed, thus obtaining the trained image anomaly detection model.

13. The method according to claim 12, characterized in that, The process of updating the image anomaly detection model to be trained based on the reconstructed features of normal samples, the restored features of abnormal samples, and the original features of normal images, and then continuing training until training is completed, to obtain the trained image anomaly detection model, includes: The sample reconstruction loss is determined based on the sample reconstruction difference between the reconstructed features of the normal sample and the original features of the normal image; The sample recovery loss is determined based on the sample recovery difference between the abnormal sample recovery features and the original features of the normal image; The image anomaly detection model to be trained is updated based on the sample reconstruction loss and the sample recovery loss, and then trained until training is completed, thus obtaining the trained image anomaly detection model.

14. The method according to claim 13, characterized in that, The method further includes: Determine a first feature difference between the reconstructed features of the normal sample and the original features of the normal image, and determine a second feature difference between the restored features of the abnormal sample and the original features of the abnormal image; Using the image anomaly detection model to be trained, at least one level of pixel feature extraction is performed on the first feature difference and the second feature difference respectively to obtain the first pixel anomaly result and the second pixel anomaly result; The pixel anomaly detection loss is determined based on the anomaly region label carried by the anomaly sample image, the first pixel anomaly result, and the second pixel anomaly result; The process of updating the image anomaly detection model to be trained based on the sample reconstruction loss and the sample recovery loss, and then continuing training until training is completed to obtain the trained image anomaly detection model, includes: The model target loss is obtained by fusing the sample reconstruction loss, the sample recovery loss, and the pixel anomaly detection loss. The image anomaly detection model to be trained is updated based on the target loss of the model and then trained again until training is completed, thus obtaining the trained image anomaly detection model.

15. A method for processing an image anomaly detection model, characterized in that, The method includes: Acquire normal sample images, reference sample images associated with the normal sample images, and abnormal sample images; Feature extraction is performed on the normal sample image, the reference sample image, and the abnormal sample image respectively to obtain the original normal image features of the normal sample image, the original reference image features of the reference sample image, and the original abnormal image features of the abnormal sample image. The image anomaly detection model to be trained encodes the original features of the normal image, the original features of the reference image, and the original features of the abnormal image, respectively, to obtain the normal sample encoding features of the normal sample image, the prompt sample encoding features of the reference sample image, and the abnormal sample encoding features of the abnormal sample image. The image anomaly detection model to be trained performs at least one level of combined decoding processing based on the normal sample encoding features and the prompt sample encoding features to obtain the first combined decoding feature output by the at least one level of combined decoding processing, and obtains the normal sample reconstruction features for the normal sample image based on the first combined decoding feature. The image anomaly detection model to be trained performs at least one level of combined decoding processing based on the anomaly sample encoding features and the cue sample encoding features to obtain a second combined decoding feature output by at least one level of combined decoding processing, and obtains anomaly sample recovery features for the anomaly sample image based on the second combined decoding features. The image anomaly detection model to be trained is updated based on the normal sample reconstruction features, the abnormal sample recovery features, and the normal image original features, and then trained until training is completed to obtain a trained image anomaly detection model; the trained image anomaly detection model is used to perform image anomaly detection on the input image.

16. The method according to claim 15, characterized in that, The process of updating the image anomaly detection model to be trained based on the reconstructed features of normal samples, the restored features of abnormal samples, and the original features of normal images, and then continuing training until training is complete, to obtain a trained image anomaly detection model, includes: The sample reconstruction loss is determined based on the difference between the reconstructed features of normal samples and the original features of normal images; the sample recovery loss is determined based on the difference between the recovered features of abnormal samples and the original features of normal images. The image anomaly detection model to be trained is updated based on the sample reconstruction loss and the sample recovery loss, and then trained again until training is completed, thus obtaining a trained image anomaly detection model.

17. The method according to claim 16, characterized in that, The method further includes: The first feature difference between the reconstructed features of normal samples and the original features of normal images, and the second feature difference between the restored features of abnormal samples and the original features of abnormal images are determined respectively. Using the image anomaly detection model to be trained, at least one level of pixel feature extraction is performed on the first feature difference and the second feature difference to obtain the first pixel anomaly result and the second pixel anomaly result; the pixel anomaly detection loss is determined based on the anomaly region label carried by the anomaly sample image, the first pixel anomaly result and the second pixel anomaly result. The process of updating the image anomaly detection model to be trained based on the sample reconstruction loss and the sample recovery loss, and then continuing training until training is complete, to obtain a trained image anomaly detection model, includes: The model target loss is obtained by fusing sample reconstruction loss, sample recovery loss, and pixel anomaly detection loss. The training process is repeated until the training of the image anomaly detection model is completed, based on the model's objective loss.

18. An image anomaly detection device, characterized in that, The device includes: The image processing module is used to acquire the image to be detected and encode the original image features of the image to be detected to obtain the detection coding features of the image to be detected. The prompt feature acquisition module is used to determine the original prompt features associated with the image to be detected, and to encode the original prompt features to obtain prompt encoded features; the original prompt features are extracted from a reference image corresponding to the image to be detected that does not contain anomalies. The combined decoding processing module is used to perform at least one level of combined decoding processing based on the detection coding features and the prompt coding features to obtain the combined decoding features output by the at least one level of combined decoding processing. In each level of combined decoding processing, attention features are extracted for the target detection coding features and target prompt coding features input at the level to obtain the prompt coding features output at the level. Attention features are also extracted for the prompt coding features output at the level and the target detection coding features to obtain the detection coding features output at the level. A reconstruction feature acquisition module is used to obtain image reconstruction features for the image to be detected based on the combined decoding features; The reconstruction difference processing module is used to obtain anomaly detection results for the image to be detected based on the reconstruction difference between the image reconstruction features and the original image features.

19. The apparatus according to claim 18, characterized in that, The combined decoding processing module is further configured to: obtain a first key feature and a first value feature based on the target detection encoding features input at the current level, and obtain a first query feature based on the target prompt encoding features input at the current level; perform cross-attention feature extraction on the first key feature, the first value feature and the first query feature to obtain the prompt encoding features output at the current level.

20. The apparatus according to claim 19, characterized in that, The combined decoding processing module is further configured to: extract cross-attention features based on the first key feature, the first value feature, and the first query feature using an attention mechanism to obtain a first cross-attention feature; perform residual fusion of the first cross-attention feature and the first query feature to obtain a fused feature; perform feature mapping at least once on the fused feature to obtain a mapped feature; and perform residual fusion of the mapped feature and the fused feature to obtain the prompt encoding feature for the output of the current level.

21. The apparatus according to claim 18, characterized in that, The combined decoding processing module is further configured to: obtain a second key feature and a second value feature based on the prompt encoding features output by the current level, and obtain a second query feature based on the target detection encoding features input by the current level; perform cross-attention feature extraction on the second key feature, the second value feature and the second query feature to obtain the detection encoding features output by the current level.

22. The apparatus according to claim 21, characterized in that, The combined decoding processing module is further configured to: extract cross-attention features based on the second key feature, the second value feature and the second query feature, to obtain a second cross-attention feature; and perform residual fusion of the second cross-attention feature and the second query feature to obtain the detection coding feature output at the current level.

23. The apparatus according to claim 18, characterized in that, The at least one level includes multiple levels; the combined decoding processing module is configured to: in the combined decoding processing of the first level among the multiple levels, perform attention feature extraction on the detection coding feature and the prompt coding feature input to the first level to obtain the prompt coding feature output by the first level, and perform attention feature extraction on the prompt coding feature output by the first level and the detection coding feature to obtain the detection coding feature output by the first level; for each level starting from the second level among the multiple levels, perform combined decoding processing of the current level based on the prompt coding feature and the detection coding feature output by the previous level to obtain the prompt coding feature and the detection coding feature output by the current level, until the combined decoding processing of the last level among the multiple levels outputs the combined decoding feature of the multiple levels.

24. The apparatus according to claim 18, characterized in that, The reconstruction feature acquisition module is further configured to: perform cross-attention feature extraction on the detection coding features and cue coding features included in the combined decoding features to obtain image reconstruction features for the image to be detected.

25. The apparatus according to claim 18, characterized in that, The reconstruction difference processing module is further configured to: determine the reconstruction difference between the image reconstruction features and the original image features; map the reconstruction difference to obtain anomaly mapping parameters; and determine the anomaly detection result for the image to be detected based on the anomaly mapping parameters.

26. The apparatus according to claim 25, characterized in that, The device further includes a pixel feature processing module, used for: extracting pixel features at least one level for the reconstruction differences to obtain pixel anomaly results; the reconstruction difference processing module is also used for weighted fusion of the anomaly mapping parameters and the pixel anomaly results to obtain anomaly fusion results; and determining anomaly detection results for the image to be detected based on the anomaly fusion results.

27. The apparatus according to claim 18, characterized in that, The image processing module is further configured to: extract the original image features of the image to be detected, and perform at least one level of attention feature encoding on the original image features to obtain the detection encoding features of the image to be detected; the prompt feature acquisition module is further configured to: determine the original prompt features associated with the image to be detected from the prompt feature pool, wherein the prompt feature pool includes at least one candidate original prompt feature extracted based on the reference image; Attention feature encoding at least one level is performed on the original cue features to obtain cue encoded features.

28. The apparatus according to claim 27, characterized in that, The prompt feature acquisition module is further configured to: determine a prompt feature pool; perform feature matching between the original features of the image and each candidate prompt original feature in the prompt feature pool to obtain a feature matching result; and determine the candidate prompt original feature corresponding to the feature matching result that indicates feature matching as the prompt original feature associated with the image to be detected.

29. The apparatus according to any one of claims 18 to 28, characterized in that, The device is implemented using an image anomaly detection model. The device also includes a model training module, used for: acquiring normal sample images, reference sample images associated with the normal sample images, and abnormal sample images; extracting features from the normal sample images, the reference sample images, and the abnormal sample images respectively to obtain the original normal image features of the normal sample images, the original reference image features of the reference sample images, and the original abnormal image features of the abnormal sample images; and encoding the original normal image features, the original reference image features, and the original abnormal image features using the image anomaly detection model to be trained, respectively, to obtain the normal sample encoding features of the normal sample images, the prompt sample encoding features of the reference sample images, and the abnormal sample encoding features of the abnormal sample images. The image anomaly detection model to be trained is subjected to at least one level of combined decoding processing based on the normal sample encoding features and the cue sample encoding features to obtain a first combined decoding feature output by at least one level of combined decoding processing, and a normal sample reconstruction feature for the normal sample image is obtained based on the first combined decoding feature; the image anomaly detection model to be trained is subjected to at least one level of combined decoding processing based on the anomaly sample encoding features and the cue sample encoding features to obtain a second combined decoding feature output by at least one level of combined decoding processing, and anomaly sample recovery feature for the anomaly sample image is obtained based on the second combined decoding feature; the image anomaly detection model to be trained is updated based on the normal sample reconstruction feature, the anomaly sample recovery feature and the original normal image feature and then trained until training is completed to obtain the trained image anomaly detection model.

30. The apparatus according to claim 29, characterized in that, The model training module is further configured to: determine the sample reconstruction loss based on the sample reconstruction difference between the normal sample reconstruction features and the original features of the normal image; determine the sample recovery loss based on the sample recovery difference between the abnormal sample recovery features and the original features of the normal image; update the image anomaly detection model to be trained based on the sample reconstruction loss and the sample recovery loss, and continue training until training is completed, thereby obtaining the trained image anomaly detection model.

31. The apparatus according to claim 30, characterized in that, The model training module is further configured to: determine a first feature difference between the reconstructed features of the normal sample and the original features of the normal image, and determine a second feature difference between the restored features of the abnormal sample and the original features of the abnormal image; and through the image anomaly detection model to be trained, perform at least one level of pixel feature extraction for the first feature difference and the second feature difference respectively to obtain a first pixel anomaly result and a second pixel anomaly result. The pixel anomaly detection loss is determined based on the abnormal region label carried by the abnormal sample image, the first pixel anomaly result, and the second pixel anomaly result; the sample reconstruction loss, the sample recovery loss, and the pixel anomaly detection loss are fused to obtain the model target loss; the image anomaly detection model to be trained is updated based on the model target loss and then trained until training is completed to obtain the trained image anomaly detection model.

32. An image anomaly detection model processing device, characterized in that, The device includes: The sample image acquisition module is used to acquire normal sample images, reference sample images associated with the normal sample images, and abnormal sample images; The sample feature extraction module is used to extract features from the normal sample image, the reference sample image, and the abnormal sample image respectively, to obtain the original normal image features of the normal sample image, the original reference image features of the reference sample image, and the original abnormal image features of the abnormal sample image. The sample feature encoding module is used to encode the original features of the normal image, the original features of the reference image, and the original features of the abnormal image using the image anomaly detection model to be trained, respectively, to obtain the normal sample encoding features of the normal sample image, the prompt sample encoding features of the reference sample image, and the abnormal sample encoding features of the abnormal sample image. The normal sample reconstruction module is used to perform at least one level of combined decoding processing based on the normal sample encoding features and the prompt sample encoding features through the image anomaly detection model to be trained, to obtain the first combined decoding feature output by the at least one level of combined decoding processing, and to obtain the normal sample reconstruction feature for the normal sample image based on the first combined decoding feature. An abnormal sample recovery module is used to perform at least one level of combined decoding processing based on the abnormal sample encoding features and the prompt sample encoding features through the image anomaly detection model to be trained, to obtain a second combined decoding feature output by the at least one level of combined decoding processing, and to obtain an abnormal sample recovery feature for the abnormal sample image based on the second combined decoding feature. The model update module is used to update the image anomaly detection model to be trained based on the normal sample reconstruction features, the abnormal sample recovery features, and the normal image original features, and then continue training until training is completed to obtain a trained image anomaly detection model; the trained image anomaly detection model is used to perform image anomaly detection on the input image.

33. The apparatus according to claim 32, characterized in that, The model update module is further configured to: determine the sample reconstruction loss based on the sample reconstruction difference between the reconstructed features of normal samples and the original features of normal images; determine the sample recovery loss based on the sample recovery difference between the recovered features of abnormal samples and the original features of normal images; update the image anomaly detection model to be trained based on the sample reconstruction loss and the sample recovery loss, and continue training until training is completed to obtain the trained image anomaly detection model.

34. The apparatus according to claim 32, characterized in that, The model update module is further configured to: determine the first feature difference between the reconstructed features of normal samples and the original features of normal images, and the second feature difference between the restored features of abnormal samples and the original features of abnormal images; extract pixel features at least one level for the first feature difference and the second feature difference using the image anomaly detection model to be trained, to obtain the first pixel anomaly result and the second pixel anomaly result; determine the pixel anomaly detection loss based on the abnormal region label carried by the abnormal sample image, the first pixel anomaly result, and the second pixel anomaly result; fuse the sample reconstruction loss, the sample restoration loss, and the pixel anomaly detection loss to obtain the model target loss; update the image anomaly detection model to be trained based on the model target loss and continue training until training is completed, to obtain the trained image anomaly detection model.

35. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 17.

36. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 17.

37. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 17.

Images