Multi-mode prompt memory unsupervised continuous anomaly detection method and system

Abstract

The invention discloses an unsupervised continuous anomaly detection method and system for multi-mode prompt memory, and relates to the technical field of computer vision and industrial image detection. The method comprises the following steps: acquiring industrial images to form a data set; an unsupervised anomaly detection model is constructed, the unsupervised anomaly detection model comprises a visual branch and a text branch which are respectively used for extracting visual features and text features, learnable visual prompts and learnable text prompts are introduced into the unsupervised anomaly detection model, and then the visual features and the text features are fused by using a self-adaptive fusion mechanism to obtain an anomaly detection result; training learnable visual prompts and learnable text prompts in the unsupervised anomaly detection model by using the data set; and performing anomaly detection on the industrial data by using the trained unsupervised anomaly detection model. According to the method, multi-modal information can be fused in the learning process, the continuous learning ability is achieved, and efficient unsupervised anomaly detection of industrial products is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and industrial image detection, and in particular to an unsupervised continuous anomaly detection method and system with multimodal prompt memory. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Early anomaly detection methods for industrial products primarily relied on supervised learning, which involves training a large number of labeled normal and abnormal samples to build a classification model. However, in real-world industrial production scenarios, large-scale labeled data is difficult to obtain. Therefore, unsupervised anomaly detection, a learning paradigm that does not rely on labeled data, emerged and garnered widespread attention. Unsupervised anomaly detection learns the characteristic distribution of normal data and classifies data that deviates from the normal pattern as anomalies, thus avoiding the reliance on large amounts of labeled data.

[0004] However, existing unsupervised anomaly detection methods have numerous limitations. For example, the computational burden of the one-to-one anomaly detection paradigm increases with the number of categories, and unified models are prone to catastrophic forgetting when products change frequently. These limitations limit their effectiveness in practical applications. Most mainstream methods rely on single-modal data features, such as using only visual information for anomaly detection. However, in complex industrial scenarios, single-modal features often fail to fully capture the diversity and complexity of normal patterns. For example, when detecting surface defects, relying solely on visual features may not accurately distinguish between normal texture changes and true defects, resulting in a high false positive rate. Multimodal data (such as text descriptions and depth information) typically contains richer information, describing objects from different perspectives, helping to more accurately characterize normal patterns and identify anomalies. However, multimodal data represents more information, and due to the lack of label guidance in unsupervised learning, it is more susceptible to noise, thereby reducing sensitivity. Furthermore, concatenating multimodal data during unsupervised learning exacerbates the curse of dimensionality, affecting the accuracy and efficiency of detection results. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide an unsupervised continuous anomaly detection method and system with multimodal prompt memory, which can integrate multimodal information in the learning process and has the ability of continuous learning, thereby realizing efficient unsupervised anomaly detection of industrial products.

[0006] In order to achieve the above object, the present invention is implemented through the following technical solutions: A first aspect of the present invention provides an unsupervised continuous anomaly detection method for multimodal cue memory, comprising the following steps: Obtain industrial image datasets and preprocess the industrial images; An unsupervised anomaly detection model is constructed, including a visual branch and a text branch, which are used to extract visual features and text features respectively. Learnable visual cues and text cues are introduced into the unsupervised anomaly detection model. Visual cues are used to guide visual feature extraction, while text cues are used to guide text feature extraction. An adaptive fusion mechanism is then used to fuse visual and text features to obtain anomaly detection results. Leveraging the dataset to train learnable visual cues and learnable textual cues in unsupervised anomaly detection models; Use the trained unsupervised anomaly detection model to perform anomaly detection on industrial data.

[0007] A second aspect of the present invention provides an unsupervised continuous anomaly detection system with multimodal cue memory, comprising: A data acquisition module is configured to acquire industrial images to form a data set and pre-process the industrial images; a model building module configured to build an unsupervised anomaly detection model, including a visual branch and a text branch for extracting visual features and text features, respectively; introducing learnable visual cues and learnable text cues into the unsupervised anomaly detection model; using visual cues to guide visual feature extraction; using text cues to guide text feature extraction; and then using an adaptive fusion mechanism to fuse visual features and text features to obtain anomaly detection results; a model training module configured to train a learnable visual cue and a learnable textual cue in an unsupervised anomaly detection model using the dataset; The anomaly detection module is configured to perform anomaly detection on industrial data using the trained unsupervised anomaly detection model.

[0008] A third aspect of the present invention provides a computer-readable storage medium storing a computer program, which is suitable for being loaded by a processor and executing the steps in the unsupervised continuous anomaly detection method for multimodal cue memory as described in the first aspect of the present invention.

[0009] A fourth aspect of the present invention provides a computer device, comprising: a processor adapted to execute a computer program; A computer-readable storage medium having a computer program stored therein, wherein when the computer program is executed by the processor, the method for unsupervised continuous anomaly detection of multimodal prompt memory as described in the first aspect of the present invention is implemented.

[0010] One or more of the above technical solutions have the following beneficial effects: The present invention discloses an unsupervised continuous anomaly detection method and system with multimodal prompt memory. First, the industrial image data is preprocessed and divided into a training set and a test set. Then, the pseudo-abnormal image generation and refinement feature prompt module, text prompt design and learnable text prompt module, continuous multimodal prompt memory library and anomaly detection model defect semantic guidance adaptive fusion mechanism are built. The training set is used to train the overall framework, and the test set is used for inference. In terms of multimodal learning, attempts are made to combine multiple modal data such as text and images, and improve the performance of anomaly detection by cross-modal feature alignment and fusion. In terms of continuous learning, the catastrophic forgetting problem is alleviated by introducing a continuous learning framework, structural contrast learning and other methods. Therefore, the present invention effectively avoids the dependence on a large amount of labeled data and the problem of catastrophic forgetting in multiple scenarios, and provides an unsupervised continuous industrial image anomaly detection framework model.

[0011] In the process of processing existing multimodal data Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] Figure 1 This is a flow chart of the unsupervised continuous anomaly detection method for multimodal cue memory in Example 1 of the present invention; Figure 2 This is a structural diagram of the unsupervised anomaly detection model framework in Example 1 of the present invention; Figure 3 This is a structural diagram of a learnable text prompt module in Example 1 of the present invention; Figure 4 This is a structural diagram of a continuous multimodal prompt memory bank in Example 1 of the present invention; Figure 5 This is a structural diagram of the defect semantics-guided adaptive fusion mechanism in Example 1 of the present invention. DETAILED DESCRIPTION

[0014] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0015] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or their combinations; The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0016] Example 1: The first embodiment of the present invention provides an unsupervised continuous anomaly detection method based on multimodal prompt memory, such as Figure 1 and Figure 2 As shown, the following steps are included: S1: Obtain industrial image datasets and preprocess the industrial images.

[0017] In a specific implementation, the preprocessed data set is divided into a training set and a test set. The unsupervised anomaly detection model is trained using the training set, and the unsupervised anomaly detection model is tested using the test set. The training set only contains normal images of industrial images. To meet the scenario setting of unsupervised anomaly detection, the industrial image categories in the test set are consistent with the industrial image categories in the training set, and the test set contains various types of abnormal images and normal images of industrial images.

[0018] First, these industrial images Perform preprocessing operations. Use the following formula to convert industrial images: .

[0019] in, represents the transformed dataset, It means unifying all industrial images to the same size. represents the image dimension, It is the scaled size, indicating the height and width of the image. Represents the size of the image to be cropped from the center of the image. is the target size. and represents the mean and variance. It represents the linear mapping of the image value range from [0, 255] to [0.0, 1.0], which becomes a tensor that can be input to the neural network. The training set is recorded as , the test set is denoted as Contains normal images and abnormal images, represents a normal image, Indicates an abnormal image.

[0020] S2: Build an unsupervised anomaly detection model, including a visual branch and a text branch, which are used to extract visual features and text features respectively, and introduce learnable visual cues and learnable text cues into the unsupervised anomaly detection model.

[0021] S21: Preliminary construction of unsupervised anomaly detection model.

[0022] S211: Generate an abnormal image by adding noise using a pseudo abnormal image data generation module.

[0023] In a specific embodiment, in order to increase the diversity of the training set, the model is exposed to more different types of abnormal situations and learns a wider range of feature representations, thereby improving its generalization ability in unseen abnormal situations. A pseudo abnormal image data generation module is constructed in the unsupervised anomaly detection model. For the images in the given training set, Each channel performs the following operations: , .

[0024] in, represents the image processed by the pseudo abnormal image data generation module, is the set of real numbers, is the pixel coordinate of the image, represents Gaussian noise, and . represents the noise mask. , indicating that noise is added at this position; Indicates no noise is added.

[0025] S212: Construct a visual branch and a text branch, and use the visual branch and the text branch to extract visual features and text features respectively.

[0026] In a specific embodiment, the unsupervised anomaly detection model uses a dual-branch collaborative reasoning method to extract visual features and text features, and then combines an adaptive fusion mechanism with a dynamic normalization strategy to effectively fuse visual and text features. The dual branches include a visual branch and a text branch, both of which are composed of a pre-trained ViT model. The input of the visual branch and the input of the text branch both include normal image data, noisy image data, and text data, wherein the normal image data comes from the training set, the noisy image data comes from the pseudo-anomaly image data generation module, and the text data is generated using a text semantic template.

[0027] In the visual branch, we use pre-trained visual transformers As the visual backbone network, it extracts multi-level features, among which the fifth-layer features are used for task identification and anomaly detection due to their characteristics of balancing semantics and detail information. Specifically, the task identifier is first constructed by sampling the farthest point. and visual features Calculate the nearest neighbor distance to achieve fast matching of task identities to obtain normal features ; Subsequently, the extracted visual features are compared with the normal features stored in the multimodal cue memory bank Perform block-wise nearest neighbor distance calculation to generate preliminary block-level anomaly scores : .

[0028] in, is a normal feature of the multimodal cue memory bank, for The corresponding category.

[0029] In the text branch, Layer 5 Features Its rich local detail features are extracted and combined with learnable text prompts The embedding vector is used to perform cross-modal cosine similarity calculation to obtain the text-guided anomaly score. : .

[0030] in, is the local weight of the feature, i represents the spatial position index in the image, j represents the word or token index in the text prompt, is the dot product of the vectors, is the Euclidean norm of the vector.

[0031] S22: Introducing learnable visual cues and learnable textual cues in unsupervised anomaly detection models.

[0032] Among them, visual cues are used to guide visual feature extraction, text cues are used to guide text feature extraction, and then an adaptive fusion mechanism is used to fuse visual features and text features to obtain anomaly detection results.

[0033] S221: Construct a refined feature cue module and introduce hierarchically refined learnable visual cues to guide unsupervised anomaly detection of visual features.

[0034] In a specific embodiment, a hierarchical and refined visual cue is introduced. To guide unsupervised anomaly detection. Task-aware visual cues Depend on It is composed of learnable vectors of dimension l×C, where represents the cue length, and C represents the channel dimension. These visual cues are uniformly distributed ( ) to initialize: .

[0035] During the task adaptation and reasoning phases, visual cues are Integrate it into the input features of each layer of the pre-trained visual branch ViT hierarchical model for prefix tuning. This process realizes the layer-by-layer propagation of task-specific information, which can be formally expressed as: .

[0036] in, Indicates the backbone network The output features of the layer, Indicates from Layer characteristics, Indicates that it is specifically for The visual cues learned by the layer are used to promote fine-grained task adaptation, and t represents task t.

[0037] S222: Introducing learnable textual cues to guide unsupervised anomaly detection of text features.

[0038] S2221: Build a text prompt design module and introduce text semantic templates containing learnable text prompts.

[0039] In a specific embodiment, Figure 3 As shown, a learnable text semantic template S is introduced, which is in the form of: S = "Aphotoofa[class]with[P]".

[0040] Among them, [class] represents a specific category label (such as bottle), and [P] is a text prompt composed of N learnable vectors, which is recorded as: .

[0041] Set N = 5, and initialize all vectors to standard normal distribution, which is used to dynamically model the semantic feature distribution of normal samples. The text feature extraction module, i.e., the backbone for extracting text features in the text branch, mainly comes from the pre-trained CLIP text encoder. The output of the text encoder is recorded as : .

[0042] in is a learnable text semantic hint template, is a pre-trained text encoder.

[0043] S2222: Construct a learnable text prompt module to adaptively optimize the learnable text prompt by fusing the inherent semantic information of the image.

[0044] In a specific embodiment, in order to solve the problem that the artificially designed text prompt templates have limitations in describing the normal or abnormal information of industrial images, and the text prompts do not take into account the actual image semantic information, this embodiment adaptively optimizes the learnable text prompts by integrating the inherent semantic information of the image, thereby achieving more efficient anomaly segmentation and detection capabilities. Specifically, in the task adaptation stage, the learnable text prompts are optimized by contrastive learning using noise-enhanced pseudo-anomaly samples. The process is as follows: First, the image of the training set in step S211 Image features obtained by pre-trained visual encoder Capture information and perform image semantic self-attention learning. The formula is expressed as: .

[0045] in, represents the image features after self-attention learning, represents a linear layer, is the activation function.

[0046] Secondly, the text prompt features obtained by the pre-trained text encoder First, self-optimization learning is performed, and the formula is expressed as: .

[0047] in, Represents the text prompt features after self-optimization learning.

[0048] Ultimately, and After the dimension alignment operation, the element-by-element weighted fusion operation is performed: .

[0049] in, For optimized learnable text prompts, and Represented as the dimension change and dimension expansion operation of the feature vector, Represented as an element-wise multiplication operation.

[0050] S23: Integrate learnable textual cues and refined visual cues to build a continuous multimodal cue memory bank.

[0051] In a specific implementation, although the traditional multi-category anomaly detection method can identify multiple categories at the same time, it requires all categories to be available for training at one time, which is impractical in real applications. In order to solve the problem of unsupervised multimodal anomaly detection in a continuous learning scenario, this embodiment combines learnable text prompts and refined visual prompts to build a continuous multimodal prompt memory library. Figure 4 As shown, through the four-tuple structure Fusion of task identifiers (K), learnable textual hints (T), refined visual hints (V) and normal feature storage (N) to achieve multimodal feature collaborative optimization and efficient knowledge accumulation. For sequentially arriving tasks t, a quadruple is constructed for it. to dynamically adapt to different tasks while guiding the model to perform unsupervised anomaly detection.

[0052] During the task adaptation phase, the task identifier Use farthest point sampling ) Filter key embeddings from the block-level features extracted from the frozen pre-trained vision branch ViT hierarchical model. This process builds low-dimensional, highly discriminative task representations and stores them in the multimodal cue memory. , for task-independent prediction. Since the task identity may not always be available in real-world scenarios, task-agnostic prediction is essential. For the training set of task t ,in is the number of images in task t, Indicates the size of a single image. Since the task identity contains both text details and high-level information, this embodiment uses ViT extraction of a specific layer instead of the last embedding to extract the identity information related to the task: .

[0053] in, is the number of image patch features, Represents the dimension of image features, The features extracted from ViT of layer j, j is selected as 5. Define the task set , then the identity of the task identifier K can be expressed as: .

[0054] in, is the number of tasks, is the farthest point sampling, for t represents the task t, n is the length of the identity representation, 、 They respectively represent the identity of the current task and the identities of each task stored in the multimodal cue memory bank.

[0055] In the task reasoning stage, for a test image , extracting identity information through a frozen pre-trained vision branch ViT hierarchical model , and calculate With task identifier The highest similarity is used to identify the task identity. In addition, this embodiment uses a fine-tuned pre-trained visual branch ViT hierarchical model to extract normal features and form a normal feature library. Similar to the task identification K, core set sampling (CSS) is used to compress the extracted image block level features to obtain the final normal feature library , used to calculate the abnormal score by extracting the normal features and storing the normal features F in the multimodal prompt memory library during the nearest neighbor search. Define the normal feature set ,in is the number of normal image block features. It can be expressed as: , , .

[0056] in, For core set sampling, Normal characteristics The number of image patch features in the core set, Normal features extracted for the task adaptation phase The core set of the space. Through continuous iterative optimization of multimodal prompts, efficient task knowledge accumulation and transfer are achieved. Text prompts focus on modeling normal patterns at the semantic level, while visual prompts enhance local feature adaptation. The two share task identifiers to form complementary feature representations, enabling cross-modal interaction.

[0057] S24: Enhancing segmentation performance in anomaly detection using an adaptive fusion mechanism guided by defect semantics.

[0058] In a specific embodiment, Figure 5 As shown in the figure, in order to fully utilize the complementary advantages of multimodal information, this embodiment proposes an adaptive fusion mechanism based on defect semantics to enhance the segmentation performance in anomaly detection. In order to enhance the robustness of the anomaly score, a dynamic Sigmoid normalization mechanism is designed. The core of this mechanism is to use hyperparameters k and b to control the steepness and center position of the Sigmoid function to achieve the optimization of the score. Among them, k is set to a fixed value of 2, because searching a and b at the same time will bring a large computational burden. Then for the hyperparameters , parameter is realized through candidate value search strategy Adaptive update to adapt to different tasks. Specifically, define the discrete step length set , in each training iteration, based on the current Generate a set of candidate values .in, is the step length, represents a set of discrete steps. Then, using each candidate After normalization, the anomaly score is calculated and the best candidate is selected by evaluating the short-term detection performance on the validation set. , and its update rules are as follows: .

[0059] in, Indicates the current candidate Corresponding validation set performance indicators.

[0060] Furthermore, the improved normalization function Defined as: .

[0061] in, represents the preliminary anomaly score.

[0062] By dynamically searching the center position of the function, it can adaptively compress the normal sample score interval and amplify the response difference of the abnormal area, thereby achieving more accurate abnormality positioning. Finally, the normalized and Upsample to the same resolution and generate memory-based anomaly maps Aligning anomaly maps with multimodal Final anomaly score map Generated through the following dynamic fusion strategy: .

[0063] in, Represents the fusion coefficient, which is used to control the fusion of the anomaly score maps of the vision branch and the text branch.

[0064] S3: Use the dataset to train learnable visual cues and learnable textual cues in unsupervised anomaly detection models.

[0065] like Figure 2 As shown in the figure, the anomaly detection model framework is built using a pseudo-abnormal image generation module, a refined feature prompt module, a text prompt design module, a learnable text prompt module, a continuous multimodal prompt memory library, and a defect semantic-guided adaptive fusion mechanism, and the training dataset is input into the built anomaly detection model for training.

[0066] First, the learnable text hints are optimized using mean squared error as the loss function: .

[0067] in, represents the mean square error loss function, represents the output of the model, Indicates the corresponding label value, This method enables the model to align the feature distribution of normal samples with textual prompts without manual annotation, which is conducive to the learning of normal patterns.

[0068] Secondly, the introduction of structured contrast loss further improves the compactness of visual features in the continuous multimodal cue memory library to better compress normal features and save storage space. The structured contrast loss function The formula is: , , .

[0069] In a given loss function, Representation characteristics The embedding at position (i, j), ,and Represents the feature corresponding to this position in the segmentation result generated by the pre-trained Segment Anything Model (SAM) Tags, , (p, q) represents the two-dimensional position coordinates of the feature map. and The value of is 1. Through continuous iterative optimization of multimodal prompts, effective task knowledge accumulation and transfer are achieved, which improves the robustness and cross-task generalization ability of unsupervised anomaly detection.

[0070] S4: Use the trained unsupervised anomaly detection model to perform anomaly detection on industrial data.

[0071] Example 2: A second embodiment of the present invention provides an unsupervised continuous anomaly detection system with multimodal prompt memory, including: A data acquisition module is configured to acquire industrial images to form a data set and pre-process the industrial images; a model building module configured to build an unsupervised anomaly detection model, including a visual branch and a text branch for extracting visual features and text features, respectively; introducing learnable visual cues and learnable text cues into the unsupervised anomaly detection model; using visual cues to guide visual feature extraction; using text cues to guide text feature extraction; and then using an adaptive fusion mechanism to fuse visual features and text features to obtain anomaly detection results; a model training module configured to train a learnable visual cue and a learnable textual cue in an unsupervised anomaly detection model using the dataset; The anomaly detection module is configured to perform anomaly detection on industrial data using the trained unsupervised anomaly detection model.

[0072] Example 3: Embodiment 3 of the present invention provides a computer-readable storage medium storing a computer program, which is suitable for being loaded by a processor and executing the steps of the unsupervised continuous anomaly detection method for multimodal prompt memory as described in embodiment 1 of the present invention.

[0073] Example 4: A fourth embodiment of the present invention provides a computer device, comprising: a processor adapted to execute a computer program; A computer-readable storage medium having a computer program stored therein, wherein when the computer program is executed by the processor, the steps of the unsupervised continuous anomaly detection method for multimodal cue memory as described in the first embodiment of the present invention are implemented.

[0074] The steps involved in the above embodiments 2, 3 and 4 correspond to those in the method embodiment 1. For the specific implementation methods, please refer to the relevant description part of the embodiment 1.

[0075] Those skilled in the art will appreciate that the units and algorithmic steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data processing device such as a server or data center that integrates one or more available media. Available media can include magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)). The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any technical object of a person skilled in the art that can be easily conceived of within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. An unsupervised continuous anomaly detection method with multimodal cue memory, characterized in that: The following steps are involved: Obtain industrial image datasets and preprocess the industrial images; An unsupervised anomaly detection model is constructed, including a visual branch and a text branch, which are used to extract visual features and text features respectively. Learnable visual cues and text cues are introduced into the unsupervised anomaly detection model. Visual cues are used to guide visual feature extraction, while text cues are used to guide text feature extraction. An adaptive fusion mechanism is then used to fuse visual and text features to obtain anomaly detection results. Leveraging the dataset to train learnable visual cues and learnable textual cues in unsupervised anomaly detection models; Use the trained unsupervised anomaly detection model to perform anomaly detection on industrial data.

2. The unsupervised continuous anomaly detection method for multimodal cue memory according to claim 1, characterized in that: The preprocessed dataset is divided into a training set and a test set. The training set is used to train the unsupervised anomaly detection model, and the test set is used to test the unsupervised anomaly detection model. The training set only contains normal images of industrial images. To meet the scenario setting of unsupervised anomaly detection, the industrial image categories in the test set are consistent with those in the training set. The test set contains various types of abnormal images and normal images of industrial images.

3. The unsupervised continuous anomaly detection method for multimodal cue memory according to claim 1, characterized in that: The specific steps to build an unsupervised anomaly detection model are: Initially build an unsupervised anomaly detection model; Introducing learnable visual cues and learnable textual cues into unsupervised anomaly detection models; Integrate learnable textual cues with refined visual cues to build a continuous multimodal cue memory bank; Enhance segmentation performance in anomaly detection using an adaptive fusion mechanism guided by defect semantics.

4. The unsupervised continuous anomaly detection method for multimodal cue memory according to claim 3, characterized in that: The specific steps for initially building an unsupervised anomaly detection model are: Generate abnormal images by adding noise using a pseudo abnormal image data generation module; Construct a visual branch and a text branch, and use the visual branch and text branch to extract visual features and text features respectively.

5. The unsupervised continuous anomaly detection method for multimodal cue memory according to claim 4, characterized in that: Both the visual branch and the text branch are composed of the pre-trained ViT model.

6. The unsupervised continuous anomaly detection method for multimodal cue memory according to claim 3, characterized in that: The specific steps for introducing learnable visual cues and learnable textual cues into the unsupervised anomaly detection model are: Construct a refined feature cue module, introducing hierarchically refined learnable visual cues to guide unsupervised anomaly detection using visual features; Learnable text prompts are introduced to guide unsupervised anomaly detection of text features. Specifically, a text prompt design module is constructed, a text semantic template containing learnable text prompts is introduced, and a learnable text prompt module is constructed. The learnable text prompts are adaptively optimized by fusing the inherent semantic information of the image.

7. The unsupervised continuous anomaly detection method for multimodal cue memory according to claim 1, characterized in that: The specific steps for training the learnable visual cues and learnable textual cues in the unsupervised anomaly detection model using the dataset are: Optimize learnable textual cues using mean squared error as the loss function; Structured contrast loss is introduced to further improve the compactness of visual features in the persistent multimodal cue memory bank.

8. An unsupervised continuous anomaly detection system with multimodal cue memory, characterized in that: include: A data acquisition module is configured to acquire industrial images to form a data set and pre-process the industrial images; a model building module configured to build an unsupervised anomaly detection model, including a visual branch and a text branch for extracting visual features and text features, respectively; introducing learnable visual cues and learnable text cues into the unsupervised anomaly detection model; using visual cues to guide visual feature extraction; using text cues to guide text feature extraction; and then using an adaptive fusion mechanism to fuse visual features and text features to obtain anomaly detection results; a model training module configured to train a learnable visual cue and a learnable textual cue in an unsupervised anomaly detection model using the dataset; The anomaly detection module is configured to perform anomaly detection on industrial data using the trained unsupervised anomaly detection model.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which is suitable for being loaded by a processor and executing the unsupervised continuous anomaly detection method for multimodal cue memory according to any one of claims 1 to 7.

10. A computer device, characterized in that: include: a processor adapted to execute a computer program; A computer-readable storage medium having a computer program stored therein, wherein the computer program, when executed by the processor, implements the unsupervised continuous anomaly detection method for multimodal cue memory according to any one of claims 1 to 7.

Images