A few-shot defect identification method driven by cross-modal text semantics

Abstract

This invention discloses a few-sample defect recognition method based on cross-modal text semantics, belonging to the field of image processing technology. Addressing the problem of insufficient generalization of detection models caused by the scarcity of abnormal samples and the dynamic evolution of defect types in industrial quality inspection scenarios, this invention establishes a dynamic feature recombination mechanism and adaptive discrimination boundaries to accurately identify unknown defect types with only a small amount of normal data. By matching text descriptions with image features, this invention generates simulated samples that closely resemble the morphology of real defects on normal samples. When encountering unseen defect types, it automatically adjusts the image texture contrast standard based on text semantics, accurately distinguishing subtle differences between normal and abnormal areas without relying on real defect data, thereby improving the accuracy of sample defect recognition.

Description

Technical Field

[0001] This invention relates to image processing technology, and in particular to a few-sample defect identification method based on cross-modal text semantics. Background Technology

[0002] Image anomaly detection is the process of identifying abnormal regions or defective targets in an image that deviate from the normal pattern. It has wide applications in many fields, such as detecting product defects in industrial production, identifying disease symptoms in medical imaging, and monitoring machine behavior to provide early warning of malfunctions. In industrial quality inspection scenarios, this technology can accurately locate defects such as surface cracks and coating peeling by analyzing the surface texture and structural features of products in real time.

[0003] Traditional methods for training anomaly detection models typically involve training a classifier model on labeled datasets containing both normal and anomalous images. On the one hand, the dynamic changes in defect types in industrial scenarios result in insufficient timeliness of labeled datasets; on the other hand, a large amount of labeled data is required for effective model training. Due to the dynamic and evolving nature of anomalies, anomaly detection models can quickly become obsolete. Traditional learning models often require extensive retraining and updates to adapt to these changes, which is both time-consuming and labor-intensive. In contrast, few-shot learning models aim to learn new patterns from minimal or no labeled examples, allowing for faster adaptation to new contexts and situations, and significantly reducing costs and manual intervention.

[0004] In few-shot anomaly detection tasks, the lack of data for the target scene is a core challenge. Due to the unpredictability of real-world scenarios, directly relying on publicly available data to train models often fails to adapt to specific detection needs. In conventional methods, unoptimized new models are prone to overfitting general data features, leading to severe performance degradation in unknown target domains. This phenomenon stems from the model's lack of implicit understanding of the target scene, making it unable to extract cross-scene discrimination rules from basic data. Therefore, building a pre-trained architecture with strong transfer capabilities has become a key breakthrough. The core contradiction that current technology urgently needs to address lies in: how to guide models to establish general anomaly representations through public data resources while maintaining flexible adaptability to unknown scenarios, ultimately achieving accurate detection and localization with limited target domain data support. Summary of the Invention

[0005] Purpose of the invention: To address the above problems, the purpose of this invention is to provide a few-sample defect identification method based on cross-modal text semantics. By analyzing the inherent relationship between text description and visual features, an anomaly judgment mechanism without labeled data is established, which supports rapid model generation under few-sample conditions, significantly reduces data dependence, and improves the generalization ability to unknown anomaly types in dynamic scenes.

[0006] Technical solution: This invention is based on a few-shot defect identification method driven by cross-modal text semantics, including a training phase and a testing phase;

[0007] During the training phase:

[0008] Obtain the global and local features of a normal image, and store the local features of the normal image into a reference library;

[0009] Construct normal prefixes and abnormal introductory words, generate a normal description set based on the normal prefixes, generate an abnormal description set based on the abnormal introductory words, and perform feature extraction to obtain a normal description feature set and an abnormal description feature set;

[0010] The global features of a normal image are combined with normal descriptive features and abnormal descriptive features respectively to generate normal feature vectors and abnormal feature vectors.

[0011] Synthetic anomalous images are obtained using anomalous feature vectors;

[0012] Normal images and synthetic anomalous images are labeled, and a training set is constructed using normal images and synthetic anomalous images. A binary classification model is then trained using the training set.

[0013] During the testing phase:

[0014] A test set was constructed using normal images and defective images;

[0015] Select any image from the test set as the image to be detected, and extract the global and local image features of the image to be detected;

[0016] The distances between the normal feature vector and the abnormal feature vector and the global image features of the image to be detected are calculated using the nearest neighbor algorithm, and an image-level anomaly score is generated based on the distance difference.

[0017] The image to be detected is input into the trained binary classification model to obtain the prediction result;

[0018] The image-level anomaly score is obtained by combining the image-level anomaly score and the prediction result. The image-level anomaly score is then compared with a preset threshold to obtain the judgment result.

[0019] Furthermore, the steps for obtaining global and local features of a normal image include:

[0020] The preprocessed normal image is input into the image encoder, where it is divided into pixel block sequences via a block embedding layer. Global features are then extracted using a multi-head self-attention mechanism. and local features , where c represents the number of image blocks after the spatial image is divided, and d represents the feature dimension of each image block encoded in the visual language model Clip. The local feature maps of normal images are stored in the reference library S. The local feature maps of normal images in the reference library are used to calculate the cosine similarity with the local feature maps of the query image during the testing phase.

[0021] Furthermore, the steps of generating a normal description set based on normal prefixes and an abnormal description set based on abnormal introductory words include:

[0022] Let the normal prefix be denoted as , This represents the total number of evolutionary normal prefixes. Indicates one of the evolutionary normal prefixes;

[0023] Normal prefix prompts The target object name is fused with the target object name using a bidirectional attention mechanism, resulting in the following structure: From the normal semantic templates, N normal description templates are formed, among which... Indicates the target object category name, attention mechanism Using the scaled dot product method, This represents the concatenation operation of vector sequences;

[0024] Let the abnormality introductory word be denoted as , This indicates the total number of abnormal introductory words. This indicates one of the abnormal introductory words;

[0025] Category Name By concatenating these elements after each exception introductory word, a set of specific exception text descriptions is obtained, represented as follows: The template-type exception description set is ,form There are 10 abnormal semantic templates, where M represents the total number of categories of the corresponding labeled defects;

[0026] Through dynamic weight fusion module abnormal guide word sequence With normal semantic templates Association, construction form is Evolvable anomalous templates, forming There are several evolvable anomalous templates, among which... Indicates bilinear feature interaction. To control the size of the evolvable template using hyperparameters;

[0027] Will include A set of normal description templates { } as a normal description set, An abnormal semantic template and A set of evolvable anomalous templates As a set of exception descriptions.

[0028] Furthermore, the steps of extracting features to obtain the normal description feature set and the abnormal description feature set include:

[0029] The normal description set { The input is fed into the text encoder to obtain the normal descriptive feature set, represented as... ;in The text encoder representing the Clip visual language model;

[0030] Set of exception descriptions The input is fed into a text encoder to obtain a set of anomaly description features, represented as follows: .

[0031] Furthermore, the normal description feature set and the abnormal description feature set are optimized, and the process includes:

[0032] global features Normal descriptive feature set and abnormal description feature set The input is fed into the control boundary module to calculate the mean of the normal descriptive features, using the following formula:

[0033] ,

[0034] In the formula, This indicates a normal description template index;

[0035] Construct the feature similarity loss function, the formula is:

[0036] ,

[0037] In the formula, This represents the natural exponential function. For temperature coefficient, This represents the negative normalized dot product similarity. This represents a feature vector in the template-type anomaly description features. This represents a feature vector in an evolutionary anomalous template; the loss... Force global features of normal images Compared with the mean of normal descriptive features The similarity is higher than the similarity with the anomaly description features;

[0038] Implement multi-prototype space constraints and calculate the mean of anomaly descriptive features using the following formula:

[0039] ,

[0040] The formula for calculating the center of the artificial anomaly template is: ,

[0041] The formula for calculating the center of an evolutionary anomaly template is: ,

[0042] Construct a dual distance loss function, the formula of which is:

[0043] ,

[0044] in, Representing global features Expected value Represents the square of the Euclidean distance. This is the distance to the boundary threshold. The first term constrains the global features of the normal image, representing the distribution alignment intensity coefficients. distance Euclidean distance ratio Nearly at least The unit, the second term reduces the distribution difference between the two types of abnormal prototypes;

[0045] The overall optimization objective is After updating the evolvable parameters using gradient descent, the optimized normal description set is obtained. }, and by An abnormal semantic template and A set of anomaly descriptions consisting of evolvable anomaly templates The mean of the normal descriptive features and the mean of the anomaly description features Store it in reference library S.

[0046] Furthermore, the steps of combining the global features of a normal image with normal descriptive features and abnormal descriptive features respectively to generate normal feature vectors and abnormal feature vectors include:

[0047] global features Through linear projection layer Mapped to This aligns the dimensions of global features with those of normal and abnormal descriptive features.

[0048] For each normal descriptive feature Perform cross-modal fusion: using a gating mechanism Generate normal feature vectors The formula is:

[0049] ,

[0050] In the formula, Represents element-wise product. The matrix represents the evolutionary parameters. Activated for Sigmoid;

[0051] Describe the features of each anomaly Calculate cross-modal attention gating weights , Evolvable parameters; hybrid features are generated through gating weighting. All After being stitched along the channel dimension and passed through a fully connected layer The projection yields the final anomalous feature vector. The formula is:

[0052] ;

[0053] Storing normal feature vectors and abnormal feature vectors To the reference library .

[0054] Furthermore, the steps for obtaining a synthesized anomalous image using anomalous feature vectors include:

[0055] Utilizing anomaly feature vectors Constructing an abnormal feature vector set ,Will The input is fed into the decoding module of a pre-trained text-image generator to obtain a tensor. , This represents a pre-trained image-text generator that generates images based on text features. H represents the number of normal samples in the training dataset, H represents the height of the final synthetic anomalous image output by the generator, and W represents the width of the final synthetic anomalous image output by the generator.

[0056] When the network reaches the final layer, the pixel value range of the tensor z is constrained by the hyperbolic tangent function, compressing the original output to... An interval is represented as:

[0057] ,

[0058] In the formula, It is represented as the output tensor after compression by the hyperbolic tangent function, which is the pixel value of the synthesized image;

[0059] Then, by using a linear transformation, the values ​​are mapped to the [0,255] range of a standard RGB image to obtain detection samples containing synthetic anomalous images. The formula is: ,

[0060] Through constraints Achieve resolution from Upgraded step by step to ,in The initial feature map base size parameters are used as input to generate the network. For the first The upsampling ratio of the deconvolution layer. For network depth;

[0061] For each sample and each exception description The generation process is performed independently, and the resulting set of synthetic anomaly images is represented as follows:

[0062] , This represents a synthetic abnormal image generated by combining the nth normal prompt template corresponding to the kth normal image sample and the mth abnormal description.

[0063] Furthermore, during the testing phase, the steps for extracting global and local image features of the image to be detected include:

[0064] Let the standardized RGB query image be denoted as The resolution is fixed at H=W=256. The input image encoder extracts features through a multi-head self-attention mechanism using 12 Transformer layers to obtain global image features. The formula is:

[0065] ,

[0066] In the formula, Represents a non-linear activation function. The original global semantic vector output by the image encoder. It is an evolutionary orthogonal projection matrix. Representation layer normalization;

[0067] Will The input is a multi-scale feature extractor, which consists of four levels of convolutional blocks, resulting in four sets of local feature maps:

[0068] ,

[0069] ,

[0070] ,

[0071] ,

[0072] In the formula, , , , These represent the first-level convolutional block, the second-level convolutional block, the third-level convolutional block, and the fourth-level convolutional block, respectively. Each level of convolutional block contains three 3×3 convolutional layers and ReLU activation.

[0073] For each level of local feature map Perform spatial dimension unrolling and L2 normalization to generate a local image feature set, as shown in the formula:

[0074] ,

[0075] In the formula, , Indicates the first Level feature map in spatial location The feature vector at that location, The Euclidean norm of a vector;

[0076] To align with the local feature dimensions used during the training phase, the local image feature set is... Through linear mapping layer Projecting the features onto a feature space with the same dimensions and semantic space as the text features extracted by CLIP yields the final set of local image features. = Each local eigenvector is normalized on a unit Euclidean sphere.

[0077] Furthermore, the steps of calculating the distance between the normal feature vector and the abnormal feature vector and the global image features of the image to be detected using the nearest neighbor algorithm, and generating an image-level anomaly score based on the distance difference, include:

[0078] Calculate global features separately With storage to reference library Normal eigenvectors in , abnormal feature vector The cosine similarity between the test image and the training features is used to evaluate the similarity between the test image and the training features. The formulas are as follows:

[0079] ,

[0080] ;

[0081] In the formula, The L2 norm of a vector;

[0082] The formula for calculating image-level anomaly scores is:

[0083] .

[0084] Furthermore, the steps of combining the image-level anomaly score and the prediction result to obtain the image-level anomaly detection score, and comparing the image-level anomaly score with a preset threshold to obtain the judgment result include:

[0085] Through absolute value weighted fusion With classification probability Fractional operations are used to obtain image-level anomaly detection scores. The formula is:

[0086] ,

[0087] In the formula, Indicates the fusion weighting coefficient. This indicates the binary classification prediction result;

[0088] When image-level anomaly score The image to be detected is determined to be an abnormal image at a certain time. This represents the classification decision threshold determined through receiver operating characteristic curve analysis.

[0089] Furthermore, when the image to be detected is an anomalous image, the process of detecting the predicted anomalous segmentation map of the anomalous image includes:

[0090] For each spatial location in the local feature map P eigenvectors The feature vector of each spatial location For a local region of the corresponding image, calculate The cosine similarity between the mean of normal descriptive features and the mean of abnormal descriptive features is calculated using the following formula:

[0091] ,

[0092] In the formula, For temperature parameters, Represents the dot product of vectors; Represents the local features after normalization This represents the mean of normal descriptive features in the reference library S. This represents the mean of the anomaly description features in the reference library S;

[0093] pass Constructing a preliminary semantic anomaly score map Each element in the semantic anomaly score graph The larger the value, the higher the probability of an anomaly in the region;

[0094] And select from reference library S Minimum similarity value of feature vectors at the same level The formula is:

[0095] ,

[0096] In the formula, express A local visual feature vector, This represents the local features of the test image currently being processed. The feature layer level it belongs to;

[0097] Minimum similarity value The mapping to anomaly scores is expressed by the following formula:

[0098] ,

[0099] By measuring the minimum matching degree between the local features of the query image and the local features of the normal image, the similarity range [-1,1] is linearly transformed to [0,1]. The larger the value, the higher the probability of an anomaly. All anomaly scores are used to construct an anomaly heatmap. Spatial resolution and input feature map Consistency directly reflects the degree to which each region deviates from the normal pattern;

[0100] calculate and The nonlinear summation fusion number is used as the prediction anomaly score map. The formula is:

[0101] ,

[0102] In the formula, Represents the predicted anomaly score plot The value for each spatial location;

[0103] Predicting anomaly score maps Multi-scale morphological optimization and sub-pixel-level boundary refinement operations are performed. The predicted anomaly score map first generates an initial binary mask through dynamic thresholding. An improved OTSU algorithm is then used to calculate the adaptive segmentation threshold. The formula is:

[0104] ,

[0105] In the formula, The inter-class variance of the grayscale histogram of the heatmap. This is the local contrast compensation factor. It is a sliding window. For local gradient magnitude, For gradient operators, and These represent the partial derivatives in the horizontal and vertical directions, respectively. This represents the total number of pixels.

[0106] If and only if At that time, binary mask =1 indicates an exception, otherwise =0 indicates normal operation, generating the initial mask. .

[0107] Beneficial effects: Compared with the prior art, the significant advantages of this invention are:

[0108] This invention effectively solves the problems of insufficient model generalization caused by the scarcity of abnormal samples and the dynamic evolution of defect types in industrial quality inspection scenarios through cross-modal text semantic driving and dynamic feature recombination mechanism; with the help of cross-modal semantic fusion technology, only normal images and text descriptions are needed to automatically construct the abnormal feature space, and combine dynamically generated synthetic defect samples to train a lightweight model, which can accurately identify unknown defects even without real abnormal data, greatly reducing the model iteration cost and annotation workload;

[0109] By combining multimodal feature recombination and implicit contrastive learning to achieve synergistic optimization of pixel-level anomaly localization and image-level judgment, this invention can effectively handle industrial scenarios such as multi-object stacking and complex backgrounds. Its adaptive discrimination boundary mechanism can dynamically adjust the detection threshold according to the features of the object region, and accurately label the defect boundary for each pixel by fusing the average. While reducing the manual re-inspection rate, it improves the ability to capture subtle defects and the generalization performance of complex scenarios, and can maintain stable detection capabilities even under conditions of few samples. Attached Figure Description

[0110] Figure 1 This is a diagram of the architecture of the present invention during the training phase;

[0111] Figure 2 This is a flowchart of the training phase.

[0112] Figure 3 This is a schematic diagram of the module structure for providing the prompt;

[0113] Figure 4 This is a structural diagram of the control boundary module;

[0114] Figure 5 This is a schematic diagram of the feature synthesis module.

[0115] Figure 6 This is an architecture diagram of the present invention during the testing phase;

[0116] Figure 7 This is a flowchart for the testing phase. Detailed Implementation

[0117] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments.

[0118] The few-shot defect identification method based on cross-modal text semantic driving described in this embodiment includes two main stages: a training stage and a testing stage. It utilizes a pre-trained visual language model to extract joint representations of image and semantic information, constructing a detection framework that can dynamically adapt to different sample scenarios. This invention establishes an anomaly judgment mechanism that does not require labeled data by analyzing the inherent relationship between text descriptions and visual features, supporting rapid model generation under few-shot conditions. Compared to traditional training modes that rely on massive labeled samples, this invention achieves lightweight modeling through semantic driving, significantly reducing data dependence while improving the generalization ability to unknown anomaly types in dynamic scenes.

[0119] This invention does not directly use real abnormal images for training, but instead generates multi-scale feature extraction through text descriptions. During the training phase, normal images without abnormalities (e.g., standard images of industrial products) and text describing normal or abnormal states (e.g., "no defects" and "damaged") are used. The normal images are processed by a pre-trained visual language model (Clip), which outputs global semantic features and local spatial feature maps representing local details. These local spatial feature maps are stored in a visual reference library to construct a reference baseline. Normal prefixes are semantically combined with the target object category name to generate a normal description set. Abnormal text suffixes are associated with normal description templates through dynamic weight fusion to generate an abnormal description set containing abnormal semantics. Features of the normal description set are extracted using a text encoder in the Clip model, transforming the abnormal descriptions into abnormal description features. Then, the normal description features and the generated abnormal description features are jointly input into a control boundary module, where hyperparameters are introduced to ensure that the distance between the normal global features and the normal description features is less than the distance between the normal global features and the abnormal description features.

[0120] The global features of normal images and normal text features are fused using mean simplification to form a combined feature vector, which is used to define the comprehensive features of a "normal object". Simultaneously, anomaly descriptions are combined with normal image features to generate synthetic anomaly vectors. These vectors are input into a text-image generator, which outputs synthetic images with specific defect morphologies, simulating real-world defects (such as cracks or stains). Normal images are labeled "normal", and synthetic anomaly images are labeled "abnormal". The original normal images and synthetic anomaly images together constitute the training dataset, used to train a binary classification machine learning model capable of distinguishing between normal and anomalous images.

[0121] During the testing phase, a query image to be detected is input, and the image encoder of the Clip model extracts global image features and local spatial feature maps of the image to be detected. The distances between the global features of the image to be detected and the normal and abnormal feature vectors generated during the training phase are calculated using the nearest neighbor algorithm: the nearest distance between the global features of the query image and the normal feature vector represents "normal similarity," and the nearest distance between the global features and the abnormal feature vector represents "abnormal similarity." The difference between these two distances generates an image-level anomaly score, which reflects the degree to which the image deviates from the normal state. The image to be detected is then input into the trained binary classification model to obtain the prediction result. Finally, the image-level anomaly detection score is obtained by combining the image-level anomaly score and the prediction result.

[0122] The local spatial feature maps of normal images stored in the reference library are compared pixel-by-pixel with the local feature maps of the query image, generating a high-resolution anomaly heatmap using a minimum distance metric. A semantic anomaly score map is generated by calculating the cosine similarity between the query image's local feature map and the mean values ​​of normal and anomaly descriptive features stored in the reference library. The anomaly heatmap and semantic anomaly score map are fused using a nonlinear summation to obtain a predicted anomaly score map. Thresholding and binarization are applied to the predicted anomaly score map to output a pixel-level anomaly region mask map with precise defect locations.

[0123] The CLIP model is the core module used in both training and testing phases. Inputs include normal images (training phase) or detected images (testing phase) and positive / negative text descriptions (e.g., "normal", "damaged"). An image encoder converts the image into image features, and a text encoder converts the text descriptions into corresponding text feature vectors. The output is image and text features in a unified embedding space, enabling cross-modal comparison of images and text, providing fundamental support for subsequent feature fusion and anomaly detection.

[0124] The training dataset consists entirely of normal RGB images of the objects to be detected, while the test set includes both normal and defective images. During the training phase, only a small number of normal image samples are required.

[0125] Combination Figure 1 and Figure 2 As shown, the training phase includes the following steps:

[0126] Step 11: Obtain the global and local feature maps of the normal image, and store the local feature maps of the normal image into the reference library.

[0127] Furthermore, the steps for obtaining global and local feature maps of a normal image include:

[0128] The preprocessed normal image is input into the image encoder, where it is divided into pixel block sequences via a block embedding layer. Global features are then extracted using a multi-head self-attention mechanism. and local feature maps , where c represents the number of image patches after the spatial image is divided, and d represents the feature dimension of each image patch encoded in the visual language model Clip.

[0129] In one example, a structured positive and negative semantic template is constructed, comprising a positive text library and a negative text library. The positive text library integrates basic descriptions (such as "smooth and flawless surface") and enhanced contextual combinations (such as "intact {category} captured from multiple angles"), while the negative text library is generated based on a defect type knowledge graph, covering morphological anomalies (such as "edge cracks"), texture defects (such as "corrosion spots"), and complex semantic perturbations (such as "damage {category} in low-contrast images"), ensuring that the positive and negative text sizes reach 200 and 300 respectively.

[0130] For a set of K defect-free industrial product RGB images { The images undergo preprocessing, with a unified resolution of 512×512 pixels. Each frame contains 1 to 3 target objects to avoid occlusion and stacking. Then, a normalization operation is performed: pixel values ​​are normalized to [-1, 1], using a cubic polynomial interpolation method based on the Mitchell-Netravali kernel function. The interpolation formula is as follows:

[0131] ,

[0132] In the formula, x represents the relative distance (in pixels, with direction) between the pixel sampling position and the adjacent integer pixels. This represents the interpolation weight at a distance of x. The parameter a = -0.5 is used to balance interpolation sharpness and smoothness. This value ensures that the geometric deformation error is less than 0.5 pixels. Background noise is filtered out by segmentation using a fixed threshold of 0.5.

[0133] To ensure that the effective area of ​​the target object is retained by ≥85%, area ratio constraints are required: Let the area of ​​the original segmented region be... The area of ​​the retained effective region is , must meet Discrete noise points are removed through region connectivity analysis to meet the target. The center cropping specification is determined based on the minimum bounding rectangle of the target region, assuming the original image size is... The center coordinates of the target area are The cropping window size is preset. (For example, taking 80% of the shorter side of the image as a baseline), the final cropping area expands symmetrically around the center point, i.e. The cropping is confined to the image boundaries to ensure key features are concentrated within the cropping box. This setting balances effective region preservation and key information integrity through area threshold constraints and geometric center localization.

[0134] The Clip model includes an image encoder and a text encoder. The image encoder uses a ViT-B / 16 variant. The preprocessed image is input into the image encoder, and after being divided into a 16×16 pixel block sequence by a block embedding layer, global semantic features are extracted through a multi-head self-attention mechanism. In this example, the feature dimension d=768, and the local spatial feature map. 196 corresponds to a 14×14 spatial grid, with each grid point associated with a 768-dimensional feature vector, preserving fine-grained structural information at a scale of 1 / 16 of the original image. Local feature maps of normal images are stored in a reference library S. These local feature maps are used to calculate cosine similarity with the local feature maps of the query image during the testing phase.

[0135] Step 12: Construct normal prefixes and abnormal guide words, generate a normal description set based on normal prefixes, generate an abnormal description set based on abnormal guide words, and perform feature extraction to obtain a normal description feature set and an abnormal description feature set.

[0136] The Prompt generation module (PGM) takes a normal text prefix and an exception prompt as input. The normal text prefix is ​​concatenated with the object name to generate a normal description; the normal description is then concatenated with the exception prompt to generate an exception description containing exception semantics.

[0137] In one example, normal prefixes include "a clean" and "a normal"; abnormal suffixes include "damaged {}", "broken {}", "{} with flaw", "{} with defect", "{} with damage", etc., and category names can be placed inside the curly braces {}.

[0138] Combination Figure 3 Furthermore, the steps of generating a normal description set based on normal prefixes and an abnormal description set based on abnormal introductory words include:

[0139] Let the normal prefix be denoted as , This represents the total number of evolutionary normal prefixes. Indicates one of the evolutionary normal prefixes;

[0140] Normal prefix prompts The target object name is fused with the target object name using a bidirectional attention mechanism, resulting in the following structure: From the normal semantic templates, N normal description templates are formed, among which... Indicates the target object category name, attention mechanism Using the scaled dot product method, This represents the concatenation operation of vector sequences;

[0141] Let the abnormality introductory word be denoted as , This indicates the total number of abnormal introductory words. This indicates one of the abnormal introductory words;

[0142] Category Name By concatenating these elements after each exception introductory word, a set of specific exception text descriptions is obtained, represented as follows: The template-based exception description set is as follows: ,form There are 10 abnormal semantic templates, where M represents the total number of categories of the corresponding labeled defects;

[0143] Through dynamic weight fusion module abnormal guide word sequence With normal semantic templates Association, construction form is Evolvable anomalous templates, forming There are several evolvable anomalous templates, among which... Indicates bilinear feature interaction. To control the size of the evolvable template using hyperparameters;

[0144] Will include A set of normal description templates { } as a normal description set, An abnormal semantic template and A set of evolvable anomalous templates As a set of exception descriptions.

[0145] Furthermore, the steps of extracting features to obtain the normal description feature set and the abnormal description feature set include:

[0146] The normal description set { The input is fed into the text encoder of the Clip model to obtain the normal descriptive feature set, represented as... ;in The text encoder representing the Clip visual language model;

[0147] Set of exception descriptions The input is fed into a text encoder to obtain a set of anomaly description features, represented as follows: .

[0148] Combination Figure 4 Furthermore, the normal description feature set and the abnormal description feature set are optimized, and the process includes:

[0149] Construct a dual-path feature constraint mechanism to constrain global features Normal descriptive feature set and abnormal description feature set The input is fed into the Control Boundary Module (CMM) to calculate the mean of the normal descriptive features, using the following formula:

[0150] ,

[0151] In the formula, This indicates a normal description template index;

[0152] Construct the feature similarity loss function, the formula is:

[0153] ,

[0154] In the formula, This represents the natural exponential function. For temperature coefficient, This represents a feature vector in the template-type anomaly description features. This represents a feature vector in an evolutionary anomaly template. This represents the negative normalized dot product similarity. The feature dimension; this similarity loss Force global features of normal images Compared with the mean of normal descriptive features The similarity is higher than the similarity with the anomaly description features;

[0155] Implement multi-prototype space constraints and calculate the mean of anomaly descriptive features. Artificial Anomaly Template Center With the center of the evolutionary anomaly template Construct a dual distance loss function, the formula of which is:

[0156] ,

[0157] in, Representing global features Expected value Represents the square of the Euclidean distance. This is the distance to the boundary threshold. The first term constrains the global features of the normal image, representing the distribution alignment intensity coefficients. distance Euclidean distance ratio Nearly at least The unit, the second term reduces the distribution difference between the two types of abnormal prototypes;

[0158] The overall optimization objective is After updating the evolvable parameters using gradient descent, the optimized normal description set is obtained. }, and by An abnormal semantic template and A set of anomaly descriptions consisting of evolvable anomaly templates The optimization makes the normal description set closer to the global features of a normal image, and the normal and abnormal descriptions are more clearly separated. The mean of the normal description features is also improved. and the mean of the anomaly description features Store in reference library .

[0159] Step 13: Combine the global features of the normal image with the normal descriptive features and the abnormal descriptive features respectively to generate normal feature vectors and abnormal feature vectors.

[0160] Furthermore, the steps of combining the global features of a normal image with normal descriptive features and abnormal descriptive features respectively to generate normal feature vectors and abnormal feature vectors include:

[0161] Global features of normal images Normal descriptive features and anomaly description feature set The input is fed into the Feature Synthesis Module (FSM). A schematic diagram of the Feature Synthesis Module is shown below. Figure 5 As shown, global features are synthesized in the feature synthesis module. Through linear projection layer Mapped to This aligns the dimensions of global features with those of normal and abnormal descriptive features.

[0162] For each normal descriptive feature Perform cross-modal fusion: using a gating mechanism Generate normal feature vectors The formula is:

[0163] ,

[0164] In the formula, Represents element-wise product. The matrix represents the evolutionary parameters. Activated for Sigmoid;

[0165] Describe the features of each anomaly Calculate cross-modal attention gating weights , Evolvable parameters; hybrid features are generated through gating weighting. All After being stitched along the channel dimension and passed through a fully connected layer The projection yields the final anomalous feature vector. The formula is:

[0166] ;

[0167] Storing normal feature vectors and abnormal feature vectors To the reference library .

[0168] Normal descriptive feature set Global features of normal images Perform feature alignment using bilinear fusion ,in Linear projection layer in the module Gating parameters , All with a learning rate of 5× End-to-end optimization enables normal-abnormal semantic decoupling and dynamic partitioning of the feature space.

[0169] The feature synthesis module during the training phase takes as input the global features, normal descriptive features, and abnormal descriptive features of the normal image generated by the Clip model. It fuses the global features and normal descriptive features into a combined feature vector, representing the comprehensive features of a "normal object." The output is used to define a reference baseline for the normal state, enhancing the model's ability to express normal features. The abnormal descriptive features are then concatenated with the global features of the normal image to generate a synthetic abnormal vector, which is used to subsequently generate abnormal images.

[0170] Step 14: Use the abnormal feature vectors to obtain the synthesized abnormal image.

[0171] Furthermore, the steps for obtaining a synthesized anomalous image using anomalous feature vectors include:

[0172] Utilizing anomaly feature vectors Constructing an abnormal feature vector set ,Will The input is fed into the decoding module of a pre-trained text-image generator to obtain a tensor. , This represents a pre-trained image-text generator that generates images based on text features. H represents the number of normal samples in the training dataset, H represents the height of the final synthetic anomalous image output by the generator, and W represents the width of the final synthetic anomalous image output by the generator.

[0173] When the network reaches the final layer, the pixel value range of the tensor z is constrained by the hyperbolic tangent function, compressing the original output to... An interval is represented as:

[0174] ,

[0175] In the formula, This is represented as the output tensor after compression using the hyperbolic tangent function, i.e., the pixel values ​​of the synthesized image; this operation compresses the original output to... interval;

[0176] Then, by using a linear transformation, the values ​​are mapped to the [0,255] range of a standard RGB image to obtain detection samples containing synthetic anomalous images. The formula is: ,

[0177] Through constraints Achieve resolution from Upgraded step by step to This ensures that the resolution increase factor is strictly matched with the network depth, among which The basic size parameters of the initial feature map corresponding to the input network are... For the first The upsampling ratio of the deconvolution layer. For network depth, H is set to 256 in this example. =8, ;

[0178] For each sample and each exception description The generation process is performed independently, and the resulting set of synthetic anomaly images is represented as follows:

[0179] , This represents a synthetic abnormal image generated by combining the nth normal prompt template corresponding to the kth normal image sample and the mth abnormal description.

[0180] Synthetic abnormal image set This will be used as training data to input into the subsequent classification model.

[0181] As the image generation module during the training phase, GAN takes anomaly feature vectors and anomaly text features as input. Based on the combination of text descriptions and image features, it generates synthetic anomaly images that simulate real defects (such as cracked industrial parts). The output synthetic anomaly images and real normal images together constitute the training set, solving the problem of insufficient real anomaly data and improving the model's generalization ability to anomaly scenarios.

[0182] Step 15: Label the normal images and the synthetic anomalous images, use the normal images and the synthetic anomalous images to form a training set, and use the training set to train a binary classification model.

[0183] Normal images are labeled "normal," and synthesized abnormal images are labeled "abnormal." A two-stage optimization strategy is used during training: the linear projection layer is fixed for the first 50 epochs. Parameters, only optimize the classification head parameters. The initial learning rate is set to 5×1. In the last 150 cycles, all parameters are jointly optimized, and the learning rate is... =1×1 Optimizer momentum parameters Weight decay coefficient The final output is a binary classification model result. : .

[0184] Combination Figures 6 to 7 As shown, the testing phase includes the following steps:

[0185] Step 21: Construct a test set using normal images and defective images.

[0186] In this example, images in which objects have no abnormalities or defects are considered normal images; images in which objects have abnormalities, such as damage, are considered defective images.

[0187] Step 22: Select any image from the test set as the image to be detected, and extract the global image features and local image features of the image to be detected.

[0188] Furthermore, during the testing phase, the steps for extracting global and local image features of the image to be detected include:

[0189] Let the standardized RGB query image be denoted as The resolution is fixed at H=W=256. The input image encoder extracts features through a multi-head self-attention mechanism using 12 Transformer layers to obtain global image features. The formula is:

[0190] ,

[0191] In the formula, Represents a non-linear activation function. The original global semantic vector output by the image encoder. It is an evolutionary orthogonal projection matrix; Representation layer normalization;

[0192] Will The input is a multi-scale feature extractor, which consists of four levels of convolutional blocks, resulting in four sets of local feature maps:

[0193] ,

[0194] ,

[0195] ,

[0196] ,

[0197] In the formula, , , , These represent the first-level convolutional block, the second-level convolutional block, the third-level convolutional block, and the fourth-level convolutional block, respectively. Each level of convolutional block contains three 3×3 convolutional layers and ReLU activation.

[0198] For each level of local feature map Perform spatial dimension unrolling and L2 normalization to generate a local image feature set, as shown in the formula:

[0199] ,

[0200] In the formula, , Indicates the first Level feature map in spatial location The feature vector at that location, This represents the Euclidean norm of the vector. To align with the dimensions of the local features used during training, [the following is used]. Through linear mapping layer Projecting onto a unified feature space, we finally obtain = Each local eigenvector is normalized on a unit Euclidean sphere.

[0201] global image features and local image feature set Together they constitute a bimodal feature set ( , The data is directly transmitted to the anomaly scoring module, whereby... Driven image-level anomaly classification, Used to generate pixel-level location heatmaps.

[0202] Step 23: The distance between the normal feature vector and the abnormal feature vector and the global image features of the image to be detected is calculated using the nearest neighbor algorithm, and an image-level anomaly score is generated based on the distance difference.

[0203] Furthermore, the steps of calculating the distance between the normal feature vector and the abnormal feature vector and the global image features of the image to be detected using the nearest neighbor algorithm, and generating an image-level anomaly score based on the distance difference, include:

[0204] Calculate global features separately With storage to reference library Normal eigenvectors in , abnormal feature vector The cosine similarity between the test image and the training features is used to evaluate the similarity between the test image and the training features. The formulas are as follows:

[0205] ,

[0206] ;

[0207] In the formula, The L2 norm of a vector;

[0208] The formula for calculating image-level anomaly scores is:

[0209] ;

[0210] Step 24: Input the image to be detected into the trained binary classification model to obtain the prediction result;

[0211] Step 25: Combine the image-level anomaly score and the prediction result to obtain the image-level anomaly detection score. Compare the image-level anomaly score with the preset threshold to obtain the judgment result.

[0212] Furthermore, the steps of combining the image-level anomaly score and the prediction result to obtain the image-level anomaly detection score, and comparing the image-level anomaly score with a preset threshold to obtain the judgment result include:

[0213] Through absolute value weighted fusion With classification probability Fractional operations are used to obtain image-level anomaly detection scores. The formula is:

[0214] ,

[0215] In the formula, Indicates the fusion weighting coefficient. This indicates the binary classification prediction result;

[0216] When image-level anomaly score The image to be detected is determined to be an abnormal image at a certain time. This represents the classification decision threshold determined through receiver operating characteristic curve analysis.

[0217] Furthermore, when the image to be detected is an anomalous image, the process of detecting the predicted anomalous segmentation map of the anomalous image includes:

[0218] For each spatial location in the local feature map P eigenvectors The feature vector of each spatial location For a local region of the corresponding image, calculate The cosine similarity between the mean of normal descriptive features and the mean of abnormal descriptive features is calculated using the following formula:

[0219] ,

[0220] In the formula, This is a temperature parameter; the default value is 0.01. Represents the dot product of vectors; Represents the local features after normalization This represents the mean of normal descriptive features in the reference library S. This represents the mean of the anomaly description features in the reference library S;

[0221] pass Constructing a preliminary semantic anomaly score map Each of its elements The larger the value, the higher the probability of an anomaly in that area;

[0222] And select with Minimum similarity value of feature vectors at the same level The formula is:

[0223] ,

[0224] In the formula, express A local visual feature vector, This represents the local features of the test image currently being processed. The feature layer level it belongs to;

[0225] Minimum similarity value The mapping to anomaly scores is expressed by the following formula:

[0226] ,

[0227] By measuring the minimum matching degree between the local features of the query image and the local features of the normal image, the similarity range [-1,1] is linearly transformed to [0,1]. The larger the value, the higher the probability of an anomaly. All anomaly scores are used to construct an anomaly heatmap. Its spatial resolution is similar to that of the input feature map. Consistency directly reflects the degree to which each region deviates from the normal pattern;

[0228] calculate and The nonlinear summation fusion number is used as the prediction anomaly score map. The formula is:

[0229] ,

[0230] In the formula, This represents the value at each spatial location in the predicted anomaly score map;

[0231] Predicting anomaly score maps Multi-scale morphological optimization and sub-pixel-level boundary refinement operations are performed. The predicted anomaly score map first generates an initial binary mask through dynamic thresholding. An improved OTSU algorithm is then used to calculate the adaptive segmentation threshold. The formula is:

[0232] ,

[0233] In the formula, The inter-class variance of the grayscale histogram of the heatmap. This is the local contrast compensation factor. It is a sliding window. For local gradient magnitude, For gradient operators, and These represent the partial derivatives in the horizontal and vertical directions, respectively. This represents the total number of pixels.

[0234] If and only if At that time, binary mask =1 indicates an exception, otherwise =0 indicates normal operation, generating the initial mask. .

Claims

1. A few-sample defect identification method based on cross-modal text semantics, characterized in that, Includes training and testing phases; During the training phase: Obtain the global and local features of a normal image, and store the local features of the normal image into a reference library; Construct normal prefixes and abnormal introductory words, generate a normal description set based on the normal prefixes, generate an abnormal description set based on the abnormal introductory words, and perform feature extraction to obtain a normal description feature set and an abnormal description feature set; The global features of a normal image are combined with normal descriptive features and abnormal descriptive features respectively to generate normal feature vectors and abnormal feature vectors. Synthetic anomalous images are obtained using anomalous feature vectors; Normal images and synthetic anomalous images are labeled, and a training set is constructed using normal images and synthetic anomalous images. A binary classification model is then trained using the training set. During the testing phase: A test set was constructed using normal images and defective images; Select any image from the test set as the image to be detected, and extract the global and local image features of the image to be detected; The distances between the normal feature vector and the abnormal feature vector and the global image features of the image to be detected are calculated using the nearest neighbor algorithm, and an image-level anomaly score is generated based on the distance difference. The image to be detected is input into the trained binary classification model to obtain the prediction result; The image-level anomaly score is obtained by combining the image-level anomaly score and the prediction result. The image-level anomaly score is then compared with a preset threshold to obtain the judgment result. The steps to obtain global and local features of a normal image include: The preprocessed normal image is input into the image encoder, where it is divided into pixel block sequences via a block embedding layer. Global features are then extracted using a multi-head self-attention mechanism. and local features , where c represents the number of image patches after the spatial image is divided, d represents the feature dimension of each image patch encoded in the visual language model Clip, and the local feature map of the normal image is stored in the reference library S; The steps for generating a normal description set based on normal prefixes and an abnormal description set based on abnormal introductory terms include: Let the normal prefix be denoted as , This represents the total number of evolutionary normal prefixes. Indicates one of the evolutionary normal prefixes; Normal prefix prompts The target object name is fused with the target object name using a bidirectional attention mechanism, resulting in the following structure: From the normal semantic templates, N normal description templates are formed, among which... Indicates the target object category name, attention mechanism Using the scaled dot product method, This represents the concatenation operation of vector sequences; Let the abnormality introductory word be denoted as , This indicates the total number of abnormal introductory words. This indicates one of the abnormal introductory words; Category Name By concatenating these elements after each exception introductory word, a set of specific exception text descriptions is obtained, represented as follows: The template-type exception description set is ,form There are 10 abnormal semantic templates, where M represents the total number of categories of the corresponding labeled defects; Through the dynamic weight fusion module abnormal guide word sequence With normal semantic templates Association, construction form is Evolvable anomalous templates, forming There are several evolvable anomalous templates, among which... Indicates bilinear feature interaction. To control the size of the evolvable template using hyperparameters; Will include A set of normal description templates { } as a normal description set, An abnormal semantic template and A set of evolvable anomalous templates As a set of exception descriptions.

2. The few-sample defect identification method based on cross-modal text semantic driving according to claim 1, characterized in that, The steps involved in feature extraction to obtain normal and abnormal description feature sets include: The normal description set { The input is fed into the text encoder to obtain the normal descriptive feature set, represented as... ;in The text encoder representing the Clip visual language model; Set of exception descriptions The input is fed into a text encoder to obtain a set of anomaly description features, represented as follows: .

3. The few-sample defect identification method based on cross-modal text semantic driving according to claim 2, characterized in that, The optimization process for the normal and abnormal feature sets includes: global features Normal descriptive feature set and abnormal description feature set The input is fed into the control boundary module to calculate the mean of the normal descriptive features, using the following formula: ， In the formula, This indicates a normal description template index; Construct the feature similarity loss function, the formula is: ， In the formula, This represents the natural exponential function. For temperature coefficient, This represents the negative normalized dot product similarity. This represents a feature vector in the template-type anomaly description features. This represents a feature vector in an evolutionary anomalous template; the loss... Force global features of normal images Compared with the mean of normal descriptive features The similarity is higher than the similarity with the anomaly description features; Implement multi-prototype space constraints and calculate the mean of anomaly descriptive features using the following formula: ， The formula for calculating the center of the artificial anomaly template is: , The formula for calculating the center of an evolutionary anomaly template is: , Construct a dual distance loss function, the formula of which is: ， in, Representing global features Expected value Represents the square of the Euclidean distance. This is the distance to the boundary threshold. The first term constrains the global features of the normal image, representing the distribution alignment intensity coefficients. distance Euclidean distance ratio Nearly at least The unit, the second term reduces the distribution difference between the two types of abnormal prototypes; The overall optimization objective is After updating the evolvable parameters using gradient descent, the optimized normal description set is obtained. }, and by An abnormal semantic template and A set of anomaly descriptions consisting of evolvable anomaly templates The mean of the normal descriptive features and the mean of abnormal descriptive features Store it in reference library S.

4. The few-sample defect identification method based on cross-modal text semantic driving according to claim 3, characterized in that, The steps for combining the global features of a normal image with normal descriptive features and abnormal descriptive features to generate normal feature vectors and abnormal feature vectors include: global features Through linear projection layer Mapped to This aligns the dimensions of global features with those of normal and abnormal descriptive features. For each normal descriptive feature Perform cross-modal fusion: using a gating mechanism Generate normal feature vectors The formula is: ， In the formula, Represents element-wise product. The matrix represents the evolutionary parameters. Activated for Sigmoid; Describe the features of each anomaly Calculate cross-modal attention gating weights , Evolvable parameters; hybrid features are generated through gating weighting. All After being stitched along the channel dimension and passed through a fully connected layer The projection yields the final anomalous feature vector. The formula is: ； Storing normal feature vectors and abnormal feature vectors To the reference library .

5. The few-sample defect identification method based on cross-modal text semantic driving according to claim 4, characterized in that, The steps for obtaining a synthetic anomalous image using anomalous feature vectors include: Utilizing anomaly feature vectors Constructing an abnormal feature vector set ,Will The input is fed into the decoding module of a pre-trained text-image generator to obtain a tensor. , This represents a pre-trained image-text generator that generates images based on text features. H represents the number of normal samples in the training dataset, H represents the height of the final synthetic anomalous image output by the generator, and W represents the width of the final synthetic anomalous image output by the generator. When the network reaches the final layer, the pixel value range of the tensor z is constrained by the hyperbolic tangent function, compressing the original output to... An interval is represented as: ， In the formula, It is represented as the output tensor after compression by the hyperbolic tangent function, which is the pixel value of the synthesized image; Then, by using a linear transformation, the values ​​are mapped to the [0,255] range of a standard RGB image to obtain detection samples containing synthetic anomalous images. The formula is: , Through constraints Achieve resolution from Upgraded step by step to ,in The initial feature map base size parameters are used as input to generate the network. For the first The upsampling ratio of the deconvolution layer. For network depth; For each sample and each exception description The generation process is performed independently, and the resulting set of synthetic anomaly images is represented as follows: , This represents a synthetic abnormal image generated by combining the nth normal prompt template corresponding to the kth normal image sample and the mth abnormal description.

6. The few-sample defect identification method based on cross-modal text semantic driving according to claim 5, characterized in that, During the testing phase, the steps for extracting global and local image features of the image to be detected include: Let the standardized RGB query image be denoted as The resolution is fixed at H=W=256. The input image encoder extracts features through a multi-head self-attention mechanism using 12 Transformer layers to obtain global image features. The formula is: ， In the formula, Represents a non-linear activation function. The original global semantic vector output by the image encoder. It is an evolutionary orthogonal projection matrix. Representation layer normalization; Will The input is a multi-scale feature extractor, which consists of four levels of convolutional blocks, resulting in four sets of local feature maps: ， ， ， ， In the formula, , , , These represent the first-level convolutional block, the second-level convolutional block, the third-level convolutional block, and the fourth-level convolutional block, respectively. Each level of convolutional block contains three 3×3 convolutional layers and ReLU activation. For each level of local feature map Perform spatial dimension unrolling and L2 normalization to generate a set of local image features, as shown in the formula: ， In the formula, , Indicates the first Level feature map in spatial location The feature vector at that location, The Euclidean norm of a vector; To align with the local feature dimensions used during the training phase, the local image feature set is... Through linear mapping layer Projecting the features onto a feature space with the same dimensions and semantic space as the text features extracted by CLIP yields the final set of local image features. = Each local eigenvector is normalized on a unit Euclidean sphere.

7. The few-sample defect identification method based on cross-modal text semantic driving according to claim 6, characterized in that, The steps of calculating the distance between the normal feature vector and the abnormal feature vector and the global image features of the image to be detected using the nearest neighbor algorithm, and generating an image-level anomaly score based on the distance difference, include: Calculate global features separately With storage to reference library Normal eigenvectors in , abnormal feature vector The cosine similarity between the test image and the training features is used to evaluate the similarity between the test image and the training features. The formulas are as follows: ， ； In the formula, The L2 norm of a vector; The formula for calculating image-level anomaly scores is: 。 8. The few-sample defect identification method based on cross-modal text semantic driving according to claim 7, characterized in that, The steps for obtaining an image-level anomaly detection score by combining the image-level anomaly score and the prediction result, and then comparing the image-level anomaly score with a preset threshold to obtain the judgment result include: Through absolute value weighted fusion With classification probability Fractional operations are used to obtain image-level anomaly detection scores. The formula is: ， In the formula, Indicates the fusion weighting coefficient. This indicates the binary classification prediction result; When image-level anomaly score The image to be detected is determined to be an abnormal image at a certain time. This represents the classification decision threshold determined through receiver operating characteristic curve analysis.

9. The few-sample defect identification method based on cross-modal text semantic driving according to claim 8, characterized in that, When the image to be detected is an anomalous image, the process of detecting the predicted anomalous segmentation map of the anomalous image includes: For each spatial location in the local feature map P eigenvectors The feature vector of each spatial location For a local region of the corresponding image, calculate The cosine similarity between the mean of normal descriptive features and the mean of abnormal descriptive features is calculated using the following formula: ， In the formula, For temperature parameters, Represents the dot product of vectors; Represents the local features after normalization This represents the mean of normal descriptive features in the reference library S. This represents the mean of the anomaly description features in the reference library S; pass Constructing a preliminary semantic anomaly score map Each element in the semantic anomaly score graph The larger the value, the higher the probability of an anomaly in the region; And select from the reference library S Minimum similarity value of feature vectors at the same level The formula is: ， In the formula, express A local visual feature vector, This represents the local features of the test image currently being processed. The feature layer level it belongs to; Minimum similarity value The mapping to anomaly scores is expressed by the following formula: ， By measuring the minimum matching degree between the local features of the query image and the local features of the normal image, the similarity range [-1,1] is linearly transformed to [0,1]. The larger the value, the higher the probability of an anomaly. All anomaly scores are used to construct an anomaly heatmap. Spatial resolution and input feature map Consistency directly reflects the degree to which each region deviates from the normal pattern; calculate and The nonlinear summation fusion number is used as the prediction anomaly score map. The formula is: ， In the formula, Represents the predicted anomaly score plot The value for each spatial location; Predicting anomaly score maps Multi-scale morphological optimization and sub-pixel-level boundary refinement operations are performed. The predicted anomaly score map first generates an initial binary mask through dynamic thresholding. An improved OTSU algorithm is then used to calculate the adaptive segmentation threshold. The formula is: ， In the formula, The inter-class variance of the grayscale histogram of the heatmap. This is the local contrast compensation factor. It is a sliding window. For local gradient magnitude, For gradient operators, and These represent the partial derivatives in the horizontal and vertical directions, respectively. This represents the total number of pixels. If and only if At that time, binary mask =1 indicates an exception, otherwise =0 indicates normal operation, generating the initial mask. .

Images