Small sample industrial anomaly detection method based on prompt learning

Abstract

The invention discloses a small sample industrial anomaly detection method based on prompt learning. The method comprises the steps that (1) word splitting is carried out on a normal prompt text and an abnormal prompt text used for industrial image anomaly detection to obtain character sequences, word boundary marks are attached to the tail of the character sequences to form initial sub-word units, and the abnormal prompt text is an industrial image judgment text to be subjected to anomaly detection; performing adaptive byte pair code merging operation on the sub-word units to obtain merged sub-word units; 2) coding the sub-word units to obtain a normal prompt text context feature vector and an abnormal prompt text context feature vector; the method comprises the steps of (1) obtaining a context feature vector, (2) carrying out importance evaluation and semantic non-linear expression enhancement to obtain a context feature vector, (4) obtaining an abnormal confidence coefficient and a semantic difference score between the abnormal confidence coefficient and the context feature vector according to a difference value, and carrying out calculation to obtain a comprehensive abnormal score, and (5) judging whether categories described by an industrial image and an abnormal prompt text are consistent or not according to the comprehensive abnormal score.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision and artificial intelligence, and in particular relates to a small-sample industrial anomaly detection method based on cue learning. Background Technology

[0002] Industrial visual inspection, as a crucial component of quality control in modern industrial production, places higher demands on the generalization ability, anomaly detection accuracy, and training sample requirements of the models used. In practical applications, the difficulty in collecting and the low frequency of anomaly samples, coupled with high annotation costs, lead to extreme imbalance in model training data. This makes small-sample anomaly detection one of the current research focuses.

[0003] Although these traditional methods have achieved some success on specific datasets, they generally suffer from the following technical challenges.

[0004] The feature has poor generalization ability: it is difficult to effectively generalize to unseen abnormal scenarios outside the training set.

[0005] Lack of multimodal understanding: Modeling is limited to the visual domain, failing to fully utilize multimodal information such as textual semantics or category descriptions in context for assisted recognition.

[0006] Slow task adaptation and high adaptation costs: The model needs to be retrained or undergo complex parameter tuning for new types of anomalies.

[0007] Meanwhile, multimodal pre-trained models, exemplified by CLIP, have demonstrated powerful general semantic concept capture capabilities and zero-shot generalization potential through large-scale joint learning of images and text. This has inspired researchers to apply them to tasks such as open-class recognition and general object detection, and they are gradually being introduced into the field of few-shot industrial anomaly detection.

[0008] Existing methods that directly apply CLIP-based approaches to industrial visual inspection still have significant shortcomings: 1) Lack of direction awareness mechanism: Existing methods do not explicitly model the "semantic offset direction" (or "residual direction") between normal and abnormal semantics, resulting in the lack of effective directional alignment guidance when the model performs image and text feature matching on local abnormal regions. 2) Lack of consideration of word-level semantic selection mechanism: There is a lack of an effective strategy to focus on key information regions in the image, making it difficult to cope with the complex challenges of abnormal regions having localization and multi-scale distribution in the image. Summary of the Invention

[0009] This invention proposes a small-sample industrial anomaly detection method based on cue learning, aiming to solve the problems of missing semantic offset guidance mechanism and insufficient accuracy of local anomaly localization in existing technologies.

[0010] 1. Construct a multimodal enhanced visual-language joint representation model: This invention provides a word structure-aware adaptive word segmentation strategy to enhance the text representation capabilities of the CLIP visual-language model and improve its semantic understanding and discrimination performance in fine-grained semantic scenarios. This strategy, as a core component of the model's text processing, is achieved through the following two key technological innovations: (1) Adaptive byte pair encoding: This method introduces a maximum merging limit parameter K to the traditional BPE algorithm, which enhances the adaptability of modeling local semantic structures while ensuring computational efficiency. (2) Word masking annotation mechanism: In order to further enhance the structured semantic modeling capability of the model in the text encoding stage, this invention proposes a word-level semantic masking annotation mechanism to establish a traceable semantic alignment relationship between the sub-word level and the word level.

[0011] 2. Establish residual feature hinting project: The aim is to explicitly characterize the semantic shift relationship between normal and abnormal prompts by performing fine-grained modeling and nonlinear fine-tuning of the contextual semantic features of text prompts, thereby achieving accurate probability estimation of abnormal categories. The residual feature prompting engineering proposed in this invention includes a key prompt extraction module, a residual feature adapter, and a residual learning module, which can effectively extract semantic residual information and calculate the anomaly confidence probability within a vision-language model framework.

[0012] (1) Key prompt extraction module, which is used to quantify and filter the importance of different sub-words in the context feature vector in order to generate key prompt features that can represent the core of global semantics.

[0013] (2) Residual feature adapter, whose core function is to perform semantic fine-tuning on the context features output by the key prompt extraction module, and to explicitly model the semantic residual direction between normal and abnormal prompts, thereby improving the model’s sensitivity to abnormal semantic shifts and its expressive ability.

[0014] (3) Anomaly detection discriminator, which is used to measure the anomaly and predict the probability of the context residual feature vector output by the residual feature adapter. This module adopts a lightweight three-layer multilayer perceptron (MLP) structure, and achieves efficient mapping from high-dimensional features to anomaly probabilities through multilayer linear transformation, nonlinear activation and normalization operations.

[0015] 3. Construct an interpretable scoring system: An interpretability scoring mechanism based on the fusion of semantic difference and residual matching is used to perform high-precision discrimination and confidence quantification of anomaly prompts, thereby achieving reliable interpretation and robust output of anomaly detection results. This scoring system integrates two core indicators: semantic difference score and residual confidence, constructing a comprehensive anomaly assessment method with direction awareness and semantic alignment capabilities. Specifically, the scoring mechanism of this invention mainly includes the following two parts: Semantic difference score calculation module. This module is used to characterize the degree of semantic offset between the normal prompt context and the abnormal prompt context, thereby quantifying the difference between the two in the semantic space; Comprehensive Score Calculation Module. This module is used to comprehensively measure the confidence level of the input prompts' anomalies and generate a final comprehensive anomaly score.

[0016] The technical solution of this invention is a small-sample industrial anomaly detection method based on cue learning, the steps of which include: 1) The normal prompt text and the abnormal prompt text used for industrial image anomaly detection are split into character sequences by word, and a word boundary marker "" is added to the end of each sequence to form an initial sub-word unit. The abnormal prompt text is the industrial image judgment text to be detected. An adaptive byte pair encoding merging operation is performed on the sub-word unit, and the merged sub-word unit is output. 2) Encode the sub-word units using a text encoder to generate a context feature vector; the context feature vector includes the context feature vector of the normal prompt text. and the contextual feature vector of the error message text ; 3) Utilize the key prompt extraction module to extract the contextual feature vector of the normal prompt text. An importance assessment is performed to obtain a contextual key vector representing the semantics of the entire normal prompt text. ;Utilize residual feature adapters to apply the context key vectors By performing a non-linear representation to enhance semantics, we obtain a contextual feature vector that enhances the normal prompt text. ; and utilize the key prompt extraction module to extract contextual feature vectors from abnormal prompt text. An importance assessment is performed to obtain a contextual key vector representing the semantics of the entire anomaly alert text. Using residual feature adapters to analyze the key vectors of anomaly alert text semantics By performing a nonlinear representation to enhance semantics, we obtain a contextual feature vector for residual enhancement of the anomaly prompt text. ; 4) Utilize the residual learning module to process the context feature vectors Context feature vector The difference is calculated to obtain the context residual feature vector ctx_res, which is then input into the anomaly alert discriminator to obtain the anomaly confidence score residual_logits; 5) Calculate the context feature vector using the semantic deviation calculation module. With context feature vector The cosine distance between them is used as the semantic difference score token_diff_score; a comprehensive anomaly score is calculated based on the anomaly confidence score residual_logits and the semantic difference score token_diff_score. 6) Determine whether the industrial image to be detected is consistent with the category described in the anomaly prompt text based on the comprehensive anomaly score.

[0017] Preferably, the residual feature adapter adopts an unbiased two-layer fully connected bottleneck structure, wherein the first linear layer compresses the input D-dimensional feature vector into... The original dimension D is then mapped back through the ReLU activation function and the second linear layer.

[0018] Preferably, D′=D / 4D'.

[0019] Preferably, the key prompt extraction module linearly scores each word feature in the input feature vector using a learnable weight vector, then uses Gumbel-Softmax to generate normalized feature weights and sums them up for all word features in the input feature vector to obtain the context key vector of the feature vector.

[0020] Preferably, the anomaly detection discriminator adopts a multilayer perceptron structure: in the first layer perceptron, the input vector is transformed by linear transformation. Projecting onto a semantic space of a specified dimension yields the mapping result. Then on A nonlinear feature transformation is performed by applying the ReLU activation function, and the feature distribution is stabilized by batch normalization to obtain intermediate feature representations. In the second-layer perceptron, for A linear mapping is performed to compress the feature dimension, extracting deep semantic patterns related to the anomaly direction. After ReLU activation and batch normalization, an intermediate feature vector is obtained. In the third-layer perceptron, for Perform a linear mapping to reduce its dimension to a single scalar s, then pass it through the Sigmoid activation function to normalize the scalar s to the interval (0,1), and output the final anomaly confidence score residual_logits.

[0021] Preferably, the comprehensive anomaly score Where α is a weight balancing parameter used to control the contribution ratio of anomaly confidence (residual_logits) to semantic difference score (token_diff_score).

[0022] Preferably, the method for performing adaptive byte pair encoding merging operation on sub-word units is as follows: Set a limit of merging rounds K, and perform a maximum of K rounds of adaptive byte pair encoding merging operation on the sub-word unit. In each round of merging operation, traverse all adjacent character pairs in the current sub-word unit. For each adjacent character pair, query the sorting position of the adjacent character pair in the pre-built merging priority table, select the character pair with the highest sorting position of the adjacent character pair as the optimal merging object for this round, and merge it with the adjacent character pair. If the length of the current sub-word unit is 1, there are no adjacent character pairs in the previous sub-word unit, or the number of iterations reaches the maximum number of merging times K, then terminate the iteration. Finally, output the merged sub-word unit.

[0023] Preferably, the final output is the merged sub-word unit and the word attribution sequence aligned with the sub-word unit. The word attribution sequence is used for the whole word masking task in the pre-training stage, and a masking operation is uniformly applied to all sub-words of the same word as a word-level supervision signal.

[0024] Preferably, the normal prompt text is a general template; the abnormal prompt text is text generated by adding an abnormal description suffix to the normal prompt text using semantic splicing technology.

[0025] This invention addresses the shortcomings of existing vision-language model-based industrial anomaly detection methods in terms of semantic modeling granularity, cross-modal alignment accuracy, and discriminative interpretability. It proposes an anomaly detection method that integrates improved vision-language joint representation, residual feature hinting engineering, and an interpretable scoring mechanism to achieve systematic optimization. The results are as follows.

[0026] First, the improved vision-language joint modeling enhances the stability and local perception capability of cross-modal semantic alignment. On the language side, adaptive lexical selection and whole-word-level contextual modeling strategies effectively preserve the semantic integrity of the prompt text, providing high-quality linguistic guidance signals for cross-modal matching. Together, these two approaches enable fine-grained alignment of text representations within a shared semantic space.

[0027] Second, the residual feature hinting engineering achieves explicit modeling of the semantic differences between normal and abnormal texts. The key hint extraction module uses learnable weight vectors and a Gumbel-Softmax soft selection mechanism to evaluate and weighted aggregate the importance of sub-word-level text features, generating key vectors representing the core semantics. The residual feature adapter adopts an unbiased two-layer bottleneck structure to nonlinearly enhance the residual between the key vector and the observed semantics, improving feature representation while maintaining the original semantic direction. Based on this enhanced residual feature, the residual learning module outputs the anomaly confidence probability through a lightweight multilayer perceptron. This enables the model to complete anomaly identification based on the semantic residual direction.

[0028] Third, the interpretability scoring mechanism provides a quantitative basis and traceability path for anomaly decision-making. The anomaly score output by the system is obtained by nonlinear mapping of residual features, and its value reflects the degree of semantic deviation. At the same time, the soft weights generated during the key hint extraction process can serve as indicators of the contribution of each semantic unit. This design enables the detection results to have both a probabilistic output form and support semantic interpretation at the feature level, which helps to establish credible and auditable judgment logic in industrial scenarios.

[0029] Fourth, the overall architecture adopts a modular design and does not rely on a specific backbone network. The residual hinting engineering and scoring mechanism can be embedded into the inference process of the existing CLIP vision-language alignment model, facilitating its transfer and application across different industrial inspection tasks. This method can maintain stable detection performance even under small sample training conditions and exhibits good robustness and generalization ability when facing real-world conditions such as complex materials, weak defects, or changes in illumination. Attached Figure Description

[0030] Figure 1 This is a flowchart of the method of the present invention.

[0031] Figure 2 The results of anomaly detection obtained on the public dataset MVTecAD using the method proposed in this invention.

[0032] Figure 3 The results of anomaly detection obtained on the public dataset Visa using the method proposed in this invention. Detailed Implementation

[0033] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0034] See Figure 1 As shown, a small-sample industrial anomaly detection method based on cue learning includes the following steps.

[0035] Step S1: Construct a word structure-aware adaptive word segmentation strategy.

[0036] To achieve anomaly detection in industrial images, this invention proposes a word structure-aware adaptive word segmentation strategy for the language side of the CLIP vision-language model, which has the following two key technological innovations.

[0037] Step S1-1 Adaptive Byte-Pair Encoding (BPE Algorithm): This step introduces a maximum merging count limit parameter K and, based on a pre-built byte-pair merging priority table (bpe_simple_vocab_16e6.txt.gz file), implements a controlled word combination process: First, the normal prompt text and the abnormal prompt text used for industrial image anomaly detection are split into character sequences, and word boundary markers "" are appended to the end to form initial word units. The normal prompt text is a manually designed general template (e.g., a photo of a {name}, where name is the type of item in the image to be detected); the abnormal prompt text is the industrial image judgment text to be detected, which is generated by adding an abnormal description suffix after the normal prompt using semantic splicing technology (e.g., "a photo of a {name} with defects"); then, parameter K is set to limit the number of subsequent merging rounds; finally, a maximum of K rounds of BPE merging operations are performed. By limiting the maximum number of iterations, the complexity of the BPE merging process can be effectively controlled, preventing excessively deep merging or convergence failures in extreme cases. Each round iterates through all adjacent character pairs in the current sub-word unit, queries their ranking position in the merge priority table generated from the vocabulary, and selects the character pair with the highest ranking (i.e., the highest merge priority) as the optimal merge target for this round. If there are no adjacent character pairs in the current sub-word unit, the iteration terminates. After each merge, the merge counter is incremented and the sub-word unit is updated as the input for the next round. When the length of the merged sub-word unit is 1, it indicates that further splitting is impossible, and the iteration ends early. This step terminates when the number of iterations reaches the maximum number of merges K, there are no mergeable character pairs, or the length of the merged sub-word is a single unit. Finally, the sub-word unit is concatenated with spaces to form the output string as the BPE encoding result. This method effectively avoids semantic fragmentation of compound terms caused by excessive subdivision by limiting the merging depth, thereby improving the model's expressive ability in fine-grained semantic scenarios.

[0038] Step S1-2, based on adaptive BPE, generates sub-word encoding and a synchronized mask sequence: Building upon step S1-1, this step further implements sub-word-level encoding and position-synchronized mask sequence construction for the input prompt text. Specifically, firstly, the normal and abnormal prompt texts are segmented according to a preset regular expression, and the basic word sequence is extracted. For the obtained word sequence, it is encoded into a byte sequence using UTF-8, and each byte is converted into a corresponding sub-word using a byte mapping table (constructing a bijective mapping from 0–255 byte values ​​to Unicode characters). Secondly, the adaptive BPE module described in step S1-1 is called to perform up to K rounds of merging operations on the byte word, generating the corresponding sub-word unit. The dictionary index corresponding to each output sub-word is recorded synchronously. Finally, this step outputs two sequences: one is the sub-word unit ID generated by the BPE word table mapping; the other is the word belonging sequence strictly aligned with the sub-word unit. The word attribution sequence can be used for whole-word masking tasks in the pre-training stage, applying the [MASK] operation uniformly to all sub-words of the same word as a word-level supervision signal.

[0039] Step S2: Establish a residual feature cueing project. This step aims to fine-tune the semantic representation of contextual cues and explicitly model the semantic shift between normal and abnormal categories, thereby calculating the probability distribution of the abnormal category. This project includes a key cue extraction module, a residual feature adapter, and a residual learning module. Its input is the feature vector containing contextual information generated by the CLIP text encoder from the word units output in steps S1-2, i.e., the contextual feature vector. This includes the contextual feature vector of the normal cue text. and the contextual feature vector of the error message text .

[0040] Step S2-1-1: The key prompt extraction module extracts the context feature vector of the acquired normal prompt text. ( The importance of features is evaluated using the number of sub-words (D being the feature dimension). The key prompt extraction module uses a learnable weight vector to evaluate the contextual feature vector of the normal prompt text. Each sub-word feature is linearly scored. Gumbel noise is introduced, specifically Gumbel-Softmax is used to generate normalized feature weights, and these weights are then used to sum the features of all sub-words to obtain a context key vector representing the semantics of the entire normal prompt text. .

[0041] Step S2-1-2: The residual feature adapter converts the context key vector of the normal prompt text semantics obtained in step S2-1-1 into a single vector. This method enhances the semantics through nonlinear representation. The residual feature adapter employs an unbiased, two-layer fully connected bottleneck structure, where the first linear layer compresses the dimension from D to [value missing]. (For example, D′=D / 4D'), after passing through the ReLU activation function, the second linear layer maps back to the original dimension D, outputting a contextual feature vector for normal prompt text enhancement. The calculation process of this module is as shown in formula (1): (1) in, Key vectors for normal text semantics In mathematical expressions, In the first layer of structure, It is a weight matrix. ReLU is an activation function that introduces a nonlinear transformation on the input. Dimensionality reduction is performed. In the second layer structure, Yes, it's a weight matrix. The dimensionality of the output that has already been reduced in the first layer structure is increased so that the final output dimension is consistent with the initial input.

[0042] Step S2-2-1: The key prompt extraction module extracts the contextual feature vector of the acquired abnormal prompt text. ( The importance of each sub-word feature is evaluated using the number of sub-words (D being the feature dimension). The key hint extraction module linearly scores each sub-word feature using a learnable weight vector. After introducing Gumbel noise, Gumbel-Softmax is used to generate normalized feature weights, and all sub-word features are weighted and summed accordingly to obtain the context key vector representing the semantics of the entire anomaly hint text. .

[0043] Step S2-2-2: The residual feature adapter uses the key vector of the anomaly prompt text semantics obtained in step S2-2-1. It employs a nonlinear representation to enhance semantics. This residual feature adapter uses an unbiased, two-layer fully connected bottleneck structure, where the first linear layer compresses the dimension from D to [value missing]. (For example, D′=D / 4D'), after passing through the ReLU activation function, the second linear layer maps back to the original dimension D, outputting the context feature vector for anomaly message residual enhancement. The calculation process for this module is as shown in formula (2): (2) in, It is the key vector of abnormal text semantics In mathematical expressions, In the first layer of structure, It is a weight matrix. ReLU is an activation function that introduces a nonlinear transformation on the input. Dimensionality reduction is performed. In the second layer structure, Yes, it's a weight matrix. The dimensionality of the output that has been reduced in the first layer structure is increased so that the final output dimension is consistent with the initial input.

[0044] Step S2-3: The residual learning module enhances the feature vector of the normal prompt text output in step S2-1-2. And the contextual feature vector of the anomaly prompt text residual enhancement in step S2-2-2 The difference is calculated to obtain the context residual feature vector ctx_res. ctx_res is then input into an anomaly alert discriminator, which employs a multilayer perceptron structure: in the first layer of the perceptron, the input vector is transformed linearly... Projecting onto a semantic space of a specified dimension yields the mapping result. Subsequently, regarding A ReLU activation function is applied to introduce a nonlinear feature transformation, and the feature distribution is stabilized through batch normalization to obtain intermediate feature representations. In the second-layer perceptron, for A linear mapping is performed to compress the feature dimension from the set 128 dimensions to 64 dimensions, further extracting deep semantic patterns related to the anomaly direction. After ReLU activation and batch normalization, an intermediate feature vector is obtained. In the third-layer perceptron, for A linear mapping is performed to reduce the dimension to a single scalar output s. Then, the scalar is normalized to the interval (0,1) using the Sigmoid activation function, outputting the final anomaly confidence probability residual_logits, which measures the potential anomaly of the prompt. The calculation process is shown in formula (3): (3) Where x is the mathematical representation of the context residual feature vector ctx_res. These are the weights of the first layer perceptron in the MLP. It is the bias of the first layer perceptron of the MLP. These are the weights of the second-layer perceptron in the MLP. It is the bias of the second-layer perceptron of the MLP. These are the weights of the third-layer perceptron in the MLP. It is the bias of the third layer perceptron in the MLP. It is the Sigmoid activation function, and the final output value of residual_logits is in the range of [0,1].

[0045] Step S3: Design an interpretability scoring mechanism. This scoring system integrates two key indicators to provide an efficient and interpretable basis for anomaly detection and identification.

[0046] Step S3-1: Semantic difference score is a measure of the semantic offset between the context of a normal prompt and the context of an abnormal prompt in the semantic space. The semantic deviation calculation module is used to calculate the context feature vector of the normal prompt text residual enhancement based on the output of step S2-1-2. The contextual feature vector of the anomaly prompt text residual enhancement output in step S2-2-2. The cosine distance between the two is used to calculate the degree of deviation between the two in the semantic space, thus characterizing the semantic offset between the normal and abnormal prompts, represented by the token_diff_score. If the token_diff_score is small, it means that the abnormal prompt and the normal prompt are very close in the semantic space; if the token_diff_score is large, it means that the abnormal prompt differs significantly from the normal semantics, and the model can more easily identify potential anomalies.

[0047] Step S3-2: The holistic score is an ensemble metric used to comprehensively measure anomaly confidence. This score integrates the anomaly confidence score `residual_logits` and the semantic difference score `token_diff_score` output from step S2-3, and fuses them using a weighted mechanism to finally output a holistic anomaly score normalized to the [0,1] interval. First, the semantic difference score `token_diff_score` is normalized to its maximum value, with a small constant of 1e-6 added during normalization to prevent division by zero errors, resulting in a new variable `token_score_norm` with a value between [0,1]. Second, weights are assigned to the anomaly confidence score `residual_logits` and the semantic difference score `token_diff_score` using an equal weighting method. Finally, the key dimension information of the residual matching score (`residual_logits`) and `token_score_norm` is integrated using addition to obtain the holistic score. Using this comprehensive scoring mechanism, the anomaly confidence score can effectively support anomaly localization. The calculation process is shown in formula (4): (4) Here, α is a weighting parameter used to control the contribution ratio of anomaly confidence (residual_logits) and semantic difference score (token_diff_score). For example, in this embodiment, α is set to 0.5, meaning that semantic confidence and contextual difference each account for 50% of the weight.

[0048] This comprehensive score directly reflects the consistency between the industrial anomaly detection image results and the categories described in the anomaly warning text. A higher score indicates stronger consistency, while a lower score indicates weaker consistency.

[0049] Although specific embodiments of the invention have been disclosed for illustrative purposes to aid in understanding and implementing the invention, those skilled in the art will understand that various substitutions, variations, and modifications are possible without departing from the spirit and scope of the invention and the appended claims. Therefore, the invention should not be limited to the content disclosed in the preferred embodiments, and the scope of protection claimed by the invention is defined by the claims.

Claims

1. A small-sample industrial anomaly detection method based on cue learning, comprising the following steps: 1) The normal prompt text and the abnormal prompt text used for industrial image anomaly detection are split into character sequences by word, and word boundary markers "" are added to the end of each sequence to form initial sub-word units. The abnormal prompt text is the industrial image judgment text to be detected. An adaptive byte pair encoding merging operation is performed on the sub-word units, and the merged sub-word units are output. 2) Encode the sub-word units using a text encoder to generate a context feature vector; the context feature vector includes the context feature vector of the normal prompt text. and the contextual feature vector of the error message text ; 3) Utilize the key prompt extraction module to extract the contextual feature vector of the normal prompt text. An importance assessment is performed to obtain a contextual key vector representing the semantics of the entire normal prompt text. ;Utilize residual feature adapters to apply the context key vectors By performing a non-linear representation to enhance semantics, we obtain a contextual feature vector that enhances the normal prompt text. ; and utilize the key prompt extraction module to extract contextual feature vectors from abnormal prompt text. An importance assessment is performed to obtain a contextual key vector representing the semantics of the entire anomaly alert text. Using residual feature adapters to analyze the key vectors of anomaly alert text semantics By performing a nonlinear representation to enhance semantics, we obtain a contextual feature vector for residual enhancement of the anomaly prompt text. ; 4) Utilize the residual learning module to process the context feature vectors Context feature vector The difference is calculated to obtain the context residual feature vector ctx_res, which is then input into the anomaly alert discriminator to obtain the anomaly confidence score residual_logits; 5) Calculate the context feature vector using the semantic deviation calculation module. With context feature vector The cosine distance between them is used as the semantic difference score token_diff_score; a comprehensive anomaly score is calculated based on the anomaly confidence score residual_logits and the semantic difference score token_diff_score. 6) Determine whether the industrial image to be detected is consistent with the category described in the anomaly prompt text based on the comprehensive anomaly score.

2. The method according to claim 1, characterized in that, The residual feature adapter employs an unbiased, two-layer fully connected bottleneck structure, where the first linear layer compresses the input D-dimensional feature vector into... The original dimension D is then mapped back through the ReLU activation function and the second linear layer.

3. The method according to claim 2, characterized in that, D′=D / 4D'.

4. The method according to claim 1, 2, or 3, characterized in that, The key prompt extraction module linearly scores each word feature in the input feature vector using a learnable weight vector, then uses Gumbel-Softmax to generate normalized feature weights and sums them up for all word features in the input feature vector to obtain the context key vector of the feature vector.

5. The method according to claim 1, 2, or 3, characterized in that, The anomaly detection discriminator employs a multilayer perceptron structure: in the first-layer perceptron, the input vector is transformed through a linear transformation. Projecting onto a semantic space of a specified dimension yields the mapping result. Then on A nonlinear feature transformation is performed by applying the ReLU activation function, and the feature distribution is stabilized by batch normalization to obtain intermediate feature representations. In the second-layer perceptron, for A linear mapping is performed to compress the feature dimension, extracting deep semantic patterns related to the anomaly direction. After ReLU activation and batch normalization, an intermediate feature vector is obtained. ; In the third-layer perceptron, for Perform a linear mapping to reduce its dimension to a single scalar s, then pass it through the Sigmoid activation function to normalize the scalar s to the interval (0,1), and output the final anomaly confidence score residual_logits.

6. The method according to claim 1, 2, or 3, characterized in that, The comprehensive anomaly score Where α is a weight balancing parameter used to control the contribution ratio of anomaly confidence (residual_logits) to semantic difference score (token_diff_score).

7. The method according to claim 1, characterized in that, The method for performing adaptive byte pair encoding merging operation on sub-word units is as follows: Set a limit of merging rounds K, and perform a maximum of K rounds of adaptive byte pair encoding merging operation on the sub-word unit. In each round of merging operation, traverse all adjacent character pairs in the current sub-word unit. For each adjacent character pair, query the sorting position of the adjacent character pair in the pre-built merging priority table, select the character pair with the highest sorting position of the adjacent character pair as the optimal merging object for this round, and merge it with the adjacent character pair. If the length of the current sub-word unit is 1, there are no adjacent character pairs in the previous sub-word unit, or the number of iterations reaches the maximum number of merging times K, then terminate the iteration. Finally, output the merged sub-word unit.

8. The method according to claim 7, characterized in that, The final output is the merged sub-word unit and the word attribution sequence aligned with the sub-word unit. The word attribution sequence is used for the whole word masking task in the pre-training stage, and a masking operation is uniformly applied to all sub-words of the same word as a word-level supervision signal.

9. The method according to claim 1, characterized in that, The normal prompt text is a general template; the abnormal prompt text is text generated by adding an abnormal description suffix to the normal prompt text using semantic splicing technology.

Images