A Visual Language Large Model for Industrial Anomaly Detection Integrating Adaptive Cueing Learning

Abstract

This invention relates to the field of industrial anomaly detection technology, and more particularly to an industrial anomaly detection method integrating adaptive prompting learning with a large visual language model. The method includes: extracting features from an industrial image to be detected; encoding a pre-set semantic text input into a text encoder of a pre-trained visual language model; decoding the image features and text features input into an image decoder of the pre-trained visual language model; modeling the abnormal visual information in the pixel-level anomaly response map using an adaptive prompting module to obtain a final prompting representation adaptively matched to the current industrial scene; concatenating the final prompting representation with image features, and inputting the multimodal joint features into a large language model for autoregressive decoding processing, outputting text information containing industrial anomaly judgment results and anomaly region descriptions. This invention can combine abnormal visual information to generate scene-adaptive prompts and output anomaly descriptions, improving the accuracy of industrial anomaly detection.

Description

Technical Field

[0001] This invention relates to the field of industrial anomaly detection technology, and in particular to an industrial anomaly detection method that integrates adaptive cue learning with a large visual language model. Background Technology

[0002] Industrial anomaly detection is a key technology in intelligent manufacturing and industrial vision inspection. Its main objective is to automatically identify and locate surface defects, structural anomalies, and process deviations during production, which is crucial for ensuring production quality and improving efficiency. With the increasing complexity of manufacturing equipment and the growing diversity of products, industrial anomaly detection systems face numerous challenges in practical applications, including sample scarcity, diverse anomaly types, and dynamic changes in the production environment. This places higher demands on the robustness and generalization ability of detection algorithms. Currently, most existing methods are based on traditional machine vision or deep learning technologies. However, in real-world industrial scenarios, anomaly samples often exhibit low incidence and high uncertainty. Obtaining large-scale, fully labeled anomaly data is not only costly but also difficult to achieve in some special scenarios. To reduce reliance on anomaly samples, existing technologies have proposed anomaly detection strategies based solely on normal sample modeling, identifying anomalies by learning the feature distribution of normal samples. However, anomaly detection strategies based solely on normal sample modeling are typically highly dependent on specific products or scenarios. When the detection object, production process, or environment changes, the model needs to be retrained or significantly adjusted, resulting in poor engineering adaptability and difficulty in meeting the needs of flexible production.

[0003] In recent years, visual language models, leveraging their advantage of jointly modeling visual and semantic information, have demonstrated significant potential in cross-modal understanding and generalization capabilities, providing a new technical path for industrial anomaly detection. However, in industrial applications, existing visual language models often rely on fixed or manually designed text prompts to guide model reasoning, making it difficult to accurately depict the subtle, localized, and non-semantic features commonly found in industrial anomalies. This limits the model's detection performance in complex industrial scenarios. Furthermore, existing industrial anomaly detection results are mostly output in numerical form, lacking semantic descriptions of the causes and regions of anomalies. This hinders maintenance personnel from quickly understanding the anomaly and making decisions, and also restricts the further integration and application of anomaly detection results in intelligent manufacturing systems.

[0004] Chinese Patent Publication No. CN121121767B discloses a zero-shot anomaly detection method and system based on a triple-perception learning-enhanced visual language model. The method includes: extracting global and local visual features from an input image; correcting local features in a deep network through a spatial perception attention enhancement module during the visual encoding process, generating fine-grained attribute text descriptions for anomalous visual features; performing deep semantic alignment between the attribute text descriptions and general text prompts through an attribute perception guidance module; calculating the similarity between the enhanced visual features and optimized text features to generate a pixel-level anomaly segmentation map; and in the inference stage, converting the segmentation map into spatial attention weights through an anomaly perception reconstruction module, feeding them back to the visual encoder to generate the final global feature representation and calculate anomaly scores. This approach only extracts image and text features and obtains anomaly detection results through similarity calculation. It cannot combine anomalous visual information to generate scene-adaptive prompts and output anomaly descriptions, resulting in a lack of scene adaptability in industrial anomaly detection and reduced accuracy. Summary of the Invention

[0005] To address this issue, the present invention provides a visual language large model-based industrial anomaly detection method that integrates adaptive prompting learning. This method overcomes the problem in existing technologies where the inability to combine abnormal visual information to generate scene-adaptive prompts and output anomaly descriptions leads to a lack of scene adaptability in industrial anomaly detection and reduces its accuracy.

[0006] To achieve the above objectives, this invention provides a method for industrial anomaly detection based on a large visual language model that integrates adaptive cue learning, comprising the following steps:

[0007] S1. Input the industrial image to be detected into the image encoder of the pre-trained visual language model to extract features and obtain image features;

[0008] S2. Input the preset semantic text into the text encoder of the pre-trained visual language model to encode it and obtain text features;

[0009] S3. Input the image features and text features into the image decoder of the pre-trained visual language model for decoding, and generate industrial image anomaly detection results containing image-level anomaly scores and pixel-level anomaly response maps;

[0010] S4. Construct an adaptive prompting module, and model the abnormal visual information in the pixel-level abnormal response map through the adaptive prompting module to obtain the final prompting representation that adaptively matches the current industrial scene.

[0011] S5. The final prompt representation is concatenated with image features to obtain multimodal joint features. The multimodal joint features are then input into a large language model for autoregressive decoding processing, and text information containing industrial anomaly determination results and anomaly region descriptions is output.

[0012] The technical principle of this application is as follows: After obtaining the pixel-level anomaly response map, the adaptive prompting process dynamically filters and reorganizes the image features of the corresponding positions output by the image encoder based on the visual feature distribution of the high-response areas in the pixel-level anomaly response map. Then, through a learnable feature mapping and attention pooling operation, the pixel-level anomaly response map is compressed into a set of final prompting representations that adaptively match the current industrial scene. This final prompting representation no longer relies on fixed manually designed text templates or general category words, but is directly generated by the pixel features of the visual anomaly area, realizing an end-to-end implicit mapping from the anomaly visual pattern to the language prompting space. Subsequently, the final prompting representation and image features are spliced ​​and fused in the feature dimension to construct a multimodal joint feature containing visual cues of the anomaly area. This multimodal joint feature is input into a large language model for autoregressive decoding. Based on the visual semantic constraints in the multimodal joint feature, the large language model generates text information containing the industrial anomaly judgment result and the specific description of the corresponding anomaly area word by word.

[0013] Compared with existing technologies, the advantages of this application are as follows: By constructing an adaptive prompting module, the abnormal visual information in the pixel-level abnormal response map is modeled, which can generate a final prompting representation that adaptively matches the current industrial scene, making the final prompting representation more consistent with the specific characteristics of industrial abnormalities and improving scene adaptability; by concatenating the final prompting representation with image features to obtain multimodal joint features, the visual abnormality information and prompting semantic information are integrated, enhancing the correlation of multimodal features; by inputting the multimodal joint features into a large language model for autoregressive decoding, the textual information of industrial abnormality judgment results and abnormal area descriptions is output, clearly presenting the specific situation of the abnormality and effectively improving the accuracy of industrial abnormality detection.

[0014] Furthermore, S1 includes the following steps:

[0015] S11. Preprocess the industrial image to be detected to obtain a preprocessed industrial image, and input the preprocessed industrial image into the image encoder for image segmentation and linear projection to obtain an image block embedding sequence.

[0016] S12. The image block embedding sequence is input into multiple coding layers of the image encoder for hierarchical coding to obtain the intermediate feature map corresponding to each coding layer.

[0017] S13. Perform global average pooling on the intermediate feature map corresponding to each coding layer to obtain the global feature vector corresponding to each coding layer, and calculate the attention weight corresponding to each coding layer based on the global feature vector.

[0018] S14. Based on the attention weights corresponding to each coding layer, the intermediate feature maps corresponding to the coding layer are weighted and fused to obtain a fused feature map. The fused feature map is then subjected to layer normalization and spatial flattening to obtain image features.

[0019] In this scheme, by performing multi-level encoding and attention-weighted fusion on the industrial images to be inspected, multi-scale semantic features of the industrial images to be inspected are effectively extracted, key regional information is highlighted, irrelevant noise is suppressed, the discriminative power and robustness of feature expression are enhanced, and the accuracy of industrial defect detection is improved.

[0020] Furthermore, S2 includes the following steps:

[0021] S21. Obtain a preset semantic text containing normal state semantic description text and abnormal state semantic description text, and perform word segmentation and tagging processing on the normal state semantic description text and abnormal state semantic description text respectively to obtain the corresponding normal state tagging sequence and abnormal state tagging sequence.

[0022] S22. Based on the text encoder of the pre-trained visual language model, perform context encoding on the normal state label sequence and the abnormal state label sequence respectively to obtain the corresponding normal state label embedding sequence and abnormal state label embedding sequence.

[0023] S23. Perform mean pooling on the embedding vectors corresponding to all label positions in the normal state label embedding sequence to obtain the global semantic vector of the normal state, and perform mean pooling on the embedding vectors corresponding to all label positions in the abnormal state label embedding sequence to obtain the global semantic vector of the abnormal state.

[0024] S24. Compare and perform semantic modulation processing on the global semantic vector of the normal state and the global semantic vector of the abnormal state to obtain the modulated semantic vector. Then, perform layer normalization and affine transformation processing on the modulated semantic vector to obtain text features that match the dimensional structure of image features.

[0025] In this scheme, by acquiring and processing semantic description texts of normal and abnormal states, word segmentation and context encoding are used to fully explore the contextual relationships of semantics. Then, mean pooling is used to obtain global semantic vectors of normal and abnormal states, which can accurately distinguish the differences between normal and abnormal semantics. Contrastive semantic modulation is used to enhance the discriminativeness of semantic features. Combined with layer normalization and affine transformation, the dimensional structure of text features and image features are matched, which can improve the regularity and adaptability of text semantic features, making text features more in line with the expression needs of visual features.

[0026] Furthermore, S3 includes the following steps:

[0027] S31. Use image features as a query matrix, text features as a key matrix and a value matrix, and input the query matrix, key matrix and value matrix into the cross-attention layer of the image decoder for feature interaction to obtain the aligned image feature matrix;

[0028] S32. Perform pixel-by-pixel similarity analysis on the text features and the aligned image feature matrix to obtain the similarity matrix;

[0029] S33. Perform bilinear interpolation upsampling on the similarity matrix to obtain a pixel-level anomaly response map corresponding to the spatial size of the industrial image to be detected. Analyze the pixel-level anomaly response map to obtain an image-level anomaly score. Output the pixel-level anomaly response map and the image-level anomaly score as the industrial image anomaly detection result.

[0030] In this scheme, high-resolution pixel-level anomaly response maps and image-level scores are generated through cross-attention interaction of image features and text features and pixel-by-pixel similarity analysis, thereby achieving accurate anomaly localization and overall judgment and improving detection accuracy.

[0031] Furthermore, in step S32, the mathematical expression for pixel-by-pixel similarity analysis is: In the formula, The text features and the aligned image feature matrix are represented at pixel positions. Similarity value at, This indicates the pixel position in the aligned image feature matrix. The feature vector at that location, Represents the text feature vector. This indicates the pixel position in the aligned image feature matrix. eigenvectors at location The length of the mold, Represents text feature vectors The length of the mold, This represents a constant used to prevent the denominator from being zero.

[0032] In this scheme, normalized dot product similarity calculation is used to stably measure the matching degree between image features and text features of each pixel, and a constant is introduced to prevent the denominator from being zero, thereby improving numerical stability and achieving accurate and robust pixel-level anomaly localization.

[0033] Furthermore, in S33, image-level anomaly scoring... The mathematical expression is: In the formula, Represents a pixel-level anomaly response map. These represent the weighting coefficients used to balance the maximum anomalous response and statistical characteristics. This represents the maximum value in the pixel-level anomaly response graph. This represents the arithmetic mean of the pixel-level anomaly response map. This represents the standard deviation in the pixel-level anomaly response graph. This represents the weighting factor of the arithmetic mean term. This represents the weighting coefficient of the standard deviation term.

[0034] This solution combines the maximum anomaly response with overall statistical features to balance local significant anomalies with global distribution information, effectively improving the accuracy and reliability of image-level anomaly scoring. It more comprehensively reflects the overall anomaly situation of the image, avoids the one-sidedness problem caused by single feature evaluation, and makes the anomaly recognition results more in line with the needs of actual scenarios.

[0035] Furthermore, S4 includes the following steps:

[0036] S41. Construct an adaptive prompting module containing a meta-prompt generation network. Use the variable convolution in the meta-prompt generation network to perform feature aggregation on spatial regions in the industrial image anomaly detection results where the abnormal response value is higher than the preset adaptive threshold, and obtain an initial region prompting set.

[0037] S42. The initial region cue set is weighted based on the spatial attention mechanism to obtain the weighted region cue vector;

[0038] S43. Perform adaptive linear fusion of the weighted regional prompt vector and the preset basic prompt, and output the final prompt representation that matches the current industrial scene.

[0039] In this solution, an adaptive prompting module with a numeric prompting generation network is constructed. Variable convolution is used to aggregate abnormal responses and highlight regional features. The prompting vector is then optimized through spatial attention weighting. Combined with adaptive fusion of basic prompts, a prompting representation that fits the current industrial scenario can be generated. This accurately focuses on abnormal regions, improves the scenario adaptability and targeting of prompts, and makes the prompting guidance for industrial image anomaly detection more accurate and effective.

[0040] Furthermore, in S42, the mathematical expression for the spatial attention mechanism is: In the formula, This represents the weighted region hint vector. This represents the total number of region cue vectors in the initial region cue set. Indicates the first The query vector is obtained by linearly transforming the region suggestion vector. Indicates the first The key vector is obtained by linear transformation of the region hint vector. Indicates the first The transpose matrix of the key vector obtained by linear transformation of the region hint vector. The dimension of the key vector. Represented by natural constant An exponential function with base 0. Indicates the first The query vector is obtained by linearly transforming the region suggestion vector. Indicates the first The key vector is obtained by linear transformation of the region hint vector. Indicates the first The transpose matrix of the key vector obtained by linear transformation of the region hint vector. Indicates the first The value vector obtained by linear transformation of the region hint vector.

[0041] In this scheme, by performing linear transformation correlation and exponential normalization weighted allocation on the cue vectors of each region, the importance of different region cue can be accurately distinguished, key effective features are automatically strengthened and redundant interference information is weakened, so that the weighted region cue vector representation is more accurate and focused, and the feature discrimination and expression effectiveness are improved.

[0042] Furthermore, S5 includes the following steps:

[0043] S51. Based on the channel dimension, the final prompt representation and image features are concatenated to obtain multimodal joint features. The multimodal joint features are then input into the embedding projection layer of the large language model for dimension alignment to obtain the dimension-aligned feature sequence.

[0044] S52. Input the dimension-aligned feature sequence into the decoder network of the large language model for autoregressive inference to obtain the conditional probability distribution of each output position.

[0045] S53. Decode the conditional probability distribution of each output position using a greedy search strategy, and output text information containing industrial anomaly judgment results and anomaly area descriptions.

[0046] In this solution, multimodal joint features are formed by splicing cue representations and image features along the channel dimension. Dimensional alignment is completed by embedding a projection layer, and then autoregressive inference is performed using a decoder network. Through greedy search decoding, text information containing industrial anomaly judgment results and anomaly region descriptions can be generated, achieving efficient fusion of multimodal features, making the detection results intuitive and readable, and accurately outputting standardized text descriptions of industrial anomalies.

[0047] Furthermore, in S52, the mathematical expression for autoregressive inference is: In the formula, This represents the text tag generated at time step t. This represents the historical text sequence generated before time step t. This represents the dimension-aligned feature sequence. This represents the output at time step t, given a known historical text sequence and dimensionally aligned feature sequence. The conditional probability distribution, The decoder network representing the large language model is in the first... The hidden state vector output at each time step Indicates the first The hidden state vector output at each time step Feature sequences aligned with dimensions The joint representation obtained by splicing. The representation is used for joint representation. The weight matrix for linear transformation. Indicates will The output is mapped to the weight matrix of the vocabulary space. This represents the bias vector of the hidden layer in the decoder network. This represents the bias vector of the output layer in the decoder network. This represents a function that transforms an input vector into a probability distribution. This represents the hyperbolic tangent activation function.

[0048] In this scheme, by fusing the feature information of historical text sequences and dimensional alignment, and processing it with linear transformation and activation functions, the conditional probability distribution of each text tag is accurately calculated. This improves the logical coherence of text generation, makes the lexical space mapping more in line with actual needs, ensures the stable and reliable output of the decoder, and enhances the accuracy of text information generation. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating an industrial anomaly detection method based on a large visual language model incorporating adaptive prompting learning, according to an embodiment of the present invention.

[0050] Figure 2 This is a schematic diagram of the framework of an industrial anomaly detection method based on a large visual language model that integrates adaptive prompting learning, according to an embodiment of the present invention.

[0051] Figure 3 This is a schematic diagram of the adaptive prompting learning process according to an embodiment of the present invention;

[0052] Figure 4 This is a schematic diagram of the ablation experiment results in this embodiment;

[0053] Figure 5 This is a schematic diagram illustrating the experimental results of comparing different large language models in this embodiment. Detailed Implementation

[0054] The following detailed description illustrates the specific implementation method:

[0055] like Figure 1 The diagram shown illustrates a flowchart of an industrial anomaly detection method based on a large visual language model that integrates adaptive cue learning, comprising the following steps:

[0056] S1. Input the industrial image to be detected into the image encoder of the pre-trained visual language model to extract features and obtain image features;

[0057] S2. Input the preset semantic text into the text encoder of the pre-trained visual language model to encode it and obtain text features;

[0058] S3. Input the image features and text features into the image decoder of the pre-trained visual language model for decoding, and generate industrial image anomaly detection results containing image-level anomaly scores and pixel-level anomaly response maps;

[0059] S4. Construct an adaptive prompting module, and model the abnormal visual information in the pixel-level abnormal response map through the adaptive prompting module to obtain the final prompting representation that adaptively matches the current industrial scene.

[0060] S5. The final prompt representation is concatenated with image features to obtain multimodal joint features. The multimodal joint features are then input into a large language model for autoregressive decoding processing, and text information containing industrial anomaly determination results and anomaly region descriptions is output.

[0061] like Figure 2The diagram illustrates the framework of an industrial anomaly detection method based on a large visual language model integrating adaptive prompting learning, according to an embodiment of the present invention. It employs a two-stage architecture of unsupervised training and few-shot inference, with the core achieving accurate anomaly detection through multimodal feature interaction and dynamic prompt enhancement. During the training phase, a frozen text encoder encodes normal / abnormal text descriptions, while a frozen image encoder extracts visual features from both the query image and normal images. Normal image features are stored in a repository, matched with query image features using cosine similarity, and then fed into a learnable adaptive prompting learner along with text features via a linear layer to generate appropriate dynamic prompts. Simultaneously, text features are multiplied by a matrix and projected through a linear layer, then added element-wise with image features and input into a learnable image decoder to generate an anomaly detection result image, completing unsupervised training. During the inference phase, the adaptive prompting learner transforms multimodal features into prompts adapted to the large language model, which are then fed into the frozen large language model along with the input text, outputting anomaly detection results in natural language form and generating an anomaly detection result image for visual localization. This application achieves industrial anomaly detection with both semantic interpretability and localization accuracy in small sample scenarios by bridging visual features and large language models through adaptive prompting learning and combining normal feature constraints from the repository.

[0062] Specifically, S1 includes the following steps:

[0063] S11. Preprocess the industrial image to be detected to obtain a preprocessed industrial image, and input the preprocessed industrial image into the image encoder for image segmentation and linear projection to obtain an image block embedding sequence.

[0064] S12. The image block embedding sequence is input into multiple coding layers of the image encoder for hierarchical coding to obtain the intermediate feature map corresponding to each coding layer.

[0065] S13. Perform global average pooling on the intermediate feature maps corresponding to each coding layer to obtain the global feature vector corresponding to each coding layer, and calculate the attention weight corresponding to each coding layer based on the global feature vector. The mathematical expression for the attention weight corresponding to each coding layer is as follows: In the formula, Indicates the first Attention weights corresponding to the layer encoding layer Indicates the first The intermediate feature map corresponding to the layer encoding layer. This represents the activation function. This indicates a global average pooling operation. Indicates the first The learnable weight vector corresponding to the layer encoding layer, Indicates the first The learnable bias terms corresponding to the layer encoding layer;

[0066] S14. Based on the attention weights corresponding to each coding layer, the intermediate feature maps corresponding to that coding layer are weighted and fused to obtain a fused feature map. The fused feature map is then subjected to layer normalization and spatial flattening to obtain image features. The fused feature map... The mathematical expression is: In the formula, Indicates the total number of coding layers. This indicates a bilinear interpolation upsampling operation.

[0067] Further, in step S11, Gaussian filtering is first used to remove random noise from the industrial image to be detected, resulting in a denoised industrial image. The size of the filter kernel is determined according to the noise intensity of the industrial image to be detected. For example, when the noise is low, a 3×3 filter kernel is selected. Then, pixel value normalization processing is performed on the denoised industrial image to map the pixel values ​​to the 0-1 range. The normalization parameter is determined according to the pixel value range of the image. For example, when the pixel value range of the image is 0-255, normalization is achieved by dividing the pixel value by 255, resulting in a preprocessed industrial image. The preprocessed industrial image is then input into an image encoder. The image encoder divides the preprocessed industrial image into blocks according to a preset block size. The preset block size is set according to the image resolution and feature extraction requirements. For example, a 16×16 pixel image block is selected. Then, each image block is mapped to a feature vector of a fixed dimension through a linear projection layer. The learnable parameters of the linear projection layer are determined according to the image block size and the feature vector dimension. For example, when the image block is 16×16×3, the image block is projected into a feature vector of dimension 768, and finally the image block embedding sequence is obtained.

[0068] In step S12, the total number of encoding layers is set according to the depth requirements of feature extraction, for example, 6 encoding layers. Each encoding layer includes a multi-head self-attention mechanism and a feedforward neural network. The number of heads in the multi-head self-attention mechanism is determined according to the feature attention requirements, for example, 12 heads are set for the multi-head self-attention mechanism. The hidden layer dimension of the feedforward neural network is set according to the feature processing capability, for example, the hidden layer dimension of the first feedforward neural network is set to 3072. The image patch embedding sequence passes through each encoding layer sequentially. The shallow encoding layer mainly extracts the basic texture features of the image, while the deep encoding layer focuses on extracting the high-level semantic features of the image. After passing through each encoding layer, the corresponding intermediate feature map is output. The size of the intermediate feature map is determined according to the downsampling ratio of the encoding layer. For example, the size of the intermediate feature map output by the shallow encoding layer is 64×64, and the size of the intermediate feature map output by the deep encoding layer is 8×8.

[0069] In step S13, the average value of all pixel values ​​in each channel of each intermediate feature map is calculated to obtain a single value. The values ​​of all channels are combined to form the global feature vector corresponding to that encoding layer. The dimension of the global feature vector is the same as the number of channels in the intermediate feature map. The number of channels is determined according to the feature extraction requirements; for example, the number of channels in the intermediate feature map is set to 256. Subsequently, the attention weights corresponding to each encoding layer are calculated based on the global feature vector. The learnable weight vector and learnable bias term are determined according to the model training requirements and initial values ​​are set through random initialization, such as the learnable weight vector. The dimension is 1×256, the initial values ​​follow a normal distribution N(0, 0.01), and the learnable bias term... The initial value is set to 0.

[0070] In step S14, the intermediate feature maps corresponding to each coding layer are first scaled to the same size using bilinear interpolation upsampling. The upsampling factor is set according to the size of the deepest intermediate feature map; for example, if the deepest intermediate feature map is 8×8, all intermediate feature maps are upsampled to 64×64. Then, each upsampled intermediate feature map is multiplied pixel-by-pixel with its corresponding attention weight, and all results are summed pixel-by-pixel to obtain a fused feature map. The fused feature map is then subjected to layer normalization, with the parameters determined based on the number of channels in the fused feature map. The normalized fused feature map is then spatially flattened, converting the two-dimensional feature map into a one-dimensional vector. The length of the one-dimensional vector is the product of the size of the fused feature map and the number of channels, ultimately yielding the image features. The dimension of the image features is determined according to the actual application requirements; for example, the dimension of the image features is set to 4096.

[0071] Specifically, S2 includes the following steps:

[0072] S21. Obtain a preset semantic text containing normal state semantic description text and abnormal state semantic description text, and perform word segmentation and tagging processing on the normal state semantic description text and abnormal state semantic description text respectively to obtain the corresponding normal state tagging sequence and abnormal state tagging sequence.

[0073] S22. Based on the text encoder of the pre-trained visual language model, perform context encoding on the normal state label sequence and the abnormal state label sequence respectively to obtain the corresponding normal state label embedding sequence and abnormal state label embedding sequence.

[0074] S23. Perform mean pooling on the embedding vectors corresponding to all label positions in the normal state label embedding sequence to obtain the global semantic vector of the normal state, and perform mean pooling on the embedding vectors corresponding to all label positions in the abnormal state label embedding sequence to obtain the global semantic vector of the abnormal state.

[0075] S24. Perform semantic modulation processing on the global semantic vectors of the normal state and the abnormal state to obtain the modulated semantic vector. Then, perform layer normalization and affine transformation on the modulated semantic vector to obtain text features that match the dimensional structure of the image features. Among them, the modulated semantic vector... The mathematical expression is: In the formula, Represents the global semantic vector of the normal state. Represents the global semantic vector of abnormal states. Represents the modulation balance coefficient. This represents the element-wise multiplication operator for vectors. This represents the vector concatenation operator. This represents the L2 norm of a vector.

[0076] Further, in step S21, the specific content of the normal state semantic description text is preset according to the requirements of the industrial inspection scenario. For example, the content of the normal state semantic description text is "the surface is smooth without scratches and the color is uniform", and the content of the abnormal state semantic description text is "the surface has scratches or uneven color". After obtaining the preset semantic text, the normal state semantic description text and the abnormal state semantic description text are segmented and labeled by a word-based word segmentation method. The word segmentation tool uses a sub-word segmenter adapted to the pre-trained visual language model. The preset word list size inside the sub-word segmenter is determined according to the training stage of the pre-trained visual language model. For example, the preset word list size is 49408 words. During the word segmentation process, the continuous text sequence is segmented according to the byte-to-byte encoding merging rule. At the same time, a start marker is added to the beginning of the segmented word sequence and an end marker is added to the end, finally obtaining the corresponding normal state label sequence and abnormal state label sequence.

[0077] In step S22, the pre-trained visual language model is determined based on task requirements and data volume. For example, the CLIP (Contrastive Language-Image Pre-training) model is selected as the pre-trained visual language model. The CLIP model's text encoder adopts a Transformer architecture, containing 6 encoder layers, a hidden layer dimension of 512, and 8 attention heads. The normal state label sequence and the abnormal state label sequence are respectively input into the text encoder of the pre-trained visual language model. The text encoder captures the contextual association between words in the sequence through a self-attention mechanism, extracts and encodes features for each label, and outputs the normal state label embedding sequence and the abnormal state label embedding sequence after linear transformation and activation function processing, respectively. The embedding vector dimension of each label in the embedding sequence is 512.

[0078] In step S23, the mean pooling operation includes: summing the embedding vectors at all marker positions in the normal state marker embedding sequence element by element along the corresponding feature dimension and then dividing by the total number of marker positions, thereby aggregating the marker embedding sequence of arbitrary length into a normal state global semantic vector with a fixed dimension. The dimension of the normal state global semantic vector is consistent with the hidden dimension of the text encoder, for example, 768 dimensions. The same mean pooling operation is applied to the embedding vectors corresponding to all marker positions in the abnormal state marker embedding sequence, that is, the embedding vectors corresponding to each marker position in the abnormal state marker embedding sequence are accumulated along the dimension and averaged to obtain an abnormal state global semantic vector with the same dimension as the normal state global semantic vector.

[0079] In step S24, the modulation balance coefficient The modulation balance coefficient is pre-set as a fixed constant based on the required contrast strength between normal and abnormal semantics. The value is set to 0.7 to emphasize the preservation of semantic information in the normal state. The specific operations of the contrast semantic modulation processing include: first, calculating the product modulation term of the global semantic vector of the abnormal state and the global semantic vector of the normal state at the corresponding element position; then, weighting and combining the global semantic vector of the normal state with the product modulation term and normalizing and scaling it using the L2 norm of the concatenated vectors to generate a modulated semantic vector with contrast enhancement characteristics; subsequently, performing a layer normalization operation on the modulated semantic vector. The layer normalization layer contains learnable scaling parameters and offset parameters. For example, the scaling parameters are initialized to a vector of all 1s and the offset parameters are initialized to a vector of all 0s. Layer normalization calculates the mean and variance of the modulation semantic vector along the feature dimension and performs standardization, recentering, and scaling. Then, an affine transformation operation is performed on the output of the layer normalization. The affine transformation is implemented through a fully connected layer. The dimensional structure of the weight matrix of the fully connected layer is jointly determined by the image feature dimension and the modulation semantic vector dimension. For example, when the image feature dimension is 1024 and the modulation semantic vector dimension is 768, the weight matrix of the affine transformation fully connected layer is 1024 rows × 768 columns. After linear mapping, text features that perfectly match the dimensional structure of the image features are obtained.

[0080] Specifically, S3 includes the following steps:

[0081] S31. Use image features as a query matrix, text features as a key matrix and a value matrix, and input the query matrix, key matrix and value matrix into the cross-attention layer of the image decoder for feature interaction to obtain the aligned image feature matrix;

[0082] S32. Perform pixel-by-pixel similarity analysis on the text features and the aligned image feature matrix to obtain the similarity matrix;

[0083] S33. Perform bilinear interpolation upsampling on the similarity matrix to obtain a pixel-level anomaly response map corresponding to the spatial size of the industrial image to be detected. Analyze the pixel-level anomaly response map to obtain an image-level anomaly score. Output the pixel-level anomaly response map and the image-level anomaly score as the industrial image anomaly detection result.

[0084] Specifically, in step S32, the mathematical expression for pixel-by-pixel similarity analysis is: In the formula, The text features and the aligned image feature matrix are represented at pixel positions. Similarity value at, This indicates the pixel position in the aligned image feature matrix. The feature vector at that location, Represents the text feature vector. This indicates the pixel position in the aligned image feature matrix. eigenvectors at location The length of the mold, Represents text feature vectors The length of the mold, This represents a constant used to prevent the denominator from being zero.

[0085] Specifically, in S33, the image-level anomaly score The mathematical expression is: In the formula, Represents a pixel-level anomaly response map. These represent the weighting coefficients used to balance the maximum anomalous response and statistical characteristics. This represents the maximum value in the pixel-level anomaly response graph. This represents the arithmetic mean of the pixel-level anomaly response map. This represents the standard deviation in the pixel-level anomaly response graph. This represents the weighting factor of the arithmetic mean term. This represents the weighting coefficient of the standard deviation term.

[0086] Further, in step S31, image features are used as the query matrix, while text features are assigned to the key matrix and value matrix respectively, ensuring that the dimensions of the query matrix, key matrix, and value matrix match each other. The structure of the cross-attention layer of the image decoder is determined according to the accuracy requirements and feature dimensions of industrial image detection. For example, a three-layer fully connected layer is used as the core structure of the cross-attention layer, with 256 neurons in each layer, and the input and output dimensions are consistent with the dimensions of the image features and text features. Subsequently, the query matrix, key matrix, and value matrix are synchronously input into the cross-attention layer of the image decoder. The association weights between image features and text features are calculated through the attention mechanism. The calculated association weights are normalized, and then combined with the value matrix for feature fusion operation, gradually completing the deep interactive alignment of image features and text features. Finally, the output is an aligned image feature matrix with dimensions completely consistent with the input image feature matrix.

[0087] In step S32, the position of each pixel in the aligned image feature matrix is ​​first extracted. The corresponding feature vectors are used, while the text feature vectors are fixed. Feature normalization is then applied to ensure the pixel positions in the aligned image feature matrix. The feature vectors of the image and the text feature vectors reside in the same feature space and have the same dimension. In the implementation of pixel-by-pixel similarity analysis, for each pixel position of the aligned image feature matrix... The image feature vector at the pixel location is multiplied element-wise with the text feature vector, and a constant is added to prevent the denominator from being zero. This constant is determined based on the distribution of the product of the text feature vector and the feature vectors at each pixel location in the aligned image feature matrix. For example, by finding the minimum value of this product, the constant is set to 1e-6 to avoid errors with a zero denominator during calculation. This process is repeated for all pixel locations in the aligned image feature matrix, calculating the similarity value for each pixel. After all calculations are complete, the similarity values ​​for all pixel locations are combined to form a complete similarity matrix.

[0088] In step S33, the scaling ratio of bilinear interpolation upsampling is first determined. This scaling ratio is based on the ratio of the spatial size of the industrial image to be detected to the spatial size of the similarity matrix. For example, when the spatial size of the industrial image to be detected is 512×512 and the spatial size of the similarity matrix is ​​64×64, the scaling ratio of bilinear interpolation upsampling is set to 8. The similarity matrix is ​​upsampled using a bilinear interpolation algorithm. Interpolation operations fill the pixel gaps generated during the upsampling process, ensuring that the spatial size of the upsampled matrix completely corresponds to the spatial size of the industrial image to be detected, resulting in a pixel-level anomaly response map. Subsequently, the pixel-level anomaly response map is analyzed to obtain an image-level anomaly score, where weighting coefficients are used to balance the maximum anomaly response and statistical features. Weighting coefficient of the arithmetic mean term Weighting coefficients of the standard deviation term All were determined using a grid search method based on the detection results of a large number of industrial image samples. For example, [the following text is incomplete and likely refers to a different method:] Set it to 0.4. Set it to 0.3. Set the value to 0.3. Calculate the maximum value, arithmetic mean, and standard deviation of the pixel-level anomaly response map to obtain the image-level anomaly score. Finally, output the pixel-level anomaly response map and the image-level anomaly score as the industrial image anomaly detection result.

[0089] Specifically, S4 includes the following steps:

[0090] S41. Construct an adaptive prompting module containing a meta-prompt generation network. Use the variable convolution in the meta-prompt generation network to perform feature aggregation on spatial regions in the industrial image anomaly detection results where the abnormal response value is higher than the preset adaptive threshold, and obtain an initial region prompting set.

[0091] S42. The initial region cue set is weighted based on the spatial attention mechanism to obtain the weighted region cue vector;

[0092] S43. Perform adaptive linear fusion of the weighted regional prompt vector and the preset basic prompt, and output the final prompt representation that matches the current industrial scene.

[0093] Specifically, in S42, the mathematical expression for the spatial attention mechanism is: In the formula, This represents the weighted region hint vector. This represents the total number of region cue vectors in the initial region cue set. Indicates the first The query vector is obtained by linearly transforming the region suggestion vector. Indicates the first The key vector is obtained by linear transformation of the region hint vector. Indicates the first The transpose matrix of the key vector obtained by linear transformation of the region hint vector. The dimension of the key vector. Represented by natural constant An exponential function with base 0. Indicates the first The query vector is obtained by linearly transforming the region suggestion vector. Indicates the first The key vector is obtained by linear transformation of the region hint vector. Indicates the first The transpose matrix of the key vector obtained by linear transformation of the region hint vector. Indicates the first The value vector obtained by linear transformation of the region hint vector.

[0094] Furthermore, such as Figure 3 The diagram illustrates the adaptive prompting learning process of this invention. Through multi-branch collaboration and adaptive fusion, it achieves dynamic prompt enhancement of input features, comprising three key components. First, the meta-prompt generation network serves as the core branch, employing a stacked convolutional neural network structure. It uses consecutive [convolution-activation function-max pooling] units to perform multi-level feature extraction and downsampling on the input features, mining deep semantic information from the original input to generate meta-prompt features adapted to the current task, achieving dynamic prompt generation rather than fixed initialization. Second, the adaptive prompting module introduces two auxiliary enhancement branches: the first is a learnable basic prompt embedding branch, providing trainable basic prompt embeddings; the second is a spatial attention branch, which modulates the spatial dimension of the input features, automatically focusing on key regions, suppressing redundant information, and enhancing the spatial expressive power of the features to obtain spatial attention features. Finally, the adaptive prompting module dynamically weights and fuses the meta-prompt features, basic prompt embeddings, and spatial attention features, organically integrating multi-source prompt information to ultimately output a final prompt representation adapted to the input, achieving adaptive prompting learning.

[0095] In step S41, an adaptive prompting module containing a meta-prompt generation network is first constructed. The structure of the meta-prompt generation network is determined based on the accuracy requirements and feature dimensions of industrial image anomaly detection. For example, it uses two variable convolutional layers and one fully connected layer to form the meta-prompt generation network. The kernel size of the variable convolutional layers is set to 3×3, and the number of neurons in the fully connected layer is set to 128. The preset adaptive threshold is determined based on the distribution of abnormal response values ​​from a large number of industrial image anomaly detection results. For example, by statistically calculating the mean of abnormal response values ​​plus one standard deviation, the preset adaptive threshold is set to 0.7. Through the variable convolution in the meta-prompt generation network, features are extracted and aggregated for spatial regions in the industrial image anomaly detection results where the abnormal response values ​​are higher than the preset adaptive threshold, ultimately obtaining an initial set of region prompts containing multiple region prompt vectors.

[0096] In step S42, a linear transformation is first performed on each region hint vector in the initial region hint set to obtain the query vector and key vector corresponding to each region hint vector. The parameters of the linear transformation are determined by training with stochastic gradient descent based on the feature dimensions of the initial region hint set. For example, the dimension of the linear transformation matrix is ​​set to 128×64. The structure of the spatial attention mechanism is determined based on the number of region hint vectors in the initial region hint set. For example, a single-head attention structure is used, with the attention head dimension set to 64. The attention weight of each region hint vector is calculated using the spatial attention mechanism. The attention weights of all region hint vectors are normalized, and then each region hint vector is weighted and summed with its corresponding attention weight. This process is repeated for all region hint vectors in the initial region hint set, and the weighted region hint vector is obtained after completing the weighted calculation.

[0097] In step S43, the preset basic prompts are determined based on the type of industrial scenario and common anomaly features. For example, for a mechanical parts inspection scenario, the preset basic prompts are set as fixed vectors containing the normal contour and texture features of the parts, with a vector dimension of 128. The fusion coefficient of the adaptive linear fusion is determined based on the similarity between the weighted region prompt vector and the preset basic prompts. For example, by calculating the cosine similarity between the weighted region prompt vector and the preset basic prompts, the fusion coefficient of the adaptive linear fusion is set to 0.6. The weighted region prompt vector and the preset basic prompts are linearly fused according to the fusion coefficient to adjust the feature distribution after fusion, making the features more consistent with the anomaly detection requirements of the current industrial scenario, and finally outputting the final prompt representation that matches the current industrial scenario.

[0098] Specifically, S5 includes the following steps:

[0099] S51. Based on the channel dimension, the final prompt representation and image features are concatenated to obtain multimodal joint features. The multimodal joint features are then input into the embedding projection layer of the large language model for dimension alignment to obtain the dimension-aligned feature sequence.

[0100] S52. Input the dimension-aligned feature sequence into the decoder network of the large language model for autoregressive inference to obtain the conditional probability distribution of each output position.

[0101] S53. Decode the conditional probability distribution of each output position using a greedy search strategy, and output text information containing industrial anomaly judgment results and anomaly area descriptions.

[0102] Specifically, in S52, the mathematical expression for autoregressive inference is: In the formula, This represents the text tag generated at time step t. This represents the historical text sequence generated before time step t. This represents the dimension-aligned feature sequence. This represents the output at time step t, given a known historical text sequence and dimensionally aligned feature sequence. The conditional probability distribution, The decoder network representing the large language model is in the first... The hidden state vector output at each time step Indicates the first The hidden state vector output at each time step Feature sequences aligned with dimensions The joint representation obtained by splicing. The representation is used for joint representation. The weight matrix for linear transformation. Indicates will The output is mapped to the weight matrix of the vocabulary space. This represents the bias vector of the hidden layer in the decoder network. This represents the bias vector of the output layer in the decoder network. This represents a function that transforms an input vector into a probability distribution. This represents the hyperbolic tangent activation function.

[0103] Further, in step S51, the channels of the final prompt representation are superimposed with the channels of the image features to obtain multimodal joint features. The concatenation method is determined according to the channel order of the final prompt representation and the image features; for example, the channels of the final prompt representation are concatenated after the image feature channels. The structure of the embedding projection layer of the large language model is determined according to the dimension of the multimodal joint features and the input dimension of the large language model. For example, a fully connected layer is used as the embedding projection layer, the input dimension is set to the sum of the number of channels of the final prompt representation and the number of channels of the image features (e.g., 256 + 512 = 768), and the output dimension is set to 768. The multimodal joint features are input into the embedding projection layer of the large language model, and the dimension of the multimodal joint features is adjusted by linear transformation to make it consistent with the input dimension of the large language model, finally obtaining the dimension-aligned feature sequence.

[0104] In step S52, the decoder network structure of the large language model is first determined. The decoder network structure is determined based on the actual needs of autoregressive inference; for example, a 12-layer Transformer decoder structure is used, with the hidden state vector dimension set to 768. The dimension-aligned feature sequence is input into the decoder network to start the autoregressive inference process. Inference starts from the initial hidden state vector. At each time step, the decoder network of the large language model is first... The hidden state vectors output at each time step are concatenated with the dimension-aligned feature sequence to obtain a joint representation. , , , All are determined using gradient descent based on model training data. For example, The dimensions are set to 768×1536. The dimension is set to 768. The dimensions are set to 768×10000. The dimension is set to 10000, and then processed sequentially by the hyperbolic tangent activation function and the Softmax function to finally obtain the conditional probability distribution of each output position.

[0105] In step S53, the execution rules of the greedy search strategy are first determined. These rules are based on the accuracy and efficiency requirements of text generation. For example, it is set that only the text marker with the highest probability value in the conditional probability distribution of each output position is selected as the current output during each decoding process. The greedy search strategy decodes the conditional probability distribution of each output position, starting from the first output position, filtering the conditional probability distribution of each position one by one, selecting the text marker with the highest probability, and iterating until a preset end marker is generated. The end marker is determined according to the industrial text generation requirements; for example, the end marker is set to "". <end>Decoding stops when an end marker is generated. All generated text markers are integrated sequentially, and the final output contains text information including the industrial anomaly determination result and an anomaly area description. The industrial anomaly determination result clearly states whether an anomaly exists, and the anomaly area description explains the location and specific characteristics of the anomaly.

[0106] Based on the above content, conduct experimental analysis:

[0107] The proposed method was evaluated using the MVTec-AD public dataset. This dataset contains 3629 training images and 1725 test images across 15 different categories. The training images consist only of normal images, while the test images include both normal and abnormal images. Image resolutions range from 700×700 to 1024×1024.

[0108] like Figure 4 The diagram shows the ablation experiment results of this embodiment, comprehensively comparing and validating the baseline method, the method that only integrates the adaptive cue learner, and the method that combines Llama-2-7B but does not use LoRA (low-rank adaptive). Looking at the three core metrics—L_AUROC (image-level area under the receiver operating characteristic curve), P_AUROC (pixel-level area under the receiver operating characteristic curve), and Accuracy—the baseline method's metrics are 0.972, 0.921, and 0.924, respectively. After introducing only the adaptive cue learner, the metrics improved to 0.974, 0.920, and 0.928, respectively, initially validating the effectiveness of adaptive cue learning for feature enhancement. Furthermore, this application, based on the integration of the adaptive cue learner and Llama-2-7B, further introduces LoRA technology, achieving significant breakthroughs in all metrics: L_AUROC remains at the optimal level of 0.973, P_AUROC improves to 0.923, and Accuracy reaches 0.942, significantly surpassing other comparative methods. This demonstrates that the technical solution of combining adaptive cue learning with Llama-2-7B and adapting it with LoRA fine-tuning can more efficiently bridge visual features with semantic information of large language models. It retains the dynamic adaptability of cue learning and achieves efficient optimization of large model parameters through LoRA. Ultimately, it outperforms other methods in terms of detection accuracy and index stability, highlighting the core advantages of this patent in small-sample industrial anomaly detection scenarios.

[0109] like Figure 5 As shown, this is a schematic diagram illustrating the comparative experimental results of different large language models in this embodiment. To verify the universality and superiority of this application, three mainstream 7B parameter large language models—Vicuna-7B, Qwen-7B, and Llama-2-7B—were selected, and comparative experiments were conducted on the three core industrial anomaly detection indicators: I_AUROC, P_AUROC, and Accuracy. Figure 5 As can be seen, in the I_AUROC metric, Vicuna-7B achieved the best performance with 0.974, followed closely by Llama-2-7B with 0.973, and Qwen-7B with 0.966. All three models achieved excellent anomaly detection capabilities. In the P_AUROC metric, Llama-2-7B led with a score of 0.922, followed by Vicuna-7B with 0.920 and Qwen-7B with 0.902. In terms of accuracy, the solution in this application, combined with Llama-2-7B, significantly outperformed Vicuna-7B (0.928) and Qwen-7B (0.780) with a score of 0.942, demonstrating a significant performance advantage. In summary, the adaptive prompting learning framework proposed in this application can be adapted to different mainstream large language models. Among them, the solution with Llama-2-7B achieves optimal performance across the three core metrics, ensuring both image-level and pixel-level anomaly detection accuracy and the highest classification accuracy. This fully verifies the versatility and robustness of this application and provides a superior solution for industrial anomaly detection tasks.

[0110] The above are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.< / end>

Claims

1. An industrial anomaly detection method based on a large visual language model incorporating adaptive cue learning, characterized in that: Includes the following steps: S1. Input the industrial image to be detected into the image encoder of the pre-trained visual language model to extract features and obtain image features; S2. Input the preset semantic text into the text encoder of the pre-trained visual language model to encode it and obtain text features; S3. Input the image features and text features into the image decoder of the pre-trained visual language model for decoding, and generate industrial image anomaly detection results containing image-level anomaly scores and pixel-level anomaly response maps; S4. Construct an adaptive prompting module, and model the abnormal visual information in the pixel-level abnormal response map through the adaptive prompting module to obtain the final prompting representation that adaptively matches the current industrial scene. S5. The final prompt representation is concatenated with the image features to obtain multimodal joint features. The multimodal joint features are then input into a large language model for autoregressive decoding processing, and text information containing industrial anomaly judgment results and anomaly region descriptions is output. S3 includes the following steps: S31. Use image features as a query matrix, text features as a key matrix and a value matrix, and input the query matrix, key matrix and value matrix into the cross-attention layer of the image decoder for feature interaction to obtain the aligned image feature matrix; S32. Perform pixel-by-pixel similarity analysis on the text features and the aligned image feature matrix to obtain the similarity matrix; S33. Perform bilinear interpolation upsampling on the similarity matrix to obtain a pixel-level anomaly response map corresponding to the spatial size of the industrial image to be detected. Analyze the pixel-level anomaly response map to obtain an image-level anomaly score. Output the pixel-level anomaly response map and the image-level anomaly score as the industrial image anomaly detection result. S4 includes the following steps: S41. Construct an adaptive prompting module containing a meta-prompt generation network. Use the variable convolution in the meta-prompt generation network to perform feature aggregation on spatial regions in the industrial image anomaly detection results where the abnormal response value is higher than the preset adaptive threshold, and obtain an initial region prompting set. S42. The initial region cue set is weighted based on the spatial attention mechanism to obtain the weighted region cue vector; S43. Perform adaptive linear fusion between the weighted regional cue vector and the preset basic cue, and output the final cue representation that matches the current industrial scene; S5 includes the following steps: S51. Based on the channel dimension, the final prompt representation and image features are concatenated to obtain multimodal joint features. The multimodal joint features are then input into the embedding projection layer of the large language model for dimension alignment to obtain the dimension-aligned feature sequence. S52. Input the dimension-aligned feature sequence into the decoder network of the large language model for autoregressive inference to obtain the conditional probability distribution of each output position. S53. Decode the conditional probability distribution of each output position using a greedy search strategy, and output text information containing industrial anomaly judgment results and anomaly area descriptions.

2. The industrial anomaly detection method based on a large visual language model incorporating adaptive cueing learning as described in claim 1, characterized in that: S1 includes the following steps: S11. Preprocess the industrial image to be detected to obtain a preprocessed industrial image, and input the preprocessed industrial image into the image encoder for image segmentation and linear projection to obtain an image block embedding sequence. S12. The image block embedding sequence is input into multiple coding layers of the image encoder for hierarchical coding to obtain the intermediate feature map corresponding to each coding layer. S13. Perform global average pooling on the intermediate feature map corresponding to each coding layer to obtain the global feature vector corresponding to each coding layer, and calculate the attention weight corresponding to each coding layer based on the global feature vector. S14. Based on the attention weights corresponding to each coding layer, the intermediate feature maps corresponding to the coding layer are weighted and fused to obtain a fused feature map. The fused feature map is then subjected to layer normalization and spatial flattening to obtain image features.

3. The industrial anomaly detection method based on a large visual language model incorporating adaptive cueing learning as described in claim 1, characterized in that: S2 includes the following steps: S21. Obtain a preset semantic text containing normal state semantic description text and abnormal state semantic description text, and perform word segmentation and tagging processing on the normal state semantic description text and abnormal state semantic description text respectively to obtain the corresponding normal state tagging sequence and abnormal state tagging sequence. S22. Based on the text encoder of the pre-trained visual language model, perform context encoding on the normal state label sequence and the abnormal state label sequence respectively to obtain the corresponding normal state label embedding sequence and abnormal state label embedding sequence. S23. Perform mean pooling on the embedding vectors corresponding to all label positions in the normal state label embedding sequence to obtain the global semantic vector of the normal state, and perform mean pooling on the embedding vectors corresponding to all label positions in the abnormal state label embedding sequence to obtain the global semantic vector of the abnormal state. S24. Compare and perform semantic modulation processing on the global semantic vector of the normal state and the global semantic vector of the abnormal state to obtain the modulated semantic vector. Then, perform layer normalization and affine transformation processing on the modulated semantic vector to obtain text features that match the dimensional structure of the image features.

4. The industrial anomaly detection method based on a large visual language model incorporating adaptive cueing learning as described in claim 1, characterized in that: In step S32, the mathematical expression for pixel-by-pixel similarity analysis is: In the formula, The text features and the aligned image feature matrix are represented at pixel positions. Similarity value at, This indicates the pixel position in the aligned image feature matrix. The feature vector at that location, Represents the text feature vector. This indicates the pixel position in the aligned image feature matrix. eigenvectors at location The length of the mold, Represents text feature vectors The length of the mold, This represents a constant used to prevent the denominator from being zero.

5. The industrial anomaly detection method based on a large visual language model incorporating adaptive cueing learning as described in claim 1, characterized in that: In S33, image-level anomaly score The mathematical expression is: In the formula, Represents a pixel-level anomaly response map. These represent the weighting coefficients used to balance the maximum anomalous response and statistical characteristics. This represents the maximum value in the pixel-level anomaly response graph. This represents the arithmetic mean of the pixel-level anomaly response map. This represents the standard deviation in the pixel-level anomaly response graph. This represents the weighting factor of the arithmetic mean term. This represents the weighting coefficient of the standard deviation term.

6. The industrial anomaly detection method based on a large visual language model incorporating adaptive cueing learning as described in claim 1, characterized in that: In S42, the mathematical expression for the spatial attention mechanism is: In the formula, This represents the weighted region hint vector. This represents the total number of region cue vectors in the initial region cue set. Indicates the first The query vector is obtained by linearly transforming the region suggestion vector. Indicates the first The key vector is obtained by linear transformation of the region hint vector. Indicates the first The transpose matrix of the key vector obtained by linear transformation of the region hint vector. This represents the dimension of the key vector. Represented by natural constant An exponential function with base 0. Indicates the first The query vector is obtained by linearly transforming the region suggestion vector. Indicates the first The key vector is obtained by linear transformation of the region hint vector. Indicates the first The transpose matrix of the key vector obtained by linear transformation of the region hint vector. Indicates the first The value vector obtained by linear transformation of the region hint vector.

7. The industrial anomaly detection method based on a large visual language model incorporating adaptive cueing learning as described in claim 1, characterized in that: In S52, the mathematical expression for autoregressive inference is: In the formula, This represents the text tag generated at time step t. This represents the historical text sequence generated before time step t. This represents the dimension-aligned feature sequence. This represents the output at time step t, given a known historical text sequence and dimensionally aligned feature sequence. The conditional probability distribution, The decoder network representing the large language model is in the first... The hidden state vector output at each time step Indicates the first The hidden state vector output at each time step Feature sequences aligned with dimensions The joint representation obtained by splicing. The representation is used for joint representation. The weight matrix for linear transformation. Indicates will The output is mapped to the weight matrix of the vocabulary space. This represents the bias vector of the hidden layer in the decoder network. This represents the bias vector of the output layer in the decoder network. This represents a function that transforms an input vector into a probability distribution. This represents the hyperbolic tangent activation function.

Images