Small sample industrial defect image segmentation method and system based on text background perception enhancement and storage medium

Abstract

The application discloses a kind of based on text background perception enhancement small sample industrial defect image segmentation method, system and storage medium, comprising: according to industrial defect image set corresponding support-query pair is constructed;Establish text background perception enhancement model;Utilize all support-query pair training text background perception enhancement model;The query image to be segmented, corresponding support image and mask label are input into the text background perception enhancement model trained, and the output result of small sample segmentation model is obtained as the segmentation prediction result of corresponding query image.By fusing multimodal language-visual modeling mechanism and joint activation map optimization strategy, the accuracy and efficiency of defect detection are greatly improved, small sample, low-cost detection is realized, and the generalization ability is strong.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of computer vision and industrial automation detection, and particularly relates to a small sample industrial defect image segmentation method based on text background perception enhancement, a system and a storage medium. BACKGROUND

[0002] Industrial defect detection is a key link in intelligent manufacturing systems, and its precision and stability are directly related to product quality control and the automation level of the production line. In recent years, deep learning methods have made significant progress in this field, especially convolutional neural networks (CNN) and Transformer architectures have shown excellent performance in defect recognition tasks. However, industrial defect data often has the following characteristics: first, defect samples are scarce, and it is difficult to obtain large-scale labeled data; second, defect morphology is complex and variable, with obvious uncertainty and domain specificity; third, labeling costs are high and rely heavily on professional knowledge, which makes it difficult for traditional fully supervised learning methods to be widely used in real industrial environments.

[0003] To overcome the above problems, researchers introduced few-shot learning (FSL) mechanism to utilize limited labeled samples to complete the recognition task of new defect types. At the same time, in recent years, the development of multi-modal pre-training models, especially the breakthrough of CLIP model in joint modeling of text and images, provides a new opportunity for introducing language guidance in industrial vision tasks. However, based on this, existing methods mostly focus on text modeling of foreground categories, ignoring the important role of background information in model discrimination in defect segmentation. There are still many limitations: first, relying on single foreground category text prompts cannot provide sufficient semantic discrimination, especially in complex backgrounds with visual interference or semantic overlap, which can easily lead to model misjudgment; second, the potential information value of background categories is ignored, and the semantic co-occurrence relationship between background and foreground cannot be used to improve the model's context understanding ability; third, the quality of activation map generation is poor, often with insufficient foreground response or excessive background area response, resulting in unstable segmentation results and limited generalization ability. SUMMARY

[0004] The purpose of the present application is to address the above problems by providing a small sample industrial defect image segmentation method based on text background perception enhancement, system and storage medium, which greatly improves the precision and efficiency of defect detection, realizes small sample and low cost detection, and has strong generalization ability.

[0005] To achieve the above purpose, the technical solution adopted by the present application is as follows:

[0006] The small sample industrial defect image segmentation method based on text background perception enhancement provided by the present application comprises the following steps:

[0007] S1, constructing a corresponding support-query pair according to an industrial defect image set wherein, is a query image, and the support set , is a first support image, is a mask label of the first support image, is a number of support images;

[0008] S2, establishing a text background perception enhancement model to perform the following operations:

[0009] S21, acquiring foreground text and multiple background texts of a defect area of each support image, respectively;

[0010] S22, inputting the query image into an image encoder of a CLIP model to correspondingly extract a multi-level visual feature and a multi-level attention weight matrix, and inputting foreground text and each background text of a support image belonging to a category into a text encoder of the CLIP model to correspondingly obtain a foreground text embedding vector and each background text embedding vector;

[0011] S23, generating a corresponding coarse-grained activation map by using a Grad-CAM method according to the multi-level visual feature of the query image, the foreground text embedding vector and each background text embedding vector;

[0012] S24, acquiring a static refined activation map and a dynamic refined activation map according to the coarse-grained activation map, the multi-level visual feature and the multi-level attention weight matrix;

[0013] S25, inputting the query image, the corresponding support image and the mask label into a few-shot segmentation model, and taking the static refined activation map and the dynamic refined activation map as attention guide signals;

[0014] S3, training the text background perception enhancement model by using all support-query pairs;

[0015] S4, inputting a query image to be segmented, a corresponding support image and a mask label into the trained text background perception enhancement model, and taking an output result of the few-shot segmentation model as a segmentation prediction result of the corresponding query image.

[0016] Preferably, the query image is an industrial defect image in the industrial defect image set, the support image is an industrial defect image with a pre-labeled defect category, and the mask label of the support image is a binary graph or a multi-class label graph of all pixel points in the corresponding support image.

[0017] Preferably, the foreground text and the plurality of background texts of the defect area of each support image are generated by a DeepSeek-R1 large language model according to a preset question and answer template; the multi-level visual features , are the first layer visual features, and the multi-level attention weight matrix , is the first layer attention weight matrix, , is the total number of layers of the image encoder, is the height of the query image, is the width of the query image.

[0018] Preferably, the corresponding coarse-grained activation map is generated by a Grad-CAM method, specifically as follows:

[0019] S231, the cosine similarity of the visual feature of each spatial position of the first layer visual feature in the multi-level visual feature to the foreground text embedding vector and each background text embedding vector is calculated respectively, is the total number of layers of the image encoder;

[0020] S232, each cosine similarity is normalized to form a probability vector by a softmax function;

[0021] S233, the cross-entropy loss of the probability vector and the true label is taken as a coarse-grained activation map generation loss function, the foreground of the true label is 1, and the background is 0;

[0022] S234, the partial derivative of the coarse-grained activation map generation loss function to the visual feature of the spatial position of the first channel of the first layer visual feature is calculated as the gradient of the visual feature of the spatial position of the corresponding channel, is the height direction index of the query image, is the width direction index of the query image, , , is the height of the query image, is the width of the query image;

[0023] S235, the global average pooling of all gradients of each channel of the first layer visual feature is performed respectively to obtain the global gradient of the corresponding channel, and the global gradient of each channel is used to weight and sum the visual feature of the corresponding channel of the first layer visual feature, and then the ReLU activation function is used to generate a coarse-grained activation map.

[0024] Preferably, the static refined activation map and the dynamic refined activation map are obtained according to the coarse-grained activation map, the multi-level visual feature and the multi-level attention weight matrix, specifically as follows:

[0025] S241, obtaining the static refined matrix and the dynamic refined matrix based on the joint refinement strategy according to the multi-level visual feature and the multi-level attention weight matrix;

[0026] S242, performing matrix multiplication operation on the coarse-grained activation map and the static refined matrix and the dynamic refined matrix respectively, and obtaining the static refined activation map and the dynamic refined activation map correspondingly.

[0027] Preferably, after obtaining the static refined activation map and the dynamic refined activation map according to the coarse-grained activation map, the multi-level visual feature and the multi-level attention weight matrix, the pseudo label is generated from the dynamic refined activation map according to the preset threshold, that is, when the activation value of each spatial position in the dynamic refined activation map is greater than or equal to the preset threshold, it is considered as the foreground region of the pseudo label, otherwise, it is considered as the background region of the pseudo label.

[0028] The joint refinement strategy includes a static refinement strategy and a dynamic refinement strategy, wherein:

[0029] The static refinement strategy performs the following operations:

[0030] The channel average of each layer attention weight matrix in the multi-level attention weight matrix is performed, and then the Sinkhorn algorithm is used to obtain a normalized matrix;

[0031] The normalized matrix and the transpose of the normalized matrix are subjected to matrix multiplication operation to obtain a high-order optimization matrix;

[0032] The larger value of each spatial position in the high-order optimization matrix and the normalized matrix is taken to generate a static refined matrix;

[0033] The dynamic refinement strategy performs the following operations:

[0034] The convolution operation is performed on the splicing of all layers of visual features in the multi-level visual feature in the channel dimension An intermediate layer feature is obtained;

[0035] The shape remodeling operation is performed on the intermediate layer feature to form a remodeled feature, and then the matrix multiplication operation is performed on the remodeled feature and the transpose of the remodeled feature to form an initial dynamic refined matrix;

[0036] The difference between each layer attention weight matrix in the multi-level attention weight matrix and the corresponding element in the initial dynamic refined matrix is calculated respectively, and the difference of all elements in the corresponding layer is added to form a corresponding layer fusion feature;

[0037] The fusion features of each layer are averaged to obtain the average matrix, and the attention weight matrix with higher than the average matrix in the multi-level attention weight matrix is ​​selected as the middle layer attention matrix;

[0038] The selected middle layer attention matrix is ​​averaged over the channels to obtain the middle layer optimization matrix;

[0039] The initial dynamic refinement matrix is ​​multiplied pixel by pixel with the intermediate layer optimization matrix to obtain the dynamic refinement matrix of the current training round, and the cross entropy loss between the dynamic refinement matrix of the current training round and the pseudo label is used as the loss function of the dynamic refinement matrix of the next training round.

[0040] Preferably, the segmentation loss of the few-shot segmentation model is the cross entropy loss between the segmentation prediction result of the query image and the true mask label.

[0041] Preferably, the few-shot segmentation model is a PFENet model or a HDMNet model.

[0042] A small sample industrial defect image segmentation system based on text background perception enhancement includes a data acquisition module, a model building module, a model training module and a prediction module, wherein:

[0043] Data acquisition module, used to construct corresponding support-query pairs based on industrial defect image sets ,in, For the query image, the support set , For the Support images, For the Mask labels for the supported images, The number of supported images;

[0044] The model building module is used to build a text context-aware enhancement model to perform the following operations:

[0045] respectively obtaining a foreground text and a plurality of background texts of a defect area of ​​each supporting image;

[0046] The query image is input into the image encoder of the CLIP model, and multi-level visual features and multi-level attention weight matrices are extracted accordingly. The foreground text and background text of the support image of the corresponding category are input into the text encoder of the CLIP model, and the foreground text embedding vector and background text embedding vector are obtained accordingly.

[0047] According to the multi-level visual features of the query image, the foreground text embedding vector and the background text embedding vector, the Grad-CAM method is used to generate the corresponding coarse-grained activation map;

[0048] According to the coarse-grained activation map, the multi-level visual feature and the multi-level attention weight matrix, a static refined activation map and a dynamic refined activation map are obtained.

[0049] The query image, the corresponding support image and the mask label are input into the few-shot segmentation model, and the static refined activation map and the dynamic refined activation map are used as attention guide signals.

[0050] The model training module is used for training the text background perception enhancement model by using all support-query pairs.

[0051] The prediction module is used for inputting the query image to be segmented, the corresponding support image and the mask label into the trained text background perception enhancement model, and obtaining an output result of the few-shot segmentation model as a segmentation prediction result of the corresponding query image.

[0052] A small sample industrial defect image segmentation storage medium based on text background perception enhancement is used for storing a computer program, and the computer program is executed by a processor to realize any one of the above small sample industrial defect image segmentation methods based on text background perception enhancement.

[0053] Compared with the prior art, the beneficial effects of the present application are:

[0054] The present application aims at the problems of sample scarcity, difficult and high-cost labeling and insufficient model generalization ability in industrial defect detection tasks, especially in the realistic background that traditional few-shot learning methods ignore background semantic information, resulting in poor robustness of segmentation results. A small sample industrial defect image segmentation method based on text background perception enhancement is proposed, which significantly improves the generalization ability and semantic discrimination ability of the industrial defect segmentation model under the condition of labeled sample scarcity, greatly improves the precision and efficiency of defect detection. Specifically:

[0055] 1) According to the industrial defect image set, the corresponding support-query pair is constructed, a small amount of support image with labeled information and corresponding mask label are read, and a query image with unknown category is input, a large language model (such as DeepSeek-R1) is combined with a preset question and answer template to automatically generate foreground text and background text related to defects, a foreground-background semantic double semantic guide mechanism is introduced on the basis of the image mode, the semantic perception information of the model is enriched, the model is guided to perceive the target defect area more finely, and the understanding ability of the model to the semantic structure is enhanced;

[0056] 2) input the query image into the image encoder of the CLIP model to extract multi-scale image features (multi-level visual features and multi-level attention weight matrix), input the foreground text and background text into the text encoder of the CLIP model to extract corresponding foreground text embedding vector and each background text embedding vector, and generate an initial activation map (coarse-grained activation map) by using the Grad-CAM method, that is, the visual and language features are jointly extracted by the CLIP multi-modal model, and the activation area is further refined by the joint optimization strategy to avoid the problems of insufficient foreground response or background interference, that is, under the foreground-background double semantic guidance mechanism, the multi-modal image-text alignment modeling and joint activation map optimization strategy are combined, the foreground response and background noise suppression ability are improved based on the static noise suppression constraint and dynamic pseudo-mask supervision, so that the perception ability of the model to the target area and the suppression ability of the model to the background noise are improved under the condition of only a small amount of labeled samples, thereby improving the segmentation ability of the model, realizing high-precision identification of industrial defect targets and effective filtering of background noise, and significantly improving the robustness and adaptability of the industrial defect segmentation task;

[0057] 3) by fully mining the semantic co-occurrence relationship between the foreground and the background, the problem that the existing few-shot segmentation method only relies on the foreground prompt and ignores the background information is broken through, and high-precision segmentation of the industrial defect area can be realized under the condition of only a small amount of labeled samples, and the data labeling cost is significantly reduced. It is suitable for various industrial visual defect detection scenes and has good engineering deployment ability and generalization ability;

[0058] 4) the static refined activation map and the dynamic refined activation map are used as attention guidance signals, and the query image, the corresponding support image and the mask label are input into the few-shot segmentation model for training and guiding the model to complete the defect region segmentation output of the query image until the model converges, and the model weight generated by training can be used for segmentation of the industrial defect image;

[0059] The present application does not require a large amount of labeled data, but only relies on a few support images to realize high-robustness and high-precision industrial defect segmentation, and by deeply fusing text and image information, the prior information provided by the background semantics is fully utilized to guide the model to perform structure perception and background suppression in defect recognition, thereby significantly improving the accuracy, stability and industrial applicability of the small sample segmentation, and the structure is simple and has strong adaptability, especially suitable for small sample detection tasks in various complex industrial scenes, and has good practical engineering application value. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 The flowchart of the small sample industrial defect image segmentation method based on text background perception enhancement of the present application;

[0061] Figure 2Schematic diagram of the structure of the text background perception enhancement model of the present invention;

[0062] Figure 3 is a flow chart of the joint refinement strategy of the present invention;

[0063] Figure 4 Comparison diagrams of support images, mask labels of support images, query images, manual annotation results of query images, and segmentation prediction results of query images using the method of the present invention, wherein (A) is the support image, (B) is the mask label of the support image, (C) is the query image, (D) is the manual annotation result of the query image, and (E) is the segmentation prediction result of the query image using the method of the present invention. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0065] It should be noted that when a component is referred to as being "connected" to another component, it may be directly connected to the other component or there may be an intermediate component. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art in the art of this application. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0066] Example 1:

[0067] like Figures 1-4 As shown in FIG, a small sample industrial defect image segmentation method based on text background perception enhancement includes the following steps:

[0068] S1. Construct corresponding support-query pairs based on industrial defect image sets ,in, For the query image, the support set , For the Support images, For the Mask labels for the supported images, The number of supported images.

[0069] In one embodiment, the query image is an industrial defect image in an industrial defect image set, the support image is an industrial defect image with defect categories pre-labeled, and the mask label of the support image is a binary image or a multi-class labeled image corresponding to all pixels in the support image.

[0070] wherein the support-query pair constitutes a basic task unit in the small sample segmentation task, and the organization form is : Each query image corresponds to a set of support images containing defect category labels, that is, a support-query pair composed of one or more support images and a single query image. In terms of quantity, one or more support images ( ) provide semantic guidance for the same query image, thereby realizing accurate segmentation of the same category defect region in the query image.

[0071] Support images refer to image samples pre-labeled with defect categories in the small sample segmentation task, usually containing only a small number of representative defect instances, providing limited category information for semantic guidance to guide the subsequent text background perception enhanced model to recognize the same category defects in the query image; The mask label is a binary graph or multi-class label graph indicating whether each pixel in the support image belongs to the defect region. Specifically, different categories are represented by the value of each pixel in the support image, that is, the same category is represented by the same pixel color, which is used to provide accurate spatial location information and category information. The query image is the target image that needs to be segmented by the text background perception enhanced model according to the category clues provided by the support image, and its defect category is consistent with that of the support image but has no explicit annotation information. The query image also reflects the generalization reasoning ability of the text background perception enhanced model (segmentation model) under weak supervision, and is the key to evaluating the ability of the text background perception enhanced model to recognize new categories under low data conditions.

[0072] Further, the defect category labeling of the support image, that is, the pixel-level segmentation labeling of the defect region in the support image, specifically refers to giving a foreground or background label to each pixel position of the defect region in the industrial defect image. It is usually represented as a two-dimensional matrix of the same size as the industrial defect image, and each element value is: 1 (or a certain positive integer category number) indicates that the pixel belongs to the defect region (foreground), and 0 indicates that the pixel is background. The industrial defect image set can be data from multiple industrial scenes such as metal processing, semiconductor packaging, photovoltaic glass, textile materials, and automobile manufacturing. By collecting image data containing defect features (such as steel plate cracks, wafer defects, glass damage, fabric speckles, etc.), it provides the necessary data basis for subsequent support-query pair construction for defect segmentation, and can be applied to related fields or generalized to other fields.

[0073] S2, establishing a text background perception enhanced model to perform the following operations:

[0074] S21, acquiring the foreground text and the plurality of background texts of the defect region of each support image, respectively;

[0075] ​S22, input the query image into the image encoder of the CLIP model to correspondingly extract multi-level visual features and a multi-level attention weight matrix, and input the foreground text and the background texts of the support image of the corresponding category into the text encoder of the CLIP model to correspondingly obtain a foreground text embedding vector and background text embedding vectors;

[0076] S23, generating a corresponding coarse-grained activation map by using a Grad-CAM method according to the multi-level visual features of the query image, the foreground text embedding vector and the background text embedding vectors;

[0077] S24, obtaining static refinement activation maps and dynamic refinement activation maps according to the coarse-grained activation map, the multi-level visual features and the multi-level attention weight matrix;

[0078] S25, inputting the query image, the corresponding support image and the mask label into the few-shot segmentation model, and taking the static refinement activation map and the dynamic refinement activation map as attention guide signals.

[0079] In an embodiment, the foreground text and the plurality of background texts of the defect area of each support image are generated by using a DeepSeek-R1 large language model according to a preset question and answer template; the multi-level visual features , are the first layer visual features, the multi-level attention weight matrix , is the first layer attention weight matrix, , is the total number of layers of the image encoder, is the height of the query image, is the width of the query image.

[0080] Wherein, for each support image, the foreground text and the background texts related to the defect area are automatically generated by using a DeepSeek-R1 large language model based on a preset question and answer template (text prompt word and text feature generation). The foreground text describes the typical semantic features of the target defect area, and the background text automatically infers the background information having a semantic co-occurrence relationship with the defect area by using the preset question and answer template, thereby providing multi-modal semantic support for the subsequent segmentation task. As shown in the following table, the preset question and answer template is as follows: Figure 2

[0081] Which objects are most likely to appear in an image containing { },but are not part of {​

[0082] Which objects are most likely to be the background of {

[0083] Combine the answers to the two questions and output in['###'] format.

[0084] ['Intact Steel', 'SmoothMetal Surface', 'Brushed Metal', 'Polished Stainless Steel', 'Coated Steel','Flat Steel Sheet', 'Clean Metal Plate', 'Uniform Steel Surface', 'NormalRolled Surface','Well-Finished Steel']

[0085] A clear origami { ​​​The foreground of the queried image is replaced by the category text of the foreground, indicating the defect type or target object of interest, such as the foreground text: A clear origami{Steel Laceration Defect}; the output background text is A clear origami {###}, [###] represents the background category list generated by the DeepSeek-R1 large language model associated with the foreground category, and the background text is A clear origami {Intact Steel}, A clear origami {Clean Metal Plate}, …, A clear origami {Normal Rolled Surface}. Through the preset question and answer template, the background text that coexists with the foreground text of the queried image with high probability can be obtained.

[0086] The CLIP model is composed of an image encoder and a text encoder, which are used to map images and texts into a unified semantic space. The technology is well known to those skilled in the art and will not be repeated here. In this embodiment, the CLIP model is referred to in the following literature: Radford A, Kim J W, Hallacy C, et al. Learning transferable visual models from natural language supervision [C] / / International conference on machine learning. PmLR, 2021: 8748-8763.

[0087] The query image is input into the image encoder of the CLIP model, and the multi-level visual features are extracted and the multi-level attention weight matrix Meanwhile, the foreground text and each background text of the support image belonging to the category are input into the text encoder of the CLIP model, and the foreground text embedding vector is obtained

[0088]

[0089] wherein, is the image encoder of the CLIP model, is the text encoder of the CLIP model, is the foreground text of the support image, is the i-th background text of the support image, is the foreground text embedding vector, is the i-th background text embedding vector, and is the i-th background text embedding vector. background text embedding vectors, , The amount of background text.

[0090] In one embodiment, the Grad-CAM method is used to generate the corresponding coarse-grained activation map, as follows:

[0091] S231, calculate the multi-level visual features The cosine similarity between the visual features of each spatial position of the layer visual feature and the foreground text embedding vector and the background text embedding vector, is the total number of layers of the image encoder;

[0092] S232, normalizing each cosine similarity using a softmax function to form a corresponding probability vector;

[0093] S233, the cross entropy loss between the probability vector and the true label is used as the coarse-grained activation map to generate the loss function, where the foreground of the true label is 1 and the background is 0;

[0094] S234, calculate the coarse-grained activation map to generate the loss function for the The first layer of visual features Spatial position in the channel The partial derivative of the visual feature of the corresponding channel is the spatial position The gradient of the visual features, is the height direction index of the query image, is the width index of the query image, , , To query the height of the image, is the width of the query image;

[0095] S235, the All gradients of each channel of the visual features of the layer are globally averaged and pooled to obtain the global gradient of the corresponding channel, and the global gradient of each channel is used to average the gradient of the first layer. The visual features of the corresponding channels of the layer visual features are weighted summed, and then the coarse-grained activation map is generated through the ReLU activation function.

[0096] Specifically, if Figure 2 As shown, the coarse-grained activation map The generation process is as follows:

[0097] 1) Calculate the first The cosine similarity between the visual features of each spatial position of the layer visual feature and the foreground text embedding vector and the background text embedding vector is as follows:

[0098]

[0099] in, is the text embedding vector, that is, the foreground text embedding vector or the background text embedding vector, For the Layer visual features spatial location Visual features With text embedding vector cosine similarity (response strength), is the Euclidean norm;

[0100] 2) Normalize each cosine similarity through the softmax function to form a corresponding probability vector ;

[0101] 3) The probability vector and the true label The cross entropy loss is used as a coarse-grained activation map Loss function during generation , the formula is as follows:

[0102]

[0103] in, is the cross entropy loss, the true label It consists of a one-hot encoding of the category to which the query image belongs, such as 1 for foreground and 0 for background;

[0104] 4) For the Grad-CAM method, the calculation of its weight depends on the target category score in the multi-level visual features. The gradient of the visual features of the first layer (that is, the last layer, the image encoder of the CLIP model has 12 layers in total), then The first layer of visual features Spatial position in the channel The gradient of the visual feature , the formula is as follows:

[0105]

[0106] in, Loss function representing coarse-grained activation maps For the first Layer visual features No. Spatial position in the channel The partial derivative of the visual feature is The first layer of visual features Spatial position in the channel The gradient of the visual feature , i.e. coarse-grained activation map loss function in the generation process the first layer visual feature of the first channel of the spatial position , k = 1 ~ K, K is the total channel number of the first layer visual feature, the gradient serves as the basis of Grad-CAM weight, guiding the model to focus on the area that has significant contribution to the specific class;

[0107] 5) generating coarse-grained activation map , the formula is as follows:

[0108]

[0109] , wherein, is the global gradient of the first channel, represented as , is the visual feature of the first channel of the first layer visual feature, and ReLU is the ReLU activation function, used to retain the area with positive contribution (positive value).

[0110] In an embodiment, according to the coarse-grained activation map, the multi-level visual feature and the multi-level attention weight matrix, the static refined activation map and the dynamic refined activation map are obtained, specifically as follows:

[0111] S241, according to the multi-level visual feature and the multi-level attention weight matrix, the static refined matrix and the dynamic refined matrix are obtained based on the joint refinement strategy;

[0112] S242, the coarse-grained activation map is subjected to matrix multiplication operation with the static refined matrix and the dynamic refined matrix respectively, and the static refined activation map and the dynamic refined activation map are obtained correspondingly.

[0113] wherein, in order to improve the accuracy and segmentation guiding ability of the coarse-grained activation map, the joint refinement strategy based on the static refinement strategy and the dynamic refinement strategy is designed. The static refinement strategy utilizes the background suppression constraint to obtain the static refined matrix , suppressing the background noise. The dynamic refinement strategy introduces a learnable layer to supervise the generation of the dynamic refined matrix , improving the response of the foreground area. Finally, the output matrix of the two refinement strategies is used to generate different refined high-quality activation maps (the static refined activation map and the dynamic refined activation map ); and the current dynamic refined activation map is screened through a threshold (the preset threshold generate a corresponding pseudo label in a manner of further supervise the generation of a dynamic refinement matrix for the next training round As shown in Figure 2 , is a static refinement activation map, is a dynamic refinement activation map, is a static refinement matrix, is a dynamic refinement matrix.

[0114] In an embodiment, after obtaining the static refinement activation map and the dynamic refinement activation map according to the coarse-grained activation map, the multi-level visual feature and the multi-level attention weight matrix, the pseudo label is generated from the dynamic refinement activation map according to a preset threshold, that is, when the activation value of each spatial position in the dynamic refinement activation map is greater than or equal to the preset threshold, it is considered as a foreground region of the pseudo label, otherwise, it is considered as a background region of the pseudo label;

[0115] The joint refinement strategy includes a static refinement strategy and a dynamic refinement strategy, wherein:

[0116] The static refinement strategy performs the following operations:

[0117] The channel average of each layer attention weight matrix in the multi-level attention weight matrix is obtained, and then the Sinkhorn algorithm is used to obtain a normalized matrix;

[0118] The normalized matrix is multiplied by the transpose of the normalized matrix to obtain a high-order optimization matrix;

[0119] The larger value of each spatial position in the high-order optimization matrix and the normalized matrix is taken to generate a static refinement matrix;

[0120] The dynamic refinement strategy performs the following operations:

[0121] The convolution operation is performed on the spliced all-layer visual features in the channel dimension in the multi-level visual feature to obtain an intermediate layer feature;

[0122] The shape of the intermediate layer feature is reshaped to form a reshaped feature, and then the reshaped feature is multiplied by the transpose of the reshaped feature to form an initial dynamic refinement matrix;

[0123] The difference between each layer attention weight matrix in the multi-level attention weight matrix and the corresponding element in the initial dynamic refinement matrix is calculated respectively, and the difference values of all elements in the corresponding layer are added to form a corresponding layer fusion feature;

[0124] The channel average of each layer fusion feature is obtained to obtain an average matrix, and the attention weight matrix higher than the average matrix in the multi-level attention weight matrix is selected as an intermediate layer attention matrix;

[0125] The selected intermediate layer attention matrix is channel-averaged to obtain an intermediate layer optimization matrix;

[0126] The initial dynamic refinement matrix is pixel-by-pixel multiplied with the intermediate layer optimization matrix to obtain a dynamic refinement matrix of the current training round, and the cross-entropy loss of the dynamic refinement matrix of the current training round and the pseudo label is taken as a loss function of a dynamic refinement matrix of the next training round.

[0127] Specifically, as shown in Figure 3 , the joint refinement strategy includes a static refinement strategy (static refinement branch) and a dynamic refinement strategy (dynamic refinement branch). Among them:

[0128] In the static refinement strategy, the channel-averaged (to obtain an average weight matrix ) Sinkhorn algorithm (Sinkhorn) is used to obtain a normalization matrix . According to the normalization matrix , a high-order optimization matrix is obtained, and the formula is as follows:

[0129]

[0130] Among them, is the transpose of the normalization matrix , and is matrix multiplication.

[0131] Then, the corresponding values of the high-order optimization matrix and the normalization matrix at each spatial position are selected as the larger one as the output, which can realize the fusion of the information contained in the two matrices, thereby generating the final static refinement matrix .

[0132] The dynamic refinement matrix is a learnable dynamic matrix. First, an intermediate layer feature needs to be obtained according to all the layer visual features (12 layer visual features of the image encoder of the CLIP model) in the multi-level visual features, and the formula is as follows:

[0133]

[0134] Among them, is a convolution operation, is splicing in the channel dimension;

[0135] According to the intermediate layer feature , an initial dynamic refinement matrix is calculated, and the formula is as follows:

[0136]

[0137] in, Represents the shape reshaping operation, using the permute function, is transposed;

[0138] Next, the initial dynamic refinement matrix is ​​refined by the attention weight matrix of each layer First, calculate the Layer attention weight matrix With the initial dynamic refinement matrix The difference of the corresponding elements in the corresponding layer is added to form the fusion feature of the corresponding layer , to indicate and The actual similarity is then calculated by channel to get the average matrix , only select the actual similarity higher than the average matrix The attention weight matrix is ​​used as the intermediate layer attention matrix (Contrast and retain the high value index.) Finally, the selected middle layer attention matrix is ​​averaged over the channels to obtain the middle layer optimization matrix .Will and Pixel-by-pixel multiplication to obtain the final dynamic refinement matrix . Loss function of dynamic refinement matrix , the formula is as follows:

[0139]

[0140] in, is the cross entropy loss, is matrix multiplication.

[0141] In the joint refinement strategy, the static refinement strategy uses static statistics to suppress interference from non-target areas, and the dynamic refinement strategy uses the activation map generated by the previous optimization step as a pseudo-mask to enhance the response of the foreground area. After joint optimization, the two output more refined and robust static refinement activation maps and dynamic refinement activation maps, guiding the segmentation model to focus on the target defect area more accurately, with strong stability, versatility and engineering application prospects.

[0142] In one embodiment, the segmentation loss of the few-shot segmentation model is the cross entropy loss between the segmentation prediction result of the query image and the true mask label.

[0143] Specifically, the segmentation loss of the few-shot segmentation model is , the formula is as follows:

[0144]

[0145] wherein, is the segmentation prediction result of the query image, is the real mask label.

[0146] In an embodiment, the few-shot segmentation model is a PFENet model or a HDMNet model.

[0147] wherein, the few-shot segmentation model refers to any segmentation model existing in the field of few-shot segmentation, such as a PFENet model, a HDMNet model, etc., which is used to extract features of the support image and fuse structural semantic information from static refined activation maps and dynamic refined activation maps to guide the few-shot segmentation model to segment the defect region in the query image. The static refined activation maps and dynamic refined activation maps serve as attention guiding signals for the few-shot segmentation model to enhance the response of the defect-related region in the query image and suppress background interference information, i.e., to enhance the focusing ability of the few-shot segmentation model on the target defect region, guide it to complete the segmentation prediction of the target defect region in the query image, and output the segmentation prediction result corresponding to the query image.

[0148] S3, training a text background perception enhancement model using all support-query pairs.

[0149] S4, inputting the query image to be segmented, the corresponding support image and the mask label into the trained text background perception enhancement model to obtain the output result of the few-shot segmentation model as the segmentation prediction result of the corresponding query image.

[0150] wherein, the segmentation prediction result of the query image can accurately mark the shape, position and boundary of the defect, which is used in the industrial detection system to highlight the abnormal region in real time, facilitate subsequent calculation of the morphological information of the defect such as the perimeter and area, and facilitate the confirmation and judgment of the operator, thereby improving the detection explainability.

[0151] The present application is directed to the problems of sample scarcity, labeling difficulty and insufficient model generalization ability in industrial defect detection tasks, especially in the realistic background that traditional few-shot learning methods ignore background semantic information, leading to poor robustness of segmentation results. A small sample industrial defect image segmentation method based on text background perception enhancement is proposed. The method automatically generates foreground and background texts with the help of DeepSeek-R1 large language model, introduces a foreground-background semantic linkage mechanism based on image modalities, extracts visual and language features jointly through CLIP multimodal model, and generates an initial activation map (coarse-grained activation map) using Grad-CAM method. Further, the activation region is refined through a joint optimization strategy to guide the segmentation model to focus more accurately on the target defect area, with strong stability, universality and engineering application prospects.

[0152] For ease of understanding, the following will be described in detail through specific embodiments.

[0153] As shown in Figure 1 The core of the method of the present embodiment is to introduce a foreground-background dual semantic guidance mechanism and to improve defect segmentation accuracy under the condition of very few labeled samples through a joint activation map refinement strategy. The main steps are as follows:

[0154] Step 1, data reading and support-query task construction: read support images and query images from the industrial defect image set, construct support-query pairs, support images have defect label information and can be used to provide foreground semantic guidance, while query images do not contain annotation information and are used as the target to be predicted by the model. Extract the defect category name in the support image to construct the foreground text prompt word, such as "Steel Laceration Defect", etc., to guide the model to focus on the potential defect area in the image.

[0155] Step 2, foreground and background text generation: generate background class text prompts through the DeepSeek-R1 large language model. This process takes the preset question and answer template as input, combined with the defect category, which is used in the specific embodiment as: "Which objects are most likely to appear in an image containing {Defect}, but are not part of {Steel Laceration Defect}? and Which objects are most likely to be the background of {Steel Laceration Defect}?" The reasoning obtains background keywords that co-occur with the foreground semantics but are mutually exclusive in category. Finally, the foreground text and background text are input into the CLIP model together to perform semantic encoding and cross-modal alignment. Defect represents each specific defect category in the industrial defect image set.

[0156] Step 3, input the image data obtained in steps 1 and 2 and the foreground and background text into the CLIP model: extract image features through the image encoder and obtain semantic embedding representation through the text encoder. Then use the Grad-CAM method to generate a preliminary foreground and background activation map (coarse-grained activation map), which can roughly reflect the possible defect area or background interference area in the image. However, due to the inaccuracy of the coarse-grained activation map, further refinement is still needed to enhance its guidance and reliability.

[0157] Step 4, to optimize the coarse-grained activation map, a joint refinement strategy is used to process the coarse-grained activation map through a static refinement branch and a dynamic refinement branch: in the static refinement branch, first average the attention weight matrix of each layer obtained by the CLIP model, then normalize it through Sinkhorn, then fuse the high-order optimization matrix and the normalized matrix information to generate a static refinement matrix , which suppresses the abnormally high response area and removes the false activation area; in the dynamic refinement branch, feature fusion is performed through a learnable layer (the cross-entropy loss between the dynamic refinement matrix of the current training round and the pseudo label is used as the loss function of the dynamic refinement matrix of the next training round), that is, the visual features obtained by the CLIP model are optimized and learned to obtain an initial dynamic refinement matrix , and the generation of the dynamic refinement matrix is guided by the initial dynamic refinement matrix , driving the update of the intermediate layer attention, thereby enhancing the foreground area response and optimizing the edge details. Finally, the refinement results of the two branches are fused with the coarse-grained activation map Multiplication, output high-quality refined guide activation map (static refined activation map and dynamic refined activation map ), as the guide input for subsequent model training. The dynamic refined activation map is processed through a preset threshold , and values greater than or equal to the preset threshold in the dynamic refined activation map are retained to generate corresponding supervision pseudo-labels to guide the generation of the next round of dynamic refined matrix .

[0158] Step 5, the refined guide activation map is used as an attention guide signal in the feature channel of the query image to improve the attention ability of the model to the defect area. At the same time, the semantic features extracted from the support image are fused with the enhanced image features in the query image, and the final segmentation prediction result is output through the few-shot segmentation model , and the cross-entropy loss calculation is performed with the real mask label corresponding to the query image to guide the model learning.

[0159] Step 6, model training, in this embodiment, the number of training rounds is set to 300, during which the model will automatically cycle steps 3 to 5, and the threshold parameter is used to determine whether the model converges, and when the model does not appear in the training of rounds of precision improvement, the training is stopped. In this embodiment, the threshold parameter is set to 100, otherwise, the model will be trained to 300 rounds and then stopped.

[0160] The specific embodiments of the present application use the public industrial surface defect few-shot dataset FSSD-12 as the industrial defect image set for verification. The dataset has Steel_Ws (wrinkle), Steel_Se (edge scratch), Steel_Sc (middle scratch), Steel_Rp (roller impression), Steel_Ri (rust inclusion), Steel_Pk (pit knock), Steel_Pa (repair area), Steel_Os (oxidation spot), Steel_Op (oil stain), Steel_Ld (crack defect), Steel_Ia (inclusion area), Steel_Am (wear mark) a total of 12 defect types, and a total of more than 2000 images. In this embodiment, the PFENet model is used as the few-shot segmentation model Meanwhile, Steel_Ws (wrinkles), Steel_Se (edge scratches), Steel_Sc (middle scratches), Steel_Rp (roller print marks), Steel_Ri (rust inclusions), Steel_Pk (pit knock), Steel_Pa (repair area), Steel_Os (oxidation spots), Steel_Op (oil stains) categories are taken as the training set, and Steel_Ld (crack defects), Steel_Ia (inclusion area), Steel_Am (wear marks) categories are taken as the test set. In the training and test process, 1 or 5 images in a certain category are randomly selected as support images, and other images in the category are taken as query images. According to the number of support images used, 1-shot (single sample learning) and 5-shot (five sample learning) are corresponded. In addition, in the experimental design, Fold-x (x = 0, 1, 2) is used for testing. Fold-x represents a cross-category split strategy, which is commonly used to simulate the generalization ability under different task distributions in small sample segmentation. Each Fold-x represents a combination of dividing 12 defect categories in the data set into training classes (support classes) and test classes (query classes) to build a small sample segmentation task. The segmentation accuracy index uses mIoU (mean intersection over union) as the main segmentation index to comprehensively reflect the recognition and segmentation ability of the model on the target defect area. The 1-Shot segmentation result is as shown in Table 1. Figure 4 As shown in Table 1. Figure 4 As can be seen from Table 1, the method of the present application has a very high segmentation accuracy compared with the manual annotation result, effectively saving manpower, Figure 4 In addition, as can be seen from Table 1, the method of the present application can achieve optimal performance under 1-shot (single sample learning) and 5-shot (five sample learning) settings, with mIoU of 66.4% and 68.7% respectively, which is significantly better than other comparison methods. For example, under 1-shot, it is improved by 11.1% compared with the traditional PFENet model. The results verify the robustness and precise segmentation ability of the method under the condition of sample scarcity.

[0161] Table 1 Comparison of mIoU index detection results of different model methods on FSSD-12 data set

[0162]

[0163] Compared with existing segmentation model methods (such as BAM model, HSNet model, PFENet model, DCP model and CPANet model), the method has higher precision, meets the use demand of actual industrial detection, and greatly saves the labeling cost of manual work.

[0164] Embodiment 2

[0165] A small sample industrial defect image segmentation system based on text background perception enhancement includes a data acquisition module, a model building module, a model training module and a prediction module, wherein:

[0166] The data acquisition module is configured to construct a corresponding support-query pair according to an industrial defect image set , wherein is a query image, and the support set , is the first support image, is the mask label of the first support image, is the number of support images.

[0167] The model building module is configured to establish a text background perception enhancement model to perform the following operations:

[0168] Respectively acquire foreground text and multiple background texts of the defect area of each support image;

[0169] Input the query image into the image encoder of the CLIP model to correspondingly extract multi-level visual features and a multi-level attention weight matrix, and input the foreground text and each background text of the support image belonging to the category into the text encoder of the CLIP model to correspondingly obtain a foreground text embedding vector and each background text embedding vector;

[0170] According to the multi-level visual features, the foreground text embedding vector and each background text embedding vector of the query image, a corresponding coarse-grained activation map is generated by using the Grad-CAM method;

[0171] According to the coarse-grained activation map, the multi-level visual features and the multi-level attention weight matrix, a static refined activation map and a dynamic refined activation map are acquired;

[0172] Input the query image, the corresponding support image and the mask label into the few-shot segmentation model, and input the static refined activation map and the dynamic refined activation map as attention guide signals;

[0173] The model training module is configured to train the text background perception enhancement model by using all support-query pairs;

[0174] The prediction module is configured to input the query image to be segmented, the corresponding support image, and the mask label into the trained text context-aware enhanced model to obtain an output result of the few-shot segmentation model as a segmentation prediction result of the corresponding query image.

[0175] Specifically, based on the same inventive concept as in Embodiment 1, the data acquisition module in this embodiment is configured to load support images and query images from local or remote storage to construct corresponding support-query pairs. The model building module is configured to construct foreground cues based on the defect categories labeled in the support images, and to generate background cues that coexist with the foreground semantics by combining a DeepSeek-R1 large language model and a preset question and answer template. In addition, the model building module also supports performing image preprocessing operations such as uniform size adjustment, brightness enhancement, and filter denoising on the images in sequence before generating the foreground text and the background text, so as to improve the image quality and provide stable input for the subsequent operations. The preset question and answer template is used to guide the DeepSeek-R1 large language model to automatically infer the context background cues related to the target defect, so as to further enrich the semantic expression of the image-text pair; and finally, a set of foreground text and background text word groups is output, which is used to guide the subsequent image feature alignment and attention modeling. Then, the support image and the query image are input into the image encoder of the CLIP model that is pre-trained and frozen, to extract the multi-level visual features and the multi-level attention weight matrix thereof; meanwhile, the generated foreground text and background text are input into the text encoder of the CLIP model that is pre-trained and frozen, to extract the corresponding semantic vectors (foreground text embedding vectors and background text embedding vectors), and a semantic alignment relationship between the visual features and the text features is established in the shared semantic space of the image encoder and the text encoder. The pre-training refers to directly using the pre-trained weights, and the image encoder and the text encoder of the CLIP model do not participate in the training. According to the multi-level visual features of the query image, the foreground text embedding vectors, and the background text embedding vectors, the cosine similarity between them is calculated as the response strength; and the Grad-CAM method is used to perform gradient backpropagation on the classification layer associated with the query image and the text category, to obtain a saliency heat map. Finally, the text response map and the gradient heat map are fused to generate an initial activation map (coarse-grained activation map), and the cross-entropy loss function is used for supervised optimization. According to the multi-level visual features and the multi-level attention weight matrix, a static refinement matrix and a dynamic refinement matrix are obtained based on a joint refinement strategy. The joint refinement strategy includes a static refinement branch and a dynamic refinement branch. The static refinement branch constructs a high-order optimization matrix based on the average attention weight matrix to enhance the consistency of the discriminative regions; the dynamic refinement branch uses the activation map generated by the current model to be transformed into a learnable pseudo mask, and through the strategy of interactive learning, the diversity and stability of the activation map are improved; finally, the static refinement activation map and the dynamic refinement activation map and the coarse-grained activation map are fused through matrix multiplication. , generate a current optimized static refinement activation map and a dynamic refinement activation map . Based on the static refinement activation map and the dynamic refinement activation map guide the few-shot segmentation model to segment the defect area in the query image to enhance the focusing ability of the model on the defect area; and fuse the semantic features of the query image and the support image at multiple scales, to generate a final small sample segmentation prediction result of the industrial defect through the few-shot segmentation model The model training module is configured to train the text background perception enhanced model using all support-query pairs. The prediction module is configured to input the query image to be segmented, the corresponding support image, and the mask label into the trained text background perception enhanced model, to obtain an output result of the few-shot segmentation model as a segmentation prediction result of the corresponding query image. According to the output segmentation prediction result combined with the evaluation index, the quality inspection system or manual review can be used, which is beneficial to improve the detection accuracy and efficiency of image defects.

[0176] The system is proposed in view of the problems of sample scarcity, labeling difficulty, and insufficient model generalization ability in industrial defect detection tasks, especially in the realistic background that traditional few-shot learning methods ignore background semantic information, leading to poor robustness of segmentation results. The system automatically generates foreground text and background text with the help of the DeepSeek-R1 large language model, introduces a foreground-background semantic linkage mechanism based on the image modality, jointly extracts visual and language features through the CLIP multi-modal model, and generates an initial activation map (coarse-grained activation map) using the Grad-CAM method. Further, the activation area is refined through a joint optimization strategy to guide the segmentation model to focus more accurately on the target defect area, with strong stability, universality, and engineering application prospects.

[0177] For specific limitations of the few-shot industrial defect image segmentation system based on text background perception enhancement, refer to the limitations of the few-shot industrial defect image segmentation method based on text background perception enhancement in the above, which will not be repeated here. Each module in the above few-shot industrial defect image segmentation system based on text background perception enhancement can be realized by software, hardware, and combinations thereof, in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0178] The memory and the processor are directly or indirectly electrically connected to realize the transmission or interaction of data. For example, the elements can be electrically connected to each other through one or more communication buses or signal lines. The memory stores a computer program executable on the processor. The processor realizes the small sample industrial defect image segmentation method based on text background perception enhancement in the embodiment of the application by running the computer program stored in the memory.

[0179] The memory can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory is used to store a computer program. The processor executes the computer program after receiving an execution instruction.

[0180] Embodiment 3:

[0181] A small sample industrial defect image segmentation storage medium based on text background perception enhancement is used to store a computer program. The computer program is executed by a processor to realize the small sample industrial defect image segmentation method based on text background perception enhancement as in embodiment 1.

[0182] The computer program stored in the small sample industrial defect image segmentation storage medium based on text background perception enhancement is executed by the processor to realize the small sample industrial defect image segmentation method based on text background perception enhancement as in embodiment 1, thereby completing the small sample segmentation task of industrial defects. The processor can be an integrated circuit chip with data processing capability. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. The processor can realize or execute the methods disclosed in the embodiments of the application. The general-purpose processor can be a microprocessor or the processor can be any conventional processor, etc. According to the segmentation prediction result combined with the evaluation index, the quality inspection system or manual review can be used, which is beneficial to improve the detection accuracy and efficiency of image defects.

[0183] The storage medium is proposed in view of the problems of sample scarcity, labeling difficulty and insufficient model generalization ability in industrial defect detection tasks, especially in the realistic background that traditional few-shot learning methods ignore background semantic information, resulting in poor robustness of segmentation results. By means of the DeepSeek-R1 large language model to automatically generate foreground text and background text, the foreground-background semantic linkage mechanism is introduced on the basis of the image mode, the visual and language features are jointly extracted by the CLIP multi-modal model, and the initial activation map (coarse-grained activation map) is generated by the Grad-CAM method. Further, the activation region is refined by the joint optimization strategy, guiding the segmentation model to focus on the target defect area more accurately, and having strong stability, universality and engineering application prospect.

[0184] The technical features of the above-described embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above-described embodiments are not described, but as long as the combinations of the technical features do not contradict, they should be considered within the scope of the present disclosure.

[0185] The above-described embodiments only express the specific and detailed embodiments described in the present application, but should not be construed as limiting the scope of the application. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A small sample industrial defect image segmentation method based on text background perception enhancement, characterized by: The steps include: S1. Construct corresponding support-query pairs based on industrial defect image sets ,in, For the query image, the support set , For the Support images, For the Mask labels for the supported images, The number of supported images; S2. Build a text context-aware enhancement model to perform the following operations: S21, respectively obtaining a foreground text and multiple background texts of a defective area of ​​each supporting image; S22, input the query image into the image encoder of the CLIP model, extract the multi-level visual features and the multi-level attention weight matrix, and input the foreground text and each background text of the support image of the corresponding category into the text encoder of the CLIP model, and obtain the foreground text embedding vector and each background text embedding vector; S23, generating a corresponding coarse-grained activation map using the Grad-CAM method based on the multi-level visual features of the query image, the foreground text embedding vector, and the background text embedding vectors; S24, obtaining a static refined activation map and a dynamic refined activation map according to the coarse-grained activation map, the multi-level visual features, and the multi-level attention weight matrix; S25. Input the query image, the corresponding support image, and the mask label into the few-shot segmentation model, and use the static refined activation map and the dynamic refined activation map as attention guidance signals; S3. Use all support-query pairs to train a text context-aware enhancement model; S4. Input the query image to be segmented, the corresponding support image and the mask label into the trained text background perception enhancement model, and obtain the output result of the few-shot segmentation model as the segmentation prediction result of the corresponding query image.

2. The small sample industrial defect image segmentation method based on text-background perception enhancement according to claim 1, characterized in that: The query image is an industrial defect image in the industrial defect image set, the support image is an industrial defect image with defect categories pre-labeled, and the mask label of the support image is a binary image or a multi-class labeled image corresponding to all pixels in the support image.

3. The small sample industrial defect image segmentation method based on text-background perception enhancement according to claim 1, characterized in that: The foreground text and multiple background texts of the defect area of ​​each supporting image are generated by using the DeepSeek-R1 large language model according to the preset question-answering template; the multi-level visual features , For the layer visual features, the multi-level attention weight matrix , For the layer attention weight matrix, , is the total number of layers of the image encoder, To query the height of the image, The width of the query image.

4. The small sample industrial defect image segmentation method based on text-background perception enhancement according to claim 1, characterized in that: The Grad-CAM method is used to generate the corresponding coarse-grained activation map, as follows: S231, calculate the multi-level visual features The cosine similarity between the visual features of each spatial position of the layer visual feature and the foreground text embedding vector and the background text embedding vector, is the total number of layers of the image encoder; S232, normalizing each cosine similarity using a softmax function to form a corresponding probability vector; S233, the cross entropy loss between the probability vector and the true label is used as the coarse-grained activation map to generate the loss function, where the foreground of the true label is 1 and the background is 0; S234, calculate the coarse-grained activation map to generate the loss function for the The first layer of visual features Spatial position in the channel The partial derivative of the visual feature of the corresponding channel is the spatial position The gradient of the visual features, is the height direction index of the query image, is the width index of the query image, , , To query the height of the image, is the width of the query image; S235, the All gradients of each channel of the visual features of the layer are globally averaged and pooled to obtain the global gradient of the corresponding channel, and the global gradient of each channel is used to average the gradient of the first layer. The visual features of the corresponding channels of the layer visual features are weighted summed, and then the coarse-grained activation map is generated through the ReLU activation function.

5. The small sample industrial defect image segmentation method based on text background perception enhancement according to claim 1 is characterized in that: According to the coarse-grained activation map, the multi-level visual features and the multi-level attention weight matrix, the static refined activation map and the dynamic refined activation map are obtained as follows: S241. Obtaining a static refinement matrix and a dynamic refinement matrix based on a joint refinement strategy according to the multi-level visual features and the multi-level attention weight matrix; S242. Perform matrix multiplication operations on the coarse-grained activation map with the static refinement matrix and the dynamic refinement matrix, respectively, to obtain a static refinement activation map and a dynamic refinement activation map, respectively.

6. The small sample industrial defect image segmentation method based on text-background perception enhancement according to claim 5, characterized in that: After obtaining the static refined activation map and the dynamic refined activation map based on the coarse-grained activation map, the multi-level visual features, and the multi-level attention weight matrix, a pseudo label is generated from the dynamic refined activation map according to a preset threshold, that is, when the activation value of each spatial position in the dynamic refined activation map is greater than or equal to the preset threshold, it is considered to be a foreground area of ​​the pseudo label; otherwise, it is considered to be a background area of ​​the pseudo label; The joint refinement strategy includes a static refinement strategy and a dynamic refinement strategy, wherein: The static refinement strategy performs the following operations: The attention weight matrix of each layer in the multi-level attention weight matrix is ​​averaged and the normalized matrix is ​​obtained using the Sinkhorn algorithm; Perform matrix multiplication operation on the normalized matrix and the transpose of the normalized matrix to obtain a high-order optimized matrix; Take the larger value of the high-order optimization matrix and the normalized matrix at each spatial position to generate a static refinement matrix; The dynamic refinement strategy performs the following operations: Concatenate all the visual features of the multi-level visual features in the channel dimension and perform convolution operation Obtaining intermediate layer features; Perform a shape reshaping operation on the intermediate layer features to form a reshaped feature, and then perform a matrix multiplication operation on the reshaped feature and the transpose of the reshaped feature to form an initial dynamic refinement matrix; Calculate the difference between the attention weight matrix of each layer in the multi-level attention weight matrix and the corresponding element in the initial dynamic refinement matrix, and add the differences of all elements in the corresponding layer to form the fusion feature of the corresponding layer; The fusion features of each layer are averaged to obtain the average matrix, and the attention weight matrix with higher than the average matrix in the multi-level attention weight matrix is ​​selected as the middle layer attention matrix; The selected middle layer attention matrix is ​​averaged over the channels to obtain the middle layer optimization matrix; The initial dynamic refinement matrix is ​​multiplied pixel by pixel with the intermediate layer optimization matrix to obtain the dynamic refinement matrix of the current training round, and the cross entropy loss between the dynamic refinement matrix of the current training round and the pseudo label is used as the loss function of the dynamic refinement matrix of the next training round.

7. The small sample industrial defect image segmentation method based on text background perception enhancement according to claim 1 is characterized in that: The segmentation loss of the few-shot segmentation model is the cross entropy loss between the segmentation prediction result of the query image and the true mask label.

8. The small sample industrial defect image segmentation method based on text background perception enhancement according to claim 1, characterized in that: The few-shot segmentation model is a PFENet model or a HDMNet model.

9. A small sample industrial defect image segmentation system based on text background perception enhancement, characterized by: It includes data acquisition module, model building module, model training module and prediction module, among which: The data acquisition module is used to construct corresponding support-query pairs based on the industrial defect image set. ,in, For the query image, the support set , For the Support images, For the Mask labels for the supported images, The number of supported images; The model building module is used to build a text background perception enhancement model to perform the following operations: respectively obtaining a foreground text and a plurality of background texts of a defect area of ​​each supporting image; The query image is input into the image encoder of the CLIP model, and multi-level visual features and multi-level attention weight matrices are extracted accordingly. The foreground text and background text of the support image of the corresponding category are input into the text encoder of the CLIP model, and the foreground text embedding vector and background text embedding vector are obtained accordingly. According to the multi-level visual features of the query image, the foreground text embedding vector and the background text embedding vector, the Grad-CAM method is used to generate the corresponding coarse-grained activation map; Obtain static refined activation maps and dynamic refined activation maps based on the coarse-grained activation map, multi-level visual features, and multi-level attention weight matrix; The query image, the corresponding support image, and the mask label are fed into the few-shot segmentation model, and the static refined activation map and the dynamic refined activation map are used as attention guidance signals. The model training module is used to train the text context perception enhancement model using all support-query pairs; The prediction module is used to input the query image to be segmented, the corresponding support image and the mask label into the trained text background perception enhancement model, and obtain the output result of the few-shot segmentation model as the segmentation prediction result of the corresponding query image.

10. A storage medium for small sample industrial defect image segmentation based on text-background perception enhancement, used to store a computer program, characterized by: When the computer program is executed by a processor, the method for segmenting small sample industrial defect images based on text background perception enhancement is implemented as described in any one of claims 1 to 8.

Images