Industrial defect detection method based on vision-language cues

Abstract

The present invention relates to the technical field of industrial scene defect detection, and in particular, provides an industrial defect detection method based on visual-verbal cues. The method comprises building a feature extraction network for a target detection model; building a visual-verbal cuing module based on the feature extraction network; building a visual-verbal detection module and connecting the feature extraction network and the visual-verbal cuing module to construct an industrial defect detection model; training the constructed industrial defect detection model, encapsulating and deploying the trained industrial defect detection model, and obtaining defect detection results using the trained industrial defect detection model. This method accurately distinguishes industrial defects, accelerates the identification and location of industrial defect targets, and thus improves the accuracy of industrial defect image detection.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial scene defect detection, and in particular to an industrial defect detection method based on vision-language prompts, which is used for industrial defect detection. Background Art

[0002] Product defect detection in industrial scenarios plays a vital role in protecting the personal and property safety of users. While deep learning methods have achieved significant success in defect detection in industrial scenarios, expert models for these scenarios typically utilize unimodal or multimodal data from a single domain and strictly adhere to a category-aware detection approach, limiting the model's ability to process multi-scenario data and generalize to open datasets. Recently, the Segment Everything model has achieved significant success in the field of natural scene images. The Segment Everything model not only possesses powerful interactive capabilities but also demonstrates strong zero-shot capabilities in fields such as remote sensing and medicine. The Segment Everything model is unique in its design of human-computer interaction prompts, allowing segmentation based on user-provided point, line, and box prompts.

[0003] However, due to the significant differences in imaging conditions, environments, scale, and distribution between industrial defect images and natural scenes, the application of pre-trained and hinted approaches to segmentation-all-in-one models faces two major challenges. Existing hinting approaches to segmentation-all-in-one models rely on user-provided hints (points, boxes, coarse masks) to segment objects in target images when handling complex scenes and large numbers of images. User familiarity with the target objects significantly impacts the effectiveness of specific hints. Furthermore, existing methods use coarse visual hints, such as colored boxes or circles, which often lead to suboptimal performance due to the inclusion of irrelevant and noisy pixels. To address these issues, some methods incorporate semantic association models to obtain pseudo-masks of the objects. CPT and ReCLIP leverage visual hinting techniques to establish relationships between part instances. CoCoOp further transforms image-generated hints into conditional inputs and dynamically combines them with language hints. These methods ignore false positives in the pseudo-masks and exhibit high sensitivity to hyperparameters. Consequently, they rely heavily on the quality of the pseudo-masks and suffer from poor generalization. Summary of the Invention

[0004] In view of this, the present invention provides an industrial defect detection method based on vision-language cues to accurately distinguish industrial defects, accelerate the identification and positioning of industrial defect targets, and thus improve the accuracy of industrial defect image detection.

[0005] In a first aspect, the present invention provides an industrial defect detection method based on visual-linguistic cues, the method comprising:

[0006] Step S1: Building a feature extraction network for the target detection model;

[0007] Step S2: Building a visual-language prompt module based on the feature extraction network in step S1;

[0008] Step S3: Building a vision-language detection module and connecting the feature extraction network in step S1 with the vision-language prompt module in step S2 to construct an industrial defect detection model;

[0009] Step S4: train the industrial defect detection model constructed in step S3, encapsulate and deploy the trained industrial defect detection model, and obtain defect detection results through the trained industrial defect detection model.

[0010] Optionally, the feature extraction network is composed of a segmentation model, a convolutional expert model, a language encoder, and a mask domain adaptation module; the segmentation model branch, the convolutional expert model branch, and the language encoder branch extract features in parallel;

[0011] The segmentation model branch includes an encoder structure, wherein the encoder structure is divided into a first attention module, a second attention module, a third attention module and a fourth attention module;

[0012] The convolution expert model branch includes a first convolution module, a second convolution module, a third convolution module and a fourth convolution module.

[0013] Optionally, the structure of the mask field adaptation module includes:

[0014] (1) Align the feature maps of the first, second, third, and fourth attention modules of the split-all-large-model branches with the feature maps of the first, second, third, and fourth convolution modules of the convolution expert model branches. The formula is as follows:

[0015] L = γ × D (L1 (F i ), L2(S j ));

[0016] Among them, L1 and L2 represent mapping functions, F i , i∈{1, 2, 3, 4} represents the feature maps in the first attention module, the second attention module, the third attention module, and the fourth attention module that split all large model branches; S j , j∈{1, 2, 3, 4} represents the feature maps in the first convolution module, the second convolution module, the third convolution module, and the fourth convolution module of the convolution expert model branch; D represents the alignment loss function, which is used to penalize different F i and Sj The distance between them, γ is the weight to balance the two losses; the formula of D is:

[0017]

[0018] (2) Divide the features of the first attention module, the second attention module, the third attention module and the fourth attention module of all large model branches into N groups according to the feature block size of 32 to obtain the neighborhood multi-scale features where N∈2 k , k∈N + , i∈{1, 2, 3, 4};

[0019] (3) According to the random sampling method, some feature blocks are evenly masked. The two-dimensional index of each feature block is defined as Randomly sample M indices to get a binary mask, and i Multiply to obtain the multi-scale features of the mask The formula is:

[0020]

[0021]

[0022] in, is the range of M indexes extracted; is a binary mask, and P is the operation of segmenting the image into image blocks;

[0023] (4) Obtaining multi-scale features of the mask Finally, a mask decoder is built to reconstruct the masked features and obtain the features of the first attention module, the second attention module, the third attention module and the fourth attention module of all the branches of the segmentation model after reconstruction. Its mask decoder formula is:

[0024]

[0025] (5) Introduce a multi-scale domain optimization function to additionally optimize the domain adaptation layer. When the optimization function is infinitely small, the optimization of the domain adaptation layer will stop. Its optimization function F target The formula is:

[0026]

[0027] Among them, n=4 represents the number of different multi-scale features, S j , j∈{1, 2, 3, 4} represents the feature maps in the first convolution module, the second convolution module, the third convolution module and the fourth convolution module of the convolution expert model branch.

[0028] Optionally, the visual-language prompt module in step S2 is composed of a refined visual prompt module and a visual-language interaction module.

[0029] Optionally, the structure of the refined visual prompt module includes:

[0030] (6) The feature maps of the first attention module, the second attention module, the third attention module and the fourth attention module of all the large model branches after segmentation are reconstructed. Each pixel is scored by the energy evaluation function to obtain the uncertainty score M of each pixel position k , the formula is:

[0031]

[0032] Where k is the normalization constant, N i is the set of adjacent pixels of the i-th pixel, is the similarity measure between the i-th pixel and the j-th pixel;

[0033] (7) Image block segmentation with different neighborhood sizes M k ; For the position of a single pixel in the feature space, the set of neighboring pixels It is expressed as:

[0034]

[0035] Where (h′, w′) is the relative position of the pixel;

[0036] (8) Given a set of segmented neighboring pixels, a sparse selection mechanism is used to select high-frequency pixels. Sparse sample selection is completed by selecting the top-k pixels whose internal pixel values ​​are greater than the mean. The formula is:

[0037]

[0038] in, express Any pixel in represents the pixel-level mean in the neighborhood space, Represents the selected sparse samples;

[0039] (9) Introduce an IoU-based optimization mechanism to iteratively optimize the activation mechanism of features; the goal of the optimization mechanism is to make the selected high-frequency pixels fall within the area of ​​the real frame; the optimization mechanism uses an additional detection head to Perform regression prediction and optimize the effect of feature activation through CIoU.

[0040] Optionally, the structure of the visual language interaction module includes:

[0041] (11) The feature maps of the first attention module, the second attention module, the third attention module and the fourth attention module of all the large model branches after segmentation are reconstructed. and After multiplication, the feature map S in the first convolution module, the second convolution module, the third convolution module and the fourth convolution module of the convolution expert model branch j Add together to get the refined visual feature F L , the formula is:

[0042]

[0043] (12) By embedding the text feature T and image feature F generated by the text encoder L ∈R CxHxW (L∈{1, 2, 3, 4}), use the maximum Sigmoid attention to query the semantic features of text-image matching; aggregate text features into image features, the formula is:

[0044]

[0045] Among them, F image-text Represents the refined visual-linguistic features, σ represents the activation function Sigmoid, Represents the transposed matrix of a text embedding containing multiple keywords.

[0046] Optionally, the structure of the vision-language detection module in step S3 includes:

[0047] (15) Output N object position predictions through regression, use position regression to locate the position of industrial defect objects, use task-aligned ground truth label assignment to select the correct object position prediction, and use each selected object position prediction as a positive sample to match the text as a classification label;

[0048] (16) On the text feature embedding T, the region-text contrast loss is calculated by the similarity between the object region and the text T and the cross entropy between the object and the text; the IOU loss and the distributed focal loss are used to optimize the prediction of the object position; among them, when the input image contains the real box, and are calculated, otherwise they are not calculated.

[0049] In a second aspect, an embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the industrial defect detection method based on visual-language cues in the first aspect or any possible implementation of the first aspect.

[0050] In a third aspect, an embodiment of the present invention provides an electronic device comprising: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, enable the device to execute the industrial defect detection method based on visual-language cues in the first aspect or any possible implementation of the first aspect.

[0051] In the technical solution provided by the present invention, the method includes building a feature extraction network of a target detection model; building a visual-language prompt module based on the feature extraction network; building a visual-language detection module, and connecting the feature extraction network and the visual-language prompt module to build an industrial defect detection model; training the constructed industrial defect detection model, encapsulating and deploying the trained industrial defect detection model, and obtaining defect detection results through the trained industrial defect detection model. This method accurately distinguishes industrial defects, accelerates the identification and positioning of industrial defect targets, and thus improves the accuracy of industrial defect image detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 A flowchart of an industrial defect detection method provided by an embodiment of the present invention;

[0054] Figure 2 A schematic diagram of an industrial defect detection model provided by an embodiment of the present invention;

[0055] Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0057] It should be understood that the embodiments described are only a portion of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0058] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the embodiments of the present invention are also intended to include plural forms, unless the context clearly indicates other meanings.

[0059] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the associated objects.

[0060] The word "if," as used herein, may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0061] Figure 1 A flowchart of an industrial defect detection method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes:

[0062] Step S1: Build a feature extraction network for the target detection model.

[0063] In the embodiment of the present invention, the input image and language, such as Figure 2 As shown in the figure, the feature extraction network consists of a segmentation model, a convolutional expert model, a language encoder, and a mask domain adaptation module; the segmentation model branch, the convolutional expert model branch, and the language encoder branch extract features in parallel;

[0064] The segmentation model branch includes an encoder structure, wherein the encoder structure is divided into a first attention module, a second attention module, a third attention module and a fourth attention module;

[0065] The convolution expert model branch includes a first convolution module, a second convolution module, a third convolution module and a fourth convolution module.

[0066] In the embodiment of the present invention, the structure of the mask domain adaptation module includes:

[0067] (1) Align the feature maps of the first, second, third, and fourth attention modules of the split-all-large-model branches with the feature maps of the first, second, third, and fourth convolution modules of the convolution expert model branches. The formula is as follows:

[0068] L = γ × D (L1 (F i ), L2(S j ));

[0069] Among them, L1 and L2 represent mapping functions, F i , i∈{1, 2, 3, 4} represents the feature maps in the first attention module, the second attention module, the third attention module, and the fourth attention module that split all large model branches; S j , j∈{1, 2, 3, 4} represents the feature maps in the first convolution module, the second convolution module, the third convolution module, and the fourth convolution module of the convolution expert model branch; D represents the alignment loss function, which is used to penalize different F i and S j The distance between them, γ is the weight to balance the two losses; the formula of D is:

[0070]

[0071] (2) Divide the features of the first attention module, the second attention module, the third attention module and the fourth attention module of all large model branches into N groups according to the feature block size of 32 to obtain the neighborhood multi-scale features where N∈2 k , k∈N + , i∈{1, 2, 3, 4};

[0072] (3) According to the random sampling method, some feature blocks are evenly masked. The two-dimensional index of each feature block is defined as Randomly sample M indices to get a binary mask, and i Multiply to obtain the multi-scale features of the mask The formula is:

[0073]

[0074]

[0075] in, is the range of M indexes extracted; is a binary mask, and P is the operation of segmenting the image into image blocks;

[0076] (4) Obtaining multi-scale features of the mask Finally, a mask decoder is built to reconstruct the masked features and obtain the features of the first attention module, the second attention module, the third attention module and the fourth attention module of all the branches of the segmentation model after reconstruction. Its mask decoder formula is:

[0077]

[0078] (5) Introduce a multi-scale domain optimization function to additionally optimize the domain adaptation layer. When the optimization function is infinitely small, the optimization of the domain adaptation layer will stop. Its optimization function F target The formula is:

[0079]

[0080] Among them, n=4 represents the number of different multi-scale features, S i , j∈{1, 2, 3, 4} represents the feature maps in the first convolution module, the second convolution module, the third convolution module and the fourth convolution module of the convolution expert model branch.

[0081] Step S2: Building a visual-language prompt module based on the feature extraction network in step S1.

[0082] In the embodiment of the present invention, the visual-language prompt module in step S2 is composed of a detailed visual prompt module and a visual-language interaction module.

[0083] In the embodiment of the present invention, the structure of the detailed visual prompt module includes:

[0084] (6) The feature maps of the first attention module, the second attention module, the third attention module and the fourth attention module of all the large model branches after segmentation are reconstructed. Each pixel is scored by the energy evaluation function to obtain the uncertainty score M of each pixel position k , the formula is:

[0085]

[0086] Where k is the normalization constant, N i is the set of adjacent pixels of the i-th pixel, is the similarity measure between the i-th pixel and the j-th pixel;

[0087] (7) Image block segmentation with different neighborhood sizes M k ; For the position of a single pixel in the feature space, the set of neighboring pixels It is expressed as:

[0088]

[0089] Where (h′, w′) is the relative position of the pixel;

[0090] (8) Given a set of segmented neighboring pixels, a sparse selection mechanism is used to select high-frequency pixels. Sparse sample selection is completed by selecting the top-k pixels whose internal pixel values ​​are greater than the mean. The formula is:

[0091]

[0092] in, express Any pixel in represents the pixel-level mean in the neighborhood space, Represents the selected sparse samples;

[0093] (9) Introduce an IoU-based optimization mechanism to iteratively optimize the activation mechanism of features; the goal of the optimization mechanism is to make the selected high-frequency pixels fall within the area of ​​the real frame; the optimization mechanism uses an additional detection head to Perform regression prediction and optimize the effect of feature activation through CIoU.

[0094] In the embodiment of the present invention, the structure of the visual language interaction module includes:

[0095] (11) The feature maps of the first attention module, the second attention module, the third attention module and the fourth attention module of all the large model branches after segmentation are reconstructed. and After multiplication, the feature map S in the first convolution module, the second convolution module, the third convolution module and the fourth convolution module of the convolution expert model branch j Add together to get the refined visual feature F L , the formula is:

[0096]

[0097] (12) By embedding the text feature T and image feature F generated by the text encoder L ∈R CxHxW (L∈{1, 2, 3, 4}), use the maximum Sigmoid attention to query the semantic features of text-image matching; aggregate text features into image features, the formula is:

[0098]

[0099] Among them, F image-text Represents the refined visual-linguistic features, σ represents the activation function Sigmoid, Represents the transposed matrix of a text embedding containing multiple keywords.

[0100] Step S3: Build a vision-language detection module and connect the feature extraction network in step S1 and the vision-language prompt module in step S2 to build an industrial defect detection model.

[0101] In the embodiment of the present invention, the structure of the vision-language detection module in step S3 includes:

[0102] (15) Output N object position predictions through regression, use position regression to locate the position of industrial defect objects, use task-aligned ground truth label assignment to select the correct object position prediction, and use each selected object position prediction as a positive sample to match the text as a classification label;

[0103] (16) On the text feature embedding T, the region-text contrast loss is calculated by the similarity between the object region and the text T and the cross entropy between the object and the text; the IOU loss and the distributed focal loss are used to optimize the prediction of the object position; among them, when the input image contains the real box, and are calculated, otherwise they are not calculated.

[0104] Step S4: train the industrial defect detection model constructed in step S3, encapsulate and deploy the trained industrial defect detection model, and obtain defect detection results through the trained industrial defect detection model.

[0105] In the technical solution provided by the present invention, the method includes building a feature extraction network of a target detection model; building a visual-language prompt module based on the feature extraction network; building a visual-language detection module, and connecting the feature extraction network and the visual-language prompt module to build an industrial defect detection model; training the constructed industrial defect detection model, encapsulating and deploying the trained industrial defect detection model, and obtaining defect detection results through the trained industrial defect detection model. This method accurately distinguishes industrial defects, accelerates the identification and positioning of industrial defect targets, and thus improves the accuracy of industrial defect image detection.

[0106] Each step of the embodiment of the present invention may be performed by an electronic device, including but not limited to a mobile phone, a tablet computer, a portable PC, a desktop computer, etc.

[0107] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is run, the electronic device where the computer-readable storage medium is located is controlled to execute the above-mentioned embodiment of the industrial defect detection method based on visual-language prompts.

[0108] Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, the electronic device 21 includes: a processor 211, a memory 212, and a computer program 213 stored in the memory 212 and executable on the processor 211. When the computer program 213 is executed by the processor 211, the industrial defect detection method based on visual-language cues in the embodiment is implemented. To avoid repetition, they are not described here one by one.

[0109] The electronic device 21 includes, but is not limited to, a processor 211 and a memory 212. Those skilled in the art will understand that Figure 3 It is only an example of the electronic device 21 and does not constitute a limitation of the electronic device 21. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0110] The processor 211 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0111] The memory 212 can be an internal storage unit of the electronic device 21, such as a hard disk or memory of the electronic device 21. The memory 212 can also be an external storage device of the electronic device 21, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash memory card, etc. equipped on the electronic device 21. Furthermore, the memory 212 can also include both an internal storage unit of the electronic device 21 and an external storage device. The memory 212 is used to store computer programs and other programs and data required by the network device. The memory 212 can also be used to temporarily store data that has been output or is about to be output.

[0112] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0113] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for industrial defect detection based on visual-linguistic cues, characterized in that: The method comprises: Step S1: Building a feature extraction network for the target detection model; Step S2: Building a visual-language prompt module based on the feature extraction network in step S1; Step S3: Building a vision-language detection module and connecting the feature extraction network in step S1 with the vision-language prompt module in step S2 to construct an industrial defect detection model; Step S4: training the industrial defect detection model constructed in step S3, encapsulating and deploying the trained industrial defect detection model, and obtaining defect detection results through the trained industrial defect detection model; The feature extraction network consists of a segmentation model, a convolutional expert model, a language encoder, and a mask domain adaptation module; the segmentation model branch, the convolutional expert model branch, and the language encoder branch extract features in parallel; The segmentation model branch includes an encoder structure, wherein the encoder structure is divided into a first attention module, a second attention module, a third attention module and a fourth attention module; The convolution expert model branch includes a first convolution module, a second convolution module, a third convolution module and a fourth convolution module; The visual-language prompt module in step S2 is composed of a refined visual prompt module and a visual-language interaction module; The structure of the detailed visual prompt module includes: (6) The feature maps of the first attention module, the second attention module, the third attention module and the fourth attention module of all the large model branches after segmentation are reconstructed. Each pixel is scored by the energy evaluation function to obtain the uncertainty score M of each pixel position k , the formula is: Where k is the normalization constant, N i is the set of adjacent pixels of the i-th pixel, is the similarity measure between the i-th pixel and the j-th pixel; (7) Image block segmentation with different neighborhood sizes M k ; For the position of a single pixel in the feature space, the set of neighboring pixels It is expressed as: Where (h′, w′) is the relative position of the pixel; (8) Given a set of segmented neighboring pixels, a sparse selection mechanism is used to select high-frequency pixels. Sparse sample selection is completed by selecting the top-k pixels whose internal pixel values ​​are greater than the mean. The formula is: in, express Any pixel in represents the pixel-level mean in the neighborhood space, Represents the selected sparse samples; (9) Introduce an IoU-based optimization mechanism to iteratively optimize the activation mechanism of features; the goal of the optimization mechanism is to make the selected high-frequency pixels fall within the area of ​​the real frame; the optimization mechanism uses an additional detection head to Perform regression prediction and optimize the effect of feature activation through CIoU; The structure of the visual language interaction module includes: (11) The feature maps of the first attention module, the second attention module, the third attention module and the fourth attention module of all the large model branches after segmentation are reconstructed. and After multiplication, the feature map S in the first convolution module, the second convolution module, the third convolution module and the fourth convolution module of the convolution expert model branch j Add together to get the refined visual feature F L , the formula is: (12) By embedding the text feature T and image feature F generated by the text encoder L ∈R CxHxW (L∈{1,2,3,4}), use the maximum Sigmoid attention to query the semantic features of text-image matching; aggregate text features into image features, the formula is: Among them, F image-text Represents the refined visual-linguistic features, σ represents the activation function Sigmoid, Represents the transposed matrix of the text embedding containing multiple keywords; The structure of the vision-language detection module in step S3 includes: (15) Output N object position predictions through regression, use position regression to locate the position of industrial defect objects, use task-aligned ground truth label assignment to select the correct object position prediction, and use each selected object position prediction as a positive sample to match the text as a classification label; (16) On the text feature embedding T, the region-text contrast loss is calculated by the similarity between the object region and the text T and the cross entropy between the object and the text; the IOU loss and the distributed focal loss are used to optimize the prediction of the object position; among them, when the input image contains the real box, and are calculated, otherwise they are not calculated.

2. The method according to claim 1, characterized in that The structure of the mask domain adaptation module includes: (1) Align the feature maps of the first, second, third, and fourth attention modules of the split-all-large-model branches with the feature maps of the first, second, third, and fourth convolution modules of the convolution expert model branches. The formula is as follows: L=γ×D(L1(F i ),L2(S j )); Among them, L1 and L2 represent mapping functions, F i ,i∈{1,2,3,4} represents the feature maps in the first attention module, the second attention module, the third attention module and the fourth attention module that split all large model branches; S j ,j∈{1,2,3,4} represents the feature maps in the first convolution module, the second convolution module, the third convolution module and the fourth convolution module of the convolution expert model branch; D represents the alignment loss function, which is used to penalize different F i and S j The distance between them, γ is the weight to balance the two losses; the formula of D is: (2) Divide the features of the first attention module, the second attention module, the third attention module and the fourth attention module of all large model branches into N groups according to the feature block size of 32 to obtain the neighborhood multi-scale features where N∈2 k ,k∈N + ,i∈{1,2,3,4}; (3) According to the random sampling method, some feature blocks are evenly masked. The two-dimensional index of each feature block is defined as Randomly sample M indices to get a binary mask, and i Multiply to obtain the multi-scale features of the mask The formula is: in, is the range of M indexes extracted; is a binary mask, and P is the operation of segmenting the image into image blocks; (4) Obtaining multi-scale features of the mask Finally, a mask decoder is built to reconstruct the masked features and obtain the features of the first attention module, the second attention module, the third attention module and the fourth attention module of all the branches of the segmentation model after reconstruction. Its mask decoder formula is: (5) Introduce a multi-scale domain optimization function to additionally optimize the domain adaptation layer. When the optimization function is infinitely small, the optimization of the domain adaptation layer will stop. Its optimization function F target The formula is: Among them, n=4 represents the number of different multi-scale features, S j ,j∈{1,2,3,4} represents the feature maps in the first convolution module, the second convolution module, the third convolution module and the fourth convolution module of the convolution expert model branch.

3. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the computer-readable storage medium is controlled to execute the industrial defect detection method based on visual-linguistic prompts according to any one of claims 1 to 2.

4. An electronic device, characterized in that: include: one or more processors; Memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to perform the industrial defect detection method based on visual-language cues as described in any one of claims 1 to 2.

Images