Industrial defect generation method based on physical constraint and language prompt

Abstract

The invention relates to the technical field of industrial scene defect detection, and particularly provides an industrial defect generation method based on physical constraint and language prompt. The method comprises the following steps: designing an open scene feature optimization module; designing a multi-prompt feature optimization module; designing an attention-driven physical modeling module; through the open scene feature optimization module, the multi-prompt feature optimization module and the attention-driven physical modeling module, the industrial defect generation model is constructed, reasoning is performed on the industrial defect generation model, the industrial defect is generated, and the method improves the accuracy of industrial defect generation.

Description

Technical Field

[0001] This invention relates to the field of industrial defect detection technology, and in particular to a method for generating industrial defects based on physical constraints and language prompts. Background Technology

[0002] The core objective of visual defect detection is to identify rare patterns that deviate from the normal distribution, relying solely on normal or defective samples. This task is highly valuable in scenarios such as industrial quality inspection and medical imaging. However, the rarity and unpredictability of defects themselves, along with the high cost of annotation, make traditional supervised methods difficult to implement.

[0003] In recent years, researchers have turned to unsupervised or self-supervised paradigms, using synthetic pseudo-defects to simulate rare patterns and train detection models. While this approach has made significant progress on closed datasets, challenges remain. First, existing defect synthesis methods are almost entirely trained or designed on closed categories. Whether based on crop-paste data augmentation or relying on fine-tuning generative models, their defect semantics are constrained by the training set distribution. These strategies quickly fail when faced with new objects and defects in open scenes. These methods cannot conceive of never-before-seen defects, nor can they transfer known defects to entirely new categories. Generating defects with arbitrary open scene semantics and categories from arbitrary text or images without retraining or relying on defect samples remains an unsolved challenge. Second, different defects exhibit complexities in image representation and imaging conditions, making it difficult for existing generation methods to adhere to established textual and physical constraints. For example, glass breakage produces local refractive distortions. These phenomena are caused by the coupling of material properties and lighting interactions. However, existing generation methods focus almost exclusively on the shape defects themselves, ignoring the material's response under physical lighting and failing to incorporate fine-grained semantics (such as position and lighting) in the text description into the optimization objective. This results in significant visual differences between the synthesized results and the real images or language descriptions. Summary of the Invention

[0004] In view of this, the present invention provides an industrial defect generation method based on physical constraints and language prompts to improve the accuracy of industrial defect generation.

[0005] In a first aspect, the present invention provides a method for generating industrial defects based on physical constraints and language prompts, the method comprising: Step 1: Design an open scene feature optimization module; Step 2: Design a multi-prompt feature optimization module; Step 3: Design an attention-driven physics modeling module; Step 4: Construct an industrial defect generation model using the open scene feature optimization module, the multi-cue feature optimization module, and the attention-driven physical modeling module, and then infer from it to generate industrial defects.

[0006] Optionally, the open scene feature optimization module in step 1 includes open vocabulary contrast loss, pixel scorer, dynamic mask generator, and denoiser; the open scene feature optimization module allows the model to perform secondary fine-tuning of defective features while maintaining image quality, so as to control the generation of open scene text-image; a. The open-vocabulary contrastive loss is used to score the similarity of each pixel of the latent features, and its expression is: ; in, and are the normalized vectors of the image and text encoders, respectively; sin is the similarity function; The temperature parameter is learnable; The score is calculated based on pixel-level loss. Used to quantify the degree of matching between each sample and the target text description; b. The pixel scorer is used to calculate the score. The average value is used to measure the central tendency of batch scores, and its expression is: ; in, B For batch size, The threshold for the dynamic mask generator is used to distinguish the pixel-by-pixel quality of the generated image; c. Dynamic mask generator Used to filter pixels that exceed the pixel scoring threshold; its value is determined by the score. With threshold The comparison result determines its expression, which is: ; d. The denoiser is used to denoise the mask generated by the dynamic mask generator, assuming the sample score... Greater than the threshold If noise is added to the generated image, then noise is added; otherwise, the generated image remains unchanged. The expression for this is: ; in, To generate an initial image; Used to control noise vector The strength; This is a noise vector sampled from a standard normal distribution, used to simulate the natural variability of the data.

[0007] Optionally, the multi-cue feature optimization module in step 2 includes an image defect embedding optimization function and an image gradient updater; f. The image defect embedding optimization function optimizes the quality of defect generation by using the differences between multiple cues and latent features. The function calculates the matching relationship between multiple text cue embeddings and the image embedding, and optimizes the generated image based on the matching relationship. Its expression is: ; in, Image embedding; For text prompt embedding; J=S, D, N represent simple defect embedding, detailed defect embedding, and normal defect embedding, respectively; The optimization goal is to minimize the difference between the text hint embedding and the image embedding, thereby optimizing the text hint to describe the image content; g. Loss value obtained by optimizing the image defect embedding function The image gradient updater updates latent features online, and its expression is: ; in, This is the learning rate.

[0008] Optionally, the attention-driven physics modeling module in step 3 includes: h. Potential features Combining with noise N yields new potential defect features. To ensure the authenticity of defect locations, a defect mask is generated using a sparse attention map of latent features. Used to indicate the location where defects are generated; generates sparse attention maps. The expression is: ; in, This is the original attention graph after text cross-attention; For dark masking, For shape masks, their expressions are as follows: ; in, S The center position of the feature shape; P For pixel position; σ Here, is the distributed control parameter; exp is the exponential transformation. ; i. Apply the dark mask and shape mask Add them together to obtain the defect mask. Potential features after defect location constraints Its expression is: ; in, X n The current potential features are then defined; finally, through an encoder-decoder structure, physical lighting constraints on the defect generation region are generated, resulting in the potential features after lighting constraints. Its expression is: ; in, For encoder.

[0009] Optionally, step 4 includes constructing an industrial defect generation model by cascading the open scene feature optimization module, the multi-cue feature optimization module, and the attention-driven physical modeling module; the industrial defect generation model does not need to be trained and can be directly used for inference to generate industrial defects.

[0010] In a second aspect, embodiments of the present invention provide a computer-readable storage medium comprising a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to execute the industrial defect generation method based on physical constraints and language prompts in the first aspect or any possible implementation thereof.

[0011] Thirdly, embodiments of the present invention provide an electronic device, including: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to perform the industrial defect generation method based on physical constraints and language prompts in the first aspect or any possible implementation of the first aspect.

[0012] The technical solution provided by this invention includes a method that designs an open scene feature optimization module; designs a multi-cue feature optimization module; designs an attention-driven physical modeling module; and constructs an industrial defect generation model by using the open scene feature optimization module, the multi-cue feature optimization module, and the attention-driven physical modeling module, and then infers from the model to generate industrial defects. This method improves the accuracy of industrial defect generation. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A flowchart illustrating an industrial defect generation method based on physical constraints and language prompts provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of the industrial defect generation model provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0017] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention are also intended to include the plural forms unless the context clearly indicates otherwise.

[0018] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0019] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0020] Figure 1 A flowchart of an industrial defect generation method based on physical constraints and language prompts provided in an embodiment of the present invention is shown below. Figure 1 As shown, the method includes: Step 1: Design an open scene feature optimization module.

[0021] In this embodiment of the invention, the open scene feature optimization module in step 1 includes an open vocabulary contrast loss, a pixel scorer, a dynamic mask generator, and a denoiser; the open scene feature optimization module allows the model to perform secondary fine-tuning of defective features while maintaining image quality, which is used to control the generation of open scene text-images. a. The open-vocabulary contrastive loss is used to score the similarity of each pixel of the latent features, and its expression is: ; in, and are the normalized vectors of the image and text encoders, respectively; sin is the similarity function; The temperature parameter is learnable; The score is calculated based on pixel-level loss. Used to quantify the degree of matching between each sample and the target text description; b. The pixel scorer is used to calculate the score. The average value is used to measure the central tendency of batch scores, and its expression is: ; in, B For batch size, The threshold for the dynamic mask generator is used to distinguish the pixel-by-pixel quality of the generated image; c. Dynamic mask generator Used to filter pixels that exceed the pixel scoring threshold; its value is determined by the score. With threshold The comparison result determines its expression, which is: ; d. The denoiser is used to denoise the mask generated by the dynamic mask generator, assuming the sample score... Greater than the threshold If noise is added to the generated image, then noise is added; otherwise, the generated image remains unchanged. The expression for this is: ; in, To generate an initial image; Used to control noise vector The strength; This is a noise vector sampled from a standard normal distribution, used to simulate the natural variability of the data.

[0022] Step 2: Design a multi-prompt feature optimization module.

[0023] In this embodiment of the invention, the multi-hint feature optimization module in step 2 includes an image defect embedding optimization function and an image gradient updater; f. The image defect embedding optimization function optimizes the quality of defect generation by using the differences between multiple cues and latent features. The function calculates the matching relationship between multiple text cue embeddings and the image embedding, and optimizes the generated image based on the matching relationship. Its expression is: ; in, Image embedding; For text prompt embedding; J=S, D, N represent simple defect embedding, detailed defect embedding, and normal defect embedding, respectively; The optimization goal is to minimize the difference between the text hint embedding and the image embedding, thereby optimizing the text hint to describe the image content; g. Loss value obtained by optimizing the image defect embedding function The image gradient updater updates latent features online, and its expression is: ; in, This is the learning rate.

[0024] Step 3: Design an attention-driven physics modeling module.

[0025] In this embodiment of the invention, the attention-driven physics modeling module in step 3 includes: h. Potential features Combining with noise N yields new potential defect features. To ensure the authenticity of defect locations, a defect mask is generated using a sparse attention map of latent features. Used to indicate the location where defects are generated; generates sparse attention maps. The expression is: ; in, This is the original attention graph after text cross-attention; For dark masking, For shape masks, their expressions are as follows: ; in, S The center position of the feature shape; P For pixel position; σ Here, is the distributed control parameter; exp is the exponential transformation. ; i. Apply the dark mask and shape mask Add them together to obtain the defect mask. Potential features after defect location constraints Its expression is: ; in, X n The current potential features are then defined; finally, through an encoder-decoder structure, physical lighting constraints on the defect generation region are generated, resulting in the potential features after lighting constraints. Its expression is: ; in, For encoder.

[0026] Step 4: Construct an industrial defect generation model using the open scene feature optimization module, the multi-cue feature optimization module, and the attention-driven physical modeling module, and then infer from it to generate industrial defects.

[0027] In embodiments of the present invention, such as Figure 2 As shown, step 4 includes constructing an industrial defect generation model by cascading the open scene feature optimization module, the multi-cue feature optimization module, and the attention-driven physical modeling module; the industrial defect generation model does not need to be trained and can be directly used for inference to generate industrial defects.

[0028] The technical solution provided by this invention includes a method that designs an open scene feature optimization module; designs a multi-cue feature optimization module; designs an attention-driven physical modeling module; and constructs an industrial defect generation model by using the open scene feature optimization module, the multi-cue feature optimization module, and the attention-driven physical modeling module, and then infers from the model to generate industrial defects. This method improves the accuracy of industrial defect generation.

[0029] The various steps in the embodiments of the present invention can be performed by an electronic device. This electronic device includes, but is not limited to, tablet computers, portable PCs, and desktop computers.

[0030] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is running, it controls the electronic device containing the computer-readable storage medium to execute the above-described embodiment of the industrial defect generation method based on physical constraints and language prompts.

[0031] Figure 3 A schematic diagram of an electronic device provided in an embodiment of the present invention, such as... Figure 3As shown, the electronic device 21 includes a processor 211, a memory 212, and a computer program 213 stored in the memory 212 and executable on the processor 211. When the computer program 213 is executed by the processor 211, it implements the industrial defect generation method based on physical constraints and language prompts in the embodiment. To avoid repetition, it will not be described in detail here.

[0032] Electronic device 21 includes, but is not limited to, processor 211 and memory 212. Those skilled in the art will understand that... Figure 3 This is merely an example of electronic device 21 and does not constitute a limitation on electronic device 21. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0033] The processor 211 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0034] The memory 212 can be an internal storage unit of the electronic device 21, such as a hard disk or RAM of the electronic device 21. The memory 212 can also be an external storage device of the electronic device 21, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or FlashCard equipped on the electronic device 21. Furthermore, the memory 212 can include both internal and external storage units of the electronic device 21. The memory 212 is used to store computer programs and other programs and data required by network devices. The memory 212 can also be used to temporarily store data that has been output or will be output.

[0035] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0036] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating industrial defects based on physical constraints and verbal prompts, characterized in that, The method includes: Step 1: Design an open scene feature optimization module; Step 2: Design a multi-prompt feature optimization module; Step 3: Design an attention-driven physics modeling module; Step 4: Construct an industrial defect generation model using the open scene feature optimization module, the multi-cue feature optimization module, and the attention-driven physical modeling module, and then infer from it to generate industrial defects.

2. The method according to claim 1, characterized in that, The open scene feature optimization module in step 1 includes open vocabulary contrast loss, pixel scorer, dynamic mask generator, and denoiser; the open scene feature optimization module allows the model to perform secondary fine-tuning of defective features while maintaining image quality, which is used to control the generation of open scene text-image. a. The open-vocabulary contrastive loss is used to score the similarity of each pixel of the latent features, and its expression is: ； in, and are the normalized vectors of the image and text encoders, respectively; sin is the similarity function; The temperature parameter is learnable; The score is calculated based on pixel-level loss. Used to quantify the degree of matching between each sample and the target text description; b. The pixel scorer is used to calculate the score. The average value is used to measure the central tendency of batch scores, and its expression is: ； in, B For batch size, The threshold for the dynamic mask generator is used to distinguish the pixel-by-pixel quality of the generated image; c. Dynamic mask generator Used to filter pixels that exceed the pixel scoring threshold; its value is determined by the score. With threshold The comparison result determines its expression, which is: ； d. The denoiser is used to denoise the mask generated by the dynamic mask generator, assuming the sample score... Greater than the threshold If noise is added to the generated image, then noise is added; otherwise, the generated image remains unchanged. The expression for this is: ； in, To generate an initial image; Used to control noise vector The strength; This is a noise vector sampled from a standard normal distribution, used to simulate the natural variability of the data.

3. The method according to claim 2, characterized in that, The multi-hint feature optimization module in step 2 includes an image defect embedding optimization function and an image gradient updater. f. The image defect embedding optimization function optimizes the quality of defect generation by using the differences between multiple cues and latent features. The function calculates the matching relationship between multiple text cue embeddings and the image embedding, and optimizes the generated image based on the matching relationship. Its expression is: ； in, Image embedding; For text prompt embedding; J=S, D, N represent simple defect embedding, detailed defect embedding, and normal defect embedding, respectively; The optimization goal is to minimize the difference between the text hint embedding and the image embedding, thereby optimizing the text hint to describe the image content; g. Loss value obtained by optimizing the image defect embedding function The image gradient updater updates latent features online, and its expression is: ； in, This is the learning rate.

4. The method according to claim 3, characterized in that, The attention-driven physics modeling module in step 3 includes: h. Potential features Combining with noise N yields new potential defect features. To ensure the authenticity of defect locations, a defect mask is generated using a sparse attention map of latent features. Used to indicate the location where defects are generated; generates sparse attention maps. The expression is: ； in, This is the original attention graph after text cross-attention; For dark masking, For shape masks, their expressions are as follows: ； in, S The center position of the feature shape; P For pixel position; σ Here, is the distributed control parameter; exp is the exponential transformation. ； i. Apply the dark mask and shape mask Add them together to obtain the defect mask. Potential features after defect location constraints Its expression is: ； in, X n The current potential features are then defined; finally, through an encoder-decoder structure, physical lighting constraints on the defect generation region are generated, resulting in the potential features after lighting constraints. Its expression is: ； in, For encoder.

5. The method according to claim 4, characterized in that, Step 4 involves constructing an industrial defect generation model by cascading the open scene feature optimization module, the multi-cue feature optimization module, and the attention-driven physical modeling module. The industrial defect generation model does not require training and can be directly used for inference to generate industrial defects.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the industrial defect generation method based on physical constraints and language prompts as described in any one of claims 1 to 5.

7. An electronic device, characterized in that, include: One or more processors; Memory; And one or more computer programs, wherein the one or more computer programs are stored in the memory, the one or more computer programs including instructions that, when executed by the device, cause the device to perform the industrial defect generation method based on physical constraints and language prompts as described in any one of claims 1 to 5.

Images