Defect image diffusion generation method and device, equipment and storage medium

Abstract

The invention discloses a defect image diffusion generation method and device, equipment and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: determining a target defect type and a defect size parameter corresponding to a target defect according to a natural language prompt text input by a user; building a defect physical vector based on the target defect type and the defect size parameter; adjusting the diffusion noise amplitude in the diffusion process based on the defect physical vector, and generating an initial defect image corresponding to the initial workpiece noise image; and performing physical consistency verification on the initial defect image, and outputting a target defect image according to a verification result. The diffusion noise amplitude in the diffusion process is adjusted based on the defect physical vector corresponding to the target defect, the initial defect image is generated, and the target defect image is output according to the verification result of the initial defect image, so that the problem that the defect image generation method in the prior art cannot pay attention to the physical size of the defect is solved. And the accuracy of the generated defect image is not high.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for generating a defective image diffusion. Background Art

[0002] In industrial manufacturing and visual inspection, the generation and simulation of defect images is crucial for improving product quality, optimizing production processes, and increasing inspection efficiency. Traditional defect image generation methods rely primarily on the collection and processing of actual samples, which is time-consuming and labor-intensive, and struggles to capture all possible defect types and morphologies.

[0003] Currently, existing approaches typically generate defect images based on Generative Adversarial Networks (GANs) through adversarial training of a generator and a discriminator. However, this approach primarily focuses on the visual appearance of defects and fails to explicitly model or dynamically control the physical dimensions of the defects (such as width, depth, length, and area), resulting in low accuracy in the generated defect images. Summary of the Invention

[0004] The main purpose of this application is to provide a defect image diffusion generation method, device, equipment and storage medium, aiming to solve the technical problem that the defect image generation method in the existing technology cannot pay attention to the physical size of the defect, resulting in low accuracy of the generated defect image.

[0005] To achieve the above objectives, the present application proposes a method for generating a defect image diffusion, the method comprising: Determine the target defect type and defect size parameters corresponding to the target defect according to the natural language prompt text input by the user; Constructing a defect physical vector corresponding to the target defect based on the target defect type and the defect size parameter; Adjusting the diffusion noise amplitude during the diffusion process based on the defect physical vector to generate an initial defect image corresponding to the initial workpiece noise image; Physical consistency verification is performed on the initial defect image, and a target defect image is output according to the verification result.

[0006] In one embodiment, the step of constructing a defect physical vector corresponding to the target defect based on the target defect type and the defect size parameter includes: Mapping the target defect type to a category index using a preset domain vocabulary; Determine the size unit and size value according to the defect size parameter; Performing unit normalization processing on the defect size parameter based on the size unit and the size value to obtain a standard defect size parameter; A defect physical vector corresponding to the target defect is constructed based on the category index and the standard defect size parameter.

[0007] In one embodiment, the step of constructing the defect physical vector corresponding to the target defect based on the category index and the standard defect size parameter includes: constructing a raw physical parameter list based on the category index and the standard defect size parameter; Performing linear normalization processing on the defect size parameters in the original physical parameter list using a preset scale reference table to obtain normalized continuous size parameters; A defect physical vector corresponding to the target defect is constructed based on the category index and the normalized continuous size parameter.

[0008] In one embodiment, the step of adjusting the diffusion noise amplitude during the diffusion process based on the defect physical vector to generate an initial defect image corresponding to the initial workpiece noise image includes: Generate benchmark noise parameters through benchmark timing function; Mapping the defect physical vector into a noise scheduling weight coefficient using a multi-layer perceptron; The diffusion noise amplitude in the diffusion process is adjusted based on the reference noise parameter, the time-varying gating coefficient and the noise scheduling weight coefficient to generate an initial defect image corresponding to the initial workpiece noise image.

[0009] In one embodiment, the step of adjusting the diffusion noise amplitude during the diffusion process based on the baseline noise parameter, the time-varying gating coefficient, and the noise scheduling weight coefficient to generate an initial defect image corresponding to the initial workpiece noise image includes: Adjusting the diffusion noise amplitude during the diffusion process based on the reference noise parameter, the time-varying gating coefficient, and the noise scheduling weight coefficient; The initial workpiece noise image is subjected to denoising and denoising processing respectively according to the adjusted diffusion noise amplitude to obtain the target workpiece noise feature map; Based on the information fusion attention mechanism, the defect physical vector is used as the query vector and the target workpiece noise feature map is used as the key value to inject into each attention layer of the conditional diffusion generation module; An initial defect image is generated by the conditional diffusion generation module based on the defect physical vector and the target workpiece noise feature map.

[0010] In one embodiment, the step of performing physical consistency verification on the initial defect image and outputting a target defect image according to the verification result includes: Predicting defect physical indicators corresponding to the initial defect image using a preset defect measurement network; determining a defect error between the defect physical indicator and the defect physical vector; Physical consistency verification is performed on the initial defect image based on the defect error, and a target defect image is output according to the verification result.

[0011] In one embodiment, the step of performing physical consistency verification on the initial defect image based on the defect error and outputting a target defect image according to the verification result includes: determining a defect error threshold corresponding to the initial defect image based on a process tolerance; comparing the defect error with the defect error threshold; If the defect error does not exceed the defect error threshold, it is determined that the initial defect image meets the physical consistency requirement, and the initial defect image is output as a target defect image.

[0012] In addition, to achieve the above-mentioned purpose, the present application also proposes a defect image diffusion generation device, the device comprising: A defect information determination module is used to determine the target defect type and defect size parameters corresponding to the target defect according to the natural language prompt text input by the user; A physical vector construction module, configured to construct a defect physical vector corresponding to the target defect based on the target defect type and the defect size parameter; a defect image generating module, configured to adjust the diffusion noise amplitude during the diffusion process based on the defect physical vector, and generate an initial defect image corresponding to the initial workpiece noise image; The defect image verification module is used to perform physical consistency verification on the initial defect image and output a target defect image according to the verification result.

[0013] In addition, to achieve the above-mentioned purpose, the present application also proposes a defect image diffusion generation device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the defect image diffusion generation method as described above.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the steps of the defect image diffusion generation method described above are implemented.

[0015] The present application provides a defect image diffusion generation method, which discloses determining a target defect type and defect size parameters corresponding to a target defect based on a natural language prompt text input by a user; constructing a defect physical vector corresponding to the target defect based on the target defect type and defect size parameters; adjusting the diffusion noise amplitude in the diffusion process based on the defect physical vector to generate an initial defect image corresponding to the initial workpiece noise image; performing physical consistency verification on the initial defect image, and outputting a target defect image based on the verification result; since the present invention can adjust the diffusion noise amplitude in the diffusion process based on the defect physical vector corresponding to the target defect to generate an initial defect image, and output the target defect image based on the physical consistency verification result of the initial defect image, it solves the technical problem that the defect image generation method in the prior art cannot pay attention to the physical size of the defect, resulting in low accuracy of the generated defect image. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 A schematic diagram of the process flow provided for the first embodiment of the defect image diffusion generation method of the present application; Figure 2 This is an example image of the target defect in the defect heat map generation method of this application; Figure 3 A schematic diagram of the process flow provided for the second embodiment of the defect image diffusion generation method of the present application; Figure 4 A flowchart of the third embodiment of the defect image diffusion generation method of the present application is provided; Figure 5 This is a model structure diagram of the defect image diffusion generation method of this application; Figure 6 This is a schematic diagram of the module structure of the defect image diffusion generation device according to an embodiment of the present application; Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the defect image diffusion generation method in the embodiment of the present application.

[0019] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0021] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The main solution of the embodiment of the present application is: determining the target defect type and defect size parameters corresponding to the target defect based on the natural language prompt text input by the user; constructing the defect physical vector corresponding to the target defect based on the target defect type and defect size parameters; adjusting the diffusion noise amplitude in the diffusion process based on the defect physical vector to generate an initial defect image corresponding to the initial workpiece noise image; performing physical consistency verification on the initial defect image, and outputting the target defect image based on the verification result.

[0023] Since the existing method of generating defect images based on generative adversarial networks through adversarial training of generators and discriminators mainly focuses on the visual appearance of defects, it is unable to explicitly model and dynamically control the physical size of the defects, resulting in low accuracy of the generated defect images.

[0024] The present application provides a solution that can adjust the diffusion noise amplitude in the diffusion process based on the defect physical vector corresponding to the target defect, generate an initial defect image, and output the target defect image based on the physical consistency verification result of the initial defect image, thereby solving the technical problem that the defect image generation method in the prior art cannot pay attention to the physical size of the defect, resulting in low accuracy of the generated defect image.

[0025] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program execution capabilities, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the aforementioned functions, such as a defective image diffusion generation device. This embodiment and the following embodiments will be described below using a defective image diffusion generation device as an example (hereinafter referred to as the device).

[0026] Based on this, the embodiment of the present application provides a method for generating a defect image diffusion, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the defect image diffusion generation method of the present application.

[0027] In this embodiment, the defect image diffusion generation method includes steps S10 to S40: Step S10: determining the target defect type and defect size parameters corresponding to the target defect according to the natural language prompt text input by the user.

[0028] It should be understood that the target defect may be the defect that needs to be simulated and generated currently; accordingly, the natural language prompt text may be text describing relevant information of the target defect, for example, “a crack defect with a width of 0.5 mm”, etc.

[0029] It is understandable that the above-mentioned target defect type can be a defect type corresponding to the target defect, such as scratches, cracks, pits, etc., and this embodiment does not limit this. The above-mentioned defect size parameter can be a parameter used to describe the size information of the target defect, such as the specific value, material, unit, etc. of the size, length, and width of the defect, and this embodiment does not limit this. Figure 2 , Figure 2 This is an example diagram of the target defect in the defect heat map generation method of this application. The target defect in this application can be Figure 2 The shown external holes, paint holes, sintering (depth of 0.01mm), indentations, dark cracks (depth of 0.01mm), as well as internal holes, sand holes, slag inclusions (width of 0.3mm), etc.

[0030] In this embodiment, the user can enter a natural language prompt to specify the type and size parameter range of the defect to be generated. After receiving the natural language prompt, the system can perform regular text segmentation on the prompt, extract keywords, and obtain the target defect type and defect size parameters corresponding to the target defect. This allows the model to learn and generate the corresponding defect image based on the target defect type and defect size parameters. At the same time, these parameters can also serve as constraints during the model training process, helping the model better learn the characteristics and distribution patterns of defects.

[0031] Step S20: constructing a defect physical vector corresponding to the target defect based on the target defect type and the defect size parameters.

[0032] It should be noted that the above-mentioned defect physical vector can be a vector used to describe the physical properties of the target defect. In this embodiment, the physical properties of the target defect can include but are not limited to the type of defect (such as scratches, cracks, corrosion spots, etc.), size (such as length, width, diameter, etc.), shape (such as linear, circular, irregular, etc.), position (such as coordinate position in the image), direction (such as the inclination angle of the scratch) and severity (such as the depth of corrosion, the depth of the scratch), etc.

[0033] Furthermore, the step S20 includes: Step S201: Mapping the target defect type to a category index using a preset domain vocabulary.

[0034] It should be understood that the above-mentioned preset domain vocabulary may be a vocabulary that defines standardized terms in the field of industrial defect detection and the relationships between the terms; and the above-mentioned category index may be a numerical value used to uniquely identify the defect type.

[0035] In actual applications, models usually require numerical input rather than text input, so in this application, defect types can be converted into category indexes to simplify the input processing of the model. First, this application can pre-build a preset domain vocabulary to associate the defect type corresponding to each workpiece in the industrial field with a unique numerical index. After determining the target defect type corresponding to the target defect based on the natural language prompt text input by the user, the system can map the target defect type to the corresponding category index by querying the preset domain vocabulary. For example, for the defect type "crack", the preset domain vocabulary is queried and its corresponding category index is found to be "1". At this time, the defect type "crack" can be mapped to the numerical value "1". In this embodiment, by mapping the defect type to the category index, it can be ensured that different users or systems maintain consistent descriptions of the same defect type.

[0036] Step S202: Determine the size unit and size value according to the defect size parameter.

[0037] It can be understood that the above-mentioned dimensional units and dimensional values ​​can be parameters used to jointly describe the specific size of the target defect, wherein the dimensional value represents the specific size of the defect, and the dimensional unit specifies the measurement reference, for example, millimeter (mm), micrometer (μm), pixel, etc. This embodiment does not impose any restrictions on this.

[0038] Step S203: performing unit normalization processing on the defect size parameter based on the size unit and the size value to obtain a standard defect size parameter.

[0039] It should be understood that the above standard defect size parameters can be size parameters after unit normalization. In this embodiment, the unit normalization function can be used. Based on the size units and size values, all defect sizes are converted to millimeters to obtain standard defect size parameters. The unit normalization formula is as follows:

[0040] in, Represent the width, depth, length and area of ​​the defect respectively.

[0041] Step S204: constructing a defect physical vector corresponding to the target defect based on the category index and the standard defect size parameter.

[0042] Specifically, step S204 includes: constructing an original physical parameter list based on the category index and the standard defect size parameter; performing linear normalization processing on the defect size parameters in the original physical parameter list through a preset scale reference table to obtain normalized continuous size parameters; and constructing a defect physical vector corresponding to the target defect based on the category index and the normalized continuous size parameters.

[0043] It should be noted that the above original physical parameter list may be a parameter list consisting of category indexes and size parameters after unification of units. In this embodiment, the original physical parameter list may be in the form of ,in, Represents the category index.

[0044] It should be noted that the above-mentioned preset scale reference table can be a tool for defining and standardizing the mapping relationship between the physical size of the defect and the image pixel size. It can clarify the correspondence between the physical scale of the defect in the real scene and the pixel value in the generated defect image, thereby ensuring the scale consistency of the generated defect image. In this embodiment, a preset scale reference table can be introduced to linearly map the continuous defect size parameters to [0,1] to obtain the normalized continuous size parameters, and construct the defect physical vector v based on the category index and the normalized continuous size parameters, thereby solving the problem of inconsistent units and dimensions in industrial scenarios and facilitating unified model processing. Among them, the formula for linear normalization of the defect size parameters in this embodiment is as follows:

[0045]

[0046]

[0047]

[0048]

[0049] Where, Identify the normalized continuous size parameters, namely the normalized defect width, depth, length and area respectively; and Represent the minimum and maximum width of the target defect type in the training data; and represent the minimum and maximum depths of the target defect type in the training data; and Respectively represent the minimum and maximum length of the target defect type in the training data; and Represent the minimum and maximum areas of the target defect type in the training data, respectively. 、 、 、 、 、 、 and All can be obtained from the preset scale benchmark table.

[0050] It should be understood that in this embodiment, the defect physical vector can be generated by encoding the category index corresponding to the defect type and the physical parameters of the defect (such as width w, depth d, length l, and area s). During encoding, the category dimension is encoded using one-hot encoding or learnable embedding, while the normalized continuous dimension parameters remain in numerical form and are subsequently directly involved in noise scheduling and attention. In practical applications, the defect physical vector is used to represent the physical properties of the defect and can be used to dynamically adjust the noise amplitude during the subsequent diffusion process to ensure that the defects in the generated defect image conform to actual physical laws.

[0051] Step S30: adjusting the diffusion noise amplitude during the diffusion process based on the defect physical vector to generate an initial defect image corresponding to the initial workpiece noise image.

[0052] It should be noted that the diffusion noise amplitude may be the intensity of the Gaussian noise added to the original image during the diffusion process, wherein the larger the noise amplitude is, the stronger the added noise is, and the higher the degree of image damage is.

[0053] It should be noted that the initial workpiece noise image may be a Gaussian noise image automatically generated by the system after receiving the natural language prompt text input by the user. The initial defect image may be a defect image generated by performing noise addition and denoising processing on the initial workpiece noise image.

[0054] In practical applications, excessively large noise amplitudes can cause the defect images generated by the conditional diffusion model to be distorted or unreasonable, while excessively small noise amplitudes may limit the diversity of the generated defect images. When generating defect images, certain defect types (such as tiny scratches) may require smaller noise amplitudes to maintain details, while other defect types (such as large-area corrosion) may require larger noise amplitudes to generate reasonable morphologies. Therefore, this embodiment can adjust the diffusion noise amplitude according to the physical vector of the defect to strike a balance between generation quality and diversity. Among them, the conditional diffusion model is a generative model based on step-by-step denoising of random noise. The model generates high-quality images in a step-by-step denoising process by learning data distribution, including two stages: forward diffusion (noise addition) and reverse diffusion (noise removal). In this embodiment, the diffusion model can be used to convert the initial workpiece noise image into pure noise (forward diffusion) by gradually adding noise, and then the initial defect image is generated by step-by-step denoising (reverse generation).

[0055] Step S40: performing physical consistency verification on the initial defect image, and outputting a target defect image according to the verification result.

[0056] It should be noted that the target defect image can be a defect image that meets the physical description requirements of the target defect. In practical applications, the physical parameters of the initial defect image generated by the conditional diffusion model often differ to a certain extent from the physical parameters of the target defect. Therefore, to improve the accuracy of the generated defect image, this embodiment can perform physical consistency verification on the initial defect image to verify whether the initial defect image meets the physical consistency requirements. If so, the initial defect image is determined to be qualified and output as the target defect image.

[0057] In practical applications, traditional diffusion methods based on image processing, GANs, or text guidance mainly focus on the visual appearance of defects and are unable to explicitly model and dynamically control the physical parameters of defects (such as width, depth, length, area, etc.). As a result, the generated samples cannot meet the high consistency requirements of industrial applications in terms of physical scale. The defect image diffusion generation method proposed in this application can directly couple the physical properties of the defect to the diffusion noise amplitude, making it possible to dynamically control the noise amplitude during forward or reverse diffusion, achieving "speaking by size, random noise generation". Moreover, this mechanism does not require additional conditional graphs (edge, depth, etc.) and can maintain the consistency of physical scales such as crack width and pit depth at the pixel level.

[0058] The present embodiment provides a defect image diffusion generation method, which discloses determining a target defect type and defect size parameters corresponding to a target defect based on natural language prompt text input by a user; constructing a defect physical vector corresponding to the target defect based on the target defect type and defect size parameters; adjusting the diffusion noise amplitude in the diffusion process based on the defect physical vector to generate an initial defect image corresponding to the initial workpiece noise image; performing physical consistency verification on the initial defect image, and outputting a target defect image based on the verification result; since the present embodiment can adjust the diffusion noise amplitude in the diffusion process based on the defect physical vector corresponding to the target defect to generate an initial defect image, and output the target defect image based on the physical consistency verification result of the initial defect image, it solves the technical problem that the defect image generation method in the prior art cannot pay attention to the physical size of the defect, resulting in low accuracy of the generated defect image.

[0059] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 3 , Figure 3 A flow chart of the second embodiment of the defect image diffusion generation method of the present application is provided.

[0060] In this embodiment, step S30 includes steps S301 to S303: Step S301: Generate a reference noise parameter using a reference timing function.

[0061] It should be understood that the above-mentioned reference timing function can be a function used to control the change of noise addition during the generation process, for example, a linear schedule function, wherein the linear schedule function is a time scheduling strategy commonly used in diffusion models, which gradually adds noise to the data by linearly increasing the noise intensity, and eventually makes the data close to a standard Gaussian distribution. In this embodiment, the reference noise parameter can be generated based on the reference timing function and the diffusion step size. ,in, represents the benchmark timing function, t represents the diffusion step size, that is, the number of iterations of the diffusion process (t = 0 is the original image, t = T is complete noise), which is used to control the degree of noise accumulation or removal.

[0062] Step S302: Utilize a multi-layer perceptron to map the defect physical vector into a noise scheduling weight coefficient.

[0063] It can be understood that the multilayer perceptron can be a basic artificial neural network model. In GAN, the multilayer perceptron MLP can be used as part of the generator or discriminator to extract and transform features.

[0064] Step S303: adjusting the diffusion noise amplitude during the diffusion process based on the reference noise parameter, the time-varying gating coefficient, and the noise scheduling weight coefficient, to generate an initial defect image corresponding to the initial workpiece noise image.

[0065] It should be noted that the aforementioned time-varying gating coefficient can be a feature used to adjust the connection strength or importance between different layers or different physical features in the generator or discriminator. In this embodiment, a noise scheduling function can be constructed based on the baseline noise parameter, the time-varying gating coefficient, and the noise scheduling weight coefficient. By adjusting the parameters in the noise scheduling function, the amplitude of the diffusion noise during the diffusion process is dynamically adjusted to generate an initial defect image corresponding to the initial workpiece noise image.

[0066] It should be noted that in the existing diffusion model, the noise scheduling (i.e., the degree of denoising) at each step is usually fixed. However, this application can use a multi-layer perceptron to The defect physical vector Mapped to noise scheduling weight coefficient , which makes it possible to dynamically adjust the noise amplitude of each step according to the defect physical vector v corresponding to the target defect, so that the physical parameters of the target defect can be explicitly modeled and dynamically controlled, and then generate a defect image that meets the physical scale requirements.

[0067] Specifically, step S303 includes: adjusting the diffusion noise amplitude in the diffusion process based on the baseline noise parameter, the time-varying gating coefficient and the noise scheduling weight coefficient; performing denoising and denoising processing on the initial workpiece noise image according to the adjusted diffusion noise amplitude to obtain the target workpiece noise feature map; injecting the defect physical vector as the query vector and the target workpiece noise feature map as the key value into each attention layer of the conditional diffusion generation module based on the information fusion attention mechanism; generating an initial defect image based on the defect physical vector and the target workpiece noise feature map through the conditional diffusion generation module.

[0068] In this embodiment, a noise scheduling function can be constructed based on the reference noise parameter, the time-varying gating coefficient and the noise scheduling weight coefficient. , which can be specifically expressed as:

[0069] Where, represents the benchmark timing function; represents a multilayer perceptron; represents the time-varying gating coefficient, which is often set to be exponentially decreasing to strengthen the physical constraints in the later stage. Through and The product term of can dynamically inject the defect physical properties into the noise scheduling process, thereby achieving fine-grained physically controllable generation.

[0070] It should be noted that the diffusion model in this embodiment is provided with a conditional diffusion generation module, which is based on UNet and Cross-Attention. UNet is a symmetrical encoder-decoder structure used to achieve efficient noise prediction and image generation; Cross-Attention is a cross-attention mechanism that allows the model to use the physical property information of the defect to guide the generation of the defect image while processing image features. In this embodiment, feature encoding and decoding can be performed using the UNet structure in the conditional diffusion generation module, and the defect physical vector is injected as a query vector into each layer of the conditional diffusion generation module. At the same time, the image features in the target workpiece noise feature map are used as key values ​​for training, and the initial defect image is finally output.

[0071] In this embodiment, the training goal of the conditional diffusion generation module is to ensure that the generated defect image is not only visually realistic, but also that its physical parameters (such as crack width and pit depth) are strictly consistent with the target values. The corresponding training loss L can be:

[0072] in, To pre-train the defect measurement network, is the loss weight coefficient, which is used to balance the noise prediction loss of the diffusion model and the optimization weight of the physical parameter regression loss L_Physics, is the standard diffusion model loss (denoised prediction error), The defect image generated The L2 norm error between the predicted physical parameters and the defect physical vector v.

[0073] In this embodiment, a reference noise parameter is generated by a reference timing function; a defect physical vector is mapped to a noise scheduling weight coefficient using a multilayer perceptron; the diffusion noise amplitude in the diffusion process is adjusted based on the reference noise parameter, the time-varying gating coefficient, and the noise scheduling weight coefficient to generate an initial defect image corresponding to the initial workpiece noise image; because the defect physical vector can be mapped to the noise scheduling weight coefficient by a multilayer perceptron in this embodiment, the diffusion noise amplitude can be subsequently dynamically adjusted according to the defect physical vector, so that the physical parameters of the target defect can be explicitly modeled and dynamically controlled, thereby generating a defect image that meets the physical scale requirements.

[0074] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above introduction and will not be described in detail later. Figure 4 , Figure 4 A flow chart illustrating the third embodiment of the defect image diffusion generation method of the present application.

[0075] In this embodiment, step S40 includes steps S401 to S403: Step S401: using a preset defect measurement network to predict defect physical indicators corresponding to the initial defect image.

[0076] It should be noted that the above-mentioned preset defect measurement network can be a network for predicting the physical indicators of the initial defect image, wherein the defect physical indicators corresponding to the initial defect image can be physical indicators such as the width, depth, length, and area of ​​the defect in the initial defect image.

[0077] Step S402: Determine the defect error between the defect physical indicator and the defect physical vector.

[0078] It can be understood that the above-mentioned defect error can be a value used to characterize the difference between the defect physical index and each physical parameter in the defect physical vector.

[0079] Step S403: performing physical consistency verification on the initial defect image based on the defect error, and outputting a target defect image according to the verification result.

[0080] Specifically, step S403 includes: determining a defect error threshold corresponding to the initial defect image based on process tolerance; comparing the defect error with the defect error threshold; if the defect error does not exceed the defect error threshold, determining that the initial defect image meets the physical consistency requirements, and outputting the initial defect image as a target defect image.

[0081] It should be understood that the process tolerance can be the allowable range of size or parameter variation, for example, ±5µm. In practical applications, the maximum and minimum allowable values ​​of the physical parameters corresponding to the defect can be determined based on the process tolerance, and the maximum and minimum allowable values ​​of the physical parameters can be determined as the defect error threshold. In this embodiment, the defect physical indicators predicted by the preset defect measurement network can be and defect physical vector Subtract the defect error and calculate the absolute value of the defect error. Defect Error Threshold For comparison, if , it means that the initial defect image meets the physical consistency requirements and is determined to be qualified. At this time, the initial defect image can be output as the target defect image, otherwise it can be discarded or re-sampled.

[0082] In the specific implementation, refer to Figure 5 , Figure 5 This is the model structure diagram of the defect image diffusion generation method of this application. Figure 5 As shown in the figure, the system can first receive the natural language prompt text Prompt input by the user, which can contain a text description of the defect type and its size, such as "a crack defect with a width of 0.5mm". At this time, the system can automatically generate an initial workpiece noise image as the starting input of the diffusion model. Then, the model can perform regular text segmentation on the natural language prompt text, extract key information from it, including defect type and size parameters, and use the domain vocabulary to map the defect type to a category index, and then use the unit normalization function to calculate the defect type. All dimensions are converted to millimeters to obtain standard defect size parameters, and the scale reference table S is introduced to perform linear normalization on the defect size parameters in the original physical parameter list. The continuous physical quantity is linearly mapped to [0,1] to obtain the normalized continuous size parameters. The defect physical vector v corresponding to the target defect is constructed based on the category index and the normalized continuous size parameters. Its dimensions include defect type, width, depth, etc. The defect physical vector v is then vector-embedded and input into the fully connected layer (FC) and LayerNorm module to obtain the standardized defect physical vector v for subsequent image generation control. The adaptive noise scheduling module in the system can receive the standardized defect physical vector v and map the defect physical vector to the noise scheduling weight coefficient through the multi-layer perceptron Tiny-MLP. In this embodiment, the multilayer perceptron includes two layers: MLP layer 1 and MLP layer 2. MLP layer 1: FC(d—>n)+ReLU (used to extract nonlinear semantic features in physical vectors); MLP layer 2: FC(n—>1) (outputs a scalar, indicating the noise amplification or reduction coefficient corresponding to the current size condition). Then, the adaptive noise scheduling module can be based on the cosine benchmark function. , Sigmoid gate function and noise scheduling weight coefficient Construct a noise scheduling function and use it to adjust the diffusion noise amplitude in the diffusion process (including forward diffusion and reverse denoising sampling) . In addition, the conditional diffusion generation module in the system (the network backbone is UNet and Cross-Attention) can gradually restore the noise feature map to the final image. Specifically, the module can first use the UNet structure (including encoder L1, encoder L2, bottleneck layer, decoder L1 and decoder L2) for feature encoding and decoding, and inject the defect physical vector v into each layer of the network. The injection method is Cross-Attention. Among them, encoders L1, L2 and L3 all use 3×3 convolution kernels for convolution operations, and downsample the feature map after two consecutive identical convolution operations, reducing it to 1 / 2 of the original. The number of output channels of encoders L1, L2 and L3 convolutions is 64, 128 and 256 respectively; the bottleneck layer can use 3×3 convolution kernels for convolution operations, and the number of output channels of convolution is 512; decoders L1, L2 and L3 can first upsample the feature map to twice the original size, and then use 3×3 convolution kernels for two consecutive identical convolution operations to finally output the initial defect image. Finally, the system can verify the physical consistency of the initial defect image through the physical consistency verification module. Specifically, the module can first use the preset defect measurement network to predict the defect physical index corresponding to the initial defect image and determine the defect error between the index and the defect physical vector. , then the defect error and the defect error threshold For comparison, if , it means that the initial defect image meets the physical consistency requirements and is determined to be qualified. At this time, the initial defect image can be output as the target defect image, otherwise it can be discarded or re-sampled.

[0083] In this embodiment, a method is disclosed for predicting defect physical indicators corresponding to an initial defect image using a preset defect measurement network; determining a defect error between the defect physical indicator and a defect physical vector; performing physical consistency verification on the initial defect image based on the defect error, and outputting a target defect image based on the verification result; since this embodiment can perform physical consistency verification on the initial defect image based on the defect error between the defect physical indicator corresponding to the initial defect image and the defect physical vector, it can ensure that the error of the generated defect image is always within the allowable error range, thereby ensuring the quality of the generated defect image.

[0084] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the defect image diffusion generation method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0085] This application also provides a defect image diffusion generation device, please refer to Figure 6 , the defect image diffusion generating device comprises: The defect information determination module 10 is used to determine the target defect type and defect size parameters corresponding to the target defect according to the natural language prompt text input by the user; A physical vector construction module 20 is configured to construct a defect physical vector corresponding to the target defect based on the target defect type and the defect size parameter; A defect image generating module 30 is configured to adjust the diffusion noise amplitude during the diffusion process based on the defect physical vector to generate an initial defect image corresponding to the initial workpiece noise image; The defect image verification module 40 is configured to perform physical consistency verification on the initial defect image and output a target defect image according to the verification result.

[0086] The defect image diffusion generation device provided in this application utilizes the defect image diffusion generation method described in the aforementioned embodiments, resolving the technical issue with existing defect image generation methods, which fail to consider the physical size of defects, resulting in low accuracy in the generated defect images. Compared to existing technologies, the defect image diffusion generation device provided in this application achieves the same beneficial effects as the defect image diffusion generation method described in the aforementioned embodiments. Other technical features of the defect image diffusion generation device are the same as those disclosed in the aforementioned embodiments and are not further elaborated here.

[0087] The present application provides a defect image diffusion generation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the defect image diffusion generation method in the above-mentioned embodiment 1.

[0088] Reference below Figure 7 , which shows a schematic structural diagram of a defect image diffusion generation device suitable for implementing embodiments of the present application. The defect image diffusion generation device in embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 7 The defective image diffusion generating device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0089] like Figure 7As shown, the defect image diffusion generation device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to programs stored in a read-only memory 1002 or loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the defect image diffusion generation device. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems may be connected to the input / output interface 1006: an input device 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; a storage device 1003 including, for example, a magnetic tape or hard disk; and a communication device 1009. The communication device 1009 can allow the defect image diffusion generation device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a defect image diffusion generation device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have alternatively.

[0090] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are performed.

[0091] The defect image diffusion generation device provided in this application utilizes the defect image diffusion generation method described in the aforementioned embodiment to address the technical issues surrounding defect image diffusion generation. Compared to the prior art, the defect image diffusion generation device provided in this application achieves the same beneficial effects as the defect image diffusion generation method described in the aforementioned embodiment. Other technical features of the defect image diffusion generation device are the same as those disclosed in the aforementioned embodiment and are not further elaborated here.

[0092] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0093] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0094] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the defect image diffusion generation method in the above embodiment.

[0095] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0096] The computer-readable storage medium may be included in the defective image diffusion generation device; or may exist independently without being assembled into the defective image diffusion generation device.

[0097] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the defect image diffusion generation device, the defect image diffusion generation device is enabled to: determine the target defect type and defect size parameters corresponding to the target defect based on the natural language prompt text input by the user; construct a defect physical vector corresponding to the target defect based on the target defect type and defect size parameters; adjust the diffusion noise amplitude in the diffusion process based on the defect physical vector to generate an initial defect image corresponding to the initial workpiece noise image; perform physical consistency verification on the initial defect image, and output the target defect image based on the verification result.

[0098] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0099] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0100] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0101] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned defect image diffusion generation method. This computer-readable storage medium addresses the technical issue in existing defect image generation methods that fail to consider the physical size of defects, resulting in low accuracy in the generated defect images. Compared to existing technologies, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the defect image diffusion generation method provided in the aforementioned embodiments and are not further elaborated here.

[0102] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A defect image diffusion generation method, characterized in that: The method includes: Determine the target defect type and defect size parameters corresponding to the target defect according to the natural language prompt text input by the user; Constructing a defect physical vector corresponding to the target defect based on the target defect type and the defect size parameter; Adjusting the diffusion noise amplitude during the diffusion process based on the defect physical vector to generate an initial defect image corresponding to the initial workpiece noise image; Physical consistency verification is performed on the initial defect image, and a target defect image is output according to the verification result.

2. The method according to claim 1, wherein The step of constructing a defect physical vector corresponding to the target defect based on the target defect type and the defect size parameter includes: Mapping the target defect type to a category index using a preset domain vocabulary; Determine the size unit and size value according to the defect size parameter; Performing unit normalization processing on the defect size parameter based on the size unit and the size value to obtain a standard defect size parameter; A defect physical vector corresponding to the target defect is constructed based on the category index and the standard defect size parameter.

3. The method according to claim 2, wherein The step of constructing a defect physical vector corresponding to the target defect based on the category index and the standard defect size parameter includes: constructing a raw physical parameter list based on the category index and the standard defect size parameter; Performing linear normalization processing on the defect size parameters in the original physical parameter list using a preset scale reference table to obtain normalized continuous size parameters; A defect physical vector corresponding to the target defect is constructed based on the category index and the normalized continuous size parameter.

4. The method according to claim 1, wherein The step of adjusting the diffusion noise amplitude in the diffusion process based on the defect physical vector to generate an initial defect image corresponding to the initial workpiece noise image includes: Generate benchmark noise parameters through benchmark timing function; Mapping the defect physical vector into a noise scheduling weight coefficient using a multi-layer perceptron; The diffusion noise amplitude in the diffusion process is adjusted based on the reference noise parameter, the time-varying gating coefficient and the noise scheduling weight coefficient to generate an initial defect image corresponding to the initial workpiece noise image.

5. The method according to claim 4, wherein The step of adjusting the diffusion noise amplitude in the diffusion process based on the reference noise parameter, the time-varying gating coefficient, and the noise scheduling weight coefficient to generate an initial defect image corresponding to the initial workpiece noise image includes: Adjusting the diffusion noise amplitude during the diffusion process based on the reference noise parameter, the time-varying gating coefficient, and the noise scheduling weight coefficient; The initial workpiece noise image is subjected to denoising and denoising processing respectively according to the adjusted diffusion noise amplitude to obtain the target workpiece noise feature map; Based on the information fusion attention mechanism, the defect physical vector is used as the query vector and the target workpiece noise feature map is used as the key value to inject into each attention layer of the conditional diffusion generation module; An initial defect image is generated based on the defect physical vector and the target workpiece noise feature map by the conditional diffusion generation module.

6. The method according to any one of claims 1 to 5, characterized in that The step of performing physical consistency verification on the initial defect image and outputting a target defect image according to the verification result includes: Predicting defect physical indicators corresponding to the initial defect image using a preset defect measurement network; determining a defect error between the defect physical indicator and the defect physical vector; Physical consistency verification is performed on the initial defect image based on the defect error, and a target defect image is output according to the verification result.

7. The method according to claim 6, wherein The step of performing physical consistency verification on the initial defect image based on the defect error and outputting a target defect image according to the verification result includes: determining a defect error threshold corresponding to the initial defect image based on a process tolerance; comparing the defect error with the defect error threshold; If the defect error does not exceed the defect error threshold, it is determined that the initial defect image meets the physical consistency requirement, and the initial defect image is output as a target defect image.

8. A defect image diffusion generating device, characterized in that: The device comprises: A defect information determination module is used to determine the target defect type and defect size parameters corresponding to the target defect according to the natural language prompt text input by the user; A physical vector construction module, configured to construct a defect physical vector corresponding to the target defect based on the target defect type and the defect size parameter; a defect image generating module, configured to adjust the diffusion noise amplitude during the diffusion process based on the defect physical vector, and generate an initial defect image corresponding to the initial workpiece noise image; The defect image verification module is used to perform physical consistency verification on the initial defect image and output a target defect image according to the verification result.

9. A defect image diffusion generating device, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the defect image diffusion generation method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the defect image diffusion generation method according to any one of claims 1 to 7 are implemented.

Images