A method and apparatus for generating industrial defects of arbitrary dimensions

Abstract

The present application relates to the technical field of computer vision and image generation, in particular to a method and device for generating industrial defects of arbitrary scale. The method comprises, in the diffusion model training stage, performing multi-scale cropping operation on the original industrial defect image to construct a complete multi-scale image and mask set, and training through joint scale loss and cross-scale perception loss, so that the shared learnable text embedding e can capture rich defect representation with location awareness; in the diffusion model inference generation stage, performing the post-generation and pre-fusion denoising strategy on the normal defect-free image to decouple the high-fidelity construction process of the industrial defect of arbitrary scale and the seamless fusion process of the global background, and obtain the industrial defect image. The present application can generate an industrial defect image that is seamlessly fused with the background, providing a good foundation for subsequent industrial defect detection.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image generation technology, and in particular to a method and apparatus for generating industrial defects at any scale. Background Technology

[0002] In modern manufacturing, industrial defect detection plays a crucial role in product quality control. However, in real-world industrial scenarios, defect samples are often extremely scarce and costly to collect, severely limiting the development of reliable visual inspection systems. To alleviate the shortage of defect samples, existing methods primarily rely on diffusion models to synthesize pseudo-defect images. However, despite some progress in these methods, they still face significant limitations when dealing with small-scale defects (such as extremely low-resolution scratches and tiny blemishes). The main reason is that mainstream diffusion models require extreme downsampling in their architecture (e.g., compressing images to 1 / 8 of their original size using a VAE encoder), and the cross-attention mechanism controlling semantic generation within the model is typically most effective at extremely low resolutions. This extreme compression mechanism leads to a significant loss of feature information from small-scale defects in the latent space, making it difficult for diffusion models to learn and reconstruct the high-frequency details of these minute defects, thus failing to meet the needs of generating industrial-grade micro-defect data.

[0003] To address the shortcomings of standard diffusion models in learning small targets, existing fine-tuning strategies are gradually shifting towards local cropping-based strategies, which extract local image patches containing small defects for training. However, fine-tuning strategies based solely on local cropping isolate small targets from the global background of the original image, disrupting the contextual constraints between defect features and specific locations on industrial products, resulting in generated defects failing to seamlessly integrate with the background. To preserve contextual positional constraints, existing approaches attempt to combine enlarged local cropped patches with the original full image for joint training. However, in this multi-scale transfer scenario, directly enlarging tiny defects inevitably introduces significant interpolation errors and cross-scale feature inconsistencies, leading to difficulties in cross-scale information fusion. More seriously, in the inference generation stage, existing text-guided image generation methods typically complete defect generation and background fusion simultaneously within a single denoising stage. However, due to the low resolution of cross-attention maps, the single-stage denoising process is easily dominated by large-scale background textures and structures, causing the already subtle small-scale defect details to be submerged by background information or even ignored, ultimately leading to defect generation failure.

[0004] In summary, although diffusion models and fine-tuning strategies based on local cropping provide preliminary ideas for the generation of industrial defect images, existing technologies still cannot effectively resolve the contradiction between fine-grained texture acquisition, interpolation distortion, and seamless integration with the global background when dealing with minute defects.

[0005] Therefore, overcoming the shortcomings of the existing technology is an urgent problem to be solved in this technical field. Summary of the Invention

[0006] The technical problem this invention aims to solve is that, when dealing with minute industrial defects, it is still impossible to effectively resolve the contradiction between fine-grained texture acquisition, interpolation distortion, and seamless integration with the global background.

[0007] The embodiments of the present invention adopt the following technical solutions: Firstly, this invention provides a method for generating industrial defects at arbitrary scales, specifically: during the diffusion model training phase, multi-scale cropping operations are performed on the original industrial defect image to construct a complete set of multi-scale images and masks. And by training with joint scale loss and cross-scale perception loss, the shared learnable text embedding e can capture rich defective representations with position awareness. During the diffusion model inference generation stage, normal, defect-free images are processed. A post-generation fusion and denoising strategy is implemented to decouple the high-fidelity construction process of industrial defects at arbitrary scale from the seamless fusion process of the global background, thereby obtaining an industrial defect image.

[0008] Preferably, the method further includes: The original industrial defect images are cropped and scaled to obtain a complete multi-scale image and mask set containing everything from local micro-textures to the global full image. ; Complete multi-scale image and mask set The data is aggregated into a training set, which is then input into the diffusion model so that the diffusion model can be based on the noisy latent variables generated during the noisy stage. The learnable text embedding e is used to predict noise, and the difference between the predicted noise and the random target noise is calculated to obtain the joint scale loss. ; Calculate cross-scale perceptual loss based on adjacent-scale images in a complete multi-scale image. ; Combined with the aforementioned joint scale loss With cross-scale perception loss Construct the total training loss function And based on the total training loss function Learnable text embeddings Update and optimize the model to complete the training of the diffusion model.

[0009] Preferably, the joint scale loss The joint scaling loss function is obtained as follows: ; in, Indicates joint-scale loss, Indicates the first Images at each scale at time step The corresponding noisy latent variable, This represents the introduced random target noise. For the noise predictor of the model, Indicates the first Masks corresponding to images at each scale Let n represent the Hadamard product, n be the total number of scales, and e represent the learnable text embedding e.

[0010] Preferably, the method further includes: Adjust the original industrial defect image to the specified generated resolution. × Obtain the image with maximum scale cropping. and defect mask ; Extracting the defect mask The minimum bounding box of the connected components is obtained by expanding the minimum bounding box into a square based on its longer side, then cropping the region corresponding to the square and scaling it to the generated resolution. × Obtain the image with the smallest scale crop. and the corresponding defect mask ; Centered on the geometric center of the minimum bounding box, the clipping side length is gradually increased geometrically according to the set clipping expansion factor α; the intermediate-scale clipping regions clipped after each expansion of the field of view are uniformly adjusted to the generated resolution. A series of intermediate-scale input sequences were obtained. Finally, the complete multi-scale image and mask set is obtained by combining them. .

[0011] Preferably, the calculation of cross-scale perceptual loss based on adjacent scale images in the complete multi-scale image is... ,include: Clean latent variables are obtained by denoising latent variables at various scales using the predicted noise. The decoder outputs the predicted denoised image. ; High-resolution scale prediction map The defective region in the map is mapped according to the corresponding side length. Perform secondary cropping and adjust to the generated resolution. To obtain aligned high-resolution prediction blocks ; Using a pre-defined feature extractor Calculate denoised image and high-resolution prediction blocks The perceptual feature similarity in the corresponding region of the mask is used as the cross-scale perceptual loss. .

[0012] Preferably, the cross-scale sensing loss The cross-scale sensing loss function is obtained as follows: ; in, This represents the loss of perception across scales. For the first Masks corresponding to images at each scale For denoised images, For high-resolution prediction blocks This represents the preset feature extractor, where n is the total number of scales.

[0013] Preferably, the method further includes: Based on Berlin noise, for normal, defect-free images Randomly generated defect mask ; Using the same multi-scale cropping operation as during training, normal, defect-free images are... Construct a set of normal image cropping blocks and masks at multiple scales. ; At the minimum scale, use the smallest normal cropped image. Minimum normal cropped image Corresponding mask and trained text embeddings As input, a pre-defined number of denoising steps are used for mixing and diffusion in the mask. Generate minimum-scale initial defects with high-fidelity fine-grained texture within the region; For each subsequent large-scale level, the defect image generated at the previous scale is... Previous scale mask Normal cropped image at the current scale Compared with the current scale mask As input, output the defect image at the current scale. and current scale mask ; Repeat the above process in a loop until the maximum scale is reached, outputting an industrial defect image that blends seamlessly with the global background.

[0014] Preferably, the method further includes: Defect image from the previous scale and the previous scale mask According to the downsampling factor Resolution reduction is performed to obtain a downsampled defect map. and downsampling mask ; Using downsampling masks From the downsampled defect map Extract the valid defect pixels and paste them directly onto the normal image at the current scale. The current scale mask At the specified precise location, pixel-level replacement is achieved to obtain the stitched image; Perform forward diffusion noise addition on the stitched image and mask it at the current scale. Under spatial constraints, local denoising and reconstruction are performed using a preset number of denoising steps to output a defect image at the current scale. .

[0015] In a second aspect, the present invention provides an arbitrary-scale industrial defect generation apparatus, the apparatus comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor to perform the arbitrary-scale industrial defect generation method of the first aspect.

[0016] Thirdly, the present invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors to perform the arbitrary-scale industrial defect generation method described in the first aspect.

[0017] Compared with existing technologies, the beneficial effects of this invention are as follows: In the diffusion model training stage, a complete multi-scale image and mask set is constructed based on the original industrial defect image. By combining scale loss and cross-scale perceptual loss, the update direction of the learnable text embedding e is guided, enabling the shared learnable text embedding e to capture rich defect representations with location awareness. Furthermore, in the diffusion model inference generation stage, by performing pre-generation and post-fusion denoising on normal industrial images, large-scale background information can be prevented from obscuring small-scale defect features. Based on the combined use of the training and inference processes provided by this invention, industrial defect images that seamlessly blend with the global background can be generated. These generated industrial defect images are used as training samples to expand the training dataset of the industrial defect detection model, alleviating the problem of insufficient defect samples and thus improving the recognition accuracy of the industrial defect detection model. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0019] Figure 1 This is a flowchart illustrating an arbitrary-scale industrial defect generation method provided in an embodiment of the present invention. Figure 2 This is a flowchart of a method for training a diffusion model in an arbitrary-scale industrial defect generation method provided by an embodiment of the present invention; Figure 3 This is a flowchart of a method for constructing a complete multi-scale image and mask set in an arbitrary-scale industrial defect generation method provided by an embodiment of the present invention; Figure 4 This is a schematic diagram of a multi-scale image and corresponding defect mask in an arbitrary-scale industrial defect generation method provided by an embodiment of the present invention; Figure 5 This is a flowchart of a method for calculating cross-scale sensing loss in an arbitrary-scale industrial defect generation method provided by an embodiment of the present invention; Figure 6 This is a flowchart of a method for generating industrial defect images in an arbitrary-scale industrial defect generation method provided by an embodiment of the present invention. Figure 7 This is a flowchart of a method for generating industrial defect images in an arbitrary-scale industrial defect generation method provided in an embodiment of the present invention. Figure 8 This is a flowchart of the diffusion model training stage in an arbitrary-scale industrial defect generation method provided by an embodiment of the present invention; Figure 9 This is a flowchart of the diffusion model inference stage in an arbitrary-scale industrial defect generation method provided by an embodiment of the present invention; Figure 10 This is a schematic diagram of an arbitrary-scale industrial defect generation device provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0021] Unless the context otherwise requires, throughout the specification and claims, the term "comprising" is interpreted as openly inclusive, meaning "including, but not limited to." In the description of the specification, terms such as "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples" are intended to indicate that a particular feature, structure, material, or characteristic associated with that embodiment or example is included in at least one embodiment or example of this disclosure. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics mentioned may be included in any suitable manner in any one or more embodiments or examples; that is, although they may be incorporated into embodiments or examples using the above terms for reasons such as order and position, it does not limit them to be incorporated in combination by a single embodiment or example.

[0022] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more. Furthermore, for example, the description may use the prefix "A" or "B" to describe the same type of nouns as two independent entities. In this case, the corresponding features defined with "A" and "B" are used only to distinguish between similar entities and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features.

[0023] In the description of this invention, the expression “A and / or B” (where A and B are used to formally represent specific features) will be used. The corresponding expression includes the following three combinations: only A, only B, and a combination of A and B.

[0024] As used in this invention, “about,” “approximately,” or “approximately” includes the stated value and the average value within an acceptable range of deviation from a particular value, wherein the acceptable range of deviation is determined by a person skilled in the art taking into account the measurement under discussion and the error associated with the measurement of the particular quantity (i.e., the limitations of the measurement system).

[0025] Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0026] Example 1: like Figure 1 As shown, Embodiment 1 of the present invention provides a method for generating industrial defects at any scale, which specifically includes the following steps: Step 101: During the diffusion model training phase, perform multi-scale cropping operations on the original industrial defect images to construct a complete multi-scale image and mask set. Furthermore, by training with joint scale loss and cross-scale perception loss, the shared learnable text embedding e is able to capture rich, position-aware defect representations.

[0027] Step 102: In the diffusion model inference generation stage, normal, defect-free images are processed. A post-generation fusion and denoising strategy is implemented to decouple the high-fidelity construction process of industrial defects at arbitrary scale from the seamless fusion process of the global background, thereby obtaining an industrial defect image.

[0028] In this embodiment, during the diffusion model training phase, a complete multi-scale image and mask set is constructed based on the original industrial defect images. By combining scale loss and cross-scale perceptual loss, the update direction of the learnable text embedding *e* is guided, enabling the shared learnable text embedding *e* to capture rich defect representations with location awareness. Furthermore, during the diffusion model inference generation phase, by performing pre-generation and post-fusion denoising on normal industrial images, large-scale background information can be prevented from obscuring small-scale defect features. Based on the combined use of the training and inference processes provided by this invention, industrial defect images that seamlessly blend with the global background can be generated. These generated industrial defect images are used as training samples to expand the training dataset of the industrial defect detection model, alleviating the problem of insufficient defect samples and thereby improving the recognition accuracy of the industrial defect detection model.

[0029] See Figure 2 Step 101 specifically includes the following steps: Step 201: Crop and scale the original industrial defect image to obtain a complete multi-scale image and mask set containing everything from local micro-textures to the global full image. .

[0030] A complete multi-scale image is a collection of images at multiple different scales. It can be categorized into large-scale, medium-scale, and small-scale images. Large-scale images contain defects and the most background, while small-scale images contain defects and the least background. There are multiple medium-scale images, each containing defects and progressively increasing background. It's important to note that "most background," "least background," and "progressively increasing background" are relative terms. Using a small-scale image as a reference, the larger the image scale, the richer the background information it contains. Therefore, large-scale images typically contain the most background.

[0031] In one embodiment, industrial defect images are represented at multiple scales through cropping and scaling operations to obtain a complete set of multi-scale images and masks. The mask has the same scale as the corresponding scale image. The mask uses 0 and 1 to represent the entire image, where 0 represents the background area and 1 represents the defect area. To crop the image to the minimum scale, Defect mask for cropping images at the minimum scale. To crop the image to the maximum scale, This is the defect mask for the image cropped at the maximum scale, where n is the total number of scales. n starts from 1 and increases with size. As n increases, the scale of the corresponding image becomes larger and contains more background.

[0032] Step 202: Combine the complete multi-scale image and mask set The data is aggregated into a training set, which is then input into the diffusion model so that the diffusion model can be based on the noisy latent variables generated during the noisy stage. The learnable text embedding e is used to predict noise, and the difference between the predicted noise and the random target noise is calculated to obtain the joint scale loss. .

[0033] Specifically, this invention introduces a shared, learnable text embedding e, which serves as the carrier of defect semantics. During the forward noise addition stage, random target noise aligned with the spatial position of the mask region is dynamically generated. This random target noise is injected only into the region corresponding to the mask, thereby achieving a differentiated noise addition strategy for defect perception. The random target noise can be directly generated using Gaussian noise.

[0034] In the reverse denoising stage, the noise latent variable at the current time step t is used as input, and noise prediction is performed by combining the joint conditions of the current time step t and the learnable text embedding e. The mean square error between the predicted noise and the target noise at each individual scale is calculated to obtain the joint scale loss. .

[0035] In one embodiment, joint dimensional loss The joint scaling loss function is obtained as follows: ; in, Indicates joint-scale loss, Indicates the first Images at each scale at time step The corresponding noisy latent variable, This represents the introduced random target noise. For the noise predictor of the model, Indicates the first Masks corresponding to images at each scale Let n represent the Hadamard product, n be the total number of scales, and e represent the learnable text embedding e.

[0036] Step 203: Calculate cross-scale perceptual loss based on adjacent scale images in the complete multi-scale image. .

[0037] To suppress upsampling interpolation distortion caused by scaling operations, this embodiment of the invention further designs a cross-scale sensing loss function (wherein, cross-scale sensing loss...). (Obtained through a cross-scale perceptual loss function) During training, the diffusion model is prompted to minimize the perceptual feature differences of defect regions between adjacent scale images, thus forcing the diffusion model to focus on learning defect features.

[0038] In one embodiment, the cross-scale sensing loss function is as follows: ; in, This represents the loss of perception across scales. For the first Masks corresponding to images at each scale For denoised images, For high-resolution prediction blocks This represents the preset feature extractor, where n is the total number of scales.

[0039] Step 204: Combine the joint scale loss With cross-scale perception loss Construct the total training loss function And based on the total training loss function Learnable text embeddings Update and optimize the model to complete the training of the diffusion model.

[0040] In one embodiment, the total training loss function as follows: ; in, Denotes the total training loss function. Indicates joint-scale loss, This represents the loss of perception across scales. The parameters are used to balance the two parts of the loss function.

[0041] In one implementation, The value is 0.001, which is just an example and can be set according to the actual application.

[0042] See Figure 3 Step 201 specifically includes the following steps: Step 301: Adjust the original industrial defect image to the specified generation resolution. Obtain the image with maximum scale cropping. and defect mask .

[0043] Among them, the generated resolution The resolution of the final industrial defect image to be generated.

[0044] In one embodiment, the input raw industrial defect image is adjusted to a specified generated resolution. Obtain the image with maximum scale cropping. and its corresponding defect mask .

[0045] Step 302: Extract the defect mask The minimum bounding box of the connected components is obtained by expanding the minimum bounding box into a square based on its longer side, then cropping the region corresponding to the square and scaling it to the generated resolution. Obtain the image with the smallest scale crop. and the corresponding defect mask .

[0046] Among them, defect mask The minimum bounding box of a connected component refers to the smallest rectangle in a binary defect mask that can completely contain all pixels within a given connected component.

[0047] Step 303: Using the geometric center of the minimum bounding box as the center, gradually expand the clipping side length geometrically according to the set clipping expansion factor α; uniformly adjust the intermediate-scale clipping regions clipped after each expansion of the field of view to the generated resolution. A series of intermediate-scale input sequences were obtained. Finally, the complete multi-scale image and mask set is obtained by combining them. .

[0048] Among them, the length of the cut side satisfies ,and That is, the original side length after expanding from the minimum bounding box. Based on this, the cropping expansion factor α is gradually expanded geometrically. For each cropping side length obtained, an intermediate-scale cropping region is cropped according to the cropping side length. According to this cropping method, the background contained in each intermediate-scale cropping region obtained at this time also gradually increases, and each intermediate-scale image expands the background outward from the defect as the center.

[0049] Among them, the total number of scales It is adaptively determined by the original size of the defective region.

[0050] See Figure 4 It showcases a complete set of multi-scale images and masks, including images cropped at the maximum scale. and defect mask Minimum scale image cropping and the corresponding defect mask and a series of intermediate-scale input sequences .

[0051] See Figure 5 Step 203 specifically includes the following steps: Step 401: Use the predicted noise to denoise the latent variables at each scale to obtain clean latent variables. The decoder outputs the predicted denoised image. .

[0052] The predicted noise is the noise predicted by the diffusion model in step 202 during the reverse denoising stage, using the noisy latent variable at the current time step t as input and combining the joint conditions of time step t and the learnable text embedding e. Based on the predicted noise, the latent variables at each scale can be denoised to obtain clean latent variables.

[0053] Step 402: Convert the high-resolution scale prediction map The defective region in the map is mapped according to the corresponding side length. Perform secondary cropping and adjust to the generated resolution. To obtain aligned high-resolution prediction blocks .

[0054] Wherein, the mapping side length , To generate resolution, and To trim the side length, This maps the side length. The cropped image is then adjusted to the generated resolution. To obtain aligned high-resolution prediction blocks .

[0055] Step 403: Utilize the preset feature extractor Calculate denoised image and high-resolution prediction blocks The perceptual feature similarity in the corresponding region of the mask is used as the cross-scale perceptual loss. .

[0056] Figures 2-5 The technical details of the training phase are explained in detail below. Figure 6 and Figure 7 For technical details regarding the reasoning phase, please refer to [link / reference]. Figure 6 The reasoning stage specifically includes the following steps: Step 501: Based on Berlin noise, obtain a normal, defect-free image. Randomly generated defect mask .

[0057] Step 502: Using the same multi-scale cropping operation as during training, crop the normal, defect-free image. Construct a set of normal image cropping blocks and masks at multiple scales. .

[0058] In one embodiment, the same multi-scale cropping logic as steps 301 to 303 is used for normal industrial images. Extract and construct multi-scale normal image cropping blocks and mask sets. Where n is the total number of scales, and the value of n starts from 1. As the value of n increases, the scale of the normal image cropping block also gradually increases.

[0059] Step 503: At the minimum scale, crop the image with minimal normal cropping. Minimum normal cropped image Corresponding mask and trained text embeddings As input, a pre-defined number of denoising steps are used for mixing and diffusion in the mask. Generate minimum-scale initial defects with high-fidelity fine-grained textures within the region.

[0060] Specifically, obtain the smallest normally cropped image where n is 1. Minimize normal image cropping Minimum normal cropped image Corresponding mask and trained text embeddings As input, a hybrid diffusion algorithm is executed using a preset number of denoising steps, in the mask. Generate minimum-scale initial defects with high-fidelity fine-grained texture within the region, while preserving the minimum normal cropped image. In non-mask The original features of the region are used to obtain a defect image containing the initial defect at the smallest scale. .

[0061] In one implementation, the preset number of denoising steps is 50 steps. This is just an example and can be adjusted according to the actual application.

[0062] Step 504: For each subsequent large-scale level, generate the defect image from the previous scale. Previous scale mask Normal cropped image at the current scale Compared with the current scale mask As input, output the defect image at the current scale. and current scale mask .

[0063] In one embodiment, for each subsequent large-scale level (where...) (The defect image generated at the previous scale) Previous scale mask Normal cropped image at the current scale Compared with the current scale mask As input, output the defect image at the current scale. and current scale mask The current scale refers to the physical size or relative proportion of the defect in the image.

[0064] Step 505: Repeat the above process alternately until the maximum scale is reached, and output an industrial defect image that blends seamlessly with the global background.

[0065] Repeat step 504, using the output of the current scale as the input for the next scale, gradually increasing the size of the defect until the generated defect scale reaches the maximum scale (i.e., i takes the value n), and finally outputting an industrial defect image that seamlessly blends with the global background at full resolution. (in, According to the above method, it is possible to generate industrial defects of any scale.

[0066] See Figure 7 Step 504 specifically includes the following steps: Step 601: Transfer the defect image from the previous scale and mask According to the downsampling factor Resolution reduction is performed to obtain a downsampled defect map. and downsampling mask .

[0067] Among them, downsampling defect map .

[0068] Step 602: Using a downsampling mask From the downsampled defect map Extract the valid defect pixels and paste them directly onto the normal image at the current scale. The current scale mask At the specified precise location, pixel-level replacement is performed to obtain the stitched image.

[0069] Pixel-level replacement is represented as: , Indicates the downsampling mask. This represents the defect image from the previous scale. Indicates the current scale mask. This represents a normal image at the current scale.

[0070] The formula for pixel-level replacement is described below, specifically: Obtain the downsampled defect image from the previous scale. The corresponding downsampling mask For defective pixels with a value of 1, replace the defective pixel in the normal image at the current scale. The corresponding current scale mask The position with a value of 1; and ensuring a normal image at the current scale. The corresponding current scale mask The original pixels at positions with a value of 0 remain unchanged.

[0071] Step 603: Perform forward diffusion noise addition on the stitched image and mask it at the current scale. Under spatial constraints, local denoising and reconstruction are performed using a preset number of denoising steps to output a defect image at the current scale. .

[0072] In one embodiment, diffusion forward noise is performed on the stitched image after pasting overlay, and a defect mask is applied at the current scale. Under spatial constraints, local denoising and reconstruction are performed using a preset number of denoising steps to eliminate texture discontinuities and misalignments at the pasting boundaries, outputting a fusion defect image of the normal image at the current scale. .

[0073] In one implementation, the preset number of denoising steps is 10 steps, which is shorter than the preset number of denoising steps in step 503. This is just an example, and the value can be adjusted according to the actual application.

[0074] The following is combined with Figure 8 This invention describes the training process provided by taking an original industrial defect image as input, cropping and scaling the original industrial defect image to obtain a complete multi-scale image and mask set containing everything from local micro-textures to the global full image. ; A complete set of multi-scale images and masks The data are aggregated into a training set, which is then input into the diffusion model. This allows the diffusion model to inject random target noise into the masked region during the noise addition stage, using learnable text embeddings e, and to gradually recover the image during the denoising stage. Furthermore, it utilizes the latent noise variables generated during the noise addition stage. The predicted noise is calculated by determining the joint scale loss between the predicted noise and the random target noise. ; Calculate cross-scale perceptual loss based on adjacent scale images in a complete multi-scale image Combined with the aforementioned joint scale loss With cross-scale perception loss Construct the total training loss function And based on the total training loss function Learnable text embeddings Update and optimize the model to complete the training of the diffusion model.

[0075] The following is combined with Figure 9 The inference process provided by this invention is described as follows: based on Berlin noise, it is a normal, defect-free image. Randomly generated defect mask Using the same multi-scale cropping operation as during training, normal, defect-free images are cropped. Construct a set of normal image cropping blocks and masks at multiple scales. To provide background at various scales; at the smallest scale, to crop the image with minimal normal cropping. Minimum normal cropped image Corresponding mask and trained text embeddings As input, a pre-defined number of denoising steps are used for mixing and diffusion in the mask. Generate the smallest-scale initial defect with high-fidelity fine-grained texture within the region; for each subsequent large-scale level, the defect image generated at the previous scale is used. Previous scale mask Normal cropped image at the current scale Compared with the current scale mask As input, output the defect image at the current scale. and current scale mask The above process is repeated in a loop until the maximum scale is reached, outputting an industrial defect image that blends seamlessly with the global background.

[0076] Example 2: Based on the arbitrary-scale industrial defect generation method provided in the foregoing embodiments, the present invention also provides an arbitrary-scale industrial defect generation device that can be used to implement the above method, such as... Figure 10 The diagram shown is a schematic representation of the device architecture according to an embodiment of the present invention. The arbitrary-scale industrial defect generation device of this embodiment includes one or more processors 21 and a memory 22. Wherein, Figure 10 Take a processor 21 as an example.

[0077] Processor 21 and memory 22 can be connected via a bus or other means. Figure 10 Taking the example of a connection between China and Israel via a bus.

[0078] The memory 22 serves as a non-volatile computer-readable storage medium for an arbitrary-scale industrial defect generation method. It can be used to store non-volatile software programs and non-volatile computer-executable programs, such as the arbitrary-scale industrial defect generation method in the foregoing embodiments. The processor 21 executes various functional applications and data processing of the arbitrary-scale industrial defect generation device by running the non-volatile software programs, instructions, and modules stored in the memory 22, thereby realizing the arbitrary-scale industrial defect generation method of the foregoing embodiments.

[0079] Memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 22 may include memory remotely located relative to processor 21, which can be connected to processor 21 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0080] The program instructions / modules are stored in memory 22 and, when executed by one or more processors 21, execute the arbitrary-scale industrial defect generation method described in the foregoing embodiments.

[0081] This invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors, for example... Figure 10 One of the processors 21 can enable the one or more processors to execute the arbitrary-scale industrial defect generation method in the foregoing embodiments.

[0082] It is worth noting that the information interaction and execution process between the modules and units in the above-mentioned device are based on the same concept as the processing method embodiment of the present invention. For details, please refer to the description in the method embodiment of the present invention, and will not be repeated here.

[0083] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0084] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for generating industrial defects at any scale, characterized in that, include: During the diffusion model training phase, multi-scale cropping operations are performed on the original industrial defect images to construct a complete set of multi-scale images and masks. And by training with joint scale loss and cross-scale perception loss, the shared learnable text embedding e can capture rich defective representations with position awareness. During the diffusion model inference generation stage, normal, defect-free images are processed. A post-generation fusion and denoising strategy is implemented to decouple the high-fidelity construction process of industrial defects at arbitrary scale from the seamless fusion process of the global background, thereby obtaining an industrial defect image.

2. The method for generating industrial defects at any scale according to claim 1, characterized in that, The method further includes: The original industrial defect images are cropped and scaled to obtain a complete multi-scale image and mask set containing everything from local micro-textures to the global full image. ; Complete multi-scale image and mask set The data is aggregated into a training set, which is then input into the diffusion model so that the diffusion model can be based on the noisy latent variables generated during the noisy stage. The learnable text embedding e is used to predict noise, and the difference between the predicted noise and the random target noise is calculated to obtain the joint scale loss. ; Calculate cross-scale perceptual loss based on adjacent-scale images in a complete multi-scale image. ; Combined with the aforementioned joint scale loss With cross-scale perception loss Construct the total training loss function And based on the total training loss function Learnable text embeddings Update and optimize the model to complete the training of the diffusion model.

3. The method for generating industrial defects at any scale according to claim 2, characterized in that, The joint scale loss The joint scaling loss function is obtained as follows: ； in, Indicates joint-scale loss, Indicates the first Images at each scale at time step The corresponding noisy latent variable, This represents the introduced random target noise. For the noise predictor of the model, Indicates the first Masks corresponding to images at each scale Let n represent the Hadamard product, n be the total number of scales, and e represent the learnable text embedding e.

4. The method for generating industrial defects at any scale according to claim 2, characterized in that, The method further includes: Adjust the original industrial defect image to the specified generated resolution. × Obtain the image with maximum scale cropping. and defect mask ; Extracting the defect mask The minimum bounding box of the connected components is obtained by expanding the minimum bounding box into a square based on its longer side, then cropping the region corresponding to the square and scaling it to the generated resolution. Obtain the image with the smallest scale crop. and the corresponding defect mask ; Centered on the geometric center of the minimum bounding box, the clipping side length is gradually increased geometrically according to the set clipping expansion factor α; the intermediate-scale clipping regions clipped after each expansion of the field of view are uniformly adjusted to the generated resolution. × A series of intermediate-scale input sequences were obtained. Finally, the complete multi-scale image and mask set is obtained by combining them. .

5. The method for generating industrial defects at any scale according to claim 2, characterized in that, The cross-scale perceptual loss is calculated based on adjacent-scale images in a complete multi-scale image. ,include: Clean latent variables are obtained by denoising latent variables at various scales using the predicted noise. The decoder outputs the predicted denoised image. ; High-resolution scale prediction map The defective region in the map is mapped according to the corresponding side length. Perform secondary cropping and adjust to the generated resolution. To obtain aligned high-resolution prediction blocks ; Using a pre-defined feature extractor Calculate denoised image and high-resolution prediction blocks The perceptual feature similarity in the corresponding region of the mask is used as the cross-scale perceptual loss. .

6. The method for generating industrial defects at any scale according to claim 2, characterized in that, The cross-scale sensing loss The cross-scale sensing loss function is obtained as follows: ； in, This represents the loss of perception across scales. For the first Masks corresponding to images at each scale For denoised images, For high-resolution prediction blocks, This represents the preset feature extractor, where n is the total number of scales.

7. The method for generating industrial defects at any scale according to claim 1, characterized in that, The method further includes: Based on Berlin noise, for normal, defect-free images Randomly generated defect mask ; Using the exact same multi-scale cropping operation as during training, normal, defect-free images are... Construct a set of normal image crop blocks and masks at multiple scales. ; At the minimum scale, use the smallest normal cropped image. Minimum normal cropped image Corresponding mask and trained text embeddings As input, a pre-defined number of denoising steps are used for mixing and diffusion in the mask. Generate minimum-scale initial defects with high-fidelity fine-grained texture within the region; For each subsequent large-scale level, the defect image generated at the previous scale is... Previous scale mask Normal cropped image at the current scale Compared with the current scale mask As input, output the defect image at the current scale. and current scale mask ; Repeat the above process in a loop until the maximum scale is reached, outputting an industrial defect image that blends seamlessly with the global background.

8. The method for generating industrial defects at any scale according to claim 7, characterized in that, The method further includes: Defect image from the previous scale and the previous scale mask According to the downsampling factor Resolution reduction is performed to obtain a downsampled defect map. and downsampling mask ; Using downsampling masks From the downsampled defect map Extract the valid defect pixels and paste them directly onto the normal image at the current scale. The current scale mask The stitched image is obtained at the specified precise location; Perform forward diffusion noise addition on the stitched image and mask it at the current scale. Under spatial constraints, local denoising and reconstruction are performed using a preset number of denoising steps to output a defect image at the current scale. .

9. An arbitrary-scale industrial defect generation device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor for performing the arbitrary-scale industrial defect generation method as described in any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that, The computer storage medium stores computer-executable instructions, which are executed by one or more processors to perform the arbitrary-scale industrial defect generation method as described in any one of claims 1-8.

Images