Industrial defect local generation method and device, equipment and storage medium

Abstract

The present application relates to the technical field of defect generation, and particularly relates to an industrial defect local generation method, device, equipment and storage medium, the method first performs hard mask enhancement processing on the obtained seed hard mask to obtain a binary hard mask, performs downsampling processing on the binary hard mask, determines an unmasked area based on the obtained latent space hard mask, performs filling processing on the latent space hard mask based on the unmasked area, packs the obtained conditional latent variable and a pre-constructed noise latent variable according to a preset packing rule, sets a conditional channel identifier, inputs the obtained context condition data into a pre-constructed initial defect generation model to perform multi-round iteration training processing, updates the low-rank adaptive weight of the initial defect generation model, performs local defect generation processing on the obtained to-be-generated defect image based on the obtained target defect generation model, and obtains a defect synthesis image, so as to realize controllable generation of defects in a specified area and obtain a high-quality defect synthesis image.

Description

Technical Field

[0001] This invention relates to the field of defect generation technology, and in particular to a method, apparatus, equipment and storage medium for localized generation of industrial defects. Background Technology

[0002] In industrial visual inspection scenarios, defect identification, localization, and segmentation models for industrial material surfaces heavily rely on training data that is abundant, evenly distributed in terms of categories, and reliably labeled. In real-world production environments, surface defects typically exhibit a long-tail distribution. While common defect samples are plentiful, rare categories and fine-grained morphological variants occur infrequently and are complex and varied in shape, leading to poor generalization performance of detection models when faced with out-of-distribution samples. Furthermore, defect data acquisition and manual labeling are costly, and some rare defects cannot be reproduced as needed. Relying solely on real-world samples makes it difficult to support the training of high-performance detection models.

[0003] Data augmentation and sample synthesis are effective ways to compensate for sample deficiencies, but existing technologies all have certain limitations. Real-world image acquisition and annotation are time-consuming and costly, making it difficult to fully cover various forms and conditions of rare defects. Traditional image enhancement methods can only change the underlying statistical features of an image and cannot generate new semantic structures of defects. Copy-paste synthesis is prone to lighting and texture mismatches, leading to synthesis artifacts. Physical simulation modeling is costly to maintain, and there is a distribution gap between simulated and real images. General generation models globally modify image content, making it difficult to achieve accurate defect generation in specified areas and also destroying the effective structural information of the original image. Conventional image restoration is prone to overfitting, making it difficult to produce transferable defect appearances. Introducing multi-branch control editing chains significantly increases system complexity and deployment difficulty. In summary, existing technologies cannot simultaneously achieve controllable local generation, fidelity in non-edited areas, transferable defect priors, and low deployment costs. The industry urgently needs a defect sample generation solution that can controllably synthesize target defects in specified areas of industrial images, preserve the original background appearance, and has low parameter tuning costs. Summary of the Invention

[0004] In order to overcome the shortcomings of the prior art, the present invention aims to provide a method, apparatus, equipment and storage medium for local generation of industrial defects, which aims to solve the problems of scarce industrial defect samples, poor controllability of local generation of existing generation technology and easy destruction of background, so as to realize the controllable generation of defects in a specified area, preserve the original background and obtain a high-quality defect synthesis image.

[0005] The first aspect of this invention provides a method for local generation of industrial defects, comprising: acquiring pre-stored industrial defect samples, the industrial defect samples including a seed hard mask; performing hard mask enhancement processing on the seed hard mask to obtain a binary hard mask; performing downsampling processing on the binary hard mask to obtain a latent space hard mask; determining unmasked regions based on the latent space hard mask, and performing filling processing on the latent space hard mask based on the unmasked regions to obtain conditional latent variables; packaging the conditional latent variables and pre-constructed noise latent variables according to a preset packaging rule, and setting conditional channel identifiers to obtain contextual conditional data; inputting the contextual conditional data into a pre-constructed initial defect generation model for multi-round iterative training processing to update the low-rank adaptation weights of the initial defect generation model to obtain a target defect generation model; acquiring a defect image to be generated, and performing local defect generation processing on the defect image to be generated based on the target defect generation model to obtain a composite defect image.

[0006] Optionally, in a first implementation of the first aspect of the present invention, obtaining the pre-stored industrial defect sample includes: obtaining a defect image; scaling the defect image based on a preset target resolution to obtain a target image; using a preset defect category template to perform text instruction construction processing on the target image to obtain a text instruction corresponding to the defect image; using a variational encoder to map the target image to a latent space to obtain image latent variables; using a text encoder to perform cue encoding processing on the text instruction to obtain a cue embedding; obtaining a bounding box corresponding to the defect image; performing a bounding box union operation on the bounding box to obtain the seed hard mask; integrating the image latent variables, the cue embedding, and the seed hard mask to obtain the industrial defect sample, and storing the industrial defect sample.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the step of performing hard mask enhancement processing on the seed hard mask to obtain a binary hard mask includes: performing random sampling processing on the seed hard mask to obtain at least one connected patch; verifying the connected patch based on a preset constraint rule; when the verification passes, using the verified connected patch as the binary hard mask; when the verification fails, performing repeated random sampling processing on the seed hard mask, and repeatedly verifying the connected patch obtained from the repeated random sampling processing; if the verification still fails after the number of repeated random sampling processing reaches a preset maximum number of samplings, then using the seed hard mask as the binary hard mask.

[0008] Optionally, in a third implementation of the first aspect of the present invention, the step of determining the unmasked region based on the latent space hard mask and filling the latent space hard mask based on the unmasked region to obtain the conditional latent variable includes: determining the region in the latent space hard mask where the mask value is 0 as the unmasked region; calculating channel statistics based on the latent variable data corresponding to the unmasked region; and filling the latent variable channels of the latent space hard mask based on the channel statistics to obtain the conditional latent variable.

[0009] Optionally, in a fourth implementation of the first aspect of the present invention, the step of inputting the contextual condition data into a pre-constructed initial defect generation model for multiple rounds of iterative training to update the low-rank adaptation weights of the initial defect generation model and obtain a target defect generation model includes: acquiring preset flow matching algorithm rules; randomly generating random noise samples based on the flow matching algorithm rules and determining the regression target corresponding to the random noise samples; inputting the random noise samples and the contextual condition data into the initial defect generation model for forward prediction processing to obtain a prediction output result; using a preset loss function, calculating the model regression error based on the prediction output result and the regression target; iteratively updating the low-rank adaptation weights of the initial defect generation model based on the model regression error; and outputting the target defect generation model when a preset iteration stopping condition is met.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, the step of acquiring the defect image to be generated and performing local defect generation processing on the defect image to be generated based on the target defect generation model to obtain a composite defect image includes: acquiring an original input image; preprocessing the original input image to obtain the defect image to be generated, and performing hole-filling processing on the defect image to be generated to obtain a conditional image; acquiring defect category information and a hard mask, and constructing a defect semantic prompt based on the defect category information; inputting the conditional image and the defect semantic prompt into the target defect generation model for sampling processing to obtain a sampling result; and linearly mixing the sampling result and the defect image to be generated, combined with the hard mask, in the area outside the hard mask according to a preset feathering weight to obtain the composite defect image.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, the step of preprocessing the original input image to obtain the defect image to be generated, and performing hole-filling processing on the defect image to obtain a conditional image, includes: scaling the original input image proportionally based on a preset longest side constraint rule to obtain a scaled image; converting the scaled image into a standard color array, and performing side length alignment processing on the standard color array based on a preset variational autoencoder size requirement to obtain the defect image to be generated; and performing hole-filling processing on the defect image to be generated and the hard mask based on a preset hole-filling strategy to obtain the conditional image.

[0012] A second aspect of the present invention provides an industrial defect local generation device, comprising: a hard mask enhancement module for acquiring pre-stored industrial defect samples, the industrial defect samples including a seed hard mask, performing hard mask enhancement processing on the seed hard mask to obtain a binary hard mask; a downsampling module for performing downsampling processing on the binary hard mask to obtain a latent space hard mask; a filling module for determining unmasked regions based on the latent space hard mask, and performing filling processing on the latent space hard mask based on the unmasked regions to obtain conditional latent variables; a packaging module for packaging the conditional latent variables and pre-constructed noise latent variables according to a preset packaging rule, and setting conditional channel identifiers to obtain contextual conditional data; a training module for inputting the contextual conditional data into a pre-constructed initial defect generation model for multi-round iterative training processing to update the low-rank adaptation weights of the initial defect generation model to obtain a target defect generation model; and a defect generation module for acquiring a defect image to be generated, performing local defect generation processing on the defect image to be generated based on the target defect generation model to obtain a composite defect image.

[0013] A third aspect of the present invention provides an industrial defect localization generation device, the industrial defect localization generation device comprising: a memory and at least one processor, the memory storing instructions; the at least one processor calling the instructions in the memory to cause the industrial defect localization generation device to perform the various steps of the industrial defect localization generation method described in any of the preceding claims.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of the industrial defect localization method described in any of the preceding claims.

[0015] In the technical solution of this invention, pre-stored industrial defect samples are first acquired. These samples include seed hard masks. The seed hard masks are then enhanced to obtain binary hard masks. The binary hard masks are then downsampled to obtain latent space hard masks. Unmasked regions are determined based on the latent space hard masks, and the latent space hard masks are filled based on the unmasked regions to obtain conditional latent variables. The conditional latent variables are then packaged with pre-constructed noise latent variables according to preset packaging rules, and conditional channel identifiers are set to obtain contextual condition data. The contextual condition data is then input into a pre-constructed initial defect generation model for multi-round iterative training to update the low-rank adaptation weights of the initial defect generation model, resulting in a target defect generation model. A defect image to be generated is then acquired. Based on the target defect generation model, local defect generation processing is performed on the defect image to be generated to obtain a composite defect image. This invention aims to solve the problems of scarce industrial defect samples, poor controllability of local generation in existing generation technologies, and easy destruction of the background. It achieves controllable generation of defects in specified areas, preserves the original background, and obtains a high-quality composite defect image. Attached Figure Description

[0016] Figure 1 A logic flowchart of the industrial defect localization generation method provided in the embodiments of the present invention; Figure 2 This is a schematic diagram of image comparison provided for an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the industrial defect localization generation device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an industrial defect localization generation device provided in an embodiment of the present invention. Detailed Implementation

[0017] This invention provides a method, apparatus, device, and storage medium for locally generating industrial defects. In this invention, the terms "first," "second," "third," "fourth," etc. (if present)," in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0018] The industrial defect local generation method, apparatus, equipment, and storage medium provided in this invention can be applied to scenarios such as industrial visual inspection sample expansion, detection model training, and industrial intelligent active learning pipelines, including but not limited to rare defect sample synthesis, fine-grained defect morphology generation, controllable defect editing in local areas, and packaged output of synthesized samples. The core of this invention lies in enhancing and latent space transforming the seed hard mask, constructing conditional latent variables and contextual conditional data, and obtaining the target defect generation model by iteratively training and updating the low-rank adaptation weights of the model. Relying on latent space condition constraints, low-rank adaptation fine-tuning, and a locally controllable generation mechanism, it solves the problems of long-tail scarcity of industrial defect samples, insufficient local controllability of defect generation, and easy destruction of non-edited backgrounds. It provides low-fine-tuning cost and high-fidelity underlying algorithm support for industrial visual defect detection model training and industrial defect synthesis data generation systems.

[0019] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the industrial defect localization method in this invention includes: 101. Obtain a pre-stored industrial defect sample, the industrial defect sample including a seed hard mask, and perform hard mask enhancement processing on the seed hard mask to obtain a binary hard mask. In this embodiment, the pre-stored industrial defect samples are pre-processed and saved data. The acquisition process involves reading the defect image, scaling it to the target resolution to obtain the target image, constructing corresponding text instructions based on the defect category template, mapping the target image to the latent space using a variational encoder to obtain image latent variables, performing cue encoding on the text instructions using a text encoder to obtain cue embeddings, extracting the bounding boxes corresponding to the defect image and performing a union operation on the bounding boxes to obtain a seed hard mask, and integrating the image latent variables, cue embeddings, and seed hard mask for storage. The seed hard mask is generated by the union operation of the bounding boxes corresponding to the defect image and serves as the spatial constraint reference for subsequent mask transformations. Hard mask enhancement processing uses a seed hard mask as a constraint. One or more connected patches are randomly sampled and generated within the seed hard mask. These connected patches are then validated according to predetermined constraint rules. Validation primarily controls the generated region to not exceed the seed hard mask's range, while also limiting the region coverage ratio, patch area ratio, and number of patches. If the sampled connected patches pass validation, they are output as a binary hard mask. If validation fails, the sampling and validation process is repeated. When the maximum number of samples is reached and the constraints are still not met, the seed hard mask is directly used as the binary hard mask. The binary hard mask is the binarized mask data output after hard mask enhancement processing, used to delineate the effective spatial range for defect generation. This processing method does not fix the mask shape used in the early training stage; sampling transformation is completed online during the training phase, generating diverse hole regions. This reduces the model's dependence on fixed rectangular masks, suppresses overfitting, and fully promotes the effective learning and formation of prior knowledge of defect appearance.

[0020] 102. Perform downsampling processing on the binary hard mask to obtain the latent space hard mask; In this embodiment, downsampling transforms the pixel-level binary hard mask to a resolution scale that matches the model's latent space, achieving spatial dimension alignment. After this transformation, a latent space hard mask is obtained. The latent space hard mask is a mask expression that maps the binary hard mask to the latent space resolution. It can mark defects at the latent space level to generate corresponding spatial ranges, allowing mask constraints to directly act on the latent variable calculation process. This eliminates the constraint deviation caused by the resolution mismatch between the pixel domain and the latent space, ensuring that subsequent latent variable filling operations can accurately distinguish different action areas based on the spatial mask. This achieves the adaptation and connection between mask conditions and latent space operations, providing a reliable spatial constraint basis for the construction of subsequent conditional latent variables.

[0021] 103. Determine the unmasked area based on the latent space hard mask, and fill the latent space hard mask based on the unmasked area to obtain the conditional latent variable; In this embodiment, the region with a mask value of 0 within the latent space hard mask is the unmasked region. This region corresponds to the background part of the image where no defects need to be generated, thus determining the unmasked region. Subsequently, the latent variable data corresponding to the unmasked region is extracted to solve the channel statistics. Then, the obtained channel statistics are used to perform a filling operation on the latent variable channels corresponding to the latent space hard mask, finally generating conditional latent variables. The conditional latent variables are the latent space representation obtained after latent space filling processing, carrying the texture and statistical features of the background region. This filling method can complete the latent variable completion of the mask position by relying on the statistical information of the real background itself, which can reduce the risk of information leakage of the original defect geometry through the conditional path, maintain the original color tone and texture statistical features of the industrial material surface, and make the filled area consistent with the overall latent space representation style, providing a reliable latent space conditional input for subsequent model training.

[0022] 104. Package the conditional latent variables and pre-constructed noise latent variables according to preset packaging rules, and set conditional channel identifiers to obtain contextual conditional data; In this embodiment, the preset packaging rules are set according to the data splicing logic of the token dimension. First, the noise latent variables and conditional latent variables are each converted into image token sequences. The splicing process along the token dimension is completed with the noise sequence arranged first and the conditional sequence arranged last. When performing the packaging operation, the conditional latent variables and the pre-constructed noise latent variables are encapsulated as image token sequences, and then the token dimension is spliced ​​and combined according to the predetermined sequence order. The conditional channel identifier is implemented by generating position identifiers for the two types of sequences. The first dimension of the conditional sequence identifier is assigned a value of 1 to complete the marking of the conditional image tokens and distinguish the noise part from the conditional reference part. After packaging and identifier setting, the contextual conditional data is obtained. This data integrates the two types of token information, namely noise and condition, and carries the position markers used for network recognition. This processing method can standardize the data organization form when latent variables enter the network, enabling the model to effectively distinguish between noise input and conditional reference information, ensuring that the latent space conditional constraints can be correctly read by the network, and providing a regular input basis for the sample construction of subsequent flow matching discrete scheduling and the updating of low-rank adaptation parameters.

[0023] 105. Input the contextual condition data into the pre-constructed initial defect generation model for multiple rounds of iterative training to update the low-rank adaptation weights of the initial defect generation model and obtain the target defect generation model. In this embodiment, the pre-constructed initial defect generation model is a base pre-trained flow matching generation network. The network architecture follows the original base network, reserving only a finely tuned low-rank adaptation branch. The base main parameters remain frozen throughout the process and do not participate in gradient updates. During multi-round iterative training, a preset flow matching algorithm rule is first obtained. Random noise samples are generated based on this rule and matched with the corresponding regression target. Then, the random noise samples, along with contextual condition data, are fed into the initial defect generation model to perform forward prediction and output the prediction result. Subsequently, a preset loss function is called to combine the prediction output and the regression target to solve the model regression error. The loss calculation will assign a region weight greater than 1 to the masked region in addition to the basic regression error, thereby improving the fitting constraint of the defect-corresponding region. Then, the obtained regression error is used for backpropagation, and gradient iteration updates are performed only on the low-rank adaptation weights. The base network main parameters remain unchanged. The sample generation, forward inference, error calculation, and parameter update process are repeatedly looped until the preset iteration stop condition is triggered, at which point the training is terminated. The low-rank adaptation weights of the initial defect generation model are a small set of parameters appended to the base network. They do not modify the original base backbone weights, but only achieve capability transfer for downstream defect generation tasks through low-rank matrix mapping. After all iterations, the target defect generation model is obtained. This model inherits the original generation capabilities of the base network and learns the industrial defect-oriented generation logic guided by masking. After training, the adaptation weights, along with the scheme identifier, condition construction method, masking enhancement strategy, and key hyperparameters, are retained as training metadata. This training paradigm can fully reuse the image prior knowledge learned by the base network pre-training, relying only on a small number of low-rank adaptation parameters to complete task adaptation. This reduces the computational overhead and overfitting risk associated with fine-tuning the complete model. The weighted loss in the masked region further enhances the generation fitting effect in the defect region, ensuring that the final model can output the expected defect generation results based on the latent space constraints.

[0024] 106. Obtain the defect image to be generated, and perform local defect generation processing on the defect image to be generated based on the target defect generation model to obtain a composite defect image.

[0025] In this embodiment, the defect image to be generated is image data adapted to the input specifications of the target defect generation model after format conversion, size scaling and side length alignment. When acquiring the image, the original input image is read, and proportional scaling is performed according to the longest side constraint rule. The scaled image is converted into a standard color array, and side length alignment is performed according to the size requirements of the variational autoencoder. The defect image to be generated can be obtained by completing the preprocessing of the original input image. If no mask is directly provided, a hard mask is generated by rasterizing the normalized bounding box set. Alternatively, context clipping can be enabled to extract a local area according to the target union expansion ratio for generation operation. After generation, the processed local area is pasted back into the complete image. When performing local defect generation processing on the image to be generated based on the target defect generation model, a hard mask is first obtained. A pre-set hole-filling strategy is used to perform hole-filling processing on the image to be generated and the hard mask to obtain a conditional image. At the same time, defect category information is read, and a defect semantic prompt is constructed based on the defect category information. The conditional image and defect semantic prompt are fed into the target defect generation model to perform sampling processing to obtain the sampling result. The operation process can enable the reinforcement guidance mechanism and negative prompts as needed to constrain the generated content. Then, the sampling result and the image to be generated are combined with the hard mask. In the area outside the hard mask, a pre-set feathering weight is used to perform linear blending to eliminate the boundary discontinuity between the generated area and the original image. Finally, a defect synthesis image is obtained. The defect synthesis image is the output result after completing the local defect regeneration. At the same time, the image, mask, category, control parameters and metadata are encapsulated and integrated to form a complete synthetic sample. This processing workflow can unify the input image size to adapt to the input constraints of the model encoder. Contextual cropping can reduce the actual computation area and reduce the computational overhead in the inference stage. Hole filling provides reasonable image prior conditions for the model. Feathering and blending operations ensure a natural transition between the generated region and the original image, avoiding harsh splicing boundaries. Combined with defect semantic prompts and optional guiding constraints, it can accurately control the category and manifestation of defect generation. Finally, it outputs a defect synthesis image carrying complete meta-information, which can directly serve the defect sample expansion task in industrial inspection scenarios.

[0026] In this embodiment of the invention, obtaining pre-stored industrial defect samples includes: acquiring a defect image; scaling the defect image based on a preset target resolution to obtain a target image; using a preset defect category template to construct text instructions on the target image to obtain text instructions corresponding to the defect image; using a variational encoder to map the target image to a latent space to obtain image latent variables; using a text encoder to perform cue encoding on the text instructions to obtain cue embeddings; acquiring bounding boxes corresponding to the defect image; performing bounding box union operations on the bounding boxes to obtain the seed hard mask; integrating the image latent variables, the cue embeddings, and the seed hard mask to obtain the industrial defect sample, and storing the industrial defect sample.

[0027] In this embodiment, the defect image carries real defect and background texture information of various material surfaces in industrial scenarios. The preset target resolution is set based on the input size requirements during model training, adapting to the input constraints of the subsequent variational encoder. When scaling the defect image based on the preset target resolution, the original defect image is scaled to adjust its size to the specifications corresponding to the target resolution, resulting in the target image. The target image is the image that meets the model's input size requirements after scaling. The preset defect category template is a predefined text generation paradigm for industrial defect types, set based on all defect categories included in the dataset. When using the preset defect category template to construct text instructions for the target image, the defect category corresponding to the sample is read, and the category information is filled into the template to generate text content describing the defect attributes. The text instructions corresponding to the defect image are the instruction text used to describe the sample defect category, providing semantic input for subsequent text encoding. A variational encoder is an autoencoder network for image compression, constructed from an encoder and a decoder. The process of mapping a target image to a latent space using a variational encoder involves feeding the target image into the encoder, where multiple layers of the network extract image features and compress dimensionality, transforming the pixel-domain image into a low-dimensional latent space. The latent variables of the image represent the feature representation of the target image in the latent space. A text encoder is a pre-trained text semantic coding network built upon publicly available pre-trained language models. The process of encoding text instructions using a text encoder involves inputting the text instruction into the encoder, where the network extracts semantic features and outputs a prompt embedding. The prompt embedding represents a high-dimensional semantic feature vector corresponding to the text instruction. The bounding boxes corresponding to the defect images are the defect location selection information obtained from the annotations of the dataset. These can be obtained by reading the annotation files paired with the defect images within the dataset. The process of performing a bounding box union operation involves finding the union of the coverage areas of all defect bounding boxes in the same image, merging overlapping or adjacent selection regions. The seed hard mask is the mask output from the bounding box union operation, serving as the spatial constraint benchmark for subsequent mask transformations. By integrating image latent variables, cue embeddings, and the seed hard mask, industrial defect samples are obtained and stored. Stored samples can be cached and reused using metadata summaries, avoiding the computational overhead of repeated encoding. The entire processing flow completes the preprocessing and persistent storage of multimodal information such as images, text, and masks. The final mask used for training is not fixed at this stage. In subsequent training iterations, diverse binary hard masks are obtained through online sampling based on the seed hard mask, improving the diversity of hole shapes, reducing the model's dependence on fixed rectangular masks, mitigating the risk of overfitting, and simultaneously caching latent space and text features in advance to reduce repeated encoding operations during training, effectively reducing the overall computational consumption during the training phase.

[0028] In this embodiment of the invention, the step of performing hard mask enhancement processing on the seed hard mask to obtain a binary hard mask includes: performing random sampling processing on the seed hard mask to obtain at least one connected patch; verifying the connected patch based on preset constraint rules; when the verification passes, using the verified connected patch as the binary hard mask; when the verification fails, performing repeated random sampling processing on the seed hard mask, and repeatedly verifying the connected patch obtained from the repeated random sampling processing; if the verification still fails after the number of repeated random sampling processing reaches a preset maximum number of samplings, then using the seed hard mask as the binary hard mask.

[0029] In this embodiment, random sampling of the seed hard mask uses the area defined by the seed hard mask as the spatial constraint boundary. Random sampling is performed within this area, outputting one or more image regions whose internal pixels are connected. At least one connected patch represents the number of independent connected image regions that can be produced in a single sampling process. The connected patch is a binary region where pixels within the mask are interconnected, used to constitute the hole range corresponding to defect generation. Preset constraint rules include that the generated region cannot exceed the spatial boundary of the seed hard mask, the coverage ratio of the seed hard mask region needs to reach a set lower limit, the patch area ratio is within a predefined numerical range, and the total number of patches falls within a set upper and lower limit range. The lower limit of the seed hard mask region coverage ratio, the numerical range of the patch area ratio, and the upper and lower limits of the number of patches are all custom hyperparameters adapted to industrial defect generation training, without fixed values. They are comprehensively set and optimized based on the actual morphological characteristics of industrial defects, the defect distribution patterns in the dataset, and the model training effect. The constraint rules are set by combining the actual morphological characteristics of industrial defects with the distribution characteristics of training samples, taking into account both the diversity of hole morphology and the rationality of the defect generation region. Verifying connected patches involves checking each sampled connected patch's metrics one by one. This includes comparing the patch's spatial range, coverage ratio, area proportion, and number of patches to ensure they all match the constraint rules. Verification passes if all metrics of the sampled connected patches meet the constraint rules. Passing verification means that the randomly sampled connected patches are suitable for defect generation training in terms of spatial range, scale, and number, and are qualified as regions for generating holes. Therefore, the verified connected patches are used as binary hard masks, which define the effective spatial range for defect generation during the training phase. Verification fails when sampled connected patches exceed spatial limits, have insufficient coverage ratio, deviate from the area proportion range, or exceed the limit in number. Failure to verify means that the shape and scale of the currently sampled connected patches do not meet the training requirements and cannot be directly used as regions for defect generation. Repeated random sampling is performed on the seed hard mask, and repeated verification is performed on the connected patches obtained by repeated random sampling. This means that when the verification result does not meet the conditions, the sampling and index verification process is continuously iterated to generate new connected patches and check each constraint index one by one until a result that meets the conditions is obtained or the upper limit of the number of samplings is reached.The preset maximum number of samplings is a pre-configured iteration termination threshold, set based on the computational cost and sample processing efficiency during the training phase. This is used to avoid the sampling process from getting stuck in an infinite loop. If the verification still fails after the preset maximum number of random samplings is reached, the seed hard mask is used as a binary hard mask. This fallback strategy avoids excessive computational resources being consumed by continuous sampling, ensuring the stable progress of the training process. The entire hard mask enhancement process does not fix the mask shape during the preprocessing stage. The sampling transformation is completed online during the training iteration process, which can generate holes with rich and varied shapes. This reduces the model's learning dependence on fixed rectangular masks, suppresses overfitting, helps the model fully learn the prior knowledge of the appearance of industrial defects, and improves the generalization ability of defect generation in the subsequent inference stage.

[0030] In this embodiment of the invention, the step of determining the unmasked region based on the latent space hard mask and filling the latent space hard mask based on the unmasked region to obtain conditional latent variables includes: determining the region in the latent space hard mask where the mask value is 0 as the unmasked region; calculating channel statistics based on the latent variable data corresponding to the unmasked region; and filling the latent variable channels of the latent space hard mask based on the channel statistics to obtain the conditional latent variables.

[0031] In this embodiment, regions with a mask value of 0 in the latent space hard mask are defined as unmasked regions. This means using the pixel values ​​of the latent space hard mask as the basis for region division to distinguish the image background from the region to be generated. The mask value is a binary identifier parameter used in the latent space to characterize the region attributes, and the magnitude of the value corresponds to different functional attributes of the image region. A mask value of 0 indicates that the corresponding location belongs to the background region where defect generation is not required, and is therefore defined as an unmasked region. A mask value of 1 indicates that the target region for defect generation and editing needs to be performed, which is the effective range for model defect generation. The latent variable data corresponding to the unmasked region is the multidimensional feature data corresponding to the background region after the image is mapped to the latent space by the variational encoder, carrying the texture, color, and grayscale statistical information of the original image background. The channel statistics are calculated based on the latent variable data corresponding to the unmasked region. This involves extracting the latent variable feature values ​​of all dimensions of the background region, calculating the mean and variance of the data distribution of each feature channel, and obtaining the channel statistics by performing a global statistical solution on the effective background feature data. The channel statistics can accurately characterize the overall distribution pattern and feature tone of the image background region in each channel of the latent space. Latent variable channels are independent dimensional channels representing latent space features. Different channels store detailed features and semantic information at different levels of the image, collectively forming a complete latent variable feature matrix. Filling the latent variable channels of the latent space hard mask based on channel statistics involves replacing the latent variable channel data corresponding to areas with a mask value of 1 with the solved background channel statistics one by one. This utilizes the feature distribution patterns of the real background to smooth out the latent variable information of defective areas, completing the latent space-level region filling and reconstruction. This processing method can complete the latent variable completion of defective areas based on the feature statistical patterns of the real background itself, effectively eliminating residual leakage of the original defect geometric structure and texture information, avoiding interference from the original defect information on model training, and ensuring that the latent variable features after filling are consistent in style and distribution with the overall image background features. It fully preserves the original tone and texture characteristics of the industrial material surface, providing a clean, regular, and realistic latent space condition input for subsequent model training and defect generation, significantly improving the accuracy of local defect generation and background fidelity.

[0032] In this embodiment of the invention, the step of inputting the contextual condition data into a pre-constructed initial defect generation model for multiple rounds of iterative training to update the low-rank adaptation weights of the initial defect generation model and obtain a target defect generation model includes: acquiring preset flow matching algorithm rules; randomly generating random noise samples based on the flow matching algorithm rules and determining the regression target corresponding to the random noise samples; inputting the random noise samples and the contextual condition data into the initial defect generation model for forward prediction processing to obtain a prediction output result; using a preset loss function, calculating the model regression error based on the prediction output result and the regression target; iteratively updating the low-rank adaptation weights of the initial defect generation model based on the model regression error; and outputting the target defect generation model when a preset iteration stopping condition is met.

[0033] In this embodiment, a preset flow matching algorithm rule is obtained. The flow matching algorithm rule is a diffusion flow scheduling logic configured to adapt to the image local defect generation task. It is jointly set according to the original operation mechanism of the base flow matching diffusion image to the image pipeline and the training requirements of industrial defect local regeneration. It is used to control the scheduling logic and sample pairing method for noise evolution. Random noise samples are randomly generated based on the flow matching algorithm rule. The random noise samples are noise feature representations that follow a specified distribution in the latent space dimension, corresponding to the latent variable input in the initial stage of the diffusion process. The generation process samples a specified time step according to the time scheduling of flow matching, constructs a multidimensional noise tensor aligned with the latent space resolution dimension, and completes the construction of random noise samples. The regression target corresponding to the random noise sample is determined simultaneously. The regression target is the latent space feature true value that is expected to be approximated at the corresponding time step under the flow matching framework. It represents the feature state that the noise sample should converge to along the flow evolution path. The regression target is calculated based on the flow field mapping relationship of flow matching, combined with the latent variables of the real sample and the sampling time step. Random noise samples and contextual condition data are input into the initial defect generation model for forward prediction processing. The initial defect generation model uses a pre-trained flow matching generation network as its base, with all backbone parameters of the base remaining frozen. Only low-rank adaptation branches that can participate in parameter updates are built in. The forward prediction process involves the model receiving two types of input data and then using a multi-layer network structure to perform layer-by-layer deduction of latent space features and flow field fitting. Combining the spatial constraints and prior image information carried by the contextual condition data, the evolution trend of random noise samples is calculated, and the prediction output is the latent space flow field prediction feature obtained by the model fitting. A pre-defined loss function is adopted, which uses the basic regression error of flow matching as the calculation basis. An additional masking region weight mechanism is introduced, which assigns a region weight greater than 1 to the masking region corresponding to the defect generation, thereby strengthening the constraint of the defect region. The model regression error is calculated based on the predicted output and the regression target. The regression error represents the feature difference between the network's predicted output and the true regression target. During the calculation process, differentiated weights are assigned to different image regions according to the spatial mask. The base weight is used for the non-masked background region, and a weighting coefficient greater than the base weight is applied to the defect generation region inside the mask. The feature differences of all spatial locations are weighted and statistically analyzed to obtain the overall model regression error. The low-rank adaptation weights of the initial defect generation model are iteratively updated based on the model regression error. The low-rank adaptation weights are a small set of additional parameters mounted inside the base network. The downstream task capability is transferred in the form of low-rank matrix mapping. The parameters of the base backbone network are kept frozen throughout the training process and do not participate in gradient updates. During the iterative update process, the model regression error is backpropagated to solve only the gradient information corresponding to the low-rank adaptation weights. The gradient iteration of the weight parameters is completed by the optimizer, and the low-rank matrix mapping relationship is continuously corrected to adapt to the local generation task of industrial defects.The preset iteration stopping condition is used to terminate the multi-round training iteration process. It is set according to the business task type, the number of training steps for a single class of samples, and the loss convergence status. The iteration stopping condition can be reaching a preset training step threshold or the model regression error decreasing to a specified convergence threshold. When the preset iteration stopping condition is met, the target defect generation model is output.

[0034] The entire training process fully reuses the prior image knowledge learned in the pre-training stage of the pedestal network. It only relies on a small number of low-rank adaptation weights to complete the downstream task adaptation, without the need to fine-tune the parameters of the complete pedestal backbone. This effectively reduces the computational power consumption in the training stage and alleviates the risk of overfitting caused by large-scale parameter updates. The masking region weighting mechanism introduced by the loss function further amplifies the error constraint of the defect generation region and improves the fitting accuracy of the corresponding defect position. The final output target defect generation model not only retains the pedestal's original image generation capability, but also learns the inherent logic of industrial defect orientation generation under mask guidance. After training, the low-rank adaptation weights, scheme identifiers, condition construction methods, mask enhancement strategies, and key hyperparameters are retained simultaneously, providing a complete and usable model running foundation for the defect generation task in the subsequent inference stage.

[0035] In this embodiment of the invention, the step of acquiring the defect image to be generated and performing local defect generation processing on the defect image to be generated based on the target defect generation model to obtain a composite defect image includes: acquiring an original input image; preprocessing the original input image to obtain the defect image to be generated; and performing hole-filling processing on the defect image to be generated to obtain a conditional image; acquiring defect category information and a hard mask; constructing a defect semantic prompt based on the defect category information; inputting the conditional image and the defect semantic prompt into the target defect generation model for sampling processing to obtain a sampling result; and linearly mixing the sampling result and the defect image to be generated, combined with the hard mask, in the region outside the hard mask according to a preset feathering weight to obtain the composite defect image.

[0036] In this embodiment, the original input image is taken from the sample image to be expanded obtained from an industrial scene. The preprocessing of the original input image involves scaling it proportionally according to the longest side constraint rule to obtain a scaled image, converting the scaled image into a standard color array, and then aligning the side lengths according to the input size requirements of the variational autoencoder to output a defect image to be generated that conforms to the input specifications of the target defect generation model. This process can simultaneously obtain a hard mask. The hard mask is used to distinguish the defect area to be generated from the original background area within the image. If existing mask data is available, it can be used directly. If no mask is provided, the hard mask is obtained through rasterization of the normalized bounding box set. The normalized bounding box set comes from the defect annotation information carried by the input sample, corresponding to the bounding boxes of each defect target in the image. The bounding box coordinates have been normalized, and the coordinate numbers are... The values ​​are mapped to a range of 0 to 1, unaffected by the original image resolution. Rasterization maps this set of normalized defect bounding boxes onto the pixel grid corresponding to the current defect image to be generated. All pixel positions enclosed by the bounding boxes are marked as the defect region to be generated, and the remaining pixel positions are marked as the original background region. When a single image contains multiple defect targets, the normalized bounding boxes corresponding to all defects are combined and then rasterized. This set of bounding boxes can come from the output results of the previous defect detection model or from defect annotation data obtained by manual annotation. The process can also optionally enable a contextual cropping mechanism to crop a local area of ​​the image according to the target union expansion ratio for subsequent calculations. After the generation calculation is completed, the local processing results are pasted back into the complete image, thereby reducing the actual calculation range and lowering the overall computational cost in the inference stage. Hole filling processing of the defect image to be generated is performed based on hard masking as the basis for region division. A preset hole filling strategy is called to perform content filling operations on the defect image to be generated, finally obtaining a conditional image. The conditional image carries the background prior of the original image and provides image-level constraints as input data for the model sampling process. Defect category information corresponds to the category labels of various defects in industrial scenarios, derived from sample annotations or external detection output results. When constructing defect semantic prompts, defect category information is used as the core basis, transforming it into semantic guidance information that the model can recognize. Defect semantic prompts are used to constrain the content generated by the model at the semantic level, limiting the type characteristics of the output defects. Inputting the conditional image and defect semantic prompts into the target defect generation model for sampling processing is a flow-matching image-to-image inference operation. During the operation stage, reinforcement guidance mechanisms and negative prompts can be enabled as needed to further constrain the performance of the generated content. The generation calculation of local defect content is completed through latent space flow evolution deduction within the model, outputting the sampling results. The sampling results are image data carrying newly generated defect content obtained through model inference.The sampling results and the defect image to be generated are combined with a hard mask to perform linear blending. The area outside the hard mask refers to the original image background area not covered by the hard mask. The preset feathering weight is used to control the pixel fusion ratio between the sampling results and the defect image to be generated. This weight is set according to the need to suppress hard boundaries of splicing and ensure the continuity of the image appearance. In the operation process, for the background area outside the hard mask, the feathering weight is used to complete the linear blending of the two sets of image pixels, weaken the boundary discontinuity between the generated area and the original background, avoid the artifacts caused by hard splicing, and finally obtain the defect synthesis image. The entire processing flow adapts to the input constraints of the variational autoencoder through size normalization, compresses inference computation through context pruning, provides reliable image priors for the model through hole filling, and achieves controllable defect generation type and morphology through semantic cues and optional guiding constraints. Feathered weighted pixel fusion ensures pixel fidelity and appearance continuity of the generated image, achieving a smooth transition between the defect region and the original background. The output defect synthetic image is accompanied by complete image and category association information. The generated samples can be further encapsulated into metadata, which can be directly used for sample expansion in industrial defect detection tasks, providing high-quality synthetic samples for detection model training and active learning pipelines. At the same time, in the subsequent use stage, the detection box, segmentation mask, category, and source traceability metadata can be exported synchronously, which facilitates hierarchical evaluation and ratio control, and supports the iterative improvement of the model's generalization ability under real working conditions.

[0037] like Figure 2 As shown, Figure 2 The diagram shows the image comparison. (a) is the original image, which is the original input image collected from an industrial scene. The image contains real background texture and original defect spots. (b) is the inference condition image, which is the condition image obtained after hole filling. The blue box in the image corresponds to the defect area to be generated defined by the hard mask. The pixels inside this area are uniformly filled with gray. The background area completely retains the pixel information of the original image, thus informing the target defect generation model to regenerate the defect content only within the box area, while the background remains unchanged. (c) is the generated image, which is the sampling result output by the target defect generation model. The model only generates new defect textures within the hard mask area corresponding to the blue box. The non-generated areas outside the box have darkening distortion and cannot be directly used as the final image. (d) is the feathered synthesis result, which is the final defect synthesis image. The newly generated defect texture in the mask area of ​​the sampling result is extracted and linearly mixed with the defect image to be generated according to the hard mask and the preset feathering weight. The mask edge is smoothed and the real background of the original image is reused to eliminate the hard boundary artifacts of the box caused by direct splicing, resulting in a complete synthetic sample with real background texture and defects regenerated by the model.

[0038] In this embodiment of the invention, the preprocessing of the original input image to obtain the defect image to be generated, and the hole-filling processing of the defect image to be generated to obtain a conditional image, includes: scaling the original input image proportionally based on a preset longest side constraint rule to obtain a scaled image; converting the scaled image into a standard color array, and aligning the standard color array according to a preset variational autoencoder size requirement to obtain the defect image to be generated; and performing hole-filling processing on the defect image to be generated and the hard mask based on a preset hole-filling strategy to obtain the conditional image.

[0039] In this embodiment, the preset longest side constraint rule is set based on the upper limit of computing power in the model inference stage and the requirement to retain effective image information. This rule limits the target pixel value of the longest side of the image. When performing proportional scaling processing on the original input image based on the preset longest side constraint rule, the scaling factor is determined by the ratio of the original long side size to the target long side size of the original input image. The image width and height are synchronously transformed according to the same scaling factor. While changing the overall resolution of the image, the aspect ratio of the original image remains unchanged, resulting in a scaled image. The scaled image is image data that maintains the aspect ratio of the original image and completes the resolution adjustment. Converting the scaled image into a standard color array is to parse the pixel information of the image into multi-dimensional tensor data with channel dimensions arranged in an ordered manner. The standard color array is an image tensor with pixel information normalized and adapted to the reading format of the subsequent encoder. The preset variational autoencoder size requirement is set based on the convolution downsampling factor of the variational autoencoder network. The width and height of the image are required to be integer multiples of the downsampling factor of the network. Based on the preset variational autoencoder size requirement, the standard color array is subjected to side length alignment processing. This involves rounding up the width and height of the scaled image and padding it. The width and height of the image are adjusted to the nearest neighbor pixel size that meets the factor constraint to obtain the defect image to be generated. The defect image to be generated is image data that has been scaled, format converted and side length aligned, fully retains the image content and meets the input specifications of the target defect generation model. The preset hole-filling strategy aims to preserve the original image statistical features of the non-masked area and avoid introducing additional color shifts and texture breaks during the generation process. Based on the preset hole-filling strategy, the process of hole-filling is performed on the defect image to be generated and the hard mask. The image region is divided by the hard mask, the original pixel information of the background region outside the hard mask is preserved, and pixel filling is performed on the defect area to be generated covered by the hard mask. This provides a reasonable image prior basis for flow matching sampling. After processing, a conditional image is obtained. The conditional image takes into account the real statistical features of the original image background and can provide image-level conditional constraint input data for the sampling process of the target defect generation model.The original input image is taken from the sample image to be expanded obtained from an industrial scene. This process can simultaneously acquire a hard mask, which is used to distinguish the defect area to be generated from the original background area within the image. If existing mask data is available, it can be used directly. If no mask is provided, the hard mask is obtained through a rasterization operation on the normalized bounding box set. The normalized bounding box set originates from the defect annotation information carried by the input sample, corresponding to the bounding boxes of each defect target in the image. The bounding box coordinates have been normalized, mapping the coordinate values ​​to the range of 0 to 1, unaffected by the original image resolution. The rasterization operation maps this set of normalized defect bounding boxes to the current area to be generated. On the pixel grid corresponding to the defect image, all pixel positions enclosed by the bounding box are marked as the defect region to be generated, and the remaining pixel positions are marked as the original background region. When there are multiple defect targets in a single image, the normalized bounding boxes corresponding to all defects are combined and then rasterized. This bounding box set can come from the output results of the previous defect detection model or from defect annotation data obtained by manual annotation. The process can also optionally enable a contextual cropping mechanism to crop a local area of ​​the image according to the expansion ratio of the target union to participate in subsequent calculations. After the calculation is completed, the local processing result is pasted back into the complete image, thereby reducing the actual calculation range and reducing the overall computational cost in the inference stage. Defect category information corresponds to the category labels of various defects in industrial scenarios, which come from sample annotations or external detection output results. When constructing defect semantic prompts, defect category information is used as the core basis to transform category information into semantic guidance information that the model can recognize. Defect semantic prompts are used to constrain the content generated by the model at the semantic level and limit the type characteristics of the output defects. The conditional image and defect semantic prompts are input into the target defect generation model for sampling processing. This is a flow-matching image-to-image inference operation. During the operation stage, reinforcement guidance mechanisms and negative prompts can be enabled as needed to further constrain the appearance of the generated content. The generation calculation of local defect content is completed through latent space flow evolution inference within the model, and the sampling result is output. The sampling result is image data carrying newly generated defect content obtained through model inference. The sampling result and the defect image to be generated are combined with a hard mask to perform linear blending operation. The area outside the hard mask refers to the original image background area not covered by the hard mask. The preset feathering weight is used to control the pixel fusion ratio between the sampling result and the defect image to be generated. This weight is set according to the need to suppress hard boundaries of splicing and ensure the continuity of image appearance. During the operation, for the background area outside the hard mask, the feathering weight is used to complete the linear blending of the two sets of image pixels, weakening the boundary discontinuity between the generated area and the original background, avoiding artifacts caused by hard splicing, and finally obtaining the defect synthesis image.The entire processing flow relies on the longest side constraint and side length alignment to normalize the input image, controlling the computational overhead of the inference stage while ensuring the integrity of image information. Variational autoencoder size alignment ensures that the network encoder can complete latent space mapping normally. Contextual cropping further compresses the actual computation area. The hole-filling strategy reuses the background statistics of the original image to avoid color shift and texture breakage in the synthesized image. Semantic prompts and optional guiding constraints jointly achieve controllability of defect generation type and morphology. Feathered weighted pixel fusion operation ensures pixel fidelity and appearance continuity of the generated image, achieving a smooth transition between the defect area and the original background. After generation, the detection box, segmentation mask, category, and traceability metadata can be simultaneously encapsulated and exported to form a synthetic sample with complete traceability information. This can be directly used for sample expansion in industrial defect detection tasks, providing high-quality synthetic samples for detection model training and active learning pipelines. It can also be used for hierarchical evaluation and ratio control based on synthetic samples to reasonably control the proportion of synthetic samples in the training set, reduce the risk of domain shift, and support the iterative improvement of model generalization ability under real working conditions.

[0040] The above describes the method for generating localized industrial defects in embodiments of the present invention. The following describes the apparatus for generating localized industrial defects in embodiments of the present invention. Please refer to [link / reference]. Figure 3 One embodiment of the industrial defect localization device in this invention includes: Hard mask enhancement module 301: used to acquire pre-stored industrial defect samples, the industrial defect samples including seed hard masks, and perform hard mask enhancement processing on the seed hard masks to obtain binary hard masks. Downsampling module 302: used to perform downsampling processing on the binary hard mask to obtain a latent space hard mask; Filling module 303: used to determine the unmasked area based on the latent space hard mask, and fill the latent space hard mask based on the unmasked area to obtain conditional latent variables; Packaging module 304: used to package the conditional latent variables and pre-constructed noise latent variables according to preset packaging rules, and set conditional channel identifiers to obtain contextual conditional data; Training module 305: used to input the contextual condition data into the pre-constructed initial defect generation model for multiple rounds of iterative training to update the low-rank adaptation weights of the initial defect generation model and obtain the target defect generation model; Defect generation module 306: used to acquire the defect image to be generated, and perform local defect generation processing on the defect image to be generated based on the target defect generation model to obtain a composite defect image.

[0041] Based on the same ideas as the methods in the above embodiments, the apparatus provided in this application can implement the methods in the above embodiments.

[0042] above Figure 3 The industrial defect localization generation device in this embodiment of the invention is described in detail from the perspective of modular functional entities. The industrial defect localization generation device in this embodiment of the invention is described in detail from the perspective of hardware processing.

[0043] Figure 4 This is a schematic diagram of the structure of an industrial defect localization generation device 400 provided in an embodiment of the present invention. The industrial defect localization generation device 400 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 410 (e.g., one or more processors) and a memory 420, and one or more storage media 430 (e.g., one or more mass storage devices) storing application programs 433 or data 432. The memory 420 and storage media 430 can be temporary or persistent storage. The program stored in the storage media 430 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the industrial defect localization generation device 400. Furthermore, the processor 410 may be configured to communicate with the storage media 430 and execute the series of instruction operations in the storage media 430 on the industrial defect localization generation device 400 to implement the steps of the industrial defect localization generation method provided in the above-described method embodiments.

[0044] The industrial defect localization generation device 400 may also include one or more power supplies 440, one or more wired or wireless network interfaces 450, one or more input / output interfaces 460, and / or one or more operating systems 431, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 4 The illustrated structure of the industrial defect localization generation device does not constitute a limitation on the industrial defect localization generation device, and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0045] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the industrial defect localization method.

[0046] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0047] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0048] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for locally generating industrial defects, characterized in that, include: Obtain a pre-stored industrial defect sample, the industrial defect sample including a seed hard mask, and perform hard mask enhancement processing on the seed hard mask to obtain a binary hard mask. The binary hard mask is downsampled to obtain the latent space hard mask; The unmasked region is determined based on the latent space hard mask, and the latent space hard mask is filled based on the unmasked region to obtain the conditional latent variable; The conditional latent variables and pre-constructed noise latent variables are packaged according to a preset packaging rule, and a conditional channel identifier is set to obtain contextual conditional data. The contextual condition data is input into the pre-constructed initial defect generation model for multiple rounds of iterative training to update the low-rank adaptation weights of the initial defect generation model, thereby obtaining the target defect generation model. A defect image to be generated is obtained, and a local defect generation process is performed on the defect image to be generated based on the target defect generation model to obtain a composite defect image.

2. The method for locally generating industrial defects according to claim 1, characterized in that, The acquisition of pre-stored industrial defect samples includes: A defect image is acquired, and the defect image is scaled based on a preset target resolution to obtain a target image; The target image is processed using a preset defect category template to construct text instructions, thereby obtaining text instructions corresponding to the defect image; A variational encoder is used to map the target image to the latent space to obtain the image latent variables; The text instructions are encoded using a text encoder to obtain the embedded prompts. Obtain the bounding box corresponding to the defect image, perform a bounding box union operation on the bounding box, and obtain the seed hard mask; The industrial defect sample is obtained by integrating the image latent variables, the cue embedding, and the seed hard mask, and the industrial defect sample is stored.

3. The method for locally generating industrial defects according to claim 1, characterized in that, The step of performing hard mask enhancement processing on the seed hard mask to obtain a binary hard mask includes: Random sampling is performed on the seed hard mask to obtain at least one connected patch; The connected patches are verified based on preset constraint rules; When the verification passes, the connected patches that pass the verification are used as the binary hard mask; If the verification fails, the seed hard mask is subjected to repeated random sampling, and the connected patches obtained by the repeated random sampling are repeatedly verified. If the verification still fails after the number of repeated random sampling reaches the preset maximum number of samplings, the seed hard mask is used as the binary hard mask.

4. The method for locally generating industrial defects according to claim 1, characterized in that, The process of determining the unmasked region based on the latent space hard mask and filling the latent space hard mask based on the unmasked region to obtain conditional latent variables includes: The region in the latent space hard mask where the mask value is 0 is defined as the unmasked region; Calculate channel statistics based on the latent variable data corresponding to the unmasked area; The latent variable channels of the latent space hard mask are filled based on the channel statistics to obtain the conditional latent variable.

5. The method for locally generating industrial defects according to claim 1, characterized in that, The step of inputting the contextual condition data into a pre-constructed initial defect generation model for multiple rounds of iterative training to update the low-rank adaptation weights of the initial defect generation model and obtain the target defect generation model includes: Obtain preset flow matching algorithm rules, randomly generate random noise samples based on the flow matching algorithm rules, and determine the regression target corresponding to the random noise samples; The random noise samples and the contextual condition data are input into the initial defect generation model for forward prediction processing to obtain the prediction output results. The regression error of the model is calculated based on the predicted output and the regression target using a preset loss function. The low-rank adaptation weights of the initial defect generation model are iteratively updated based on the model regression error. When the preset iteration stopping condition is met, the target defect generation model is output.

6. The method for locally generating industrial defects according to claim 1, characterized in that, The process of acquiring the defect image to be generated and performing local defect generation processing on the defect image to be generated based on the target defect generation model to obtain a composite defect image includes: The original input image is acquired, the original input image is preprocessed to obtain the defect image to be generated, and the defect image to be generated is filled with holes to obtain the conditional image. Obtain defect category information and hard mask, and construct defect semantic prompts based on the defect category information; The conditional image and the defect semantic prompts are input into the target defect generation model for sampling processing to obtain the sampling results; The sampling results and the defect image to be generated are combined with the hard mask, and linearly blended in the area outside the hard mask according to a preset feathering weight to obtain the composite defect image.

7. The method for locally generating industrial defects according to claim 6, characterized in that, The process of preprocessing the original input image to obtain the defect image to be generated, and then performing hole-filling processing on the defect image to obtain a conditional image, includes: The original input image is scaled proportionally based on a preset longest side constraint rule to obtain a scaled image. The scaled image is converted into a standard color array, and the standard color array is aligned based on the preset variational autoencoder size requirements to obtain the defect image to be generated. The defect image to be generated and the hard mask are filled with holes based on a preset hole-filling strategy to obtain the conditional image.

8. An industrial defect localization generation device, characterized in that, include: Hard mask enhancement module: used to acquire pre-stored industrial defect samples, the industrial defect samples including seed hard masks, and perform hard mask enhancement processing on the seed hard masks to obtain binary hard masks. Downsampling module: used to downsample the binary hard mask to obtain a latent space hard mask; Fill module: used to determine the unmasked area based on the latent space hard mask, and fill the latent space hard mask based on the unmasked area to obtain conditional latent variables; Packaging module: used to package the conditional latent variables and pre-constructed noise latent variables according to preset packaging rules, and set conditional channel identifiers to obtain contextual conditional data; Training module: used to input the contextual condition data into the pre-constructed initial defect generation model for multiple rounds of iterative training to update the low-rank adaptation weights of the initial defect generation model and obtain the target defect generation model; Defect generation module: used to acquire the defect image to be generated, and perform local defect generation processing on the defect image to be generated based on the target defect generation model to obtain a composite defect image.

9. An industrial defect localization generation device, characterized in that, The industrial defect localization generation device includes: a memory and at least one processor, wherein the memory stores instructions; At least one of the processors invokes the instructions in the memory to cause the industrial defect localization generation device to perform the steps of the industrial defect localization generation method as claimed in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the steps of the industrial defect localization method as described in any one of claims 1-7.

Images