A method and system for generating a chiseled surface defect image sample

Abstract

The application discloses a kind of engraved surface defect image sample generation method and system, it is related to artificial intelligence and data processing field.The method comprises: the engraved surface defect image real sample associated with first corpus parameter and first defect description text are combined as training set;Defect image sample generation model is trained using the training set;Mask chart of the obtained engraved surface normal image real sample is extracted;Using the defect image sample generation model trained, based on the engraved surface normal image real sample, the mask chart and second defect description text, engraved surface defect image sample is generated.Compared with prior art, the application does not need a large number of engraved surface defect image real sample, the defect image sample generation model trained is strong in generalization, and the engraved surface defect image sample generated is high in authenticity, which is beneficial to reduce the training difficulty of neural network model for defect detection and improve its accuracy.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and data processing technology, and more specifically, to a method and system for generating image samples of faceted defects. Background Technology

[0002] The etched surfaces on the engine serve to indicate information such as the engine manufacturer, specifications, performance, features, manufacturing process, application, and batch number. They also facilitate retrieval of lost engines and prevent unauthorized modification. Figure 1 As shown, the national standard GB 7258 clearly stipulates that vehicles must have their engine model and serial number engraved on the engine. If there are defects in the engraved surface, the vehicle may not be able to be registered normally. Therefore, defects in the engraved surface need to be detected during the engine manufacturing process. When using a deep learning-based model to detect engine defects, the detection model needs to be trained. However, the training process faces the problem of a small number of defect samples, which can lead to overfitting in the trained detection model.

[0003] The existing main technical solutions for expanding defective image samples mainly involve data augmentation methods such as flipping (horizontal or vertical), random cropping, random rotation, and scaling, as well as adding random noise (such as Gaussian noise and salt-and-pepper noise) or perturbing input features (such as random occlusion, changing color, brightness, contrast, etc.) to enhance the diversity of defective image sample datasets.

[0004] However, among the methods mentioned above, data augmentation methods do not take into account the characteristics and distribution of the dataset, and require a large number of original real defect samples, thus having limited improvement on model training; random noise addition methods are difficult to control the noise quality during the noise addition process, and may lead to errors in training labels, making them unsuitable for scenarios requiring fine reasoning, such as faceted defect recognition; while feature perturbation methods involve the modification or transformation of the original data, and can only operate on existing features, unable to introduce completely new features. In addition, excessively strong perturbations can cause the trained model to pay too much attention to these perturbations, thus making it impossible for the trained model to accurately generalize to new data. Summary of the Invention

[0005] To overcome the limitation of limited training data for engraved surface defect detection in the prior art, the present invention provides a method and system for generating engraved surface defect image samples.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0007] Firstly, a method for generating faceted defect image samples includes:

[0008] The real samples of faceted defect images associated with the first corpus parameters are combined with the first defect description text to form a training set; wherein, the first defect description text is a text prompt used to characterize the defect type of the real samples of faceted defect images.

[0009] A defect image sample generation model is trained using the training set; wherein the defect image sample generation model is configured with the first corpus parameters, and the first corpus parameters are used to instruct the defect image sample generation model to determine the defect type pointed to by the first defect description text;

[0010] Extract the mask image of the real sample of the obtained normal image of the etched surface;

[0011] Using the trained defect image sample generation model, based on the real sample of the normal image of the engraved surface, the mask image, and the second defect description text, an engraved surface defect image sample is generated; wherein, the second defect description text refers to a text prompt used to characterize the defect type of the engraved surface defect image sample.

[0012] Secondly, a system for generating faceted defect image samples includes:

[0013] The data acquisition module is used to acquire real samples of faceted defect images, real samples of faceted normal images, first corpus parameters, first defect description text, and second defect description text; wherein, the first defect description text is a text prompt used to characterize the defect type of the real sample of faceted defect image, and the second defect description text is a text prompt used to characterize the defect type of the generated faceted defect image sample.

[0014] The training module is used to associate the first corpus parameters with the real samples of the faceted defect images and combine them with the first defect description text to form a training set; it is also used to train a defect image sample generation model using the training set; wherein the defect image sample generation model is configured with the first corpus parameters, which are used to instruct the defect image sample generation model to determine the defect type pointed to by the first defect description text.

[0015] The image generation module is used to extract a mask image of a real sample of a normal image of a punched surface; it is also used to load the trained defect image sample generation model and generate the punched surface defect image sample based on the real sample of the normal image of a punched surface, the mask image and the second defect description text.

[0016] Thirdly, an electronic device includes:

[0017] Memory is used to store executable instructions or computer programs.

[0018] A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the method described in the first aspect.

[0019] Fourthly, a computer program product includes a computer program or computer-executable instructions, which, when executed by a processor, implement the method described in the first aspect.

[0020] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0021] This invention discloses a method for generating defect image samples. A defect image sample generation model is trained using a constructed training set, such that specific first corpus parameters in the model are associated with real samples of faceted defect images. In subsequent model usage, the trained defect image sample generation model, based on real samples of normal faceted images, corresponding mask images, and second defect description text, can generate a large number of diverse faceted defect image samples, exhibiting good generalization performance. Compared to existing technologies, this invention does not require a large number of real faceted defect image samples, the trained defect image sample generation model has strong generalization ability, and the generated faceted defect image samples have high realism. This helps reduce the training difficulty of neural network models used for defect detection and improves their accuracy. Attached Figure Description

[0022] Figure 1 A schematic diagram of a faceted engine.

[0023] Figure 2 This is a flowchart illustrating the method for generating surface defect image samples in Embodiment 1 of this application;

[0024] Figure 3 This is another flowchart illustrating the method for generating surface defect image samples in Embodiment 1 of this application;

[0025] Figure 4 This is a schematic diagram of the engraved character area in Embodiment 1 of this application;

[0026] Figure 5 This is an example diagram of the character region mask in Embodiment 1 of this application;

[0027] Figure 6 This is an example diagram of the non-character region mask in Embodiment 1 of this application;

[0028] Figure 7 This is an example diagram of the character outline mask in Embodiment 1 of this application;

[0029] Figure 8 This is a schematic diagram of the diffusion generation process based on a non-character region mask map in Embodiment 1 of this application;

[0030] Figure 9 This is a schematic diagram of the diffusion generation process based on character region mask map in Embodiment 1 of this application;

[0031] Figure 10 This is an example image of the defect region in the engraved surface defect image sample generated in Embodiment 1 of this application;

[0032] Figure 11 This is an example image of another defect region in the engraved surface defect image sample generated in Embodiment 1 of this application;

[0033] Figure 12 This is an example image of a defect area with a defect type of scratch in Embodiment 1 of this application;

[0034] Figure 13 This is another example image of a defect area with a scratch type in Embodiment 1 of this application;

[0035] Figure 14 This is a schematic diagram of the structure of the surface defect image sample generation system in Embodiment 2 of this application. Detailed Implementation

[0036] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0037] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0038] To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions;

[0039] It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0040] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0041] Example 1

[0042] This embodiment provides a method for generating image samples of faceted defects. (See attached document.) Figure 2 ,include:

[0043] The real samples of faceted defect images associated with the first corpus parameters are combined with the first defect description text to form a training set; wherein, the first defect description text is a text prompt used to characterize the defect type of the real samples of faceted defect images.

[0044] A defect image sample generation model is trained using the training set; wherein the defect image sample generation model is configured with a first corpus parameter, the first corpus parameter being used to instruct the defect image sample generation model to determine the defect type pointed to by the first defect description text;

[0045] Extract the mask image of the real sample of the obtained normal image of the etched surface;

[0046] Using the trained defect image sample generation model, based on the real sample of the normal image of the engraved surface, the mask image, and the second defect description text, an engraved surface defect image sample is generated; wherein, the second defect description text refers to a text prompt used to characterize the defect type of the engraved surface defect image sample.

[0047] The method described in this embodiment trains a defect image sample generation model using a training set constructed based on real samples of faceted defect images and a first defect description text. First corpus parameters are configured for the defect image sample generation model, associating specific first corpus parameters in the model with real samples of faceted defect images. This allows the model to generate faceted defect image samples of the defect type of interest based on the second defect description text, resulting in a model with strong generalization ability. Furthermore, the extracted mask image can improve the realism of the generated faceted defect image samples. Compared to existing technologies, the method described in this embodiment can generate a large number of faceted defect image samples without collecting a large number of real faceted defect image samples, thus reducing the difficulty of constructing the training set for the detection model and improving the accuracy of the detection model trained based on the generated faceted defect image samples.

[0048] In some examples, the configuration of the first corpus parameters for the defective image sample generation model is completed by adding the first corpus parameters to the tokenizer of the defective image sample generation model.

[0049] In some examples, the first corpus parameter can be a simple descriptive text (such as a phrase) of the defect type, and the first defect descriptive text can be a complex descriptive text paragraph that includes the first corpus parameter and / or its synonyms. Similarly, the second defect descriptive text can be a complex descriptive text paragraph that includes the first corpus parameter and / or its synonyms.

[0050] In some examples, the defect types include scratches, stains, peeling, etc.

[0051] It should be understood that the real sample of the etched surface defect image refers to the real sample of the etched surface of the engine with defects, and the real sample of the etched surface normal image refers to the real sample of the etched surface of the engine without defects.

[0052] In some specific implementation processes, refer to Figure 3 Industrial cameras were used to acquire images of multiple engine etched surfaces. 5-10 real samples of each defect type of etched surface were collected to form a etched surface defect sample library. 1000 real samples of normal etched surface images were collected to form a normal etched surface sample library.

[0053] In some preferred embodiments, the mask image includes a region mask image;

[0054] The extracted mask image of the real sample of the normal image of the engraved surface includes:

[0055] The character detection algorithm is used to identify the engraved character regions in the real sample of the normal image of the engraved surface. (See reference) Figure 4 ;

[0056] A region mask map of the real sample of the normal image of the engraved surface is randomly generated based on the engraved character region.

[0057] In some alternative embodiments, a diffusion model is used as the defect image sample generation model;

[0058] The generation of the faceted defect image sample includes:

[0059] The real sample of the normal image of the etched surface is input into the defect image sample generation model, and the real sample of the normal image of the etched surface is encoded to obtain the latent encoded image.

[0060] The region mask is downsampled to make its shape consistent with the latent coded image;

[0061] Based on the downsampled region mask, the location for diffusion generation on the latent coded image is determined, and forward diffusion is performed by iteratively adding noise to this location to obtain a noisy image.

[0062] Using the second defect description text, the noisy image is back-diffused to obtain a denoised image;

[0063] The denoised image is decoded to obtain the faceted defect image sample, which is then used as the output of the defect image sample generation model.

[0064] In one specific implementation, the denoised image obtained after 50 iterations is subjected to VAE decoding to output the corresponding facet defect image sample.

[0065] Furthermore, the region mask image includes a character region mask image (corresponding to defects in the character region, see [reference]). Figure 5 ) and non-character region mask (the corresponding defect is in the non-character region, see Figure 6 The mask image also includes a character outline mask image (see...). Figure 7 );

[0066] After obtaining the denoised image through backdiffusion, the method further includes:

[0067] Based on the region mask used to determine the diffusion generation location, it is determined whether to perform character region feature reconstruction on the denoised image; wherein, when the region mask is a character region mask, the character region feature is reconstructed on the denoised image using the character contour mask, and then the denoised image after character region feature reconstruction is subjected to backdiffusion or decoding; when the region mask is a non-character region mask, character region feature reconstruction is not performed on the denoised image.

[0068] It should be noted that when generating images by diffusion within the character region, the original engraved characters are destroyed, affecting the realism of the final defect image sample. In the above embodiment, the character parts that do not need to be reconstructed in the latent feature image (i.e., the denoised image) are replaced with the original features by extracting the character contour mask image. This allows for the diffusion generation of defects within the character region while ensuring that the engraved characters are consistent with the original sample, thereby improving the realism of the generated engraved surface defect image sample.

[0069] It should be understood that by obtaining the coordinate information of the character region and the coordinate information of the non-character region, a corresponding region mask map can be established.

[0070] In a specific implementation, in the defect image sample generation model, for diffusion generation using a non-character region mask image, see [reference needed]. Figure 8 The process includes:

[0071] 1) Perform VAE encoding on real samples of normal faceted images to obtain the latent encoded image Z. init ;

[0072] 2) Downsample the mask image of the non-character region corresponding to the location where the defect needs to be generated, ensuring that its shape is consistent with the Z-axis. init Consistency;

[0073] 3) Determine the position in Z using the downsampled non-character region mask.init The location where image diffusion needs to be generated, and noise needs to be added to that location;

[0074] 4) Call the function containing parameters of the first corpus (e.g.) <block-huahen>The second descriptive text of the location is used to remove noise (Denoise);

[0075] 5) After 50 iterations, VAE decoding is performed to generate corresponding engraved surface defect image samples.

[0076] In another specific implementation, for diffusion generation using a character region mask image, please refer to [reference needed]. Figure 9 The process includes:

[0077] 1) Perform VAE encoding on real samples of normal faceted images to obtain the latent encoded image Z. init ;

[0078] 2) Downsample the mask image of the non-character region corresponding to the location where the defect needs to be generated, ensuring that its shape is consistent with the Z-axis. init Consistency;

[0079] 3) Use the downsampled character region mask to determine the position in Z. init The location where image diffusion needs to be generated, and noise needs to be added to that location;

[0080] 4) Call the function containing parameters of the first corpus (e.g.) <block-huahen>The second descriptive text of the text removes noise at that location;

[0081] 5) Use the character contour mask to replace the character parts in the Latent Feature that do not need to be reconstructed with the original features of the original image;

[0082] 6) After 50 iterations, VAE decoding is performed to generate corresponding engraved surface defect image samples.

[0083] As a non-limiting example, the defect region in the faceted defect image sample generated by diffusion using the aforementioned defect image sample generation model is as follows: Figure 10 , Figure 11 As shown.

[0084] Furthermore, the mask image of the extracted normal sample of the etched surface also includes:

[0085] The CEDN (Fully Convolutional Encoder-Decoder Network) model is used to obtain the outline of the engraved character in the real sample of the normal image of the engraved surface;

[0086] Generate a character outline mask of the engraved character based on the engraved character outline.

[0087] Furthermore, the diffusion model is a Stable Diffusion model;

[0088] During the training process of the defect image sample generation model, only the weight parameters of the text encoder in the Stable Diffusion model are trained.

[0089] In some examples, the Stable Diffusion V1.5 base model is used for training. During training, the parameters of the VAE (Variational Autoencoder) and UNet (U-shaped network) models in the base model are frozen, and only the weights of the token_embeddings of the CLIP text encoder are trained, which greatly reduces the training cost of the model. After training, the image inpainting scheme of the Stable Diffusion model can be used to generate faceted defect image samples.

[0090] Those skilled in the art should understand that the diffusion model can also be the GLIDE model, the DALLE2 model, etc.

[0091] In some preferred embodiments, the process of constructing the training set includes:

[0092] The corresponding defect region of the real sample of the engraved surface defect image is cropped into an independent defect region image, see reference. Figure 12 , Figure 13 The first corpus parameters are associated with the defect type of the defect region image, and the size of the defect region image is adjusted by interpolation.

[0093] Configure the first defect description text for the defect region image, and combine the text vector with the corresponding defect region image to form the training set.

[0094] In some specific implementation processes, defect area cropping and defect area image storage are performed based on defect type.

[0095] Those skilled in the art should understand that the above preferred embodiments can generate defect region images of any size, so that defects of the same type in the final generated faceted defect image sample can be of different sizes.

[0096] In some examples, bicubic interpolation is used to resize the defect region image.

[0097] In other examples, bilinear interpolation is used to resize the image of the defective region.

[0098] Example 2

[0099] This embodiment provides a system for generating image samples of faceted defects, used to implement the method in Embodiment 1. (See attached document.) Figure 14 ,include:

[0100] The data acquisition module is used to acquire real samples of faceted defect images, real samples of faceted normal images, first corpus parameters, first defect description text, and second defect description text; wherein, the first defect description text is a text prompt used to characterize the defect type of the real sample of faceted defect image, and the second defect description text is a text prompt used to characterize the defect type of the generated faceted defect image sample.

[0101] The training module is used to associate the first corpus parameters with the real samples of the faceted defect images and combine them with the first defect description text to form a training set; it is also used to train a defect image sample generation model using the training set; wherein the defect image sample generation model is configured with the first corpus parameters, which are used to instruct the defect image sample generation model to determine the defect type pointed to by the first defect description text.

[0102] The image generation module is used to extract a mask image of a real sample of a normal image of a punched surface; it is also used to load the trained defect image sample generation model and generate the punched surface defect image sample based on the real sample of the normal image of a punched surface, the mask image and the second defect description text.

[0103] It is understood that the apparatus of this embodiment corresponds to the method of embodiment 1 above, and the options in embodiment 1 above are also applicable to this embodiment, so they will not be described again here.

[0104] Example 3

[0105] This embodiment provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set. The at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor, causing the processor to perform some or all of the steps of the method provided in Embodiment 1 of this application.

[0106] It is understood that the storage medium can be transient or non-transient. Exemplarily, the storage medium includes, but is not limited to, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0107] By way of example, the processor may be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0108] In some examples, a computer program product is provided, which can be implemented by hardware, software, or a combination thereof. As a non-limiting example, the computer program product can be embodied in the storage medium, or it can be embodied in a software product, such as an SDK (Software Development Kit).

[0109] As a non-limiting example, a computer program product is provided, comprising a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer program or computer-executable instructions from the computer-readable storage medium, and executes the computer-executable instructions, causing the electronic device to perform some or all of the steps of the method described in the embodiments of this application.

[0110] In some examples, a computer program is provided, including computer-readable code, wherein, when the computer-readable code is run in a computer device, a processor in the computer device performs some or all of the steps for implementing the method.

[0111] This embodiment also proposes an electronic device, including a memory and a processor. The memory stores at least one instruction, at least one program, code set, or instruction set. When the processor executes the at least one instruction, at least one program, code set, or instruction set, it implements some or all of the steps of the method described in Embodiment 1.

[0112] In some examples, a hardware entity of the electronic device is provided, including: a processor, a memory, and a communication interface; wherein the processor typically controls the overall operation of the electronic device; the communication interface is used to enable the electronic device to communicate with other terminals or servers via a network; the memory is configured to store instructions and applications executable by the processor, and may also cache data to be processed or already processed (including but not limited to image data, audio data, voice communication data, and video communication data) to be processed by the processor and various modules in the electronic device, and may be implemented using flash memory or random access memory (RAM).

[0113] A processor may include one or more processing elements. Therefore, a processor may include one or more integrated circuits (ICs) configured to perform the functions of the processor. Furthermore, each integrated circuit may include circuitry (e.g., a first circuit, a second circuit, and other circuitry) configured to perform the functions of the processor.

[0114] Furthermore, data can be transferred between the processor, communication interface, and memory via a bus, which can include any number of interconnected buses and bridges, connecting various circuits of one or more processors and memories together.

[0115] It is understood that the options in Embodiment 1 above also apply to this embodiment, so they will not be described again here.

[0116] The same or similar labels correspond to the same or similar parts;

[0117] The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this application.

[0118] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0119] In different specific implementations, the methods or systems described in this application can be implemented in software, hardware, or a combination thereof. Furthermore, the order of the method steps can be changed, and various elements can be added, reordered, combined, omitted, or modified.

[0120] Obviously, the above embodiments of this application are merely examples for clearly illustrating this application, and are not intended to limit the implementation of this application, nor are they intended to limit this application. For those skilled in the art, other variations or modifications can be made based on the above description. The separate structural / functional modules or units can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. The structure and function of the separate components can be implemented as a combined structure or component. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of the claims of this application.

Claims

1. A method for generating a scuffed surface defect image sample, the method comprising: include: ​ The real samples of faceted defect images associated with the first corpus parameters are combined with the first defect description text to form a training set; wherein, the first defect description text is a text prompt used to characterize the defect type of the real samples of faceted defect images. A defect image sample generation model is trained using the training set; wherein the defect image sample generation model is configured with the first corpus parameters, and the first corpus parameters are used to instruct the defect image sample generation model to determine the defect type pointed to by the first defect description text; Extract the mask image of the real sample of the obtained normal image of the etched surface; Using the trained defect image sample generation model, based on the real sample of the normal image of the engraved surface, the mask image, and the second defect description text, an engraved surface defect image sample is generated; wherein, the second defect description text refers to a text prompt used to characterize the defect type of the engraved surface defect image sample. The mask image includes a region mask image; A diffusion model is used as the model for generating the defective image samples; The generation of the faceted defect image sample includes: The real sample of the normal image of the etched surface is input into the defect image sample generation model, and the real sample of the normal image of the etched surface is encoded to obtain the latent encoded image. The region mask image is downsampled to make its size consistent with the latent coded image; Based on the downsampled region mask image, the position for diffusion generation on the latent coded image is determined, and forward diffusion is performed by iteratively adding noise to this position to obtain a noisy image. Using the second defect description text, the noisy image is back-diffused to obtain a denoised image; The denoised image is decoded to obtain the faceted defect image sample, which is used as the output of the defect image sample generation model; The region mask image includes a character region mask image and a non-character region mask image; the mask image also includes a character outline mask image; After obtaining the denoised image through backdiffusion, the method further includes: Based on the region mask used to determine the diffusion generation location, it is determined whether to perform character region feature reconstruction on the denoised image; wherein, when the region mask is a character region mask, the character region feature is reconstructed on the denoised image using the character contour mask, and then the denoised image after character region feature reconstruction is subjected to backdiffusion or decoding; when the region mask is a non-character region mask, character region feature reconstruction is not performed on the denoised image.

2. The method of claim 1, wherein, The extracted mask image of the real sample of the normal image of the engraved surface includes: A character detection algorithm is used to identify the engraved character regions in the real sample of the normal image of the engraved surface; A region mask map of a normal image of the engraved surface is randomly generated based on the engraved character region.

3. The method for generating image samples of faceted defects according to claim 1, characterized in that, The mask image of the real sample of the normal image of the etched surface obtained by extraction also includes: The CEDN model was used to obtain the outline of the engraved character in the real sample of the normal image of the engraved surface; Generate a character outline mask of the engraved character based on the engraved character outline.

4. The method for generating image samples of faceted defects according to claim 1, characterized in that, The diffusion model is the Stable Diffusion model. During the training process of the defect image sample generation model, only the weight parameters of the text encoder in the Stable Diffusion model are trained.

5. A method for generating image samples of faceted defects according to any one of claims 1-4, characterized in that, The process of constructing the training set includes: The corresponding defect region of the real sample of the faceted defect image is cropped into an independent defect region image. The first corpus parameters are associated with the defect type of the defect region image, and the size of the defect region image is adjusted by interpolation. Configure the first defect description text for the defect region image, and combine the text vector with the corresponding defect region image to form the training set.

6. A system for generating image samples of faceted defects, characterized in that, include: The data acquisition module is used to acquire real samples of faceted defect images, real samples of faceted normal images, first corpus parameters, first defect description text, and second defect description text; wherein, the first defect description text is a text prompt used to characterize the defect type of the real sample of faceted defect image, and the second defect description text is a text prompt used to characterize the defect type of the generated faceted defect image sample. The training module is used to associate the first corpus parameters with the real samples of the faceted defect images and combine them with the first defect description text to form a training set; it is also used to train a defect image sample generation model using the training set; wherein the defect image sample generation model is configured with the first corpus parameters, which are used to instruct the defect image sample generation model to determine the defect type pointed to by the first defect description text. The image generation module is used to extract a mask image of a real sample of a normal image of a punched surface; it is also used to load the trained defect image sample generation model and generate the punched surface defect image sample based on the real sample of the normal image of a punched surface, the mask image and the second defect description text. The mask image includes a region mask image; A diffusion model is used as the model for generating the defective image samples; The generation of the faceted defect image sample includes: The real sample of the normal image of the etched surface is input into the defect image sample generation model, and the real sample of the normal image of the etched surface is encoded to obtain the latent encoded image. The region mask image is downsampled to make its size consistent with the latent coded image; Based on the downsampled region mask image, the position for diffusion generation on the latent coded image is determined, and forward diffusion is performed by iteratively adding noise to this position to obtain a noisy image. Using the second defect description text, the noisy image is back-diffused to obtain a denoised image; The denoised image is decoded to obtain the faceted defect image sample, which is used as the output of the defect image sample generation model; The region mask image includes a character region mask image and a non-character region mask image; the mask image also includes a character outline mask image; After obtaining the denoised image through backdiffusion, the system determines whether to perform character region feature reconstruction on the denoised image based on the region mask used to determine the diffusion generation location. Specifically, when the region mask is a character region mask, character region feature reconstruction is performed on the denoised image using the character contour mask, and then backdiffusion or decoding is performed on the reconstructed denoised image. When the region mask is a non-character region mask, character region feature reconstruction is not performed on the denoised image.

7. An electronic device, characterized in that, include: Memory is used to store executable instructions or computer programs. A processor, configured to execute computer-executable instructions or computer programs stored in the memory, implements the method of any one of claims 1-5.

8. A computer program product, comprising a computer program or computer-executable instructions, characterized in that, When the computer program or computer-executable instructions are executed by a processor, the method described in any one of claims 1-5 is implemented.

Images