A grey fabric defect generation method, system, medium and computer

Abstract

The present application relates to a kind of grey defect generation method, system, medium and computer, the present application proposes a kind of condition diffusion model that can be controlled by text information (prompt word) and picture information specific grey specific defect type generation.This kind of technology based on AI GC generates grey defect, compared with traditional defect generation method, its defect form is rich, various, realistic can effectively improve the detection ability of detection model, effectively solve the problem that there is lack of defect sample and defect sample acquisition is more difficult in the existing grey defect detection field, compared with manually making grey defect, the generated grey defect is more natural and real, can effectively improve the generalization migration ability of defect detection model.

Description

Technical Field

[0001] This invention relates to the field of textile defect detection technology, and more specifically, to a method, system, medium, and computer for generating defects in greige fabric. Background Technology

[0002] Deep neural networks, with their ability to automatically extract image features, are gradually replacing traditional manual feature extraction methods in computer vision fields such as image recognition and object detection. In industrial quality inspection, the difficulty in collecting defect samples has led to an increasing demand for the automatic generation of such samples. Fabric defect detection is a crucial task in the textile industry. With the advancement of the national strategy of digital industrial upgrading, automated fabric defect detection is replacing manual inspection, and deep learning has become an important method for industrial visual defect detection. However, deep learning requires a large number of defect samples. In the absence of sufficient sample data, how to quickly collect or generate defect samples remains a significant obstacle to fabric defect detection.

[0003] Supervised training of deep learning models relies on manually labeled data. However, the types of defects in raw fabrics are numerous and varied in actual production scenarios, with some defects occurring very infrequently, making defect data collection quite difficult. Furthermore, for pre-trained models, facing the ever-changing new fabric types, requiring defect data collection before model retraining for each new fabric type would significantly hinder practical product application. Therefore, the technology of creating defects using software or models has become a crucial step in the implementation process.

[0004] Current traditional data synthesis methods for detecting defects in greige fabrics mainly involve synthesizing new defect images from existing greige fabric images through image fusion techniques or color and geometric transformations. These synthesis schemes either leave obvious synthetic traces or the synthesized defect morphologies are too similar to those already present in the dataset. These issues severely limit the improvement of the model's transferability and generalization capabilities offered by the synthesized defect images. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a method, system, medium and computer for generating fabric defects, so as to overcome the shortcomings of the existing technology in the process of generating fabric defect samples, which are easy to leave obvious synthesis traces and have poor optimization and improvement of model transfer and generalization.

[0006] The above-mentioned technical objective of the present invention is achieved through the following technical solution: a method for generating defects in raw fabric, comprising:

[0007] S1. Obtain several photos of the first raw fabric, create a first prompt word for each photo of the first raw fabric, and obtain several image-text pairs that correspond one-to-one with the photos of the first raw fabric. Record the several image-text pairs as the full dataset.

[0008] S2. Randomly select a predetermined number of image-text pairs with defects from the full dataset, and denot them as defect image-text pairs; add a mask image to the defect location of the first fabric photo of each defect image-text pair to obtain the corresponding mask photo that does not contain defects.

[0009] S3. Create a second prompt word for each of the masked photos to obtain several masked image-text pairs that correspond one-to-one with the masked photos. Record the several masked image-text pairs as a conditional control dataset.

[0010] S4. Input several full datasets into a pre-established stable diffusion model for training to obtain a basic diffusion model; input the conditional control dataset into the basic diffusion model to adjust the basic diffusion model and obtain the corresponding conditional diffusion model.

[0011] S5. Obtain a third prompt word to describe the defect of the greige fabric and a second greige fabric photo that does not contain the defect. Input the third prompt word and the second greige fabric photo into the conditional diffusion model to obtain the corresponding greige fabric image containing the defect.

[0012] Optionally, the step of creating a first cue word for each first greige fabric photograph includes:

[0013] The first prompt word contains several category information tags, which include: fabric type, fabric structure, key raw material information, warp yarn count, weft yarn count, and defect type of the fabric.

[0014] Optionally, the step of creating a first prompt for each first greige fabric photograph further includes:

[0015] Retrieve all subcategories contained in each category information label, and create a token ID for each subcategory.

[0016] Optionally, the step of inputting several of the full datasets into a pre-established stable diffusion model for training to obtain a basic diffusion model further includes:

[0017] All the first fabric photos are input into the stable diffusion model for training. The encoder and decoder of the VAE layer in the stable diffusion model are disabled. The VAE layer is only used for compressing and decompressing image information.

[0018] All the first prompt words are input into the stable diffusion model for training, and the MES function output by the stable diffusion model is obtained;

[0019] The MES function is used to constrain the text encoding layer and the Unet layer to obtain the corresponding basic diffusion model.

[0020] Optionally, the step of inputting the conditional control dataset into the basic diffusion model and adjusting the basic diffusion model to obtain the corresponding conditional diffusion model includes:

[0021] Enable the encoder and decoder of the VAE layer in the base diffusion model; freeze the trainable parameters of the Unet layer in the base diffusion model.

[0022] All the masked photos are input into the basic diffusion model, and the first image parameters output by the encoder of the VAE layer in the basic diffusion model are obtained;

[0023] Input all the first fabric photos corresponding to the mask photos into the basic diffusion model, and obtain the second image parameters output by the encoder of the VAE layer in the basic diffusion model;

[0024] The first image parameters are input into a preset ControlNet layer for processing to obtain the third image parameters;

[0025] The second image parameters are input into the Unet layer of the basic diffusion model for processing to obtain the fourth image parameters;

[0026] Calculate the mean square error between the noise of the third image parameter and the noise of the fourth image parameter, and use the mean square error as the loss function;

[0027] The loss function is used to constrain the ControlNet layer to obtain the conditional diffusion model.

[0028] Optionally, the full dataset contains more than 1 million image-text pairs;

[0029] The predetermined number of defective image-text pairs are randomly selected from the full dataset, and the predetermined number is greater than 100,000.

[0030] Optionally, the step of using the MES function to constrain the text encoding layer and the Unet layer to obtain the corresponding basic diffusion model includes:

[0031] The learning rate of the stable diffusion model is between 10. -6 Up to 10 -5 The stable diffusion model has more than 50 training rounds.

[0032] A system for generating defects in raw fabric, comprising:

[0033] Image-text pair generation module: used to acquire several first fabric photos, create a first prompt word for each first fabric photo, obtain several image-text pairs that correspond one-to-one with the first fabric photos, and record the several image-text pairs as the full dataset.

[0034] Masked photo generation module: used to randomly extract a predetermined number of image-text pairs with defects from the full dataset, denoted as defect image-text pairs; add a mask image to the defect location of the first fabric photo of each defect image-text pair to obtain the corresponding masked photo without defects.

[0035] Mask image-text pair generation module: used to generate a second prompt word for each of the masked photos, to obtain several mask image-text pairs that correspond one-to-one with several masked photos, and to record several mask image-text pairs as a condition control dataset;

[0036] Conditional diffusion model generation module: used to input several full datasets into a pre-established stable diffusion model for training to obtain a basic diffusion model; input the conditional control dataset into the basic diffusion model to adjust the basic diffusion model and obtain the corresponding conditional diffusion model;

[0037] Defect image synthesis module: used to obtain a third prompt word describing the defect of the greige fabric and a second greige fabric photo that does not contain the defect. The third prompt word and the second greige fabric photo are both input into the conditional diffusion model to obtain the corresponding greige fabric image containing the defect.

[0038] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0039] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.

[0040] In summary, this invention offers the following advantages: This application proposes a conditional diffusion model that allows for conditional control of the generation of specific defect types in a particular fabric using textual information (prompt words) and image information. This AIGC-based fabric defect generation technique, compared to traditional defect generation methods, provides richer, more diverse, and more realistic defect morphologies, effectively improving the detection capabilities of the detection model. It effectively solves the problems of insufficient defect samples and difficulty in collecting defect samples in the existing field of fabric defect detection. Compared to manually creating fabric defects, the generated defects are more natural and realistic, effectively improving the generalization and transfer capabilities of the defect detection model. Attached Figure Description

[0041] Figure 1 This is a flowchart of a method for generating defects in raw fabric according to the present invention;

[0042] Figure 2 This is a structural diagram of a fabric defect generation system according to the present invention;

[0043] Figure 3 This is an internal structural diagram of a computer device in an embodiment of the present invention.

[0044] In the diagram: 1. Image-text pair generation module; 2. Masked photo generation module; 3. Masked image-text pair generation module; 4. Conditional diffusion model generation module; 5. Defect image synthesis module. Detailed Implementation

[0045] To make the objectives, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Several embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein.

[0046] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature.

[0047] In this invention, unless otherwise expressly specified and limited, "above" or "below" a second feature can include direct contact between the first and second features, or contact between the first and second features through another feature between them. Furthermore, "above," "over," and "on top" of a second feature includes the first feature being directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" of a second feature includes the first feature being directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature. The terms "vertical," "horizontal," "left," "right," "above," "below," and similar expressions are for illustrative purposes only and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed or operated in a specific orientation, and therefore should not be construed as limiting the invention.

[0048] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0049] This invention provides a method for generating defects in raw fabric, such as... Figure 1 As shown, it includes:

[0050] S1. Obtain several photos of the first raw fabric, create a first prompt word for each photo of the first raw fabric, and obtain several image-text pairs that correspond one-to-one with the photos of the first raw fabric. Record the several image-text pairs as the full dataset.

[0051] S2. Randomly select a predetermined number of image-text pairs with defects from the full dataset, and denot them as defect image-text pairs; add a mask image to the defect location of the first fabric photo of each defect image-text pair to obtain the corresponding mask photo that does not contain defects.

[0052] S3. Create a second prompt word for each of the masked photos to obtain several masked image-text pairs that correspond one-to-one with the masked photos. Record the several masked image-text pairs as a conditional control dataset.

[0053] S4. Input several full datasets into a pre-established stable diffusion model for training to obtain a basic diffusion model; input the conditional control dataset into the basic diffusion model to adjust the basic diffusion model and obtain the corresponding conditional diffusion model.

[0054] S5. Obtain a third prompt word to describe the defect of the greige fabric and a second greige fabric photo that does not contain the defect. Input the third prompt word and the second greige fabric photo into the conditional diffusion model to obtain the corresponding greige fabric image containing the defect.

[0055] In practical applications, to quickly synthesize a large number of qualified photos of fabric with defects, the stable diffusion model first needs to be trained. To obtain a large number of training samples, some photos of fabric with or without defects need to be acquired first, denoted as the first fabric photo. To enable the diffusion model to recognize the feature information contained in the image, a cue word needs to be created for each fabric photo. Specifically, the cue word must include the fabric type, weave structure information, key raw material information, warp yarn denier (D), and weft yarn denier (D). If the image contains defects, the defect type information should also be included. For example, the cue word for a good product photo of jet-jet TR plain weave fabric could be: "Closeup detail of white fabric texture background, with a label of penqi, 1-1pingwen, TR, 133D, 133D". Here, "penqi" is the fabric type, "1-1pingwen" is the weave structure, "TR" is the raw material information, and the two "133D"s are the warp and weft yarn deniers, respectively. The denier (D) of the fabric is denier, and 1D means that 9000 meters of yarn weighs 1g. Using this rule, we eventually obtain a dataset of image-text pairs with a sample size of approximately one million, which is called the full dataset.

[0056] Then, 100,000 image-text pairs containing defects are extracted from the full dataset, and masks are added to the defect locations in these first fabric photos containing defects to mask the shape of the defects, resulting in corresponding masked photos without defects. Second prompt words are generated for these masked photos, and the second prompt words and masked photos together form masked image-text pairs.

[0057] First, the entire dataset is input into the open-source stable diffusion model for basic training. This enables the model to obtain the basic correspondence between text and image in image-text pairs, resulting in the trained basic diffusion model. In other words, the diffusion model can autonomously generate basic images of fabric based on the input prompts.

[0058] Then, using ControlNet technology, the basic diffusion model is fine-tuned to obtain the conditional diffusion model. This involves comparing the masked image with the corresponding image of the fabric containing the defect, allowing the diffusion model to learn the features corresponding to the location of the defect in the fabric. In practical applications, based on the input keywords used to describe the defect, the model can then generate the relevant defect on the fabric image.

[0059] After obtaining the conditional defect model, the conditional diffusion model can be used to synthesize fabric defects on the fabric photograph. Since in the actual production process, photos of the fabric without defects are usually taken. For some less common types of defects, it may be difficult to capture them. Therefore, a second fabric photograph without defects needs to be input into the model. Then, according to the type of fabric defect that people want, prompt words are used to describe the defect. In this way, fabric defects of a specific size and type can be generated at a specified location on the second fabric photograph.

[0060] In summary, this invention addresses the problems of insufficient defect samples and difficulties in defect sample collection in fabric defect detection scenarios. Based on defect image data in a reserve fabric type library, it employs ControlNet fine-tuning technology in the general image domain to train a conditional diffusion model that can conditionally control the generation of specific defect types in specific fabrics using text information (prompt words) and image information. Compared to traditional defect generation methods, this AIGC-based fabric defect generation technology offers richer, more diverse, and more realistic defect morphologies, effectively improving the detection capabilities of the detection model.

[0061] Furthermore, the creation of a first prompt for each first greige fabric photograph includes:

[0062] The first prompt word contains several category information tags, which include: fabric type, fabric structure, key raw material information, warp yarn count, weft yarn count, and defect type of the fabric.

[0063] Furthermore, the step of creating a first prompt for each first greige fabric photograph also includes:

[0064] Retrieve all subcategories contained in each category information label, and create a token ID for each subcategory.

[0065] In practical applications, each category information label for greige fabric contains several subcategories. For example, the category information label "fabric weave structure" includes subcategories such as: satin weave, twill weave, plain weave, composite twill weave, curved twill weave, herringbone twill weave, shaded twill weave, and corduroy fabric weave. A token ID is created for each subcategory, allowing the corresponding token ID to represent that subcategory.

[0066] Furthermore, the step of inputting several of the full datasets into a pre-established stable diffusion model for training to obtain a basic diffusion model also includes:

[0067] All the first fabric photos are input into the stable diffusion model for training. The encoder and decoder of the VAE layer in the stable diffusion model are disabled. The VAE layer is only used for compressing and decompressing image information.

[0068] All the first prompt words are input into the stable diffusion model for training, and the MES function output by the stable diffusion model is obtained;

[0069] The MES function is used to constrain the text encoding layer and the Unet layer to obtain the corresponding basic diffusion model.

[0070] In practical applications, training the stable diffusion model using the full dataset involves training the text encoding layer and the Unet layer in the model, enabling the stable diffusion model to correlate the meaning of the prompt words with the information in the fabric image.

[0071] Furthermore, the step of inputting the conditional control dataset into the basic diffusion model and adjusting the basic diffusion model to obtain the corresponding conditional diffusion model includes:

[0072] Enable the encoder and decoder of the VAE layer in the base diffusion model; freeze the trainable parameters of the Unet layer in the base diffusion model.

[0073] All the masked photos are input into the basic diffusion model, and the first image parameters output by the encoder of the VAE layer in the basic diffusion model are obtained;

[0074] Input all the first fabric photos corresponding to the mask photos into the basic diffusion model, and obtain the second image parameters output by the encoder of the VAE layer in the basic diffusion model;

[0075] The first image parameters are input into a preset ControlNet layer for processing to obtain the third image parameters;

[0076] The second image parameters are input into the Unet layer of the basic diffusion model for processing to obtain the fourth image parameters;

[0077] Calculate the mean square error between the noise of the third image parameter and the noise of the fourth image parameter, and use the mean square error as the loss function;

[0078] The loss function is used to constrain the ControlNet layer to obtain the conditional diffusion model.

[0079] In practical applications, the encoder of the VAE layer converts the image into low-level image information, and the decoder converts the low-level image information into a specific image according to predetermined rules. Therefore, in the actual process of generating a photo of fabric containing defects, the low-level image information containing defect information is directly generated based on the input prompt words. Then, the second input fabric photo is encoded into low-level image information. The two low-level image information are fused together and then decoded to generate a photo of fabric containing defects. Since the diffusion model is also a generative model, the intuition behind the diffusion model comes from physics. Therefore, noise is introduced, and then an attempt is made to generate an image through denoising. The working principle of the diffusion model is to learn the information attenuation caused by noise and then use the learned pattern to generate an image. Therefore, in the actual training process, the masked part on the masked image can be regarded as noise on the fabric photo. Then, based on the keywords, the noise is eliminated at the corresponding positions, and the defect is regenerated. Since the generation of defects needs to be constrained by the original image, the noise recognition is constrained by a loss function. ControlNet can constrain the regenerated defects based on the original image, so that the fine-tuned diffusion model is equipped with the constraints of ControlNet, thus obtaining the conditional diffusion model.

[0080] Furthermore, the full dataset contains more than 1 million image-text pairs;

[0081] The predetermined number of defective image-text pairs are randomly selected from the full dataset, and the predetermined number is greater than 100,000.

[0082] Furthermore, the step of using the MES function to constrain the text encoding layer and the Unet layer to obtain the corresponding basic diffusion model includes:

[0083] The learning rate of the stable diffusion model is between 10. -6 Up to 10 -5 The stable diffusion model has more than 50 training rounds.

[0084] In summary, this conditional diffusion model can effectively meet the needs for generating defects in grey fabrics. It can synthesize defect images of characteristic defect types for specific fabric types and regions. Furthermore, by using fabric type label information and defect type from prompts, as well as images with location mask information, as control conditions, the model can synthesize defect images of already collected fabric types or new fabric types, even when only good quality fabric is available.

[0085] like Figure 2 As shown, the present invention also provides a fabric defect generation system comprising:

[0086] Image-text pair generation module: used to acquire several first fabric photos, create a first prompt word for each first fabric photo, obtain several image-text pairs that correspond one-to-one with the first fabric photos, and record the several image-text pairs as the full dataset.

[0087] Masked photo generation module: used to randomly extract a predetermined number of image-text pairs with defects from the full dataset, denoted as defect image-text pairs; add a mask image to the defect location of the first fabric photo of each defect image-text pair to obtain the corresponding masked photo without defects.

[0088] Mask image-text pair generation module: used to generate a second prompt word for each of the masked photos, to obtain several mask image-text pairs that correspond one-to-one with several masked photos, and to record several mask image-text pairs as a condition control dataset;

[0089] Conditional diffusion model generation module: used to input several full datasets into a pre-established stable diffusion model for training to obtain a basic diffusion model; input the conditional control dataset into the basic diffusion model to adjust the basic diffusion model and obtain the corresponding conditional diffusion model;

[0090] Defect image synthesis module: used to obtain a third prompt word describing the defect of the greige fabric and a second greige fabric photo that does not contain the defect. The third prompt word and the second greige fabric photo are both input into the conditional diffusion model to obtain the corresponding greige fabric image containing the defect.

[0091] For specific limitations regarding a fabric defect generation system, please refer to the limitations of a fabric defect generation method described above, which will not be repeated here. Each module in the aforementioned fabric defect generation system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0092] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and the database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. When the computer program is executed by the processor, it implements a method for generating defects in fabric.

[0093] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0094] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps: including:

[0095] S1. Obtain several photos of the first raw fabric, create a first prompt word for each photo of the first raw fabric, and obtain several image-text pairs that correspond one-to-one with the photos of the first raw fabric. Record the several image-text pairs as the full dataset.

[0096] S2. Randomly select a predetermined number of image-text pairs with defects from the full dataset, and denot them as defect image-text pairs; add a mask image to the defect location of the first fabric photo of each defect image-text pair to obtain the corresponding mask photo that does not contain defects.

[0097] S3. Create a second prompt word for each of the masked photos to obtain several masked image-text pairs that correspond one-to-one with the masked photos. Record the several masked image-text pairs as a conditional control dataset.

[0098] S4. Input several full datasets into a pre-established stable diffusion model for training to obtain a basic diffusion model; input the conditional control dataset into the basic diffusion model to adjust the basic diffusion model and obtain the corresponding conditional diffusion model.

[0099] S5. Obtain a third prompt word to describe the defect of the greige fabric and a second greige fabric photo that does not contain the defect. Input the third prompt word and the second greige fabric photo into the conditional diffusion model to obtain the corresponding greige fabric image containing the defect.

[0100] In one embodiment, creating a first cue word for each first greige fabric photograph includes:

[0101] The first prompt word contains several category information tags, which include: fabric type, fabric structure, key raw material information, warp yarn count, weft yarn count, and defect type of the fabric.

[0102] In one embodiment, creating a first cue word for each first greige fabric photograph further includes:

[0103] Retrieve all subcategories contained in each category information label, and create a token ID for each subcategory.

[0104] In one embodiment, the step of inputting several full datasets into a pre-established stable diffusion model for training to obtain a basic diffusion model further includes:

[0105] All the first fabric photos are input into the stable diffusion model for training. The encoder and decoder of the VAE layer in the stable diffusion model are disabled. The VAE layer is only used for compressing and decompressing image information.

[0106] All the first prompt words are input into the stable diffusion model for training, and the MES function output by the stable diffusion model is obtained;

[0107] The MES function is used to constrain the text encoding layer and the Unet layer to obtain the corresponding basic diffusion model.

[0108] In one embodiment, the step of inputting the conditional control dataset into the basic diffusion model and adjusting the basic diffusion model to obtain the corresponding conditional diffusion model includes:

[0109] Enable the encoder and decoder of the VAE layer in the base diffusion model; freeze the trainable parameters of the Unet layer in the base diffusion model.

[0110] All the masked photos are input into the basic diffusion model, and the first image parameters output by the encoder of the VAE layer in the basic diffusion model are obtained;

[0111] Input all the first fabric photos corresponding to the mask photos into the basic diffusion model, and obtain the second image parameters output by the encoder of the VAE layer in the basic diffusion model;

[0112] The first image parameters are input into a preset ControlNet layer for processing to obtain the third image parameters;

[0113] The second image parameters are input into the Unet layer of the basic diffusion model for processing to obtain the fourth image parameters;

[0114] Calculate the mean square error between the noise of the third image parameter and the noise of the fourth image parameter, and use the mean square error as the loss function;

[0115] The loss function is used to constrain the ControlNet layer to obtain the conditional diffusion model.

[0116] In one embodiment, the full dataset contains more than 1 million image-text pairs;

[0117] The predetermined number of defective image-text pairs are randomly selected from the full dataset, and the predetermined number is greater than 100,000.

[0118] In one embodiment, constraining the text encoding layer and the Unet layer using the MES function to obtain the corresponding basic diffusion model includes:

[0119] The learning rate of the stable diffusion model is between 10. -6 Up to 10 -5 The stable diffusion model has more than 50 training rounds.

[0120] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0121] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0122] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for generating defects in grey fabric, characterized in that, include: S1. Obtain several photos of the first raw fabric, create a first prompt word for each photo of the first raw fabric, and obtain several image-text pairs that correspond one-to-one with the photos of the first raw fabric. Record the several image-text pairs as the full dataset. S2. Randomly select a predetermined number of image-text pairs with defects from the full dataset, denoted as defect image-text pairs; add a mask image to the defect location of the first raw fabric photo of each defect image-text pair to obtain the corresponding mask photo that does not contain defects. S3. Create a second prompt word for each of the masked photos to obtain several masked image-text pairs that correspond one-to-one with the masked photos. Record the several masked image-text pairs as a conditional control dataset. S4. Input several full datasets into a pre-established stable diffusion model for training to obtain a basic diffusion model; input the conditional control dataset into the basic diffusion model to adjust the basic diffusion model and obtain the corresponding conditional diffusion model. S5. Obtain a third prompt word to describe the defect of the fabric and a second fabric photo that does not contain the defect. Input the third prompt word and the second fabric photo into the conditional diffusion model to obtain the corresponding fabric image containing the defect. The process of creating a first prompt for each first greige fabric photograph includes: The first prompt word contains several category information tags, which include: fabric type of greige fabric, fabric structure of greige fabric, key raw material information of greige fabric, warp yarn count of greige fabric, weft yarn count of greige fabric, and defect type of greige fabric. The step of inputting several full datasets into a pre-established stable diffusion model for training to obtain a basic diffusion model also includes: All the first fabric photos are input into the stable diffusion model for training. The encoder and decoder of the VAE layer in the stable diffusion model are disabled. The VAE layer is only used for compressing and decompressing image information. All the first prompt words are input into the stable diffusion model for training, and the MES function output by the stable diffusion model is obtained; The MES function is used to constrain the text encoding layer and the Unet layer to obtain the corresponding basic diffusion model; The step of inputting the conditional control dataset into the basic diffusion model and adjusting the basic diffusion model to obtain the corresponding conditional diffusion model includes: Enable the encoder and decoder of the VAE layer in the base diffusion model; freeze the trainable parameters of the Unet layer in the base diffusion model. All the masked photos are input into the basic diffusion model, and the first image parameters output by the encoder of the VAE layer in the basic diffusion model are obtained; Input all the first fabric photos corresponding to the mask photos into the basic diffusion model, and obtain the second image parameters output by the encoder of the VAE layer in the basic diffusion model; The first image parameters are input into a preset ControlNet layer for processing to obtain the third image parameters; The second image parameters are input into the Unet layer of the basic diffusion model for processing to obtain the fourth image parameters; Calculate the mean square error between the noise of the third image parameter and the noise of the fourth image parameter, and use the mean square error as the loss function; The loss function is used to constrain the ControlNet layer to obtain the conditional diffusion model.

2. The method for generating defects in grey fabric according to claim 1, characterized in that, The process of creating a first prompt for each first greige fabric photograph also includes: Retrieve all subcategories contained in each category information label, and create a tokenID for each subcategory.

3. The method for generating defects in grey fabric according to claim 1, characterized in that, The full dataset contains more than 1 million image-text pairs; The predetermined number of defective image-text pairs are randomly selected from the full dataset, and the predetermined number is greater than 100,000.

4. The method for generating defects in grey fabric according to claim 1, characterized in that, The step of using the MES function to constrain the text encoding layer and the Unet layer to obtain the corresponding basic diffusion model includes: The learning rate of the stable diffusion model is between 10. -6 Up to 10 -5 The stable diffusion model has more than 50 training rounds.

5. A fabric defect generation system based on the method of claim 1, characterized in that, include: Image-text pair generation module: used to acquire several first fabric photos, create a first prompt word for each first fabric photo, obtain several image-text pairs that correspond one-to-one with the first fabric photos, and record the several image-text pairs as the full dataset. Masked photo generation module: used to randomly extract a predetermined number of image-text pairs with defects from the full dataset, denoted as defect image-text pairs; add a mask image to the defect location of the first fabric photo of each defect image-text pair to obtain the corresponding masked photo without defects. Mask image-text pair generation module: used to generate a second prompt word for each of the masked photos, to obtain several mask image-text pairs that correspond one-to-one with several masked photos, and to record several mask image-text pairs as a condition control dataset; Conditional diffusion model generation module: used to input several full datasets into a pre-established stable diffusion model for training to obtain a basic diffusion model; input the conditional control dataset into the basic diffusion model to adjust the basic diffusion model and obtain the corresponding conditional diffusion model; Defect image synthesis module: used to obtain a third prompt word describing the defect of the greige fabric and a second greige fabric photo that does not contain the defect. The third prompt word and the second greige fabric photo are both input into the conditional diffusion model to obtain the corresponding greige fabric image containing the defect.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

Images