Image generation system with LORA model combined with clothing modular design

Abstract

The invention discloses an image generation system combining an LORA model with clothing modular design. The image generation system comprises a processor, a user input module, an LORA model training module, a clothing and model automatic production module, a buffer library and a database, the LORA model training module is bidirectionally connected with the processor, and the LORA model training module comprises a feature extraction unit, a stylized LORA model training unit and a clothing image generation unit. According to the image generation system combining the LORA model with the clothing modular design, the LORA technology and the clothing modular design concept are deeply fused, a 1 + N LORA model concept is proposed, a clothing design AI workflow is reconstructed through one profile LORA and N clothing modular LORA, clothing styles meeting market requirements and fashion trends can be generated through LORA training and a Stable Diffusion model, and meanwhile, the LORA model is integrated with the clothing modular design concept, so that the image generation efficiency is improved. The garment is split into a plurality of modules through modular design for training of the LORA models, and a user can also superpose the plurality of LORA models.

Description

Technical Field

[0001] The present invention relates to the technical field of clothing design, and specifically to an image generation system combining the LORA model with clothing modular design. Background Art

[0002] Clothing ODM and OBM enterprises are urgently required to closely follow the rapidly changing fashion trends and meet the increasingly diverse and personalized needs of consumers. Consumers' requirements for clothing styles, materials, environmental protection, etc. are constantly increasing, which prompts enterprises to accelerate the product replacement speed, and the number of new models launched each year continues to increase.

[0003] Referring to the patent publication number "CN112699261B", it discloses a clothing image automatic generation system and method, including: a clothing image retrieval module, an image attribute automatic annotation module for performing attribute annotation on the collected clothing images, and a clothing image automatic generation module; the clothing image retrieval module includes: image search by text and image search by image, where the image for image search by image is the clothing image taken by the user and / or the image directly input for query; the image attribute automatic annotation module constructs a relevant image annotation library.

[0004] As shown in the above technology, in terms of pain points, the traditional design process is cumbersome and time-consuming, seriously restricting the market response speed of enterprises. According to industry research, in the traditional design mode, it takes an average of 3 - 6 months for a new clothing to be launched from the design concept to the final product on the market. During this period, designers need to invest a lot of time and energy in sketch drawing, modification and improvement, and due to human factors, it is difficult to accurately control the continuity of the design style. In terms of the strong desire to improve design efficiency, many enterprises have strong demands. In the past, relying on traditional design teams and tools, it took 4 weeks to design a new clothing. After introducing artificial intelligence design assistance tools, the design cycle was shortened to within 2 weeks, and the product launch speed was greatly improved. Clothing ODM and OBM enterprises hope to enhance their market competitiveness through efficient design and quickly launch products that meet market demands. Therefore, the LORA technology and the clothing modular design concept are deeply integrated to customize an exclusive display and iteration service system for the vertical clothing design field for enterprises. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, the present invention provides an image generation system combining the LORA model with clothing modular design, which solves the problem that the existing clothing design speed needs to be rapidly improved.

[0006] To achieve the above objectives, the present invention is realized through the following technical solutions: An image generation system combining the LORA model with clothing modular design, including a processor, a user input module, a LORA model training module, a clothing and model automatic production module, a buffer library and a database;

[0007] The LORA model training module is bidirectionally connected to the processor. The LORA model training module includes a feature extraction unit, a stylized LORA model training unit, and a clothing image generation unit. The signal output end of the feature extraction unit is connected to the signal input end of the stylized LORA model training unit, and the signal output end of the stylized LORA model training unit is connected to the signal input end of the clothing image generation unit.

[0008] Preferably, the clothing image retrieval module is bidirectionally connected to the processor, and the clothing image retrieval module retrieves images searched on the Internet by user input text and user input images.

[0009] Preferably, the clothing and model automatic generation module is bidirectionally connected to the processor. The clothing and model automatic generation module includes a character model production unit, a clothing and character model adaptation unit, and a clothing and character model output unit. The character model production unit produces a corresponding three-dimensional character model according to the character body type data collected by the input module.

[0010] Preferably, the user input module is bidirectionally connected to the processor. The input module is used to collect the body type data input by the user and to manually adjust the clothing generated by the clothing image generation unit. The processor is bidirectionally connected to the buffer library, and the buffer library is bidirectionally connected to the database. The buffer library is used to cache the body type data input by the user input module and the clothing images generated by the clothing image generation unit.

[0011] Preferably, the feature extraction unit extracts feature factors from the input clothing images. The feature factors include clothing styles, clothing colors, local designs, and fashion trends. The local designs include necklines and cuffs.

[0012] Preferably, the stylized LORA model training unit sorts out exclusive training data sets for different design elements and trains independent stylized LORA models. Through multi-LORA combination, the generated clothing not only conforms to the fashion trend but also meets the personalized needs of users. The clothing image generation is performed on the Stablediffusion platform.

[0013] Preferably, the stylized LORA model training unit completes the training of the clothing model through the following steps:

[0014] Image screening: By screening the corresponding clothing popular online and offline, finally 90 images of the corresponding clothing are selected;

[0015] Uniform cropping of images: The image resolution is uniformly adjusted to 1024*1024 size by cropping and scaling to meet the training requirements of the stablediffusion XL model;

[0016] Assemble into a training set: Organize and assemble the selected images;

[0017] Batch and detail labeling of images: Use a plugin to label the images, write the corresponding English of the clothing as the first one and uniformly write it into the description text corresponding to the image, and then manually mark the details according to the characteristics of each image

[0018] Start training: Main training parameter settings: Each image is trained 5 times, the number of training epochs is set to 10, the training batch size is 1 image, the learning rate is set to 0.00001, and training is carried out using stablediffusion XL as the base model;

[0019] Generate models: Since there are unstable factors during the training process of the LORA model, which are likely to cause overfitting of the model training, so a model is saved every 2 epochs. Therefore, five LORA models are obtained each time of training. Use the xyz chart plugin to test the five models under different weights to select the model with the best training effect and its corresponding weight. Finally, fine-tune the various parameters and description text according to the training results. Finally, through six times of training, compare 30 models and their Loss value curves to obtain an ideal model, and finally use stablediffusion to generate the effect diagram;

[0020] Modular LORA model training: For the model training of each part of the clothing, screen the pictures of the neckline and cuffs with the largest quantity;

[0021] Adjust the picture labels and training parameters according to the training results: Finally, 58 pictures and 22 pictures are respectively selected for training. Similarly, the image resolution is uniformly adjusted to 1024*1024 size by cropping and scaling, and use a plugin to label the pictures. And insert locally enlarged pictures in the effect diagram of the whole piece of clothing to improve the model training effect. The main training parameters of the LORA model are also set. Finally, two neckline and two cuff style LORA models are obtained through eleven tests;

[0022] Final model generation: By using the models in an overlay manner, users can also modify the local parts of the clothing, thereby achieving the goal of modular design. Modify the cuffs or neckline of the clothing through local redrawing and overlaying the Lora model, and generate the final model.

[0023] Preferably, in step seven, each part of the clothing is the main body of the clothing, sleeves, neckline, cuffs and hem.

[0024] Beneficial effects

[0025] The present invention provides an image generation system that combines a LORA model with modular clothing design. Compared with the prior art, it has the following beneficial effects:

[0026] 1. In the image generation system that combines the LORA model with modular clothing design, by deeply integrating LORA technology with the concept of modular clothing design, the 1+N LORA model concept is proposed. Through 1 silhouette LORA and N modular clothing LORAs, the AI workflow for clothing design is reconstructed. Through LORA training, the Stable Diffusion model can generate clothing styles that meet market demands and fashion trends. At the same time, through modular design, the clothing is split into multiple modules for LORA model training. Users can also freely select the styles of each part of the clothing by superimposing multiple LORA models, realizing personalized customization and creating unique clothing designs.

[0027] 2. In the image generation system that combines the LORA model with modular clothing design, through the clothing and model automatic generation module, the generated clothing images and character models are dressed and matched. The parameters of the character model are input by the user, and thus the designed clothing effect can be better displayed.

[0028] 3. In the image generation system that combines the LORA model with modular clothing design, by setting up a buffer library, the pictures and design parameters in the process of generating clothing images are cached, avoiding data loss in case of system crashes or accidental operation to close the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is the principle block diagram of the present invention;

[0030] Figure 2 is the principle block diagram of the clothing and model automatic generation module of the present invention;

[0031] Figure 3 is the principle block diagram of the LORA model training module of the present invention;

[0032] Figure 4 is the flowchart of the stylized LORA model training of the present invention;

[0033] Figure 5 is the schematic diagram of the module for splitting each part of the clothing of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0035] Please refer to Figures 1 - 5 , the LORA model combined with the image generation system for clothing modular design provides two technical solutions:

[0036] The first implementation manner: includes a processor, a user input module, a LORA model training module, a clothing and model automatic production module, a buffer library and a database;

[0037] The LORA model training module is bidirectionally connected to the processor. The LORA model training module includes a feature extraction unit, a stylized LORA model training unit, and a clothing image generation unit. The signal output end of the feature extraction unit is connected to the signal input end of the stylized LORA model training unit, and the signal output end of the stylized LORA model training unit is connected to the signal input end of the clothing image generation unit. The stylized LORA model training unit sorts out exclusive training data sets for different design elements and trains independent stylized LORA models. Through multi-LORA combination, the generated clothing not only conforms to the fashion trend but also meets the personalized needs of users. The clothing image generation unit is generated on the Stablediffusion platform.

[0038] The clothing image retrieval module is bidirectionally connected to the processor, and the clothing image retrieval module retrieves by searching for pictures on the Internet with the text input by the user and the pictures input by the user. The user input module is bidirectionally connected to the processor. The input module is used to collect the body type data input by the user and to manually adjust the clothing generated by the clothing image generation unit. The processor is bidirectionally connected to the buffer library, and the buffer library is bidirectionally connected to the database. The buffer library is used to cache the body type data input by the user input module and the clothing images generated by the clothing image generation unit. By setting the buffer library, the pictures and design parameters in the process of generating clothing images are cached to avoid data loss when the system crashes or is accidentally operated to close.

[0039] By deeply integrating LORA technology with the concept of modular clothing design, the 1+N LORA model concept is proposed. Through 1 silhouette LORA and N modular clothing LORAs, the AI workflow for clothing design is reconstructed. Through LORA training, the StableDiffusion model can generate clothing styles that meet market demands and fashion trends. At the same time, through modular design, the clothing is disassembled into multiple modules for Lora model training. Users can also freely select the styles of each part of the clothing by superimposing multiple LORA models, achieving personalized customization and creating unique clothing designs.

[0040] The second implementation method, the main difference from the first implementation method is that: the clothing and model automatic generation module is connected to the processor bidirectionally. The clothing and model automatic generation module includes a character model production unit, a clothing and character model adaptation unit, and a clothing and character model output unit. The character model production unit produces a corresponding three-dimensional character model based on the character body type data collected by the input module.

[0041] When in use, the first step: first determine the silhouette, select the customized LORA model, that is, through the clothing image retrieval module, and uniformly adjust the image resolution to 1024*1024 size through cropping and scaling to meet the training requirements of the stablediffusion XL model. Then assemble it into a training set: organize and assemble the selected pictures, batch and detail label the pictures: and use a plug-in to label the pictures. The plug-in is stable-diffusion-webui-wd14-tagger. Write the corresponding English of the clothing as the first one and uniformly write it into the description text corresponding to the image. Then manually label the details according to the characteristics of each picture, input the prompt words, such as describing what style of clothes you want and what style you don't want, select the appropriate silhouette to modify the details, and input the selected pictures into the LORA model training module to start training: The main training parameters are set as follows: each picture is trained 5 times, the number of training epochs is set to 10, the training batch is 1 picture, and the learning rate is set to 0.00001. Use stablediffusion XL as the base model for training;

[0042] Generate the model: Since there are unstable factors during the LORA model training process, which are likely to cause overfitting in model training, so set to save one model every 2 epochs. Therefore, five LORA models are obtained each time of training. Use the xyz graph plug-in to test the five models under different weights to select the model with the best training effect and its corresponding weight. Finally, fine-tune the various parameters and the description text according to the training results. Finally, through six times of training, compare 30 models and their Loss value curves to obtain the ideal model, and finally use stablediffusion to generate the effect picture;

[0043] Modular LORA model training: For the model training of each part of the clothing, screen the pictures of the collar and cuffs with the largest quantity;

[0044] Adjust the picture labels and training parameters according to the training results: Finally, 58 pictures and 22 pictures were selected for training respectively. The image resolution was uniformly adjusted to 1024*1024 size by cropping and scaling, and the pictures were marked using a plugin. And local enlarged pictures were interspersed in the effect picture of the whole piece of clothing to improve the model training effect. The main training parameters of the LORA model were also set. Finally, two LORA models of collar styles and two LORA models of cuff styles were obtained through eleven tests;

[0045] Second step: Then modify the details. Circle the details to be modified in the picture. If you want to modify the cuffs, just circle the cuffs. Select the corresponding modular LORA model. If you circle the cuffs, use the LORA for modifying the cuffs. Enter the prompt words to describe what kind of detailed style you want, such as entering to change the cuffs to the elastic band style, and select the final effect picture;

[0046] Third step: Match the clothing image generated by the LORA model training module with the character model parameters and display them in the virtual space.

[0047] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variation thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0048] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made in these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An image generation system combining LORA model with clothing modular design, characterized in that: It includes a processor, a user input module, a LORA model training module, a clothing and model automatic production module, a buffer library and a database; The LORA model training module is bidirectionally connected to the processor. The LORA model training module includes a feature extraction unit, a stylized LORA model training unit, and a clothing image generation unit. The signal output end of the feature extraction unit is connected to the signal input end of the stylized LORA model training unit, and the signal output end of the stylized LORA model training unit is connected to the signal input end of the clothing image generation unit.

2. The image generation system combining the LORA model and the modular design of clothing according to claim 1, characterized in that: The clothing image retrieval module is bidirectionally connected to the processor, and the clothing image retrieval module retrieves images by searching the Internet for pictures entered by the user and pictures entered by the user.

3. The image generation system combining the LORA model and the modular design of clothing according to claim 1, wherein: The clothing and model automatic generation module is bidirectionally connected to the processor. The clothing and model automatic generation module includes a character model production unit, a clothing and character model adaptation unit, and a clothing and character model output unit. The character model production unit produces a corresponding three-dimensional character model according to the body type data of the character collected by the input module.

4. The image generation system combining the LORA model and clothing modular design according to claim 1, characterized in that: The user input module is bidirectionally connected to the processor. The input module is used to collect the body type data input by the user and to manually adjust the clothing generated by the clothing image generation unit. The processor is bidirectionally connected to the buffer library, and the buffer library is bidirectionally connected to the database. The buffer library is used to cache the body type data input by the user input module and the clothing images generated by the clothing image generation unit.

5. A LORA model combined with clothing modular design image generation system according to claim 1, characterized in that: The feature extraction unit extracts feature factors from the input clothing images. The feature factors include clothing styles, clothing colors, local designs, and fashion trends. The local designs include necklines and cuffs.

6. The image generation system combining the LORA model and the modular design of clothing according to claim 1, wherein: The stylized LORA model training unit sorts out exclusive training data sets for different design elements and trains independent stylized LORA models. Through the combination of multiple LORAs, the generated clothing not only conforms to the fashion trends but also meets the personalized needs of users. The clothing image generation is carried out on the Stablediffusion platform.

7. The image generation system combining the LORA model with the modular design of clothing according to claim 1, characterized in that: The stylized LORA model training unit completes the training of the clothing model through the following steps: Step 1: Picture screening: By screening the corresponding clothing popular online and offline, finally select 90 images of the corresponding clothing; Step 2: Picture unified cropping: Uniformly adjust the image resolution to 1024*1024 size through cropping and scaling to meet the training requirements of the stablediffusion XL model; Step 3: Aggregate into a training set: Organize and aggregate the selected pictures; Step 4: Batch and detail label the pictures: And use a plug-in to label the pictures, write the corresponding English of the clothing as the first and uniformly write it into the description text corresponding to the image, and then manually label the details according to the characteristics of each picture; Step 5: Start training: Main training parameter settings: Each picture is trained 5 times, the number of training epochs is set to 10, the training batch size is 1, the learning rate is set to 0.00001, and the training is carried out with stablediffusion XL as the base model; Step 6: Generate models: Since there are unstable factors during the training process of the LORA model, which are likely to cause overfitting in model training, one model is saved every 2 epochs. Therefore, five LORA models are obtained in each training. The xyz chart plugin is used to test the five models with different weights to select the model with the best training effect and its corresponding weight. Finally, according to the training effect, various parameters and the description text are fine-tuned. Finally, through six trainings, after comparing 30 models and their Loss value curves, an ideal model is obtained, and finally stablediffusion is used to generate the effect picture; Step 7: Modular LORA model training: For the model training of each part of the clothing, screen the pictures of the collar and cuffs with the largest quantity; Step 8: Adjust the picture labels and training parameters according to the training results: Finally, 58 pictures and 22 pictures are selected for training respectively. The image resolution is uniformly adjusted to 1024*1024 by cropping and scaling, and the pictures are labeled using the plugin. And local enlarged pictures are interspersed in the effect picture of the whole piece of clothing to improve the model training effect. The main training parameters of the LORA model are also set. Finally, two LORA models of the collar and two cuffs styles are obtained through eleven tests respectively; Step 9: Generate the final model: By using the models in an overlay manner, users can also modify the local part of the clothing, thereby achieving the goal of modular design. Modify the cuffs or collar of the clothing by local redrawing and overlaying the Lora model, and generate the final model.

8. The image generation system combining the LORA model and the modular design of clothing according to claim 1, characterized in that: In the said Step 7, each part of the clothing is the main body body, sleeves, collar, cuffs and hem.

Images