Device and method for automatically realizing text graph training

Through automated literary picture training devices and methods, the problems of low data preparation and model training efficiency and unstable image quality in literary picture generation technology are solved, and efficient and automated literary picture generation is achieved, improving application effect and convenience.

CN119068081APending Publication Date: 2024-12-03SHANGHAI HUANZE INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410906657.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-08
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing literary and biographical image generation technology faces problems such as insufficient quality and quantity of training data sets, long model complexity and training time, unstable image quality and high dependence on professional and technical personnel, resulting in insufficient application effect and convenience.

Method used

Provide a device and method for automatically realizing literary and artistic training, including picture preprocessing module, parameter adjustment module and training module, through automatic cropping and labeling, dynamically adjusting the learning rate and training round number, using deep learning algorithms and stable diffusion technology , realize efficient and automated literary and artistic picture training process.

Benefits of technology

It significantly improves training efficiency and image generation quality, reduces resource consumption and dependence on professional and technical personnel, improves the reliability and stability of user experience and model training, and is suitable for multiple application scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068081A_ABST
    Figure CN119068081A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a device and method for automatically achieving text image training, and the device comprises a picture preprocessing module which is used for completing the preprocessing work of an input image; the parameter adjusting module is used for adjusting various parameters required by training; the training module is used for receiving the preprocessed data and the adjusted parameters, directly starting model training and outputting a training result; the display module is used for displaying the generation effect of each LoRA file by using a Stable Diffusion technology; compared with the prior art, the method has the advantages of being more efficient and higher in automation degree, the application effect and use convenience of the text map generation technology are improved, the overall performance, efficiency, quality, user experience and the like of the text map training device are remarkably improved, and a solid foundation is laid for wide application of the image generation technology.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to the field of artificial intelligence technology, in particular to a device and method for automatically implementing text graph training. [Background Technology]

[0002] As an emerging artificial intelligence technology, Text-to-Image generation technology has made significant progress in recent years driven by the development of generative adversarial networks (GANs) and deep learning algorithms. This technology generates images that match the description by inputting natural language text descriptions, and has been widely used in image generation, image enhancement, virtual reality and other fields.

[0003] Currently, the technology of generating text images mainly relies on models such as Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs). For example, StackGAN achieves the generation of high-resolution images by dividing the image generation process into two stages; AttnGAN improves the details and consistency of generated images by introducing an attention mechanism. These methods have significantly improved the quality and accuracy of text image generation, but they still face some challenges:

[0004] (1) Insufficient quality and quantity of training datasets: The training of text-based graph generation models relies on large-scale, high-quality image and text description datasets. However, collecting and annotating such datasets is expensive, and the diversity and coverage of the datasets may also be insufficient.

[0005] (2) Model complexity and long training time: Existing text graph generation models are generally complex, and the training process involves a lot of parameter tuning and iteration, which takes a long time to train and consumes a lot of computing resources;

[0006] (3) Unstable quality of generated images: Although the latest technologies have significantly improved the quality of generated images, the generated images may still have problems such as blur and distortion, especially when dealing with complex and abstract text descriptions;

[0007] (4) High dependence on professional technicians: During the training, optimization, and deployment of the model, professional technicians are required to perform a lot of manual intervention and tuning, which increases labor costs and maintenance difficulty.

[0008] In summary, the existing technology still has many limitations and challenges in practical applications, and there is an urgent need for a more efficient and more automated device to improve the application effect and ease of use of the text map generation technology. [Summary of the invention]

[0009] The purpose of the present invention is to solve the above-mentioned deficiencies and provide a more efficient and higher degree of automation device for automatically realizing Vincent image training, thereby improving the application effect and ease of use of Vincent image generation technology, significantly improving the overall performance, efficiency, quality and user experience of the Vincent image training device, and laying a solid foundation for the widespread application of image generation technology.

[0010] In one aspect, the present invention provides a device for automatically implementing text graph training, comprising:

[0011] Image preprocessing module, used to complete the preprocessing of input images;

[0012] Parameter adjustment module, used to adjust various parameters required for training;

[0013] The training module is used to receive the preprocessed data and adjusted parameters, directly start the model training, and output the training results;

[0014] The display module is used to use the Stable Diffusion technology to show the generation effect of each LoRA file.

[0015] In a possible implementation, the image preprocessing module includes an image cropping unit and an image labeling unit. The image cropping unit crops the image according to the size and requirements of the input image to meet the training requirements of the model; the image labeling unit automatically adds labels to the input image and generates corresponding text descriptions.

[0016] In another possible implementation, the parameter adjustment module dynamically adjusts the learning rate according to the performance during training and sets the number of rounds of model training to ensure that the model achieves optimal performance within an appropriate time. At the same time, according to specific training tasks and model requirements, the relevant hyperparameters are adjusted to optimize the training process.

[0017] In another possible implementation, the training module includes a data input unit, a model training unit and a result output unit. The data input unit is used to receive preprocessed images and corresponding text description data; the model training unit trains the input data based on a deep learning algorithm to generate a text-based graph model; and the result output unit saves the trained model parameters in the LoRA file format.

[0018] In other possible implementations, the display module includes an input LoRA file unit and an image generation unit. The input LoRA file unit is used to load the generated LoRA file, and the image generation unit generates a high-quality image based on the LoRA file to display the actual effect of the model.

[0019] In another aspect, the present invention provides a method for automatically implementing text graph training, comprising the following steps:

[0020] 1) Image preprocessing: preprocess the input image and automatically complete the image cropping and labeling operations;

[0021] 2) Parameter adjustment: Optimize the model training process by dynamically adjusting the parameters required for training, including but not limited to the learning rate and the number of training rounds;

[0022] 3) Model training: receiving preprocessed data and adjusted parameters, directly starting model training, and outputting training results;

[0023] 4) Generation and display: Use Stable Diffusion technology to display the generation effect of each LoRA file.

[0024] In a possible implementation, image preprocessing includes the following steps:

[0025] Step 101, receiving input images: collecting image data sets;

[0026] Step 102: cropping the image: cropping the original image to ensure that all images have a uniform size and meet the input requirements of the model;

[0027] Step 103, adding labels to images: automatically adding labels to each image and generating text descriptions;

[0028] Step 104, output preprocessed data: transmit the cropped image and corresponding text label data to the parameter adjustment step.

[0029] In another possible implementation, the parameter adjustment includes the following steps:

[0030] Step 201, setting training parameters: setting parameters including but not limited to learning rate and number of training rounds;

[0031] Step 202, optimize parameter settings: dynamically adjust the learning rate and the number of training rounds according to the performance of the model during the training process to optimize the training effect.

[0032] In another possible implementation, model training includes the following steps:

[0033] Step 301, receiving preprocessing data and parameters: obtaining cropped images and label data from the image preprocessing step, and obtaining set training parameters from the parameter adjustment step;

[0034] Step 302, perform model training: start the model training process, use the deep learning algorithm to train the input data, and dynamically adjust the learning rate and the number of training rounds according to the feedback of the parameter adjustment step during the training process;

[0035] Step 303, output training results: After the training is completed, the model parameters are saved as a LoRA file for subsequent application and display.

[0036] In other possible implementations, generating and displaying include the following steps:

[0037] Step 401, load LoRA file: load the LoRA file generated after training;

[0038] Step 402, generating an image using stable diffusion technology: generating a high-quality image based on the LoRA file and the input text description;

[0039] Step 403: Result display: display the generated image.

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] (1) Significantly improve training efficiency:

[0042] Automated preprocessing: The image preprocessing module of the present invention can automatically complete the image cropping and labeling operations, reducing a lot of manual intervention and improving the efficiency of data preparation;

[0043] Automatic parameter adjustment: The parameter adjustment module of the present invention optimizes the model training process by dynamically adjusting parameters such as the learning rate and the number of training rounds, reduces the time for repeated experiments and manual tuning, and improves the overall training efficiency.

[0044] (2) Significantly reduce resource consumption:

[0045] Efficient computing resource utilization: This invention optimizes the utilization of computing resources and reduces the dependence on high-performance computing devices during the training process through scientific and reasonable parameter settings and deep learning algorithms.

[0046] Reduce hardware requirements: Minimize reliance on expensive hardware equipment, allowing more small and medium-sized scientific research institutions and enterprises to deploy and use the device at a lower cost.

[0047] (3) Improve image generation quality:

[0048] High-quality image output: Through the stable diffusion technology (Stable Diffusion), the display module of the present invention can generate high-resolution and high-quality images, ensuring that the generated effect meets the application requirements.

[0049] Enhanced details and consistency: The deep learning algorithm and attention mechanism used in the device of the present invention can effectively improve the details and consistency of image generation, making the image more realistic and complete.

[0050] (4) Significantly improve the level of automation:

[0051] Full process automation: From image preprocessing, parameter adjustment to model training and result display, the present invention has achieved a high degree of automation in each link, reducing the dependence on professional technicians.

[0052] Multi-task support: The device of the present invention can handle multiple training tasks at the same time, and automatically schedule and manage various tasks, thereby improving work efficiency and device utilization.

[0053] (5) Enhanced applicability and scalability:

[0054] Adaptable to different application scenarios: The device of the present invention has flexible configuration capabilities and is suitable for data sets of different sizes and types. It is widely used in multiple fields such as image generation, artistic creation, and video production.

[0055] Modular design: The device of the present invention adopts a modular design, and each module is independent of each other, which is easy to customize and expand, providing convenience for future technical upgrades and functional expansion.

[0056] (6) Optimize user experience:

[0057] Simplified operation process: The present invention simplifies the operation process through a friendly user interface design, so that even non-professionals can quickly use the device to complete the task of text map training.

[0058] Real-time feedback and display: The display module of the present invention can generate and display image effects in real time, so that users can view and evaluate training results in time and optimize the use experience.

[0059] (7) Enhance model training reliability and stability:

[0060] Intelligent error correction and adjustment: The device of the present invention has built-in intelligent monitoring and tuning functions, which can automatically detect and adjust abnormal situations during training to ensure the stability and reliability of training tasks.

[0061] In summary, the present invention provides a more efficient and highly automated device and method to improve the application effect and ease of use of the Vincent graph generation technology; specifically, the present invention has achieved significant improvements in the overall performance, efficiency, quality and user experience of the Vincent graph training device, laying a solid foundation for the widespread application of image generation technology. [Drawings]

[0062] Figure 1It is a schematic diagram of the structure of the device of the present invention;

[0063] Figure 2 It is a schematic flow diagram of the method of the present invention;

[0064] Figure 3 It is a schematic diagram of specific implementation steps of the method of the present invention;

[0065] Figure 4 It is a structural schematic diagram of the electronic device of the present invention;

[0066] Figure 5 This is a schematic diagram of image cutting and marking of the present invention;

[0067] Figure 6 It is a schematic diagram of parameter adjustment of the present invention;

[0068] Figure 7 It is a schematic diagram of the training results of the present invention. [Specific implementation method]

[0069] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present invention. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0070] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described below in conjunction with the accompanying drawings and specific embodiments:

[0071] As attached Figure 1 As shown, in some embodiments, the present invention provides a device for automatically implementing text image training, including: an image preprocessing module, used to complete the preprocessing of the input image; a parameter adjustment module, used to adjust the various parameters required for training; a training module, used to receive the preprocessed data and the adjusted parameters, directly start the model training, and output the training results; a display module, used to use stable diffusion Stable Diffusion technology to display the generation effect of each LoRA file.

[0072] Among them, the image preprocessing module includes an image cropping unit and an image labeling unit. The image cropping unit crops the image according to the size and requirements of the input image to meet the training requirements of the model; the image labeling unit automatically adds labels to the input image and generates corresponding text descriptions. The parameter adjustment module dynamically adjusts the learning rate according to the performance during the training process, and sets the number of rounds of model training to ensure that the model achieves optimal performance within an appropriate time; at the same time, according to the specific training tasks and model requirements, adjust the relevant hyperparameters to optimize the training process. The training module includes a data input unit, a model training unit and a result output unit. The data input unit is used to receive the preprocessed image and the corresponding text description data; the model training unit trains the input data based on the deep learning algorithm to generate a text-based image model; the result output unit saves the trained model parameters in the LoRA file format. The display module includes an input LoRA file unit and an image generation unit. The input LoRA file unit is used to load the generated LoRA file, and the image generation unit generates high-quality images based on the LoRA file to display the actual effect of the model.

[0073] As attached Figure 2 and attached Figure 3 As shown, in some other embodiments, the present invention provides a method for automatically implementing the training of a text map, comprising the following steps:

[0074] 1) Image preprocessing: preprocess the input image and automatically complete the image cropping and labeling operations;

[0075] 2) Parameter adjustment: Optimize the model training process by dynamically adjusting the parameters required for training, including but not limited to the learning rate and the number of training rounds;

[0076] 3) Model training: receiving preprocessed data and adjusted parameters, directly starting model training, and outputting training results;

[0077] 4) Generation and display: Use Stable Diffusion technology to display the generation effect of each LoRA file.

[0078] In some other embodiments, the present invention provides a device for automatically implementing text graph training, which mainly includes the following modules:

[0079] The image preprocessing module is used to complete the preprocessing of the input image. The specific functions include: (1) Image cropping: crop the image according to the size and requirements of the input image to meet the training requirements of the model. (2) Image labeling: automatically add labels to the input image and generate corresponding text descriptions as part of the training data.

[0080] The parameter adjustment module is used to adjust the parameters required for training, including: (1) Learning rate: dynamically adjust the learning rate according to the performance during training to improve training efficiency and effect. (2) Number of training rounds: set the number of rounds of model training to ensure that the model achieves optimal performance within an appropriate time. (3) Other hyperparameters: adjust related hyperparameters according to specific training tasks and model requirements to optimize the training process.

[0081] The training module receives the preprocessed data and adjusted parameters, directly starts the model training, and outputs the training results. The main functions include: (1) Data input: receiving the preprocessed image and the corresponding text description data. (2) Model training: based on the deep learning algorithm, the input data is trained to generate a text-based graph model. (3) Result output: the trained model parameters are saved in the LoRA file (.safetensors file) format for subsequent applications.

[0082] The display module uses the Stable Diffusion technology to display the generated effect of each LoRA file. The specific functions include: (1) Input LoRA file: load the generated LoRA file. (2) Image generation: generate high-quality images based on the LoRA file to show the actual effect of the model.

[0083] In some other embodiments, a LoRA (Low-Rank Adaptation) for training clothing images is provided, and the specific steps are as follows:

[0084] 1. Image preprocessing module

[0085] Step 1. Receive input images: We collected a dataset of images containing various types of clothing, including T-shirts, shirts, pants, skirts, etc.

[0086] Step 2: Image cropping: The image preprocessing module crops the original image to ensure that all images have uniform sizes and meet the input requirements of the model.

[0087] Step 3: Label the images: The image preprocessing module automatically labels each image and generates text descriptions. These labels will be input into the training module as training data along with the images.

[0088] Step 4: Output preprocessed data: The cropped image and the corresponding text label data are transferred to the parameter adjustment module.

[0089] 2. Parameter adjustment module

[0090] Step 5: Set training parameters

[0091] Learning rate: 0.001 (based on initial settings, can be adjusted dynamically during training)

[0092] Number of training rounds: 50 rounds (adjusted according to the size of the dataset and the complexity of the model)

[0093] Other hyperparameters: such as batch size, optimizer type (such as Adam or SGD), etc.

[0094] Step 6: Optimize parameter settings

[0095] According to the performance of the model during training, the parameter adjustment module dynamically adjusts the learning rate and the number of training rounds to optimize the training effect.

[0096] 3. Training Module

[0097] Step 7, receiving preprocessing data and parameters: the training module obtains the cropped image and label data from the image preprocessing module, and obtains the set training parameters from the parameter adjustment module.

[0098] Step 8: Execute model training: The training module starts the model training process and uses the deep learning algorithm to train the input data. During the training process, the learning rate and number of training rounds are dynamically adjusted according to the feedback from the parameter adjustment module to ensure that the model is trained in the optimal state.

[0099] Step 9. Output training results: After training is completed, the training module saves the model parameters as a LoRA file (.safetensors format) for subsequent application and display.

[0100] 4. Display module

[0101] Step 10. Load LoRA file: The display module loads the LoRA file generated after training.

[0102] Step 11. Generate an image using stable diffusion technology: The display module generates a high-quality image based on the LoRA file and the input text description. For example, if the input text description is "blue jeans", the display module will generate a corresponding image of blue jeans.

[0103] Step 12: Result display: The display module displays the generated images to facilitate users to observe and evaluate the training effect.

[0104] In addition, it is considered as another specific application example of the present invention, and the steps are as follows:

[0105] (1) Data preparation: 500 clothing images of different styles and colors were collected, each with a corresponding text description. After preprocessing, 500 cropped and labeled image and description pairs were obtained.

[0106] (2) Parameter setting: Set the initial learning rate to 0.001, the number of training rounds to 50, the batch size to 32, and use the Adam optimizer.

[0107] (3) Model training: After 50 rounds of training, the trained LoRA file (e.g. output.safetensors) is output.

[0108] (4) Generation and display: Load the .safetensors file, input the description "red dress", and the display module generates a high-quality image of a red dress for the user to inspect and verify.

[0109] In some other embodiments of the present invention, an electronic device 400 is disclosed, as shown in the attached figure. Figure 4 As shown, the electronic device includes a processor and a memory, the processor and the memory are connected via a bus, and the memory stores a program that can be run on the processor. When the program is executed by the processor, the electronic device implements the above method. Specifically, the hardware composition of the electronic device includes: one or more processors 401; memory 402; display 403; one or more application programs (not shown); and one or more computer programs 404, and the above devices can be connected via one or more communication buses 405. The one or more computer programs 404 are stored in the above memory 402 and are configured to be executed by the one or more processors 401, and the one or more computer programs 404 include instructions.

[0110] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a computer, the method described in the above method embodiment is implemented. For specific beneficial effects, please refer to the above method embodiment. The present invention also provides a computer program product, and when the computer program product is executed by a computer, the method described in the above method embodiment is implemented. For specific beneficial effects, please refer to the above method embodiment.

[0111] Through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0112] Each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0113] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: flash memory, mobile hard disk, read-only memory, random access memory, disk or optical disk and other media that can store program codes.

[0114] The above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit it. Under the idea of ​​the present invention, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present invention as described above. Although the present invention is described in detail with reference to the above embodiments, it should be understood by those skilled in the art that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

[0115] The present invention is not limited to the above-mentioned implementation modes, and any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement modes and shall be included in the protection scope of the present invention.

Claims

1. A device for automatically implementing text graph training, characterized in that: include: Image preprocessing module, used to complete the preprocessing of input images; Parameter adjustment module, used to adjust various parameters required for training; The training module is used to receive the preprocessed data and adjusted parameters, directly start the model training, and output the training results; The display module is used to use the Stable Diffusion technology to show the generation effect of each LoRA file.

2. The device according to claim 1, characterized in that: The image preprocessing module includes an image cropping unit and an image labeling unit. The image cropping unit crops the image according to the size and requirements of the input image to meet the training requirements of the model; the image labeling unit automatically adds labels to the input image and generates corresponding text descriptions.

3. The device according to claim 1, characterized in that: The parameter adjustment module dynamically adjusts the learning rate according to the performance during training and sets the number of rounds of model training to ensure that the model achieves optimal performance within an appropriate time. At the same time, it adjusts relevant hyperparameters and optimizes the training process according to specific training tasks and model requirements.

4. The device according to claim 1, characterized in that: The training module includes a data input unit, a model training unit and a result output unit, wherein the data input unit is used to receive the preprocessed image and the corresponding text description data; The model training unit trains the input data based on the deep learning algorithm to generate a text graph model; the result output unit saves the trained model parameters in the LoRA file format.

5. The device according to claim 1, characterized in that: The display module includes a LoRA file input unit and an image generation unit. The LoRA file input unit is used to load the generated LoRA file. The image generation unit generates a high-quality image based on the LoRA file to display the actual effect of the model.

6. A method for automatically implementing text graph training, characterized in that: The following steps are involved: 1) Image preprocessing: preprocess the input image and automatically complete the image cropping and labeling operations; 2) Parameter adjustment: Optimize the model training process by dynamically adjusting the parameters required for training, including but not limited to the learning rate and the number of training rounds; 3) Model training: receiving preprocessed data and adjusted parameters, directly starting model training, and outputting training results; 4) Generation and display: Use Stable Diffusion technology to display the generation effect of each LoRA file.

7. The method according to claim 6, characterized in that Image preprocessing includes the following steps: Step 101, receiving input images: collecting image data sets; Step 102: cropping the image: cropping the original image to ensure that all images have a uniform size and meet the input requirements of the model; Step 103, adding labels to images: automatically adding labels to each image and generating text descriptions; Step 104, output preprocessed data: transmit the cropped image and corresponding text label data to the parameter adjustment step.

8. The method according to claim 6, characterized in that Parameter adjustment includes the following steps: Step 201, setting training parameters: setting parameters including but not limited to learning rate and number of training rounds; Step 202, optimize parameter settings: dynamically adjust the learning rate and the number of training rounds according to the performance of the model during the training process to optimize the training effect.

9. The method according to claim 6, characterized in that Model training consists of the following steps: Step 301, receiving preprocessing data and parameters: obtaining cropped images and label data from the image preprocessing step, and obtaining set training parameters from the parameter adjustment step; Step 302, perform model training: start the model training process, use the deep learning algorithm to train the input data, and dynamically adjust the learning rate and the number of training rounds according to the feedback of the parameter adjustment step during the training process; Step 303, output training results: After the training is completed, the model parameters are saved as a LoRA file for subsequent application and display.

10. The method according to claim 6, characterized in that The generation and display process includes the following steps: Step 401, load LoRA file: load the LoRA file generated after training; Step 402, generating an image using a stable diffusion technique: generating a high-quality image based on the LoRA file and the input text description; Step 403: Result display: display the generated image.