A system for automatically generating pictures from text

Abstract

The application is suitable for the field of computers and provides a system for automatically generating pictures from text, which comprises a text information import module arranged in the system and used for importing description text information into a picture calling model; a picture calling module used for the picture calling model to call pictures consistent with the description text information from a picture library to form a calling picture set; a parameter acquisition module used for acquiring a random value, an iteration number and an update step distance; a basic picture setting module used for selecting a picture from the calling picture set as a basic picture according to the random value; a basic picture iteration module used for importing the basic picture into a picture generation model and regenerating the picture according to the iteration number and the update step distance; and a target picture export module used for exporting the generated target picture from the picture generation model when the iteration number is completed. Different target pictures can be directly generated and exported according to the input description text, and the workload of manual drawing is reduced.

Description

Technical Field

[0001] This invention belongs to the field of computer science, and in particular relates to a system for automatically generating images from text. Background Technology

[0002] Intelligent recognition technology uses computer systems, scanning equipment, and photographic equipment as infrastructure to intelligently identify target data. This technology is widely used in image-to-text conversion, such as the scanners and text recognition software we use every day.

[0003] Image recognition is an important technology in the information age, created to enable computers to process large amounts of physical information instead of humans. With the development of computer technology, our understanding of image recognition technology has deepened. The process of image recognition technology includes information acquisition, preprocessing, feature extraction and selection, classifier design, and classification decision. This paper briefly analyzes the introduction of image recognition technology, its technical principles, and pattern recognition, and then introduces neural network image recognition technology, nonlinear dimensionality reduction image recognition technology, and its applications.

[0004] As people's work pace and workload increase, and automation is promoted, the one-way conversion from image to text can no longer meet people's work needs. The need for automatically generating images from text is also increasing. For example, industries such as animation, games, and advertising magazines that require a large number of images as backgrounds or scenes need automated image generation processes. Summary of the Invention

[0005] This invention provides a system for automatically generating images from text, aiming to solve the problem that there is no image-text converter that can automatically generate an image based on a text description.

[0006] The present invention is implemented as follows: a system for automatically generating images from text includes:

[0007] The text information acquisition module is used to acquire a descriptive text information; the descriptive text information is a description of the final target image; the descriptive text information includes at least the image style and theme;

[0008] The text information import module is used to import descriptive text information into the image call model;

[0009] The image retrieval module is used by the image retrieval model to retrieve images from the image library that match the descriptive text information, forming a set of retrieved images;

[0010] The parameter acquisition module is used to obtain random values, iteration count, and update step size;

[0011] The base image setting module is used to select an image from the set of images to be called as the base image according to a random value.

[0012] The basic image iteration module is used to import the basic image into the image generation model and regenerate the image according to the number of iterations and the update step; the update step is the distance the region moves in each iteration; the number of iterations is the number of times the image generation model iterates and changes the image.

[0013] The target image export module is used to export the generated target image from the image generation model when the iteration count is complete.

[0014] As an improvement to the present invention: the training method of the image retrieval model includes:

[0015] A large number of images and their descriptions are imported into the image retrieval model for training.

[0016] During training, the image call model automatically recognizes the content in the image, converts it into recognized text, and adds it to the description information. When the model recognizes the image, it only recognizes the main elements in the image. The main elements include the main color, image style, theme, and scene.

[0017] By allowing users to add descriptions themselves or manually adding descriptions, a correspondence is established between images and their corresponding descriptions, which are then stored in the image library.

[0018] As another improvement of the present invention: the training method of the image generation model includes:

[0019] A large number of images are imported into the image generation model for training;

[0020] The image generation model is continuously learned and trained to generate images corresponding to a specified theme. The image generation model includes a generator and a discriminator, which are alternately optimized and trained using an adversarial neural network.

[0021] As another improvement to the present invention: the image calling module specifically includes:

[0022] The keyword analysis unit is used to analyze the effective keywords in the descriptive text information of the image call model; the effective keywords include at least one of image style descriptive words, image theme descriptive words, and image color descriptive words.

[0023] The image search unit is used to search for image description information in the image library based on valid keywords. When the image description information contains valid keywords, the image corresponding to the image description information is copied and retrieved, and temporarily stored in the base image set.

[0024] As a further aspect of the present invention: the base image can also be obtained by uploading a specified base image through the user terminal, wherein the specified base image is a patterned image or a blank image; when the specified base image is a blank image, the values ​​of the iteration number and the update step size should be increased accordingly.

[0025] As a further embodiment of the present invention: the basic image setting module specifically includes:

[0026] Image set sorting unit, used to randomly sort the images in the image set according to their order;

[0027] The base image determination unit is used to select an image from the image set whose sequence number is equal to the random value from the current random sorting of the image set as the base image.

[0028] As an optimized solution of the present invention: the basic image iteration module specifically includes:

[0029] The basic image import unit is used to import basic images into the image generation model;

[0030] An iterative region setting unit is used to set the area of ​​each iterative region according to the number of iterations. The number of iterations is inversely proportional to the area of ​​each iterative region. Within the set number of iterations, the entire base image can be iterated at least once according to the area of ​​each iterative region set accordingly.

[0031] An iterative region movement unit is used to move the iterative region once per iteration according to the update step size; the iterative region starts moving from at least one side of the base image; after each move, the iterative region intersects with the position of the previous iterative region;

[0032] The image modification unit is used to modify the image area covered by the iteration region using the generator in the image generation model according to the image style in the descriptive text information.

[0033] The beneficial effects of this invention are as follows: By setting up a text information acquisition module, a text information import module, an image calling module, a parameter acquisition module, a basic image setting module, a basic image iteration module, and a target image export module within the system, the following functions are described: The text information acquisition module is used to acquire a descriptive text information, which includes at least the image style and theme; the text information import module is used to import the descriptive text information into the image calling model; the image calling module is used to call images from the image library that match the descriptive text information to form a set of calling images; the parameter acquisition module is used to acquire random values, iteration counts, and update step sizes; the basic image setting module is used to select an image from the set of calling images as the base image according to the random values; the basic image iteration module is used to import the base image into the image generation model and regenerate the image according to the iteration counts and update step sizes; the target image export module is used to export the generated target image from the image generation model when the iteration count is complete. This allows for the automatic generation and export of different target images directly based on the input descriptive text, increasing the image generation rate and reducing the workload of manual drawing. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the internal structure of a system that automatically generates images from text.

[0035] Figure 2 This is a flowchart of an image retrieval model training method in a system that automatically generates images from text.

[0036] Figure 3 This is a schematic diagram of the internal structure of an image calling module in a system that automatically generates images from text.

[0037] Figure 4 This is a schematic diagram of the internal structure of the basic image setting module in a system that automatically generates images from text.

[0038] Figure 5 This is a schematic diagram of the internal structure of a basic image iteration module in a system that automatically generates images from text. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0040] This invention establishes a system comprising a text information acquisition module, a text information import module, an image retrieval module, a parameter acquisition module, a basic image setting module, a basic image iteration module, and a target image export module. The text information acquisition module acquires descriptive text information, which includes at least the image style and theme. The text information import module imports the descriptive text information into the image retrieval model. The image retrieval module retrieves images from an image library that match the descriptive text information, forming a set of retrieved images. The parameter acquisition module acquires random values, iteration counts, and update step sizes. The basic image setting module selects an image from the retrieved image set as the base image based on the random values. The basic image iteration module imports the base image into the image generation model and regenerates the image according to the iteration count and update step size. The target image export module exports the generated target image from the image generation model when the iteration count is complete. This system can automatically generate and export different target images directly based on the input descriptive text, increasing the image generation rate and reducing the workload of manual drawing.

[0041] Figure 1 This diagram illustrates the internal structure of a system for automatically generating images from text, according to an embodiment of the present invention. The system includes:

[0042] The text information acquisition module 100 is used to acquire a descriptive text information; the descriptive text information is a description of the final target image; the descriptive text information includes at least the image style and theme;

[0043] The text information import module 200 is used to import descriptive text information into the image call model;

[0044] Image retrieval module 300 is used by the image retrieval model to retrieve images from the image library that match the descriptive text information, forming a set of retrieved images;

[0045] The parameter acquisition module 400 is used to acquire random values, iteration counts, and update step sizes. It saves the acquired parameters and generates an iteration template for direct use later, requiring only changes to the text information and basic images.

[0046] The base image setting module 500 is used to select an image from the set of called images according to a random value as the base image;

[0047] The basic image iteration module 600 is used to import the basic image into the image generation model and regenerate the image according to the number of iterations and the update step; the update step is the distance the region moves in each iteration; the number of iterations is the number of times the image generation model iterates and changes the image.

[0048] The target image export module 700 is used to export the generated target image from the image generation model when the iteration count is complete.

[0049] Before iterative updates, you can set the size of the exported image, ensuring that updates are performed within the specified size range during iteration. You can also set the quality of the exported image during the export process.

[0050] Besides selecting images from the image set as base images by setting random values, the base images can also be obtained by uploading specific base images from the user's client. These specific base images can be patterned images or blank images; when the specified base image is a blank image, the iteration count and update step size should be increased accordingly. Furthermore, a reference weight ratio can be set to determine the similarity between the final exported image and the base image, causing significant deviations during the iteration process. Of course, to reduce image copyright disputes, the reference weight ratio can be appropriately lowered, making the target pattern significantly different from the base pattern.

[0051] Figure 2 The flowchart illustrates a method for training an image retrieval model in a system that automatically generates images from text, according to an embodiment of the present invention. The training method for the image retrieval model includes:

[0052] Step S10: Import a large number of images and image description information into the image retrieval model for training; the description information includes the scene, main objects, content, style, author, etc. in the image;

[0053] Step S11: During training, the image call model automatically recognizes the content in the image, converts it into recognized text, and adds it to the description information. When the model recognizes the image, it only recognizes the main elements in the image. The main elements include the main color, image style, theme, and scene, such as colors that occupy a large area, objects that occupy a large area, or scenes. However, due to the limitations of automatic recognition, abstract objects are not easy to be recognized. At this time, the color, shape, etc. of abstract objects are recognized as much as possible to make up for the lack of recognition of objects.

[0054] Step S12: Establish a correspondence between the image and the corresponding description information by adding description information manually or independently, and save it in the image library.

[0055] In one embodiment, the training method for the image generation model includes:

[0056] Step S20: Import a large number of images into the image generation model for training;

[0057] Step S21: The image generation model is continuously learned and trained to generate images corresponding to the specified theme; the image generation model includes a generator and a discriminator, which are alternately optimized and trained using an adversarial neural network.

[0058] It should be noted that during model training, different types of images need to be trained separately to ensure better learning results and prevent training confusion and unsatisfactory results caused by mixing multiple images.

[0059] A Generative Adversarial Network (GAN) consists of two models: a generative model and a discriminative model. The generative model's task is to generate instances that appear natural and realistic, similar to the original data. The discriminative model's task is to determine whether a given instance appears natural and realistic or is artificially created (real instances come from the dataset, while fake instances come from the generative model).

[0060] This can be viewed as a zero-sum game. The generator tries to deceive the discriminator, while the discriminator strives not to be deceived. Through alternating optimization training, both models can be improved, but ultimately, we want a generative model with very high performance, whose generated products are indistinguishable from real ones.

[0061] During training, the goal of the generative network G is to generate as many realistic images as possible to deceive the network D, while the goal of D is to try to distinguish the fake images generated by G from the real images. In this way, G and D constitute a dynamic "game process", and the final equilibrium point is the Nash equilibrium point.

[0062] Ideally, the images used during training should be the same for both the image generation model and the image retrieval model. This allows for verification of the quality of the images produced after the retrieval model retrieves the images and the generation model generates them.

[0063] Figure 3 This diagram illustrates the internal structure of an image calling module in a system for automatically generating images from text, according to an embodiment of the present invention. The image calling module 300 specifically includes:

[0064] The keyword analysis unit 310 is used to analyze the effective keywords in the descriptive text information of the image call model; the effective keywords include at least one of image style descriptive words, image theme descriptive words, and image color descriptive words.

[0065] Image search call unit 320 is used to search for image description information in the image library based on valid keywords. When the image description information contains valid keywords, the image corresponding to the image description information is copied and retrieved, and temporarily stored in the call base image set.

[0066] Figure 4 This diagram illustrates the internal structure of a basic image setting module in a system for automatically generating images from text, according to an embodiment of the present invention. The basic image setting module 500 specifically includes:

[0067] Image set sorting unit 510 is used to randomly sort the images in the image set according to their order;

[0068] The base image determination unit 520 is used to select an image from the image set whose sequence number is equal to the random value from the current random sorting of the image set as the base image.

[0069] Figure 5 This diagram illustrates the internal structure of a basic image iteration module in a system for automatically generating images from text, according to an embodiment of the present invention. The basic image iteration module 600 specifically includes:

[0070] The basic image import unit 610 is used to import basic images into the image generation model;

[0071] The iteration region setting unit 620 is used to set the area of ​​each iteration region according to the number of iterations; the number of iterations is inversely proportional to the area of ​​each iteration region, and within the set number of iterations, the entire base image can be iterated at least once according to the area of ​​each iteration region set accordingly;

[0072] The iterative region moving unit 630 is used to move the iterative region once according to the update step size in each iteration; the iterative region starts to move from at least one side of the base image; after each move, the iterative region intersects with the position of the previous iterative region;

[0073] Image modification unit 640 is used to modify the image area covered by the iteration area according to the image style in the descriptive text information using the generator in the image generation model.

[0074] During an iteration, since iteration takes time, multiple images can be uploaded continuously to generate iteration tasks while waiting. These tasks are independent of each other, forming a queue to wait for processing.

[0075] In order for the above methods and systems to operate smoothly, the system may include more or fewer components than those described above, or combine certain components, or different components, in addition to the various modules mentioned above. For example, it may include input / output devices, network access devices, buses, processors, and memory.

[0076] The processor referred to can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. This processor is the control center of the system, connecting various parts of the user terminal via various interfaces and lines.

[0077] The aforementioned memory can be used to store computer and system programs and / or modules. The aforementioned processor performs the various functions by running or executing computer programs and / or modules stored in the memory, and by accessing data stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store the operating system, application programs required for at least one function, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital cards (SD cards), flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0078] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0079] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0080] The above-described embodiments are merely preferred embodiments of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A system for automatically generating a picture from a text, characterized in that, The system comprises: a text information acquisition module, configured to acquire a piece of description text information; the description text information is a description of a final target picture; and the description text information comprises at least a picture style and a theme; a text information import module, configured to import the description text information into a picture calling model; a picture calling module, configured to call pictures conforming to the description text information from a picture library to form a calling picture set by the picture calling model; in the training process of the picture calling model, the picture calling model automatically identifies the content in the pictures and converts the content into recognized text, which is added to the description information; when the model identifies the pictures, the model only identifies the main elements in the pictures; the main elements comprise a main color, a picture style, a theme and a scene; a parameter acquisition module, configured to acquire random values, iteration times and update step distances; and the acquired various parameters are saved to generate an iteration template; a basic picture setting module, configured to select a picture from the calling picture set as a basic picture according to the random values; the basic picture can also be obtained by uploading a specified basic picture from a user end; the specified basic picture is a patterned picture or a blank picture; when the specified basic picture is a blank picture, the values of the iteration times and the update step distances are increased accordingly; and the basic picture setting module is further configured to set a reference weight ratio to determine the similarity between the finally exported picture and the basic picture, so that a major deviation occurs in the iteration process; a basic picture iteration module, configured to import the basic picture into a picture generation model to regenerate the picture according to the iteration times and the update step distances; the update step distance is the distance of the movement of the iteration area each time; and the iteration times are the number of times of the iteration of the picture generation model to the picture; the basic picture iteration module is configured to set the size of the exported picture before iteration and update, so that the iteration and update are performed within the specified size range; and the quality of the exported picture is set when the picture is exported; the basic picture iteration module specifically comprises an iteration area setting unit and an iteration area movement unit; the iteration area setting unit is configured to set the area of the iteration area each time according to the iteration times; the iteration times are inversely proportional to the size of the iteration area each time; and according to the set area of the iteration area each time, the entire basic picture can be iterated at least once within the set iteration times; and the iteration area movement unit is configured to move the iteration area according to the update step distance each time; the iteration area starts to move from at least one side of the basic picture; and the iteration area intersects with the position of the last iteration area each time after the iteration area moves. a target picture export module, configured to export the generated target picture from the picture generation model when the iteration times are completed.

2. The system for automatically generating a picture from text of claim 1, wherein, The training method of the picture calling model comprises: importing a large number of pictures and picture description information into the picture calling model for training; establishing a corresponding relationship between the pictures and the corresponding description information by means of self-added description information and artificially added description information, and saving the pictures in a picture library.

3. The system for automatically generating a picture from text of claim 1, wherein, The training method of the picture generation model comprises: A large number of picture classification is imported into the picture generation model for training; The picture generation model is continuously learned and trained to generate pictures of corresponding themes according to specified themes; the picture generation model includes a generator and a discriminator, and the two are alternately optimized and trained by using an adversarial neural network.

4. The system for automatically generating a picture from text of claim 1, wherein, The picture calling module specifically includes: A keyword analysis unit is configured to analyze effective keywords in the description text information by the picture calling model; the effective keywords include at least one of picture style description words, picture theme description words and picture color description words; A picture search calling unit is configured to search picture description information in a picture library according to the effective keywords, copy and call out a picture corresponding to the picture description information when the picture description information contains the effective keywords, and temporarily store the picture in the calling basic picture set.

5. The system for automatically generating a picture from text of claim 1, wherein, The basic picture setting module specifically includes: A picture set sorting unit is configured to randomly sort the pictures in the calling basic picture set according to an order; A basic picture determination unit is configured to select a picture with a sequence number equal to a random value as a basic picture from the calling basic picture set according to the random value in the current random sorting of the calling basic picture set.

6. The system for automatically generating a picture from text of claim 1, wherein, The basic picture iteration module specifically includes: A basic picture import unit is configured to import the basic picture into the picture generation model; A picture modification unit is configured to change a picture area covered by the iteration area according to a picture style in the description text information by using the generator in the picture generation model.

Images