Image generation method and device capable of capturing description details, equipment and medium

Through CLIP network splitting and step-by-step rendering technology, combining Gaussian noise and color dispersion, the problems of generation stability and detail capture in the image generation model are solved, and high-quality and diverse image generation is achieved, suitable for long text content generation .

CN118918026APending Publication Date: 2024-11-08UNICOM WOYUEDU TECH CULTURE CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410844795.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-27
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

In the field of image generation models, generating stable images that are strongly related to PROMPT remains an under-resolved technical challenge. Existing methods rely on complex tool sets and cumbersome workflows, and small-sample models with stable features lead to excessive generation of images Fixation, lack of diversity and nature, suggestive words are insufficiently binding, resulting in the model ignoring important descriptive information.

Method used

By obtaining the original description text, splitting it into multiple keywords using the CLIP network, and generating the target image in combination with Gaussian noise, step-by-step rendering and resolution adjustment technology are adopted to introduce placeholder and color dispersion strategies to avoid pollution and interference. Ensure image details are captured and expressed.

Benefits of technology

It realizes better capturing human language details and generating images that are highly consistent with the original description, improving the stability and diversity of image generation, and reducing the need for manual intervention, especially in the field of long text content generation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918026A_ABST
    Figure CN118918026A_ABST
Patent Text Reader

Abstract

The invention relates to an image generation method capable of capturing description details, an image generation device capable of capturing description details, equipment and a medium. Wherein the latter segment of keywords in the multiple segments of keywords has more key information of the original description text than the previous segment of keywords. According to the method, when the human language is processed, the human language can be converted into the keyword array composed of multiple segments of keywords, compared with an original diffusion model, details in the human language can be better captured, an image highly consistent with an original description text can be generated, and the method is especially effective in the field of content generation based on long texts.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image data processing technology, and in particular to an image generation method, device, equipment and medium capable of capturing description details. Background Art

[0002] Deep learning-based generative models, such as convolutional neural networks (CNNs) and generative adversarial networks (GANs), have become the mainstream technology in the field of image generation. These models are able to generate high-quality, realistic images through a large amount of training data. GAN technology has made significant progress in recent years, from the original DCGAN to StyleGAN, and then to the recent GAN variants. GAN technology has significantly enhanced its control over image generation, gradually allowing generative models to more sensitively capture the connection between the original text and the image representation. With the leapfrog progress of industry technology, Stable Diffusion (SD) was born. SD is an artificial intelligence (AI) model based on deep learning, specially designed for generating images. It uses a model of diffusion in latent space, compresses images into latent space, and applies a diffusion process in latent space to generate new images. SD is able to generate detailed images from text descriptions, and can also be used for tasks such as image restoration, image drawing, text to image, and image to image. It converts the "image generation" process into a diffusion process that gradually removes noise, starting with random Gaussian noise, gradually removing noise through training, and ultimately outputting an image that is closer to the text description.

[0003] In the field of image generation models, generating stable images that are strongly correlated with PROMPT (prompt words) remains a technical challenge that has not been fully solved. The current state of industry technology shows that generating stable images usually requires relying on complex tool sets and cumbersome workflows.

[0004] Traditional solutions usually include: first, training a feature-stable model based on small sample data, which can capture and stabilize the features of key elements (such as characters) in the picture. Then, combined with a basic generative model (such as a generative adversarial network GAN or a variational autoencoder VAE), the information provided by the small sample model is used as a constraint to iteratively generate images. In this process, manual screening and intervention are indispensable steps to ensure that the generated images meet the preset quality requirements. However, this workflow-based approach has several significant limitations. First, due to the reliance on manual screening, the stability and efficiency of the entire process are limited. Second, the feature-stable small sample model often causes the generated images to be too fixed in storyboard and composition, lacking diversity and naturalness, thus affecting the overall viewing experience. In addition, the prompt, as a key input to guide the image generation model, is often not very constrained in practical applications, and the model is prone to ignore some important descriptive information during the decoding process. Summary of the invention

[0005] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0006] The main purpose of the embodiments of the present invention is to provide an image generation method, device, equipment and medium that can capture description details, so as to better capture the details in human language.

[0007] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present invention provides a method for generating an image capable of capturing description details, the method comprising:

[0008] Get the original description text;

[0009] Based on the CLIP network, the original description text is split into multiple segments of keywords, and the multiple segments of keywords are combined into a keyword array; wherein the latter segment of keywords in the multiple segments of keywords has more key information of the original description text than the former segment of keywords;

[0010] The target image is generated based on the image generator according to the keyword array and Gaussian noise.

[0011] In some embodiments, the multiple keywords are three keywords, and the three keywords are respectively the target of the original description text, the definition of the original description text, and the details of the original description text.

[0012] In some embodiments, the step of splitting the original description text into multiple keyword segments based on a CLIP network includes:

[0013] Extracting text features from the original description text based on a natural language processor;

[0014] Based on the CLIP network, multiple keywords are split from text features.

[0015] In some embodiments, the image generator generates an image based on the keyword array and Gaussian noise, including:

[0016] Generate an initial image according to the Gaussian noise and the three keywords;

[0017] The initial image is rendered step by step according to the three keywords to obtain a target image.

[0018] In some embodiments, in the step-by-step rendering of the initial image according to the three-segment keyword, placeholders are used to separate the three-segment keyword.

[0019] In some embodiments, rendering the initial image step by step according to the three-segment keyword to obtain the target image includes:

[0020] Rendering the initial image in steps according to the three keywords to obtain an intermediate image;

[0021] The resolution of the intermediate image is adjusted to obtain a target image.

[0022] In some embodiments, before generating the initial image according to the Gaussian noise and the three-segment keyword, the method further includes:

[0023] The plurality of groups of colors are dispersed into different segment keywords among the three segment keywords.

[0024] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present invention provides an image generation device capable of capturing description details, the image generation device capable of capturing description details comprising:

[0025] A data acquisition unit, used for acquiring original description text;

[0026] An array partitioning unit, used for splitting the original description text into multiple segments of keywords based on a CLIP network, and forming a keyword array with the multiple segments of keywords; wherein the latter segment of keywords in the multiple segments of keywords has more key information of the original description text than the former segment of keywords;

[0027] The image generating unit is used to generate a target image based on the keyword array and Gaussian noise by an image generator.

[0028] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present invention provides an electronic device, comprising: at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the image generation method that can capture descriptive details of the above-mentioned first aspect.

[0029] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the image generation method that can capture description details of the above-mentioned first aspect.

[0030] An embodiment of the present application provides an image generation method that can capture description details. The method splits the original description text into a keyword array consisting of multiple keyword segments based on a natural language processor, wherein the latter keyword segment of the multiple keyword segments has more key information of the original description text than the former keyword segment. When processing human language, the method can convert it into a keyword array consisting of multiple keyword segments. Compared with the original diffusion model, it can better capture the details in human language and generate images that are highly consistent with the original description text. It is particularly effective in the field of content generation based on long texts.

[0031] It can be understood that the beneficial effects of the second to fourth aspects compared with the related art are the same as the beneficial effects of the first aspect compared with the related art. Please refer to the relevant description in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0033] Figure 1 is a schematic flow chart of a method for generating an image capable of capturing description details provided by an embodiment;

[0034] Figure 2 is a flowchart of a method for generating an image capable of capturing description details provided by another embodiment;

[0035] Figure 3 is an architectural diagram of an image generation device that can capture description details provided by an embodiment. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0038] Deep learning-based generative models, such as convolutional neural networks (CNNs) and generative adversarial networks (GANs), have become the mainstream technology in the field of image generation. These models are able to generate high-quality, realistic images through a large amount of training data. GAN technology has made significant progress in recent years, from the original DCGAN to StyleGAN, and then to the recent GAN variants. GAN technology has significantly enhanced its control over image generation, gradually allowing generative models to more sensitively capture the connection between the original text and the image representation. With the leapfrog progress of industry technology, SD was born. SD is an artificial intelligence (AI) model based on deep learning, specially designed for generating images. It uses a model of diffusion in latent space, compresses images into latent space, and applies a diffusion process in latent space to generate new images. SD is able to generate detailed images from text descriptions, and can also be used for tasks such as image restoration, image drawing, text to image, and image to image. It converts the "image generation" process into a diffusion process that gradually removes noise, starting with random Gaussian noise, and gradually removes noise through training, and finally outputs an image that is closer to the text description.

[0039] In the field of image generation models, generating stable images that are strongly correlated with PROMPT (prompt words) remains a technical challenge that has not been fully solved. The current state of industry technology shows that generating stable images usually requires relying on complex tool sets and cumbersome workflows.

[0040] Traditional solutions usually include: first, training a feature-stable model based on small sample data, which can capture and stabilize the features of key elements (such as characters) in the picture. Then, combined with a basic generative model (such as a generative adversarial network GAN or a variational autoencoder VAE), the information provided by the small sample model is used as a constraint to iteratively generate images. In this process, manual screening and intervention are indispensable steps to ensure that the generated images meet the preset quality requirements. However, this workflow-based approach has several significant limitations. First, due to the reliance on manual screening, the stability and efficiency of the entire process are limited. Second, the feature-stable small sample model often causes the generated images to be too fixed in storyboard and composition, lacking diversity and naturalness, thus affecting the overall viewing experience. In addition, the prompt, as a key input to guide the image generation model, is often not very constrained in practical applications, and the model is prone to ignore some important descriptive information during the decoding process.

[0041] To overcome these limitations, refer to Figure 1 The present application provides some embodiments, including a method for generating an image capable of capturing description details, the method for generating an image capable of capturing description details comprising steps S110 to S130:

[0042] Step S110: Obtain original description text.

[0043] Step S120: split the original description text into multiple keyword segments based on the CLIP network, and form the multiple keyword segments into a keyword array; wherein the latter keyword segment in the multiple keyword segments has more key information of the original description text than the former keyword segment.

[0044] Step S130: Generate a target image based on the keyword array and Gaussian noise by using an image generator.

[0045] In the embodiment of the present application, the original description text is a segment of human language.

[0046] In an embodiment of the present application, the SD model includes a natural language processor, a CLIP network, and an image generator. The CLIP network is a multimodal network proposed by OpenAI, which can process text and images and realize zero-sample learning between image and text tasks. The training task of CLIP is a multimodal contrastive learning task. Given an image and a set of text descriptions, the goal of the model is to match the correct text description with the image, while mismatching other text descriptions with the image. The original description text is split into multiple segments of keywords based on the CLIP network.

[0047] In the embodiment of the present application, the target image is ultimately generated by an image generator according to a keyword array and Gaussian noise.

[0048] When processing human language, this method can convert it into a keyword array consisting of multiple keywords. Compared with the original diffusion model, it can better capture the details in human language and generate images that are highly consistent with the original description text. It is particularly effective in the field of content generation based on long texts.

[0049] In some embodiments of the present application, the multi-segment keyword is a three-segment keyword, and the three-segment keyword is respectively the target of the original description text, the definition of the original description text, and the details of the original description text.

[0050] For example, the original description text is: an oil painting of a girl sitting in the forest. Then, based on the natural language processor, three keywords are obtained, which are: [target, definition, details], that is, a painting, a great realistic oil painting, and an oil painting of a girl in the forest.

[0051] In some embodiments of the present application, splitting the original description text into multiple keyword segments based on the CLIP network includes step S1210 and step S1220:

[0052] Step S1210: extract text features from the original description text based on a natural language processor.

[0053] Step S1220: split multiple keyword segments from the text features based on the CLIP network.

[0054] In some embodiments of the present application, the image generator in step S130 generates an image according to the keyword array and Gaussian noise, including step S1310 and step S1320:

[0055] Step S1310: Generate an initial image based on Gaussian noise and three-segment keywords.

[0056] Step S1320: Render the initial image step by step according to the three keywords to obtain a target image.

[0057] In step S1320, performing step-by-step rendering can isolate and subdivide the keywords to avoid contamination.

[0058] In the above step, in which the initial image is rendered step by step according to the three segments of keywords, placeholders are used to separate the three segments of keywords. Due to the performance limitation of the original SD model, when facing long texts (especially novel texts), different descriptions may interfere with each other. This embodiment introduces placeholders, which can greatly reduce this loss.

[0059] In some embodiments of the present application, the initial image is rendered step by step according to the three-segment keyword in step S1320 to obtain the target image, including step S1321 and step S1322:

[0060] Step S1321, rendering the initial image step by step according to the three-segment keywords to obtain an intermediate image.

[0061] Step S1322: Adjust the resolution of the intermediate image to obtain the target image.

[0062] In some embodiments of the present application, before generating the initial image according to the Gaussian noise and the three-segment keyword, the method further includes:

[0063] Dispersing multiple groups of colors into different segment keywords in the three segment keywords. By dispersing multiple groups of colors into different segment keywords in the three segment keywords, the correlation between colors is reduced, thereby reducing pollution.

[0064] Reference Figure 2 For ease of understanding, an embodiment is proposed, which provides an image generation method capable of capturing description details, and the method includes the following process:

[0065] Step S910, obtain the original description text, unpack it into a combination of words and sentences of [goal, definition, details] through the CLIP network, and then parse it into numbers and store it in the input array.

[0066] Step S920: Combine the Gaussian noise input into the generator to generate an initial image.

[0067] Step S930: For the initial image, repeatedly render the initial image based on the split keywords to ensure that it is finally reflected in the output target image.

[0068] Step 5940: Use the Stable Diffusion amplifier to enlarge the image resolution of the target image and make the image detailed.

[0069] First, the original description text was obtained: an oil painting of a girl sitting in a forest.

[0070] Then, a Stable Diffusion model is built, which includes a natural language processor, a CLIP network, and an image generator.

[0071] A natural language processor extracts text features from the original description text, that is, converts it into a digital representation to capture the ideas in the text. For different texts, the amount of digital representation it converts into is also different.

[0072] The CLIP network is composed of a neural network (Encoder). The Encoder will sequentially encode the original description text one by one, and then calculate the closest keyword among many possibilities to obtain an array of multiple keyword segments. The array will be combined with Gaussian noise and input into the image generator. The array words will generate blocks at the same time, and then their blocks will be combined together. After the numbers in the array are fully input into the image generator, a blurred image is generated.

[0073] Since the existing Stable Diffusion model will automatically put the overlong input into the CLIP network in the form of a group of up to 75 inputs, pollution will occur. Therefore, this embodiment adjusts the unstable CLIP network and constructs a general description reading method. That is: the main body of a description is split into three sections, and then each short sentence is split into the entire description, and the performance of the main body is strengthened by the combined strengthening and superposition effect of multiple sections of keywords. At the same time, because the complete description of the main body is split, this method can isolate several color groups into several sections of keywords, thereby reducing pollution. Finally, in the image generator, according to the needs, use a step-by-step renderer, etc. to adjust the content or order of some sentences to make the picture closer to the required situation.

[0074] After modification based on the CLIP network, it becomes: a painting, a great realistic oil painting, a girl in the forest. The three-segment CLIP network splits the original text into the expected: [goal, definition, details]. This completes a description that can be subdivided and arrayed.

[0075] Image Generator,The image generator gradually enlarges the image and enriches the details in the process.

[0076] (3) The image generator gradually enlarges the image and enriches the details in the process.

[0077] The image generator is composed of a neural network encoder, which uses placeholders to separate the front and back to avoid contamination. Using multiple descriptions to superimpose and strengthen can have the effect of isolating contamination. Dispersing multiple groups of colors into different natural segments reduces the correlation between colors, thereby reducing contamination. The principles of the two are the same, both of which separate elements that are easily contaminated, thereby reducing confusion when generating images.

[0078] This embodiment has at least the following beneficial effects:

[0079] (1) This embodiment proposes an innovative solution, which is to allow the SD diffusion model to better understand human language and generate high-quality images by modifying the CLIP network.

[0080] (2) Generate high-quality images covering details;

[0081] The original SD has difficulty in controlling the details of a human language and lacks relevant processing measures. This embodiment enters the layered renderer from the outside so that the details can be correctly reflected;

[0082] (3) Pollution isolation;

[0083] Due to the performance limitations of the original SD model, different descriptions may interfere with each other when faced with long texts (especially novel texts). This method introduces placeholders, which can greatly alleviate this loss.

[0084] In summary, the modification of CLIP network enables it to convert human language into an array of [goal, definition, details] when processing it, and introduces distributed renderers and placeholders to isolate and segment keywords. Compared with the original diffusion SD model, it can better capture the details in human language and generate images that are highly consistent with the original description text, which is particularly effective in the field of content generation based on long text.

[0085] Reference Figure 3 This embodiment provides an image generation device capable of capturing description details, and the image generation device capable of capturing description details includes:

[0086] The data acquisition unit 1100 is used to acquire the original description text.

[0087] The array partitioning unit 1200 is used to split the original description text into multiple keyword segments based on the CLIP network, and form the multiple keyword segments into a keyword array; wherein the latter keyword segment in the multiple keyword segments has more key information of the original description text than the former keyword segment.

[0088] The image generating unit 1300 is used to generate a target image based on an image generator according to a keyword array and Gaussian noise.

[0089] It should be noted that the present device embodiment and the above-mentioned method embodiment are based on the same inventive concept, so the relevant contents of the above-mentioned method embodiment are also applicable to the present device embodiment and will not be repeated here.

[0090] The present application also provides an electronic device, which includes:

[0091] at least one memory;

[0092] at least one processor;

[0093] at least one program;

[0094] The programs are stored in the memory, and the processor executes at least one program to implement the image generation method capable of capturing description details as described above in the present disclosure.

[0095] The electronic device may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.

[0096] The electronic device according to the embodiment of the present application is described in detail below.

[0097] The processor 1600 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0098] The memory 1700 may be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 may store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the program code is stored in the memory 1700, and the processor 1600 calls and executes the image generation method capable of capturing description details in the embodiments of the present invention.

[0099] Input / output interface 1800, used to implement information input and output;

[0100] The communication interface 1900 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0101] Bus 2000 , which transmits information between various components of the device (e.g., processor 1600 , memory 1700 , input / output interface 1800 , and communication interface 1900 );

[0102] The processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 are connected to each other in communication within the device via the bus 2000 .

[0103] An embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned method for generating an image that can capture description details.

[0104] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0105] The embodiments described in the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0106] Those skilled in the art will appreciate that the technical solutions shown in the figures do not limit the embodiments of the present invention and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0107] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0108] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0109] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0110] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0111] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0112] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0113] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0114] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store programs.

[0115] The above is a specific description of the preferred implementation of the embodiments of the present application, but the embodiments of the present application are not limited to the above-mentioned implementation methods. Technical personnel familiar with the field can also make various equivalent modifications or substitutions without violating the spirit of the embodiments of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the embodiments of the present application.

Claims

1. A method for generating an image capable of capturing descriptive details, characterized in that: The method for generating an image capable of capturing description details comprises: Get the original description text; Based on the CLIP network, the original description text is split into multiple segments of keywords, and the multiple segments of keywords are combined into a keyword array; wherein the latter segment of keywords in the multiple segments of keywords has more key information of the original description text than the former segment of keywords; The target image is generated based on the image generator according to the keyword array and Gaussian noise.

2. The method for generating an image capable of capturing description details according to claim 1, characterized in that: The multiple keywords are three keywords, and the three keywords are respectively the target of the original description text, the definition of the original description text, and the details of the original description text.

3. The method for generating an image capable of capturing description details according to claim 2, characterized in that: The method of splitting the original description text into multiple keyword segments based on the CLIP network includes: Extracting text features from the original description text based on a natural language processor; Based on the CLIP network, multiple keywords are split from text features.

4. The method for generating an image capable of capturing description details according to claim 2, characterized in that: The image generator generates an image according to the keyword array and Gaussian noise, including: Generate an initial image according to the Gaussian noise and the three keywords; The initial image is rendered step by step according to the three keywords to obtain a target image.

5. The method for generating an image capable of capturing description details according to claim 3, characterized in that: In rendering the initial image step by step according to the three sections of keywords, placeholders are used to separate the three sections of keywords.

6. The method for generating an image capable of capturing description details according to claim 3, characterized in that: The step of rendering the initial image according to the three keywords to obtain the target image includes: Rendering the initial image in steps according to the three keywords to obtain an intermediate image; The resolution of the intermediate image is adjusted to obtain a target image.

7. The method for generating an image capable of capturing description details according to claim 3, characterized in that: Before generating the initial image according to the Gaussian noise and the three-segment keywords, the method further includes: The plurality of groups of colors are dispersed into different segment keywords among the three segment keywords.

8. An image generation device capable of capturing descriptive details, characterized in that: The image generating device capable of capturing description details comprises: A data acquisition unit, used for acquiring original description text; An array partitioning unit, used for splitting the original description text into multiple segments of keywords based on a CLIP network, and forming a keyword array with the multiple segments of keywords; wherein the latter segment of keywords in the multiple segments of keywords has more key information of the original description text than the former segment of keywords; The image generating unit is used to generate a target image based on the keyword array and Gaussian noise by an image generator.

9. An electronic device, characterized in that: include: at least one control processor and a memory for communicatively coupling with the at least one control processor; The memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to perform the image generation method capable of capturing descriptive details as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the image generation method capable of capturing description details as described in any one of claims 1 to 7.