Image generation method and method for processing data for image generation

Abstract

Embodiments of the present disclosure provide an image generation method and a method for processing data for image generation. The image generation method comprises: acquiring an initial prompt of an image generation task; inputting the initial prompt into a language model, and obtaining a target prompt, the language model being obtained on the basis of sample prompt pair training, said training comprising performing reinforcement learning on the basis of feedback information of a predicted prompt outputted by a language model to be trained, and the feedback information representing the image quality of a reference image pre-generated by the predicted prompt; and on the basis of the target prompt, executing the image generation task by using an image generation model, and obtaining a target image. According to the described language model, on the basis of a sample prompt pair, targeted reinforcement learning for prompt optimization is carried out on the image quality of a pre-generated reference image, targeted optimization is carried out on an initial prompt, a visually aligned target image is generated, the image quality of the generated image is improved, image generation efficiency is improved, and image generation costs are reduced.

Description

Image generation method and data processing method for image generation

[0001] Cross-reference

[0002] This disclosure claims priority to a Chinese patent application filed with the Patent Office of China on September 13, 2023, with application number 202311192928X and invention name “Image Generation Method and Data Processing Method for Image Generation”, the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0003] The embodiments of the present disclosure relate to the field of deep learning technology, and in particular to an image generation method. Background Art

[0004] With the development of deep learning technology, image generation technologies represented by Text-to-Image Synthesis (TIS) are improving production efficiency and results in multiple fields, meeting the needs of image generation.

[0005] Currently, text-to-image synthesis technology uses image generation models based on prompt text to generate corresponding images. The quality of the generated image depends on the quality of the prompt text content.

[0006] However, there's a high degree of uncertainty about whether the prompt text meets the requirements for image generation, leading to a similarly high degree of uncertainty in the quality of the generated image. In real-world image generation scenarios, when prompt text often fails to meet these requirements, repeated trial-and-error attempts to modify the prompt text and regenerate the image are often necessary. This results in significant time and computational resource consumption, and the resulting image quality is often subpar. Therefore, an efficient, low-cost, and high-quality image generation method is urgently needed.

[0007] Summary of the Invention

[0008] In view of this, embodiments of the present disclosure provide an image generation method. Embodiments of the present disclosure also relate to another image generation method, a data processing method for image generation, an image generation device, another image generation device, a data processing device for image generation, a computing device, a computer-readable storage medium, and a computer program to address technical deficiencies in the prior art.

[0009] The present disclosure provides an image generation method, including:

[0010] Get the initial prompt text for the image generation task;

[0011] Inputting an initial prompt text into a language model to obtain a target prompt text, wherein the language model is trained based on a sample prompt pair, the sample prompt pair including a first sample prompt text and a second sample prompt text, the first sample prompt text having less text content than the second sample prompt text, the training including performing reinforcement learning based on feedback information of a predicted prompt text output by the language model to be trained, the predicted prompt text being output after inputting the first sample prompt text into the language model to be trained, the feedback information representing an image quality of a reference image pre-generated from the predicted prompt text;

[0012] Based on the target prompt text, an image generation task is performed using an image generation model to obtain a target image, wherein the image generation model is a model adapted to the language model.

[0013] The present disclosure also provides an image generation method, which is applied to a cloud-side device and includes:

[0014] Receive the initial prompt text of the image generation task sent by the front end;

[0015] Inputting an initial prompt text into a language model to obtain a target prompt text, wherein the language model is trained based on a sample prompt pair, the sample prompt pair including a first sample prompt text and a second sample prompt text, the first sample prompt text having less text content than the second sample prompt text, the training including performing reinforcement learning based on feedback information of a predicted prompt text output by the language model to be trained, the predicted prompt text being output after inputting the first sample prompt text into the language model to be trained, the feedback information representing an image quality of a reference image pre-generated from the predicted prompt text;

[0016] Based on the target prompt text, an image generation task is performed using an image generation model to obtain a target image, wherein the image generation model is a model adapted to the language model;

[0017] Feed the target image back to the front end.

[0018] The present disclosure also provides a data processing method for image generation, which is applied to a cloud-side device and includes:

[0019] Acquire a target sample set, wherein the target sample set includes a plurality of sample prompt texts;

[0020] Extracting a target sample prompt text from the target sample set, wherein the target sample prompt text is any one of a plurality of sample prompt texts;

[0021] Input the target sample prompt text into the language model to be trained to obtain the target predicted prompt text;

[0022] Evaluate the image quality of the reference image pre-generated for the target prediction prompt text to obtain feedback information of the target prediction prompt text;

[0023] Based on the feedback information of the target prediction prompt text, the language model to be trained is reinforced to obtain the language model after reinforcement learning;

[0024] Feedback the model parameters of the language model after reinforcement learning to the end device.

[0025] In another embodiment of the present disclosure, an electronic device is provided. The electronic device includes: a memory and a processor, wherein:

[0026] memory and processor;

[0027] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, any of the above methods is implemented.

[0028] In yet another embodiment of the present disclosure, the present disclosure provides a computer program product, including a computer program, which implements any of the above methods when executed by a processor.

[0029] In another embodiment of the present disclosure, the present disclosure provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above methods is implemented.

[0030] In yet another embodiment of the present disclosure, the present disclosure provides a computer program, which implements any of the above methods when executed by a processor.

[0031] In another embodiment of the present disclosure, a computer-readable storage medium is provided, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, any of the above methods can be implemented.

[0032] In an embodiment of the present disclosure, an initial prompt text of an image generation task is obtained; the initial prompt text is input into a language model to obtain a target prompt text, wherein the language model is trained based on a sample prompt pair, the sample prompt pair includes a first sample prompt text and a second sample prompt text, the text content of the first sample prompt text is lower than the text content of the second sample prompt text, the training includes reinforcement learning based on feedback information of a predicted prompt text output by the language model to be trained, the predicted prompt text is output after the first sample prompt text is input into the language model to be trained, and the feedback information represents the image quality of a reference image pre-generated by the predicted prompt text; based on the target prompt text, an image generation task is performed using an image generation model to obtain a target image, wherein the image generation model is a model adapted to the language model. The language model is based on sample prompt pairs and performs targeted reinforcement learning on the image quality of pre-generated reference images to optimize the prompt text. This allows the trained language model to perform targeted optimization of the image quality of the pre-generated reference images for the initial prompt text, align the prompt texts at the visual level, and generate visually aligned target images, thereby improving the image quality of the generated images and better meeting the image generation requirements. There is no need for repeated trial and error, modifying the prompt text without a clear direction, and regenerating the image, which avoids a large amount of time and computing resources loss, improves image generation efficiency, and reduces image generation costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] FIG1 is a flow chart of an image generation method provided by an embodiment of the present disclosure;

[0034] FIG2 is a schematic diagram of a flow chart of sample expansion in an image generation method provided by an embodiment of the present disclosure;

[0035] FIG3 is a flow chart of language model training in an image generation method provided by an embodiment of the present disclosure;

[0036] FIG4 is a schematic diagram showing a front-end display in an image generation method provided by an embodiment of the present disclosure;

[0037] FIG5 is a flowchart of another image generation method provided by an embodiment of the present disclosure;

[0038] FIG6 is a flow chart of a data processing method for image generation provided by an embodiment of the present disclosure;

[0039] FIG7 is a flowchart of a processing process of an image generation method applied to a cartoon picture provided by an embodiment of the present disclosure;

[0040] FIG8 is a schematic structural diagram of an image generating device provided by an embodiment of the present disclosure;

[0041] FIG9 is a schematic structural diagram of another image generating device provided by an embodiment of the present disclosure;

[0042] FIG10 is a schematic structural diagram of a data processing device for image generation provided by an embodiment of the present disclosure;

[0043] FIG11 is a structural block diagram of a computing device provided by an embodiment of the present disclosure;

[0044] FIG12 is a structural block diagram of a processor provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0045] The following description sets forth many specific details to facilitate a full understanding of the present disclosure. However, the present disclosure can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present disclosure. Therefore, the present disclosure is not limited to the specific implementations disclosed below.

[0046] The terms used in the embodiments of the present disclosure are for the purpose of describing specific embodiments only and are not intended to limit the embodiments of the present disclosure. The singular forms "a," "an," "the," and "the" used in the embodiments of the present disclosure and the appended claims are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the embodiments of the present disclosure refers to and includes any and all possible combinations of one or more of the associated listed items.

[0047] It should be understood that although the terms first, second, etc. may be used to describe various information in the embodiments of the present disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the embodiments of the present disclosure, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0048] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0049] In the disclosed embodiments, a large model refers to a deep learning model with large-scale model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even more than ten trillion model parameters. A large model can also be called a cornerstone model / foundation model. By pre-training a large model with large-scale unlabeled corpus, a pre-trained model with more than 100 million parameters is produced. This model can adapt to a wide range of downstream tasks and has good generalization capabilities, such as a large language model (LLM) and a multi-modal pre-training model.

[0050] When large models are used in practice, only a small number of samples are needed to fine-tune the pre-trained model and it can be applied to different tasks. Large models can be widely used in natural language processing (NLP), computer vision and other fields. Specifically, they can be applied to computer vision tasks such as visual question answering (VQA), image caption (IC), and image generation, as well as natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. The main application scenarios of large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0051] First, the terms involved in the embodiments of the present disclosure are explained.

[0052] Stable Diffusion: A deep learning model for text-to-image synthesis. Given a prompt text, Stable Diffusion generates random tensors in the latent space through forward diffusion and reverse diffusion. The random tensors can be controlled by setting a random seed. A latent image is generated in the latent space. The latent image and prompt text are used as input. The latent noise in the latent space is predicted, and the latent noise is removed from the latent image to generate a new latent image. This step is repeated multiple times, and a decoder is used to convert the latent image to pixel space to obtain the target image. Stable Diffusion models include but are not limited to the DALLE series of models, the Midjourney model, and the Imagen model.

[0053] Bootstrap Language-Image Pre-training (BLIP) model: A multimodal pre-training model that uses a large amount of image and text data for training. It can be used for a variety of image and text tasks, including image generation, text generation, image classification, and text classification.

[0054] Contrastive Language-Image Pre-Training (CLIP) model: A multimodal pre-training model that uses a large amount of image and text data for training. It can be used for a variety of image and text tasks, including image generation, text generation, image classification, and text classification.

[0055] Large Language Model (LLM) model: A large language model with large model parameters. It uses a large amount of text data for training and can achieve text understanding and generation.

[0056] User preference feedback information (Pick Score, abbreviated as PS): a feedback score used to measure the preferences of a large number of users for images.

[0057] Image aesthetic feedback information (Aesthetic Score, abbreviated as AES): used to measure the feedback score of the image itself.

[0058] Reinforcement learning: A training technique for deep learning models that simulates the interaction between deep learning models and application scenarios to learn optimization strategies. The basic idea is to learn optimization strategies through continuous prediction and feedback, achieving continuous improvement based on feedback information.

[0059] Proximal Policy Optimization (PPO): A gradient strategy in reinforcement learning used to solve policy-based reinforcement learning problems. It is a gradient-based optimization algorithm that can be used to update the parameters of the policy model so that the policy model's prediction results are closer to the optimized policy. The policy network parameters are updated by minimizing the policy's loss function. Strength information techniques are used to smooth the policy update process to avoid excessive policy updates that may lead to model instability. In addition, the learning rate parameter is used to control the policy update speed to ensure model stability and convergence. It has good stability and convergence, and can achieve good performance in complex environments. It can also be used to solve various types of reinforcement learning problems.

[0060] REINFORCE strategy: A gradient strategy in reinforcement learning that uses the Monte Carlo method to estimate the expected value of the feedback information obtained from taking an action in each state, and then uses these estimates to calculate the policy gradient and update the policy parameters.

[0061] Trust Region Policy Optimization (TRPO): A gradient strategy in reinforcement learning that constructs a function L to satisfy the objective function of using function L instead of gradient update on the trust region.

[0062] KL penalty (Kullback-Leibler divergence penalty, abbreviated as KL): a measurement method for measuring the difference between two probability distributions. KL penalty is commonly used in fields such as machine learning and natural language processing to evaluate model performance and optimize model parameters. The specific definition of KL penalty is: KL(P||Q) = ∑x P(x)log(P(x) / Q(x)). Among them, P and Q are two probability distributions, x is the value of the random variable, and log is the natural logarithm function. The smaller the value of the KL penalty, the smaller the difference between the two probability distributions and the better the performance of the model. Therefore, in fields such as machine learning and natural language processing, KL penalty is often used to evaluate model performance, and the value of KL penalty is reduced by optimizing the parameters of the model.

[0063] Currently, the prompt project aims to explore how to provide image generation models with prompt text that meets image generation requirements. This has significantly improved the quality of generated images, increased image generation efficiency, and reduced image generation costs. For example, users can use large-scale language models to generate prompt text based on keywords, or optimize simple prompt text to produce high-quality prompt text.

[0064] However, directly using a large-scale language model to generate or optimize prompt text lacks a clear optimization direction for the prompt text, making it difficult to directly optimize the prompt text to optimize the image quality of the pre-generated reference image. For example, a simple prompt text is "a majestic sailing ship." Directly using a large-scale language model to optimize the prompt text to obtain the prompt text "a majestic sailing ship, with gorgeous patterns decorated on the hull and exquisite patterns embroidered on the sails. It rides the wind and waves on the sea, stirring up layers of waves." Based on this prompt text, the generated image is a Chinese-style sailing ship. The shot of the sailing ship in the image is a close-up shot, and the ship type is a shallow-water sailing ship, which does not meet the generated image requirements of "Western-style sailing ship, long-range shot, ocean-going sailing ship." The optimized prompt text needs to be repeatedly modified.

[0065] In response to the above problems, the present disclosure provides an image generation method. The present disclosure also relates to another image generation method, a data processing method for image generation, an image generation device, another image generation device, a data processing device for image generation, a computing device, a computer-readable storage medium and a computer program, which are described in detail one by one in the following embodiments.

[0066] Referring to FIG1 , FIG1 shows a flow chart of an image generation method provided by an embodiment of the present disclosure, which includes the following specific steps:

[0067] Step 102: Obtain the initial prompt text for the image generation task.

[0068] The embodiments of the present disclosure are applied to applications, websites, or mini-programs that have prompt text optimization and image generation functions. Prompt text optimization is achieved through a language model deployed on the application, website, or mini-program, and image generation is achieved through an image generation model deployed on the application, website, or mini-program.

[0069] Image generation tasks involve generating target images using image generation models. These tasks are applied to a variety of fields, including computer graphics, artistic creation, and visual effects. These tasks include, but are not limited to, game map generation, game model generation, video image generation, art painting generation, engineering model generation, and medical imaging generation. Image generation tasks can be selected by the user via the front-end or directly included in the initial prompt text, without limitation.

[0070] The initial prompt text is a natural language text that is not optimized and is used to indirectly guide the image generation model to understand the image generation task and generate the corresponding target image. For example, the initial prompt text for the game map generation task is "Please generate a castle map with corridors and secret passages." Another example is the initial prompt text for the art painting generation task is "Please generate an oil painting depicting the scenery of a certain place and bay." The initial prompt text can be selected by the user in the front-end.

[0071] For example, on an AI technology website, the website deploys a Large-Scale Language Model (LLM) and a Stable Diffusion model (Stable Diffusion). A user logs in to the website's front-end, selects "Artwork Generation Task" on the image generation page, identifies the image generation task as an art painting generation task, and enters the corresponding initial prompt text "a girl" in the input box. The website's server then retrieves the initial prompt text for the art painting generation task.

[0072] Obtain the initial prompt text for the image generation task. This provides the data foundation for prompt text optimization and image generation, which is a suggestive natural language text.

[0073] Step 104: Input the initial prompt text into the language model to obtain the target prompt text, wherein the language model is trained based on a sample prompt pair, the sample prompt pair includes a first sample prompt text and a second sample prompt text, the text content of the first sample prompt text is less than the text content of the second sample prompt text, the training includes reinforcement learning based on feedback information of the predicted prompt text output by the language model to be trained, the predicted prompt text is output after the first sample prompt text is input into the language model to be trained, and the feedback information represents the image quality of a reference image pre-generated by the predicted prompt text.

[0074] The language model is a natural language model with prompt text optimization capabilities. Prompt text optimization can be performed by expanding or rewriting the prompt text. For prompt text expansion, the language model retrieves related text based on the initial prompt text and uses this text to expand the initial prompt text. For example, if the initial prompt text is "a girl," the language model uses a search engine to retrieve the related text "big eyes," "pink cheeks," and "two long braids." It then uses this related text to expand the initial prompt text and obtain the target prompt text: "a girl with big eyes, pink cheeks, and two long braids." For prompt text rewriting, the language model rewrites the initial prompt text. For example, if the initial prompt text is "a girl," the language model rewrites the initial prompt text and obtains the target prompt text: "a doll-like girl."

[0075] The target hint text is optimized, suggestive natural language text that directly guides the image generation model to understand the image generation task and generate the corresponding target image. The target hint text and the initial hint text describe the same object, but the target hint text contains more text than the initial hint text. This can be understood as the target hint text providing a more detailed natural language description of the object than the initial hint text. For example, the initial hint text for a game map generation task might be "Generate a castle map with corridors and secret passages." The target hint text might be "Generate a medieval castle map with numerous maze-like corridors and secret passages." Another example might be "Generate an oil painting depicting a bay in Location A." The target hint text might be "Generate an Impressionist-style oil painting depicting a bay in Location A." It should be noted that there is at least one target hint text, and after obtaining the target hint text, at least one target hint text can be rendered on the front-end page for further user selection.

[0076] A sample prompt pair is a pair of prompt texts used as sample data in language model training. Any sample prompt pair consists of two sample prompt texts describing the same object: a first sample prompt text and a second sample prompt text. The first sample prompt text has a lower amount of text content, while the second sample prompt text has a higher amount of text content. The first sample prompt text has a lower amount of text content than the second sample prompt text. The first sample prompt text can serve as training data, while the second sample prompt text can serve as label data to train the language model. For example, a sample prompt pair includes two sample prompt texts describing the same object, "cat": "Please generate a picture of a cat." and "Please generate a picture of a cute kitten napping in the sun, with soft and smooth fur, eyes closed, and looking very relaxed and comfortable." The first sample prompt text has a lower amount of text content, while the second sample prompt text has a higher amount of text content.

[0077] The predicted prompt text is the optimized, suggestive natural language text output by the language model during training. The predicted prompt text describes the same object as the sample prompt pair and serves as prediction data for language model training. The predicted prompt text is directly output by the language model after inputting the sample prompt text with the least textual content from the sample prompt pair. For example, if the sample prompt text "Please generate a picture of a cat" is input into the language model, the predicted prompt text output will be "Please generate a picture of a lazy cat, stretched out, looking relaxed and at ease." The reference image is a pre-generated image for the predicted prompt text. This pre-generated image can be generated or not. The image quality of the reference image indicates how well it matches the image generation requirements; the higher the match, the higher the image quality. The image generation requirements are the general requirements for the image generation task. For example, for the image generation task of generating a picture of a cat, the general requirement is to generate a picture of a furry, young cat, rather than a picture of a hairless, elderly cat. Therefore, the image quality of the reference image generated for the predicted prompt text "Please generate a picture of a furry, young cat" is higher than the image quality of the reference image generated for the predicted prompt text "Please generate a picture of a hairless, elderly cat." For another example, for the image generation task of generating a landscape image, the general image generation requirement is to generate a panoramic landscape image, not a close-up landscape image. Therefore, the image quality of the reference image generated for the predicted prompt text "Please generate a panoramic landscape image of a bay in location A" is higher than the image quality of the reference image generated for the predicted prompt text "Please generate a close-up landscape image of a bay in location A."

[0078] The feedback information represents the image quality of the reference image pre-generated for the predicted prompt text, and represents the degree of match between the predicted prompt text and the image generation requirements. The higher the degree of match, the higher the image quality and the higher the feedback information. The feedback information corresponds to at least one image quality dimension, including but not limited to: user preference dimension and image aesthetic dimension. The user preference dimension can be understood as an evaluation dimension of whether the generated image meets the user's preferences. The more it meets the user's preferences, the higher the image quality. For example, if the prompt text is "Please generate a picture of a lotus pond blooming on a rainy day", from the perspective of user preference, the user is more inclined to generate a picture of a rainy day with lotus flowers blooming, rather than a sunny day with no lotus flowers. The image aesthetic dimension can be understood as an evaluation dimension that determines whether the generated image meets image aesthetic standards (abstract human evaluation standards for image aesthetics). The more it meets the image aesthetic standards, the higher the image quality. For example, if the prompt text is "Please generate an oil painting of an apple," from the image aesthetic dimension, an image ratio of 16:9 has higher image quality than an image ratio of 1:1, and an image with an apple placed at a ratio of (0.618:0.382) has higher image quality than an image with the apple placed in the center. Feedback information is determined by a preset feedback algorithm. This feedback algorithm can be a pre-trained feedback evaluation model for image generation requirements under at least one image quality dimension (e.g., a user preference evaluation model and / or an image aesthetic evaluation model), or a pre-trained feedback model, which is not limited here. It should be noted that the feedback information is essentially a feedback signal in reinforcement learning about whether the behavior of the predicted prompt text output by the language model meets the expectations. The more it meets the expectations, the more positive feedback it will get, and the less it meets the expectations, the more negative feedback it will get. Through such a feedback mechanism, the language model can "learn" the general image generation requirements of the image generation task, and optimize the prompt text in a targeted manner according to the general image generation requirements. The prompt text is aligned at the visual level, and a visually aligned target image can be generated in step 106.

[0079] The training of the language model includes supervised training and reinforcement learning based on sample prompt pairs. The reinforcement learning paradigm of the language model includes: (1) inputting the training data (sample prompt text with low text content) into the language model, outputting the predicted prompt text, directly determining the feedback information of the predicted prompt text, and performing reinforcement learning on the language model based on the feedback information to obtain the language model after reinforcement learning; (2) inputting the training data into the language model, outputting the predicted prompt text, generating a reference image based on the predicted prompt text using the image generation model, determining the image quality of the reference image as feedback information for the predicted prompt text, and performing reinforcement learning on the language model based on the feedback information to obtain the language model after reinforcement learning. Of the two training paradigms mentioned above, the second one introduces the image generation model, which increases the number of model parameters of the overall reinforcement learning, increases the complexity, and reduces the efficiency of reinforcement learning.

[0080] The initial prompt text is input into the language model to obtain the target prompt text. Specifically, the initial prompt text is input into the language model, and the initial prompt text is optimized to obtain the target prompt text. The prompt text optimization includes prompt text expansion or prompt text rewriting.

[0081] For example, the initial prompt text "a girl" is input into the large-scale language model LLM, and the prompt text is rewritten to obtain 5 target prompt texts: "A portrait of a girl, the girl is sighing, full of melancholy, sad and confused, with a hint of sorrow."; "A portrait of a girl, the girl is running and jumping, full of energy, healthy and charming, like the most dazzling sunshine in summer."... "A portrait of a girl, the girl is painting, quiet and elegant, her focused eyes reflecting the warm autumn sun." The 5 target prompt texts are fed back to the website front end and rendered on the image generation page.

[0082] An initial prompt text is input into a language model to obtain a target prompt text, wherein the language model is trained based on sample prompts. The training includes reinforcement learning based on feedback information of the predicted prompt text output by the language model, and the feedback information represents the image quality of a reference image pre-generated by the predicted prompt text.

[0083] Step 106: Based on the target prompt text, use the image generation model to perform the image generation task to obtain the target image, wherein the image generation model is a model adapted to the language model.

[0084] The image generation model can be a model adapted to the language model. The image generation model is an image generation model with text-to-image synthesis function. The image generation model understands and performs the image generation task based on the prompt text to obtain the target image. The image generation model generates the corresponding target image by mapping the input prompt text to the latent space (Latent Space), processing the feature vector in the latent space, and then mapping the processed feature vector to the image space (for example, pixel space). In the embodiment of the present disclosure, the model parameters of the image generation model are fixed, and the training process of the language model can be understood as adaptive training of the image generation model, in order to better adapt the performance of the image generation model. Image generation models include but are not limited to: Stable Diffusion model, DALLE series model, Midjourney series model and Imagen model.

[0085] The target image is a visual image generated based on the target prompt text. The target image is visually aligned with the target prompt text. Compared to visual images generated directly based on the initial prompt text, the target image more closely matches the image generation requirements and has higher image quality. For example, if the target prompt text is "Generate a medieval castle map with numerous maze-like corridors and secret passages," the target image is a painting-style map of a 12th-century Gothic castle with three floors, 21 corridors, and 43 secret passages.

[0086] There is at least one target prompt text. After obtaining the target prompt text, the at least one target prompt text is rendered on the front-end page for further selection by the user. Based on the target prompt text selected by the user, an image generation task can be performed using an image generation model to obtain a target image.

[0087] For example, user A selects "Portrait of a girl, painting, serene and elegant, her focused eyes reflecting the warm autumn sun" from among five target prompts displayed on the website's image generation page. Based on this target prompt, the website's server uses the Stable Diffusion model to perform the art painting generation task, generating an oil painting-style portrait: a girl painting, illuminated by autumn sunlight, wearing a beret, holding an oil paintbrush in her right hand and a paint palette in her left, facing a vertical drawing board. This oil painting-style portrait is then fed back to the website's front-end and rendered on the image generation page.

[0088] In an embodiment of the present disclosure, an initial prompt text for an image generation task is obtained; the initial prompt text is input into a language model to obtain a target prompt text, wherein the language model is trained based on sample prompt pairs, and the training includes reinforcement learning based on feedback information of the predicted prompt text output by the language model, wherein the feedback information represents the image quality of a reference image pre-generated by the predicted prompt text; based on the target prompt text, an image generation task is performed using an image generation model to obtain a target image. The language model performs targeted reinforcement learning on the image quality of the pre-generated reference image based on the sample prompt pairs, so that the trained language model can perform targeted optimization on the image quality of the pre-generated reference image for the initial prompt text, align the prompt text at the visual level, generate a visually aligned target image, improve the image quality of the generated image, better meet the image generation requirements, and eliminate the need for trial and error to modify the prompt text without a clear direction and regenerate the image, thereby avoiding a large loss of time and computing resources, improving image generation efficiency, and reducing image generation costs.

[0089] In an optional embodiment of the present disclosure, before step 104, the following specific steps may be further included:

[0090] Acquire a target sample set, wherein the target sample set includes a plurality of sample prompt texts;

[0091] Extracting a target sample prompt text from the target sample set, wherein the target sample prompt text is any one of a plurality of sample prompt texts;

[0092] Input the target sample prompt text into the language model to be trained to obtain the target predicted prompt text;

[0093] Evaluate the image quality of the reference image pre-generated for the target prediction prompt text to obtain feedback information of the target prediction prompt text;

[0094] Based on the feedback information of the target prediction prompt text, the language model to be trained is reinforced learning to obtain a language model after reinforcement learning.

[0095] The target sample set is a collection of sample prompt texts used for reinforcement learning of language models. This target sample set can be constructed manually, directly obtained from open source sample data, or obtained from public data using data acquisition tools, without limitation.

[0096] The sample prompt text is a suggestive natural language sample text used for reinforcement learning, and the sample prompt text has a corresponding description object.

[0097] The target sample prompt text is the sample prompt text that has not been optimized and is the training data in reinforcement learning.

[0098] The target prediction prompt text is the optimized prompt natural language text output by the language model in reinforcement learning. The target prediction prompt text has the same descriptive object as the target sample prompt text. It is the prediction data in reinforcement learning and realizes the evaluation of the reinforcement learning of the language model.

[0099] The feedback information of the target prediction prompt text represents the image quality of the reference image pre-generated by the target prediction prompt text, and represents the degree of matching between the target prediction prompt text and the image generation requirements. The higher the matching degree, the higher the image quality and the higher the feedback information.

[0100] The target sample prompt text is input into the language model to obtain the target predicted prompt text. The specific method is as follows: the target sample prompt text is input into the language model, the target sample prompt text is optimized, and the target predicted prompt text is obtained, wherein the prompt text optimization includes prompt text expansion or prompt text rewriting.

[0101] The image quality of the reference image pre-generated for the target prediction prompt text is evaluated to obtain feedback information of the target prediction prompt text. The specific method is: through a preset feedback algorithm, the image quality of the reference image pre-generated for the target prediction prompt text is evaluated to obtain feedback information of the target prediction prompt text.

[0102] Based on the feedback information from the target prediction prompt text, reinforcement learning is performed on the language model to be trained to obtain a language model after reinforcement learning. The specific method is as follows: Based on the feedback information from the target prediction prompt text, a preset gradient strategy is used to perform reinforcement learning on the language model to obtain a language model after reinforcement learning. The preset gradient strategy refers to updating the language model using gradient descent so that the language model can maximize the expected feedback information, that is, reinforcement learning is performed with the goal of increasing feedback information. Preset gradient strategies include but are not limited to: PPO strategy, REINFORCE strategy, and TRPO strategy.

[0103] The end condition of reinforcement learning may be a feedback information threshold, a reinforcement learning state threshold, or a reinforcement learning time threshold, which is not limited here.

[0104] Exemplarily, a target sample set is obtained, where the target sample set includes 10,000 sample prompt texts Sample Prompt_i (i∈[1,10000]), any one of which is extracted from the target sample set as the target sample prompt text Target Sample Prompt, the target sample prompt text Target Sample Prompt is input into the large-scale language model LLM, the target sample prompt text is rewritten to obtain the target prediction prompt text Target Predict Prompt, the image quality of the reference image pre-generated for the target prediction prompt text is evaluated through a preset feedback algorithm, feedback information Feedback of the target prediction prompt text is obtained, based on the feedback information of the target prediction prompt text, the large-scale language model LLM is reinforced learning using a preset PPO strategy, and the step of extracting any one of which from the target sample set as the target sample prompt text Target Sample Prompt is returned to execute until the feedback information reaches a preset feedback information threshold, thereby obtaining the large-scale language model LLM after reinforcement learning.

[0105] In the embodiment of the present disclosure, the image quality of the reference image pre-generated for the target prediction prompt text output by the language model is evaluated to obtain feedback information of the target prediction prompt text. Based on the feedback information, the language model is reinforced learned so that the language model after reinforcement learning can perform targeted optimization on the image quality of the reference image pre-generated for the initial prompt text, align the prompt text at the visual level, and subsequently generate a visually aligned target image, thereby improving the image quality of the subsequently generated image.

[0106] In an optional embodiment of the present disclosure, based on the feedback information of the target prediction prompt text, reinforcement learning is performed on the language model to be trained to obtain the language model after reinforcement learning, including the following specific steps:

[0107] Determining strength information for adjusting parameters of a language model;

[0108] Determine the learning rate of the language model based on the feedback and strength information of the target prediction prompt text;

[0109] Based on the learning rate, reinforcement learning is performed on the language model to be trained to obtain a language model after reinforcement learning.

[0110] Since the target sample set inevitably contains noise, for example, the consistency between the sample prompt texts is relatively low. The model performance of the language model for reinforcement learning under noise is insufficient. To improve the model performance of the language model for reinforcement learning, the embodiment of the present disclosure utilizes proximal policy optimization to directly optimize the feedback information, determine the learning rate of the language model, and implement reinforcement learning of the language model based on the learning rate.

[0111] The parameter adjustment strength information represents the difference in model output before and after reinforcement learning for the previous state, measuring the strength of the model parameter adjustments made by reinforcement learning in the previous state. A higher parameter adjustment strength indicates a greater adjustment between the previous and current states, while a lower parameter adjustment strength indicates a smaller adjustment between the previous and current states. This parameter adjustment strength information is used to determine the learning rate of the language model, avoiding over-adjustment of model parameters that could lead to significant deviations.

[0112] The learning rate of the language model is the adjustment rate of the model parameters of the language model in the current state. It is the optimized feedback information and represents the expected feedback information in the current state. The calculation formula of the learning rate of the language model is shown in formula (1):

[0113] Among them, D can be used to represent the target sample set, x can be used to represent the target sample prompt text, y can be used to represent the target prediction prompt text, ρ can be used to represent the language model of the current state, ρ(y|x) can be used to represent the model output of the language model of the current state, π can be used to represent the pre-initialized reference language model, which retains the model parameters of the language model in the previous state, π(y|x) can be used to represent the model output of the language model in the previous state, r(x,y) can be used to represent the feedback information of the target prediction prompt text, β can be used to represent the KL penalty coefficient, which implements adaptive KL penalty to prevent the current state from adjusting the model parameters too aggressively and deviating greatly from the reference language model. It can be used to indicate the strength of parameter adjustment. Reinforcement learning of language models can be understood as fine-tuning π.

[0114] Determining strength information for adjusting parameters of a language model, specifically by determining strength information for adjusting parameters of the language model based on an output of a reference language model and an output of a language model in a current state, wherein the reference language model is a reference strategy model based on model parameter initialization of the language model.

[0115] Based on the learning rate, reinforcement learning is performed on the language model to obtain the language model after reinforcement learning. The specific method is: with the growth of feedback information as the goal, reinforcement learning is performed on the language model based on the learning rate to obtain the language model after reinforcement learning.

[0116] For example, determining the strength information for parameter adjustment of the large-scale language model LLM According to the feedback information r(x,y) and strength information of the target prediction prompt text, the learning rate E of the large-scale language model LLM is determined by the above formula (1): x～D,y～π(.|x) , with the goal of feedback information growth, based on the learning rate, reinforcement learning is performed on the large-scale language model LLM to obtain the large-scale language model LLM after reinforcement learning.

[0117] In the disclosed embodiments, the reinforcement learning effect of the language model is improved, and the image quality of subsequently generated images is further improved.

[0118] In an optional embodiment of the present disclosure, before evaluating the image quality of the reference image pre-generated for the target predicted prompt text and obtaining feedback information of the target predicted prompt text, the following specific steps may be included:

[0119] Based on the target sample prompt text, the image generation model is used to generate the target reference image;

[0120] Evaluate the image quality of the target reference image and obtain label feedback information of the target sample prompt text, where the label feedback information is used to represent the degree of match between the target sample prompt text and the image generation requirements;

[0121] Based on the target sample prompt text and the label feedback information of the target sample prompt text, the feedback model is trained to obtain a trained feedback model;

[0122] Correspondingly, evaluating the image quality of the reference image pre-generated for the target prediction prompt text to obtain feedback information of the target prediction prompt text includes the following specific steps:

[0123] The trained feedback model is used to evaluate the image quality of the reference image pre-generated for the target prediction prompt text, and feedback information of the target prediction prompt text is obtained.

[0124] The feedback model is a predictive model for feedback information in reinforcement learning. It predicts the predicted prompt text output by the language model in the current state and the feedback information generated in subsequent image generation. The feedback model is pre-trained using label feedback information. The performance of the feedback model directly determines the effectiveness of the language model's reinforcement learning. It can be understood as predicting the image quality of a pre-generated reference image based solely on the prompt text, without generating any image.

[0125] The target reference image is a pre-generated image of the target sample prompt text. In the embodiment of the present disclosure, the target reference image is actually generated. The image quality of the target reference image represents the degree of match with the image generation requirements. The higher the match, the higher the image quality.

[0126] The label feedback information characterizes the image quality of the target reference image pre-generated from the target sample prompt text, and represents the degree of match between the target sample prompt text and the image generation requirements. The higher the match, the higher the image quality and the higher the label feedback information. The feedback information corresponds to at least one dimension, including but not limited to: user preference dimension and image aesthetic dimension. The label feedback information is used as label data (ground truth) to train the feedback model, so that the feedback model can further predict the image quality of the reference image generated based on the predicted prompt text based on the predicted prompt text.

[0127] The image quality of the target reference image is evaluated to obtain label feedback information of the target sample prompt text. Specifically, the image quality of the target reference image is evaluated under at least one image quality dimension to obtain label feedback information of the target sample prompt text.

[0128] Based on the target sample prompt text and the label feedback information of the target sample prompt text, the feedback model is trained to obtain a trained feedback model. The specific method is: based on the target sample prompt text and the label feedback information of the target sample prompt text, the feedback model is supervised trained to obtain a supervised trained feedback model.

[0129] It should be noted that, under the image generation model, the feedback model is trained so that the feedback information output by the feedback model is more adapted to the generation capability of the image generation model.

[0130] Exemplarily, based on the target sample prompt text Target Sample Prompt, a stable diffusion model Stable Diffusion is used to generate a target reference image Target Ref Image, and the image quality of the target reference image is evaluated under two image quality dimensions (user preference dimension and image aesthetic dimension) to obtain label feedback information Label Feedback of the target sample prompt text, and based on the target sample prompt text and the label feedback information of the target sample prompt text, a feedback model RM is supervised trained to obtain a feedback model after supervised training, and the feedback model is used to evaluate the image quality of the reference image pre-generated by the target prediction prompt text Target Predict Prompt to obtain feedback information of the target prediction prompt text.

[0131] In the disclosed embodiment, by performing a quality assessment on the target reference image and obtaining label feedback information, the feedback model is trained. The trained feedback model is used to accurately assess the quality of the target prediction prompt text, thereby ensuring the effect of subsequent reinforcement learning of the language model and further improving the image quality of subsequently generated images.

[0132] In an optional embodiment of the present disclosure, image quality includes a user preference dimension and an image aesthetic dimension.

[0133] Correspondingly, the image quality of the target reference image is evaluated to obtain label feedback information of the target sample prompt text, including the following specific steps:

[0134] Performing a quality assessment on the target reference image in a user preference dimension to obtain label preference feedback information of the target sample prompt text, wherein the label preference feedback information is used to characterize the image quality of the target reference image pre-generated from the target sample prompt text in the user preference dimension; and / or performing a quality assessment on the target reference image in an image aesthetic dimension to obtain label aesthetic feedback information of the target sample prompt text, wherein the label aesthetic feedback information is used to characterize the image quality of the target reference image pre-generated from the target sample prompt text in the image aesthetic dimension;

[0135] Based on the label preference feedback information and / or the label aesthetic feedback information, label feedback information of the target sample prompt text is determined.

[0136] The user preference dimension is the preference for generating images based on the prompt text, which is an image generation requirement from the user's perspective. For example, the prompt text is "Please generate a picture of a pond with blooming lotus flowers on a rainy day." From the user preference dimension, the user is more inclined to generate a picture of a rainy day with blooming lotus flowers rather than a sunny day with no lotus flowers.

[0137] The image aesthetic dimension refers to the aesthetic quality of the generated image itself, which is an image generation requirement from the perspective of image aesthetics. For example, the prompt text is "Please generate an oil painting of an apple". From the perspective of image aesthetics, an image ratio of 16:9 has higher image quality than an image ratio of 1:1. An image with an apple placed at the position of (0.618:0.382) has higher image quality than an image with the apple placed in the center.

[0138] The label preference feedback information is used to characterize the image quality of the target reference image pre-generated from the target sample prompt text under the user preference dimension. The label aesthetic feedback information is used to characterize the image quality of the target reference image pre-generated from the target sample prompt text under the image aesthetic dimension. In the embodiment of the present disclosure, the label feedback information characterizes the image quality of the target reference image pre-generated from the target sample prompt text under the user preference dimension and the image aesthetic dimension. The calculation formula of the label feedback information is shown in formula (2):

[0139] r(x,y)=α·r ps (x,y)+(1-α)·r aes (2)

[0140] Among them, r ps (x,y) can be used to represent label preference feedback information (Pick Score, PS), r aes It can be used to represent the label aesthetic feedback information (Aesthetic Score, AES), and α can be used to represent the preset balance factor. The two feedback information are weighted to obtain the label feedback information.

[0141] Correspondingly, the training of the feedback model is also carried out under the user preference dimension and the image aesthetic dimension. The loss value used to train the feedback model includes the preference loss value and the aesthetic loss value. In addition, when the image generation model is a stable diffusion model, multiple random seeds are used to control multiple random tensors and the label feedback information is averaged. The calculation formula of the preference loss value is shown in formula (3), and the calculation formula of the aesthetic loss value is shown in formula (4):

[0142] Among them, L ps is the preference loss value, r ps (x, y) is the label preference feedback information, MSE() is the mean square error calculation formula, and N is the total number of samples.

[0143] Among them, L aes Can be used to represent the aesthetic loss value, r aes(y) can be used to represent label aesthetic feedback information, MSE() can be used to represent the mean square error calculation formula, and N can be used to represent the total number of samples.

[0144] A quality assessment of the target reference image in terms of user preference is performed to obtain label preference feedback information of the target sample prompt text. Specifically, a preference model is used to perform a quality assessment of the target reference image in terms of user preference to obtain label preference feedback information of the target sample prompt text. The preference model is pre-trained based on a large-scale dataset of prompt text and user preferences, and has the function of predicting user preferences for images.

[0145] The target reference image is evaluated for its image aesthetics, and the label aesthetic feedback information of the target sample prompt text is obtained. Specifically, the aesthetic model is used to evaluate the target reference image for its image aesthetics, and the label aesthetic feedback information of the target sample prompt text is obtained. The aesthetic model is pre-trained on a large-scale dataset based on prompt text and image aesthetics, and has the function of measuring the aesthetic quality of the image itself.

[0146] Based on the label preference feedback information and / or the label aesthetic feedback information, the label feedback information of the target sample prompt text is determined. In the case of including the user preference dimension and the image aesthetic dimension, the specific method is: based on the label preference feedback information and the label aesthetic feedback information, the label feedback information of the target sample prompt text is obtained by weighted calculation.

[0147] Exemplarily, a preference model is used to perform a quality assessment on the target reference image Target Ref Image in terms of user preference dimensions to obtain label preference feedback information Label Feedback PS of the target sample prompt text Target Sample Prompt. An aesthetic model is used to perform a quality assessment on the target reference image in terms of image aesthetic dimensions to obtain label aesthetic feedback information Label Feedback AES of the target sample prompt text. Based on the label preference feedback information and the label aesthetic feedback information, a weighted calculation is performed to obtain label feedback information Label Feedback of the target sample prompt text.

[0148] In the disclosed embodiment, label feedback information is obtained from two image quality dimensions, namely user preference and image aesthetics. Targeted training of the feedback model is implemented in these two dimensions. The feedback model completed with targeted training is used to perform a quality assessment of user preference and image aesthetics on the target prediction prompt text, thereby ensuring the effect of subsequent reinforcement learning of the language model and further improving the visual alignment of the subsequently generated images in the two image quality dimensions of user preference and image aesthetics.

[0149] In an optional embodiment of the present disclosure, before inputting the target sample prompt text into the language model to obtain the target predicted prompt text, the following specific steps may be included:

[0150] Acquire a training sample set, wherein the training sample set includes a sample text pair, and the sample text pair includes a first sample prompt text and a second sample prompt text;

[0151] Inputting the first sample prompt text into the language model to be trained to obtain predicted prompt text, wherein the predicted prompt text includes the first predicted prompt text;

[0152] Based on the first predicted prompt text and the second sample prompt text, a language model to be trained is pre-trained to obtain a pre-trained language model.

[0153] The training sample set is a collection of sample prompt text used for pre-training the language model. This training sample set can be constructed manually, directly obtained from open source sample data, or obtained from public data using data acquisition tools, without limitation here.

[0154] The first sample prompt text is the sample prompt text with the lower text content in the sample prompt text pair and serves as training data for pre-training the language model to be trained. The second sample prompt text is the sample prompt text with the higher text content in the sample prompt text pair and serves as label data for pre-training the language model to be trained. The first sample prompt text and the second sample prompt text describe the same object, and the image quality of the pre-generated reference images for the two samples is uncertain.

[0155] The first predicted prompt text language model outputs the optimized suggestive natural language text during pre-training. The first predicted prompt text has the same description object as the first sample prompt text and the second sample prompt text, and is the prediction data for pre-training the language model to be trained.

[0156] The training sample set is obtained by: obtaining an initial sample set, performing paired sample expansion on the initial sample set, and obtaining a training sample set.

[0157] The first sample prompt text is input into the language model to obtain a first predicted prompt text. The specific method is: the first sample prompt text is input into the language model, the prompt text is optimized for the first sample prompt text, and the first predicted prompt text is obtained, wherein the prompt text optimization includes prompt text expansion or prompt text rewriting.

[0158] Based on the first prediction prompt text and the second sample prompt text, the language model to be trained is pre-trained to obtain a pre-trained language model. The specific method is as follows: based on the first prediction prompt text and the second sample prompt text, a loss value is calculated, and based on the loss value, the model parameters of the language model are adjusted. When the pre-training end condition is met, the pre-trained language model is obtained. The pre-training end condition includes but is not limited to: a loss value threshold, a pre-training duration, and a pre-training state threshold. In the embodiment of the present disclosure, the loss value is the loss value of the autoregressive model, and the specific calculation formula is shown in Formula (5):

[0159] Here, x can be used to represent the first sample prompt text, and yi can be used to represent the text vector token y={y1, y2...yn} of the first predicted prompt text.

[0160] Exemplarily, an initial sample set is obtained, wherein the training sample set includes 100,000 sample text pairs, the sample text pairs include a first sample prompt text Sample Prompt1_i (i∈[1,100000]) and a second sample prompt text Sample Prompt2_i (i∈[1,100000]), the text content of the second sample prompt text is higher than that of the first sample prompt text, the first sample prompt text is input into the large-scale language model LLM, the prompt text of the first sample prompt text is rewritten to obtain a first predicted prompt text Target Sample Prompt1, based on the first predicted prompt text and the second sample prompt text, a loss value Lsft is calculated, based on the loss value, the model parameters of the decoder of the large-scale language model LLM are adjusted, and when the preset pre-training end conditions are met, a pre-trained language model is obtained.

[0161] In the disclosed embodiment, supervised pre-training of the language model ensures that the language model can initially complete accurate prompt text optimization, laying the foundation for subsequent reinforcement learning.

[0162] In an optional embodiment of the present disclosure, obtaining a training sample set includes the following specific steps:

[0163] Acquire an initial sample set, wherein the initial sample set includes a plurality of sample prompt texts;

[0164] Performing text conversion on any sample prompt text to obtain a reference sample prompt text corresponding to any sample prompt text, wherein the text content volume of any sample prompt text is different from the text content volume of the reference sample prompt text;

[0165] Integrate any sample prompt text and the corresponding reference sample prompt text to obtain a training sample set.

[0166] The initial sample set is a collection of sample prompt texts before sample expansion. Multiple sample prompt texts in the initial sample set do not have the same description object (are not paired), and multiple sample prompt pairs need to be obtained through sample expansion.

[0167] Reference sample prompt text is a suggestive natural language sample text that describes the same object as the template prompt text. The sample prompt text and the reference sample prompt text differ in the amount of text content. Text content includes, but is not limited to, text length, number of words, and semantic labels of the words.

[0168] Optionally, before performing text conversion on any sample prompt text to obtain a reference sample prompt text corresponding to any sample prompt text, the following specific steps are also included:

[0169] Based on the text content of each sample prompt text, each sample prompt text is divided into a first sample prompt text and a second sample prompt text.

[0170] Perform text conversion on any sample prompt text to obtain a reference sample prompt text corresponding to any sample prompt text. The specific method is: use a text conversion algorithm to perform text conversion on any sample prompt text to obtain a reference sample prompt text corresponding to any sample prompt text, wherein the text conversion algorithm includes a text content volume enhancement algorithm and a text content volume reduction algorithm.

[0171] Optionally, integrating any sample prompt text and the corresponding reference sample prompt text to obtain a training sample set includes the following specific steps:

[0172] Based on the text screening rules, multiple sample prompt texts and corresponding reference sample prompt texts are screened to obtain screened sample prompt texts and corresponding reference sample prompt texts;

[0173] The filtered sample prompt texts and the corresponding reference sample prompt texts are integrated to obtain a training sample set.

[0174] Text filtering rules include but are not limited to: sensitive text filtering rules, erroneous text filtering rules, and aesthetic filtering rules.

[0175] Exemplarily, an initial sample set is obtained, wherein the initial sample set includes 60,000 sample prompt texts Sample Prompt_i (i∈[1,60000]), and based on the text content of each sample prompt text, each sample prompt text is divided into a first sample prompt text Sample Prompt1_i (i∈[1,30000]) and a second sample prompt text Sample Prompt2_i (i∈[1,30000]), and text conversion is performed on any sample prompt text to obtain a reference sample prompt text Sample Ref Prompt_i (i∈[1,60000]) corresponding to any sample prompt text. Based on sensitive text screening rules and aesthetic screening rules, the 60,000 sample prompt texts and the corresponding reference sample prompt texts are screened to obtain 50,000 screened sample prompt texts and the corresponding reference sample prompt texts, and the screened sample prompt texts and the corresponding reference sample prompt texts are integrated to obtain a training sample set including 100,000 sample prompt text pairs.

[0176] In the disclosed embodiment, a training sample set including multiple sample prompt pairs is obtained through sample expansion, which provides sufficient sample data for subsequent supervised pre-training of the language model.

[0177] In an optional embodiment of the present disclosure, for any sample prompt text whose text content volume is higher than a preset threshold, performing text conversion on any sample prompt text to obtain a reference sample prompt text corresponding to any sample prompt text includes the following specific steps:

[0178] Based on any sample prompt text, a prompt image corresponding to any sample prompt text is generated; based on the prompt image, a summary text of the prompt image is obtained as a reference sample prompt text corresponding to any sample prompt text;

[0179] and / or,

[0180] Perform content summary processing on any sample prompt text to obtain a reference sample prompt text corresponding to any sample prompt text.

[0181] Based on any sample prompt text, a prompt image corresponding to any sample prompt text is generated. Based on the prompt image, a summary text of the prompt image is obtained as a reference sample prompt text corresponding to any sample prompt text. The specific method is as follows: based on the second sample prompt text, a prompt image corresponding to the second sample prompt text is generated using a graph-text matching model. Based on the prompt image, a summary text of the prompt image is obtained using a graph-text matching model as a reference sample prompt text corresponding to the second sample prompt text. The graph-text matching model is a deep learning model for matching images and texts. The corresponding image is determined by extracting text features, or the corresponding text is determined by extracting image features. Graph-text matching models include but are not limited to: BLIP model and CLIP model.

[0182] Perform content summarization on any sample prompt text to obtain a corresponding reference sample prompt text. Specifically, use a language model to generate a content summary on a second sample prompt text to obtain a corresponding reference sample prompt text. Language models are natural language models with summary generation capabilities, including but not limited to Large Scale Language Models (LLMs), Transformer models, and BERT models.

[0183] Exemplarily, based on the second sample prompt text Sample Prompt2_i (i∈[1,30000]), the BLIP model is used to generate a prompt image corresponding to the second sample prompt text. Based on the prompt image, the BLIP model is used to obtain the summary text of the prompt image as the reference sample prompt text corresponding to the second sample prompt text. The large-scale language model LLM is used to perform content summary generation processing on the second sample prompt text to obtain the reference sample prompt text corresponding to the second sample prompt text.

[0184] In the disclosed embodiment, sample expansion of sample prompt texts with high text content is achieved, and reference sample prompt texts with low text content are obtained, thereby increasing the number of training samples and providing sufficient training data for subsequent supervised pre-training of the language model.

[0185] In an optional embodiment of the present disclosure, for any sample prompt text whose text content is less than a preset threshold, performing text conversion on any sample prompt text to obtain a reference sample prompt text corresponding to any sample prompt text includes the following specific steps:

[0186] Perform content expansion processing on any sample prompt text to obtain a reference sample prompt text corresponding to any sample prompt text.

[0187] Perform content expansion processing on any sample prompt text to obtain a corresponding reference sample prompt text. Specifically, the method is to use a language model to perform content expansion processing on the first sample prompt text to obtain a corresponding reference sample prompt text. The language model is a natural language model with text content expansion capabilities, including but not limited to the Large Scale Language Model (LLM), the Transformer model, and the BERT model.

[0188] Exemplarily, the large-scale language model LLM is used to perform content expansion processing on the first sample prompt text Sample Prompt1_i (i∈[1,30000]) to obtain a reference sample prompt text corresponding to the first sample prompt text.

[0189] In the disclosed embodiment, sample expansion of sample prompt texts with low text content is achieved, and reference sample prompt texts with high text content are obtained, which increases the number of training samples and provides sufficient label data for subsequent supervised pre-training of the language model.

[0190] FIG2 shows a schematic diagram of a sample expansion process in an image generation method provided by an embodiment of the present disclosure, as shown in FIG2 :

[0191] Obtain an initial sample set. According to the text content volume division rule, divide the multiple sample prompt texts in the initial sample set into sample prompt texts with high text content volume and sample prompt texts with low text content volume. For the sample prompt texts with high text content volume, use a graph-text matching model to perform content expansion processing, or use a large-scale language model to perform summary text extraction to obtain the corresponding reference sample prompt text. For the sample prompt texts with low text content volume, use a large-scale language model to perform content summary processing to obtain the corresponding reference sample prompt text. Perform data cleaning and filtering on each sample prompt text and reference sample prompt text to obtain a training sample set.

[0192] FIG3 shows a flow chart of language model training in an image generation method provided by an embodiment of the present disclosure, as shown in FIG3 :

[0193] Obtain a sample set, which includes multiple sample prompt pairs. Self-supervise pre-training a language model to obtain a pre-trained language model. Utilize a preference model and an aesthetics model to perform a quality assessment on the sample prompt text based on user preference and image aesthetics, respectively, to obtain label feedback information. Train a feedback model based on the sample prompt text and label feedback information to obtain a trained feedback model. Using the trained feedback model, perform reinforcement learning on the pre-trained language model based on proximal policy optimization. The feedback model is used to perform a quality assessment on the target prediction prompt text output by the language model, obtain feedback information on the target prediction prompt text, and adjust the model parameters of the language model based on the feedback information.

[0194] FIG4 shows a schematic diagram of a front-end display in an image generation method provided by an embodiment of the present disclosure, as shown in FIG4 :

[0195] On the front-end of the AI ​​website, users click the "Text to Image Synthesis" and "Hint Text Optimization" controls, then enter the initial hint text "A Girl" in the initial hint text input box. Several target hint texts will pop up on the right side of the page for selection: "Portrait of a girl, sighing, filled with melancholy, sad and lost, with a hint of sorrow"; "Portrait of a girl, running and jumping, full of energy, healthy and charming, like the brightest sunshine in summer"; ... "Portrait of a girl, painting, serene and elegant, her focused eyes reflecting the warm autumn sun." After the user selects the target hint text, they click the Generate control. Based on the target hint text, the image generation model performs the image generation task for text-to-image synthesis, obtains the target image, and displays it in the target image display box.

[0196] 5 , which shows a flow chart of another image generation method provided according to an embodiment of the present disclosure. The method is applied to a cloud-side device and includes the following specific steps:

[0197] Step 502: Receive the initial prompt text of the image generation task sent by the front end.

[0198] Step 504: Input the initial prompt text into the language model to obtain the target prompt text, wherein the language model is trained based on a sample prompt pair, the sample prompt pair includes a first sample prompt text and a second sample prompt text, the text content of the first sample prompt text is less than the text content of the second sample prompt text, the training includes reinforcement learning based on feedback information of the predicted prompt text output by the language model to be trained, the predicted prompt text is output after the first sample prompt text is input into the language model to be trained, and the feedback information represents the image quality of a reference image pre-generated by the predicted prompt text.

[0199] Step 506: Based on the target prompt text, use the image generation model to perform the image generation task to obtain the target image, wherein the image generation model is a model adapted to the language model.

[0200] Step 508: Feedback the target image to the front end.

[0201] The disclosed embodiments are applied to cloud-side devices of applications, websites, or mini-programs that have prompt text optimization and image generation functions. The cloud-side device is a network cloud device where the server of the application, website, or mini-program is located. It is a virtual device. A language model is deployed on the cloud-side device to implement prompt text optimization, and an image generation model is deployed to implement image generation. The front end is the front end of the application, website, or mini-program that the user logs in to, or the front end of the application platform. The cloud-side device and the end-side device are connected through a network transmission channel to transmit data and realize remote image generation. The computing power performance of the cloud-side device is higher than that of the front end.

[0202] The embodiment of the present disclosure and the embodiment of FIG. 1 are based on the same inventive concept. The specific methods of steps 502 to 506 can refer to steps 102 to 106 and will not be repeated here.

[0203] In an embodiment of the present disclosure, an initial prompt text of an image generation task sent by a front end is received; the initial prompt text is input into a language model to obtain a target prompt text, wherein the language model is trained based on sample prompts, and the training includes reinforcement learning based on feedback information of a predicted prompt text output by the language model to be trained, wherein the predicted prompt text is output after inputting a first sample prompt text into the language model to be trained, and the feedback information represents the image quality of a reference image pre-generated by the predicted prompt text; based on the target prompt text, the image generation task is performed using the image generation model to obtain a target image; and the target image is fed back to the front end. The language model is based on sample prompt pairs and performs targeted reinforcement learning on the image quality of pre-generated reference images to optimize the prompt text. This allows the trained language model to perform targeted optimization on the image quality of the pre-generated reference images for the initial prompt text, align the prompt texts at the visual level, and generate visually aligned target images, thereby improving the image quality of the generated images and better meeting the image generation requirements. There is no need for repeated trial and error, modifying the prompt text without a clear direction, and regenerating the image, which avoids a large amount of time and computing resources loss, improves image generation efficiency, and reduces image generation costs. Image generation is completed through cloud-side devices with high computing and storage performance, further improving image generation efficiency and reducing image generation costs.

[0204] In an optional embodiment of the present disclosure, after step 508, the following specific steps are further included:

[0205] Receive image feedback for the target image sent by the front end;

[0206] Based on the image feedback, adjust the model parameters of the language model.

[0207] After receiving the target image as feedback, the front end can further generate image feedback based on the target image, and further adjust the model parameters of the language model through interaction.

[0208] For example, the target image is an oil painting-style portrait: a girl painting, illuminated by autumn sunshine. The girl, wearing a beret, holds a paintbrush in her right hand and a palette in her left, facing a vertical drawing board. This oil painting-style portrait is fed back to the website frontend and rendered on the image generation page. User A then generates image feedback based on this target image, including "poor composition," "missing details," and "incorrect semantic understanding." This feedback is then fed back to the client device, which then adjusts the model parameters of the Large Scale Language Model (LLM) based on the image feedback.

[0209] Receive image feedback from the front-end for the target image and adjust the language model parameters based on the image feedback. This interactive process allows for further adjustments to the language model parameters, further improving the subsequent optimization of the language model's prompt text.

[0210] 6 , which shows a flow chart of a data processing method for image generation according to an embodiment of the present disclosure. The method is applied to a cloud-side device and includes the following specific steps:

[0211] Step 602: Acquire a target sample set, wherein the target sample set includes a plurality of sample prompt texts.

[0212] Step 604: extracting a target sample prompt text from the target sample set, wherein the target sample prompt text is any one of a plurality of sample prompt texts.

[0213] Step 606: Input the target sample prompt text into the language model to be trained to obtain the target predicted prompt text.

[0214] Step 608: Evaluate the image quality of the reference image pre-generated for the target prediction prompt text to obtain feedback information of the target prediction prompt text.

[0215] Step 610: Based on the feedback information of the target prediction prompt text, reinforcement learning is performed on the language model to be trained to obtain a language model after reinforcement learning.

[0216] Step 612: Feedback the model parameters of the language model after reinforcement learning to the terminal device.

[0217] Cloud-side devices are virtual devices that provide model training. End-side devices are physical devices that provide prompt text optimization and image generation capabilities. End-side and cloud-side devices are connected via network channels for data transmission. Cloud-side devices have higher computing power and storage capacity than end-side devices.

[0218] The embodiment of the present disclosure and the embodiment of FIG1 are based on the same inventive concept. For the specific methods of steps 602 to 610, please refer to the embodiment of performing reinforcement learning on the language model in the embodiment of FIG1 above, and will not be repeated here.

[0219] In an embodiment of the present disclosure, a target sample set is obtained, wherein the target sample set includes multiple sample prompt texts; a target sample prompt text is extracted from the target sample set, wherein the target sample prompt text is any one of the multiple sample prompt texts; the target sample prompt text is input into a language model to obtain a target prediction prompt text; the image quality of a reference image pre-generated for the target prediction prompt text is evaluated to obtain feedback information of the target prediction prompt text; based on the feedback information of the target prediction prompt text, reinforcement learning is performed on the language model to be trained to obtain a language model after reinforcement learning; and model parameters of the language model after reinforcement learning are fed back to the end-side device. By evaluating the image quality of the reference image pre-generated for the target prediction prompt text output by the language model, feedback information of the target prediction prompt text is obtained. Based on the feedback information, reinforcement learning is performed on the language model to be trained, so that the language model after reinforcement learning can perform targeted optimization on the image quality of the reference image pre-generated for the initial prompt text, align the prompt text at the visual level, and subsequently generate a visually aligned target image, thereby improving the image quality of subsequently generated images. In addition, the reinforcement learning of the language model is completed through cloud-side devices with high computing and storage performance, thereby improving the model performance of the language model obtained by reinforcement learning, improving the efficiency of reinforcement learning, and reducing the cost of reinforcement learning.

[0220] The following further describes the image generation method provided by the present disclosure, using the application of the image generation method provided by the present disclosure in generating cartoon images as an example, in conjunction with FIG7 . FIG7 shows a flowchart of the processing process of the image generation method provided by the present disclosure in an embodiment of the present disclosure, which is applied to a cartoon image and includes the following specific steps:

[0221] Step 702: Acquire an initial sample set, wherein the initial sample set includes a plurality of sample prompt texts.

[0222] Step 704: For high-sample prompt texts whose text content volume is higher than a preset threshold, based on the high-sample prompt text, a picture-text matching model is used to generate a prompt image corresponding to the high-sample prompt text; based on the prompt image, a picture-text matching model is used to obtain a summary text of the prompt image as a low-sample prompt text corresponding to the high-sample prompt text.

[0223] Step 706: For high-sample prompt texts whose text content volume is higher than a preset threshold, a large-scale language model is used to perform content summarization processing on the high-sample prompt texts to obtain low-sample prompt texts corresponding to the high-sample prompt texts.

[0224] Step 708: For low-sample prompt texts whose text content is lower than a preset threshold, a large-scale language model is used to perform content expansion processing on the low-sample prompt text to obtain a high-sample prompt text corresponding to the low-sample prompt text.

[0225] Step 710: Based on the text screening rules, multiple sample prompt pairs are screened to obtain screened sample prompt pairs, and the screened sample prompt pairs are integrated to obtain a training sample set, wherein the sample prompt text pairs include low-sample prompt texts with low text content and high-sample prompt texts with high text content.

[0226] Step 712: Input the low-sample prompt text into the language model to obtain a high-prediction prompt text with a high text content volume, and pre-train the language model to be trained based on the high-prediction prompt text and the high-sample prompt text to obtain a pre-trained language model.

[0227] Step 714: Based on the sample prompt text, use the image generation model to generate a reference image, perform a quality assessment on the reference image in terms of user preference, obtain label preference feedback information of the sample prompt text, perform a quality assessment on the reference image in terms of image aesthetics, obtain label aesthetic feedback information of the sample prompt text, and determine the label feedback information of the sample prompt text based on the label preference feedback information and the label aesthetic feedback information.

[0228] Step 716: Based on the sample prompt text and the label feedback information of the sample prompt text, the feedback model is trained to obtain a trained feedback model.

[0229] Step 718: Input the low-sample prompt text into the language model to obtain the high-prediction prompt text, use the feedback model to evaluate the image quality of the reference image pre-generated for the high-prediction prompt text, obtain feedback information of the high-prediction prompt text, and based on the feedback information of the high-prediction prompt text, use proximal strategy optimization to perform reinforcement learning on the language model to obtain a language model after reinforcement learning.

[0230] Step 720: Obtain low-prompt text with low text content for the image generation task.

[0231] Step 722: Input the low-prompt text into the language model to obtain a high-prompt text with a high text content volume.

[0232] Step 724: Based on the high-prompt text, use the image generation model to perform the image generation task to obtain the target image.

[0233] In this disclosed embodiment, a language model is first trained on supervised pairs of low-quality and high-quality sample prompts. Then, to ensure that the optimized target prompt text can pre-generate reference images that better meet user preferences and image aesthetics, a feedback model is implemented using a reinforcement learning language model with visual AI feedback. This maximizes the feedback information generated for prompts. Reinforcement learning from visual AI feedback can significantly improve the optimization of prompt text, and thus the image quality of generated images.

[0234] Corresponding to the above method embodiment, the present disclosure also provides an embodiment of an image generation device. FIG8 shows a schematic structural diagram of an image generation device provided by an embodiment of the present disclosure. As shown in FIG8 , the device includes:

[0235] A first acquisition module 802 is configured to acquire an initial prompt text for an image generation task;

[0236] A first optimization module 804 is configured to input an initial prompt text into a language model to obtain a target prompt text, wherein the language model is trained based on a sample prompt pair, the sample prompt pair including a first sample prompt text and a second sample prompt text, the first sample prompt text having less text content than the second sample prompt text, the training including performing reinforcement learning based on feedback information of a predicted prompt text output by the language model to be trained, the predicted prompt text being output after inputting the first sample prompt text into the language model to be trained, the feedback information representing an image quality of a reference image pre-generated from the predicted prompt text;

[0237] The first generation module 806 is configured to perform an image generation task based on the target prompt text using an image generation model to obtain a target image, wherein the image generation model is a model adapted to the language model.

[0238] Optionally, the device further comprises:

[0239] The reinforcement learning module is configured to obtain a target sample set, wherein the target sample set includes multiple sample prompt texts; extract a target sample prompt text from the target sample set, wherein the target sample prompt text is any one of the multiple sample prompt texts; input the target sample prompt text into a language model to be trained to obtain a target predicted prompt text; evaluate the image quality of a reference image pre-generated for the target predicted prompt text to obtain feedback information of the target predicted prompt text; and perform reinforcement learning on the language model to be trained based on the feedback information of the target predicted prompt text to obtain a language model after reinforcement learning.

[0240] Optionally, the reinforcement learning module is further configured to:

[0241] Determine the strength information for adjusting the parameters of the language model; determine the learning rate of the language model to be trained based on the feedback information and strength information of the target prediction prompt text; and perform reinforcement learning on the language model to be trained based on the learning rate to obtain a language model after reinforcement learning.

[0242] Optionally, the device further comprises:

[0243] The feedback model training module is configured to generate a target reference image based on the target sample prompt text using an image generation model; evaluate the image quality of the target reference image to obtain label feedback information of the target sample prompt text; train the feedback model based on the target sample prompt text and the label feedback information of the target sample prompt text to obtain a trained feedback model;

[0244] Correspondingly, the reinforcement learning module is further configured as follows:

[0245] The trained feedback model is used to evaluate the image quality of the reference image pre-generated for the target prediction prompt text, and feedback information of the target prediction prompt text is obtained.

[0246] Optionally, the image quality includes a user preference dimension and an image aesthetic dimension;

[0247] Correspondingly, the feedback model training module is further configured as follows:

[0248] Perform a quality assessment on the target reference image in a user preference dimension to obtain label preference feedback information of the target sample prompt text, wherein the label preference feedback information is used to characterize the image quality of the target reference image pre-generated from the target sample prompt text in the user preference dimension; and / or perform a quality assessment on the target reference image in an image aesthetic dimension to obtain label aesthetic feedback information of the target sample prompt text; determine label feedback information of the target sample prompt text based on the label preference feedback information and / or the label aesthetic feedback information, wherein the label aesthetic feedback information is used to characterize the image quality of the target reference image pre-generated from the target sample prompt text in the image aesthetic dimension.

[0249] Optionally, the device further comprises:

[0250] The language model training module is configured to obtain a training sample set, wherein the training sample set includes sample prompt pairs, and the sample text pairs include a first sample prompt text and a second sample prompt text; input the first sample prompt text into the language model to be trained to obtain a first predicted prompt text; based on the first predicted prompt text and the second sample prompt text, pre-train the language model to be trained to obtain a pre-trained language model.

[0251] Optionally, the language model training module is further configured to:

[0252] An initial sample set is obtained, wherein the initial sample set includes a plurality of sample prompt texts; text conversion is performed on any sample prompt text to obtain a reference sample prompt text corresponding to any sample prompt text, wherein the text content of any sample prompt text is different from the text content of the reference sample prompt text; and any sample prompt text and the corresponding reference sample prompt text are integrated to obtain a training sample set.

[0253] Optionally, the language model training module is further configured to:

[0254] Based on any sample prompt text, generate a prompt image corresponding to any sample prompt text, and based on the prompt image, obtain a summary text of the prompt image as a reference sample prompt text corresponding to any sample prompt text; and / or, perform content summary processing on any sample prompt text to obtain a reference sample prompt text corresponding to any sample prompt text.

[0255] Optionally, the language model training module is further configured to:

[0256] Perform content expansion processing on any sample prompt text to obtain a reference sample prompt text corresponding to any sample prompt text.

[0257] In an embodiment of the present disclosure, an initial prompt text of an image generation task is obtained; the initial prompt text is input into a language model to obtain a target prompt text, wherein the language model is trained based on a sample prompt pair, the sample prompt pair includes a first sample prompt text and a second sample prompt text, the text content of the first sample prompt text is less than the text content of the second sample prompt text, the training includes reinforcement learning based on feedback information of a predicted prompt text output by the language model to be trained, the predicted prompt text is output after the first sample prompt text is input into the language model to be trained, and the feedback information represents the image quality of a reference image pre-generated by the predicted prompt text; based on the target prompt text, an image generation task is performed using an image generation model to obtain a target image, wherein the image generation model is a model adapted to the language model. The language model is based on sample prompt pairs and performs targeted reinforcement learning on the image quality of pre-generated reference images to optimize the prompt text. This allows the trained language model to perform targeted optimization of the image quality of the pre-generated reference images for the initial prompt text, align the prompt texts at the visual level, and generate visually aligned target images, thereby improving the image quality of the generated images and better meeting the image generation requirements. There is no need for repeated trial and error, modifying the prompt text without a clear direction, and regenerating the image, which avoids a large amount of time and computing resources loss, improves image generation efficiency, and reduces image generation costs.

[0258] The above is a schematic diagram of an image generation device according to this embodiment. It should be noted that the technical solution of this image generation device and the technical solution of the aforementioned image generation method are based on the same concept. For details not described in detail in the technical solution of the image generation device, please refer to the description of the technical solution of the aforementioned image generation method.

[0259] Corresponding to the above method embodiment, the present disclosure also provides an embodiment of an image generation device. FIG9 shows a schematic diagram of the structure of another image generation device provided by an embodiment of the present disclosure. As shown in FIG9, the device is applied to a cloud-side device and includes:

[0260] Receiving module 902, configured to receive the initial prompt text of the image generation task sent by the front end;

[0261] A second optimization module 904 is configured to input the initial prompt text into a language model to obtain a target prompt text, wherein the language model is trained based on sample prompt pairs, the sample prompt pairs including a first sample prompt text and a second sample prompt text, the first sample prompt text having less text content than the second sample prompt text, the training including performing reinforcement learning based on feedback information of a predicted prompt text output by the language model to be trained, the predicted prompt text being output after inputting the first sample prompt text into the language model to be trained, the feedback information representing an image quality of a reference image pre-generated from the predicted prompt text;

[0262] The second generation module 906 is configured to perform an image generation task based on the target prompt text using an image generation model to obtain a target image, wherein the image generation model is a model adapted to the language model;

[0263] The image feedback module 908 is configured to feed back the target image to the front end.

[0264] Optionally, the device further comprises:

[0265] The model adjustment module is configured to receive image feedback for the target image sent by the front end; and adjust the model parameters of the language model based on the image feedback.

[0266] In an embodiment of the present disclosure, an initial prompt text of an image generation task sent by a front end is received; the initial prompt text is input into a language model to obtain a target prompt text, wherein the language model is trained based on sample prompts, and the training includes reinforcement learning based on feedback information of a predicted prompt text output by the language model to be trained, wherein the predicted prompt text is output after inputting a first sample prompt text into the language model to be trained, and the feedback information represents the image quality of a reference image pre-generated by the predicted prompt text; based on the target prompt text, the image generation task is performed using the image generation model to obtain a target image; and the target image is fed back to the front end. The language model is based on sample prompt pairs and performs targeted reinforcement learning on the image quality of pre-generated reference images to optimize the prompt text. This allows the trained language model to perform targeted optimization on the image quality of the pre-generated reference images for the initial prompt text, align the prompt texts at the visual level, and generate visually aligned target images, thereby improving the image quality of the generated images and better meeting the image generation requirements. There is no need for repeated trial and error, modifying the prompt text without a clear direction, and regenerating the image, which avoids a large amount of time and computing resources loss, improves image generation efficiency, and reduces image generation costs. Image generation is completed through cloud-side devices with high computing and storage performance, further improving image generation efficiency and reducing image generation costs.

[0267] The above is a schematic diagram of an image generation device according to this embodiment. It should be noted that the technical solution of this image generation device and the technical solution of the aforementioned image generation method are based on the same concept. For details not described in detail in the technical solution of the image generation device, please refer to the description of the technical solution of the aforementioned image generation method.

[0268] Corresponding to the above method embodiment, the present disclosure also provides an embodiment of a data processing device for image generation. FIG10 shows a schematic diagram of the structure of a data processing device for image generation provided by an embodiment of the present disclosure. As shown in FIG10 , the device is applied to a cloud-side device and includes:

[0269] The sample acquisition module 1002 is configured to acquire a target sample set, wherein the target sample set includes a plurality of sample prompt texts;

[0270] An extraction module 1004 is configured to extract a target sample prompt text from the target sample set, wherein the target sample prompt text is any one of a plurality of sample prompt texts;

[0271] Prediction module 1006, configured to input the target sample prompt text into the language model to be trained to obtain the target predicted prompt text;

[0272] The quality evaluation module 1008 is configured to evaluate the image quality of the reference image pre-generated for the target prediction prompt text and obtain feedback information of the target prediction prompt text;

[0273] A reinforcement learning module 1010 is configured to perform reinforcement learning on the language model to be trained based on the feedback information of the target prediction prompt text to obtain a language model after reinforcement learning;

[0274] The model feedback module 1012 is configured to feed back the model parameters of the language model after reinforcement learning to the terminal device.

[0275] In an embodiment of the present disclosure, a target sample set is obtained, wherein the target sample set includes multiple sample prompt texts; a target sample prompt text is extracted from the target sample set, wherein the target sample prompt text is any one of the multiple sample prompt texts; the target sample prompt text is input into a language model to obtain a target prediction prompt text; the image quality of a reference image pre-generated for the target prediction prompt text is evaluated to obtain feedback information of the target prediction prompt text; based on the feedback information of the target prediction prompt text, reinforcement learning is performed on the language model to be trained to obtain a language model after reinforcement learning; and model parameters of the language model after reinforcement learning are fed back to the end-side device. By evaluating the image quality of the reference image pre-generated for the target prediction prompt text output by the language model, feedback information of the target prediction prompt text is obtained. Based on the feedback information, reinforcement learning is performed on the language model to be trained, so that the language model after reinforcement learning can perform targeted optimization on the image quality of the reference image pre-generated for the initial prompt text, align the prompt text at the visual level, and subsequently generate a visually aligned target image, thereby improving the image quality of subsequently generated images. In addition, the reinforcement learning of the language model is completed through cloud-side devices with high computing and storage performance, thereby improving the model performance of the language model obtained by reinforcement learning, improving the efficiency of reinforcement learning, and reducing the cost of reinforcement learning.

[0276] The above is a schematic diagram of an image-generated data processing device according to this embodiment. It should be noted that the technical solution of this image-generated data processing device and the technical solution of the image-generated data processing method described above share the same concept. For details not described in detail in the technical solution of the image-generated data processing device, please refer to the description of the technical solution of the image-generated data processing method described above.

[0277] Figure 11 shows a block diagram of an electronic device provided by an embodiment of the present disclosure. Components of the electronic device 1100 include, but are not limited to, a memory 1110 and a processor 1120. The processor 1120 is connected to the memory 1110 via a bus 1130, and a database 1150 is used to store data.

[0278] The electronic device 1100 also includes an access device 1140 that enables the electronic device 1100 to communicate via one or more networks 1160. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1140 may include one or more of any type of network interface (e.g., a network interface controller (NIC)) of wired or wireless type, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC).

[0279] In the embodiment of the present disclosure, the above-mentioned components of the electronic device 1100 and other components not shown in FIG11 may also be connected to each other, for example, via a bus. It should be understood that the block diagram of the computing device structure shown in FIG11 is for illustrative purposes only and does not limit the scope of the present disclosure. Those skilled in the art may add or replace other components as needed.

[0280] Electronic device 1100 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, personal digital assistant, laptop computer, notebook computer, netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). Electronic device 1100 may also be a mobile or stationary server.

[0281] The processor 1120 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned image generation method or image generation data processing method.

[0282] The present disclosure also provides a processor. Figure 12 shows a block diagram of a processor for implementing the present disclosure. As shown in Figure 12, the processor 1200 is configured to run a program, wherein the program, when executed by the processor, executes the method in the above embodiment.

[0283] In an embodiment of the present disclosure, the processor 1200 may execute an operating program for an image generation method or a data processing method for image generation.

[0284] Optionally, the processor 1200 can be configured to perform the following steps: obtaining an initial prompt text for an image generation task; inputting the initial prompt text into a language model to obtain a target prompt text, wherein the language model is trained based on a sample prompt pair, the sample prompt pair includes a first sample prompt text and a second sample prompt text, the text content of the first sample prompt text is less than the text content of the second sample prompt text, the training includes reinforcement learning based on feedback information of the predicted prompt text output by the language model to be trained, the predicted prompt text is output after the first sample prompt text is input into the language model to be trained, and the feedback information represents the image quality of a reference image pre-generated by the predicted prompt text; based on the target prompt text, using the image generation model to perform the image generation task to obtain a target image, wherein the image generation model is a model adapted to the language model.

[0285] Optionally, the processor 1200 can be configured to perform the following steps: receiving an initial prompt text of an image generation task sent by the front end; inputting the initial prompt text into a language model to obtain a target prompt text, wherein the language model is trained based on a sample prompt pair, the sample prompt pair includes a first sample prompt text and a second sample prompt text, the text content of the first sample prompt text is less than the text content of the second sample prompt text, the training includes reinforcement learning based on feedback information of the predicted prompt text output by the language model to be trained, the predicted prompt text is output after the first sample prompt text is input into the language model to be trained, and the feedback information represents the image quality of a reference image pre-generated by the predicted prompt text; based on the target prompt text, using the image generation model to perform the image generation task to obtain a target image, wherein the image generation model is a model adapted to the language model; and feeding the target image back to the front end.

[0286] Optionally, the processor 1200 can be configured to perform the following steps: obtaining a target sample set, wherein the target sample set includes multiple sample prompt texts; extracting a target sample prompt text from the target sample set, wherein the target sample prompt text is any one of the multiple sample prompt texts; inputting the target sample prompt text into the language model to be trained to obtain a target prediction prompt text; evaluating the image quality of a reference image pre-generated for the target prediction prompt text to obtain feedback information of the target prediction prompt text; performing reinforcement learning on the language model to be trained based on the feedback information of the target prediction prompt text to obtain a language model after reinforcement learning; and feeding back the model parameters of the language model after reinforcement learning to the end-side device.

[0287] The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of this computing device is based on the same concept as the technical solutions of the aforementioned image generation method and image-generated data processing method. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solutions of the aforementioned image generation method or image-generated data processing method.

[0288] The embodiment of the present disclosure further provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned image generation method or image generation data processing method.

[0289] The above is a schematic diagram of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium is based on the same concept as the technical solutions of the aforementioned image generation method and image-generated data processing method. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solutions of the aforementioned image generation method or image-generated data processing method.

[0290] Computer-readable storage media may also be referred to as computer storage media. They may include data signals transmitted in baseband or as part of a carrier wave, carrying readable program code. Such transmitted data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable storage media may transmit, propagate, or transfer programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0291] The program code contained in the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, radio frequency, etc., or any suitable combination of the foregoing.

[0292] An embodiment of the present disclosure also provides a computer program product, including a non-volatile computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the above-mentioned image generation method or image generation data processing method.

[0293] The embodiments of the present disclosure further provide a computer program, which, when executed by a processor, implements the steps of the above-mentioned image generation method or image generation data processing method.

[0294] Optionally, the computer program implements the following program code when executed by a processor: obtaining an initial prompt text for an image generation task; inputting the initial prompt text into a language model to obtain a target prompt text, wherein the language model is trained based on a sample prompt pair, the sample prompt pair includes a first sample prompt text and a second sample prompt text, the text content of the first sample prompt text is less than the text content of the second sample prompt text, the training includes reinforcement learning based on feedback information of a predicted prompt text output by the language model to be trained, the predicted prompt text is output after the first sample prompt text is input into the language model to be trained, and the feedback information represents the image quality of a reference image pre-generated by the predicted prompt text; based on the target prompt text, using an image generation model to perform an image generation task to obtain a target image, wherein the image generation model is a model adapted to the language model.

[0295] Optionally, when the above-mentioned computer program is executed by the processor, the program code implements the following steps: receiving the initial prompt text of the image generation task sent by the front end; inputting the initial prompt text into the language model to obtain the target prompt text, wherein the language model is trained based on the sample prompt pair, the sample prompt pair includes a first sample prompt text and a second sample prompt text, the text content of the first sample prompt text is less than the text content of the second sample prompt text, the training includes reinforcement learning based on the feedback information of the predicted prompt text output by the language model to be trained, the predicted prompt text is output after the first sample prompt text is input into the language model to be trained, and the feedback information represents the image quality of the reference image pre-generated by the predicted prompt text; based on the target prompt text, using the image generation model to perform the image generation task to obtain the target image, wherein the image generation model is a model adapted to the language model; and feeding back the target image to the front end.

[0296] Optionally, the computer program implements the following program code when executed by a processor: obtaining a target sample set, wherein the target sample set includes multiple sample prompt texts; extracting a target sample prompt text from the target sample set, wherein the target sample prompt text is any one of the multiple sample prompt texts; inputting the target sample prompt text into the language model to be trained to obtain a target predicted prompt text; evaluating the image quality of a reference image pre-generated for the target predicted prompt text to obtain feedback information of the target predicted prompt text; performing reinforcement learning on the language model to be trained based on the feedback information of the target predicted prompt text to obtain a language model after reinforcement learning; and feeding back the model parameters of the language model after reinforcement learning to the terminal-side device.

[0297] The embodiment of the present disclosure further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned image generation method or the image generation data processing method.

[0298] The above is an illustrative embodiment of a computer program. It should be noted that the technical solution of this computer program shares the same concept as the technical solutions of the aforementioned image generation method and image generation data processing method. For details not described in detail in the technical solution of the computer program, please refer to the description of the technical solutions of the aforementioned image generation method or image generation data processing method.

[0299] The foregoing description describes embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0300] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0301] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present disclosure are not limited by the order of the actions described, because according to the embodiments of the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the present disclosure are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of the present disclosure.

[0302] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0303] The preferred embodiments of the present disclosure disclosed above are intended only to help illustrate the present disclosure. The optional embodiments do not describe all details in detail, nor do they limit the disclosure to the specific implementation methods described. Obviously, many modifications and variations can be made based on the content of the embodiments of the present disclosure. The present disclosure selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of the present disclosure, so that those skilled in the art can better understand and utilize the present disclosure. The present disclosure is limited only by the claims and their full scope and equivalents. Industrial Applicability

[0304] The solution provided by the embodiment of the present disclosure can be applied to the process of image generation to obtain the initial prompt text of the image generation task; the initial prompt text is input into the language model to obtain the target prompt text, wherein the language model is trained based on a sample prompt pair, the sample prompt pair includes a first sample prompt text and a second sample prompt text, the text content of the first sample prompt text is lower than the text content of the second sample prompt text, the training includes reinforcement learning based on the feedback information of the predicted prompt text output by the language model to be trained, the predicted prompt text is output after the first sample prompt text is input into the language model to be trained, and the feedback information represents the image quality of the reference image pre-generated by the predicted prompt text; based on the target prompt text, the image generation task is performed using the image generation model to obtain the target image, wherein the image generation model is a model adapted to the language model, thereby solving the technical problems of large waste of time and computing resources, low image generation efficiency and high image generation cost.

Claims

1. A method for generating an image, comprising: Get the initial prompt text for the image generation task; Inputting the initial prompt text into a language model to obtain a target prompt text, wherein the language model is trained based on a sample prompt pair, the sample prompt pair includes a first sample prompt text and a second sample prompt text, the text content of the first sample prompt text is less than the text content of the second sample prompt text, the training includes performing reinforcement learning based on feedback information of a predicted prompt text output by the language model to be trained, the predicted prompt text is output after the first sample prompt text is input into the language model to be trained, and the feedback information represents an image quality of a reference image pre-generated by the predicted prompt text; Based on the target prompt text, the image generation task is performed using an image generation model to obtain a target image, wherein the image generation model is a model adapted to the language model.

2. The method according to claim 1, further comprising: Acquire a target sample set, wherein the target sample set includes a plurality of sample prompt texts; Extracting a target sample prompt text from the target sample set, wherein the target sample prompt text is any one of the multiple sample prompt texts; Inputting the target sample prompt text into the language model to be trained to obtain a target predicted prompt text; Evaluating the image quality of the reference image pre-generated by the target prediction prompt text to obtain feedback information of the target prediction prompt text; Based on the feedback information of the target prediction prompt text, reinforcement learning is performed on the language model to be trained to obtain the language model after reinforcement learning.

3. The method according to claim 2, wherein the step of performing reinforcement learning on the language model to be trained based on the feedback information of the target prediction prompt text to obtain the language model after reinforcement learning comprises: Determining strength information for adjusting parameters of the language model; Determining a learning rate of the language model to be trained according to the feedback information of the target prediction prompt text and the strength information; Based on the learning rate, reinforcement learning is performed on the language model to be trained to obtain the language model after reinforcement learning.

4. The method according to claim 2 or 3, further comprising: Based on the target sample prompt text, using the image generation model, generating a target reference image; The image quality of the target reference image is evaluated to obtain label feedback information of the target sample prompt text, wherein the label feedback information is used to characterize the relationship between the target sample prompt text and the image generation requirement. degree of matching; Based on the target sample prompt text and the label feedback information of the target sample prompt text, a feedback model is trained to obtain the trained feedback model; The step of evaluating the image quality of the reference image pre-generated for the target prediction prompt text to obtain feedback information of the target prediction prompt text includes: The trained feedback model is used to evaluate the image quality of the reference image pre-generated for the target prediction prompt text, and feedback information of the target prediction prompt text is obtained.

5. The method according to claim 4, wherein the image quality comprises a user preference dimension and an image aesthetic dimension; The step of evaluating the image quality of the target reference image and obtaining label feedback information of the target sample prompt text includes: Performing a quality assessment on the target reference image in a user preference dimension to obtain label preference feedback information of the target sample prompt text, wherein the label preference feedback information is used to characterize the image quality of the target reference image pre-generated by the target sample prompt text in the user preference dimension; and / or, performing a quality assessment on the target reference image in an image aesthetic dimension to obtain label aesthetic feedback information of the target sample prompt text, wherein the label aesthetic feedback information is used to characterize the image quality of the target reference image pre-generated by the target sample prompt text in the image aesthetic dimension; Based on the label preference feedback information and / or the label aesthetic feedback information, label feedback information of the target sample prompt text is determined.

6. The method according to claim 2, further comprising: Acquire a training sample set, wherein the training sample set includes a sample prompt pair, and the sample prompt pair includes the first sample prompt text and the second sample prompt text; Inputting the first sample prompt text into the language model to be trained to obtain the predicted prompt text, wherein the predicted prompt text includes the first predicted prompt text; Based on the first predicted prompt text and the second sample prompt text, the language model to be trained is pre-trained to obtain the pre-trained language model.

7. The method according to claim 6, wherein obtaining a training sample set comprises: Acquire an initial sample set, wherein the initial sample set includes a plurality of sample prompt texts; Performing text conversion on any of the sample prompt texts to obtain a reference sample prompt text corresponding to any of the sample prompt texts, wherein the text content volume of any of the sample prompt texts is different from the text content volume of the reference sample prompt text; Integrate any of the sample prompt texts and the corresponding reference sample prompt text to obtain the training sample set.

8. The method according to claim 7, for any of the sample prompt texts whose text content volume is higher than a preset threshold, performing text conversion on any of the sample prompt texts to obtain the reference sample prompt text corresponding to any of the sample prompt texts, comprising: Based on any of the sample prompt texts, a prompt image corresponding to any of the sample prompt texts is generated, and based on the prompt image, a summary text of the prompt image is obtained as the reference sample prompt text corresponding to any of the sample prompt texts; and / or, Content summary processing is performed on any of the sample prompt texts to obtain the reference sample prompt text corresponding to any of the sample prompt texts.

9. The method according to claim 7, for any of the sample prompt texts whose text content volume is lower than a preset threshold, performing text conversion on any of the sample prompt texts to obtain the reference sample prompt text corresponding to any of the sample prompt texts, comprising: Content expansion processing is performed on any of the sample prompt texts to obtain a reference sample prompt text corresponding to any of the sample prompt texts.

10. An image generation method, applied to a cloud-side device, comprising: Receive the initial prompt text of the image generation task sent by the front end; Inputting the initial prompt text into a language model to obtain a target prompt text, wherein the language model is trained based on a sample prompt pair, the sample prompt pair includes a first sample prompt text and a second sample prompt text, the text content of the first sample prompt text is less than the text content of the second sample prompt text, the training includes performing reinforcement learning based on feedback information of a predicted prompt text output by the language model to be trained, the predicted prompt text is output after the first sample prompt text is input into the language model to be trained, and the feedback information represents an image quality of a reference image pre-generated by the predicted prompt text; Based on the target prompt text, using an image generation model to perform the image generation task to obtain a target image, wherein the image generation model is a model adapted to the language model; The target image is fed back to the front end.

11. The method according to claim 10, further comprising: Receiving image feedback for the target image sent by the front end; Based on the image feedback, model parameters of the language model are adjusted.

12. The method according to claim 10, further comprising: Acquire a target sample set, wherein the target sample set includes a plurality of sample prompt texts; Extracting a target sample prompt text from the target sample set, wherein the target sample prompt text is any one of the multiple sample prompt texts; Inputting the target sample prompt text into the language model to be trained to obtain a target predicted prompt text; Evaluating the image quality of the reference image pre-generated by the target prediction prompt text to obtain feedback information of the target prediction prompt text; Based on the feedback information of the target prediction prompt text, reinforcement learning is performed on the language model to be trained to obtain the language model after reinforcement learning.

13. The method according to claim 12, wherein the step of performing reinforcement learning on the language model to be trained based on the feedback information of the target prediction prompt text to obtain the language model after reinforcement learning comprises: Determining strength information for adjusting parameters of the language model; Determining a learning rate of the language model to be trained according to the feedback information of the target prediction prompt text and the strength information; Based on the learning rate, reinforcement learning is performed on the language model to be trained to obtain the language model after reinforcement learning.

14. The method according to claim 12 or 13, further comprising: Based on the target sample prompt text, using the image generation model, generating a target reference image; Evaluate the image quality of the target reference image to obtain label feedback information of the target sample prompt text, wherein the label feedback information is used to characterize the matching degree between the target sample prompt text and the image generation requirement; Based on the target sample prompt text and the label feedback information of the target sample prompt text, a feedback model is trained to obtain the trained feedback model; The step of evaluating the image quality of the reference image pre-generated for the target prediction prompt text to obtain feedback information of the target prediction prompt text includes: The trained feedback model is used to evaluate the image quality of the reference image pre-generated for the target prediction prompt text, and feedback information of the target prediction prompt text is obtained.

15. A data processing method for image generation, applied to a cloud-side device, comprising: Acquire a target sample set, wherein the target sample set includes a plurality of sample prompt texts; Extracting a target sample prompt text from the target sample set, wherein the target sample prompt text is any one of the multiple sample prompt texts; Inputting the target sample prompt text into the language model to be trained to obtain a target predicted prompt text; Evaluating the image quality of the reference image pre-generated by the target prediction prompt text to obtain feedback information of the target prediction prompt text; Based on the feedback information of the target prediction prompt text, reinforcement learning is performed on the language model to be trained to obtain the language model after reinforcement learning; The model parameters of the language model after reinforcement learning are fed back to the terminal side device.

16. An electronic device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 15 are implemented.

17. A computer program product, wherein: The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 15.

18. A computer program product, wherein: The invention comprises a non-volatile computer-readable storage medium storing a computer program, wherein the computer program implements the method according to any one of claims 1 to 15 when executed by a processor.

19. A computer program, wherein When the computer program is executed by a processor, the method according to any one of claims 1 to 15 is implemented.

20. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 15.

Images