An image generation method, apparatus, electronic device, storage medium, and program product

Abstract

The present disclosure relates to an image generation method and device, electronic equipment, storage medium and program product, and relates to the field of image processing. The method comprises: in response to an obtained interaction request, performing intent recognition on the interaction request based on a trained intent recognition model to determine task category information to which the interaction request belongs; inputting the task category information and the interaction request into a trained prompt generation model for processing to generate prompt information; and processing the prompt information based on a trained image generation model to generate a corresponding image. The present disclosure improves the reliability and image quality of image generation technology in application scenarios such as multi-modal interaction, high-quality generation and customization requirements.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, and more specifically, to an image generation method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] Among related technologies, AIGC (Artificial Intelligence Generated Content) is a technology that uses artificial intelligence to generate content. It is widely used in the field of image generation, and AIGC technology enables the generation of images based on interactive requests from user input.

[0003] In recent years, Artificial Intelligence Generated Content (AIGC) technology has made significant progress in the field of image generation. AIGC utilizes computer algorithms and models to automatically generate fictional, artistic, or modified content based on existing images, greatly expanding the diversity and application scenarios of image generation. Early image generation techniques mainly relied on rules and templates, resulting in limited flexibility and limited output. The rise of diffusion models has further propelled the development of this field; their powerful generation capabilities, rich detail, and high controllability have made them the mainstream technology. In the future, with the continuous development of multimodal interaction, high-quality generation, and customization needs, AIGC technology will play an even more important role in creative design, scientific research, and other fields. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides an image generation method, apparatus, electronic device, storage medium, and program product.

[0005] According to a first aspect of the present disclosure, an image generation method is provided, the method comprising: responding to an acquired interaction request, performing intent recognition on the interaction request based on a trained intent recognition model, and determining task category information to which the interaction request belongs; inputting the task category information and the interaction request into a trained prompt generation model for processing, and generating prompt information; and processing the prompt information based on a trained raw image model, and generating a corresponding image.

[0006] In some exemplary embodiments of this disclosure, in response to an acquired interaction request, performing intent recognition on the interaction request based on a trained intent recognition model to determine the task category information to which the interaction request belongs includes: in response to acquiring the interaction request of the current dialogue round, determining the historical data of the previous dialogue round corresponding to the interaction request of the current dialogue round; inputting the interaction request of the current dialogue round and the historical data of the previous dialogue round into the trained intent recognition model to perform intent recognition and determine the task category information of the current dialogue round.

[0007] In some exemplary embodiments of this disclosure, the task category information and the interaction request are input into a trained prompt generation model for processing, and the prompt information is generated by: constructing a guiding prefix for the current dialogue round based on the task category information of the current dialogue round; inputting the guiding prefix and the interaction request of the current dialogue round into the trained prompt generation model for processing, and generating the prompt information for the current dialogue round.

[0008] In some exemplary embodiments of this disclosure, the task category information and the interaction request are input into a trained prompt generation model for processing, and the prompt information is generated by: constructing a guiding prefix for the current dialogue round based on the task category information of the current dialogue round; inputting the guiding prefix and the interaction request of the current dialogue round into the trained prompt generation model for processing, and generating the prompt information for the current dialogue round.

[0009] In some exemplary embodiments of this disclosure, the task category information and the interaction request are input into a trained prompt generation model for processing. The generation of prompt information further includes: constructing a guiding prefix for the current dialogue round based on the task category information of the current dialogue round; obtaining the semantic information of the current dialogue round corresponding to the interaction request of the current dialogue round; and inputting the semantic information of the current dialogue round, the guiding prefix of the current dialogue round, and the historical data of the previous dialogue round into the trained prompt generation model for processing to generate the prompt information of the current dialogue round.

[0010] In some exemplary embodiments of this disclosure, generating a corresponding image based on the prompt information includes: inputting the prompt information of the current dialogue round into a trained graph-based model to obtain a first image of the current dialogue round; inputting the first image of the current dialogue round and the prompt information of the current dialogue round into a trained graph-based model, wherein the graph-based model outputs the image of the current round.

[0011] In some exemplary embodiments of this disclosure, inputting the first image of the current dialogue turn and the prompt information of the current dialogue turn into a trained graph-to-graph model, and the graph-to-graph model outputting the image of the current turn, includes: extracting the edge image of the first image of the current dialogue turn based on an edge image algorithm to obtain the second image of the current dialogue turn; inputting the second image of the current dialogue turn and the prompt information of the current dialogue turn into a trained graph-to-graph model, and the graph-to-graph model outputting the image of the current turn.

[0012] In some exemplary embodiments of this disclosure, the method further includes: determining whether the interaction request of the current dialogue round belongs to the topic corresponding to the historical data of the previous dialogue round based on a trained topic recognition model; updating the historical data corresponding to the topic based on the interaction request of the current dialogue round if the interaction request of the current dialogue round belongs to the topic corresponding to the historical data of the previous dialogue round; or, clearing the historical data of the previous dialogue round if the interaction request of the current dialogue round does not belong to the topic corresponding to the historical data of the previous dialogue round.

[0013] In some exemplary embodiments of this disclosure, the method further includes: detecting whether the interaction request of the current dialogue round contains Chinese information; translating the detected Chinese information into English information; obtaining the English information corresponding to historical interaction requests; and inputting the English information as the interaction request of the current dialogue round into the trained intent recognition model for intent recognition.

[0014] According to a second aspect of the present disclosure, an image generation apparatus is provided, comprising: an intent recognition unit, configured to, in response to an acquired interaction request, perform intent recognition on the interaction request based on a trained intent recognition model, and determine task category information to which the interaction request belongs; a prompt generation unit, configured to input the task category information and the interaction request into a trained prompt generation model for processing, and generate prompt information; and an image generation unit, configured to process the prompt information based on a trained image generation model, and generate a corresponding image.

[0015] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the image generation method described in any of the technical solutions above.

[0016] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, which, when the instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to execute the image generation method described in any of the technical solutions.

[0017] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the image generation method as described in any of the technical solutions.

[0018] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0019] The embodiments of this disclosure, in response to an acquired interaction request, perform intent recognition on the interaction request based on a trained intent recognition model, determine the task category information to which the interaction request belongs, improve the understanding of the interaction request, and generate corresponding images through multi-turn dialogue, making the generated images more consistent with the semantics of the interaction request. This comprehensively improves the reliability and image quality of image generation technology in application scenarios such as multimodal interaction, high-quality generation, and customization needs.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0022] Figure 1 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 1 .

[0023] Figure 2 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 2 .

[0024] Figure 3 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 3 .

[0025] Figure 4 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 4 .

[0026] Figure 5 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 5 .

[0027] Figure 6 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 6 .

[0028] Figure 7This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 7 .

[0029] Figure 8 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 8 .

[0030] Figure 9 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 9 .

[0031] Figure 10 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 10 .

[0032] Figure 11 This is a schematic diagram illustrating model training of an image generation method according to an exemplary embodiment of the present disclosure.

[0033] Figure 12 This is a schematic diagram illustrating a model application of an image generation scheme according to an exemplary embodiment of the present disclosure.

[0034] Figure 13 This is a block diagram illustrating an image generation apparatus according to an exemplary embodiment of the present disclosure.

[0035] Figure 14 This is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0036] Exemplary embodiments of this disclosure will be described in detail herein, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. Various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but can be changed as will become apparent upon understanding this disclosure, except for operations that must be performed in a particular order. Furthermore, for clarity and brevity, descriptions of features known in the art may be omitted.

[0037] The embodiments described below, which are examples of some of the embodiments of this disclosure, do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0038] The steps of the method in the exemplary embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings and examples.

[0039] Figure 1 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 1 .

[0040] The device in this embodiment can be applied to electronic devices, including smart speakers, smartphones, and smart tablet terminal devices. It can also include server-side components such as local servers and cloud servers, which can be deployed in a computer cluster consisting of one computer or multiple computers.

[0041] like Figure 1 As shown, in some embodiments, the image generation method of this disclosure includes:

[0042] In step S102, in response to the acquired interaction request, the interaction request is identified based on the trained intent recognition model to determine the task category information to which the interaction request belongs.

[0043] In this embodiment of the disclosure, in response to the acquired interaction request, the interaction request is identified by the trained intent recognition model, and the task category information to which the interaction request belongs is determined. The obtained task category information provides clear guidance for subsequent text prompt reasoning, so that the generated prompt information is more in line with the interaction request's requirements for raw images.

[0044] In exemplary embodiments of this disclosure, task category information may include, but is not limited to, any of the following categories: edit tasks, story tasks, and complicated tasks.

[0045] In the exemplary embodiments disclosed herein, editing tasks refer to explicit editing operations on elements in an image, such as adding, replacing, deleting, changing the background, or stylizing. These tasks are highly specific and relatively easy to understand.

[0046] In exemplary embodiments of this disclosure, a story task revolves around an element in an image as the main character, generating contextualized image content with coherent logic and narrative. For example, describing a dynamic narrative or a story with rich background requires a deep understanding of contextual information, but is not limited to this.

[0047] In exemplary embodiments of this disclosure, the complex task involves complex reasoning logic tasks, such as hierarchical descriptions or fuzzy expressions, such as "draw a plate of dumplings" or "make more dumplings," but is not limited thereto.

[0048] In exemplary embodiments of this disclosure, the training process of the intent recognition model corresponds to, for example: Figure 11 The LLM model training process shown in section 1102, which trains a query intent classification model, categorizes the potential intents of each query into three types: edit, story, and complex. Editing refers to operations performed on elements in an image, such as adding, replacing, deleting, changing the background, or stylizing. Editing intents are straightforward and relatively easy to understand. Story intents revolve around a specific element in the image, providing coherent and narrative descriptions, such as scene-based stories or dynamic narratives. These often contain numerous pronouns and require a deeper understanding of the context. Complex intents typically involve complex reasoning and include various special cases, such as hierarchical descriptions like "draw a plate of dumplings" or "more dumplings," making them more difficult to comprehend.

[0049] In an exemplary embodiment of this disclosure, a large language model tool (such as GPT4o) is used to generate a large number of multi-turn continuous dialogues containing a large number of different subjects and different intent categories. Common subjects include animals, plants, people, well-known IPs from movies and TV series, etc. Intent categories include editing, story, and complex categories. Each dialogue data contains 2 to 20 rounds of dialogue.

[0050] In an exemplary embodiment of this disclosure, each round of dialogue data is labeled according to intent category: edit, story, and complicated, corresponding to the editing, story, and complicated categories, respectively.

[0051] In an exemplary embodiment of this disclosure, a guiding prefix is ​​added to each dialogue. Based on the user input record (user_history), the task category to which the user's current input (user_current) belongs is determined, and one of edit, story, or complicated is output. This serves as training data for a query intent classification model.

[0052] In an exemplary embodiment of this disclosure, a query intent classification model based on an open-source large language model, such as using LoRa fine-tuning techniques, is trained to classify the intent category of a query according to the user's intent.

[0053] In step S104, the task category information and the interaction request are input into the trained prompt generation model for processing to generate prompt information.

[0054] In this embodiment of the disclosure, the latest raw image requirements are captured by the task category information and the interaction request. The task category information can more clearly guide the logic and direction of generating prompt information. Then, the task category information and the interaction request are input into the trained prompt generation model for processing, generating high-quality prompt information while maintaining the coherence between the generated prompt information and the context information.

[0055] In an exemplary embodiment of this disclosure, the training process of the topic recognition model corresponds to, as follows: Figure 11 The process of training the intent-aware text prompt reasoning model in LLM model training 1102 shown is based on multi-turn dialogue data from the historical training process. After simple manual screening and deduplication, a guiding prefix is ​​added before each turn of dialogue data, and the intent category of the query is embedded as a text prompt. Different guiding prefixes are used for different intent categories.

[0056] In an exemplary embodiment of this disclosure, for an edit-type query, the guiding prefix is ​​"This is an edit-type task that understands historical dialogue and human intent and infers the subject's state after the current input, while preserving the subject's background and details, and rewrites it into a concise and accurate English sentence."

[0057] In an exemplary embodiment of this disclosure, for a story-type query, the guiding prefix is ​​"This is a story-type task. Please understand the historical dialogue content and human intentions, develop a scenario-based description or narrative around the protagonist of the current input, supplement details and maintain the logical coherence of the narrative, and rewrite it into a concise and accurate English sentence to generate the core description of the image story."

[0058] In an exemplary embodiment of this disclosure, for complex queries, the guiding prefix is ​​"This is a complex task. Please understand the content of historical dialogues and human intentions, perform logical reasoning on the input from multiple levels and perspectives, extract the core information and perform semantic synthesis, and rewrite it into a concise and accurate English sentence that highlights the user's specific needs and multiple limiting conditions."

[0059] In an exemplary embodiment of this disclosure, a text prompt reasoning model is trained based on an open-source large language model, such as using LoRa fine-tuning technology, and simultaneously supports text prompt reasoning that is aware of three intent categories: story, editing, and complex.

[0060] In step S106, the above prompt information is processed based on the trained raw image model to generate the corresponding image.

[0061] In this embodiment of the disclosure, the above-mentioned prompt information is processed by the trained raw image model, and the corresponding image generated has higher quality, better aesthetics and more detailed expression, which greatly improves the reliability of the raw image solution and the user experience.

[0062] In the exemplary embodiments of this disclosure, in the multi-turn dialogue image generation scenario, the user's interaction requests are progressively understood through multiple dialogue turns. Context awareness and iterative optimization are achieved by combining historical data from previous dialogue turns. The historical data from previous dialogue turns accumulates all historical data under the same topic, gradually parsing implicit needs, avoiding semantic biases in single-turn image generation, and optimizing contextual coherence during iteration. Users can correct details of multi-turn dialogues in real time using natural speech, achieving more accurate and personalized image generation. Furthermore, multi-turn dialogue image generation can also address high-complexity requirements; that is, multi-turn dialogue can decompose the task into hierarchical generation, avoiding information overload or logical conflicts during single-turn generation.

[0063] In an exemplary embodiment of this disclosure, the process of generating a graph based on multiple dialogue turns includes, but is not limited to, the following steps:

[0064] The user's initial input interaction request is: "Generate a forest scene." The image generation system outputs a standard forest image corresponding to the interaction request in the first round of dialogue, and stores the historical data of the first round. The historical data includes the interaction request input in the first round, the prompt information generated based on the interaction request, and the standard forest image, and determines that the current topic is "forest".

[0065] The user's feedback in the second round of interaction requests was: "Add moonlight and fog to create a sense of mystery." Based on the second round of interaction requests, the guidance prefix was determined to be edit-oriented, and the current topic was determined to still be "forest." Then, based on the historical data from the first round and the second round of interaction requests, the image generation system adjusted the lighting and filters of the standard forest image generated in the first round to generate the forest image in the second round, and stored the historical data of the second round. The historical data included the interaction requests input in the second and first rounds, the prompts in the second and first rounds, and the image records generated in the second and first rounds, etc.

[0066] The user's third round of interaction requests were refined to include: "Add a glowing deer to the right side." Based on these requests, the guiding prefix was determined to be editable, and the current topic remained "forest." Then, based on the historical data from the first round and the second round of interaction requests, the image generation system ultimately generated a customized image that met the user's psychological expectations.

[0067] Users can continue to modify, adjust, and optimize the images by entering interaction requests in the fourth round and beyond, until the user's interaction request ends the "forest" topic.

[0068] Figure 2 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 2 .

[0069] like Figure 2 As shown, in Figure 1 Based on the image generation method shown, step S102 above may include the following steps:

[0070] In step S202, in response to the interaction request for obtaining the current dialogue round, the historical data corresponding to the above interaction request for the current dialogue round is determined.

[0071] In this embodiment of the disclosure, in response to the interaction request to obtain the current dialogue round, the historical data corresponding to the interaction request of the current dialogue round is determined. The historical data includes not only historically stored interaction requests, prompts and historical images, but also user feedback records on the generation effect of historical images, but is not limited thereto.

[0072] In an exemplary embodiment of this disclosure, the historical data of the previous dialogue round may come from database query results, log file analysis results, cached data, etc., but is not limited to these.

[0073] In step S204, the interaction request of the current dialogue round and the historical data of the previous dialogue round are input into the trained intent recognition model to perform intent recognition and determine the task category information of the current dialogue round.

[0074] In this embodiment of the disclosure, the interaction request of the current dialogue round and the historical data of the previous dialogue round are input into the trained intent recognition model to perform intent recognition, determine the task category information of the current dialogue round, and combine the historical data to provide more complete contextual information and classification guidance for the interaction request, thereby making the subsequently generated task category information more in line with the semantics of the context information and user needs.

[0075] In exemplary embodiments of this disclosure, the intent recognition model may be based on rule matching, machine learning models, and deep learning models. Machine learning models include decision trees, support vector machines, etc., and deep learning models include recurrent neural networks and their variants, long short-term memory networks, gated recurrent units, etc., but are not limited thereto.

[0076] Figure 3 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 3 .

[0077] like Figure 3 As shown, in Figure 1 Based on the image generation method shown, step S104 above may include the following steps:

[0078] In step S302, a guiding prefix for the current dialogue round is constructed based on the task category information of the current dialogue round.

[0079] In step S304, the aforementioned guiding prefix and the aforementioned interaction request of the current dialogue round are input into the trained prompt generation model for processing to generate the aforementioned prompt information of the current dialogue round.

[0080] In this embodiment of the disclosure, the aforementioned guiding prefix and the aforementioned interaction request of the current dialogue round are input into the trained prompt generation model for processing. The guiding prefix is ​​used to indicate the processing direction and logic of the interaction request, thereby making the generated prompt information of the current dialogue round more in line with the editing needs of the interaction request for historical images.

[0081] In exemplary embodiments of this disclosure, the guiding prefix may be a keyword or code determined by task category information, but is not limited thereto.

[0082] In exemplary embodiments of this disclosure, information such as the story type, theme, image purpose, and audience of an image is set by a guiding prefix, but is not limited thereto.

[0083] Figure 4 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 4 .

[0084] like Figure 4 As shown, in Figure 2 Based on the image generation method shown, step S104 above may include the following steps:

[0085] In step S402, a guiding prefix is ​​constructed based on the task category information of the current dialogue round.

[0086] In exemplary embodiments of this disclosure, in processing scenarios such as natural language processing and dialogue systems, a guiding prefix is ​​constructed using task category information. This prefix is ​​a special text identifier added before user input, which more accurately guides the model to understand the task category information.

[0087] In step S404, the interaction request of the current dialogue round, the guidance prefix of the current dialogue round, and the historical data of the previous dialogue round are input into the trained prompt generation model for processing to generate the prompt information of the current dialogue round.

[0088] In this embodiment of the disclosure, the interaction request of the current dialogue round, the guidance prefix of the current dialogue round, and the historical data of the previous dialogue round are input into the trained prompt generation model for processing. The guidance prefix is ​​used to indicate the processing direction and logic of the interaction request, and the historical data is used to provide contextual information, thereby making the generated prompt information of the current dialogue round more in line with the editing requirements of the interaction request for historical images and semantic follow-up.

[0089] Figure 5 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 5 .

[0090] like Figure 5 As shown, in Figure 2 Based on the image generation method shown, step S104 above may include the following steps:

[0091] In step S502, a guiding prefix for the current dialogue round is constructed based on the task category information of the current dialogue round.

[0092] In step S504, the semantic information corresponding to the above-mentioned interaction request in the current dialogue round is obtained.

[0093] In an exemplary embodiment of this disclosure, while determining task category information based on the intent recognition model, not only is a guiding prefix constructed, but the semantic information of the interaction request is also identified, providing a clearer direction and processing logic to the semantic information through the guiding prefix.

[0094] In step S506, the semantic information of the current dialogue round, the guiding prefix of the current dialogue round, and the historical data of the previous dialogue round are input into the trained prompt generation model for processing to generate the prompt information of the current dialogue round.

[0095] In this embodiment of the disclosure, the semantic information of the current dialogue round, the guiding prefix of the current dialogue round, and the historical data of the previous dialogue round are input into the trained prompt generation model for processing. The guiding prefix is ​​used to indicate the processing direction and logic of the interaction request, the historical data is used to provide contextual information, and the semantic information is used to describe the semantic understanding of the interaction request. This makes the generated prompt information of the current dialogue round more in line with the editing requirements of the interaction request for historical images and semantic follow-up, and also better able to express the semantic details contained in the interaction request.

[0096] In exemplary embodiments of this disclosure, the prompt generation model can be implemented using decision trees, support vector machines, recurrent neural networks, but is not limited thereto.

[0097] Figure 6This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 6 .

[0098] like Figure 6 As shown, in Figure 1 Based on the image generation method shown, step S106 above may include the following steps.

[0099] In step S602, the above-mentioned prompt information of the current dialogue round is input into the trained text-based image model to obtain the first image of the current dialogue round.

[0100] In this embodiment of the disclosure, the training process of the text-based image model corresponds to the following: Figure 11 The process of training the text-based image model in the training of the raw image model (1104) is as follows: Specifically, a dataset of images with high semantic density is collected. These images should contain multiple subjects and have complex interactions. Detailed annotation of the images is performed using open-source multimodal tools to generate ground truth (GT) labels. Each training sample consists of an image and a corresponding GT label (detailed description). The text-based image model is trained based on open-source pre-trained weights, focusing on improving the model's semantic following ability to text prompts. The training objective is to ensure that the generated images closely match the description of the text prompt (Prompt) while not omitting detailed information.

[0101] In an exemplary embodiment of this disclosure, the text-based graph model is trained and inferred at a low resolution (e.g., 512*512) to improve computational efficiency while maintaining the model's ability to parse complex semantics.

[0102] In an exemplary embodiment of this disclosure, the open-source pre-trained weights can be Stable Diffusion.

[0103] In step S604, the first image of the current dialogue round and the prompt information of the current dialogue round are input into the trained graph-generated graph model, and the graph-generated graph model outputs the image of the current dialogue round.

[0104] In this embodiment of the disclosure, based on Figures 1 to 5The image processing scheme shown here processes the interaction requests of multi-turn dialogues, and the resulting prompts not only maintain semantic coherence but also more clearly guide the user's editing needs for existing images in each dialogue turn. Furthermore, by inputting the aforementioned prompts for the current dialogue turn into the trained text-to-graph model to obtain the first image, and inputting the aforementioned first image and the aforementioned prompts for the current dialogue turn into the trained graph-to-graph model, the image output by the graph-to-graph model for the current dialogue turn contains subject attributes, complex relationships between multiple subjects, and environmental details, as well as the image editing needs corresponding to the user's interaction requests in multiple dialogue turns.

[0105] In an exemplary embodiment of this disclosure, the resolution of the first image may be set to 512*512.

[0106] In an exemplary embodiment of this disclosure, the image generation task is decoupled into two independent stages: cue following and image quality enhancement. In the first stage, the text-to-image model focuses on achieving efficient cue following capabilities by accurately understanding and generating images consistent with the cue content. The second stage employs a graph-to-image model, focusing on image quality optimization, while incorporating image information (such as Canny edge maps) from the text-to-image generation results of the first stage to ensure that the cue following effect is preserved. This phased task decoupling strategy allows the text-to-image model to focus on improving cue following capabilities, while the graph-to-image model focuses on image quality optimization, thus achieving complementary advantages. While ensuring cue following effectiveness, image quality is significantly improved, ultimately achieving high-quality image generation results.

[0107] In an exemplary embodiment of this disclosure, the text-to-graph model is trained using Finetune technology, with a focus on improving the model's semantic following ability to text prompts.

[0108] Figure 7 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 7 .

[0109] like Figure 7 As shown, in Figure 6 Based on the image generation method shown, step S106 above may include the following steps:

[0110] In step S702, the edge image of the first image in the current dialogue round is extracted to obtain the second image of the current dialogue round.

[0111] In this embodiment of the disclosure, the edge image of the image can be extracted using an edge image algorithm. The edge image of the first image in the current dialogue round is extracted using the edge image algorithm to obtain the second image.

[0112] In step S704, the second image of the current dialogue round and the prompt information of the current dialogue round are input into the trained graph-generated image model, and the graph-generated image model outputs the image of the current dialogue round.

[0113] In this embodiment of the disclosure, by inputting the second image of the current dialogue round and the prompt information of the current dialogue round into the trained graph-to-graph model, the graph-to-graph model outputs the image of the current dialogue round, which further enhances the aesthetics and detail of the images generated in multiple dialogue rounds and greatly improves the user experience.

[0114] In an exemplary embodiment of this disclosure, generating a graph based on a multi-turn dialogue may include the following stages:

[0115] (1) Initial input stage:

[0116] (1.1) In the first round, the user provides an interaction request to determine the task category information and generates the first round of text prompts, or directly uploads the initial sketch.

[0117] (1.2) The model generates the first round of low-resolution images (basic composition and tone).

[0118] (2) Multi-round optimization stage:

[0119] (2.1) Nth round of dialogue: The user’s interaction request based on the previous round of image feedback is a correction instruction, and the task category information is determined based on the correction instruction.

[0120] (2.2) Input fusion: Combine task category information, correction instructions and historical data from round 1 to round N-1 to generate the current text prompt (Prompt_n) and input it into the raw image model.

[0121] (2.3) Progressive output: The model outputs an optimized Image_n, with the resolution, details, and style of the image gradually improved.

[0122] (3) Termination conditions:

[0123] Users can end the current topic, start a new topic, or reach the preset round limit, such as completing it within 5 or 10 rounds, but are not limited to these.

[0124] In an exemplary embodiment of this disclosure, the training process of the graph-generated graph model corresponds to, as follows: Figure 11The training process of the raw image model shown in Figure 1104 involves collecting a large amount of aesthetically pleasing, high-quality image data, using the Canny operator to extract edge maps from the images, and simultaneously using open-source multimodal tools to annotate detailed descriptions and generate ground truth (GT) labels. Each training sample consists of an original image, a Canny edge map, and a corresponding GT label.

[0125] In an exemplary embodiment of this disclosure, a raw image model is trained using Finetune technology based on open-source pre-trained weights. This enables the model to generate a refined, high-quality image based on the input Canny edge map and text prompt. The goal is to further refine the image generated by the raw image model to produce a high-resolution image (e.g., 1024*1024), while simultaneously improving aesthetics and the semantic following ability of the text prompt.

[0126] In an exemplary embodiment of this disclosure, the open-source pre-trained weights may employ the ControlNet model.

[0127] Figure 8 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 8 .

[0128] like Figure 8 As shown, in Figures 2-7 Based on the image generation method shown, the image generation method may further include the following steps:

[0129] In step S802, the trained topic recognition model determines whether the interaction request in the current dialogue round belongs to the topic corresponding to the historical data in the previous dialogue round.

[0130] In this embodiment of the disclosure, the trained topic recognition model determines whether the interaction request in the current dialogue round belongs to the topic corresponding to the historical data in the previous dialogue round. If the interaction request in the current dialogue round belongs to the topic corresponding to the historical data, the current topic is maintained, and the context information of the historical data contained in the current topic is inherited.

[0131] In an exemplary embodiment of this disclosure, the trained topic recognition model determines whether the interaction request in the current dialogue round belongs to the topic corresponding to the historical data in the previous dialogue round. If the interaction request in the current dialogue round does not belong to the topic corresponding to the historical data, the topic is terminated, and the interaction request in the current dialogue round does not inherit the context information of the historical data.

[0132] In an exemplary embodiment of this disclosure, the training process of the topic recognition model corresponds to, as follows: Figure 11The process of training the topic switching recognition model in LLM model training 1102 shown is as follows: Specifically, a topic switching dataset is generated by using batch-generated multi-turn dialogue data and a random combination method, and a corresponding ground truth (gt) label is added to each data point: if a topic switch occurs, the gt is labeled as "yes"; if no topic switch occurs, the gt is labeled as "no".

[0133] Below is an exemplary embodiment of the backend code for specific topic identification:

[0134] [{"from":"human","value":f"{PREFIX}{user_history:['draw a dog.','dogwear a hat.'],user_current:'draw a cat'}"},{"from":"gpt","value":"yes"}.

[0135] In an exemplary embodiment of this disclosure, each dialogue data entry contains 2 to 20 rounds of dialogue, and a guiding prefix is ​​added to each data entry: based on the user input record user_history, it is determined whether the user's current input user_current has switched topics. If the topic has switched, "yes" is output; otherwise, "no" is output.

[0136] In this embodiment of the disclosure, a topic switching recognition model is trained based on an open-source large language model, such as using LoRA fine-tuning technology.

[0137] In an exemplary embodiment of this disclosure, a topic switching recognition model trained based on LoRA fine-tuning technology can automatically identify whether a topic switch has occurred in the user input; if a switch has occurred, it outputs "yes"; otherwise, it outputs "no".

[0138] In step S804, in response to the interaction request of the current dialogue round belonging to the topic corresponding to the historical data of the previous dialogue round, the historical data of the previous dialogue round is updated based on the interaction request of the current dialogue round.

[0139] In step S806, in response to the fact that the interaction request of the current dialogue round does not belong to the topic corresponding to the historical data of the previous dialogue round, the historical data of the previous dialogue round is cleared.

[0140] In this embodiment of the disclosure, if the interaction request in the current dialogue round does not belong to the topic corresponding to the historical data in the previous dialogue round, by clearing the historical data in the previous dialogue round, on the one hand, the storage pressure caused by historical data can be reduced, and on the other hand, the interference of irrelevant historical data on the generated image can be reduced.

[0141] Figure 9 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 9 .

[0142] like Figure 9 As shown, in Figure 8 Based on the image generation method shown, the image generation method may further include the following steps:

[0143] In step S902, in response to the interaction request of the current dialogue round belonging to the topic corresponding to the historical data of the previous dialogue round, the historical data of the previous dialogue round is updated according to the generated prompt information.

[0144] In an exemplary embodiment of this disclosure, in response to the interaction request of the current dialogue round belonging to the topic corresponding to the historical data of the previous dialogue round, the interaction request of the current dialogue round is obtained based on the topic, and after generating prompt information based on the interaction request of the current dialogue round and the historical data, the interaction request of the current dialogue round and the prompt information are stored as historical data for the next round.

[0145] In an exemplary embodiment of this disclosure, in response to the interaction request of the current dialogue round belonging to the topic corresponding to the historical data of the previous dialogue round, the interaction request of the current dialogue round is obtained based on the topic. After generating an image based on the interaction request of the current dialogue round and the historical data, the interaction request and prompt information of the current dialogue round are stored as historical data for the next round. Then, when the interaction request is received in the next round, the task classification information corresponding to the interaction request is determined, and new prompt information is obtained by combining the historical data. The prompt information is input into the trained raw image model to obtain an image.

[0146] Figure 10 This is a schematic diagram of an image generation method according to an exemplary embodiment of the present disclosure. Figure 10 .

[0147] like Figure 10 As shown, in Figure 8 Based on the image generation method shown, the image generation method may further include the following steps:

[0148] In step S1002, it is detected whether the above-mentioned interaction request in the current dialogue round contains Chinese information.

[0149] In step S1004, the detected Chinese information is translated into English.

[0150] In step S1006, the English information corresponding to the historical interaction requests is obtained.

[0151] In step S1008, the aforementioned English information is input as the interaction request for the current dialogue round into the trained intent recognition model for intent recognition.

[0152] In this embodiment of the disclosure, by translating the Chinese information in the interaction request into English information, invalid text information in the Chinese information can be filtered out. The English information is then used as the interaction request for the current dialogue round and input into the trained intent recognition model for intent recognition. This helps to further improve the accuracy and reliability of the task category information output by the intent recognition model.

[0153] In an exemplary embodiment of this disclosure, step S1004 can be implemented based on a translation model, and the training process of the translation model corresponds to, for example: Figure 11 The LLM model training process shown in 1102 involves training a translation model using large language model tools (such as GPT-4o) to generate a large number of Chinese-English translation pairs.

[0154] In the exemplary embodiments disclosed herein, each translation pair includes a Chinese text prompt and a corresponding English translation, in order to simulate the query request scenario of a real user as much as possible.

[0155] In an exemplary embodiment of this disclosure, a guiding prefix is ​​added to each generated dialogue data: the following sentence is translated into English, requiring conciseness and accuracy, and the result is converted into the format required for training, serving as training data for the translation model.

[0156] In an exemplary embodiment of this disclosure, a translation model based on an open-source large language model is trained, for example, using LoRA fine-tuning technology, to optimize the model's translation capabilities and ensure that the translation results are accurate and concise. This model can not only translate user-inputted interactive requests (queries) into English, but also remove redundant descriptions that are irrelevant to the content generated by the image.

[0157] Figure 12 This is a schematic diagram illustrating a model application of an image generation scheme according to an exemplary embodiment of the present disclosure.

[0158] like Figure 12 As shown, in Figure 1-10 The image generation method shown and Figure 11 Based on the model training shown, the image generation scheme relies on the trained LLM model 1202 and the trained raw image model 1204. The trained LLM model 1202 is applied to the topic switching recognition stage, the translation stage, the query intent classification stage, and the intent-aware text prompt reasoning stage. The trained raw image model 1204 is applied to the text-to-image stage and the image-to-image stage. The specific processing steps of each processing stage are as follows.

[0159] 1. Topic Switching Recognition Stage: Based on the user's current interaction request (query) and history (user_history), input them into the topic switching recognition model. If the model outputs "yes," it indicates that a topic switching has occurred. At this time, the history cache is cleared, and the current query is recorded as the new dialogue starting point. If the output is "no," the current query is added to the user_history to maintain the logical integrity of the continuous dialogue.

[0160] 2. Translation Stage: The system detects whether the user-input query contains Chinese characters. If so, it uses a trained translation model to translate it into English. During the translation process, redundant descriptions unrelated to the generated image content are automatically removed, ensuring a concise and accurate translation that better meets the requirements of the image generation task.

[0161] 3. Query Intent Classification Stage: The query intent classification model is used to analyze the user's current query and determine the task category corresponding to the user's needs. The model input includes the current query and the cached user_history, and the output is one of the following categories: edit, story, or complicated.

[0162] Editing tasks: These involve explicit editing operations on elements within an image, such as adding, replacing, deleting, changing the background, or stylizing. These tasks are highly specific and relatively easy to understand.

[0163] Story-based content: This category revolves around a specific element in the image as the main subject, generating coherent and narrative scene-based image content. For example, it describes dynamic narratives or stories with rich backgrounds, requiring a deep understanding of the context.

[0164] Complex tasks involve complex reasoning logic, such as hierarchical descriptions or vague expressions, like "draw a plate of dumplings" or "make more dumplings." These tasks include various special cases and are more difficult to understand.

[0165] The classification results will provide clear guidance for subsequent text prompt reasoning, making the generated text prompts more in line with user intent.

[0166] 4. Intent-Aware Text Prompt Inference Stage: Based on the intent classification results described above, and combining the user's user history and the current query as input, a high-quality text prompt containing detailed image descriptions is generated. The intent classification results explicitly guide the generation logic and direction of the prompt. User history provides contextual information, ensuring the generated prompt is coherent with the dialogue background. The current query captures the user's latest input needs, ensuring the prompt accurately matches the current intent.

[0167] For different intent categories, the following different guiding prefixes are used to construct the model input:

[0168] Editing Task: This is an editing task that involves understanding the history of the dialogue and human intent, inferring the subject's state after the current input, while preserving the subject's background and details, and rewriting it into a concise and accurate English sentence.

[0169] The Story category is a story-based task that requires understanding historical dialogue and human intent, developing a contextualized description or narrative around the current protagonist, adding details while maintaining logical coherence, and rewriting it into a concise and accurate English sentence to generate the core description of the image story.

[0170] The complex class task involves understanding historical dialogue content and human intent, performing logical reasoning on the input from multiple levels and perspectives, extracting the core information and performing semantic synthesis, and rewriting it into a concise and accurate English sentence that highlights the user's specific needs and multiple constraints.

[0171] The multi-intent-aware text prompt inference model generates a high-quality text prompt that includes a detailed description of the image content, such as the subject description of the image (e.g., object type, action, emotion), scene details (e.g., environment, lighting, color), etc.

[0172] 5. Text-to-Graph Stage: The text-to-graph model is used to generate the first-stage image, using the generated Prompt as input, ensuring that the image content closely matches the Prompt description. The generated image must include subject attributes, complex relationships between subjects, and environmental details. To improve generation speed and efficiency, the initial image resolution is set to 512*512. The image quality at this stage is preliminary, with a focus on ensuring semantic accuracy.

[0173] 6. Image-to-Image Stage: The Canny operator is used to extract edge information from the image generated from the image-to-image model. The Canny edge map is then combined with the Prompt algorithm and input into the image-to-image model to generate a high-quality second-stage image. The image-to-image model further enhances the aesthetics and detail of the image, significantly improving the user experience.

[0174] 7. Output Results Stage: The high-quality image generated from the original image is presented to the user as the final result.

[0175] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0176] Figure 13 This is a block diagram illustrating an image generation apparatus according to an exemplary embodiment of the present disclosure. The apparatus of this embodiment can be applied to electronic devices, including smart speakers, smartphones, and smart tablet terminal devices, and may also include server-side components such as local servers and cloud servers, which can be deployed in a computer or a computer cluster consisting of multiple computers.

[0177] like Figure 13 As shown, the image generation device 1300 may include: an intent recognition unit 1302, a prompt generation unit 1304, and an image generation unit 1306.

[0178] The intent recognition unit 1302 is used to respond to the interaction request for obtaining the current dialogue round, perform intent recognition on the interaction request based on the trained intent recognition model, and determine the task category information to which the above interaction request belongs in the current dialogue round.

[0179] The prompt generation unit 1304 is used to input the above-mentioned task category information and the above-mentioned interaction request of the current dialogue round into the trained prompt generation model for processing, and generate prompt information for the current dialogue round.

[0180] The image generation unit 1306 is used to process the above-mentioned prompt information of the current dialogue round based on the trained raw image model, and generate the corresponding image of the current dialogue round.

[0181] In some exemplary embodiments of this disclosure, the intent recognition unit 1302 is further configured to: in response to the acquired interaction request, determine the historical data corresponding to the interaction request of the current dialogue round; input the interaction request of the current dialogue round and the historical data of the previous dialogue round into the trained intent recognition model to perform intent recognition, and determine the task category information of the current dialogue round.

[0182] In some exemplary embodiments of this disclosure, the prompt generation unit 1304 is further configured to: construct a guiding prefix for the current dialogue turn based on the task category information of the current dialogue turn; input the guiding prefix and the interaction request of the current dialogue turn into the trained prompt generation model for processing, and generate the prompt information of the current dialogue turn.

[0183] In some exemplary embodiments of this disclosure, the prompt generation unit 1304 is further configured to: input the above-mentioned interaction request of the current dialogue round, the above-mentioned guidance prefix of the current dialogue round, and the above-mentioned historical data of the previous dialogue round into the trained prompt generation model for processing, and generate the above-mentioned prompt information of the current dialogue round.

[0184] In some exemplary embodiments of this disclosure, the prompt generation unit 1304 is further configured to: obtain semantic information corresponding to the interaction request of the current dialogue round; input the semantic information, the guiding prefix of the current dialogue round, and the historical data of the previous dialogue round into the trained prompt generation model for processing, and generate the prompt information of the current dialogue round.

[0185] In some exemplary embodiments of this disclosure, the image generation unit 1306 is further configured to: input the aforementioned prompt information of the current dialogue turn into a trained graph model to obtain a first image of the current dialogue turn; input the aforementioned first image of the current dialogue turn and the aforementioned prompt information of the current dialogue turn into a trained graph model, and the graph model outputs the aforementioned image of the current dialogue turn.

[0186] In some exemplary embodiments of this disclosure, the image generation unit 1306 is further configured to: extract the edge image of the first image in the current dialogue round based on the edge image algorithm to obtain the second image in the current dialogue round; input the second image in the current dialogue round and the prompt information in the current dialogue round into the trained graph-generated image model, and output the graph-generated image model as the image in the current dialogue round.

[0187] In some exemplary embodiments of this disclosure, the image generation apparatus 1300 is further configured to: determine, based on a trained topic recognition model, whether the interaction request of the current dialogue round belongs to the topic corresponding to the historical data of the previous dialogue round; update the historical data of the previous dialogue round based on the interaction request of the current dialogue round if the interaction request of the current dialogue round belongs to the topic corresponding to the historical data of the previous dialogue round; and clear the historical data of the previous dialogue round if the interaction request of the current dialogue round does not belong to the topic corresponding to the historical data of the previous dialogue round.

[0188] In some exemplary embodiments of this disclosure, the image generation apparatus 1300 is further configured to: respond to the interaction request of the current dialogue round belonging to the topic corresponding to the historical data of the previous dialogue round; and update the historical data of the previous dialogue round according to the generated prompt information of the current dialogue round.

[0189] In some exemplary embodiments of this disclosure, the image generation apparatus 1300 is further configured to: detect whether the interaction request of the current dialogue round contains Chinese information; translate the detected Chinese information into English information; obtain the English information corresponding to the historical interaction request; and input the English information as the interaction request of the current dialogue round into the trained intent recognition model for intent recognition.

[0190] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0191] Figure 14 This is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. For example, device 1400 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, etc.

[0192] Reference Figure 14 The device 1400 may include one or more of the following components: a processing component 1402, a memory 1404, a power supply component 14013, a multimedia component 1408, an audio component 1410, an input / output (I / O) interface 1412, a sensor component 1414, and a communication component 1416.

[0193] Processing component 1402 typically controls the overall operation of device 1400, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1402 may include one or more processors 1418 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 1402 may include one or more modules to facilitate interaction between processing component 1402 and other components. For example, processing component 1402 may include a multimedia module to facilitate interaction between multimedia component 1408 and processing component 1402.

[0194] Memory 1404 is configured to store various types of data to support the operation of device 1400. Examples of this data include instructions for any application or method operating on device 1400, contact data, phonebook data, messages, pictures, videos, etc. Memory 1404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0195] Power supply component 14013 provides power to various components of device 1400. Power supply component 14013 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 1400.

[0196] Multimedia component 1408 includes a screen that provides an output interface between the device 1400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 1408 includes a front-facing camera and / or a rear-facing camera. When the device 1400 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0197] Audio component 1410 is configured to output and / or input audio signals. For example, audio component 1410 includes a microphone (MIC) configured to receive external audio signals when device 1400 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1404 or transmitted via communication component 1416. In some embodiments, audio component 1410 also includes a speaker for outputting audio signals.

[0198] I / O interface 1412 provides an interface between processing component 1402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0199] Sensor assembly 1414 includes one or more sensors for providing status assessments of various aspects of device 1400. For example, sensor assembly 1414 may detect the on / off state of device 1400, the relative positioning of components such as the aforementioned components being the display and keypad of device 1400, changes in the position of device 1400 or a component of device 1400, the presence or absence of user contact with device 1400, the orientation or acceleration / deceleration of device 1400, and temperature changes of device 1400. Sensor assembly 1414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1414 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0200] Communication component 1416 is configured to facilitate wired or wireless communication between device 1400 and other devices. Device 1400 can access wireless networks based on communication standards, such as WiFi, 3G, 4G, 5G, other communication standards, or combinations thereof. In some embodiments of this disclosure, communication component 1416 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In some embodiments of this disclosure, the aforementioned communication component 1416 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0201] In some embodiments of this disclosure, the apparatus 1400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0202] In some embodiments of this disclosure, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1404 including instructions, which can be executed by a processor 1418 of device 1400 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0203] In some embodiments of this disclosure, a non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to execute an image generation method. The method includes: responding to an acquired interaction request; performing intent recognition on the interaction request based on a trained intent recognition model to determine the task category information to which the interaction request belongs in the current dialogue turn; inputting the task category information and the interaction request in the current dialogue turn into a trained prompt generation model for processing to generate prompt information; and processing the prompt information based on a trained raw image model to generate a corresponding image.

[0204] In some embodiments of this disclosure, a computer program product is also provided, including a computer program / instructions. When executed by a processor, the computer program / instructions implement an image generation method. The method includes: responding to an acquired interaction request; performing intent recognition on the interaction request based on a trained intent recognition model to determine the task category information to which the interaction request belongs in the current dialogue round; inputting the task category information and the interaction request in the current dialogue round into a trained prompt generation model for processing to generate prompt information; and processing the prompt information based on a trained raw image model to generate a corresponding image.

[0205] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0206] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image generation method, characterized in that, The method includes: In response to the acquired interaction request, the interaction request is identified based on the trained intent recognition model to determine the task category information to which the interaction request belongs; The task category information and the interaction request are input into the trained prompt generation model for processing to generate prompt information; The trained image model is used to process the prompt information and generate a corresponding image.

2. The image generation method according to claim 1, characterized in that, In response to the acquired interaction request, the interaction request is subjected to intent recognition based on the trained intent recognition model, and the task category information to which the interaction request belongs includes: In response to an interaction request to obtain the current dialogue round, determine the historical data of the previous dialogue round corresponding to the interaction request of the current dialogue round; The interaction request of the current dialogue round and the historical data of the previous dialogue round are input into the trained intent recognition model to perform intent recognition and determine the task category information of the current dialogue round.

3. The image generation method according to claim 1 or 2, characterized in that, The task category information and the interaction request are input into the trained prompt generation model for processing, and the generated prompt information includes: Construct a guiding prefix for the current dialogue round based on the task category information of the current dialogue round; The guidance prefix and the interaction request of the current dialogue turn are input into the trained prompt generation model for processing to generate the prompt information of the current dialogue turn.

4. The image generation method according to claim 1 or 2, characterized in that, The task category information and the interaction request are input into the trained prompt generation model for processing. The generated prompt information also includes: Construct a guiding prefix for the current dialogue round based on the task category information of the current dialogue round; The interaction request of the current dialogue round, the guiding prefix of the current dialogue round, and the historical data of the previous dialogue round are input into the trained prompt generation model for processing to generate the prompt information of the current dialogue round.

5. The image generation method according to claim 1 or 2, characterized in that, The task category information and the interaction request are input into the trained prompt generation model for processing. The generated prompt information also includes: Construct a guiding prefix for the current dialogue round based on the task category information of the current dialogue round; Obtain the semantic information of the current dialogue round corresponding to the interaction request in the current dialogue round; The semantic information of the current dialogue round, the guiding prefix of the current dialogue round, and the historical data of the previous dialogue round are input into the trained prompt generation model for processing to generate the prompt information of the current dialogue round.

6. The image generation method according to claim 1 or 2, characterized in that, Generating the corresponding image based on the prompt information includes: The prompt information of the current dialogue round is input into the trained text-based image model to obtain the first image of the current dialogue round; The first image of the current dialogue round and the prompt information of the current dialogue round are input into the trained graph-generated graph model, and the graph-generated graph model outputs the image of the current round.

7. The image generation method according to claim 6, characterized in that, The first image of the current dialogue round and the prompt information of the current dialogue round are input into the trained graph-generated graph model, and the graph-generated graph model outputs the image of the current round, including: Extract the edge image of the first image in the current dialogue round to obtain the second image in the current dialogue round; The second image of the current dialogue round and the prompt information of the current dialogue round are input into the trained graph-generated graph model, and the graph-generated graph model outputs the image of the current round.

8. The image generation method according to any one of claims 1, 2, and 7, characterized in that, Also includes: Based on the trained topic recognition model, determine whether the interaction request in the current dialogue round belongs to the topic corresponding to the historical data of the previous dialogue round; In response to an interaction request in the current dialogue round belonging to a topic corresponding to the historical data of the previous dialogue round, the historical data corresponding to the topic is updated based on the interaction request in the current dialogue round; or, If the interaction request in the current dialogue round does not belong to the topic corresponding to the historical data in the previous dialogue round, the historical data of the previous dialogue round is cleared.

9. The image generation method according to claim 8, characterized in that, Also includes: Detect whether the interaction request in the current dialogue round contains Chinese information; Translate the detected Chinese information into English information; Retrieve the English information corresponding to historical interaction requests; The English information is used as the interaction request for the current dialogue round and input into the trained intent recognition model for intent recognition.

10. An image generation apparatus, characterized in that, include: An intent recognition unit is used to respond to an acquired interaction request, perform intent recognition on the interaction request based on a trained intent recognition model, and determine the task category information to which the interaction request belongs. The prompt generation unit is used to input the task category information and the interaction request into the trained prompt generation model for processing, and generate prompt information. The image generation unit is used to process the prompt information based on the trained raw image model and generate the corresponding image.

11. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the image generation method according to any one of claims 1-9.

12. A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of a mobile terminal, enable the mobile terminal to perform the image generation method of any one of claims 1-9.

13. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the image generation method as described in any one of claims 1-9.

Images