Figure generation method, device, system and equipment and storage medium

Abstract

The invention provides a text-to-map method, device, system and equipment and a storage medium, relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models and the like, and is applied to scenes such as AIGC content generation based on artificial intelligence. According to the specific implementation scheme, a target dialogue text used for generating an image of a current round of dialogue of a target object is obtained; constructing to-be-rewritten information based on the target dialogue text and the historical dialogue condition; inputting the to-be-rewritten information into the large language model to obtain a target cue word generated by the large language model according to the requirement of the target dialogue text; the target cue word is obtained based on the text rewriting capability of the large language model; and generating an image of the current round of dialogue based on the target prompt word.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to technical fields such as computer vision, deep learning, and large models, and is applied to scenarios such as AIGC (Artificial Intelligence Generated Content) content generation based on artificial intelligence. Background Art

[0002] With the continuous development of artificial intelligence technology, the technology of generating images from text has received unprecedented exposure and attention. A series of innovative tools and products have emerged in this field, which not only greatly improves the ability to automatically generate high-quality images from text descriptions, but also opens up new possibilities for many application scenarios such as artistic creation, design conception, virtual reality, etc. Summary of the invention

[0003] The present invention provides a method, apparatus, system, device and storage medium for creating a text map.

[0004] According to one aspect of the present disclosure, a method for generating a text map is provided, comprising:

[0005] Obtain the target dialogue text for generating an image in the current round of dialogue of the target object;

[0006] Construct the information to be rewritten based on the target dialogue text and historical dialogue situations;

[0007] Input the information to be rewritten into the large language model to obtain the target prompt words generated by the large language model according to the requirements of the target dialogue text; the target prompt words are obtained based on the text rewriting ability of the large language model;

[0008] Generate an image of the current round of dialogue based on the target prompt word.

[0009] According to another aspect of the present disclosure, there is provided a Wensheng map device, comprising:

[0010] An acquisition module, used for acquiring a target dialogue text of a current round of dialogue of a target object for generating an image;

[0011] A construction module, used to construct information to be rewritten based on the target dialogue text and historical dialogue situations;

[0012] The first generation module is used to input the information to be rewritten into the large language model to obtain the target prompt words generated by the large language model according to the requirements of the target dialogue text; the target prompt words are obtained based on the text rewriting ability of the large language model;

[0013] The second generation module is used to generate an image of the current round of dialogue based on the target prompt word.

[0014] According to another aspect of the present disclosure, there is provided a Wensheng diagram system, comprising:

[0015] The prompt word optimization agent is used to obtain the target dialogue text for generating images in the current round of dialogue of the target object; construct the information to be rewritten based on the target dialogue text and the historical dialogue situation; input the information to be rewritten into the large language model to obtain the target prompt words generated by the large language model according to the requirements of the target dialogue text; send the target prompt words to the text image tool; the target prompt words are obtained based on the text rewriting ability of the large language model;

[0016] The text image tool is used to generate images based on the target prompt words and the text image model in the text image tool.

[0017] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0021] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0022] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0023] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0025] Figure 1 is a schematic flow chart of a Vincent diagram method provided according to an embodiment of the present disclosure;

[0026] Figure 2 is a schematic diagram of constructing information to be rewritten in different rounds of dialogue according to an embodiment of the present disclosure;

[0027] Figure 3 is a schematic diagram of a process for training a large language model according to an embodiment of the present disclosure;

[0028] Figure 4 is a schematic diagram of the overall framework of a Wensheng diagram method provided according to an embodiment of the present disclosure;

[0029] Figure 5 is a schematic diagram of the architecture of a Wensheng graph system provided according to an embodiment of the present disclosure;

[0030] Figure 6 is a processing flow chart of a Wensheng graph system provided according to an embodiment of the present disclosure;

[0031] Figure 7 is a schematic diagram of the structure of a Vincent graph device provided according to an embodiment of the present disclosure;

[0032] Figure 8 It is a block diagram of an electronic device for implementing the Vincent diagram method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0034] The terms "first", "second", etc. in this disclosure are used to distinguish similar objects, and are not necessarily used to describe a specific order or precedence. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, including a series of steps or units. The method, system, product or device is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] With the emergence of large models and the birth of multimodal models, text-generated graph technology has gained a lot of exposure, and many text-generated graph products have been born. The biggest feature of these text-generated graph products is that they generate one or more pictures by inputting a text. In the product form of text-generated graphs, there are also interactive conversational graphs, which means chatting with the system. Users constantly organize language to accurately express their needs, and then the text-generated graph product understands the user's needs and finally obtains the expected picture.

[0036] Therefore, the related art text image products have extremely high requirements on the user's input information, that is, the user is required to have good language expression ability.

[0037] Moreover, in the related art, traditional natural language processing technology is used to extract, identify and classify user input information in order to understand the intention of the user information. For example, the text input by the user is "draw a girl". In the related art, it is necessary to use traditional natural language processing technology to extract the subject "girl" and identify and classify it into "character" and "single subject". Then, based on the recognized content, according to the fixed format of describing the character, the character's face, clothing and other modifications are added to obtain prompt words for the text-based graph model reasoning. These supplementary methods use the dictionary method, which requires a large number of professionals to collect and organize, and has a fixed pattern and low flexibility.

[0038] In addition, the traditional natural language processing solutions used in related technologies require the adjustment of multiple functional modules as needed, such as extraction, recognition, and classification modules; this is not conducive to expanding the application to other raw image scenes. For example, when drawing people, animals, objects, and posters, it is necessary to adjust the functional effects of each module in a targeted manner according to different application scenarios, and each scenario needs to be adjusted and optimized separately.

[0039] In summary, in the related technologies, the prompt words required for text graphics still rely on traditional natural language processing technology, which requires high costs in maintenance, iteration, and expansion.

[0040] In view of this, the present disclosure provides a method for generating a text map. Figure 1 As shown, it is a flow chart of the method, which mainly includes the following contents:

[0041] S101, obtaining a target dialogue text for generating an image in a current round of dialogue of a target object.

[0042] The target dialogue text refers to the various related descriptions made by the target object in the current dialogue, which are used to generate images. The target object can express its requirements for images by text input or by voice input. If the requirements for images are expressed by text input, the target dialogue text is the text input by the target object. If the requirements for images are expressed by voice input, the voice can be converted into text to serve as the target dialogue text.

[0043] Regardless of the above methods, the target dialogue text can be regarded as the query currently input by the target object for processing. The target dialogue text may include expressions such as the theme, composition elements, picture effects, color matching, and style preferences of the image to be generated.

[0044] S102, constructing information to be rewritten based on the target dialogue text and historical dialogue situations.

[0045] In the disclosed embodiment, the historical dialogue situations can be divided into two situations: the current round of dialogue is the first round of dialogue, and the current round of dialogue is not the first round of dialogue.

[0046] For the case of non-first-round dialogue, that is, multi-round dialogue, it means that multiple rounds of interaction with the target object are performed to generate the desired image. In the case of multi-round dialogue, each round of dialogue is rewritten by the large language model as the current round of dialogue to obtain the corresponding prompt words.

[0047] Through the historical dialogue situation, the historical dialogue text of the target object and the historical prompt words generated by the large language model for the historical dialogue text are mainly sorted out. Among them, the historical dialogue text is the image generation requirement expressed by the target object before the current round of dialogue for the image to be generated. The historical dialogue situation mainly focuses on the historical prompt words. The historical dialogue text serves as auxiliary information for understanding the context for the large language model to understand and sort out the historical prompt words, so as to fully understand the requirements for generating images.

[0048] The information to be rewritten is constructed based on the target dialogue text of the current round and the historical dialogue situation. This information combines the specific requirements of the target object for generating images at present and the preferences expressed previously for the requirements for generating images, which is a more comprehensive understanding of the target object's requirements for generating images.

[0049] S103, inputting the information to be rewritten into the large language model to obtain target prompt words generated by the large language model according to the requirements of the target dialogue text; the target prompt words are obtained based on the text rewriting capability of the large language model.

[0050] The constructed information to be rewritten is input into the large language model, which analyzes the information to be rewritten and generates the target prompt words according to the requirements of the dialogue text of the current round of dialogue based on the text rewriting ability of the large language model itself. After being rewritten by the large language model, the target prompt words can more accurately express the requirements for generating images. For example, the requirements for generating images after rewriting are clearer, more detailed, and more professional.

[0051] S104, generating an image of the current round of dialogue based on the target prompt word.

[0052] After receiving the target prompt word, the pre-trained text-based graph model is used to generate an image. The generated image should reflect the key content and core elements of the target object's conversation text, ensuring that the visual output of the generated image is highly relevant to the topic expressed in the target conversation text.

[0053] In the disclosed embodiment, the information to be rewritten incorporates the latest requirements of the target object, that is, the requirements put forward by the target dialogue text, and the historical requirements of the historical dialogue situation for the generated image. The information to be rewritten is input into the large language model, and the target prompt words can be generated by using the language understanding ability, language organization ability and knowledge of the large language model itself. Among them, with the help of the text rewriting ability of the large language model, some colloquial, ambiguous, and inaccurate expressions that may exist in the original target dialogue text can be converted into prompt words that are more in line with the technical requirements of image generation and have clear and definite meanings, thereby generating high-quality prompt words to ultimately improve the quality of the generated image. In addition, relying on the language expression ability of the large language model, there is no need to perform extraction, recognition and classification operations like traditional natural language processing solutions, nor is there a need to use dictionary methods to collect and organize various template-based expressions. Therefore, the solution provided by the disclosed embodiment has low maintenance cost, convenient iterative optimization, and better scalability.

[0054] In the embodiment of the present disclosure, the information to be rewritten is constructed based on the target dialogue text and the historical dialogue situation, which can be implemented as follows: when the current round of dialogue is the first round of dialogue, Figure 2 As shown, in S201, the target conversation text and the target character are concatenated according to a preset format to obtain the information to be rewritten; wherein the value of the target character is empty, which is used to indicate that the historical conversation content is empty.

[0055] That is, in the case of the first round of dialogue, the target dialogue text and the target characters are spliced ​​into structured information to be rewritten according to a preset format.

[0056] When implemented, the preset format can be set to a format such as <target dialogue text, target character>. In the case of the first round of dialogue, the format meets the following requirements:

[0057] 1) Use the target field to express the target dialogue text, and the target dialogue text is the value of the target field. Use the dialogue start symbol and dialogue end symbol to express the start and end of the target dialogue text, so that the text graph model can recognize and verify the integrity of the target dialogue text.

[0058] 2) Use target characters to represent historical conversation records. If the target character is empty, it means that the current round of conversation is the first round of conversation and there is no historical conversation record.

[0059] In the disclosed embodiment, the target dialogue text and the target characters are spliced ​​in a preset format, which can ensure that the information to be rewritten is regular and logical, so that the large language model can understand and sort out the image generation requirements of the current round of dialogue and the specific content of the historical situation, which can reduce the understanding error of the large model and improve the accuracy of the target prompt words, thereby improving the quality of the generated image.

[0060] When the current round of dialogue is the latest round of dialogue in multiple rounds of dialogue, that is, the successor of multiple rounds of dialogue, the information to be rewritten is constructed based on the dialogue text and the historical dialogue situation, which can be achieved by the following steps: Figure 2 As shown:

[0061] S202, obtaining the historical conversation text provided by the target object in the historical conversation record, and the historical prompt words generated by the large language model according to the requirements of the historical conversation text.

[0062] S203, splicing the target conversation text and the historical conversation record according to a preset format to obtain the information to be rewritten; wherein, in the information to be rewritten, the historical conversation text and the historical prompt words of the historical conversation text are organized into an information tuple according to the association relationship, and the information tuple is the value of the target character.

[0063] Based on the above, the preset format can be set to a format such as <target dialogue text, target characters>. When the current dialogue round is the latest dialogue round in multiple dialogue rounds, and the current dialogue round is the second dialogue round, the format meets the following requirements:

[0064] 1) Use the target field to express the target dialogue text of the current dialogue round, and the target dialogue text is the value of the target field. Use the dialogue start symbol and dialogue end symbol to express the start and end of the target dialogue text, so that the text graph model can recognize and verify the integrity of the target dialogue text.

[0065] 2) Using target characters to represent historical conversation records, the target characters contain two key fields, namely the first subfield and the second subfield.

[0066] 3) The first subfield is used to represent the historical conversation text of the first round of conversation, and the historical conversation text is the value of the first subfield. Moreover, the historical conversation text uses the first start symbol and the first end symbol to represent the beginning and the end of the historical conversation text, and is used to identify and verify the integrity of the historical conversation text. Similarly, the second subfield represents the historical prompt word of the first round of conversation, and the historical prompt word is the value of the second subfield. Moreover, the historical prompt word uses the second start symbol and the second end symbol to represent the beginning and the end of the historical prompt word, and is used to identify and verify the integrity of the historical prompt word.

[0067] 4) Use the target start symbol and target end symbol of the historical conversation record to indicate the start and end position of the target character value, so as to identify and verify the integrity of the target character value.

[0068] In the embodiment of the disclosure, in the scenario of multiple rounds of dialogue, the dialogue text and the historical dialogue record are spliced ​​in a preset format, and the requirements for image generation expressed in the historical dialogue record can be collected, and then spliced ​​and integrated with the latest round of dialogue text to obtain the information to be rewritten. In the information to be rewritten, the information tuple is expressed in a formatted way, which can facilitate the large model to understand the drawing requirements and preferences, so as to improve the quality of the generated target prompt words.

[0069] Accordingly, during the creation process, the target object may refer to the generated image and change the requirements for generating the image, which may require multiple rounds of dialogue to iterate the drawing requirements and finally obtain the desired image. When the historical dialogue record includes multiple rounds of historical dialogues, there will be multiple groups of information tuples in the historical dialogue record. Among them, the historical dialogue text and the corresponding historical prompt words of each round of historical dialogue are constructed as corresponding information tuples.

[0070] Taking multiple rounds of historical dialogue as an example, the format of the prompt information to be rewritten meets the following requirements:

[0071] 1) Use the target field to express the target dialogue text of the current dialogue round, and the target dialogue text is the value of the target field. Use the dialogue start symbol and dialogue end symbol to express the start and end of the target dialogue text, so that the text graph model can recognize and verify the integrity of the target dialogue text.

[0072] 2) Using target characters to represent historical conversation records, the target characters contain two key fields, namely the first subfield and the second subfield.

[0073] 3) In the case of multiple rounds of historical conversations, each round of historical conversations includes a corresponding pair of first subfields and second subfields. The first subfield and the second subfield of the same round of historical conversations are represented by the third start symbol and the third end symbol. This allows the text graph model to accurately understand the historical conversation text and historical prompt words of the same round of historical conversations.

[0074] 4) In each round of historical conversation, the corresponding first subfield is used to represent the historical conversation text of the round of historical conversation, and the historical conversation text is the value of the first subfield. Moreover, the historical conversation text uses the first start symbol and the first end symbol to represent the beginning and end of the historical conversation text, which is used to identify and verify the integrity of the historical conversation text. Similarly, the second subfield represents the historical prompt word of the round of historical conversation, and the historical prompt word is the value of the second subfield. Moreover, the historical prompt word uses the second start symbol and the second end symbol to represent the beginning and end of the historical prompt word, which is used to identify and verify the integrity of the historical prompt word.

[0075] 5) Use the target start symbol and target end symbol of the historical conversation record to indicate the start and end position of the target character value, so as to identify and verify the integrity of the target character value.

[0076] In the disclosed embodiment, the historical conversation text of each round of historical conversation and the corresponding historical prompt words are constructed as corresponding information tuples, which can provide a structured storage method for conversation information. By arranging these information tuples by round, the evolution process of the target object's raw image demand can be clearly expressed, which enables the large language model to accurately capture the target object's raw image demand and preference, clarify the rewriting direction and purpose of the generated historical prompt words, and improve the quality of the rewritten target prompt words.

[0077] In the disclosed embodiment, the target object may generate one or more images using a piece of text. Each image has its own characteristics and may also have its own corresponding core content. Therefore, the prompt words used to generate each image are different and independent. Based on this, when the first target round of dialogue in the historical dialogue record generates multiple images, the historical prompt words of the first target round of dialogue include sub-prompt words corresponding to the multiple images.

[0078] That is, the multiple sub-prompt words of the multiple images and the historical dialogue text of the first target round dialogue are constructed as the information tuple of the first target round dialogue.

[0079] Taking any round of historical dialogue as an example, assuming that multiple images are generated in any round of historical dialogue, in addition to meeting the above requirements, the expression of the historical prompt words in any round of historical dialogue meets the following requirements:

[0080] 1) The historical prompt word in the second subfield includes multiple sub-prompt words, and the multiple sub-prompt words are separated by separators to facilitate identification of the sub-prompt words of each image.

[0081] 2) It can express the correspondence between each sub-prompt word and the corresponding image; when implementing, the sub-prompt words can be sorted in the same sorting order as the generated multiple images, thereby establishing the correspondence between the sub-prompt words and the images through the sorting position; in addition, special descriptors can be added to the second subfield to express the correspondence between the sub-prompt words and the images. For example, the first special symbol and the second special symbol can be used to express that the two special symbols contain a sub-prompt word and a corresponding image identifier. It can be understood that it is sufficient to be able to identify the correspondence between the sub-prompt words and the images, and the embodiments of the present disclosure are not limited to this.

[0082] Thus, a group of information tuples is constructed from the multiple sub-prompt words of the multiple images and the historical dialogue text of the first target round dialogue.

[0083] In the disclosed embodiment, by constructing multiple sub-prompt words of multiple images and the historical dialogue text of the first target round of dialogue into the information tuple of the first target round of dialogue, the large language model can understand the description of each image in each round of dialogue, so that the large language model can clarify the image generation requirements of the target object through the structured information tuple, so as to clarify the rewriting purpose and rewriting direction, and improve the quality of the generated target prompt words.

[0084] In some embodiments, the target object uploads a reference image and hopes to generate an image similar in style or content based on the reference image. In this case, in the current round of dialogue, if the target object provides a reference image, a text description of the reference image can be generated; then the text description of the reference image is used as the content of the historical prompt word corresponding to the previous round of dialogue of the current round of dialogue to construct the information to be rewritten.

[0085] That is, when the target object provides a reference image, a corresponding text description can be generated for this reference image by combining image recognition technology and natural language generation technology. The generated text description can be determined based on the target dialogue text of the current round of dialogue. For example, if the target object specifies to generate a similar subject with reference to a subject in the reference image, the subject can be identified based on image recognition technology and a corresponding description can be generated. If the target object specifies to generate an image with reference to the overall layout style of the reference image, the layout style of the reference image can be classified based on image recognition technology to obtain the corresponding layout style category.

[0086] The text description of the reference image can be used as an important basis for generating a new image. In the embodiment of the present disclosure, the text description of the reference image is used as the content of the historical prompt words corresponding to the previous round of dialogue of the current round of dialogue, so that the large language model can sort out the new image requirements of the target object in the form of multiple rounds of dialogue, and can process the reference image in the form of a compatible dialogue, so that the large model can understand and sort out the target dialogue text in one mode.

[0087] In the disclosed embodiment, the reference image uploaded by the target object is converted into a text description of the reference image, so that the text graph model can more easily understand the content of the reference image. The text description of the reference image is used as the content of the historical prompt words corresponding to the previous round of dialogue before the current round of dialogue to construct the information to be rewritten, so that the information contained in the reference image can be integrated with the previous dialogue information. The use of an information tuple expression form can be used for the large language model to sort out and understand the rewriting task, thereby ensuring that the generated target prompt words can meet expectations and improve the quality of the generated image.

[0088] In a multi-round conversation, as the number of conversation rounds increases, the time cost and resource consumption cost for processing and analyzing historical conversation data will increase accordingly. Therefore, in the embodiment of the present disclosure, when the number of conversation rounds in the historical conversation record is greater than m, the historical conversation text of m rounds of historical conversations is selected from the historical conversation record; wherein m is a positive integer greater than 1.

[0089] During implementation, the most recent m rounds of historical conversations can be selected for retention. This ensures that the generated image is more focused on the current drawing requirements of the target object, so that the generated image is more in line with the expectations of the target object. In addition, selecting m rounds of historical conversations from a large number of historical conversations can also effectively reduce the memory resources required to store and process a large number of historical conversations, thereby improving the efficiency of image generation.

[0090] Based on the content described above, historical dialogue refers to a series of exchanges with the target object around image generation before the current round of dialogue. In other words, historical dialogue includes all historical dialogue texts and corresponding target prompt words before the current round of dialogue.

[0091] Among them, when the current round of dialogue is the first round of dialogue, the target prompt word is the prompt word obtained by rewriting the dialogue text of the first round of dialogue by the large language model.

[0092] For example, the first round of dialogue text of the target object is "Please draw a two-dimensional anime style picture: the content is cyberpunk style, urban background, a man with a mechanical hand holding a combat knife, and a chip implanted in his back spine". The prompt word obtained by rewriting the dialogue text through the large language model can be "The picture shows a two-dimensional anime style scene. The man is in a cyberpunk city. One of his hands has been transformed into a mechanical hand, tightly holding a combat knife, looking mighty and domineering. And a chip can be clearly seen implanted in the spine of his back, revealing a mysterious combination of high technology and humans." It can be seen that after the rewriting of the large language model, the prompt word requirements are clearer and the style is more prominent.

[0093] In the disclosed embodiment, when the current round of dialogue is the first round of dialogue, the target prompt word is the prompt word obtained by rewriting the dialogue text by the large language model. Based on the rewriting ability of the large language model, the dialogue text of the target object can be converted into standardized and professional instructions that are more in line with the technical requirements of the text-based graph model image generation, which helps the text-based graph model to better understand the target object's image requirements.

[0094] When the current round of dialogue is the latest round of dialogue among multiple rounds of dialogue, the target prompt word is the prompt word obtained by rewriting the designated prompt word by the large language model; wherein the designated prompt word is the historical prompt word of the second target round of dialogue indicated by the dialogue text.

[0095] Taking the second round of dialogue as the current round as an example, if the current round of dialogue is the latest round of dialogue in multiple rounds of dialogue. If the dialogue text input by the target object of the current round of dialogue is "The knife should be longer, like the knife shown in the comic picture", the designated prompt word is the prompt word obtained by rewriting the dialogue text input by the first round of target object by the large language model, that is, "The picture shows a two-dimensional anime style scene. The man is in a cyberpunk city. One of his hands has been transformed into a mechanical hand, tightly holding the infantry knife, looking mighty and domineering. And in the spine position of his back, it can be clearly seen that a chip is implanted, revealing a mysterious sense of the combination of high technology and humans."

[0096] The prompt word obtained by rewriting the specified prompt word through the large language model can be "The screen shows a two-dimensional anime style scene. In the cyberpunk urban background, a man stands. One of his hands has been transformed into a mighty mechanical hand. In his hand, he holds a longer than usual combat knife, and the blade shines with a cold light. A chip is implanted in the spine of his back. The combination of high technology and humans presents a mysterious sense of the future."

[0097] That is, when generating an image, if the target dialogue text of the current round of dialogue does not clearly indicate which prompt word to rewrite, the historical prompt word of the previous round of dialogue is used as the designated prompt word by default. Similarly, when generating an image, if the target dialogue text of the current round of dialogue clearly indicates a historical prompt word, the historical prompt word is the designated prompt word.

[0098] When generating multiple images, if the target dialogue text of the current round of dialogue does not clearly indicate which prompt word to rewrite, and does not clearly indicate which image's prompt word to rewrite, the sub-prompt word of a randomly selected image in the previous round of dialogue is used as the designated prompt word by default. If the target dialogue text of the current round of dialogue clearly indicates which image's prompt word is for that round, the sub-prompt word of the image in the clearly specified round is used as the designated prompt word.

[0099] In the disclosed embodiment, in multiple rounds of conversations, the prompt words obtained by rewriting the specified prompt words through the large language model can be integrated with new raw image requirements based on the historical prompt words. The rewritten prompt words can provide more comprehensive and accurate instructions for the raw image model.

[0100] It should be noted that, when multiple images are generated in the historical round of dialogue, the user can specify the historical prompt word corresponding to at least one of the multiple images in the target dialogue text of the current round of dialogue to rewrite the prompt word. If no specification is made, the large language model will randomly select the historical prompt word corresponding to one image from the multiple images to rewrite the prompt word. The large language model can also intelligently identify and select the historical prompt word corresponding to the image with the best visual quality to rewrite the prompt word.

[0101] In some embodiments, for specific objects that the text graph model has never learned or that the text graph model cannot accurately process, the text graph model may encounter difficulties in generation, resulting in a large gap between the generated image and the expectations of the target object. To this end, during implementation, a professional may give a professional preset description of this type of specific object. The preset description may include a description of the appearance, material, characteristics, etc. of the specific object. The preset descriptions of these specific objects can be stored in a preset set accordingly. When the target dialogue text includes a specific object in the preset set, obtain the preset description of the specific object; then construct the information to be rewritten based on the preset description of the specific object, the dialogue text, and the historical dialogue situation.

[0102] During implementation, the preset description may be a manually defined detailed description of the specific object, which covers information such as the core features, typical forms, unique attributes, common associated elements, and general presentation methods of the specific object in different situations.

[0103] In the disclosed embodiment, the information to be rewritten is constructed based on a preset description of a specific object, a target conversation text, and historical conversation situations. The preset description of a specific object can assist the text graph model to better understand and generate an image that better meets the expectations of the target object, thereby reducing image deviations caused by the model's unfamiliarity with the specific object.

[0104] With the continuous development of artificial intelligence, in some scenarios, although relevant technologies can provide responses in the reply link while providing generated images, these responses are difficult to achieve personalized customization of the response content and cannot effectively improve the user experience and satisfaction of the target object.

[0105] In view of this, in the disclosed embodiment, the large language model can be trained to have the ability to rewrite the generated prompt words through fine-tuning, and the large language model can generate personalized replies based on fine-tuning. Specifically, it can be implemented as follows: obtaining the reply text generated by the large language model for the dialogue text based on the information to be rewritten; and feeding back the reply text with the generated image to the target object.

[0106] That is, the large language model is used to generate a reply text adapted to the target dialogue text based on the target dialogue text. Then, the generated image is fed back to the target object to provide the target object with a more intuitive and rich feedback experience. For example, if the target object's raw image requirement is "generate an ink painting", the large language model will generate the corresponding reply text based on the raw image requirement, such as "According to your requirements, a work of art in the style of ink painting has been completed. I hope you will like it.", "According to your inspiration and requirements, a work of art full of the charm of ink painting has been carefully created. I hope it will win your favor.".

[0107] In the disclosed embodiment, a large language model is used to generate targeted reply text for the target dialogue text based on the information to be rewritten, and a more personalized reply can be fed back according to the target object's current round of raw image needs, thereby increasing the degree of personalization of the interaction between the target object and the raw image model and improving the target object's user experience.

[0108] In the embodiment of the present disclosure, in order to enable the large language model for generating images to generate high-quality images according to the rewritten prompt words, the large language model can be obtained by training a base model with a certain basic framework, and the training method can be as follows: Figure 3 As shown, including:

[0109] S301, supervised training is performed on the base model based on the first training corpus set to obtain an intermediate model, wherein the sample prompt words of the first training corpus set are trained to align with the format requirements of the training corpus of the text-graph model for generating images.

[0110] The first training prediction set is a data set containing a large number of labeled input-output pairs, which is used to guide the model to learn a specific task. In the rewriting task, the first training prediction set can be composed of query samples and their corresponding excellent prompt word samples.

[0111] The base model is a pre-trained large language model that has not been optimized for the specific task of rewriting prompt words, but has a certain basic framework.

[0112] The base model is supervisedly trained based on the first training corpus set to obtain an intermediate model, that is, the base model is adjusted and optimized by learning the mapping relationship between the input and the expected output in the first training corpus set. During implementation, the model training can be performed through SFT (Supervised Fine-Tun-ing) training, so that the finally optimized base model is used as the intermediate model.

[0113] In order to align the training corpus of the text graph model used for the text graph, the sample prompt words of the first training corpus set are trained to align with the format requirements of the training corpus of the text graph model.

[0114] For example, the prompt word samples can be organized according to the format and order of the training corpus of the pre-text-graph model. In this way, the optimized intermediate model can align the expression and understanding of the text-graph model, ensuring that the generated target prompt words are adapted to the text-graph model to retain the original functional performance of the text-graph model.

[0115] S302, using a direct preference optimization training method, optimizing the intermediate model based on the second training corpus set to obtain a large language model.

[0116] The second training corpus set is different from the first training corpus set in that it may be more focused on refining the model's capabilities in certain specific aspects. For example, it may be different output examples corresponding to multiple groups of similar inputs, and may carry information such as preference evaluations of these output examples.

[0117] Using the preference relationship information in the second training corpus, the parameters of the intermediate model are adjusted through the DPO (Direct Preference Optimization) training method, which further improves the performance and expression of the intermediate model, and finally obtains an optimized large language model.

[0118] In the disclosed embodiment, supervised training of the base model based on the first training corpus set can make the obtained intermediate model more adaptable to the needs of specific scenarios. Among them, by aligning the text-graph model, not only can the text processing ability of the intermediate model be improved, but also the original image generation and processing ability of the text-graph model can be retained. Furthermore, direct preference optimization of the intermediate model based on the second training expected set can further optimize the model so that the obtained large language model can avoid the shortcomings of the model and make the generated content more in line with the needs of the target object.

[0119] It should be noted that the trained large language model supports text rewriting, obtaining target prompt words, and generating personalized replies. In the SFT stage, the base model can be trained through the rewriting task and the reply task. After obtaining the losses of each of the two tasks, the total loss is determined based on each loss, and then the parameters of the base model are optimized based on the total loss to obtain the intermediate model.

[0120] In the DPO stage, optimization is only performed for specific scenarios. It is sufficient to train the rewriting task of the intermediate model, and the reply task does not need to be trained.

[0121] In some embodiments, after aligning the base model and optimizing the prompt word, the content of the target prompt word includes at least one of the following and satisfies the following description order:

[0122] Description of the picture style, description of the picture subject and limitations, description of the picture details, description of the picture background modification, special effects, composition, color tone, clarity description, and quality description.

[0123] Among them, when the target prompt word includes multiple contents above, the relative order of the multiple contents meets the requirements of the above description order. For example, when the target prompt word includes: composition, picture style description, picture subject and limitation description, picture background modification description, special effects, then the description order of various information follows the above description order requirements, and the final description order is: picture style description, picture subject and limitation description, picture background modification description, special effects, composition.

[0124] The disclosed embodiment can adapt the expression specification of the text-generated graph model by standardizing the content of the target prompt words and the order of different contents, so as to generate high-quality target prompt words and improve the quality of the generated image.

[0125] In summary, the structure of the text graph method provided by the embodiment of the present disclosure is as follows: Figure 4 As shown, in S401, a sample of information to be rewritten can be constructed based on the format of <target dialogue text, target characters>, and sample prompt words and sample replies can be constructed based on the format of <prompt word example, reply text example>, thereby obtaining a training corpus. After constructing the training corpus, in S401, the base model is fine-tuned using the SFT and DPO training methods to obtain a large language model. The large language model can provide an interface to the outside world for providing inference and prediction services. The main function of this service is to achieve text rewriting, obtain high-quality target prompt words and personalized replies, and provide them to the downstream text graph model for use. Figure 4 In S403, the dialogue text input by the target object is obtained. In S404, the dialogue text and historical dialogue records are processed in the format of <target dialogue text, target character> to obtain the information to be rewritten. In S405, the information to be rewritten is input into the large language model to obtain the target prompt word and reply information to be provided to the downstream text graph model. After the downstream text graph model generates an image based on the target prompt word, it feeds back the reply information generated by the large language model to the target object.

[0126] In addition, in S406, during the online application process, prompt word examples and response examples can be collected and added to the training corpus to iteratively optimize the large language model. For example, for a new scenario, it is only necessary to build the corresponding training corpus to optimize the large language model. Among them, the following core contents are included:

[0127] (1) Information organization

[0128] That is, after the target object inputs the target dialogue text for generating the image, the target dialogue text and the historical dialogue are organized into a tuple of <target dialogue text, target character>. In the case where the current round is the first round of dialogue, the target character is used to indicate that the historical dialogue content is empty; in the case where the current round is not the first round of dialogue, the target character is used to indicate that the historical dialogue content is at least one group of information tuples formed by the historical dialogue text and the historical prompt words of the historical dialogue text according to the association relationship.

[0129] (2) Construction of training corpus

[0130] That is, based on the tuple of <target dialogue text, target character> organized and constructed by information organization, a high-quality rewriting model is used to rewrite and reply, and a prompt word example obtained by rewriting the dialogue text and a reply text example generated by the target dialogue text are obtained.

[0131] Then, the obtained prompt word examples and reply text examples are optimized and corrected manually to obtain the correct results, which are used as training corpus.

[0132] Of course, you can also collect actual conversation texts and corresponding replies and prompt words to construct training corpus.

[0133] (3) Data rewriting and acquisition

[0134] That is, the rewriting rules for the dialogue text used to generate the image are set, and the rewriting rules follow the following format and order to align the base model: picture style description, picture subject and limitation description, picture detail description, picture background modification description, special effects, composition, color tone, clarity description, quality description. In this way, the training corpus can be better aligned and the content description of the image can be matched to achieve the purpose of producing high-quality images.

[0135] The picture style description refers to the artistic style of the image, presenting the representative artistic characteristics of the image, such as the second dimension, illustration, Chinese style, ink painting, cartoon, animation, sketch, stick figure, landscape painting, etc. Realistic limited descriptions such as photo, realism, and reality are not artistic styles.

[0136] The subject of the image and the limited description refer to the subject presented in the image, which may include people, animals, plants, buildings, scenery, etc., as well as limited descriptions, such as an 18-year-old girl, blue sky, and white clouds.

[0137] The detailed description of the picture refers to the detailed description of the subject, which further refines and enriches the subject. For example, an 18-year-old girl has a delicate face and wears comfortable clothes with beautiful patterns on them.

[0138] The background modification description of the picture refers to adding background details to the subject to enhance the overall beauty of the picture and prevent the picture from appearing simple and empty. For example, the background is a green meadow with flowers and trees. Another example is that the background is blank, transparent, or pure color, which can make the picture more beautiful.

[0139] Special effects refer to beautifying and enriching the content of the picture, making it more exquisite, and can enhance the impact and appeal of the picture. There are words such as picture effects, light, focus, etc., such as: thick coating, perfect light and shadow, texture, ray tracing, Tyndall effect, blur, sharp focus, light and shadow, reflection, cg (Computer Graphics) rendering, 3d (Three Dimensions) rendering, multiple exposure, double exposure.

[0140] Composition refers to the position of the subject, that is, to organize the subject appropriately according to the subject's requirements to form a coordinated and complete picture. There are words such as lens, perspective, depth of field, such as long shot, close-up, center, close-up, ultra-wide angle, front view, and bird's-eye view.

[0141] Tone refers to the relative lightness and darkness of the picture, depicting the color of the subject and the background color, such as: white, gold, color, rich color, colorful, rich color, colorful.

[0142] Clarity description refers to the clarity of the overall picture quality, such as: high definition, ultra high definition, 8k high definition.

[0143] Quality descriptions refer to the texture and subjective feeling of the overall picture, such as: masterpiece, highest quality, master work, award-winning, immersive, simple, and dilapidated.

[0144] It is understandable that the supervised training of the base model is to rewrite the format and order requirements of the training corpus of the text-based graph model.

[0145] Therefore, in the inference phase, the rewriting ability of the large language model is also to align the format and order of the text graph model.

[0146] When rewriting, the earlier the rewriting order is, the more important it is. The content included in the rewriting order depends on the target dialogue text. For content that is not very important, if the target dialogue text does not mention it, it can be included in the target prompt words. If the target dialogue text requires image quality, the rewritten target prompt words will include a description of image quality. This applies to both the training stage and the inference stage.

[0147] (4) Risk Control

[0148] That is, the risk control model is used to judge the intention of the dialogue text input by the target object for generating images. If the judgment result shows that it cannot be drawn, involves risks, or violates the law, the corresponding reply text is returned, that is, the image cannot be generated, and the reason for not being able to generate the image can be returned.

[0149] (5) Training module

[0150] Based on the constructed training corpus, the base model is trained using the SFT method to meet new painting requirements and align the training corpus of the text-based image model. For specific scenarios and long-tail problems, DPO can be used for retraining to avoid model shortcomings and obtain a large language model.

[0151] After training the large language model, it can provide external services.

[0152] (6) Inference and prediction

[0153] The trained large language model is deployed to obtain online services. When the target object inputs a dialogue text request for generating an image, the information organization module organizes the target dialogue text for generating an image and the historical dialogues input by the target object into a tuple of <target dialogue text, target character>, and inputs it into the large language model. Then, the prompt words obtained by rewriting the target dialogue text and the reply text generated by the dialogue text can be obtained. The target prompt words can be input into the text-to-graph model to obtain the generated image.

[0154] (7) Effect Correction Module

[0155] The prompt words obtained by rewriting the dialogue text and the reply text generated by the dialogue text can be collected, and the effect can be corrected again manually to check whether there are any problems with the rewritten and reply content. The problematic marks can be corrected again manually and put into the training corpus for the next iterative training.

[0156] In summary, the Wenshengtu method provided by the embodiment of the present disclosure can improve the output effect of Wenshengtu products and the user experience of the products, and has the advantages of fast iteration and easy expansion. It can quickly support the productization of Wenshengtu models at low cost and obtain benefits.

[0157] Based on the same technical concept, the present disclosure also provides a cultural graph system, such as Figure 5 The following is a schematic diagram of the system architecture, including:

[0158] The prompt word optimization agent 501 is used to obtain the target dialogue text for generating an image of the current round of dialogue of the target object; construct the information to be rewritten based on the target dialogue text and the historical dialogue situation; input the information to be rewritten into the large language model to obtain the target prompt words generated by the large language model according to the requirements of the target dialogue text; send the target prompt words to the text map tool; the target prompt words are obtained based on the text rewriting ability of the large language model;

[0159] The text map tool 502 is used to generate an image based on the target prompt word and the text map model in the text map tool.

[0160] The prompt word optimization agent in the embodiment of the present disclosure constructs the information to be rewritten with reference to the large language model in the foregoing text, and performs a rewriting operation on it to obtain the target prompt word, which will not be described in detail here.

[0161] It should be noted that in the text-generated graph tool of the disclosed embodiment, the target dialogue text input by the target object can be sent to the prompt word optimization agent 501 for rewriting operation to obtain the target prompt word adapted to the text-generated graph model.

[0162] In order to generate personalized reply content for the target object, the prompt word optimization agent 501 in the embodiment of the present disclosure can also generate reply text for the target object text based on the method described above using a large language model.

[0163] like Figure 6 As shown, in S601, the prompt word optimization agent 501 generates the reply text while generating the target prompt word, and sends the target prompt word and the reply text together to the text map tool 502.

[0164] In S602, the text graph tool 502 calls the text graph model to generate an image according to the target prompt word, outputs the image to the target object 503, and optimizes the reply text provided by the agent 501 in conjunction with the output prompt word.

[0165] In S603, the target object 503 outputs the reply text while outputting the generated image.

[0166] For example, if the target text of the current round of dialogue is "The knife should be longer, like the knife in Japanese comics", the generated reply text is: "According to your requirements, a long Japanese comic-style knife has been generated." The target object of the next round of dialogue requires "Change the previous round to Chinese style, with the effect of ink painting", then the reply text is "The image has been modified to Chinese style, with the effect of ink painting."

[0167] As a result, the reply text obtained by the target object is targeted at the content of the current round of dialogue, is closely related to the current round of dialogue, and can improve the user experience.

[0168] Based on the same technical concept, the embodiment of the present disclosure also proposes a Wensheng map device 700, such as Figure 7 As shown, including:

[0169] An acquisition module 701 is used to acquire a target dialogue text for generating an image in a current round of dialogue of a target object;

[0170] A construction module 702 is used to construct information to be rewritten based on the target dialogue text and historical dialogue situations;

[0171] The first generation module 703 is used to input the information to be rewritten into the large language model to obtain the target prompt words generated by the large language model according to the requirements of the target dialogue text; the target prompt words are obtained based on the text rewriting ability of the large language model;

[0172] The second generation module 704 is used to generate an image of the current round of dialogue based on the target prompt word.

[0173] In some embodiments, wherein the building blocks are used to:

[0174] A first splicing unit is used to splice the target dialogue text and the target characters according to a preset format to obtain information to be rewritten when the current round of dialogue is the first round of dialogue;

[0175] The value of the target character is empty, which is used to indicate that the historical conversation content is empty.

[0176] In some embodiments, the building blocks include:

[0177] An extraction unit, for obtaining, when the current round of dialogue is the latest round of dialogue in multiple rounds of dialogue, a historical dialogue text provided by a target object in the historical dialogue record, and a historical prompt word generated by the large language model according to the requirements of the historical dialogue text;

[0178] A second splicing unit is used to splice the target conversation text and the historical conversation record according to a preset format to obtain the information to be rewritten;

[0179] Among them, in the information to be rewritten, the historical dialogue text and the historical prompt words of the historical dialogue text are organized into information tuples according to the association relationship, and the information tuple is the value of the target character.

[0180] In some embodiments, where the historical conversation record includes multiple rounds of historical conversations, the historical conversation text of each round of historical conversation and the corresponding historical prompt words are constructed as corresponding information tuples.

[0181] In some embodiments, when a first target round of dialogue in the historical dialogue record generates multiple images, the historical prompt words of the first target round of dialogue include sub-prompt words corresponding to the multiple images respectively.

[0182] In some embodiments, a building block comprises:

[0183] The image-to-text unit is used to generate a text description of a reference image when the target object provides a reference image in the current round of dialogue;

[0184] The determination unit is used to use the text description of the reference image as the content of the historical prompt words corresponding to the previous round of dialogue before the current round of dialogue, so as to construct the information to be rewritten.

[0185] In some embodiments, the extraction unit is specifically used to:

[0186] When the number of conversation rounds in the historical conversation record is greater than m, selecting the historical conversation text of m rounds of historical conversation from the historical conversation record;

[0187] Wherein, m is a positive integer greater than 1.

[0188] In some embodiments, when the current round of dialogue is the first round of dialogue, the target prompt word is a prompt word obtained by rewriting the target dialogue text by the large language model.

[0189] In some embodiments, when the current round of dialogue is the latest round of dialogue in multiple rounds of dialogue, the target prompt word is a prompt word obtained by rewriting the designated prompt word by the large language model;

[0190] The designated prompt word is a historical prompt word of the second target round of dialogue indicated by the target dialogue text.

[0191] In some embodiments, the building block is further used to:

[0192] When the target dialogue text includes a specific object in a preset set, obtaining a preset description of the specific object;

[0193] The information to be rewritten is constructed based on the preset description of the specific object, the target dialogue text, and the historical dialogue situation.

[0194] In some embodiments, a feedback module is further included for:

[0195] Obtain the reply text generated by the large language model for the target dialogue text based on the information to be rewritten;

[0196] The reply text and the generated image are fed back to the target object.

[0197] In some embodiments, a training module is further included, for training a large language model based on the following method:

[0198] The base model is supervisedly trained based on the first training corpus set to obtain an intermediate model; wherein the sample prompt words of the first training corpus set are trained to align with the format requirements of the training corpus of the text graph model of the image, wherein, when the target prompt words include multiple contents of the following, the relative order of the multiple contents meets the requirements of the following description order;

[0199] The direct preference optimization training method is adopted to optimize the intermediate model based on the second training corpus to obtain a large language model.

[0200] In some embodiments, the content of the target prompt word includes at least one of the following and satisfies the following description order:

[0201] Description of the picture style, description of the picture subject and limitations, description of the picture details, description of the picture background modification, special effects, composition, color tone, clarity description, and quality description.

[0202] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0203] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0204] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0205] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0206] like Figure 8As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0207] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0208] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the Wensheng diagram method. For example, in some embodiments, the Wensheng diagram method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the Wensheng diagram method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the Wensheng diagram method in any other appropriate manner (e.g., by means of firmware).

[0209] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0210] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0211] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0212] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0213] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0214] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0215] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0216] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating a Wensheng diagram, comprising: Obtain the target dialogue text for generating an image in the current round of dialogue of the target object; Constructing information to be rewritten based on the target dialogue text and historical dialogue situations; Inputting the information to be rewritten into a large language model to obtain a target prompt word generated by the large language model according to the requirements of the target dialogue text; The target prompt word is obtained based on the text rewriting capability of the large language model; Based on the target prompt word, an image of the current round of dialogue is generated.

2. The method according to claim 1, wherein: The constructing the information to be rewritten based on the target dialogue text and the historical dialogue situation includes: In the case where the current round of dialogue is the first round of dialogue, the target dialogue text and the target characters are concatenated according to a preset format to obtain the information to be rewritten; The value of the target character is empty, which is used to indicate that the historical conversation content is empty.

3. The method according to claim 1, wherein: The constructing the information to be rewritten based on the target dialogue text and the historical dialogue situation includes: In the case where the current round of dialogue is the latest round of dialogue in multiple rounds of dialogue, obtaining a historical dialogue text provided by the target object in the historical dialogue record, and a historical prompt word generated by the large language model according to the requirements of the historical dialogue text; splicing the target conversation text and the historical conversation record according to a preset format to obtain the information to be rewritten; Among them, in the information to be rewritten, the historical dialogue text and the historical prompt words of the historical dialogue text are organized into information tuples according to the association relationship, and the information tuple is the value of the target character.

4. The method according to claim 3, wherein: In the case where the historical dialogue record includes multiple rounds of historical dialogues, the historical dialogue text of each round of historical dialogue and the corresponding historical prompt words are constructed into corresponding information tuples.

5. The method according to claim 3, wherein: In the case where a plurality of images are generated in the first target round dialogue in the historical dialogue record, the historical prompt words of the first target round dialogue include sub-prompt words corresponding to the plurality of images respectively.

6. The method according to claim 2 or 3, further comprising: In the current round of dialogue, if the target object provides a reference image, generating a text description of the reference image; The text description of the reference image is used as the content of the historical prompt words corresponding to the previous round of dialogue of the current round of dialogue, so as to construct the information to be rewritten.

7. The method according to claim 3, wherein: The obtaining of the historical conversation text provided by the target object in the historical conversation record includes: When the number of conversation rounds in the historical conversation record is greater than m, selecting historical conversation texts of m rounds of historical conversations from the historical conversation record; Wherein, m is a positive integer greater than 1.

8. The method according to any one of claims 1 to 7, wherein: In the case where the current round of dialogue is the first round of dialogue, the target prompt word is a prompt word obtained by the large language model rewriting the target dialogue text.

9. The method according to any one of claims 1 to 7, wherein: In the case where the current round of dialogue is the latest round of dialogue in multiple rounds of dialogue, the target prompt word is a prompt word obtained by rewriting the designated prompt word by the large language model; The designated prompt word is a historical prompt word of the second target round of dialogue indicated by the target dialogue text.

10. The method according to any one of claims 1 to 9, wherein: The constructing the information to be rewritten based on the target dialogue text and the historical dialogue situation includes: When the target dialogue text includes a specific object in a preset set, obtaining a preset description of the specific object; The information to be rewritten is constructed based on a preset description of the specific object, the target dialogue text, and the historical dialogue situation.

11. The method according to any one of claims 1 to 10, further comprising: Obtaining a reply text generated by the large language model for the target dialogue text based on the information to be rewritten; The reply text and the generated image are fed back to the target object.

12. The method according to any one of claims 1 to 11, wherein: The large language model is trained based on the following method: The base model is supervisedly trained based on the first training corpus set to obtain an intermediate model; wherein the sample prompt words of the first training corpus set are aligned to generate the format requirements of the training corpus of the text-graph model of the image; The large language model is obtained by optimizing the intermediate model based on the second training corpus set using a direct preference optimization training method.

13. The method according to any one of claims 1 to 12, wherein: The content of the target prompt word includes at least one of the following and satisfies the following description order: Description of the picture style, description of the picture subject and limitations, description of the picture details, description of the picture background modification, special effects, composition, color tone, clarity description, and quality description.

14. A Wensheng map device, comprising: An acquisition module, used for acquiring a target dialogue text of a current round of dialogue of a target object for generating an image; A construction module, used to construct the information to be rewritten based on the target dialogue text and the historical dialogue situation; A first generating module, configured to input the information to be rewritten into a large language model, and obtain a target prompt word generated by the large language model according to the requirements of the target dialogue text; The target prompt word is obtained based on the text rewriting capability of the large language model; The second generation module is used to generate the image of the current round of dialogue based on the target prompt word.

15. The device according to claim 14, wherein: The building blocks are used to: A first splicing unit is used for splicing the target dialogue text and the target characters according to a preset format to obtain the information to be rewritten when the current round of dialogue is the first round of dialogue; The value of the target character is empty, which is used to indicate that the historical conversation content is empty.

16. The device according to claim 14, wherein: The building blocks include: an extraction unit, configured to obtain, when the current round of dialogue is the latest round of dialogue in multiple rounds of dialogue, a historical dialogue text provided by the target object in the historical dialogue record, and a historical prompt word generated by the large language model according to the requirements of the historical dialogue text; A second splicing unit, used for splicing the target conversation text and the historical conversation record according to a preset format to obtain the information to be rewritten; Among them, in the information to be rewritten, the historical dialogue text and the historical prompt words of the historical dialogue text are organized into information tuples according to the association relationship, and the information tuple is the value of the target character.

17. The device according to claim 16, wherein: In the case where the historical dialogue record includes multiple rounds of historical dialogues, the historical dialogue text of each round of historical dialogue and the corresponding historical prompt words are constructed into corresponding information tuples.

18. The device according to claim 16, wherein: In the case where a plurality of images are generated in the first target round dialogue in the historical dialogue record, the historical prompt words of the first target round dialogue include sub-prompt words corresponding to the plurality of images respectively.

19. The device according to claim 15 or 16, wherein the building module comprises: A picture-to-text unit, configured to generate a text description of a reference image when the target object provides a reference image in the current round of dialogue; The determination unit is used to use the text description of the reference image as the content of the historical prompt words corresponding to the previous round of dialogue before the current round of dialogue, so as to construct the information to be rewritten.

20. The device according to claim 16, wherein: The extraction unit is specifically used for: When the number of conversation rounds in the historical conversation record is greater than m, selecting historical conversation texts of m rounds of historical conversations from the historical conversation record; Wherein, m is a positive integer greater than 1.

21. The device according to any one of claims 14 to 20, wherein: In the case where the current round of dialogue is the first round of dialogue, the target prompt word is a prompt word obtained by the large language model rewriting the target dialogue text.

22. The device according to any one of claims 14 to 20, wherein: In the case where the current round of dialogue is the latest round of dialogue in multiple rounds of dialogue, the target prompt word is a prompt word obtained by rewriting the designated prompt word by the large language model; The designated prompt word is a historical prompt word of the second target round of dialogue indicated by the target dialogue text.

23. The device according to any one of claims 14 to 22, wherein: The building blocks are also used to: When the target dialogue text includes a specific object in a preset set, obtaining a preset description of the specific object; The information to be rewritten is constructed based on a preset description of the specific object, the target dialogue text, and the historical dialogue situation.

24. The apparatus according to any one of claims 14 to 23, further comprising a feedback module, configured to: Obtaining a reply text generated by the large language model for the target dialogue text based on the information to be rewritten; The reply text and the generated image are fed back to the target object.

25. The apparatus according to any one of claims 14 to 24, further comprising a training module, configured to train the large language model based on the following method: The base model is supervisedly trained based on the first training corpus set to obtain an intermediate model; wherein, The sample prompt words of the first training corpus set are aligned to generate the format requirements of the training corpus of the text graph model of the image, wherein, when the target prompt words include multiple contents of the following, the relative order of the multiple contents meets the requirements of the following description order; The large language model is obtained by optimizing the intermediate model based on the second training corpus set using a direct preference optimization training method.

26. The device according to any one of claims 14 to 25, wherein: The content of the target prompt word includes at least one of the following and satisfies the following description order: Description of the picture style, description of the picture subject and limitations, description of the picture details, description of the picture background modification, special effects, composition, color tone, clarity description, and quality description.

27. A Wensheng graph system, comprising: A prompt word optimization agent is used to obtain the target dialogue text of the current dialogue round of the target object for generating images; Constructing information to be rewritten based on the target dialogue text and historical dialogue situations; inputting the information to be rewritten into a large language model to obtain target prompt words generated by the large language model according to the requirements of the target dialogue text; sending the target prompt words to a text map tool; the target prompt words are obtained based on the text rewriting capability of the large language model; A text map tool is used to generate an image based on the target prompt word and a text map model in the text map tool.

28. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 13.

29. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-13.

30. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 13.

Images