Method and device for generating object image according to text

Abstract

The invention provides a method and device for generating an object image according to a text, and the method comprises the steps: obtaining a system instruction which is used for indicating the description of a target object according to a specified dimension; the system instruction and the target text are input into a large language model, key features, extracted by the large language model, of a target object in the target text are obtained, and the key features conform to the specified dimension; and processing the key features by using a text graph large model to generate a target image of the target object. The image generation efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of image generation technology, and in particular to a method and device for generating an object image based on text. Background Art

[0002] In film and television production and related content creation fields, character design is crucial. Traditional content often consists solely of textual descriptions, lacking visual elements. This limits the creators' and readers' intuitive understanding of the characters. In traditional productions, establishing a character's visual identity often requires hiring a dedicated professional art team. However, this requires thorough reading of the text before developing the character design, making this process inefficient. For example, character design based on a script requires a dedicated art team to thoroughly study the script before drawing the character, which is highly inefficient. Summary of the Invention

[0003] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides a method and device for generating an object image based on text.

[0004] In a first aspect, the present application provides a method for generating an object image based on text, the method comprising:

[0005] Obtaining a system instruction, wherein the system instruction is used to instruct to describe the target object according to a specified dimension;

[0006] Inputting the system instruction and the target text into a large language model to obtain key features of a target object in the target text extracted by the large language model, wherein the key features conform to the specified dimension;

[0007] The key features are processed using a large Wensheng graph model to generate a target image of the target object.

[0008] Optionally, the system instruction and the target text are input into a large language model, and the key features of the target object in the target text extracted by the large language model include:

[0009] Inputting the system instruction and target text into a large language model;

[0010] Searching the target text for relevant paragraphs describing the target object using the large language model;

[0011] Extracting the candidate features of the target object from the relevant paragraphs in combination with the context information, and obtaining a candidate feature set based on the plurality of relevant paragraphs;

[0012] Extract key features that match the specified dimension from the set of candidate features.

[0013] Optionally, extracting the candidate features of the target object from the relevant paragraphs in combination with the context information includes:

[0014] Performing word segmentation on the relevant paragraphs to obtain multiple words;

[0015] The entity of the target object is identified by using named entity recognition, and the attribute features of the vocabulary related to the target object are marked according to context information.

[0016] Optionally, the target object is a target character, and in the process of extracting the candidate features of the target object from the relevant paragraphs in combination with the context information, the method further includes:

[0017] Determining the emotions, actions, and inner monologue of the target character in the relevant paragraphs in combination with contextual information;

[0018] The character of the target character is determined based on the emotion, the action, and the inner monologue.

[0019] Optionally, after obtaining the key features of the target object in the target text extracted by the large language model, the method further includes:

[0020] Obtaining the adjusted features after the user modifies the key features;

[0021] The large language model is fine-tuned according to the adjustment features and the target text, so that the large language model learns to generate key features that meet user expectations.

[0022] Optionally, processing the key features using a large Wensheng graph model to generate a target image of the target object includes:

[0023] Integrating the key features into a structured object profile, and generating prompt words based on the structured object profile, wherein the prompt words are natural language descriptions that integrate the key features;

[0024] Get the image style specified by the user;

[0025] The target image of the target object is generated by using the cultural image model according to the prompt word and the image style.

[0026] Optionally, the target text is a script, the target object is a target character, and after generating a target image of the target object, the method further includes:

[0027] analyzing target facial features of the target character;

[0028] The target facial features are compared with the facial features of multiple actors in the image library for key points, and the actor who best fits the target role is selected.

[0029] In a second aspect, the present application provides a device for generating an object image based on text, the device comprising:

[0030] An acquisition module, configured to acquire a system instruction, wherein the system instruction is used to instruct to describe a target object according to a specified dimension;

[0031] an input / output module, configured to input the system instruction and the target text into a large language model, and obtain key features of a target object in the target text extracted by the large language model, wherein the key features conform to the specified dimension;

[0032] The processing module is used to process the key features using the Wensheng graph model to generate a target image of the target object.

[0033] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0034] Memory for storing computer programs;

[0035] The processor is configured to implement any of the steps of the method for generating an object image based on text when executing a program stored in the memory.

[0036] In a fourth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any of the steps of the method for generating an object image based on text is implemented.

[0037] The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art:

[0038] The method provided in the embodiment of the present application uses a large language model to extract key features of the target object in the text, and then uses a large text-image model to generate an image of the target object that meets the key features. By automating the process of feature extraction and image generation, the conversion time from text description to visual image is shortened. Compared with the existing technology that relies on an art team to design the image, the present application improves the efficiency of image generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] Figure 1 A schematic diagram of the hardware environment for a method of generating an object image based on text provided in an embodiment of the present application;

[0042] Figure 2 A flow chart of a method for generating an object image based on text provided in an embodiment of the present application;

[0043] Figure 3 A schematic diagram of a character generated according to an embodiment of the present application;

[0044] Figure 4 A schematic diagram of a process for generating character images based on a script provided in an embodiment of the present application;

[0045] Figure 5 A schematic diagram of the structure of a device for generating an object image based on text provided in an embodiment of the present application;

[0046] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of this application and have no specific meaning. Therefore, "module" and "component" can be used interchangeably.

[0049] In order to solve the problem mentioned in the background technology, according to one aspect of the embodiments of the present application, an embodiment of a method for generating an object image based on text is provided.

[0050] Optionally, in the embodiment of the present application, the above method for generating an object image based on text can be applied to Figure 1 In the hardware environment composed of the terminal 101 and the server 103 shown in FIG. Figure 1As shown, the server 103 is connected to the terminal 101 through a network. The user uploads text and instructions through the terminal, and the server parses the text to generate an image of the target object. A database 105 can be set on the server or independently of the server to provide data storage services for the server 103. The above-mentioned network includes but is not limited to: wide area network, metropolitan area network or local area network, and the terminal 101 includes but is not limited to PC, mobile phone, tablet computer, etc.

[0051] The following will be combined with specific implementation methods to provide a method for generating an object image based on text provided by the embodiment of the present application in detail, which is applied to a server, such as Figure 2 The specific steps are as follows:

[0052] Step 201: Obtaining a system instruction, wherein the system instruction is used to instruct to describe a target object according to a specified dimension;

[0053] Step 202: Input the system instruction and the target text into the large language model to obtain key features of the target object in the target text extracted by the large language model, wherein the key features conform to the specified dimensions;

[0054] Step 203: Use the text-based image model to process the key features and generate a target image of the target object.

[0055] Optionally, the target objects include but are not limited to: characters, props, set scenes, etc.

[0056] This application is applicable to the field of generating images of target objects based on text content. Application areas include, but are not limited to, literary creation, game development, advertising and marketing. Optionally, target objects include, but are not limited to, characters, props, and scene construction. Examples include generating script characters, script props, script scene construction, game characters, artwork, and suspect portraits.

[0057] The system instructions clearly specify the dimensions along which the target object should be described. Once the system receives the instructions and the relevant target text (such as a script excerpt or novel chapter), it passes this information to the large language model. The large language model uses its powerful natural language processing capabilities to deeply analyze the text content and extract the key features of the target object according to the dimensions specified in the system instructions. The large language model then passes these key features as input to the large text-based image generation model, which combines image generation technology and deep learning algorithms to generate a high-quality image of the target object based on the provided feature description.

[0058] System instructions can be generated based on user input or automatically. If generated based on user input, the user input can specify the target object and the specific dimensions along which to describe it. For example, if a text contains 10 characters, the user can specify that character 1 and character 2 be described along certain dimensions. If the system instructions are automatically generated, each character can be described along all dimensions, based on the objects and their dimensions in the text.

[0059] For example, the system instruction is "Based on user input, infer a character description. The answer should be simple and clear, and should include the following dimensions: name, gender, age, occupation, era, appearance, clothing, and accessories. Among them, age is expressed in numbers, appearance specifically refers to facial features, and accessories include shoes and other accessories that are characteristic of the character."

[0060] The user provides a script clip about "Li Lianhua". The large language model extracts all key features related to "Li Lianhua" from the script clip according to the system's instructions, and obtains the following content:

[0061] Name: Li Lianhua;

[0062] Gender: Male;

[0063] Age: 30-40 years old;

[0064] Occupation: Master of the Sigu Sect, doctor, traveling healer;

[0065] Era: Jianghu era;

[0066] Appearance: Gentle, pale, and calm;

[0067] Clothing: Simple clothing, suitable for a secluded life;

[0068] Accessories: No obvious accessories, possibly some medical tools.

[0069] The Wenshengtu model uses the above key features as prompts to generate character image diagrams. Figure 3 Schematic diagram of the generated character image.

[0070] Optionally, the large language model can adopt GPT (Generative Pre-trained Transformer, translated into Chinese as generative pre-trained transformer). This application does not impose any specific restrictions on the type of large language model.

[0071] Optionally, the large-scale model of the cultural image can adopt Stable Diffusion (image generation technology based on deep learning) or Midjourney (image generation technology based on generative adversarial network), etc. This application does not impose any specific restrictions on the type of the large-scale model of the cultural image.

[0072] In this application, a large language model is used to extract the key features of the target object in the text, and then a large text-image model is used to generate an image of the target object that meets the key features. By automating the process of feature extraction and image generation, the conversion time from text description to visual image is shortened. Compared with the existing technology that relies on an art team to design the image, this application improves the efficiency of image generation.

[0073] Furthermore, the combination of a large language model and an image generation model bridges the gap between the large language model's superior semantic understanding capabilities and the high-performance image generation capabilities of the large text-based image model. The resulting images are not only aesthetically pleasing but also highly faithful to the original descriptions, improving the accuracy of the generated images' alignment with the original text descriptions. Furthermore, users can flexibly adjust commands to meet their needs for more personalized results.

[0074] As an optional implementation, in step 202, inputting the system instruction and the target text into the large language model, and obtaining the key features of the target object in the target text extracted by the large language model includes the following steps:

[0075] Step S11: inputting system instructions and target text into the large language model;

[0076] Step S12: searching the target text for relevant paragraphs describing the target object using the large language model;

[0077] Step S13: extracting candidate features of the target object from relevant paragraphs in combination with context information, and obtaining a candidate feature set based on multiple relevant paragraphs;

[0078] Step S14: extract key features that match the specified dimension from the set of features to be selected.

[0079] The system first receives system instructions and target text (such as scripts, novel chapters, etc.), and this information is passed to the large language model. The large language model first performs a preliminary scan of the entire text to identify all relevant paragraphs that may involve the target object. This step relies on natural language processing technology, especially named entity recognition and semantic similarity matching algorithms, to accurately locate descriptions related to the target object. After locating the relevant paragraphs describing the target object, the large language model further analyzes the specific content in these paragraphs, extracts a series of candidate features based on contextual information, and then combines the candidate features of multiple related paragraphs to generate a candidate feature set. Finally, the system selects the most suitable key features from the candidate feature set based on the dimensions specified in the system instructions. This process ensures that the final output content strictly follows the dimensions in the system instructions while maintaining a high degree of accuracy and consistency.

[0080] If the system instructions are generated based on user instructions, then this application allows users to customize the extracted dimensions through instructions. Users can flexibly adjust the instructions according to their own needs to obtain more personalized and satisfactory results. Then, by combining contextual information and multi-paragraph analysis, the accuracy and comprehensiveness of feature extraction are improved. Finally, through dimension matching, it is ensured that the generated features fully conform to the user's specified dimensions, thereby improving the accuracy of feature extraction.

[0081] As an optional implementation, in step S13, extracting the selected features of the target object from the relevant paragraphs in combination with the context information includes: performing word segmentation on the relevant paragraphs to obtain multiple words; identifying the entities of the target object using named entity recognition, and marking the attribute features of the words related to the target object based on the context information.

[0082] After locating the relevant paragraphs describing the target object, the large language model first performs word segmentation on these paragraphs. Word segmentation is the process of dividing text into individual words or phrases. This provides the basic text units for subsequent feature extraction and analysis, ensuring that each word can be processed independently. For example, the target paragraph is: Li Lianhua is the leader of the Si Gu Men from the Jianghu era, and is also a traveling doctor. The word segmentation result is: ["Li Lianhua", "is", "a", "from", "Jianghu era", "of", "leader of the Si Gu Men", "at the same time", "also", "is", "a", "traveling doctor"].

[0083] After word segmentation is completed, the system uses named entity recognition technology for further processing to identify specific types of entities from the text, such as names of people, places, time, etc. For this application, the focus is on identifying entities that describe the target object (such as a character) and marking the vocabulary and attribute features related to the object based on contextual information. For example, "Jianghu Era" is identified as a time period, "Master of Sigumen" is identified as an occupation, and "Itinerant Doctor" is identified as an occupation. Subsequently, the key features can be selected by matching the attribute features with the specified dimensions. This method is applicable to any object, including but not limited to characters, props, and set designs.

[0084] This application uses word segmentation and named entity recognition to accurately identify and extract entities and features of objects in text, avoiding missing important information; through context understanding, it captures more implicit feature information, making the generated object image more comprehensive and three-dimensional.

[0085] As an optional implementation, if the target object is a target character, then in the process of extracting the target object's selected features from the relevant paragraphs in combination with the context information, the method also includes: determining the target character's emotions, actions, and inner monologue in the relevant paragraphs in combination with the context information; and determining the target character's personality based on the emotions, actions, and inner monologue.

[0086] If the target object is a character, some of the character's personality is not directly expressed in the text, but is reflected through the character's emotions, actions and inner monologue. In order to capture the character's personality more accurately, it is necessary to collect and gather the character's emotions, actions and inner monologue in the text.

[0087] Specifically, sentiment analysis algorithms (such as sentiment classification models based on deep learning) are used to identify emotional vocabulary and expressions in texts. For example, if the text describes "Li Lianhua smiled slightly, and a hint of helplessness flashed in her eyes," it can be identified that the action of "smiling" carries the emotional color of "helplessness."

[0088] Determine the character's emotional state (such as happiness, sadness, anger, etc.), extract the character's specific actions through verb recognition and syntactic analysis, and understand the context and meaning of these actions. For example, if the text mentions "he gently stroked the hilt of the sword, his eyes firm," the action of "stroking the hilt" can be analyzed and associated with the "firm eyes," implying the character's determination and preparation.

[0089] Use semantic understanding and thematic induction techniques to deeply interpret the character's inner activities and reveal their deep-seated thoughts and motivations. For example, when there is an inner monologue in the text, "In this troubled time, I can only rely on the medical skills in my hands to save the dying and the wounded. Although I wander around, I always have a sense of peace in my heart, hoping to bring a ray of light to the world", the character's sense of mission and inner peace can be extracted.

[0090] After capturing the target character's emotions, actions, and inner monologue, Big Language further integrates this information to infer the character's personality traits. Personality is one of the core qualities of a character, and it can be applied to the subsequent generation of the character's image. Personality can be reflected in the distribution of the character's facial features, facial features, body shape, and skin condition, making the generated character image more consistent with the character's personality in the text.

[0091] As an optional implementation, after obtaining the key features of the target object in the target text extracted by the large language model, the method also includes: obtaining adjusted features after the user modifies the key features; and fine-tuning the large language model based on the adjusted features and the target text so that the large language model learns to generate key features that meet the user's expectations.

[0092] After the large language model successfully extracts and generates key features of the target object, the system provides a user interface or API to review and modify these features. Users can adjust or supplement certain features based on their needs and preferences, such as modifying a character's appearance or adding new personality traits. Once the user has completed feature adjustments, the system uses these adjusted features along with the original target text as new training data to fine-tune the large language model.

[0093] This application introduces a memory module that enables the large language model to remember previously adjusted feature patterns, quickly responding to similar requests. Through extensive fine-tuning training, the model's generalization ability is improved, enabling it to generate high-quality features for different types of text. The fine-tuned model not only better understands the feature descriptions in the original text, but also learns the user's preferred feature adjustment methods. This means that when processing similar text in the future, the model can automatically generate key features that are closer to the user's expectations without frequent manual intervention.

[0094] This application achieves the goal of learning and optimizing the large language model from user feedback by obtaining users' adjustments to key features and fine-tuning the large language model accordingly, thereby improving the level of personalized service and enhancing the model's learning ability and adaptability.

[0095] As an optional implementation, in step 203, processing the key features using the large model of the text image to generate a target image of the target object includes the following steps:

[0096] Step S21: Integrate the key features into a structured object profile, and generate prompt words based on the structured object profile, wherein the prompt words are natural language descriptions that integrate the key features;

[0097] Step S22: Obtain the image style specified by the user;

[0098] Step S23: Generate a target image of the target object according to the prompt words and image style using the large model of text and image.

[0099] After successfully extracting and organizing the key features of the target object, the system integrates these features into a structured object profile. This profile is presented in the form of key-value pairs, with each keyword corresponding to a specific attribute (such as name, gender, age, occupation, etc.), forming a data structure that is easy to understand and use.

[0100] The following is an example of an object file:

[0101] json

[0102] {"name":"Li Lianhua",

[0103] "Gender":"Male",

[0104] "Age":"30-40 years old",

[0105] "Occupation": ["Master of Sigumen", "Itinerant Doctor"],

[0106] "Era": "Jianghu Era",

[0107] "Appearance": "gentle and pale",

[0108] "clothing": "simple clothing",

[0109] "Accessories": "No obvious accessories or medical supplies"

[0110] }

[0111] Next, the system automatically generates detailed prompts based on the structured object profile. The prompt is a natural language description that integrates all key features and is designed to be directly input into the Wenshengtu model to generate the corresponding visual image. For example, the prompt is "Generate a male character named Li Lianhua, aged between 30 and 40, the head of the Sigumen sect and a traveling doctor, living in the Jianghu era. He has a gentle appearance, pale complexion, wears simple clothes, and wears almost no ornaments, but always carries some medical tools with him."

[0112] The system provides a user interface or API that allows users to specify the style of the target object image they wish to generate. This style can be a specific artistic genre (such as realism, cartoon, futurism, etc.) or a specific emotional expression (such as warmth, coolness, mystery, etc.). The system then passes the generated prompt word and the user-specified image style to the Wenshengtu model. The model combines advanced image generation technology and deep learning algorithms to generate a high-quality target object image based on the provided prompt word and style requirements.

[0113] In this application, structured object profiles and detailed prompt words ensure that the generated images are not only beautiful but also highly faithful to the original description. In addition, by meeting the specific requirements of different image styles, the diverse needs of users are met.

[0114] As an optional implementation method, the target text is a script, the target object is a target character, and after generating a target image of the target object, the method also includes: analyzing the target facial features of the target character; comparing the target facial features with the facial features of multiple actors in the image library for key points, and selecting the actor who best fits the target role.

[0115] After successfully generating the target image of the target character, the system further performs a detailed analysis of the facial features in the image. This step aims to extract and quantify the facial features of the target character, including but not limited to the position, shape and size of facial organs such as eyes, nose, mouth, jaw line, as well as soft biometric features such as skin texture, skin color, and expression features. The system compares the key points of the above-extracted target facial features with the pre-built actor facial feature database. This process is not limited to the matching of static facial features, but also involves the simulation and comparison of dynamic features, such as the angle of the corners of the mouth when smiling, the frequency of blinking, etc., to ensure that the selected actor can highly fit the requirements of the target role in both static and dynamic performance. Based on the above comparison results, the system finally selects the actor who best fits the target role. This process may combine automated comparison scoring and human expert review opinions to ensure the best selection.

[0116] Through automated facial feature analysis and key point comparison, this application's system can quickly select the actors most suitable for the target role, significantly reducing the time and cost of casting. At the same time, precise comparison technology ensures accurate casting and reduces errors caused by subjective judgment. Detailed facial feature analysis and multi-dimensional comparison make the generated character images more realistic and vivid, ensuring that the actors are not only similar in appearance but also reflect the character's personality traits in their performance.

[0117] This application provides a flow chart of generating character images based on a script or novel, such as Figure 4As shown in the figure, after the user enters the command, the large language model is used to analyze the system command and script to generate the key features of the character image. Here, the user is supported to manually adjust the key features. Then the large model of text and image combines the key features into a prompt, generates an image based on the prompt, and outputs the image.

[0118] Based on the same technical concept, the embodiment of the present application also provides a device for generating an object image according to text, such as Figure 5 As shown, the device includes:

[0119] An acquisition module 501 is configured to acquire a system instruction, wherein the system instruction is used to instruct to describe a target object according to a specified dimension;

[0120] An input / output module 502 is configured to input the system instructions and the target text into the large language model, and obtain key features of the target object in the target text extracted by the large language model, wherein the key features conform to the specified dimensions;

[0121] The processing module 503 is used to process the key features using the cultural image model to generate a target image of the target object.

[0122] Optionally, the input / output module 502 is used to:

[0123] Input system instructions and target text into the large language model;

[0124] Use a large language model to find relevant paragraphs describing the target object from the target text;

[0125] Combined with the context information, candidate features of the target object are extracted from the relevant paragraphs, and a set of candidate features is obtained based on multiple relevant paragraphs;

[0126] Extract key features that match the specified dimension from the set of candidate features.

[0127] Optionally, the input / output module 502 is used to:

[0128] Perform word segmentation on the relevant paragraphs to obtain multiple words;

[0129] Named entity recognition is used to identify the entities of the target object, and the attribute features of the words related to the target object are marked according to the context information.

[0130] Optionally, the target object is a target character, and the device is further configured to:

[0131] Determine the target character's emotions, actions, and inner monologue in relevant paragraphs by combining contextual information;

[0132] Determine the personality of your target character based on emotions, actions, and inner monologue.

[0133] Optionally, the device is further used to:

[0134] Obtain the adjusted features after the user modifies the key features;

[0135] Fine-tune the large language model based on the adjusted features and target text so that the large language model learns to generate key features that meet user expectations.

[0136] Optionally, the processing module 503 is configured to:

[0137] Integrate the key features into a structured object profile, and generate prompt words based on the structured object profile, wherein the prompt words are natural language descriptions that integrate the key features;

[0138] Get the image style specified by the user;

[0139] The target image of the target object is generated according to the prompt words and image style using the text-image model.

[0140] Optionally, the target text is a script, and the device is further configured to:

[0141] Analyze the target facial features of the target character;

[0142] The target facial features are compared with the facial features of multiple actors in the image library for key points, and the actor who best fits the target role is selected.

[0143] Based on the same technical concept, an embodiment of the present invention further provides an electronic device, such as Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0144] Memory 603, used for storing computer programs;

[0145] The processor 601 is configured to execute the program stored in the memory 603 by performing the following steps:

[0146] Obtaining a system instruction, wherein the system instruction is used to instruct to describe the target object according to a specified dimension;

[0147] Inputting the system instruction and the target text into the large language model to obtain key features of the target object in the target text extracted by the large language model, wherein the key features conform to the specified dimensions;

[0148] The key features are processed using a large model of text and image to generate a target image of the target object.

[0149] Optionally, the system instructions and the target text are input into the large language model, and the key features of the target object in the target text extracted by the large language model include:

[0150] Input system instructions and target text into the large language model;

[0151] Use a large language model to find relevant paragraphs describing the target object from the target text;

[0152] Combined with the context information, candidate features of the target object are extracted from the relevant paragraphs, and a set of candidate features is obtained based on multiple relevant paragraphs;

[0153] Extract key features that match the specified dimension from the set of candidate features.

[0154] Optionally, the candidate features of the target object extracted from the relevant paragraphs in combination with the context information include:

[0155] Perform word segmentation on the relevant paragraphs to obtain multiple words;

[0156] Named entity recognition is used to identify the entities of the target object, and the attribute features of the words related to the target object are marked according to the context information.

[0157] Optionally, the target object is a target character, and in the process of extracting candidate features of the target object from relevant paragraphs in combination with context information, the method further includes:

[0158] Determine the target character's emotions, actions, and inner monologue in relevant paragraphs by combining contextual information;

[0159] Determine the personality of your target character based on emotions, actions, and inner monologue.

[0160] Optionally, after obtaining the key features of the target object in the target text extracted by the large language model, the method further includes:

[0161] Obtain the adjusted features after the user modifies the key features;

[0162] Fine-tune the large language model based on the adjusted features and target text so that the large language model learns to generate key features that meet user expectations.

[0163] Optionally, using a large model of text and image to process key features and generate a target image of the target object includes:

[0164] Integrate the key features into a structured object profile, and generate prompt words based on the structured object profile, wherein the prompt words are natural language descriptions that integrate the key features;

[0165] Get the image style specified by the user;

[0166] The target image of the target object is generated according to the prompt words and image style using the text-image model.

[0167] Optionally, the target text is a script, the target object is a target character, and after generating a target image of the target object, the method further includes:

[0168] Analyze the target facial features of the target character;

[0169] The target facial features are compared with the facial features of multiple actors in the image library for key points, and the actor who best fits the target role is selected.

[0170] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0171] The communication interface is used for communication between the above electronic device and other devices.

[0172] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0173] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0174] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0175] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer is enabled to execute any one of the methods in the above embodiments.

[0176] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0177] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0178] The foregoing description is intended only to provide specific embodiments of the present invention, which will enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown herein, but is intended to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for generating an object image based on text, characterized in that: The method comprises: Obtaining a system instruction, wherein the system instruction is used to instruct to describe the target object according to a specified dimension; Inputting the system instruction and the target text into a large language model to obtain key features of a target object in the target text extracted by the large language model, wherein the key features conform to the specified dimension; The key features are processed using a large Wensheng graph model to generate a target image of the target object.

2. The method according to claim 1, characterized in that The system instruction and the target text are input into a large language model, and the key features of the target object in the target text extracted by the large language model include: Inputting the system instruction and the target text into a large language model; Searching the target text for relevant paragraphs describing the target object by using the large language model; Extracting the candidate features of the target object from the relevant paragraphs in combination with the context information, and obtaining a candidate feature set based on a plurality of the relevant paragraphs; A key feature matching the specified dimension is extracted from the set of candidate features.

3. The method according to claim 2, characterized in that Extracting the candidate features of the target object from the relevant paragraphs in combination with the context information includes: Performing word segmentation processing on the relevant paragraphs to obtain multiple words; The entity of the target object is identified by using named entity recognition, and the attribute features of the vocabulary related to the target object are marked according to context information.

4. The method according to claim 3, characterized in that The target object is a target character, and in the process of extracting the candidate features of the target object from the relevant paragraphs in combination with the context information, the method further includes: Determine the emotion, action and inner monologue of the target character in the relevant paragraph in combination with context information; The character of the target character is determined based on the emotion, the action and the inner monologue.

5. The method according to claim 1, characterized in that After obtaining the key features of the target object in the target text extracted by the large language model, the method further includes: Obtaining the adjusted features after the user modifies the key features; The large language model is fine-tuned according to the adjustment features and the target text, so that the large language model learns to generate key features that meet user expectations.

6. The method according to claim 1, characterized in that The key features are processed by using the Wensheng graph model to generate a target image of the target object, including: Integrate the key features into a structured object profile, and generate a prompt word according to the structured object profile, wherein the prompt word is a natural language description that integrates the key features; Get the image style specified by the user; The target image of the target object is generated by using the Wensheng picture model according to the prompt word and the image style.

7. The method according to claim 1, characterized in that The target text is a script, the target object is a target character, and after generating a target image of the target object, the method further includes: Analyzing target facial features of the target character; The target facial features are compared with the facial features of multiple actors in the image library for key points, and the actor who best fits the target role is selected.

8. A device for generating an object image based on text, characterized in that: The device comprises: An acquisition module, used for acquiring a system instruction, wherein the system instruction is used for instructing to describe a target object according to a specified dimension; An input-output module, used for inputting the system instruction and the target text into a large language model, and obtaining key features of a target object in the target text extracted by the large language model, wherein the key features conform to the specified dimension; The processing module is used to process the key features using the Wensheng graph model to generate a target image of the target object.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 7 are implemented.

Images