Image generation method, device, electronic device and medium

Abstract

The embodiment of the present invention provides an image generation method, device, electronic device and medium, and relates to the field of computer technology. The method includes: inputting a target image prompt word as input data into a preset large language model, parsing the target image prompt word through the large language model, and obtaining the orientation information of one or more specified persons in the vehicle interior space; determining the position code corresponding to the one or more specified persons according to the orientation information; generating a target image according to the position code corresponding to the one or more specified persons, the target image prompt word and the preset text map model, wherein the target image includes a specified vehicle and one or more specified persons, and the one or more specified persons are located at the position corresponding to the position code in the specified vehicle. The method guides the generation of the text map model through the position code, and can stably generate any vehicle interior environment, thereby improving the production efficiency and the quality of visual content generation.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image generation method, device, electronic equipment and medium. Background Art

[0002] The text-based graph technology is a large-scale model technology based on deep learning. It generates corresponding images by parsing and understanding the input description text. It can be widely used in creative design, advertising production, game development, virtual reality and other fields. However, the current text-based graph large-scale model generation effect is unstable, and it is necessary to generate large batches of text-based graph models to produce images that meet user needs.

[0003] Taking the generation of in-car scenes as an example, the current Wenshengtu large model has a general effect in restoring the structure of the vehicle's interior and is prone to violate the logic of reality. For example, the images generated by the current Wenshengtu large model cannot accurately draw driving actions (such as two people driving together in the front seat or one person driving in the back seat). Even if accurate driving actions are drawn, it is difficult to control the position of each person in a multi-person scene in the car. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present invention provide an image generation method, device, electronic device and medium.

[0005] In a first aspect, an embodiment of the present invention provides an image generation method, comprising:

[0006] Inputting the target image prompt word as input data into a preset large language model, so as to parse the target image prompt word through the large language model to obtain the position information of one or more designated persons in the space inside the vehicle; the target image prompt word includes a position description text of the one or more designated persons in the space inside the vehicle and a vehicle type description text;

[0007] Determining, according to the position information of the one or more designated persons in the vehicle interior space, position codes corresponding to the one or more designated persons; the position codes are used to indicate the positions of the one or more designated persons in the vehicle;

[0008] A target image is generated according to the position codes corresponding to the one or more specified persons, the target image prompt words and a preset text-based graph model, wherein the target image includes a vehicle corresponding to the vehicle type description text and the one or more specified persons, and the one or more specified persons are located at positions corresponding to the position codes in the vehicle.

[0009] Optionally, determining the position codes corresponding to the one or more designated persons based on the position information of the one or more designated persons in the space inside the vehicle includes: determining the position codes corresponding to the one or more designated persons based on a preset position code list and the position information of the one or more designated persons in the space inside the vehicle.

[0010] Optionally, the target image prompt word also includes a vehicle description text;

[0011] Inputting the target image prompt word as input data into a preset large language model to obtain the position information of one or more designated persons in the vehicle interior space, including: inputting the target image prompt word as input data into a preset large language model to obtain the position information of one or more designated persons in the vehicle interior space and vehicle appearance information;

[0012] The generating of the target image according to the position codes corresponding to the one or more designated persons, the target image prompt word and the preset Wensheng graph model includes: obtaining a designated vehicle model from a preset vehicle model library according to the vehicle appearance information; generating the target image according to the position codes corresponding to the one or more designated persons, the designated vehicle model, the target image prompt word and the preset Wensheng graph model.

[0013] Optionally, the target image prompt word also includes character description text;

[0014] The target image prompt word is input as input data into the preset large language model to obtain the position information of one or more specified persons in the space inside the car, including:

[0015] Inputting the target image prompt word as input data into a preset large language model to obtain description information of one or more specified persons and position information in the vehicle interior space; the description information of the one or more specified persons includes image information and / or action information of the one or more specified persons;

[0016] The method of generating a target image according to the position codes corresponding to the one or more designated persons, the target image prompt word and a preset Wensheng graph model, wherein the target image includes a vehicle corresponding to the vehicle type description text and the one or more designated persons, and the one or more designated persons are located at the position corresponding to the position code in the vehicle, includes: obtaining character models corresponding to the one or more designated persons according to the description information of the one or more designated persons; placing the character models corresponding to the one or more designated persons at the position in the vehicle corresponding to the position code in the designated vehicle model to obtain a 3D layout diagram; and rendering the 3D layout diagram based on the target image prompt word using a preset Wensheng graph model to generate a target image, wherein the target image includes a vehicle corresponding to the vehicle type description text and character models corresponding to the one or more designated persons, and the character models corresponding to the one or more designated persons are located at the position corresponding to the position code in the vehicle.

[0017] Optionally, based on the target image prompt word, rendering the 3D layout diagram using a preset text image model to generate the target image includes:

[0018] Get camera parameters;

[0019] Generate a depth map according to the camera parameters and the 3D layout map; the depth map is used to indicate the layout of the target image controlled according to the layout of the 3D layout map;

[0020] Based on the target image prompt word and the depth map, the 3D layout map is rendered using a preset text map model to generate a target image.

[0021] Optionally, the method further includes: generating an edge contour map according to the camera parameters and the 3D layout map; generating a line drawing corresponding to the target image according to the depth map, the edge contour map, the target image prompt word and a preset text image model.

[0022] Optionally, generating a line drawing corresponding to the target image according to the depth map, the edge contour map, the target image prompt word and a preset Wensheng graph model includes: binarizing the edge contour map; generating a line drawing corresponding to the target image according to the depth map, the edge contour map after binarization, the target image prompt word and a preset Wensheng graph model.

[0023] Optionally, the target image prompt word is input as input data into a preset large language model to obtain the position information of one or more specified persons in the vehicle interior space, including:

[0024] The target image prompt word and the preset question answering template are input as input data into the preset large language model to obtain the position information of one or more specified persons in the space inside the vehicle; wherein the question answering template is used to instruct the large language model to parse the logic of the target image prompt word.

[0025] In a second aspect, an embodiment of the present invention provides an image generating device, including:

[0026] A text parsing module is used to input the target image prompt word as input data into a preset large language model, so as to parse the target image prompt word through the large language model to obtain the position information of one or more specified persons in the space inside the vehicle; the target image prompt word includes a position description text of one or more specified persons in the space inside the vehicle and a vehicle type description text;

[0027] A position determination module, used to determine the position codes corresponding to the one or more designated persons according to the position information of the one or more designated persons in the vehicle interior space; the position codes are used to indicate the positions of the one or more designated persons in the vehicle;

[0028] A rendering module is used to generate a target image according to the position codes corresponding to the one or more specified persons, the target image prompt words and a preset text-based graph model, wherein the target image includes the vehicle corresponding to the vehicle type description text and the one or more specified persons, and the one or more specified persons are located at the positions corresponding to the position codes in the vehicle.

[0029] Optionally, the position determination module is used to determine the position codes corresponding to the one or more designated persons according to a preset position code list and the position information of the one or more designated persons in the vehicle interior space.

[0030] Optionally, the target image prompt word also includes a vehicle description text;

[0031] The text parsing module is also used to: input the target image prompt word as input data into a preset large language model to obtain the position information of one or more specified persons in the vehicle interior space and the vehicle appearance information;

[0032] The rendering module is used to: obtain a specified vehicle model from a preset vehicle model library according to the vehicle appearance information; and generate a target image according to the position codes corresponding to the one or more specified persons, the specified vehicle model, the target image prompt word and a preset text image model.

[0033] Optionally, the target image prompt word also includes character description text;

[0034] The text parsing module is further used to: input the target image prompt word as input data into a preset large language model to obtain description information of one or more specified persons and position information in the vehicle interior space; the description information of the one or more specified persons includes image information and / or action information of the one or more specified persons;

[0035] The rendering module is also used to: obtain character models corresponding to the one or more specified characters according to the description information of the one or more specified characters; place the character models corresponding to the one or more specified characters at the positions in the vehicle corresponding to the position codes in the specified vehicle model to obtain a 3D layout diagram; based on the target image prompt word, render the 3D layout diagram using a preset text graph model to generate a target image, wherein the target image includes the vehicle corresponding to the vehicle type description text and the character models corresponding to the one or more specified characters, and the character models corresponding to the one or more specified characters are located at the positions corresponding to the position codes in the vehicle.

[0036] Optionally, the rendering module is also used to: obtain camera parameters; generate a depth map based on the camera parameters and the 3D layout map; the depth map is used to indicate the layout of the target image controlled according to the layout of the 3D layout map; based on the target image prompt word and the depth map, render the 3D layout map using a preset Wensheng graph model to generate a target image.

[0037] Optionally, the rendering module is also used to: generate an edge contour map according to the camera parameters and the 3D layout map; generate a line drawing corresponding to the target image according to the depth map and the edge contour map, the target image prompt words and a preset text image model.

[0038] Optionally, the rendering module is also used to: binarize the edge contour map; and generate a line drawing corresponding to the target image based on the depth map, the binarized edge contour map, the target image prompt word and a preset text image model.

[0039] Optionally, the text parsing module is also used to: input the target image prompt word and the preset question answer template as input data into a preset large language model to obtain the orientation information of one or more specified persons in the space inside the vehicle; wherein the question answer template is used to instruct the large language model to parse the logic of the target image prompt word.

[0040] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; the memory is used to store computer programs; and the processor is used to implement the image generation method provided by any embodiment of the present invention when executing the program stored in the memory.

[0041] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image generation method provided by any embodiment of the present invention.

[0042] In a fifth aspect, an embodiment of the present invention provides a computer program product, which, when executed on a computer, enables the computer to execute the image generation method provided by any embodiment of the present invention.

[0043] The technical solution provided by the embodiments of the present invention brings at least the following beneficial effects:

[0044] The image generation method provided by the embodiment of the present invention inputs the target image prompt word as input data into a preset large language model, analyzes the target image prompt word through the large language model, obtains the orientation information of one or more specified persons in the vehicle interior space, and determines the position code corresponding to the one or more specified persons according to the orientation information; generates the target image according to the position code corresponding to the one or more specified persons, the target image prompt word and the preset Wensheng graph model, and the target image includes the one or more specified persons and is located at the vehicle interior position corresponding to the position code. The method uses the large language model to understand the target image prompt word, and determines the position code of one or more specified persons in the vehicle interior space in combination with the preset orientation code list, determines the layout of the one or more specified persons in the vehicle interior space according to the position code, and guides the generation of the Wensheng graph model through the layout, thereby being able to stably generate any vehicle interior environment, avoid incorrect drawing of the vehicle interior scene, effectively reduce the number of times the Wensheng graph model is generated, and improves production efficiency and the quality of visual content generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0046] Figure 1 A schematic diagram showing a flow chart of an image generating method according to an embodiment of the present invention;

[0047] Figure 2 A schematic diagram showing a flow chart of an image generating method according to another embodiment of the present invention;

[0048] Figure 3A schematic diagram showing an image generating method according to an embodiment of the present invention is shown;

[0049] Figure 4 A schematic diagram showing the structure of an image generating device according to an embodiment of the present invention is shown;

[0050] Figure 5 A schematic structural diagram of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0051] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0052] Figure 1 FIG. 2 is a flow chart of an image generation method according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0053] Step S101: input the target image prompt word as input data into a preset large language model, so as to parse the target image prompt word through the large language model to obtain the position information of one or more specified persons in the space inside the vehicle; the target image prompt word includes a position description text of one or more specified persons in the space inside the vehicle and a vehicle type description text.

[0054] Large Language Model (LLM) refers to a pre-trained language model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. Prompt refers to the input text or instruction provided to the large language model, which is used to guide the large language model to generate a specific type of response. The core of prompt is to design appropriate prompt words. Prompt words are a specific input format used to guide the pre-trained language model to produce the desired output. It can be a simple question, a longer text, or a set of instructions, a task description, or any information that can provide the context and topic required for generation. When the large language model generates content, it will first process the prompt words and then output them based on its understanding of them.

[0055] In an embodiment of the present invention, the target image prompt word may be a description text input by a user, which is used to indicate the screen content of the image to be generated (i.e., the target image). As an optional example, the target image prompt word includes but is not limited to a description of one or more specified characters in the interior space of the car and a description of the vehicle type. For example, the content of the target image prompt word may include: vehicle type: 5-seat car; descriptive text: Dad is driving, Mom is sitting in the co-pilot, and the child is sitting in the back seat on the left. The large language model parses the description text of one or more specified characters in the interior space of the car included in the target image prompt word, and outputs the orientation information of the one or more specified characters in the interior space of the car. For example, the large language model parses "Dad is driving, Mom is sitting in the co-pilot, and the child is sitting in the back seat on the left" in the target image prompt word, and outputs "main driver-Dad, co-pilot-Mom, back seat on the left-child". The large language model parses "vehicle type: 5-seat car" included in the target image prompt word, and can determine the type of vehicle, and then draw the type of vehicle in the generated target image.

[0056] In an optional embodiment, the target image prompt word also includes a vehicle description text. The vehicle description text can be used to describe the appearance of the vehicle (such as the color of the vehicle). Furthermore, the large language model understands the target image prompt word and can obtain the vehicle appearance information. Still optionally, the vehicle description text can also be used to describe the interior decoration of the vehicle. The large language model understands the target image prompt word and can obtain the vehicle interior information. As an optional example, the content of the target image prompt word may include: vehicle type: car; descriptive text: Dad is driving, mom is sitting in the passenger seat, the child is sitting in the back left seat, and the exterior color of the car is white.

[0057] In some other optional embodiments, the target image prompt word also includes a character description text. The character description text can be used to describe the image and / or action of one or more specified characters. The large language model understands the character description text and can obtain the image information and / or action information of one or more specified characters. As an optional example, the content of the target image prompt word may include: vehicle type: car; descriptive text: Dad is driving, mom is wearing a blue dress and sitting on the passenger seat, and the little girl is sitting on the left seat in the back row holding a doll.

[0058] Step S102: Determine the position codes corresponding to the one or more designated persons according to the position information of the one or more designated persons in the vehicle interior space. The position codes are used to indicate the positions of the one or more designated persons in the vehicle.

[0059] In this embodiment, different codes can be pre-assigned to different positions in the vehicle interior space, for example, seats in the vehicle are mapped to different codes, and the positions in the vehicle are indicated by the codes. Then, the position information of one or more designated persons in the vehicle interior space is mapped to the position code.

[0060] Optionally, different orientation code lists can be set for different types of vehicles. For example, for a car, in the orientation code list, No. 1 represents the main driver's seat, No. 2 represents the co-driver's seat, No. 3 represents the left side of the back row, No. 4 represents the middle position of the back row, and No. 5 represents the right side of the back row. Then, based on the vehicle type description text in the target image prompt word, the orientation code list corresponding to the type of vehicle is determined, and then based on the orientation information of one or more specified persons in the vehicle interior space and the orientation code list, the position code corresponding to one or more specified persons is determined.

[0061] In the embodiment of the present invention, by determining the position codes of one or more designated persons in the vehicle interior space, the layout of the one or more designated persons in the vehicle interior space can be accurately determined, thereby controlling the layout of the image generated by the Wensheng graph model.

[0062] Step S103: Generate a target image according to the position codes corresponding to the one or more designated persons, the target image prompt words and the preset text image model, wherein the target image includes the one or more designated persons, and the one or more designated persons are located at the positions in the vehicle corresponding to the position codes.

[0063] Among them, the preset text-generated graph model can be trained by a large number of text samples and corresponding image samples, so that it can automatically generate images that match the text description. The text-generated graph model can be trained using the training method in the prior art, and the present invention will not repeat it here. The text-generated graph model generates the expected in-vehicle image screen according to the target image prompt word, and controls the layout of the in-vehicle scene according to the position code corresponding to the one or more specified characters, so as to avoid incorrect drawing of the in-vehicle scene.

[0064] The image generation method provided by the embodiment of the present invention inputs the target image prompt word as input data into a preset large language model, analyzes the target image prompt word through the large language model, obtains the orientation information of one or more specified persons in the vehicle interior space, determines the position code corresponding to the one or more specified persons according to the preset orientation code list and the orientation information; generates the target image according to the position code corresponding to the one or more specified persons, the target image prompt word and the preset Wensheng graph model, and the target image includes the one or more specified persons and is located at the vehicle interior position corresponding to the position code. The method uses the large language model to understand the target image prompt word, and determines the position code of one or more specified persons in the vehicle interior space in combination with the preset orientation code list, determines the layout of the one or more specified persons in the vehicle interior space according to the position code, guides the generation of the Wensheng graph model through the layout, and then can stably generate any vehicle interior environment, avoids the wrong drawing of the vehicle interior scene, effectively reduces the number of times the Wensheng graph model is generated, and improves the production efficiency and the quality of visual content generation.

[0065] The image generation method provided by the embodiment of the present invention can be applied to fields such as video, advertising marketing, and games. For example, when shooting a film or TV series, producing an AI (Artificial Intelligence) video, or making a game, various in-car scenes are required. Based on the image generation method, a storyboard of one or more characters at any position in the car can be stably generated, effectively reducing the number of image generation times, thereby reducing production costs.

[0066] In an optional embodiment, the large language model can be guided by a question answering template to parse the logic of the target image prompt words, and further guided by the question answering template to disassemble the target image prompt words with more complex logic, and through a series of logically related thinking, a complete thinking process is formed, thereby accurately understanding the user's intentions and reasonably extracting information related to the character's position and action.

[0067] Therefore, the embodiment of the present invention further includes: inputting the preset question answering template and the target image prompt word as input data into the preset large language model to obtain the position information of one or more specified persons in the vehicle interior space, the vehicle appearance information and the person description information. The question answering template may include one or more prompt word samples and the response information corresponding to the prompt word samples. As an optional example, the question answering template includes:

[0068] You are a text analysis assistant. Now you are given a description: "Vehicle type: car. Descriptive text: Xiao Ming is driving, Xiao Mei is sitting in the passenger seat and is angry with him. Xiao Huang, who is sitting in the back seat of Xiao Ming, comes out to mediate." Through analysis, you find that Xiao Ming is in the driver's seat, Xiao Mei is in the passenger seat, and Xiao Huang is in the left back seat, so your answer is "Vehicle type: car; Position information: Xiao Ming - driver's seat, Xiao Mei - passenger seat, Xiao Huang - back seat on the left.

[0069] The embodiment of the present invention displays one or more examples to the large language model through a preset question answering template, and controls the logic of the large language model to parse the target image prompt words, thereby guiding the large language model to obtain a more accurate answer.

[0070] Figure 2 FIG. 2 is a flow chart showing an image generation method according to another embodiment of the present invention. Figure 2 As shown, the method includes:

[0071] Step S201: Obtain target image prompt words input by the user. The target image prompt words include vehicle type description text, one or more specified characters description text, one or more specified characters position description text in the vehicle interior space and vehicle description text. The character description text is used to indicate the image and action of the character. The position description text is used to indicate the position of the character in the vehicle interior space. The vehicle description text is used to indicate the appearance of the vehicle.

[0072] Step S202: Input the target image prompt word and the preset question answer template as input data into the preset large language model, so as to parse the target image prompt word through the large language model to obtain description information of one or more specified persons, position information of one or more specified persons in the space inside the vehicle, vehicle type information and vehicle appearance information.

[0073] Step S203: determining position codes corresponding to one or more designated persons according to a preset position code list and position information of one or more designated persons in the vehicle interior space.

[0074] Steps S201-S203 can refer to Figure 1 The illustrated embodiments are not described in detail here to avoid repetition.

[0075] Step S204: acquiring a specified vehicle model from a preset vehicle model library according to the vehicle type information and the vehicle appearance information.

[0076] The vehicle model library is a preset 3D model library, and the 3D model includes vehicle models of various types such as buses, cars, SUVs (Sport Utility Vehicles), and the like.

[0077] Step S205: Generate a 3D layout diagram according to the position codes corresponding to one or more designated persons and the designated vehicle model.

[0078] In this step, one or more humanoid models are obtained from a preset character model library according to the description information of one or more characters. The preset humanoid model library includes a variety of humanoid models, such as adult male models and adult female models of various body shapes, models of children of various ages, etc. The humanoid model has a skeleton binding, and the action can be freely adjusted. All actions have fixed codes, which can be adjusted automatically according to the action information. Then, according to the position codes corresponding to one or more characters, the humanoid models corresponding to the one or more characters are placed at the positions corresponding to the position codes of the specified vehicle model. According to the character action information extracted from the target image prompt words, the humanoid models on each position code are physically adjusted to present the specified action.

[0079] Step S206: Obtain camera parameters.

[0080] The camera parameters include camera position and focus information. The camera parameters may be preset default parameters or user-set parameters, for example, the user selects a camera position number from preset camera position numbers and sets the camera parameters according to the camera position number.

[0081] The embodiment of the present invention uses camera parameters to render images based on the camera's viewing angle, so that the images generated by the text graph model are more realistic and more in line with the real scene.

[0082] Step S207: Generate a depth map according to the camera parameters and the 3D layout map.

[0083] A depth map is an image used to represent the three-dimensional information of a scene. In a depth map, the value of each pixel (denoted as the depth value) represents the depth of the pixel from the camera. The depth value is a value between 0 and 1. The closer the object is to the camera, the closer the depth value of the corresponding pixel is to 1. The farther the object is from the camera, the closer the depth value of the corresponding pixel is to 0.

[0084] In this step, the depth of any point in the interior space of the designated vehicle model from the camera can be calculated based on the camera position and focus information in the camera parameters, and the position of each designated person in the interior space of the designated vehicle model can be determined based on the 3D layout diagram, and then the depth of each person model placed at the position from the camera can be determined. Thus, a depth map is generated based on the depth of any point in the interior space of the designated vehicle model from the camera and the depth of each person model placed in the designated vehicle model from the camera.

[0085] Step S208: Generate a target image according to the depth map, the target image prompt word and the preset text image model.

[0086] This step can be combined with the pre-trained depth model and depth map to guide the generation of the Wensheng graph model, and control the Wensheng graph model to generate a target image with a layout consistent with the layout of the 3D layout map and a sense of hierarchy and space. As an optional example, the depth map can be passed as input data into the pre-trained depth model, and the model extracts the depth features of the depth map, such as the values ​​of each pixel point, and inputs the features into the diffusion model to change the diffusion mode of the diffusion model, thereby guiding the image generation process of the Wensheng graph model. Among them, the diffusion model is a generation model that can generate new data samples based on training data. In an embodiment of the present invention, a series of pictures of in-car scenes are used as training data to train the diffusion model so that it learns the patterns of various objects in the in-car scenes in a real environment. In this embodiment, the depth features extracted from the depth map by the depth model are input into the diffusion model, and the diffusion model is guided to simulate the patterns of various objects in the in-car scene under the depth features. The diffusion mode of the diffusion model under the depth feature guides the Wensheng graph model, so that it can generate a high-quality target image.

[0087] The image generation method provided by the embodiment of the present invention inputs a target image prompt word and a preset question answering template as input data into a preset large language model, guides the large language model to parse the target image prompt word through the question answering template, obtains the orientation information of one or more specified persons in the vehicle interior space, and determines the position codes corresponding to the one or more specified persons according to the preset orientation code list and the orientation information; generates a 3D layout diagram according to the position codes corresponding to the one or more specified persons and the specified vehicle model, renders using the 3D layout diagram and camera parameters to obtain a depth map, and inputs the depth map and the target image prompt word as input data into a preset text map model to generate a target image. The method uses a large language model to understand the target image prompt word, and determines the position code of one or more specified persons in the vehicle interior space in combination with a preset orientation code list, determines the layout of the one or more specified persons in the vehicle interior space according to the position code, obtains a 3D layout map, generates a depth map using camera image parameters and the 3D layout map, and uses the depth map to control the screen layout generated by the text map model to be consistent with the layout of the 3D layout map, thereby being able to stably generate any vehicle interior environment, avoid incorrect drawing of the vehicle interior scene, effectively reduce the number of times the text map model is generated, and improve production efficiency and the quality of visual content generation.

[0088] Figure 3 FIG. 2 is a schematic diagram showing an image generation method according to another embodiment of the present invention. Figure 3 As shown, in Figure 2Based on the shown embodiment, the image generation method of this embodiment also includes: generating an edge contour map according to camera parameters and a 3D layout map; generating a line drawing corresponding to the target image according to the depth map and the edge contour map, the target image prompt words and a preset text image model.

[0089] Among them, the camera can be driven to render the picture according to the camera parameters and the 3D layout diagram to obtain a normal rendered image. The edge contour map is extracted from the rendered image using the HED algorithm. The edge contour map has a white background and a black edge line. The edge contour map is used to guide the generation of a cleaner background when the line drawing is generated. The HED algorithm (Holistically-Nested Edge Detection) is an edge detection algorithm based on deep learning, which is mainly used in the fields of image processing and computer vision.

[0090] In this embodiment, the edge contour map is encoded into a latent space vector using VAE (Variational autoEncoder), and the Wensheng graph model is drawn on this basis. Because the Wensheng graph model is affected by the preset deep model, it tends to generate an image that conforms to the layout, and the edge contour map conforms to such a layout. Therefore, when generating the line drawing, the pre-trained line drawing style Wensheng graph model is used. This large model tends to transform the style of the picture while retaining the original content to obtain the line drawing.

[0091] In an optional embodiment, after obtaining the edge contour map, it is also necessary to perform a binarization process on the edge contour map. The binarization process refers to setting the grayscale value of the pixel points on the image to 0 or 255 so that the entire image presents an obvious black and white effect. The embodiment of the present invention uses the edge contour map after the binarization process to generate a line drawing of the target image, which can effectively highlight the outlines of various objects in the line drawing and improve the image quality.

[0092] Figure 4 FIG. 2 shows a schematic diagram of the structure of an image generating device provided by an embodiment of the present invention. Figure 4 As shown, the image generating device 400 includes:

[0093] The text parsing module 401 is used to input the target image prompt word as input data into a preset large language model, so as to parse the target image prompt word through the large language model to obtain the position information of one or more specified persons in the space inside the vehicle; the target image prompt word includes a position description text of one or more specified persons in the space inside the vehicle and a vehicle type description text;

[0094] The position determination module 402 is used to determine the position codes corresponding to the one or more designated persons according to a preset position code list and the position information of the one or more designated persons in the vehicle interior space; the position codes are used to indicate the positions of the one or more designated persons in the vehicle;

[0095] The rendering module 403 is used to generate a target image according to the position codes corresponding to the one or more specified persons, the target image prompt words and the preset text-based graph model, wherein the target image includes the vehicle corresponding to the vehicle type description text and the one or more specified persons, and the one or more specified persons are located at the position corresponding to the position code in the vehicle.

[0096] Optionally, the position determination module is used to determine the position codes corresponding to the one or more designated persons according to a preset position code list and the position information of the one or more designated persons in the vehicle interior space.

[0097] Optionally, the target image prompt word also includes a vehicle description text;

[0098] The text parsing module is also used to: input the target image prompt word as input data into a preset large language model to obtain the position information of one or more specified persons in the vehicle interior space and the vehicle appearance information;

[0099] The rendering module is used to: obtain a specified vehicle model from a preset vehicle model library according to the vehicle appearance information; and generate a target image according to the position codes corresponding to the one or more specified persons, the specified vehicle model, the target image prompt word and a preset text image model.

[0100] Optionally, the target image prompt word also includes character description text;

[0101] The text parsing module is further used to: input the target image prompt word as input data into a preset large language model to obtain description information of one or more specified persons and position information in the vehicle interior space; the description information of the one or more specified persons includes image information and / or action information of the one or more specified persons;

[0102] The rendering module is also used to: obtain character models corresponding to the one or more specified characters according to the description information of the one or more specified characters; place the character models corresponding to the one or more specified characters at the positions in the vehicle corresponding to the position codes in the specified vehicle model to obtain a 3D layout diagram; based on the target image prompt word, render the 3D layout diagram using a preset text graph model to generate a target image, wherein the target image includes the vehicle corresponding to the vehicle type description text and the character models corresponding to the one or more specified characters, and the character models corresponding to the one or more specified characters are located at the positions corresponding to the position codes in the vehicle.

[0103] Optionally, the rendering module is also used to: obtain camera parameters; generate a depth map based on the camera parameters and the 3D layout map; the depth map is used to indicate the layout of the target image controlled according to the layout of the 3D layout map; based on the target image prompt word and the depth map, render the 3D layout map using a preset Wensheng graph model to generate a target image.

[0104] Optionally, the rendering module is also used to: generate an edge contour map according to the camera parameters and the 3D layout map; generate a line drawing corresponding to the target image according to the depth map and the edge contour map, the target image prompt words and a preset text image model.

[0105] Optionally, the rendering module is also used to: binarize the edge contour map; and generate a line drawing corresponding to the target image based on the depth map, the binarized edge contour map, the target image prompt word and a preset text image model.

[0106] Optionally, the text parsing module is also used to: input the target image prompt word and the preset question answer template as input data into a preset large language model to obtain the orientation information of one or more specified persons in the space inside the vehicle; wherein the question answer template is used to instruct the large language model to parse the logic of the target image prompt word.

[0107] The above device can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the image generation method provided by the embodiment of the present invention.

[0108] Figure 5 FIG. 1 is a schematic diagram showing the structure of an electronic device according to an embodiment of the present invention. Figure 5 As shown, the electronic device includes:

[0109] Processor 501, communication interface 502, memory 503 and communication bus 505, wherein the processor 501, communication interface 502, memory 503 communicate with each other via the communication bus 505.

[0110] Memory 503, used for storing computer programs;

[0111] The processor 501 is used to execute the program stored in the memory 503, and implements the following steps:

[0112] Inputting the target image prompt word as input data into a preset large language model, so as to parse the target image prompt word through the large language model to obtain the position information of one or more designated persons in the space inside the vehicle; the target image prompt word includes a position description text of the one or more designated persons in the space inside the vehicle and a vehicle type description text;

[0113] Determine the position codes corresponding to the one or more designated persons according to a preset position code list and the position information of the one or more designated persons in the vehicle interior space; the position codes are used to indicate the positions of the one or more designated persons in the vehicle;

[0114] A target image is generated according to the position codes corresponding to the one or more specified persons, the target image prompt words and a preset text-based graph model, wherein the target image includes a vehicle corresponding to the vehicle type description text and the one or more specified persons, and the one or more specified persons are located at positions corresponding to the position codes in the vehicle.

[0115] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0116] The communication interface is used for communication between the above terminal and other devices.

[0117] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0118] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0119] In another embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions. When the instructions are executed on a computer, the computer executes the image generation method described in any one of the above embodiments.

[0120] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes the image generating method described in any one of the above embodiments.

[0121] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0122] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0123] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0124] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. An image generation method, characterized in that: include: Inputting the target image prompt word as input data into a preset large language model, so as to parse the target image prompt word through the large language model to obtain the position information of one or more designated persons in the vehicle interior space; The target image prompt words include one or more texts describing the position of a designated person in the vehicle interior and a text describing the vehicle type; Determining position codes corresponding to the one or more designated persons according to the position information of the one or more designated persons in the vehicle interior space; The position code is used to indicate the position of the one or more designated persons in the vehicle; A target image is generated according to the position codes corresponding to the one or more specified persons, the target image prompt words and a preset text-based graph model, wherein the target image includes a vehicle corresponding to the vehicle type description text and the one or more specified persons, and the one or more specified persons are located at positions corresponding to the position codes in the vehicle.

2. The method according to claim 1, characterized in that: Determining the position codes corresponding to the one or more designated persons based on the position information of the one or more designated persons in the space inside the vehicle includes: determining the position codes corresponding to the one or more designated persons based on a preset position code list and the position information of the one or more designated persons in the space inside the vehicle.

3. The method according to claim 1, characterized in that The target image prompt words also include vehicle description text; The target image prompt word is input as input data into the preset large language model to obtain the position information of one or more specified persons in the space inside the car, including: Input the target image prompt words as input data into a preset large language model to obtain the position information of one or more specified persons in the vehicle interior space and the vehicle appearance information; The step of generating a target image according to the position codes corresponding to the one or more designated characters, the target image prompt words and a preset text image model includes: According to the vehicle appearance information, obtaining a specified vehicle model from a preset vehicle model library; A target image is generated according to the position codes corresponding to the one or more designated persons, the designated vehicle model, the target image prompt words and a preset text image model.

4. The method according to claim 3, characterized in that The target image prompt words also include character description text; The target image prompt word is input as input data into the preset large language model to obtain the position information of one or more specified persons in the space inside the car, including: Inputting the target image prompt word as input data into a preset large language model to obtain description information of one or more specified persons and position information in the vehicle interior space; the description information of the one or more specified persons includes image information and / or action information of the one or more specified persons; The step of generating a target image according to the position codes corresponding to the one or more designated persons, the target image prompt words and a preset text image model, wherein the target image includes a vehicle corresponding to the vehicle type description text and the one or more designated persons, and the one or more designated persons are located at positions corresponding to the position codes in the vehicle, including: Acquire character models corresponding to the one or more specified characters according to the description information of the one or more specified characters; Placing character models corresponding to the one or more designated characters at vehicle positions corresponding to the position codes in the designated vehicle model to obtain a 3D layout diagram; Based on the target image prompt word, the 3D layout diagram is rendered using a preset text image model to generate a target image, wherein the target image includes a vehicle corresponding to the vehicle type description text and a character model corresponding to the one or more specified characters, and the character model corresponding to the one or more specified characters is located at a position corresponding to the position code in the vehicle.

5. The method according to claim 4, characterized in that The method of rendering the 3D layout diagram based on the target image prompt word using a preset text image model to generate a target image includes: Get camera parameters; Generate a depth map according to the camera parameters and the 3D layout map; the depth map is used to indicate the layout of the target image controlled according to the layout of the 3D layout map; Based on the target image prompt word and the depth map, the 3D layout map is rendered using a preset text map model to generate a target image.

6. The method according to claim 5, characterized in that The method further comprises: Generate an edge contour map according to the camera parameters and the 3D layout map; A line drawing corresponding to the target image is generated according to the depth map, the edge contour map, the target image prompt word and a preset text image model.

7. The method according to claim 6, characterized in that The step of generating a line drawing corresponding to the target image according to the depth map, the edge contour map, the target image prompt word and a preset text image model includes: Binarization processing is performed on the edge contour map; A line drawing corresponding to the target image is generated according to the depth map, the edge contour map after binarization processing, the target image prompt word and a preset text image model.

8. The method according to any one of claims 1 to 7, characterized in that: The target image prompt word is input as input data into the preset large language model to obtain the position information of one or more specified persons in the space inside the car, including: The target image prompt word and the preset question answering template are input as input data into the preset large language model to obtain the position information of one or more specified persons in the space inside the vehicle; wherein the question answering template is used to instruct the large language model to parse the logic of the target image prompt word.

9. An image generating device, characterized in that: include: A text parsing module, used for inputting the target image prompt word as input data into a preset large language model, so as to parse the target image prompt word through the large language model to obtain the position information of one or more specified persons in the vehicle interior space; The target image prompt words include one or more texts describing the position of a designated person in the vehicle interior and a text describing the vehicle type; A position determination module, used to determine the position codes corresponding to the one or more designated persons according to the position information of the one or more designated persons in the vehicle interior space; The position code is used to indicate the position of the one or more designated persons in the vehicle; A rendering module is used to generate a target image according to the position codes corresponding to the one or more specified persons, the target image prompt words and a preset text-based graph model, wherein the target image includes the vehicle corresponding to the vehicle type description text and the one or more specified persons, and the one or more specified persons are located at the positions corresponding to the position codes in the vehicle.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, configured to implement the method according to any one of claims 1 to 8 when executing a program stored in a memory.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, when running on a computer, enables the computer to execute the method according to any one of claims 1 to 8.

Images