Control unit for generating multiple scene images using large language model (LLM)
By generating multiple scene image variations through control units and large language models (LLM), the problems of low efficiency and image quality in traditional methods are solved, and efficient image generation based on automation and physics is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-30
- Publication Date
- 2026-03-31
AI Technical Summary
Existing technologies are inefficient and cannot be automated when generating multiple scene images. Traditional methods are cumbersome and time-consuming, and the generated images are not based on physics or contain noise and distortion.
By receiving text prompts and attribute sets through the control unit, generating multiple scene image variations using the Large Language Model (LLM), converting attribute values to predefined ranges using a mapping table, generating multiple JSON files, and rendering the images through the Unreal Engine module, the system achieves automated and physically based image generation.
It enables the efficient generation of multiple physics-based scene image variations, reduces manual operations, improves generation efficiency, and ensures image quality.
Smart Images
Figure CN121767532A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a control unit and method for generating multiple scene images using a large language model (LLM). Background Technology
[0002] Images of specific scenes of interest are composited using various traditional rendering tools (such as Blender, Unreal Engine, and Unity) by creating and rendering images from a virtual scene. The generation of the same image can be semi-automated by describing scene variations using a markup language, allowing for multiple variations of the same scene. Scene images can be generated by manually writing scene descriptions in JSON (JavaScript Object Notation) format, which acts as input to and is fed into an internal proprietary tool containing specific key-value pairs of the image. Visual language models like DALL-E and CLIP can be used to automate scene image generation by providing textual hints for the image. Attached Figure Description
[0003] Figure 1 The illustration shows a control unit for generating multiple scene images using a Large Language Model (LLM) according to an embodiment of the present invention; and Figure 2 The diagram illustrates a flowchart of a method for generating multiple scene images in a control unit using a Large Language Model (LLM) according to the present invention. Detailed Implementation
[0004] Figure 1 The illustration shows a control unit for generating multiple scene images using a Large Language Model (LLM) according to an embodiment of the present invention. The control unit 10 receives text prompts for the scene images and a set of attributes 14(a) defined for the scene images, and maps each attribute value to a value with a corresponding predefined range listed in a mapping table before providing each attribute value as input to the LLM 12, such that each attribute value is mapped to a predefined value.
[0005] Control unit 10 extracts information about a specific attribute 14(a) present in text prompt 14 and captures the value of the specific attribute 14(a) from a mapping table for generating multiple scene image variations. Control unit 10 converts the scene image variations into multiple JSON (JavaScript Object Notation) files and transmits the converted JSON files to image generation module 16. Then, control unit 10 uses rendering technology in image generation module 16 to generate multiple scene images from the received JSON files. Image generation module 16 receives the generated / produced JSON files from control unit 10, which includes a 3D scene layout, and this scene varies based on the received JSON files. Image generation module 16 finally renders the scene and sends back an image for each JSON variation.
[0006] Furthermore, the construction of control unit 10 and the components involved in control unit 10 during scene image generation. Control unit 10 is an operational device for processing and storing information received from at least one source. Control unit 10 is selected from a group of control units including microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, one or more microchips or integrated circuits interconnected using a motherboard, hard-wired logic, software stored in a memory device and executed by a microprocessor, firmware, application-specific integrated circuits (ASICs) and / or field-programmable gate arrays (FPGAs) and / or any components that operate on signals based on operating instructions. It should be noted that the control unit can be any other kind, but is not limited to the devices described above known to those skilled in the art.
[0007] The control unit 10 receives multiple elements as input: a text prompt 12, a set of attributes 14(a) associated with the text prompt 12, and a JSON file structure. These inputs are provided by a user, in this context, an Original Equipment Manufacturer (OEM). The OEM provides the output JSON file structure 14(b) along with the text prompt 14 and the set of attributes 14(a) as input. For example, text prompt 14 could be “A white female with straight hair with a backpack,” and the set of attributes 14(a) associated with text prompt 14 could be Gender, BodyShape, HeadIndex, SkinTextureIndex, SkinTextureModifierIndex, HairIndex, EyebrowIndex, Beard, HairColor, OutfitIndex, PatternOption, PatternColor, AccessoryIndex, and ScaleFactor. However, it should be understood that the attributes in this set can vary based on user requests.
[0008] Each attribute 14(a) has a predefined range of values, which is stored in the memory 18 of the control unit 10. According to the invention, all attributes 14(a) and their predefined ranges are referred to as a mapping table. For example, a user-provided structure may have defined attribute values and corresponding predefined ranges.
[0009] { "Gender": "A", / / Options: A, B "Body Shape": "Overweight", / / Options: Overweight, Underweight, Normal Weight "Head Index": 0, / / Range: 0 to 5 "SkinTextureIndex": 0, / / Range: 0 to 5 "SkinTextureModifierIndex": 0, / / Range: 0 to 35 "HairIndex": 0, / / Range: 0 to 4 "EyebrowIndex": 0, / / Range: 0 to 5 "Beard": 0, / / Range: 0 to 2 "HairColor": 0, / / Range: 0 to 7 "OutfitIndex": 0, / / Range: 0 to 10 "PatternOption": 0, / / Range: 0 to 15 "PatternColor": 0, / / Range: 0 to 3 "AccessoryIndex": 1, / / Range: 1 to 11 "ScaleFactor": 0 / / Range: 0 to 3 } Structure 14(b), including attributes, their corresponding values, and their predefined ranges, is represented in the form of a JSON file. The output JSON file structure includes multiple attribute names and the corresponding predefined range for the value of each attribute 14(a). Subsequently, for each of these attribute value numbers, the corresponding mapping is considered as input to LLM 12 so that LLM 12 understands that each number maps to its corresponding value.
[0010] For example, the control unit maps attributes and their values like this: Mapping : Gender:{A: male, B: female} For Gender: A – {HairIndex: {0: Medium-length curly hair, 1: Long straight hair, 2: Short hair...} AccessoryIndex:{1: Drinking cup, 2: Telephone, 3: Backpack...} For Gender: B – {HairIndex: {0: Playful short curly hair, 1: Playful short straight hair, 2: Short ponytail straight hair……}、AccessoryIndex:{1: Drinking cup, 2: Telephone, 3: Brown backpack, 4: Black backpack……}.
[0011] Control unit 10 generates multiple scene image variations from a single text cue 14 using thought chain cues leveraging LLM 12, wherein for attributes unrelated to text cue 14, they vary according to corresponding predefined ranges provided in the large language model 12. Control unit 10 then develops dynamic sequential thought chain cues for LLM 12, where input attribute values are dynamically obtained from the user. Control unit 10 adds, removes, and changes attributes based on requirements and generates corresponding scene images in LLM 12. It should be understood that LLM 12 can also be an intelligent model known in the prior art.
[0012] Multiple JSON files are created by converting attribute values into predefined value numbers based on a mapping table and attribute set, and the multiple JSON files created have the same structure 14(b) attribute 14(a) and attribute values within a specified range (predefined range).
[0013] Figure 2 The diagram illustrates a flowchart of a method for generating multiple scene images using a Large Language Model (LLM) 12 in a control unit 10 according to the present invention. In step S1, a text prompt 14 for the scene image and a set of attributes 14(a) defined for the scene image are received as input. In step S2, before providing each attribute value as input to the LLM 12, each attribute value is mapped to a value with a corresponding predefined range listed in a mapping table, such that each attribute value is mapped to a predefined value.
[0014] In step S3, information regarding a specific attribute 14(a) present in the text prompt 14 is extracted, and the value of the specific attribute 14(a) is captured from the mapping table to generate multiple scene image changes. In step S4, the scene image changes are converted into multiple JSON (JavaScript Object Notation) files, and the converted JSON files are sent to the image generation module 16. In step S5, the image generation module 16 uses rendering technology to generate multiple scene images from the received JSON files.
[0015] The method is explained in detail. Traditional scene image generation methods on the market are either manual or fail to generate physically accurate images. When using the Unreal Engine module / image generation module 16 to generate images, the corresponding JSON data file must be manually written. For thousands of images, this is a very tedious and time-consuming task. To mitigate this, when using the Vision Language Model 12 to automate image generation using only text prompts 14, the output images from these models are not physically based and introduce noise and distortion into the images. Similarly, for a single text prompt 14, only a small number of images can be generated. Therefore, these methods are ineffective in production-based settings. This invention provides a solution that provides multiple scene image variations by changing attributes along with their values and via multiple JSON files developed / created through these attribute changes.
[0016] Control unit 10 receives text prompts, a set of attributes, their values, and a predefined range for attribute 14(a) as input from the user. In addition to the above, control unit 10 receives at least one JSON structure from the user containing these attributes and their values along with the predefined ranges. Control unit 10 then maps the attributes and their values based on the received text prompts 14. Using thought chain prompts and a large language model 12, information about the specific attribute 14(a) mentioned in the prompts is extracted based on the input text prompts 14. For the extracted information, attribute values are captured from the mapping table.
[0017] For all other attributes 14(a) not mentioned in the text prompt, the attribute values vary according to the range provided for each of them. Therefore, using thought chain prompts leveraging LLM 12, “N” scene variations are generated from a single text prompt 14. In this document, in addition to variations, the control unit 10 also adds, removes, or alters attributes 14(a) for the scene image and can generate corresponding scenes within the LLM chain 12.
[0018] Control unit 10 then uses LLM 12 to generate JSON data files. The “N” scene variations generated in the previous steps are acquired and converted into N JSON data files by converting the values into value numbers according to the mapping table and the attribute set 14(a). Now, the N JSON files have the same structure 14(b) with attribute 14(a) and values within a specified range that will be acquired by the image generation module 16, and the generation of scene images will begin by control unit 10. Such an image generation module 16 is an Unreal Engine module 16 used to generate the N scene images. Note that the image generation module 16 can be any other type of image generation module known in the prior art that uses these text prompts 14 and the attribute set 14(a) associated with the text prompts 14.
[0019] This complete process from text prompts to JSON generation is accomplished using a novel dynamic input-based sequential thought chain prompt that employs a large language model 12. The thought chain prompting method involves the control unit 10 creating several separate chains with dynamic inputs and outputs, and then connecting them all together to generate a sequential thought chain. The control unit 10 generates N JSONs from the received text prompts 14. These JSON files are passed as HTTP requests to an API server (not shown) running with the Unreal Engine module (image generation module 16), where images are generated. The API server running the Unreal Engine module 16 has a virtual scene setup with lighting, a camera, and corresponding properties 14(a) for the scene images.
[0020] Whenever the server receives a request from the LLM 12 side containing the generated JSON data files, the JSON files are fed to the virtual scene setup one after another. The control unit 10 changes the character's appearance and corresponding attributes based on the description, and the values of the attributes in the corresponding JSON files and the final rendering are stored in the database as various scene images.
[0021] Given a text cue 14 describing an image, the novel method disclosed above uses a dynamic thought chain cue leveraging a large language model 12 to use the cue 14 and generate N variations, where N is dynamic input obtained from the user. The generated N JSON data files, in terms of variations using the large language model 12, are compatible with physically based image generation software. Automatically transferring such JSON data files to the image generation module 16 using a custom API to generate physically based images from the JSON data files is novel and provides multiple scene images compared to traditional methods.
[0022] It should be understood that the embodiments described above are merely illustrative and do not limit the scope of the invention. Many such embodiments and other modifications and variations of the embodiments described are contemplated. The scope of the invention is defined only by the scope of the claims.
Claims
1. A control unit (10) for generating multiple scene images using a large language model (LLM) (12), the control unit (10) is adapted to: receive a text prompt (14) for a scene image and a set of attributes (14(a)) defined for the scene image; map each attribute value to a value listed in a mapping table having a corresponding predefined range before providing the each attribute value as input to the LLM (12) such that each attribute value is mapped to a predefined value; extract information for a particular attribute present in the text prompt (14) and capture the value of the particular attribute from the mapping table for generating multiple scene image variations; convert the scene image variations into multiple JSON (java script object notation) files and transfer the converted multiple JSON files to an image generation module (16); generate multiple scene images from the received multiple JSON files using rendering techniques in the image generation module (16).
2. The control unit (10) according to claim 1, wherein The control unit (10) receives a structure (14(b)) of an output JSON file along with the text prompt and the set of attributes as input.
3. The control unit (10) according to claim 2, wherein The structure (14(b)) of the output JSON file includes multiple attribute names and corresponding predefined ranges of values for each attribute.
4. The control unit (10) according to claim 1, wherein The control unit (10) generates the multiple scene image variations from a single text prompt (14) using a think chain prompt with the LLM (12).
5. The control unit (10) according to claim 1, wherein For attributes (14(a)) that are not relevant to the text prompt (14), these attributes are varied according to the corresponding predefined ranges provided in the large language model (12).
6. The control unit (10) according to claim 1, wherein The control unit (10) develops dynamic sequential think chain prompts for the LLM (12) where the input attribute values are dynamically obtained from a user.
7. The control unit (10) according to claim 1, wherein The attributes (14(a)) are added, removed, and changed based on requirements and corresponding scene images are generated in the LLM (12).
8. The control unit (10) according to claim 1, wherein The multiple JSON files are created by converting the attribute values to predefined numbers according to the mapping table and the set of attributes (14(a)).
9. The control unit (10) according to claims 2 and 8, wherein, The created multiple JSON files have attributes (14(a)) with the same structure and attribute values within a specified range (predefined range).
10. A method of generating multiple scene images using a large language model (LLM) (12) in a control unit (10), the method comprising the steps of: receiving a text prompt (14) for a scene image and a set of attributes (14(a)) defined for the scene image; mapping each attribute value to a value listed in a mapping table having a corresponding predefined range before providing the each attribute value as input to the LLM (12) such that each attribute value is mapped to a predefined value; extracting information for a particular attribute present in the text prompt (12) and capturing the value of the particular attribute from the mapping table for generating multiple scene image variations; converting the scene image changes into a plurality of JSON (java script object notation) files and transmitting the converted plurality of JSON files to an image generation module (16); generating a plurality of scene images from the received plurality of JSON files using rendering techniques in the image generation module (16).