Three-dimensional scene construction method, device and equipment

Abstract

The invention relates to the technical field of artificial intelligence and computer graphics, and provides a three-dimensional scene construction method, device and equipment, and the method comprises the steps: obtaining an original scene description, inputting the original scene description into a scene decomposition agent, obtaining a plurality of hierarchical scene descriptions decomposed by the scene decomposition agent, the scene decomposition agent is realized based on a first large language model; a target scene construction intelligent agent is utilized, a target layered scene description in the multiple layered scene descriptions serves as input, a target layered three-dimensional scene is generated, and the target scene construction intelligent agent is achieved based on a second large language model; and integrating the target layered three-dimensional scene, and determining a target three-dimensional scene. According to the technical scheme provided by one or more embodiments of the invention, the three-dimensional virtual scene can be constructed efficiently, vividly and controllably.

Description

Technical Field

[0001] The present disclosure relates to the technical fields of artificial intelligence and computer graphics, and in particular to a three-dimensional scene construction method, device and equipment. Background Art

[0002] In recent years, the demand for large-scale three-dimensional virtual scenes has increased in various applications, such as designing room and town layouts, developing video games and metaverses, and training artificial intelligence agents.

[0003] In game production, large-scale virtual 3D scenes are an indispensable part. For example, in open world games, it is necessary to build a huge world view and environment. In the fields of aviation, navigation, military training, etc., using simulators for practical operation training can effectively reduce risks and costs. For example, flight simulators, ship driving simulations, autonomous driving tests, etc. Large-scale virtual scenes are the key to achieving immersive experience. Users can freely explore and interact in these environments, such as virtual reality games, online exhibitions, etc.

[0004] However, creating large-scale 3D virtual scenes for these applications generally requires manual design, real environment scanning, and hard-coded rules, which requires a lot of manpower. Therefore, more efficient methods are needed to create large-scale 3D virtual scenes. Summary of the invention

[0005] In view of this, one or more embodiments of the present disclosure provide a three-dimensional scene construction method, device and equipment, which can efficiently, realistically and controllably construct a three-dimensional virtual scene.

[0006] In a first aspect, the present disclosure provides a three-dimensional scene construction method, the method comprising: obtaining an original scene description, and inputting the original scene description into a scene decomposition agent to obtain multiple hierarchical scene descriptions decomposed by the scene decomposition agent, wherein the scene decomposition agent is implemented based on a first large language model; using a target scene construction agent, taking a target hierarchical scene description in the multiple hierarchical scene descriptions as input to generate a target hierarchical three-dimensional scene, wherein the target scene construction agent is implemented based on a second large language model; integrating the target hierarchical three-dimensional scene to determine the target three-dimensional scene.

[0007] In a second aspect, the present disclosure provides a three-dimensional scene construction device, which includes: a scene decomposition module, which is used to obtain an original scene description and input the original scene description into a scene decomposition agent to obtain multiple hierarchical scene descriptions decomposed by the scene decomposition agent, and the scene decomposition agent is implemented based on a first large language model; a scene construction module, which is used to use a target scene construction agent, and takes a target hierarchical scene description in the multiple hierarchical scene descriptions as input to generate a target hierarchical three-dimensional scene, and the target scene construction agent is implemented based on a second large language model; a scene synthesis module, which is used to integrate the target hierarchical three-dimensional scene and determine the target three-dimensional scene.

[0008] In a third aspect, the present disclosure provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the three-dimensional scene construction method of the first aspect by executing the computer instructions.

[0009] In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the three-dimensional scene construction method of the first aspect.

[0010] In a fifth aspect, the present disclosure provides a computer program product, including computer instructions, and the computer instructions are used to enable a computer to execute the three-dimensional scene construction method of the first aspect.

[0011] The technical solution provided by one or more embodiments of the present disclosure can be used by a scene decomposition agent to hierarchically decompose a large-scale three-dimensional scene construction task into multiple subtasks, that is, the original scene description is decomposed into multiple hierarchical scene descriptions. Each subtask can be assigned to a target scene construction agent with specific functions for execution. After receiving the target hierarchical scene description, the target scene construction agent can generate a corresponding target hierarchical three-dimensional scene. By integrating multiple levels of target hierarchical three-dimensional scenes, a comprehensive large-scale three-dimensional environment can be constructed.

[0012] In the technical solutions provided by one or more embodiments of the present disclosure, both the scene decomposition agent and the target scene construction agent are implemented based on the large language model technology, so that the construction process from natural language text to three-dimensional virtual scene can be realized. The collaborative work of multiple agents forms a multi-agent system, which improves the efficiency of building three-dimensional scenes.

[0013] The technical solution provided by one or more embodiments of the present disclosure not only focuses on the overall scene understanding, but also realizes the reasoning of three-dimensional spatial relationships. In the process of constructing a large-scale three-dimensional scene, the three-dimensional scene construction task is decomposed into multi-level target layered three-dimensional scenes, which can realize multi-dimensional space layout planning. On the one hand, the realism of the constructed three-dimensional scene is enhanced; on the other hand, the single target layered three-dimensional scene can be optimized, and the constructed three-dimensional scene is more flexible and customizable. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The features and advantages of the embodiments of the present disclosure may be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present disclosure in any way. In the accompanying drawings:

[0015] Figure 1 A schematic diagram showing the steps of a method for constructing a three-dimensional scene in one embodiment of the present disclosure is shown;

[0016] Figure 2 A schematic diagram of a process for decomposing an original scene description based on a large language model in one embodiment of the present disclosure is shown;

[0017] Figure 3 A schematic diagram of a process of constructing a three-dimensional scene in one embodiment of the present disclosure is shown;

[0018] Figure 4 Another schematic diagram of a process for constructing a three-dimensional scene in one embodiment of the present disclosure is shown;

[0019] Figure 5 A schematic diagram of functional modules of a three-dimensional scene construction device in one embodiment of the present disclosure is shown;

[0020] Figure 6 A schematic diagram of the hardware structure of a computer device in one embodiment of the disclosure is shown. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0022] The three-dimensional scene construction method provided in one embodiment of the present disclosure can be applied to an electronic device with data processing capabilities. For example, the electronic device can be a personal computer, a workstation, etc. In addition, the method can also be applied to software running in the above electronic device.

[0023] See also Figure 1 A three-dimensional scene construction method provided in one embodiment of the present disclosure may include the following steps.

[0024] S1: Obtain an original scene description, and input the original scene description into a scene decomposition agent to obtain multiple hierarchical scene descriptions decomposed by the scene decomposition agent, wherein the scene decomposition agent is implemented based on the first language model.

[0025] In this embodiment, the Large Language Model (LLM) technology is an artificial intelligence technology or application. The LLM can understand and generate human language and process multiple information types, such as text, images, and audio. The LLM can promote more natural and intelligent human-computer interaction, support the reception and generation of various data, and be easy to use and safe.

[0026] In this embodiment, the scene decomposition agent implemented based on the first language model can perform complex tasks. The scene decomposition agent can use LLM as a "hub" to coordinate the sequence operations of completing tasks or respond to user requests. The key modules included in the scene decomposition agent generally also involve a planning module, a memory module and a tool call module. The scene decomposition agent can realize task automation by integrating core components such as the LLM module, the planning module and the memory module. The first language model that the scene decomposition agent relies on can be pre-trained and can efficiently complete the task of decomposing the original scene description. The first language model that the scene decomposition agent relies on can be updated and optimized.

[0027] In this embodiment, the scene decomposition agent can hierarchically decompose the large-scale 3D scene construction task into multiple subtasks, that is, the original scene description is decomposed into multiple hierarchical scene descriptions. Each subtask can be assigned to a target scene construction agent with specific functions for execution.

[0028] In this embodiment, according to the actual application scenario, decomposing the original scene description can be achieved by using prompt word engineering technology. By improving the prompt words by means of structured text, the user's original prompt words can be standardized, and the task of the first language model is to decompose the original scene description into multiple hierarchical scene descriptions, and guide the first language model to give the desired multiple hierarchical scene descriptions.

[0029] In a practical application example, the scene decomposition agent can use the first language model to parse the original scene description at multiple levels. For example, see Figure 2, an original scene description can be decomposed into five hierarchical scene descriptions: terrain description, meteorological environment description, biological community description, road and building description, and vehicle equipment description.

[0030] It should be noted that, according to different actual application scenarios, the scene decomposition agent can divide different hierarchical scene descriptions from multiple perspectives. These hierarchical scene descriptions can correspond to the celestial universe (such as the sun, moon, stars, etc.), mountains and rivers, natural phenomena (such as weather, geology, biology, chemistry, physics, etc.), products of human activities (such as buildings, transportation, tools and equipment, artworks, food, etc.), life forms (such as plants, animals, microorganisms, etc.), etc.

[0031] In some implementations, the original scene description may be input into a scene decomposition agent. The scene decomposition agent may be used to complete the original scene description to obtain an extended scene description containing multiple preset category descriptions. The scene decomposition agent may be used to decompose the extended scene description to obtain multiple hierarchical scene descriptions.

[0032] In a practical application example, the scene decomposition agent can perform multi-level analysis on the original scene description. Using the first language model, the original scene description can be expanded and deepened to form a specific and detailed description, such as topography, weather conditions, ecosystems, transportation facilities and equipment, etc. For example, the scene decomposition agent can complete the original scene description to ensure that the completed scene description covers at least five aspects of topography, weather conditions, biomes, road buildings, and vehicle equipment.

[0033] S2: Using the target scene to construct an intelligent agent, taking the target hierarchical scene description among the multiple hierarchical scene descriptions as input, and generating a target hierarchical three-dimensional scene, wherein the target scene construction intelligent agent is implemented based on the second largest language model.

[0034] In this embodiment, the target scene construction agent implemented based on the second language model can perform complex tasks. The target scene construction agent can use LLM as a "hub" to coordinate the sequence operations of completing tasks or respond to user requests. The key modules included in the target scene construction agent generally also involve a planning module, a memory module and a tool call module. The target scene construction agent can realize task automation by integrating core components such as LLM modules, planning modules and memory modules. The second language model that the target scene construction agent relies on can be pre-trained and can efficiently complete the creation of the target layered three-dimensional scene. The second language model that the target scene construction agent relies on can be updated and optimized.

[0035] In this embodiment, after receiving the target hierarchical scene description, the target scene construction agent can generate a target hierarchical three-dimensional scene accordingly. The target scene construction agent is implemented based on the large language model technology, so that the construction process from natural language text to three-dimensional virtual scene can be realized.

[0036] In this embodiment, the target scene construction agent can realize the construction of a specific three-dimensional scene, including but not limited to scene construction and design work in terms of terrain distribution, biological community distribution, road network, building location distribution, meteorological conditions, etc.

[0037] It should be noted that the first language model and the second language model can represent that the scene decomposition agent and the target scene construction agent are implemented based on different dedicated large language models. For example, the large language model A included or called by the scene decomposition agent focuses on the scene decomposition task, and the large language model B included or called by the target scene construction agent focuses on the creation of the target layered three-dimensional scene. Depending on the actual application scenario, the second language model can also be different dedicated large language models. For example, when the target scene construction agent is used to construct a three-dimensional scene of terrain and landforms, the large language model B1 included or called by it focuses on the creation of the three-dimensional scene of terrain and landforms; when the target scene construction agent is used to construct a three-dimensional scene of meteorological environment, the large language model B2 included or called by it focuses on the creation of the three-dimensional scene of meteorological environment; when the target scene construction agent is used to construct a three-dimensional scene of biological community, the large language model B3 included or called by it focuses on the creation of the three-dimensional scene of biological community; when the target scene construction agent is used to construct a three-dimensional scene of road buildings, the large language model B4 included or called by it focuses on the creation of the three-dimensional scene of road buildings; when the target scene construction agent is used to construct a three-dimensional scene of vehicle equipment, the large language model B5 included or called by it focuses on the creation of the three-dimensional scene of vehicle equipment.

[0038] The first and second language models can represent that the scene decomposition agent and the target scene construction agent are respectively implemented based on different instances of a comprehensive large language model. For example, the scene decomposition agent drives the instance C1 of the all-round large language model C to complete the scene decomposition task. The target scene construction agent drives other instances of the all-round large language model C to complete the creation of the target layered three-dimensional scene. According to different actual application scenarios, when the target scene construction agent is used to construct a three-dimensional scene of terrain and landforms, it can drive the instance C2 of the universal large language model C to complete the creation of the three-dimensional scene of terrain and landforms; when the target scene construction agent is used to construct a three-dimensional scene of meteorological environment, it can drive the instance C3 of the universal large language model C to complete the creation of the three-dimensional scene of meteorological environment; when the target scene construction agent is used to construct a three-dimensional scene of biological community, it can drive the instance C4 of the universal large language model C to complete the creation of the three-dimensional scene of biological community; when the target scene construction agent is used to construct a three-dimensional scene of road and building, it can drive the instance C5 of the universal large language model C to complete the creation of the three-dimensional scene of road and building; when the target scene construction agent is used to construct a three-dimensional scene of vehicle and equipment, it can drive the instance C6 of the universal large language model C to complete the creation of the three-dimensional scene of vehicle and equipment.

[0039] In this embodiment, the target layered scene description can be divided into regions using the target scene construction agent to obtain a target layered local description. The target scene construction agent can determine at least one of the target layered editing parameters and the target layered programmatic content generation parameters based on the target layered local description. The target layered programmatic content generation parameters may include model type parameters, random range parameters, and distribution parameters. Using the target layered editing parameters and the target layered programmatic content generation parameters, the target scene construction agent can drive the preset three-dimensional scene construction software to determine the target layered three-dimensional scene.

[0040] Specifically, the built-in functions of the target scene construction agent include but are not limited to regional division, parameterized description and software drive. The target scene construction agent can divide the target layered scene into regions through the knowledge precipitated in the second language model, combined with the target layered scene description, to obtain multiple target layered local areas. The division can be a fuzzy rough division or a precise grid division. The target scene construction agent can describe each divided target layered local area in detail. Subsequently, based on the existing target layered local description, the target scene construction agent can generate an accurate parameterized description of the local area. The parameterized description generally includes basic global parameter settings and specific programmatic content generation (PCG) parameter settings (such as the selected basic static model type, random range, distribution parameters, etc.). Finally, the target scene construction agent can use the parameterized description to drive the scene setting function (such as PCG function) of the three-dimensional scene construction software to construct a complex three-dimensional scene.

[0041] Among them, PCG is a toolset that integrates a variety of tools, which can be used to custom develop procedural content and tools in some 3D scene construction software. Through PCG, technical artists, designers, and developers can easily create tools and content of various complexities, covering a wide range from specific assets (such as buildings or ecological communities) to complete worlds. Using PCG technology, algorithms can be used to create nuanced local environments, including elements such as terrain, buildings, and vegetation. These elements focus on authenticity and diversity, thereby enhancing the realism and immersive experience of the scene. Compared with manual production, PCG technology quickly generates content that meets specific needs through programming, especially when dealing with large amounts of content or random generation, which greatly reduces labor costs.

[0042] However, in related technologies, the global planning and detailed management of large-scale and complex three-dimensional scenes usually require the assistance of human experience, and there are problems such as low efficiency, lack of consistency and authenticity. The threshold for PCG creation is high and requires certain programming skills. For designers or artists who are not familiar with the code, it is difficult to learn how to use and create PCG tools. When using PCG, developers may not be able to control every detail as accurately as the preset templates, and the controllability is limited. Manually adjusting the parameters of PCG is time-consuming and error-prone, and the process is complicated and cumbersome.

[0043] In contrast, the technical solution provided by this embodiment relies on a pre-trained large language model, which can automatically generate corresponding PCG setting parameters based on the description of a specific scene. These PCG parameters can be directly applied to some 3D scene construction software to efficiently build local environments full of random elements or large-scale content.

[0044] The technical solution provided by this embodiment, based on the large-scale three-dimensional scene layout planning capability of the second largest language model, can use LLM to understand and simulate the distribution patterns of various elements such as terrain, ecosystem, transportation facilities, weather, etc. in the real world, so as to realize the design of large-scale three-dimensional scenes. Through in-depth analysis and learning of rich spatial layout data, LLM can automatically generate logically clear regional divisions, terrain and landform design, biome configuration, road network layout, building location settings, and meteorological conditions. The three-dimensional large language model can accept three-dimensional points and text prompts with features as input, and specializes in performing various tasks related to three-dimensional space. The technical solution provided by this embodiment pays special attention to the application in overall scene understanding, three-dimensional spatial relationship reasoning and planning, which goes beyond the conventional task scope of traditional large language models or two-dimensional large language models. For three-dimensional scene construction, the technical solution provided by this embodiment focuses on creating regional feature designs that are logical and conform to spatial relationships, which can cover multi-dimensional spatial layout planning such as terrain, ecosystem, transportation facilities, vehicle distribution, and climate conditions.

[0045] In some embodiments, the target scene construction agent may include a terrain and landform agent. Using the target scene construction agent, taking the target hierarchical scene description in the multiple hierarchical scene descriptions as input to generate a target hierarchical three-dimensional scene may include: using the terrain and landform agent to divide the terrain and landform description in the target hierarchical scene description into regions to obtain a local description of the terrain and landform; the terrain and landform agent determines terrain editing parameters and terrain programmatic content generation parameters based on the local description of the terrain and landform; using the terrain editing parameters and the terrain programmatic content generation parameters, the terrain and landform agent drives the three-dimensional scene construction software to determine the terrain and landform three-dimensional scene.

[0046] In a practical application example, a terrain and landform agent can be responsible for adjusting terrain and landforms, such as mountains, rivers, lakes, etc., to meet ecological diversity and geographical authenticity. First, based on LLM, the terrain and landform layered three-dimensional scene can be regionally planned and divided into several areas, each of which generates different local feature descriptions of the terrain and landforms. For a single area, the terrain (landscape) parameters and PCG parameters of the three-dimensional scene construction software can be generated based on LLM, that is, the terrain editing parameters and the landform PCG parameters. The terrain editing function (such as the height map import function) and the PCG plug-in function of the three-dimensional scene construction software can be used to generate a terrain and landform three-dimensional scene.

[0047] In some embodiments, the target scene construction agent may include a meteorological environment agent. Using the target scene construction agent, taking the target hierarchical scene description in the multiple hierarchical scene descriptions as input to generate a target hierarchical three-dimensional scene may include: using the meteorological environment agent to divide the meteorological environment description in the target hierarchical scene description into regions to obtain a local description of the meteorological environment; the meteorological environment agent determines weather editing parameters and lighting editing parameters based on the local description of the meteorological environment; using the weather editing parameters and the lighting editing parameters, the meteorological environment agent drives the three-dimensional scene construction software to determine the meteorological environment three-dimensional scene.

[0048] In a practical application example, the meteorological environment agent can combine terrain conditions to configure a variety of meteorological environments and lighting conditions, such as sunny, rainy, snowy, foggy, etc., to create realistic weather effects. First, based on LLM, the meteorological environment layered three-dimensional scene can be regionally planned and divided into several areas, each of which generates different local feature descriptions of the meteorological environment. For a single area, the weather, lighting and other parameters of the three-dimensional scene construction software can be generated based on LLM. The weather system plug-in (such as Ultra dynamic sky) and lighting system plug-in of the three-dimensional scene construction software can be used to generate a meteorological environment three-dimensional scene.

[0049] In some embodiments, the target scene construction agent may include a biome agent. Using the target scene construction agent, taking the target hierarchical scene description in the multiple hierarchical scene descriptions as input to generate a target hierarchical three-dimensional scene may include: using the biome agent to divide the biome description in the target hierarchical scene description into regions to obtain a local description of the biome; the biome agent determines the biome programmatic content generation parameters based on the local description of the biome; using the biome programmatic content generation parameters, the biome agent drives the three-dimensional scene construction software to determine the biome three-dimensional scene.

[0050] In a practical application example, the biome agent can configure suitable vegetation communities and animal communities in combination with terrain and climate conditions to achieve the construction of a natural ecosystem. First, based on LLM, the biome layered three-dimensional scene can be regionally planned and divided into several regions, each of which generates different biome local feature descriptions. For a single region, the biome PCG parameters of the three-dimensional scene construction software can be generated based on LLM. The biome three-dimensional scene can be generated by using the PCG plug-in and biome plug-in (such as Procedural Biome Core) functions of the three-dimensional scene construction software.

[0051] In some embodiments, the target scene construction agent may include a road building agent. Using the target scene construction agent, taking the target hierarchical scene description in the multiple hierarchical scene descriptions as input to generate a target hierarchical three-dimensional scene may include: using the road building agent to divide the road building description in the target hierarchical scene description into regions to obtain a local description of the road building; the road building agent determines the road building programmatic content generation parameters based on the road building local description; using the road building programmatic content generation parameters, the road building agent drives the three-dimensional scene construction software to determine the road building three-dimensional scene.

[0052] In a practical application example, the road building intelligent agent can configure reasonable roads and buildings based on the conditions of topography, meteorological environment, and biological communities, taking into account implicit needs such as traffic flow and functional requirements. First, based on LLM, the layered three-dimensional scene of road construction can be regionally planned to generate a road and building layout that conforms to the actual design rules, divided into several areas, and each area generates a different description of the local characteristics of the road building. For a single area, the PCG parameters of the road building of the three-dimensional scene construction software can be generated based on LLM. The three-dimensional scene of the road building can be generated by using the PCG plug-in and road building plug-in (such as Procedural City Generator) of the three-dimensional scene construction software.

[0053] In some embodiments, the target scene construction agent may include a vehicle equipment agent. Using the target scene construction agent, taking the target hierarchical scene description in the multiple hierarchical scene descriptions as input to generate a target hierarchical three-dimensional scene may include: using the vehicle equipment agent to divide the vehicle equipment description in the target hierarchical scene description into regions to obtain a local description of the vehicle equipment; the vehicle equipment agent determines the vehicle equipment programmatic content generation parameters based on the vehicle equipment local description; using the vehicle equipment programmatic content generation parameters, the vehicle equipment agent drives the three-dimensional scene construction software to determine the vehicle equipment three-dimensional scene.

[0054] In a practical application example, the vehicle equipment agent can configure elements such as transportation tools and vehicle equipment according to the needs of the scene to make the scene more dynamic and story-telling. First, based on LLM, the layered three-dimensional scene of vehicle equipment can be regionally planned to generate a vehicle equipment layout that conforms to the actual design rules and is divided into several areas. Each area generates a different local feature description of the distribution of vehicle equipment. For a single area, the PCG parameters of the vehicle equipment distribution of the three-dimensional scene construction software can be generated based on LLM. Using the PCG plug-in function of the three-dimensional scene construction software, the vehicle equipment three-dimensional scene can be constructed.

[0055] S3: Integrate the target layered three-dimensional scene to determine the target three-dimensional scene.

[0056] In this embodiment, a comprehensive large-scale three-dimensional environment can be constructed by integrating multi-level target layered three-dimensional scenes.

[0057] In some embodiments, the target layered three-dimensional scene may be integrated to determine a candidate three-dimensional scene. Subsequently, the candidate three-dimensional scene may be evaluated. If the candidate three-dimensional scene meets the preset expectations, the candidate three-dimensional scene may be determined as the target three-dimensional scene. If the candidate three-dimensional scene does not meet the preset expectations, an improved scene description may be added to the original scene description, and the candidate three-dimensional scene may be updated based on the original scene description and the improved scene description.

[0058] The target 3D scene can be evaluated and optimized through user feedback or artificial intelligence algorithms. By adding missing or improved scene descriptions to the original scene description and continuously iterating this process, the quality of the scene description can be improved to match the pre-set goals and expectations.

[0059] The technical solution provided in this embodiment includes a scene decomposition agent and multiple target scene construction agents, forming a multi-agent system (Multi-Agent System, MAS). In particular, the output data of the scene decomposition agent can be input to different target scene construction agents for subsequent processing, and jointly complete the complex large-scale three-dimensional scene creation work, which belongs to a distributed processing technology. MAS combined with LLM integrates the powerful analytical and comprehension capabilities of LLM, and improves the interactive efficiency and problem-solving capabilities of the multi-agent system.

[0060] See also Figure 3 The three-dimensional scene construction method provided in one embodiment of the present disclosure can be implemented by the following steps:

[0061] 1. Construct scene decomposition agent, terrain agent, meteorological environment agent, biological community agent, road construction agent, and vehicle equipment agent.

[0062] 2. Receive the original scene description input by the user.

[0063] 3. Input the original scene description into the scene decomposition agent to complete the original scene description, ensuring that the completed description covers at least the above five aspects: topography, meteorological environment, biological community, road buildings, and vehicle equipment. Then, through hierarchical decomposition processing, enrich the content of each part layer by layer to obtain multiple hierarchical scene descriptions.

[0064] 4. Input the terrain description into the terrain agent, perform regional planning on the scene based on LLM, divide it into several regions, and generate different terrain feature descriptions for each region. For a single region, generate UE's HeightMap or LandScape parameters and PCG parameters based on LLM, namely terrain editing parameters and terrain PCG parameters.

[0065] 5. Input the meteorological environment description into the meteorological environment agent, perform regional planning on the scene based on LLM, divide it into several areas, and generate different meteorological environment feature descriptions for each area. For a single area, generate UE weather, light and other parameters based on LLM.

[0066] 6. Input the biome description into the biome agent, perform regional planning on the scene based on LLM, divide it into several regions, and generate different biome feature descriptions for each region. For a single region, generate the biome PCG parameters of the UE based on LLM.

[0067] 7. Input the road building description into the road building agent, perform regional planning on the scene based on LLM, generate road and building layouts that conform to actual design rules, divide them into several regions, and generate different road building feature descriptions for each region. For a single region, generate UE road building PCG parameters based on LLM.

[0068] 8. Input the vehicle equipment description into the vehicle equipment agent, perform regional planning on the scene based on LLM, generate a vehicle equipment layout that meets the actual design rules, divide it into several areas, and generate different vehicle equipment distribution feature descriptions for each area. For a single area, generate the UE's vehicle equipment distribution PCG parameters based on LLM.

[0069] 9. Use the parametric description obtained in the above steps 4-8 to directly drive the corresponding basic functions of the 3D scene construction software and the PCG plug-in functions (such as Procedural Content Generation, UltraDynamic Sky, Procedural Biome Core, Procedural City Generator, etc.) in the 3D scene construction software to perform multi-layer scene synthesis, thereby generating a corresponding large-scale scene.

[0070] 10. Evaluate the scenario constructed in step 9 using expert feedback or intelligent algorithm recognition, add the missing or improved scenario description to the original scenario description, and iterate steps 2-9 repeatedly to improve the quality of scenario construction until it matches the pre-set goals and expectations.

[0071] For some practical application examples, see Figure 4, users can describe the imagined scene through natural language (which can be voice input, text input, etc.) to form an original scene description. For example, the text content of an original scene description can be as follows: Welcome to "Multi-ecological Realm: Green Shadow Dawn Land", a treasure of the three-dimensional virtual world. The landscape here is rich and diverse, with forests, grasslands and continuous mountains complementing each other, and lakes and rivers interspersed among them, forming a vivid natural picture. Roads and buildings are scattered on the surface, showing signs of human settlement and development. Several vehicles of game elements are either moving or stationary, adding a sense of dynamics to this virtual world. The bright sunshine under the clear sky is like the dawn of the day, providing players with clear and visible navigation conditions. Green plants of different shades grow like trees and bushes, creating an immersive natural environment. Time is frozen in the daytime, and the distribution of minerals and natural resources on the surface suggests the possibility of economic systems and resource management. Explore this vast world and experience the rich adventures brought by the diverse ecology.

[0072] In the technical solution provided by this embodiment, the programmed PCG significantly improves the efficiency of scene construction, and the time to transform concepts into actual environments is greatly shortened; multiple agents work together to form a multi-agent system, which enhances the realism of the generated scene and makes it closer to real life; with the help of LLM knowledge, the layout and common sense rationality of the three-dimensional scene are optimized, and the logical rationality of the constructed three-dimensional scene is improved; strong customization is reflected through interactive adjustment and iterative processes, which can meet the unique needs of different users for scenes. The technical solution provided by this embodiment has a wide range of applicability and can be applied to multiple fields such as game development, virtual reality, and the construction of test scenes, and can provide users with a realistic three-dimensional environment with a high degree of customization and large scale.

[0073] The technical solution provided by one or more embodiments of the present disclosure can be used by a scene decomposition agent to hierarchically decompose a large-scale three-dimensional scene construction task into multiple subtasks, that is, the original scene description is decomposed into multiple hierarchical scene descriptions. Each subtask can be assigned to a target scene construction agent with specific functions for execution. After receiving the target hierarchical scene description, the target scene construction agent can generate a corresponding target hierarchical three-dimensional scene. By integrating multiple levels of target hierarchical three-dimensional scenes, a comprehensive large-scale three-dimensional environment can be constructed.

[0074] In the technical solutions provided by one or more embodiments of the present disclosure, both the scene decomposition agent and the target scene construction agent are implemented based on the large language model technology, so that the construction process from natural language text to three-dimensional virtual scene can be realized. The collaborative work of multiple agents forms a multi-agent system, which improves the efficiency of building three-dimensional scenes.

[0075] The technical solution provided by one or more embodiments of the present disclosure not only focuses on the overall scene understanding, but also realizes the reasoning of three-dimensional spatial relationships. In the process of constructing a large-scale three-dimensional scene, the three-dimensional scene construction task is decomposed into multi-level target layered three-dimensional scenes, which can realize multi-dimensional space layout planning. On the one hand, the realism of the constructed three-dimensional scene is enhanced; on the other hand, the single target layered three-dimensional scene can be optimized, and the constructed three-dimensional scene is more flexible and customizable.

[0076] See also Figure 5 The present disclosure also provides a three-dimensional scene construction device, the device comprising:

[0077] A scene decomposition module 100 is used to obtain an original scene description and input the original scene description into a scene decomposition agent to obtain multiple hierarchical scene descriptions decomposed by the scene decomposition agent, wherein the scene decomposition agent is implemented based on the first language model;

[0078] A scene construction module 200, configured to construct an agent using a target scene, taking a target hierarchical scene description from the plurality of hierarchical scene descriptions as input, and generating a target hierarchical three-dimensional scene, wherein the target scene construction agent is implemented based on the second language model;

[0079] The scene synthesis module 300 is used to integrate the target layered three-dimensional scene to determine the target three-dimensional scene.

[0080] In one embodiment, the scene construction module 200 is specifically used to utilize the target scene construction agent to divide the target hierarchical scene description into regions to obtain a target hierarchical local description; the target scene construction agent determines at least one of a target hierarchical editing parameter and a target hierarchical programmatic content generation parameter based on the target hierarchical local description, and the target hierarchical programmatic content generation parameters include a model type parameter, a random range parameter, and a distribution parameter; utilizing the target hierarchical editing parameters and the target hierarchical programmatic content generation parameters, the target scene construction agent drives a preset three-dimensional scene construction software to determine the target hierarchical three-dimensional scene.

[0081] In one embodiment, the target scene construction agent includes a terrain and geomorphology agent, and the scene construction module 200 is specifically used to use the terrain and geomorphology agent to divide the terrain and geomorphology description in the target hierarchical scene description into regions to obtain a local description of the terrain and geomorphology; the terrain and geomorphology agent determines terrain editing parameters and terrain programmatic content generation parameters based on the local description of the terrain and geomorphology; using the terrain editing parameters and the terrain programmatic content generation parameters, the terrain and geomorphology agent drives the three-dimensional scene construction software to determine the terrain and geomorphology three-dimensional scene.

[0082] In one embodiment, the target scene construction agent includes a meteorological environment agent, and the scene construction module 200 is specifically used to use the meteorological environment agent to divide the meteorological environment description in the target hierarchical scene description into regions to obtain a local description of the meteorological environment; the meteorological environment agent determines weather editing parameters and lighting editing parameters based on the local description of the meteorological environment; using the weather editing parameters and the lighting editing parameters, the meteorological environment agent drives the three-dimensional scene construction software to determine the three-dimensional scene of the meteorological environment.

[0083] In one embodiment, the target scene construction agent includes a biome agent, and the scene construction module 200 is specifically used to use the biome agent to divide the biome description in the target hierarchical scene description into regions to obtain a local description of the biome; the biome agent determines the biome programmatic content generation parameters based on the local description of the biome; using the biome programmatic content generation parameters, the biome agent drives the three-dimensional scene construction software to determine the biome three-dimensional scene.

[0084] In one embodiment, the target scene construction agent includes a road building agent, and the scene construction module 200 is specifically used to use the road building agent to divide the road building description in the target hierarchical scene description into regions to obtain a local description of the road building; the road building agent determines the road building programmatic content generation parameters based on the road building local description; using the road building programmatic content generation parameters, the road building agent drives the three-dimensional scene construction software to determine the road building three-dimensional scene.

[0085] In one embodiment, the target scene construction agent includes a vehicle equipment agent, and the scene construction module 200 is specifically used to use the vehicle equipment agent to divide the vehicle equipment description in the target hierarchical scene description into regions to obtain a local description of the vehicle equipment; the vehicle equipment agent determines the vehicle equipment programmatic content generation parameters based on the vehicle equipment local description; using the vehicle equipment programmatic content generation parameters, the vehicle equipment agent drives the three-dimensional scene construction software to determine the vehicle equipment three-dimensional scene.

[0086] In one embodiment, the scene decomposition module 100 is specifically used to input the original scene description into the scene decomposition agent; use the scene decomposition agent to complete the content of the original scene description to obtain an extended scene description containing multiple preset category descriptions; use the scene decomposition agent to decompose the content of the extended scene description to obtain multiple layered scene descriptions.

[0087] In one embodiment, the scene synthesis module 300 is specifically used to integrate the target layered three-dimensional scene to determine a candidate three-dimensional scene; evaluate the candidate three-dimensional scene; if the candidate three-dimensional scene meets preset expectations, determine the candidate three-dimensional scene as the target three-dimensional scene; if the candidate three-dimensional scene does not meet the preset expectations, add an improved scene description to the original scene description, and update the candidate three-dimensional scene based on the original scene description and the improved scene description.

[0088] The modules described in the above embodiments can be implemented by computer chips or products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0089] For the convenience of description, the above devices are described in terms of functions and modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0090] The present disclosure also provides a computer device having the above Figure 5 The three-dimensional scene construction device shown.

[0091] See also Figure 6 , Figure 6 is a schematic diagram of a computer device provided by an optional embodiment of the present disclosure, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0092] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0093] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0094] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0095] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0096] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0097] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0098] The present disclosure embodiment also provides a computer-readable storage medium, and the above method according to the embodiment of the present disclosure can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or is implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the three-dimensional scene construction method shown in the above embodiment is implemented.

[0099] A part of the present disclosure may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can be called or provided through the operation of the computer according to the three-dimensional scene construction method of the present disclosure. It should be understood by those skilled in the art that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0100] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A three-dimensional scene construction method, characterized in that: The method comprises: Obtaining an original scene description, and inputting the original scene description into a scene decomposition agent to obtain multiple hierarchical scene descriptions decomposed by the scene decomposition agent, wherein the scene decomposition agent is implemented based on the first language model; Using a target scene to construct an intelligent agent, taking a target hierarchical scene description from the plurality of hierarchical scene descriptions as input, to generate a target hierarchical three-dimensional scene, wherein the target scene to construct an intelligent agent is implemented based on a second language model; The target layered three-dimensional scene is integrated to determine the target three-dimensional scene.

2. The method according to claim 1, characterized in that The method of constructing an intelligent agent using a target scene, taking a target hierarchical scene description in a plurality of hierarchical scene descriptions as input, and generating a target hierarchical three-dimensional scene, comprises: Using the target scene to construct an intelligent agent, dividing the target hierarchical scene description into regions to obtain a target hierarchical local description; The target scene construction agent determines at least one of a target hierarchical editing parameter and a target hierarchical programmatic content generation parameter based on the target hierarchical local description, wherein the target hierarchical programmatic content generation parameter includes a model type parameter, a random range parameter, and a distribution parameter; By using the target layered editing parameters and the target layered programmed content generation parameters, the target scene construction agent drives the preset three-dimensional scene construction software to determine the target layered three-dimensional scene.

3. The method according to claim 1 or 2, characterized in that: The target scene construction agent includes a terrain and landform agent. The target scene construction agent uses the target hierarchical scene description in the plurality of hierarchical scene descriptions as input to generate a target hierarchical three-dimensional scene, including: Using the terrain and geomorphology agent, the terrain and geomorphology description in the target hierarchical scene description is divided into regions to obtain a local description of the terrain and geomorphology; The terrain and landform intelligent agent determines terrain editing parameters and landform procedural content generation parameters based on the local description of the terrain and landform; By using the terrain editing parameters and the landform programmatic content generation parameters, the terrain and landform intelligent agent drives the three-dimensional scene construction software to determine the terrain and landform three-dimensional scene.

4. The method according to claim 1 or 2, characterized in that: The target scene construction agent includes a meteorological environment agent, and the target scene construction agent uses the target hierarchical scene description in the plurality of hierarchical scene descriptions as input to generate a target hierarchical three-dimensional scene, including: Using the meteorological environment agent, the meteorological environment description in the target hierarchical scene description is divided into regions to obtain a local description of the meteorological environment; The meteorological environment agent determines weather editing parameters and lighting editing parameters based on the local description of the meteorological environment; By using the weather editing parameters and the lighting editing parameters, the meteorological environment intelligent agent drives the three-dimensional scene construction software to determine the meteorological environment three-dimensional scene.

5. The method according to claim 1 or 2, characterized in that: The target scene construction agent includes a biome agent, and the target scene construction agent uses the target hierarchical scene description in the plurality of hierarchical scene descriptions as input to generate a target hierarchical three-dimensional scene, including: Using the biome agent, the biome description in the target hierarchical scene description is divided into regions to obtain a local description of the biome; The biome agent determines biome procedural content generation parameters based on the biome local description; The biome programmatic content generation parameters are used to drive the three-dimensional scene construction software by the biome intelligent agent to determine the biome three-dimensional scene.

6. The method according to claim 1 or 2, characterized in that: The target scene construction agent includes a road construction agent. The target scene construction agent uses the target hierarchical scene description in the plurality of hierarchical scene descriptions as input to generate a target hierarchical three-dimensional scene, including: Using the road building intelligent agent, the road building description in the target hierarchical scene description is divided into regions to obtain a local description of the road building; The road building intelligent agent determines the road building programmatic content generation parameters based on the road building local description; The road building programmatic content generation parameters are used, and the road building intelligent agent drives the three-dimensional scene construction software to determine the road building three-dimensional scene.

7. The method according to claim 1 or 2, characterized in that: The target scene construction agent includes a vehicle equipment agent, and the target scene construction agent uses the target hierarchical scene description in the plurality of hierarchical scene descriptions as input to generate a target hierarchical three-dimensional scene, including: Using the vehicle equipment agent, dividing the vehicle equipment description in the target hierarchical scene description into regions to obtain a local description of the vehicle equipment; The vehicle equipment intelligent agent determines vehicle equipment programmatic content generation parameters based on the vehicle equipment local description; By using the vehicle equipment programmatic content generation parameters, the vehicle equipment intelligent agent drives the three-dimensional scene construction software to determine the vehicle equipment three-dimensional scene.

8. The method according to claim 1 or 2, characterized in that: The inputting the original scene description into the scene decomposition agent to obtain multiple hierarchical scene descriptions decomposed by the scene decomposition agent includes: Inputting the original scene description into the scene decomposition agent; Using the scene decomposition agent, the original scene description is completed to obtain an expanded scene description including descriptions of multiple preset categories; The scene decomposition agent is used to decompose the content of the extended scene description to obtain a plurality of the hierarchical scene descriptions.

9. The method according to claim 1 or 2, characterized in that: The step of integrating the target layered three-dimensional scene to determine the target three-dimensional scene includes: Integrate the target layered three-dimensional scene to determine a candidate three-dimensional scene; evaluating the candidate three-dimensional scene; If the candidate three-dimensional scene meets the preset expectation, determining the candidate three-dimensional scene as the target three-dimensional scene; If the candidate 3D scene does not meet the preset expectation, an improved scene description is added to the original scene description, and the candidate 3D scene is updated based on the original scene description and the improved scene description.

10. A three-dimensional scene construction device, characterized in that: The device comprises: A scene decomposition module, used to obtain an original scene description, and input the original scene description into a scene decomposition agent to obtain multiple hierarchical scene descriptions decomposed by the scene decomposition agent, wherein the scene decomposition agent is implemented based on the first language model; A scene construction module, used to construct an intelligent agent using a target scene, taking a target hierarchical scene description from the plurality of hierarchical scene descriptions as input to generate a target hierarchical three-dimensional scene, wherein the target scene construction intelligent agent is implemented based on the second language model; The scene synthesis module is used to integrate the target layered three-dimensional scene to determine the target three-dimensional scene.

11. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the three-dimensional scene construction method according to any one of claims 1 to 9 by executing the computer instructions.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the three-dimensional scene construction method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to enable a computer to execute the three-dimensional scene construction method according to any one of claims 1 to 9.

Images