An extreme complex scene-oriented generative immersive scene construction method and system
By integrating multi-source data and multi-layered driving engines, and combining them with a credibility constraint module, the problem of generating highly dynamic, strongly coupled, and credible virtual scenes in existing technologies has been solved, thus achieving efficient and realistic immersive scenario construction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SOUTHWEST JIAOTONG UNIV
- Filing Date
- 2026-04-17
- Publication Date
- 2026-07-31
AI Technical Summary
Existing technologies cannot automatically generate highly dynamic, multi-element strongly coupled, and behaviorally credible extremely complex virtual scenarios, and cannot dynamically respond to external interventions, resulting in low simulation efficiency, high cost, and insufficient behavioral credibility.
A scene knowledge graph is constructed by fusing multi-source data. Through a multi-layer driving engine, physical environment state, group behavior and key individual strategies are generated collaboratively. Combined with a credibility constraint module, real-time correction is performed to establish a three-layer dynamic coupling mechanism to realize the construction of immersive scenarios.
It achieves efficient and automated generation of large-scale, highly detailed, and complex scenes, possesses dynamic interactive capabilities for physical evolution, group behavior, and intelligent decision-making, has high realism, and supports rapid response to diverse simulation needs.
Smart Images
Figure CN122490992A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of artificial intelligence and virtual reality technology, and in particular relates to a generative immersive scenario construction method and system for extremely complex scenarios. Background Technology
[0002] In fields such as emergency management, national defense and security, and urban planning, high-fidelity simulations of extreme scenarios such as earthquakes, floods, terrorist attacks, and major fires are invaluable for contingency plan development, decision optimization, and personnel training. Traditional construction methods mainly fall into two categories: one is manual, detailed modeling based on game engines or 3D modeling software; the other is procedural generation based on Geographic Information Systems (GIS) and certain rules.
[0003] Existing technologies have fundamental limitations in dealing with extremely complex scenarios:
[0004] 1. Static fragmentation: Manual modeling is mostly static scenery, with independent element behavior, which cannot simulate the dynamic coupling of disaster chains, secondary disasters and social group behavior.
[0005] 2. Inefficient and costly: Large-scale, high-detail scenarios are time-consuming and costly, making it difficult to quickly respond to diverse simulation needs.
[0006] 3. Low credibility of behavior: The behavior of the crowd / opponent is based on simple scripts or random algorithms. The patterns are simple and the logic is simple. It cannot reflect the complexity, strategy and adaptability of behavior in real crisis, and the credibility of the inference conclusions is insufficient.
[0007] 4. Lack of self-evolution ability: The scenario is a "dead" scenario, unable to make reasonable, dynamic and overall responses based on the behavior of participants, and does not have the ability to grow and evolve.
[0008] Therefore, there is an urgent need for an immersive scenario construction technology that can automatically and efficiently generate scenarios that combine high physical realism and behavioral complexity, and can dynamically respond to external interventions. Summary of the Invention
[0009] To address the problem that existing technologies cannot automatically generate highly dynamic, multi-element strongly coupled, and behaviorally believable extremely complex virtual scenes, this invention provides a generative immersive scenario construction method and system for extremely complex scenarios.
[0010] The present invention provides a generative immersive scenario construction method for extremely complex scenarios, comprising the following steps:
[0011] Step 1: Multi-source data fusion and scene seed initialization.
[0012] By integrating heterogeneous data from multiple sources, a scene knowledge graph is constructed, and an initial unified world state W(t0) is generated to complete the initialization of the scenario seed.
[0013] The unified world state of the system at any simulation time t is:
[0014]
[0015] Where t is the system simulation time, E(t) is the macroscopic physical environment state, including terrain, physical field distribution, and infrastructure state; A(t) is the mesoscopic group intelligent agent state set; I(t) is the microscopic key intelligent agent state set; and C(t) is the constraint set composed of physical rules, behavioral logic, and domain knowledge.
[0016] Step 2: Collaborative generation based on a multi-layered driving engine.
[0017] A multi-layered driving engine is activated, including a macro-level catastrophe simulation model, a meso-level social dynamics model, and a micro-level multi-agent game model. Based on the scenario knowledge graph, physical environment states, group behaviors, and key individual strategies are generated collaboratively according to a hierarchical formal model.
[0018] Step 3: Real-time guidance and correction by the credibility constraint module.
[0019] The world state W(t) is guided and corrected in real time by the credibility constraint module to generate a legal state. This ensures that the generated content is reasonable and credible.
[0020] Step 4: Dynamic Coupling and Unified Rendering.
[0021] A three-layer dynamic coupling mechanism is established, and output is synchronized through a unified state bus to integrate multimodal immersive rendering and form an interactive immersive environment.
[0022] Step 5: Dynamic evolution of the situation and interactive response.
[0023] It receives external interaction commands (Uinput) and feeds them back to the multi-layered driving engine, driving the dynamic evolution and response of the driving scenario.
[0024] Furthermore, the multi-source heterogeneous data in step 1 includes: high-precision geospatial data, building information model (BIM) data, infrastructure network data, historical disaster data, population dynamic distribution data, traffic flow data, and domain knowledge rules.
[0025] Furthermore, in step 2:
[0026] Macro level: Disaster simulation model.
[0027] Based on the laws of fluid mechanics, structural mechanics, heat transfer, and historical statistical models, a macroscopic physical derivation function f is constructed. macro The state update formula for simulating the large-scale spatiotemporal dynamic evolution of the disaster entity is as follows:
[0028]
[0029] Where, Θ phy It is a set of physical parameters, and the output is the physical field distribution and critical infrastructure status that change over time.
[0030] Meso-level: Social dynamics model.
[0031] A group behavior model based on ABM and cellular automata social The simulation of large-scale population behavior and state changes under the influence of disasters uses the following state update formula:
[0032]
[0033] Where, Θ beh These serve as parameters for individual behavioral rules, leading to the emergence of complex social phenomena through local interactions.
[0034] Micro-level: Multi-agent game model.
[0035] Build deep reinforcement learning agent model f for key roles agent It possesses a space of objectives, resources, and strategies, and its state update formula is:
[0036]
[0037] in, For policy networks, R is the reinforcement learning reward function. The agent can perform perception, decision-making, planning, and strategic interaction with the environment and other agents.
[0038] Furthermore, the revised formula for the credibility constraint module in step 3 is as follows:
[0039]
[0040] in:
[0041] As a physical rule constraint: it ensures that the generated content conforms to basic physical laws, and the constraint determination formula is:
[0042]
[0043] For behavioral logic constraints: Based on knowledge graphs and expert rules, the rationality of behavior is verified, and the constraint determination formula is as follows:
[0044]
[0045] Interface for injecting domain knowledge: Supports experts to inject professional knowledge with rules, probability models or reward functions, and finely control the generation direction.
[0046] Furthermore, the dynamic coupling mechanism in step 4 satisfies:
[0047]
[0048] Macro-level outputs constrain the action capabilities and decision-making costs of meso- and micro-level entities, while the behavior of meso- and micro-level entities triggers macro-level state updates or participates in calculations as boundary conditions.
[0049] Furthermore, in step 5, the external interaction command U is received. input Inject it as an incentive term into the world state update function:
[0050]
[0051] Each layer of the model recalculates and updates the world state in real time based on new inputs, enabling the situation to intelligently respond to external interventions and achieve continuous dynamic evolution.
[0052] This invention discloses a generative immersive context construction system, comprising:
[0053] Data fusion and knowledge management module: Processes multi-source data, builds and maintains a scenario knowledge graph, and generates the initial world state W(t0).
[0054] The multi-layered driving engine module consists of a macro-level catastrophe simulation submodule, a meso-level social dynamics submodule, and a micro-level multi-agent game submodule, which perform hierarchical state iteration calculations.
[0055] Trustworthiness constraint module: physical rule constraint, behavioral logic constraint, domain knowledge injection interface, to complete state legality verification and correction.
[0056] Dynamic Coupling and Synchronization Bus Module: Maintains a unified world state bus to achieve three-layer data exchange, event transmission, and state synchronization.
[0057] Immersive rendering and interactive interface module: Completes multi-sensory rendering output and external interactive command reception.
[0058] The beneficial technical effects of this invention compared to existing technologies are as follows:
[0059] 1. High generation efficiency and automation: Eliminates the dependence on manual modeling, and quickly generates large-scale, highly detailed and complex scenes based on data and rules.
[0060] 2. Excellent dynamism and realism: Multi-layered collaborative coupling, with dynamic interaction capabilities for physical evolution, group behavior, and intelligent decision-making, achieving a realism far exceeding that of static and scripted simulations.
[0061] 3. High credibility of behavior: The entity's behavior is emergent and adaptive, and formal constraints ensure that it conforms to logic and common sense.
[0062] 4. Flexible and scalable: The modular architecture facilitates replacement and expansion, and the domain interface can be quickly adapted to different professional simulation needs.
[0063] 5. Support human-in-the-loop feedback simulation: Through mathematical state iteration and interactive excitation, a standardized dynamic environment is provided for human-in-the-loop feedback simulation. Attached Figure Description
[0064] Figure 1 This is a schematic diagram of the overall process of the generative immersive scenario construction method for extremely complex scenarios according to the present invention.
[0065] Figure 2 This is a schematic diagram of the multi-layer drive engine architecture and coupling relationship of the present invention.
[0066] Figure 3 This is a schematic diagram of the operation of the reliability constraint module of the present invention.
[0067] Figure 4 This is a schematic diagram of the system modules of the present invention. Detailed Implementation
[0068] The present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.
[0069] The process of a generative immersive scenario construction method for extremely complex scenarios according to the present invention is as follows: Figure 1 As shown, the specific steps include:
[0070] Step 1 (S10): Multi-source data fusion and scene seed initialization.
[0071] This approach integrates multi-source heterogeneous data, including high-precision geospatial data, Building Information Modeling (BIM) data, infrastructure network data, historical disaster data, dynamic population distribution data, traffic flow data, and domain knowledge rules. A scene knowledge graph is constructed as the seed and knowledge foundation for context generation, generating an initial unified world state W(t0) to complete the context seed initialization.
[0072] The unified world state of the system at any simulation time t is:
[0073]
[0074] Where t is the system simulation time, E(t) is the macroscopic physical environment state, including terrain, physical field distribution, and infrastructure state; A(t) is the mesoscopic group intelligent agent state set; I(t) is the microscopic key intelligent agent state set; and C(t) is the constraint set composed of physical rules, behavioral logic, and domain knowledge.
[0075] Step 2 (S20): Collaborative generation based on multi-layer driving engine.
[0076] A multi-layered driving engine is activated, including a macro-level catastrophe simulation model, a meso-level social dynamics model, and a micro-level multi-agent game model. Based on a scenario knowledge graph, physical environment states, group behaviors, and key individual strategies are collaboratively generated according to a hierarchical formalized model. For example... Figure 2 As shown, a unified world state bus 410 is set at the center; above it is the macro-level catastrophe simulation submodule 210, in the middle is the meso-level social dynamics submodule 220, and below it is the micro-level multi-agent game submodule 230; each submodule is bidirectionally connected to the unified world state bus 410, and bidirectional data coupling between layers is achieved through the bus.
[0077] Macro level: Disaster simulation model.
[0078] Based on the laws of fluid mechanics, structural mechanics, heat transfer, and historical statistical models, a macroscopic physical derivation function f is constructed. macro The state update formula for simulating the large-scale spatiotemporal dynamic evolution of the disaster entity is as follows:
[0079]
[0080] Where, Θ phy It is a set of physical parameters, and the output is the physical field distribution and critical infrastructure status that change over time.
[0081] Meso-level: Social dynamics model.
[0082] A group behavior model based on ABM (Agent-Based Modeling) and cellular automata social The simulation of large-scale population behavior and state changes under the influence of disasters uses the following state update formula:
[0083]
[0084] Where, Θ beh These serve as parameters for individual behavioral rules, leading to the emergence of complex social phenomena through local interactions.
[0085] Micro-level: Multi-agent game model.
[0086] Build deep reinforcement learning agent model f for key roles agentIt possesses a space of objectives, resources, and strategies, and its state update formula is:
[0087]
[0088] in, For policy networks, R is the reinforcement learning reward function. The agent can perform perception, decision-making, planning, and strategic interaction with the environment and other agents.
[0089] Step 3 (S30): The credibility constraint module provides real-time guidance and correction.
[0090] The world state W(t) is guided and corrected in real time by the credibility constraint module to generate a legal state. This ensures that the generated content is reasonable and credible. For example... Figure 3 As shown, the input on the left is the world state W(t) output by the multi-layer drive engine; the middle sections are, in order, the physical rule constraint 310, the behavioral logic constraint 320, and the domain knowledge injection interface 330; the output on the right is the legal state after constraint correction. The output is sent to the dynamic coupling and synchronization bus module.
[0091] The correction formula for the credibility constraint module is:
[0092]
[0093] in:
[0094] As a physical rule constraint: it ensures that the generated content conforms to basic physical laws, and the constraint determination formula is:
[0095]
[0096] For behavioral logic constraints: Based on knowledge graphs and expert rules, the rationality of behavior is verified, and the constraint determination formula is as follows:
[0097]
[0098] Interface for injecting domain knowledge: Supports experts to inject professional knowledge with rules, probability models or reward functions, and finely control the generation direction.
[0099] Step 4 (S40): Dynamic Coupling and Unified Rendering.
[0100] A three-layer dynamic coupling mechanism is established, and output is synchronized through a unified state bus to integrate multimodal immersive rendering and form an interactive immersive environment.
[0101] Coupling rules are satisfied:
[0102]
[0103] The state after constraint correction Input into a 3D graphics engine to complete real-time high-fidelity multimodal immersive rendering, creating an interactive immersive environment.
[0104] Step 5 (S50): Dynamic evolution of the situation and interactive response.
[0105] Receive external interaction commands U input Inject it as an incentive term into the world state update function:
[0106]
[0107] Each layer of the model recalculates and updates the world state in real time based on new inputs, enabling the situation to intelligently respond to external interventions and achieve continuous dynamic evolution.
[0108] This invention provides a generative immersive context construction system, such as... Figure 4 As shown, the modules arranged in sequence are: data fusion and knowledge management module 100, multi-layer driving engine module 200, credibility constraint module 300, dynamic coupling and synchronization bus module 400, and immersive rendering and interactive interface module 500; arrows indicate the data flow direction, reflecting the collaborative working logic between modules.
[0109] Specifically:
[0110] Data fusion and knowledge management module: Processes multi-source data, builds and maintains a scenario knowledge graph, and generates the initial world state W(t0).
[0111] The multi-layered driving engine module consists of a macro-level catastrophe simulation submodule, a meso-level social dynamics submodule, and a micro-level multi-agent game submodule, which perform hierarchical state iteration calculations.
[0112] Trustworthiness constraint module: physical rule constraint, behavioral logic constraint, domain knowledge injection interface, to complete state legality verification and correction.
[0113] Dynamic Coupling and Synchronization Bus Module: Maintains a unified world state bus to achieve three-layer data exchange, event transmission, and state synchronization.
[0114] Immersive rendering and interactive interface module: Completes multi-sensory rendering output and external interactive command reception.
[0115] Example:
[0116] Urban torrential rain and flooding and secondary disaster simulation scenario construction
[0117] S10 Data Fusion and Initialization:
[0118] Import DEM, drainage network, building outline, population heat map, and traffic data; construct a knowledge graph and associate it with "low-lying areas - prone to flooding", "underground passages - high risk", and "hospitals - critical infrastructure" to generate the initial world state W(t0).
[0119] S20 Multi-layer Collaborative Generation:
[0120] Macroscopic: Hydrological and Hydrodynamic Model Simulates runoff, confluence, and drainage, and outputs water depth / flow velocity, road interruption, and power grid failure.
[0121] Mid-level: Tens of thousands of citizens' agents through Through iteration, the system perceives water accumulation, traffic, and herd mentality, leading to congestion, evacuation, and gathering.
[0122] Microscopic level: Rescue convoys and stranded citizen groups through intelligent agents Complete the planning and decision-making of reinforcement learning paths.
[0123] S30 constraint coupled with S40:
[0124] physical constraints : Determine the vehicle's wading threshold and the decrease in personnel movement speed with water depth, and correct any violations.
[0125] Logical constraints Restricting ordinary citizens from engaging in professional diving rescue activities.
[0126] Coupled interaction: The status of the underground passage being flooded is updated to the bus → the meso-level agent is trapped and panics → the micro-level rescue team replans → the convoy moves to alleviate traffic / affect bridge safety.
[0127] S40 rendering and S50 interaction:
[0128] The graphics engine renders scenes of torrential rain, flooding, traffic congestion, and rescue operations; the commander issues the order to "ensure power supply to the hospital." input Inject state update functions, change the agent's objective function, and drive the continuous evolution of the situation.
[0129] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.
Claims
1. A generative immersive scenario construction method for extremely complex scenarios, characterized in that, Includes the following steps: Step 1: Multi-source data fusion and scene seed initialization; By integrating multi-source heterogeneous data, constructing a scene knowledge graph, and generating an initial unified world state W(t0), the context seed initialization is completed. The unified world state of the system at any simulation time t is: ； Where t is the system simulation time, E(t) is the macroscopic physical environment state, including terrain, physical field distribution, and infrastructure state; A(t) is the mesoscopic group intelligent agent state set; I(t) is the microscopic key intelligent agent state set; and C(t) is the constraint set composed of physical rules, behavioral logic, and domain knowledge. Step 2: Collaborative generation based on a multi-layered driving engine; A multi-layered driving engine is launched, including a macro-level catastrophe simulation model, a meso-level social dynamics model, and a micro-level multi-agent game model. Based on the scenario knowledge graph, physical environment state, group behavior and key individual strategies are generated collaboratively according to a hierarchical formal model. Step 3: Real-time guidance and correction by the credibility constraint module; The world state W(t) is guided and corrected in real time by the credibility constraint module to generate a legal state. To ensure that the generated content is reasonable and credible; Step 4: Dynamic Coupling and Unified Rendering; A three-layer dynamic coupling mechanism is established, and output is synchronized through a unified state bus to integrate multimodal immersive rendering and form an interactive immersive environment. Step 5: Dynamic Evolution of Context and Interactive Response; It receives external interaction commands (Uinput) and feeds them back to the multi-layered driving engine, driving the dynamic evolution and response of the driving scenario.
2. The generative immersive scenario construction method for extremely complex scenarios according to claim 1, characterized in that, The multi-source heterogeneous data in step 1 includes: high-precision geospatial data, building information model (BIM) data, infrastructure network data, historical disaster data, population dynamic distribution data, traffic flow data, and domain knowledge rules.
3. The generative immersive scenario construction method for extremely complex scenarios according to claim 1, characterized in that, In step 2: Macro level: Disaster simulation model; Based on the laws of fluid mechanics, structural mechanics, heat transfer, and historical statistical models, a macroscopic physical derivation function f is constructed. macro The state update formula for simulating the large-scale spatiotemporal dynamic evolution of the disaster entity is as follows: ； Where, Θ phy Given a set of physical parameters, the output is the time-varying distribution of physical fields and the status of critical infrastructure. Meso-level: Social dynamics model; A group behavior model based on ABM and cellular automata social The simulation of large-scale population behavior and state changes under the influence of disasters uses the following state update formula: ； Where, Θ beh These serve as parameters for individual behavioral rules, leading to the emergence of complex social phenomena through local interactions. Micro-level: Multi-agent game model; Build deep reinforcement learning agent model f for key roles agent It possesses a space of objectives, resources, and strategies, and its state update formula is: ； in, For policy networks, R is the reinforcement learning reward function. The agent can perform perception, decision-making, planning, and strategic interaction with the environment and other agents.
4. The generative immersive scenario construction method for extremely complex scenarios according to claim 3, characterized in that, The correction formula for the reliability constraint module in step 3 is as follows: ； in: As a physical rule constraint: it ensures that the generated content conforms to basic physical laws, and the constraint determination formula is: ； For behavioral logic constraints: Based on knowledge graphs and expert rules, the rationality of behavior is verified, and the constraint determination formula is as follows: ； Interface for injecting domain knowledge: Supports experts to inject professional knowledge with rules, probability models or reward functions, and finely control the generation direction.
5. The generative immersive scenario construction method for extremely complex scenarios according to claim 4, characterized in that, The dynamic coupling mechanism in step 4 satisfies: ； Macro-level outputs constrain the action capabilities and decision-making costs of meso- and micro-level entities, while the behavior of meso- and micro-level entities triggers macro-level state updates or participates in calculations as boundary conditions.
6. The generative immersive scenario construction method for extremely complex scenarios according to claim 5, characterized in that, In step 5, the external interaction command U is received. input Inject it as an incentive term into the world state update function: ； Each layer of the model recalculates and updates the world state in real time based on new inputs, enabling the situation to intelligently respond to external interventions and achieve continuous dynamic evolution.
7. A generative immersive context construction system implementing the method as described in any one of claims 1-6, characterized in that, include: Data fusion and knowledge management module: processes multi-source data, constructs and maintains a scenario knowledge graph, and generates the initial world state W(t0); Multi-layered driving engine module: macro-level catastrophe simulation submodule, meso-level social dynamics submodule, micro-level multi-agent game submodule, performs hierarchical state iteration calculation; Trustworthiness constraint module: physical rule constraint, behavioral logic constraint, domain knowledge injection interface, to complete state legality verification and correction; Dynamic Coupling and Synchronization Bus Module: Maintains a unified world state bus to achieve three-layer data exchange, event transmission, and state synchronization; Immersive rendering and interactive interface module: Completes multi-sensory rendering output and external interactive command reception.