An open city embodied intelligent simulation training environment construction method and system

By acquiring and calibrating open urban 3D data, generating simulation constraints and dynamic subject templates, and using a large language model to generate structured task scripts, the problem of lack of dynamic constraints in embodied intelligence training of existing urban 3D models is solved. This enables the efficient generation of embodied intelligence simulation training environments and improves the effectiveness and consistency of training data.

CN123118941APending Publication Date: 2026-11-17HONG KONG UNIV OF SCI & TECH (GUANGZHOU)
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611318949.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-28
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing 3D urban models lack dynamic subjects, passable areas, collision constraints, lane constraints, flight constraints, interactive events, and training data labels in embodied intelligent training scenarios, resulting in high costs, limited coverage, and low reuse efficiency. Random generation methods lack unified constraints on urban geometry, kinematic constraints, and interactive events, producing invalid training samples.

Method used

By acquiring open city 3D data, including mesh models and 3D Gaussian splash models, spatial calibration is performed and transformed to a unified world coordinate system to generate simulation constraint data, establish a dynamic subject template library, use large language model intelligent agents to generate structured task scripts, generate multi-subject motion trajectories and interactive events, and combine visual information to generate an embodied intelligent simulation training environment.

Benefits of technology

Significantly reduces the workload of manual configuration, ensures the executability of tasks, dynamic subjects, start and end points, motion constraints and event constraints, improves the effectiveness of training events and data consistency, generates training samples containing complex scenarios, and ensures the consistency of visual output with trajectory, collision and semantic labels.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118941A_ABST
    Figure CN123118941A_ABST
Patent Text Reader

Abstract

This invention discloses a method and system for constructing an open city embodied intelligence simulation training environment, relating to the fields of embodied intelligence and 3D simulation technology. The method includes: acquiring open city 3D data composed of a mesh model and a 3D Gaussian splash model and performing spatial calibration to align them in a unified world coordinate system; generating simulation constraint data based on the mesh model, including collision meshes, navigation meshes, lane maps, 3D flight space, and region labels; establishing a dynamic subject template library; generating structured task scripts through a large language model agent; generating multi-subject motion trajectories and forming interactive events based on the structured task scripts and simulation constraint data; generating visual information and writing it into the same training sample to obtain the simulation training environment. This invention transforms static city 3D data into a simulation environment usable for embodied intelligence training, improving the effectiveness and generation efficiency of training data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of embodied intelligence and 3D simulation technology, and particularly relates to a method and system for constructing an open city embodied intelligence simulation training environment. Background Technology

[0002] Existing 3D city models are primarily used for city display, digital twin visualization, static browsing, and offline rendering. These models can present the static appearance of roads, buildings, blocks, squares, green spaces, and city facades. However, in embodied intelligent training scenarios, 3D models with only static appearances lack dynamic subjects, traversable areas, collision constraints, lane constraints, flight constraints, interactive events, and training data labels. Therefore, existing open 3D city data needs to be further transformed into a simulation training environment capable of running dynamic simulations, generating training tasks, and outputting training samples.

[0003] Embodied intelligence training relies on a large amount of open urban dynamic data. Mobile robots need to perform tasks such as target navigation, obstacle avoidance, pedestrian avoidance, and passage through narrow areas in areas such as sidewalks, squares, and intersections; vehicles need to perform tasks such as following, slowing down, yielding, and passing through intersections in lanes, intersections, pedestrian crossings, and parking areas; drones need to perform inspection, flight around, and obstacle avoidance tasks within buildings, roads, squares, and constrained aerial spaces. Since the above tasks involve static urban structures, dynamic subject movements, and multi-subject interactions, relying on manually configuring dynamic scene and trajectory data line by line would result in high costs, limited coverage, and low reuse efficiency.

[0004] To scale up data volume, randomly generating dynamic subjects and trajectories is a common approach. However, random generation lacks uniform constraints on urban geometry, kinematic constraints, and interaction events, easily leading to invalid training samples. For example, pedestrian trajectories may cross buildings, robots may leave sidewalks, vehicles may deviate from lanes, and drones may traverse buildings, resulting in no effective avoidance, yielding, or occlusion events between dynamic subjects. Such data requires extensive post-processing and manual screening before entering the training process.

[0005] At the task configuration level, the large language model possesses capabilities for task description, scene combination, and constraint generation, enabling the generation of open city training tasks and dynamic scene configurations. Correspondingly, continuous motion trajectories still need to be generated by path planning, local obstacle avoidance, kinematic smoothing, and collision detection modules to ensure that the trajectories meet urban geometric constraints, kinematic constraints, and multi-agent interaction constraints. Thus, the large language model agent is responsible for structured tasks and constraint generation, while the trajectory generation module is responsible for generating continuous motion trajectories. The combination of these two modules reduces task configuration costs and improves the effectiveness of trajectory data.

[0006] At the level of urban 3D data, existing open urban 3D data includes both mesh models and 3D Gaussian splash models. Mesh models are used for geometric constraints, collision detection, path generation, depth occlusion, and label generation in open urban scenes, while 3D Gaussian splash models are used for realistic rendering of static urban backgrounds. By using both collaboratively in a unified world coordinate system and integrating them with dynamic subject trajectory generation, task script generation, and training data output processes, an open urban simulation training environment oriented towards embodied intelligence training can be formed.

[0007] Therefore, there is a need for a method and system for constructing an embodied intelligent simulation training environment for open cities, which can perform simulation processing on existing open city 3D data to generate open city training tasks, multi-subject motion trajectories, interactive events and multimodal training data, and ensure the consistency of output data in terms of geometric constraints, motion constraints, visual occlusion and task labels. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention provides a method for constructing an open city embodied intelligence simulation training environment, comprising: Acquire open city 3D data, which includes a mesh model and a 3D Gaussian splash model. The mesh model is used to provide the geometric structure of the open city scene, and the 3D Gaussian splash model is used to provide the static visual appearance of the open city scene. Spatial calibration is performed on the mesh model and the three-dimensional Gaussian splash model, and the mesh model and the three-dimensional Gaussian splash model are transformed to a unified world coordinate system; Generate open city simulation constraint data based on the grid model; Establish a dynamic subject template library, which contains multiple dynamic subject templates. Each dynamic subject template records the motion attributes and passage constraints of the corresponding type of dynamic subject. Input the city area summary, available area list, dynamic subject template, target task type and task difficulty into the large language model agent, and generate a structured task script through the large language model agent; Based on the structured task script and the simulation constraint data, a multi-subject motion trajectory is generated, and an interactive event is formed in the multi-subject motion trajectory according to the interactive event template in the structured task script. Visual information is generated based on the multi-subject motion trajectories, and the structured task script, the multi-subject motion trajectories, the interactive events, and the visual information are written into the same training sample to obtain the open city embodied intelligent simulation training environment.

[0009] Optionally, acquiring open city 3D data includes: A mesh model and a 3D Gaussian splash model are generated from the same set of multi-view images of an open city and the same photogrammetric results, respectively. The mesh model and the 3D Gaussian splash model share the camera pose and 3D reconstructed coordinates obtained from the photogrammetry. The spatial calibration of the mesh model and the three-dimensional Gaussian splash model includes: The same coordinate transformation is applied to the mesh model and the 3D Gaussian splash model to transform them from the 3D reconstructed coordinates to the unified world coordinate system, so that the city structure in the mesh model and the corresponding visual content in the 3D Gaussian splash model maintain spatial correspondence.

[0010] Optionally, open city simulation constraint data is generated based on the grid model, including: Based on the geometric structure of the mesh model, and combined with geographic information data mapping and / or 2D image semantic segmentation results, collision mesh, navigation mesh, lane map, 3D flight space, depth proxy mesh, and region labels are generated.

[0011] Optionally, the large language model agent includes a global route planning module and a local behavior planning module; The generation of structured task scripts through the large language model agent includes: The global route planning module determines the target location and overall route based on the task requirements and the simulation constraint data. The target location and overall route together represent the high-level action intention of the dynamic subject. The local behavior planning module generates sequentially executed local behaviors based on the overall route and dynamic subject distribution. The local behaviors include at least one of forward movement, turning, lane changing, waiting, and yielding. A structured action plan is generated based on the overall route and the local behaviors; The structured task script is generated based on the structured action plan. The structured task script includes task type, dynamic subject, starting region, target region, motion constraints, and interaction event constraints.

[0012] Optionally, based on the structured task script and the simulation constraint data, a multi-agent motion trajectory is generated, including: The starting point, target point, and key road segments in the structured task script are mapped to the passable space in the simulation constraint data, and a path search algorithm is used to generate a geometric path in the passable space. In each simulation time step, each dynamic subject calculates the desired motion direction and desired velocity along the geometric path, and updates the local motion state according to the current position, velocity and safety distance of the surrounding dynamic subjects, generating the discrete motion results of each dynamic subject. The discrete motion results are subjected to kinematic constraint-based curve smoothing and time parameterization to generate a continuous trajectory that includes timestamps, position, attitude, linear velocity, angular velocity, and behavioral state.

[0013] Optionally, interactive events are generated in the multi-subject motion trajectories according to the interactive event template in the structured task script, including: Read the interaction event template, which records the event type, executing subject, target object, occurrence area, triggering condition, target behavior, termination condition, and time constraint; When the triggering condition is met, record the start time of the event and the corresponding subject state; Generate the motion trajectory of the executing entity based on the target behavior; When the termination condition is met, record the candidate termination time and the corresponding subject state.

[0014] Optionally, after generating the multi-subject motion trajectory, the method further includes: The multi-entity motion trajectory is input into the trajectory physical executability detection module. The trajectory physical executability detection module performs collision detection, safety distance detection, region boundary crossing detection, and kinematic constraint detection on the multi-entity motion trajectory based on the external dimensions of each dynamic entity and the mesh model. The event state of the interaction event is determined based on the detection results of the collision detection, the safe distance detection, the area boundary crossing detection, and the kinematic constraint detection.

[0015] Optionally, generating visual information based on the multi-subject motion trajectories includes: Read the camera imaging parameters, installation position and orientation recorded in the dynamic subject template, and calculate the position and orientation of the camera in the unified world coordinate system based on the position and attitude of the device subject at each moment in the multi-subject motion trajectory; Based on the position and orientation of the camera, a static city background image corresponding to the 3D Gaussian splash model, a static scene depth map corresponding to the mesh model, and a dynamic subject image, dynamic subject depth map, and instance mask corresponding to the dynamic subject are generated respectively. Based on the static scene depth map and the dynamic subject depth map, the foreground and background occlusion relationships of each pixel are determined, and a composite image, depth map, and instance mask corresponding to the camera viewpoint are generated.

[0016] Optionally, the structured task script, the multi-agent motion trajectory, the interaction events, and the visual information are written into the same training sample, including: Using a unified task number, event identifier, and timestamp, the structured task script, the multi-subject motion trajectory, physical detection results, event labels, task status, camera parameters, images, depth maps, and instance masks are written into the same training sample.

[0017] This invention also provides a system for constructing an open city embodied intelligence simulation training environment, comprising: The data import module is used to acquire open city 3D data, which includes a mesh model and a 3D Gaussian splash model. The spatial calibration module is used to perform spatial calibration on the mesh model and the three-dimensional Gaussian splash model, and to convert the mesh model and the three-dimensional Gaussian splash model to a unified world coordinate system; The simulation constraint generation module is used to generate open city simulation constraint data based on the grid model. A dynamic subject template library contains multiple dynamic subject templates, and each dynamic subject template records the motion attributes and passage constraints of the corresponding type of dynamic subject; A large language model agent is used to generate structured task scripts based on the input city area summary, available area list, dynamic subject template, target task type, and task difficulty. The trajectory and event generation module is used to generate multi-subject motion trajectories based on the structured task script and the simulation constraint data, and to form interactive events in the multi-subject motion trajectories based on the interactive event templates in the structured task script. The visual rendering and output module is used to generate visual information based on the multi-subject motion trajectories, and write the structured task script, the multi-subject motion trajectories, the interactive events and the visual information into the same training sample to obtain the open city embodied intelligent simulation training environment.

[0018] Compared with the prior art, the present invention has the following advantages and technical effects: This invention, based on existing open city 3D data, constructs an embodied intelligent simulation training environment, enabling the originally static city model to possess the capabilities of dynamic subject movement, task generation, trajectory generation, interactive event construction, and training data output. By introducing a large language model intelligent agent, this invention can automatically generate structured task scripts, significantly reducing the workload of manually configuring open city training tasks and dynamic interactive scenarios, and ensuring that tasks, dynamic subjects, start and end points, motion constraints, and event constraints can all be incorporated into an executable simulation process. Based on this, the system constructs training samples containing complex scenarios such as pedestrian avoidance, vehicle yielding, robot detours, drone obstacle avoidance, and dynamic occlusion through multi-subject trajectory generation and interactive event generation mechanisms, effectively increasing the proportion of target training events appearing in the synthetic data. To ensure the rationality of dynamic subject movement, this invention also utilizes a mesh model to generate collision meshes, navigation meshes, lane maps, 3D flight spaces, depth proxy meshes, and region labels, thereby ensuring that trajectories satisfy geometric constraints, kinematic constraints, and region traffic constraints. In terms of visual presentation, the system uses a 3D Gaussian splash model to generate a static urban background, and combines a deep proxy mesh with a dynamic subject depth map for occlusion synthesis, ensuring that the final visual output is consistent with the trajectory, collision, and semantic labeling results. Furthermore, this invention also records the physical detection results and execution status of the trajectory through trajectory physical executability detection and event and task state determination, thereby improving the effectiveness and usability of the output training data. Attached Figure Description

[0019] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the collaboration between the mesh model and the three-dimensional Gaussian splash model in an embodiment of the present invention. Figure 3 This is a flowchart illustrating the interactive event generation process according to an embodiment of the present invention. Figure 4 This is a flowchart illustrating the visual occlusion synthesis process according to an embodiment of the present invention. Detailed Implementation

[0020] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0021] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0022] Example 1 like Figures 1-4 As shown, this embodiment provides a method for constructing an open city embodied intelligence simulation training environment, including: Acquire open city 3D data, which includes a mesh model and a 3D Gaussian splash model. The mesh model is used to provide the geometric structure of the open city scene, and the 3D Gaussian splash model is used to provide the static visual appearance of the open city scene. Spatial calibration is performed on the mesh model and the 3D Gaussian splash model, and the mesh model and the 3D Gaussian splash model are transformed to a unified world coordinate system; Generate open city simulation constraint data based on the grid model; Establish a dynamic subject template library, which contains multiple dynamic subject templates. Each dynamic subject template records the motion attributes and passage constraints of the corresponding type of dynamic subject. Input the city area summary, available area list, dynamic subject template, target task type and task difficulty into the large language model agent, and generate a structured task script through the large language model agent; Based on the structured task script and simulation constraint data, multi-subject motion trajectories are generated, and interactive events are formed in the multi-subject motion trajectories according to the interactive event templates in the structured task script. Visual information is generated based on the motion trajectories of multiple subjects, and the structured task script, motion trajectories of multiple subjects, interactive events and visual information are written into the same training sample to obtain an open city embodied intelligent simulation training environment.

[0023] Furthermore, acquiring open city 3D data includes: A mesh model and a 3D Gaussian splash model are generated from the same set of multi-view images of open cities and the same photogrammetric results, respectively. The mesh model and the 3D Gaussian splash model share the camera pose and 3D reconstructed coordinates obtained from the photogrammetry. Spatial calibration of the mesh model and the 3D Gaussian splash model, including: The same coordinate transformation is applied to both the mesh model and the 3D Gaussian splash model, converting them from 3D reconstructed coordinates to a unified world coordinate system, so that the city structure in the mesh model and the corresponding visual content in the 3D Gaussian splash model maintain spatial correspondence.

[0024] Specifically, importing open city 3D data and completing spatial calibration includes: First, the system acquires a mesh model and a 3D Gaussian splash model generated from the same set of multi-view images of the open city and the same photogrammetric results. The mesh model includes the geometry of the ground, roads, buildings, walls, squares, sidewalks, fixed obstacles, and urban facilities, while the 3D Gaussian splash model includes the static visual appearance of the open city area.

[0025] Based on this, the photogrammetric results record the position and orientation of the camera in three-dimensional space when each image is captured. This position and orientation are called the camera pose, and a three-dimensional reconstruction coordinate system is established accordingly. The mesh model and the three-dimensional Gaussian splash model are generated based on the same photogrammetric results, and both inherit the camera pose and three-dimensional reconstruction coordinates, thus having a consistent spatial origin, coordinate axis direction, and scale.

[0026] Subsequently, when importing the two types of models into the simulation training environment, the system applies the same coordinate transformation to both, converting them to the unified world coordinate system of the simulation training environment. After the transformation, roads, buildings, and other urban structures in the mesh model maintain spatial correspondence with the corresponding visual content in the 3D Gaussian splash model. The mesh model is used for collision detection, path planning, and depth calculation, while the 3D Gaussian splash model is used for static urban background rendering, thus ensuring consistency between geometric constraints, visual content, and the position of dynamic subjects.

[0027] Furthermore, open city simulation constraint data is generated based on the grid model, including: Based on the geometric structure of the grid model, combined with geographic information data mapping and / or 2D image semantic segmentation results, collision grids, navigation grids, lane maps, 3D flight space, depth proxy grids, and region labels are generated.

[0028] Specifically, generating open city simulation constraint data includes: After completing spatial calibration, the system generates open city simulation constraint data based on the mesh model. The open city simulation constraint data includes collision mesh, navigation mesh, lane map, 3D flight space, depth proxy mesh, and region labels.

[0029] Specifically, the system first imports publicly available geographic information data, urban road vector data, or other geographic information layers. Based on the geographic coordinates carried by the photogrammetry results, or based on the corresponding control points in the grid model and geographic information data, the system calculates the coordinate transformation between the two types of data, and maps road centerlines, road boundaries, intersections, and other area information to the unified coordinate system of the grid model. The system provides polygon editing tools, supporting the drawing and adjustment of area boundaries, splitting or merging areas, and setting area categories, traffic attributes, and connectivity relationships, allowing operators to correct the mapping results.

[0030] After completing the aforementioned geographic information mapping, the system abstracts the traffic functions of the vehicle roads, organizing the road area into a directed traffic network consisting of road segments, lanes, and their connections. Each road segment and lane records its spatial boundaries, number of lanes, speed range, and road status. The road status includes traffic direction, open or closed status, congestion level, and the types of dynamic entities allowed to pass, and can be updated according to the simulation scenario. The system generates a lane map for autonomous vehicle path planning based on the traffic network and writes the road status into the corresponding region labels and connection edge attributes.

[0031] In addition to the aforementioned vehicle and road modeling, the system further utilizes the original multi-view images from photogrammetry to supplement the semantics of areas beyond roads. The system employs 2D image semantic segmentation to identify roads, sidewalks, plazas, green spaces, buildings, and other surface categories, and projects pixel-level labels from each image onto a mesh model using camera poses obtained from photogrammetry. The system determines candidate semantics for each mesh facet through multi-view label voting, and merges continuous regions using a region growing algorithm based on the normal direction, slope, elevation continuity, and surface roughness of adjacent mesh faces. Operators then use polygon editing tools to correct region boundaries and categories, forming the final semantic surface region.

[0032] After forming the surface semantic region, the system determines the traversable area for the embodied subject based on the surface semantics, mesh geometry, and the size and mobility of the dynamic subject. For robots, pedestrians, and other ground subjects, the system generates a corresponding navigation mesh by combining the allowed traversable area category, maximum slope, maximum step height, minimum passage width, and minimum clearance. Sidewalks, plazas, courtyards, and other ordinary road surfaces that meet the mobility requirements, other than vehicular roads, can thus be configured as traversable areas for robots or pedestrians. Different types of dynamic subjects obtain navigation meshes and passage rules adapted to their capabilities.

[0033] For UAV airspace planning, the system generates a three-dimensional occupied space based on buildings and other static obstacles in the mesh model, and expands this occupied space according to the UAV's dimensions and safe distance. Areas within the scene boundary that are within the specified flight altitude range and outside the expanded occupied space are designated as default flyable spaces. Operators delineate no-fly zones, height-restricted zones, and designated flight corridors using two-dimensional polygons combined with altitude ranges or three-dimensional bounding volumes, and the system generates a three-dimensional flight space for UAV path planning accordingly.

[0034] Based on the above processing results, the system generates collision meshes and depth proxy meshes according to static mesh geometry, generates lane maps according to vehicle road function abstraction, generates navigation meshes according to surface semantics and subject motion conditions, generates three-dimensional flight space according to three-dimensional occupied space and artificial airspace constraints, and saves the category, boundary, connection relationship and traffic rules of each region as region labels.

[0035] Furthermore, the large language model agent includes a global route planning module and a local behavior planning module; Generate structured task scripts using a large language model agent, including: The global route planning module determines the target location and overall route based on task requirements and simulation constraint data. The target location and overall route together represent the high-level action intention of the dynamic subject. The local behavior planning module generates sequential local behaviors based on the overall route and dynamic subject distribution. Local behaviors include at least one of forward, turn, lane change, wait, and avoidance. Generate a structured action plan based on the overall route and local behaviors; A structured task script is generated based on the structured action plan. The structured task script includes the task type, dynamic subject, starting region, target region, motion constraints, and interaction event constraints.

[0036] Specifically, generating structured task scripts through a large language model agent includes: After the dynamic subject template library is established, the system inputs the open city area summary, available area list, dynamic subject template, target task type, and task difficulty into the large language model agent. The large language model agent includes a global route planning module and a local behavior planning module.

[0037] The global route planning module receives task requirements, urban road and spatial structure, current traffic status, and the type, size, and mobility of the dynamic subject. It comprehensively determines the target location and the overall route to that location. The target location and the overall route together represent the dynamic subject's high-level action intention. For ground-based subjects such as unmanned vehicles and robots, this module generates a global route composed of key road segments or areas based on road networks, lane maps, or passable areas, combined with road connectivity, traffic efficiency, road width, and subject size. For aerial subjects such as drones, this module generates a global route composed of aerial waypoints or flight corridors based on three-dimensional flyable space, combined with flight altitude, obstacle distribution, and subject size.

[0038] The local behavior planning module generates sequentially executed local behaviors based on the global route, road structure along the route, and dynamic subject distribution. These local behaviors include forward movement, turning, lane changing, deceleration, waiting, and avoidance, as well as the UAV's takeoff, climb, cruise, turning, descent, and landing. Each behavior includes the corresponding execution segment, triggering conditions, and motion constraints. The global route and local behaviors together form a structured action plan, which serves as input to the trajectory generation module.

[0039] The interactive event is a process in which an embodied intelligent agent, within a specific spatial area and time interval, performs a movement behavior with clearly defined triggering conditions, behavioral responses, and termination conditions, based on task requirements, road or area conditions, static obstacles, or the state of other dynamic agents. Each interactive event includes the executing agent, the object of action, the area of ​​occurrence, the triggering condition, the target behavior, and the termination condition. Robot obstacle avoidance, robot pedestrian avoidance, unmanned vehicle obstacle avoidance, unmanned vehicle lane changing, vehicle yielding, and drones avoiding buildings are all examples of interactive events.

[0040] The large language model agent outputs structured task scripts. These scripts include task type, main dynamic agent, interactive dynamic agent, starting region, target region, motion constraints, interactive event constraints, and data output requirements, and also set interactive event templates. These templates record event type, executing agent, target object, occurrence region, triggering condition, target behavior, termination condition, and time constraint.

[0041] After the structured task script is generated, the system performs format verification, region existence verification, dynamic subject type verification, start-point and target point reachability verification, and motion constraint legality verification. Once the verifications pass, the structured task script and structured motion plan enter the trajectory generation process, transforming the task requirements at the natural language level into executable data structures for subsequent simulation modules.

[0042] Specifically, the movement trajectories of multiple entities and the resulting interactive events include: The system generates multi-agent trajectories based on the Melodie subject modeling and simulation framework. Pedestrians, robots, autonomous vehicles, and drones are each established as dynamic agents. The simulation environment is organized around a navigation mesh, lane map, 3D flight space, static obstacles, and current traffic conditions. The structured motion plan, along with the dimensions, speeds, and motion capabilities of each dynamic agent, is organized into a scene configuration. The simulation model schedules the state updates of each dynamic agent according to a preset time step. The data acquisition component records the agent's position, attitude, speed, and behavior at each time step, forming multi-agent motion trajectories.

[0043] The trajectory generation employs a hierarchical planning method combining global path search and local motion calculation. The trajectory generation module calculates specific paths based on key areas, key road segments, or flight corridors defined by the overall route. The system first maps the start point, target point, and key road segments from the structured motion plan to the corresponding passable space, then searches for paths using the A* Pathfinding Algorithm on the navigation grid, lane map, or 3D pathpoint network. Path cost comprehensively considers travel distance, estimated travel time, road or space congestion, obstacle safety distance, and the size and motion capabilities of the dynamic subject, thereby generating a geometric path that conforms to the structured motion plan.

[0044] In each simulation time step, each dynamic entity calculates its desired motion direction and velocity along the geometric path, and updates its local motion state based on the current position, velocity, and safe distance of surrounding dynamic entities. Pedestrians, robots, and drones use the Optimal Reciprocal Collision Avoidance (ORCA) algorithm to calculate two-dimensional or three-dimensional collision avoidance velocities; unmanned vehicles operate along the lane map, using the Intelligent Driver Model (IDM) to calculate car-following acceleration, and the Minimizing Overall Braking Induced by Lane Changes (MOBIL) model to determine lane-changing behavior. The algorithm generates local motion results such as deceleration, waiting, avoidance, detour, and lane changing.

[0045] For tasks involving multi-agent interaction events, the simulation environment adjusts the departure time, expected speed, or waiting time of each dynamic agent based on their estimated arrival time in the interaction area, ensuring that the relevant dynamic agents arrive in the interaction area within a preset time window. The system performs kinematic constraint-based curve smoothing and time parameterization on the discrete motion results, forming a continuous trajectory that includes timestamps, positions, attitudes, linear velocities, angular velocities, and behavioral states.

[0046] The multi-subject trajectory generation module reads the interaction event template. When the triggering conditions in the event template are met, the system records the event start time and the corresponding subject state; when the termination condition is met, the system records the candidate termination time and the corresponding subject state. Taking the robot avoiding an obstacle as an example, when the obstacle is located in the robot's forward direction and enters the preset effective distance, the system records the event start time and generates a trajectory for the robot to decelerate, adjust its direction of travel, and bypass the obstacle; when the robot passes the obstacle and continues to move along the target direction, the system records the candidate termination time.

[0047] Taking the lane-changing event of an autonomous vehicle as an example, when the task script requires a lane change, or when the obstacles, slow-moving objects, or traffic conditions in the current lane meet the preset lane-changing conditions, and the adjacent lanes allow passage, the system generates a trajectory to move towards the target lane; when the autonomous vehicle enters the target lane and continues to drive along that lane, the system records the candidate end time.

[0048] The system inputs the generated dynamic subject trajectory into the trajectory physics executability detection module for verification. The trajectory physics executability detection module, based on a physics simulation engine (specifically, the NVIDIA PhysX 5 physics engine), determines the corresponding colliders based on the external dimensions of each dynamic subject and detects the trajectory using the scene mesh model. The detection includes parallel collision detection, safety distance detection, region boundary crossing detection, and kinematic constraint detection. Specifically, collision detection determines whether a collision occurs between the dynamic subject and the scene environment or other dynamic subjects; safety distance detection determines whether a preset safety distance is met between the dynamic subject and the scene environment or other dynamic subjects; region boundary crossing detection determines whether the dynamic subject enters a prohibited area or exceeds its permitted range of movement; and kinematic constraint detection determines whether the dynamic subject's velocity, acceleration, and steering changes satisfy the corresponding subject's motion characteristics.

[0049] After the trajectory physical executability test is completed, the system reads the structured task script, dynamic main trajectory, road and area labels, and physical detection results according to a unified timeline to determine the interactive event status. Interactive event statuses include pending, in progress, completed, and incomplete. When the triggering condition is met, the system sets the event status to in progress; when the termination condition is met and the corresponding trajectory passes physical detection, the system confirms the event termination time and sets the event status to completed; when the trajectory finishes running or the termination condition is not met after a specified time, the system sets the event status to incomplete.

[0050] Event tags include event identifier, event type, executing entity identifier, target identifier or type, occurrence area or lane, start time, end time, and event status. Collision, safety distance, zone crossing, and kinematic constraint detection results are recorded as independent physical detection tags.

[0051] The task status is determined based on the target requirements and the completion status of interactive events in the structured task script, including pending execution, executing, completed, and incomplete. When the dynamic subject completes the interactive events specified in the task and reaches the target location or meets other task termination conditions, the system sets the task status to completed; when the trajectory runs to completion or the task termination conditions are not met after the specified time, the system sets the task status to incomplete.

[0052] Furthermore, based on the structured task script and simulation constraint data, multi-agent motion trajectories are generated, including: The starting point, target point, and key road segments in the structured task script are mapped to the passable space in the simulation constraint data, and a path search algorithm is used to generate a geometric path in the passable space. In each simulation time step, each dynamic subject calculates the desired motion direction and desired velocity along the geometric path, and updates the local motion state according to the current position, velocity and safety distance of the surrounding dynamic subjects, generating the discrete motion results of each dynamic subject. Perform kinematic constraint-based curve smoothing and time parameterization on discrete motion results to generate continuous trajectories that include timestamps, positions, attitudes, linear velocities, angular velocities, and behavioral states.

[0053] Furthermore, based on the interaction event templates in the structured task script, interaction events are generated in the multi-subject motion trajectories, including: Read the interaction event template. The interaction event template records the event type, execution subject, target object, occurrence area, triggering condition, target behavior, termination condition and time constraint. When the triggering condition is met, record the start time of the event and the corresponding subject state; Generate the movement trajectory of the executing entity based on the target behavior; When the termination condition is met, record the candidate termination time and the corresponding subject state.

[0054] Furthermore, after generating the multi-subject motion trajectories, it also includes: The multi-subject motion trajectory is input into the trajectory physical executability detection module. Based on the shape and size of each dynamic subject and the mesh model, the trajectory physical executability detection module performs collision detection, safety distance detection, region boundary crossing detection and kinematic constraint detection on the multi-subject motion trajectory. Based on the detection results of collision detection, safe distance detection, area boundary crossing detection, and kinematic constraint detection, the event state of the interactive event is determined.

[0055] In this embodiment, the visual data generation and training sample output include: The simulated training environment generates visual information according to the camera perspective of the embodied intelligent device to be simulated. The system reads the pre-imported camera imaging parameters, installation position, and orientation from the dynamic subject template, and calculates the camera's position and orientation in a unified world coordinate system based on the position and attitude of the device subject recorded in the trajectory at each moment.

[0056] The visual rendering and occlusion synthesis module uses the camera position, orientation, and imaging parameters at the same simulation moment to generate a static city background image corresponding to the 3D Gaussian splash model, a static scene depth map corresponding to the scene mesh model, and images, depth maps, and instance masks corresponding to other dynamic subjects.

[0057] The system determines the foreground and background occlusion relationships of each pixel based on the static scene depth map and the dynamic subject depth map, and generates a synthetic image, depth map, and instance mask corresponding to the camera viewpoint of the embodied intelligent device, ensuring that the simulated visual information is synchronized with the device trajectory, camera position, and camera orientation. Dynamic occlusion is recorded as a visual visibility tag.

[0058] The training data output module uses a unified task number, event identifier, and timestamp to write structured task scripts, dynamic subject trajectories, physical detection results, event labels, task status, camera parameters, images, depth maps, and instance masks into the same training sample, so that the task, trajectory, detection results, event status, and visual information remain consistent.

[0059] Furthermore, visual information is generated based on the motion trajectories of multiple subjects, including: Read the camera imaging parameters, installation position and orientation recorded in the dynamic subject template, and calculate the position and orientation of the camera in the unified world coordinate system based on the position and attitude of the device subject at each moment in the multi-subject motion trajectory. Based on the camera's position and orientation, static city background images corresponding to the 3D Gaussian splash model, static scene depth maps corresponding to the mesh model, and dynamic subject images, dynamic subject depth maps, and instance masks corresponding to the dynamic subject are generated respectively. The foreground and background occlusion relationships of each pixel are determined based on the static scene depth map and the dynamic subject depth map, and a composite image, depth map and instance mask corresponding to the camera viewpoint are generated.

[0060] Furthermore, structured task scripts, multi-agent motion trajectories, interaction events, and visual information are written into the same training sample, including: Using a unified task number, event identifier, and timestamp, structured task scripts, multi-subject motion trajectories, physical detection results, event labels, task status, camera parameters, images, depth maps, and instance masks are written into the same training sample.

[0061] This embodiment also provides an open city embodied intelligence simulation training environment construction system, including: The data import module is used to acquire open city 3D data, which includes mesh models and 3D Gaussian splash models. The spatial calibration module is used to perform spatial calibration on the mesh model and the 3D Gaussian splash model, and to convert the mesh model and the 3D Gaussian splash model to the unified world coordinate system. The simulation constraint generation module is used to generate open city simulation constraint data based on the mesh model; The dynamic subject template library contains multiple dynamic subject templates. Each dynamic subject template records the motion attributes and passage constraints of the corresponding type of dynamic subject. Furthermore, after obtaining the simulation constraint data, the system establishes a dynamic subject template library. This library includes pedestrian templates, vehicle templates, robot templates, and drone templates. Each type of dynamic subject template records its geometric dimensions, collider, visual model, velocity range, acceleration constraints, steering constraints, passable area, safe distance, and mission capabilities.

[0062] For embodied intelligent devices such as robots, autonomous vehicles, and drones, the system also records the device's camera imaging parameters, as well as the camera's installation position and orientation relative to the device's main body, in the corresponding template. These parameters are imported into the simulation training environment from the actual camera configuration of the device to be simulated, and are used to generate visual information according to the camera's perspective of the corresponding device.

[0063] Pedestrians move on a navigation grid, generating motion based on speed range, dwell time, target area, and local obstacle avoidance rules. Vehicles move on a lane map, generating motion based on lane connections, speed limits, braking distance, and yield rules. Robots move on a navigation grid, generating motion based on target points, local obstacle avoidance, safety distance, and kinematic constraints. Unmanned aerial vehicles (UAVs) move in three-dimensional flight space, generating motion based on altitude range, waypoints, building obstacle avoidance distance, and speed constraints.

[0064] A large language model agent is used to generate structured task scripts based on the input city area summary, available area list, dynamic subject template, target task type, and task difficulty. The trajectory and event generation module is used to generate multi-subject motion trajectories based on structured task scripts and simulation constraint data, and to generate interactive events in the multi-subject motion trajectories based on interactive event templates in the structured task scripts. The visual rendering and output module is used to generate visual information based on the motion trajectories of multiple subjects, and write the structured task script, the motion trajectories of multiple subjects, interactive events and visual information into the same training sample to obtain an open city embodied intelligent simulation training environment.

[0065] Example 2 This embodiment discloses a method for constructing an open city embodied intelligence simulation training environment, including: Acquire open city 3D data, which includes a mesh model and a 3D Gaussian splash model. The mesh model is used to provide the geometric structure of the open city scene, and the 3D Gaussian splash model is used to provide the static visual appearance of the open city scene. Spatial calibration is performed on the mesh model and the three-dimensional Gaussian splash model, and the mesh model and the three-dimensional Gaussian splash model are transformed to a unified world coordinate system; Generate open city simulation constraint data based on the grid model; Establish a dynamic subject template library, which contains multiple dynamic subject templates. Each dynamic subject template records the motion attributes and passage constraints of the corresponding type of dynamic subject. Input the city area summary, available area list, dynamic subject template, target task type and task difficulty into the large language model agent, and generate a structured task script through the large language model agent; Based on the structured task script and the simulation constraint data, a multi-subject motion trajectory is generated, and an interactive event is formed in the multi-subject motion trajectory according to the interactive event template in the structured task script. Visual information is generated based on the multi-subject motion trajectories, and the structured task script, the multi-subject motion trajectories, the interactive events, and the visual information are written into the same training sample to obtain the open city embodied intelligent simulation training environment.

[0066] Specifically, the following steps are included: Step one involves importing a mesh model and a 3D Gaussian splash model of the same urban area into the system. Both models are generated from the same set of open city multi-view images and the same photogrammetric results. The mesh model includes the geometry of roads, buildings, sidewalks, plazas, walls, fixed obstacles, and ground facilities. The 3D Gaussian splash model contains the static visual appearance of the urban area.

[0067] Step two: After data import, the system uses the camera pose and 3D reconstructed coordinates obtained from photogrammetry to perform spatial calibration. Both the mesh model and the 3D Gaussian splash model inherit the same set of camera poses and 3D reconstructed coordinates, and the same coordinate transformation is applied when importing into the simulation training environment, converting to a unified world coordinate system. After coordinate transformation, the system uses the unified world coordinate system to save the city area, dynamic subject position, camera pose, and trajectory points.

[0068] Step three: After spatial calibration, the system generates collision meshes, navigation meshes, lane maps, 3D flight spaces, depth proxy meshes, and region labels. First, the system imports publicly available geographic information data and maps road, intersection, and region information to a unified coordinate system based on geographic coordinates or control points. Then, it uses polygon editing tools to correct region boundaries, categories, traffic attributes, and connectivity. After geographic information mapping, the system establishes a directed traffic network according to road segments, lanes, and their connectivity, recording traffic direction, openness, congestion level, and permitted dynamic subject types. For areas outside roads, the system projects the 2D semantic segmentation results of the original image onto the mesh model, forming candidate semantic regions through multi-view label voting and region growing, which are then corrected by operators. After forming the surface semantic regions, the system combines subject size and mobility to generate passable areas for ground subjects such as robots and pedestrians. For drones, the system determines the flyable space based on scene boundaries, flight altitude, building obstacles, and manually designated no-fly zones.

[0069] Step four: After obtaining the simulation constraint data, the system establishes a dynamic subject template library. Pedestrian templates include pedestrian dimensions, capsule colliders, walking speed, dwell time, and obstacle avoidance radius. Vehicle templates include vehicle dimensions, vehicle colliders, lane rules, maximum speed, acceleration, and braking distance. Robot templates include robot dimensions, colliders, maximum speed, steering limits, safety distance, and navigation rules. Drone templates include drone dimensions, flight altitude, maximum speed, climb rate, and building obstacle avoidance distance. Furthermore, for embodied intelligent devices such as robots, autonomous vehicles, and drones, the system also records the imaging parameters of the actual device's camera, as well as the camera's installation position and orientation relative to the subject, in the corresponding template.

[0070] Step 5: After the dynamic subject template library is established, the large language model agent receives the city area summary, the list of available areas, the dynamic subject template, the target task type, and the task difficulty. The global route planning module determines the target location and the overall route, which together represent the high-level action intention of the dynamic subject. The local behavior planning module generates behaviors such as moving forward, turning, changing lanes, waiting, and avoiding obstacles, along with their triggering conditions. Based on this, the agent outputs a structured action plan and a structured task script. The structured task script records the task type, the main dynamic subject, the interacting dynamic subjects, the starting area, the target area, motion constraints, interaction event constraints, and data output requirements. The interaction event template records the event type, the executing subject, the target object, the occurrence area, the triggering condition, the target behavior, the termination condition, and the time constraint. The interaction events include behaviors such as robot obstacle avoidance, unmanned vehicle lane changing, vehicle yielding, and drone detour.

[0071] Step Six: After the structured task script passes verification, the system runs multi-agent trajectory generation based on the Melodie subject modeling and simulation framework. The trajectory generation module calculates the specific path based on the key areas, key road segments, or flight corridors defined by the overall route in the structured action plan. First, the system generates a geometric path using the A* path search algorithm on the navigation grid, lane map, or 3D path point network. Subsequently, pedestrians, robots, and drones calculate local collision avoidance speeds using the optimal reciprocal collision avoidance algorithm, while the unmanned vehicle generates car-following and lane-changing behaviors using an intelligent driver car-following model and a lane-changing model that minimizes acceleration disturbances. For interactive events, the system adjusts the departure time, expected speed, or waiting time of the relevant subjects according to the event template. Finally, the system smooths the discrete motion results and parameterizes the time to form a continuous trajectory containing timestamps, positions, attitudes, speeds, and behavioral states.

[0072] Step seven: After obtaining the continuous trajectory, the system inputs it into the trajectory physics executability detection module. This module, based on a physics simulation engine (specifically, the NVIDIA PhysX 5 physics engine), determines the colliders based on the external dimensions of each dynamic subject and performs parallel collision detection, safety distance detection, region boundary crossing detection, and kinematic constraint detection in conjunction with the scene mesh model. After completing the above detections, the system determines the interaction event state based on the event triggering conditions, termination conditions, and physics detection results, and determines the task state based on the target position and the completion status of the interaction event.

[0073] Step eight: After the trajectory and interaction event states are determined, the visual rendering and occlusion synthesis module reads the device camera imaging parameters, installation position, and orientation recorded in the dynamic subject template, and calculates the camera's position and orientation in the unified world coordinate system at each simulation moment based on the trajectory. Subsequently, the module generates a static city background image, a static scene depth map, a dynamic subject image, a dynamic subject depth map, and an instance mask. Finally, the system performs occlusion judgment based on the static scene depth map and the dynamic subject depth map, generating open city training images with consistent occlusion relationships.

[0074] Step nine: After generating the visual information, the training data output module uses a unified task number, event identifier, and timestamp to write the structured task script, dynamic subject trajectory, physical detection results, event labels, task status, camera parameters, images, depth maps, and instance masks into the same training sample. This ensures that the task, trajectory, detection results, event status, and visual information maintain their correspondence. At this point, existing open city 3D data has been transformed into an embodied intelligent simulation training environment, forming a reusable open city training sample.

[0075] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for constructing an open city embodied intelligence simulation training environment, characterized in that, include: Acquire open city 3D data, which includes a mesh model and a 3D Gaussian splash model. The mesh model is used to provide the geometric structure of the open city scene, and the 3D Gaussian splash model is used to provide the static visual appearance of the open city scene. Spatial calibration is performed on the mesh model and the three-dimensional Gaussian splash model, and the mesh model and the three-dimensional Gaussian splash model are transformed to a unified world coordinate system; Generate open city simulation constraint data based on the grid model; Establish a dynamic subject template library, which contains multiple dynamic subject templates. Each dynamic subject template records the motion attributes and passage constraints of the corresponding type of dynamic subject. Input the city area summary, available area list, dynamic subject template, target task type and task difficulty into the large language model agent, and generate a structured task script through the large language model agent; Based on the structured task script and the simulation constraint data, a multi-subject motion trajectory is generated, and an interactive event is formed in the multi-subject motion trajectory according to the interactive event template in the structured task script. Visual information is generated based on the multi-subject motion trajectories, and the structured task script, the multi-subject motion trajectories, the interactive events, and the visual information are written into the same training sample to obtain the open city embodied intelligent simulation training environment.

2. The method for constructing an open city embodied intelligent simulation training environment according to claim 1, characterized in that, The acquisition of open city 3D data includes: A mesh model and a 3D Gaussian splash model are generated from the same set of multi-view images of an open city and the same photogrammetric results, respectively. The mesh model and the 3D Gaussian splash model share the camera pose and 3D reconstructed coordinates obtained from the photogrammetry. The spatial calibration of the mesh model and the three-dimensional Gaussian splash model includes: The same coordinate transformation is applied to the mesh model and the 3D Gaussian splash model to transform them from the 3D reconstructed coordinates to the unified world coordinate system, so that the city structure in the mesh model and the corresponding visual content in the 3D Gaussian splash model maintain spatial correspondence.

3. The method for constructing an open city embodied intelligent simulation training environment according to claim 1, characterized in that, Open city simulation constraint data is generated based on the grid model, including: Based on the geometric structure of the mesh model, and combined with geographic information data mapping and / or 2D image semantic segmentation results, collision mesh, navigation mesh, lane map, 3D flight space, depth proxy mesh, and region labels are generated.

4. The method for constructing an open city embodied intelligent simulation training environment according to claim 1, characterized in that, The large language model agent includes a global route planning module and a local behavior planning module; The generation of structured task scripts through the large language model agent includes: The global route planning module determines the target location and overall route based on the task requirements and the simulation constraint data. The target location and overall route together represent the high-level action intention of the dynamic subject. The local behavior planning module generates sequentially executed local behaviors based on the overall route and dynamic subject distribution. The local behaviors include at least one of forward movement, turning, lane changing, waiting, and yielding. A structured action plan is generated based on the overall route and the local behaviors; The structured task script is generated based on the structured action plan. The structured task script includes task type, dynamic subject, starting region, target region, motion constraints, and interaction event constraints.

5. The method for constructing an open city embodied intelligent simulation training environment according to claim 1, characterized in that, Based on the structured task script and the simulation constraint data, multi-agent motion trajectories are generated, including: The starting point, target point, and key road segments in the structured task script are mapped to the passable space in the simulation constraint data, and a path search algorithm is used to generate a geometric path in the passable space. In each simulation time step, each dynamic subject calculates the desired motion direction and desired velocity along the geometric path, and updates the local motion state according to the current position, velocity and safety distance of the surrounding dynamic subjects, generating the discrete motion results of each dynamic subject. The discrete motion results are subjected to kinematic constraint-based curve smoothing and time parameterization to generate a continuous trajectory that includes timestamps, position, attitude, linear velocity, angular velocity, and behavioral state.

6. The method for constructing an open city embodied intelligent simulation training environment according to claim 5, characterized in that, Based on the interaction event template in the structured task script, interaction events are generated in the multi-subject motion trajectories, including: Read the interaction event template, which records the event type, executing subject, target object, occurrence area, triggering condition, target behavior, termination condition, and time constraint; When the triggering condition is met, record the start time of the event and the corresponding subject state; Generate the motion trajectory of the executing entity based on the target behavior; When the termination condition is met, record the candidate termination time and the corresponding subject state.

7. The method for constructing an open city embodied intelligent simulation training environment according to claim 1, characterized in that, After generating the multi-subject motion trajectory, the method further includes: The multi-entity motion trajectory is input into the trajectory physical executability detection module. The trajectory physical executability detection module performs collision detection, safety distance detection, region boundary crossing detection, and kinematic constraint detection on the multi-entity motion trajectory based on the external dimensions of each dynamic entity and the mesh model. The event state of the interaction event is determined based on the detection results of the collision detection, the safe distance detection, the area boundary crossing detection, and the kinematic constraint detection.

8. The method for constructing an open city embodied intelligent simulation training environment according to claim 1, characterized in that, Generate visual information based on the multi-subject motion trajectories, including: Read the camera imaging parameters, installation position and orientation recorded in the dynamic subject template, and calculate the position and orientation of the camera in the unified world coordinate system based on the position and attitude of the device subject at each moment in the multi-subject motion trajectory; Based on the position and orientation of the camera, a static city background image corresponding to the 3D Gaussian splash model, a static scene depth map corresponding to the mesh model, and a dynamic subject image, dynamic subject depth map, and instance mask corresponding to the dynamic subject are generated respectively. Based on the static scene depth map and the dynamic subject depth map, the foreground and background occlusion relationships of each pixel are determined, and a composite image, depth map, and instance mask corresponding to the camera viewpoint are generated.

9. The method for constructing an open city embodied intelligent simulation training environment according to claim 1, characterized in that, Writing the structured task script, the multi-agent motion trajectory, the interaction events, and the visual information into the same training sample includes: Using a unified task number, event identifier, and timestamp, the structured task script, the multi-subject motion trajectory, physical detection results, event labels, task status, camera parameters, images, depth maps, and instance masks are written into the same training sample.

10. A system for constructing an open city embodied intelligent simulation training environment, characterized in that, include: The data import module is used to acquire open city 3D data, which includes a mesh model and a 3D Gaussian splash model. The spatial calibration module is used to perform spatial calibration on the mesh model and the three-dimensional Gaussian splash model, and to convert the mesh model and the three-dimensional Gaussian splash model to a unified world coordinate system; The simulation constraint generation module is used to generate open city simulation constraint data based on the grid model. A dynamic subject template library contains multiple dynamic subject templates, and each dynamic subject template records the motion attributes and passage constraints of the corresponding type of dynamic subject; A large language model agent is used to generate structured task scripts based on the input city area summary, available area list, dynamic subject template, target task type, and task difficulty. The trajectory and event generation module is used to generate multi-subject motion trajectories based on the structured task script and the simulation constraint data, and to form interactive events in the multi-subject motion trajectories based on the interactive event templates in the structured task script. The visual rendering and output module is used to generate visual information based on the multi-subject motion trajectories, and write the structured task script, the multi-subject motion trajectories, the interactive events and the visual information into the same training sample to obtain the open city embodied intelligent simulation training environment.