4D scene editing and rendering method and system, product and storage medium

Abstract

A 4D scene editing and rendering method and system, a product and a storage medium relate to the field of scene rendering, and the method comprises the following steps: inputting a 4D target model, a target starting point position in a 4D scene model, a target end point position in the 4D scene model and a middle position point in the 4D scene model into a simulation environment, generating a plurality of candidate motion trails through a kinematics solver, performing physical constraint verification on the candidate motion trails based on a physical engine, screening out a target motion trail meeting physical constraints, inputting the 4D target model, the 4D scene model and the target motion trail into a rendering engine, and rendering the 4D target model, the 4D scene model and the target motion trail. And generating a pose sequence of a plurality of virtual sensors based on preset sensor parameters, rendering the 4D target model, the 4D scene model and the target motion track according to the pose sequence of the virtual sensors, and generating target image data, scene image data and motion data. By implementing the method, a more natural 4D scene motion track can be generated.

Description

Technical Field

[0001] This application relates to the field of scene rendering, and particularly to a 4D scene editing and rendering method, system, product, and storage medium. Background Art

[0002] With the rapid development of autonomous driving, robotics, and computer vision technologies, the testing and verification of intelligent systems have an increasing demand for massive and diverse data. Generating realistic virtual scene data through a simulation system has become an important means to solve the difficulty of obtaining actual scene data.

[0003] In related technologies, a rule-based approach can be used to generate motion trajectories. Specifically, the system pre-sets a series of fixed motion rules and parameters, controls the motion process of virtual objects in the scene through these rules, and renders the scene based on a single sensor perspective to obtain simulation data.

[0004] However, the rendering method from a single sensor perspective results in generated data, leading to unnatural motion trajectories, unable to meet the requirements of multi-sensor fusion systems for different types and perspectives of data, and restricting the value of simulation data in practical applications. Summary of the Invention

[0005] This application provides a 4D scene editing and rendering method, system, product, and storage medium for generating more natural 4D scene motion trajectories.

[0006] In a first aspect, this application provides a 4D scene editing and rendering method, which is applied to a 4D scene editing and rendering system. The method includes: inputting a 4D target model, the target starting position in the 4D scene model, the target ending position in the 4D scene model, and the intermediate position points in the 4D scene model into a simulation environment, and generating multiple candidate motion trajectories through a kinematic solver. The candidate motion trajectory is the path for the 4D target model to reach the target ending position from the target starting position through at least one of the intermediate position points; verifying physical constraints on the candidate motion trajectories based on a physics engine, and screening out target motion trajectories that meet the physical constraints. The physical constraints include gravity, collision, and friction constraints; inputting the 4D target model, the 4D scene model, and the target motion trajectory into a rendering engine, generating pose sequences of multiple virtual sensors based on preset sensor parameters. The sensor parameters include camera internal parameters and the scanning angles of lidar; rendering the 4D target model, the 4D scene model, and the target motion trajectory according to the pose sequences of the virtual sensors to generate target image data, scene image data, and motion data. The target image data and the scene image data include video data and point cloud data, and the motion data includes position and attitude information.

[0007] By adopting the above technical solution, the 4D target model, the target starting position, the target ending position, and the intermediate position points are input into the simulation environment, and the kinematic solver generates candidate motion trajectories, providing diverse path selection for the subsequent process. Based on the physical constraint verification of the physics engine, it can ensure that the selected target motion trajectories conform to real physical laws such as gravity, collision, and friction, making the motion trajectories more reasonable and realistic. Inputting the relevant data into the rendering engine, generating a virtual sensor pose sequence according to the sensor parameters, and then rendering target image data, scene image data, and motion data containing various data types, which meets the requirements of the multi-sensor fusion system for different types and multi-view data and overcomes the limitations of single-sensor perspective rendering.

[0008] Combined with some embodiments of the first aspect, in some embodiments, before the step of inputting the 4D target model, the target starting position in the 4D scene model, the target ending position in the 4D scene model, and the intermediate position points in the 4D scene model into the simulation environment, the method further includes: obtaining a 4D target model and a 4D scene model, where the 4D target model is a dynamic three-dimensional target with a timestamp, and the 4D scene model is a dynamic three-dimensional scene with a timestamp; setting the target starting position and the target ending position in the 4D scene model, and generating a plurality of intermediate position points between the target starting position and the target ending position, where the intermediate position points are generated by random sampling.

[0009] By adopting the above technical solution, a 4D target model and a 4D scene model with timestamps are obtained. The existence of timestamps enables the models to accurately record and reflect the state changes at different times, providing a basis for the precise simulation of dynamic scenes. Setting the target starting and ending positions in the 4D scene model and generating intermediate position points by random sampling increases the diversity of the intermediate position points, enriches the possibilities of path planning, helps the kinematic solver to plan motion paths that better meet the actual scene requirements, are more flexible and diverse, and improves the quality and practicality of the entire 4D scene editing and rendering.

[0010] Combined with some embodiments of the first aspect, in some embodiments, the step of setting the target starting position and the target ending position in the 4D scene model and generating a plurality of intermediate position points between the target starting position and the target ending position specifically includes: determining the spatial coordinates of the target starting position and the spatial coordinates of the target ending position in the three-dimensional space of the 4D scene model; performing random sampling in the region between the spatial coordinates of the target starting position and the spatial coordinates of the target ending position based on a preset sampling density to obtain a candidate intermediate position set; and removing the unreachable candidate intermediate position points from the candidate intermediate position set to obtain the spatial coordinates of a plurality of intermediate position points.

[0011] By adopting the above technical solution, the target starting point and end point position coordinates are determined in the three-dimensional space of the 4D scene model, and a candidate set of intermediate positions is randomly sampled in the area between the two points based on a preset sampling density. The sampling density determines the number and distribution of sampling points and affects the richness of intermediate position points. In the candidate set, unreachable points are removed. By calculating the distance from obstacles and judging the collision situation, the safety and feasibility of the finally determined intermediate position points are ensured, so that the intermediate position points have sufficient diversity and can ensure the safety of the motion path planning.

[0012] Combined with some embodiments of the first aspect, in some embodiments, the step of removing unreachable candidate intermediate position points from the candidate set of intermediate positions to obtain the spatial coordinates of multiple intermediate position points specifically includes: calculating the minimum distance between each candidate intermediate position point and the obstacles in the 4D scene model; when the minimum distance is less than a preset safety distance threshold, judging whether the candidate intermediate position point is inside the obstacle or collides with the obstacle; if it is judged that the candidate intermediate position point is inside the obstacle or collides with the obstacle, marking the candidate intermediate position point as an unreachable point and deleting it from the candidate set of intermediate positions; after completing the reachability verification for all candidate intermediate position points, determining the remaining candidate intermediate position points as intermediate position points.

[0013] By adopting the above technical solution, calculating the minimum distance between the candidate intermediate position point and the obstacle can accurately locate their spatial relationship with the obstacle. When it is less than the safety distance threshold, it is judged whether it is inside the obstacle or in collision, and this judgment method is scientific and reasonable. Marking and deleting unreachable points ensures the safety of intermediate position points. The intermediate position points determined after comprehensive reachability verification provide reliable nodes for motion planning, avoid planning to unsafe areas, and effectively improve the safety and feasibility of the motion trajectory.

[0014] Combined with some embodiments of the first aspect, in some embodiments, the step of rendering the 4D target model, the 4D scene model, and the target motion trajectory according to the pose sequence of the virtual sensor to generate target image data, scene image data, and motion data specifically includes: sequentially obtaining each pose in the pose sequence of the virtual sensor and using each pose as an observation perspective; generating an RGB image and a depth image based on the camera internal parameters of each observation perspective, and generating point cloud data based on the lidar scanning angle of each observation perspective. The RGB image constitutes the target image data, and the depth image and the point cloud data constitute the scene image data; recording the three-dimensional spatial position coordinates and Euler angle attitude parameters of the 4D target model when moving on the target motion trajectory, and using the three-dimensional spatial position coordinates and the Euler angle attitude parameters as the motion data.

[0015] By adopting the above technical solution, the observation perspective is determined using the pose sequence to ensure the systematicness of the perspective. Images are generated based on the camera internal parameters, and the internal parameters precisely control the image generation, truly reflecting the visual information. Point cloud data is generated according to the lidar scanning angle to ensure the data accuracy, record the motion position and attitude information, and the sampling mechanism ensures the precision.

[0016] Combined with some embodiments of the first aspect, in some embodiments, after the step of rendering the 4D target model, the 4D scene model, and the target motion trajectory according to the pose sequence of the virtual sensor to generate target image data, scene image data, and motion data, the method further includes: based on the position and attitude information in the motion data, performing a playback verification on the motion process of the 4D target model on the target motion trajectory to obtain a playback verification result; if the playback verification result meets the preset motion constraint conditions, then determining the target motion trajectory as the final motion trajectory.

[0017] By adopting the above technical solution, playback verification is performed based on the motion data, providing a reliable basis for the accuracy of the motion data. Check various constraint conditions to comprehensively evaluate the rationality of the motion trajectory. Determine the final trajectory when meeting the preset conditions, and the screening mechanism ensures the quality, effectively verifying whether the trajectory meets the expectations, excluding unqualified trajectories, enhancing the reliability and stability of the actual application of the motion trajectory, and improving the accuracy and effectiveness of the system motion planning.

[0018] Combined with some embodiments of the first aspect, in some embodiments, after the step of rendering the 4D target model, the 4D scene model, and the target motion trajectory according to the pose sequence of the virtual sensor to generate target image data, scene image data, and motion data, the method further includes: adding timestamp marks to the target image data, the scene image data, and the motion data, and establishing a temporal correspondence relationship between the video data, the point cloud data, and the position and attitude information.

[0019] By adopting the above technical solution, adding timestamp marks endows the data with a time attribute, establishing a temporal correspondence relationship, enabling the data to be precisely associated according to time, and enhancing the convenience and efficiency of data management.

[0020] In a second aspect, an embodiment of the present application provides a 4D scene editing and rendering system, and the 4D scene editing and rendering system includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, and the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the 4D scene editing and rendering system to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0021] In a third aspect, an embodiment of the present application provides a computer program product containing instructions. When the computer program product runs on a 4D scene editing and rendering system, the 4D scene editing and rendering system is caused to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium including instructions. When the instructions run on a 4D scene editing and rendering system, the 4D scene editing and rendering system is caused to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0023] It can be understood that the 4D scene editing and rendering system provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the method provided in the embodiment of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, and will not be elaborated here.

[0024] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0025] 1. In the present application, by inputting a 4D target model, a target starting position, a target ending position, and an intermediate position point into a simulation environment, a kinematic solver generates candidate motion trajectories, providing diverse path selection for the subsequent process. Based on the physical constraint verification of the physics engine, it can ensure that the selected target motion trajectory conforms to real physical laws such as gravity, collision, and friction, making the motion trajectory more reasonable and realistic. By inputting relevant data into the rendering engine and generating a virtual sensor pose sequence according to sensor parameters, target image data, scene image data, and motion data containing various data types are rendered, meeting the requirements of a multi-sensor fusion system for different types and multi-perspective data, and overcoming the limitations of single-sensor perspective rendering.

[0026] 2. In the present application, by obtaining a 4D target model and a 4D scene model with timestamps, the existence of timestamps enables the model to accurately record and reflect state changes at different times, providing a basis for the precise simulation of dynamic scenes. Set the target starting and ending positions in the 4D scene model, and generate intermediate position points through random sampling. Random sampling increases the diversity of intermediate position points, enriches the possibilities of path planning, helps the kinematic solver to plan a motion path that better meets the actual scene requirements, is more flexible and diverse, and improves the quality and practicality of the entire 4D scene editing and rendering.

[0027] 3. In this application, the target start and end position coordinates are determined in the three-dimensional space of the 4D scene model, and a set of candidate intermediate positions is randomly sampled in the area between the two points based on a preset sampling density. The sampling density determines the number and distribution of sampling points, and affects the richness of intermediate position points. By removing unreachable points from the candidate set and calculating the distance from obstacles and judging the collision situation, the safety and feasibility of the finally determined intermediate position points are ensured, so that the intermediate position points have sufficient diversity and can ensure the safety of the motion path planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a flowchart of a 4D scene editing and rendering method in an embodiment of this application;

[0029] Figure 2 is another flowchart of a 4D scene editing and rendering method in an embodiment of this application;

[0030] Figure 3 is a schematic structural diagram of an entity device of a 4D scene editing and rendering system in an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] The terms used in the following embodiments of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. As used in the specification of this application, the singular forms "a", "one kind", "the above", "the" and "this" are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in this application refers to any or all possible combinations including one or more of the listed items.

[0032] Hereinafter, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of this application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0033] For ease of understanding, the method provided in this embodiment is described in a process below. Please refer to Figure 1 , which is a flowchart of a 4D scene editing and rendering method in an embodiment of this application.

[0034] S101. Input the target starting position in the 4D target model and the 4D scene model, the target ending position in the 4D scene model, and the intermediate position points in the 4D scene model into the simulation environment. Generate multiple candidate motion trajectories through a kinematic solver. The candidate motion trajectory is a path for the 4D target model to reach the target ending position from the target starting position passing through at least one of the intermediate position points.

[0035] Among them, the 4D target model represents a three-dimensional object model including the time dimension, used to describe the geometric shape, material properties, and motion characteristics of the target object. The 4D scene model refers to a three-dimensional environment model including the time dimension, used to describe the static and dynamic characteristics of the environment. The target starting position refers to the starting spatial coordinate point of motion planning, usually represented by (x, y, z). The target ending position represents the target spatial coordinate point of motion planning, also represented by (x, y, z). The intermediate position points refer to the set of spatial coordinate points that the planned path must pass through. The simulation environment refers to a computer environment used to simulate the physical world, including a physics engine and a motion planning module. The kinematic solver refers to an algorithm module for calculating the motion characteristics of an object, used to generate trajectories that satisfy kinematic constraints. The candidate motion trajectory represents a set of smooth curves connecting the starting point, intermediate points, and ending point.

[0036] This step is executed when a motion path needs to be planned for the target object. Specifically, the system first imports the 4D target model into the simulation environment, including its complete geometric information, mass distribution, and motion constraints. Then it loads the 4D scene model and establishes a spatial index structure to support fast collision detection. The system reads the coordinate information of the starting point, ending point, and intermediate position points, and uses these points as the key nodes for path planning. The kinematic solver, based on these inputs, uses numerical optimization methods to calculate multiple trajectories that satisfy the motion constraints. Each trajectory ensures that the target model can smoothly pass through at least one intermediate position point and finally reach the ending position.

[0037] In some embodiments, the candidate motion trajectories can be generated in multiple ways: Optionally, use a sampling-based path planning method. First, randomly sample path points in the space, connect these points to form a path graph, use the A* algorithm to search for the optimal path in the path graph, then use Bezier curves to smooth the path, and finally apply speed planning to generate a complete motion trajectory; Optionally, use an optimization-based trajectory generation method. Define an objective function including distance, smoothness, and dynamic constraints, use gradient descent or sequential quadratic programming methods to optimize the trajectory parameters, and generate a trajectory that satisfies the constraints by adjusting the control point positions and speed configurations. It can be understood that other trajectory generation methods can also be used to implement the generation of candidate motion trajectories, which is not limited here.

[0038] S102. Physically constrain and verify the candidate motion trajectory based on a physics engine, and filter out the target motion trajectory that meets the physical constraints. The physical constraints include gravity, collision, and friction constraints.

[0039] Among them, the physics engine refers to a software system that simulates the basic laws of the physical world and is used to calculate the physical effects in the motion of objects. Physical constraints represent various physical rules that restrict the motion of objects. Gravity constraint refers to the motion restriction generated by the action of gravity on an object, usually expressed as a vertical acceleration of -9.8 m / s². Collision constraint represents the spatial restriction that objects cannot penetrate each other. Friction constraint refers to the resistance generated when objects come into contact with each other, including static friction and dynamic friction. The target motion trajectory represents a feasible motion path that finally meets all physical constraints.

[0040] This step is executed after generating the candidate motion trajectory. Specifically, the system inputs each candidate trajectory into the physics engine for verification and calculates the motion state of the object at discrete time steps. For each time step, the system calculates the acceleration under the action of gravity, detects the collision state with the environment, and calculates the friction force at the contact point. The position and velocity of the object are updated through integral calculation to verify whether the motion violates the physical laws. The system records the verification results of each trajectory, including indicators such as whether a collision occurs, whether the friction force is sufficient, and whether the motion is stable.

[0041] In some embodiments, physical constraint verification can be achieved in various ways: Optionally, use an accurate physical simulation method to establish a complete physical model including a gravity field, collision detection, and friction calculation, perform simulation calculations at high-precision time steps, record the force, acceleration, and position changes of the object, and verify the constraint conditions through threshold judgment; Optionally, use a simplified constraint checking method to discretize the continuous motion into key time points, calculate the gravitational potential energy, detect the distance to the nearest obstacle, and estimate the required friction force at each time point, and verify the constraint conditions through algebraic calculation. It can be understood that other physical verification methods can also be used to achieve the inspection of the constraint conditions, which are not limited here.

[0042] S103. Input the 4D target model, the 4D scene model, and the target motion trajectory into the rendering engine, and generate the pose sequences of multiple virtual sensors based on the preset sensor parameters. The sensor parameters include the camera internal parameters and the scanning angle of the lidar.

[0043] Among them, the rendering engine refers to the software system used to generate the virtual scene image, including functional modules such as geometric processing, lighting calculation, and material rendering. Sensor parameters refer to the set of numerical values that describe the characteristics of virtual sensors. The camera internal parameters represent the optical characteristic parameters of the camera, including focal length, principal point coordinates, distortion coefficients, etc. The lidar scanning angle refers to the emission direction range of the laser beam, usually including the horizontal scanning angle (such as 360 degrees) and the vertical scanning angle (such as ±15 degrees). A virtual sensor refers to a software model that simulates the characteristics of an actual sensor in a simulation environment. The pose sequence represents the combination of the position and pose of the sensor in a time series, usually represented by the position coordinates (x, y, z) and Euler angles (roll, pitch, yaw).

[0044] After determining the target motion trajectory, this step is executed when generating sensor observation data. Specifically, the system first loads the 4D target model and the scene model into the rendering engine to establish a complete scene graph structure. Then it reads the preset sensor parameters, including parameters such as the resolution, field of view angle, and exposure time of the camera, as well as parameters such as the scanning frequency, angular resolution, and ranging range of the lidar. Based on the scene characteristics and observation requirements, the system calculates the optimal sensor layout plan and generates a series of observation poses. Each pose ensures that high-quality observation data can be obtained while avoiding observation blind spots.

[0045] In some embodiments, the generation of the virtual sensor pose sequence can be achieved in multiple ways: Optionally, use the pose planning method based on view entropy. First, uniformly sample candidate observation points in the scene space, then calculate the information gain of each observation point, select the position with the largest amount of information as the sensor placement point, and finally calculate the optimal observation direction according to the scene structure and generate a continuous pose sequence through smooth interpolation; Optionally, use the pose generation method based on target tracking. First, calculate the predicted position according to the target motion trajectory, then set the optimal observation distance and angle between the sensor and the target, and finally generate a sensor motion sequence that can continuously track the target. It can be understood that other methods can also be used to achieve the generation of the virtual sensor pose sequence, which is not limited here.

[0046] S104. Render the 4D target model, the 4D scene model, and the target motion trajectory according to the pose sequence of the virtual sensor to generate target image data, scene image data, and motion data. The target image data and the scene image data include video data and point cloud data, and the motion data includes position and pose information.

[0047] Among them, rendering refers to the process of converting a three-dimensional scene into a two-dimensional image or point cloud data. Target image data refers to the visual information that describes the appearance characteristics of the target object. Scene image data represents the geometric and visual characteristics of the environment. Video data refers to a sequence of continuous images, usually collected at a fixed frame rate (such as 30fps). Point cloud data represents a data format that uses a set of three-dimensional points to describe the geometric characteristics of the object surface. Motion data refers to the temporal data that describes the motion state of the object. Position information represents the coordinates (x, y, z) of the object in space. Pose information represents the rotation angle of the object, usually represented by Euler angles (roll, pitch, yaw).

[0048] This step is executed after generating the virtual sensor pose sequence. Specifically, the system performs multi-modal data rendering according to the sensor pose sequence. For each observation pose, the system calculates the projection matrix and view matrix of the camera, and uses ray tracing or rasterization methods to render the scene, generating an image containing color and depth information. At the same time, the scanning process of the lidar is simulated, the intersection points of the laser and the scene are calculated, and point cloud data with intensity information is generated. The system records the spatial position and pose angle of the target model during the movement process, and the sampling frequency is usually higher than the video frame rate to ensure the accuracy of the motion data.

[0049] In some embodiments, scene rendering and data generation can be achieved in various ways: Optionally, use a physically based rendering method. First, establish a material model of the scene, including properties such as reflectivity and roughness, then use a global illumination algorithm to calculate the light propagation and simulate the real lighting effect, and finally apply post-processing effects to generate the final image, while calculating the depth map and normal map; Optionally, use a real-time rendering method. First, use a hierarchical spatial data structure to accelerate scene rendering, then use deferred rendering technology to calculate geometric information and lighting information respectively, and finally synthesize to generate the final image, and at the same time, use GPU acceleration to achieve real-time generation of point cloud data. It can be understood that other rendering methods can also be used to achieve the generation of multi-modal data, which is not limited here.

[0050] The following further describes the more specific process of the method provided in this embodiment. Please refer to Figure 2 , which is another process schematic diagram of the 4D scene editing and rendering method in the embodiment of the present application.

[0051] S201. Obtain a 4D target model and a 4D scene model. The 4D target model is a dynamic three-dimensional target with a timestamp, and the 4D scene model is a dynamic three-dimensional scene with a timestamp.

[0052] A 4D target model refers to a dynamic object model that has geometric properties such as shape, size, and position in three-dimensional space and also includes time dimension information. For example, a pedestrian model not only includes its three-dimensional features such as appearance and volume but also includes the action sequence and corresponding timestamps when walking. A timestamp is a time mark that records the state of the model at a specific moment, usually in milliseconds or microseconds. A 4D scene model is a dynamic three-dimensional model that describes the entire environment, including static objects such as buildings, roads, and trees, as well as other dynamic elements (such as moving vehicles, weather changes, etc.) and corresponding time information.

[0053] In actual implementation, the system first reads the required 4D target model file from a preset model library. This file contains the geometric mesh data, material information, skeletal animation data, and time series data of the model. At the same time, it loads the pre-constructed 4D scene model, which may be stored in a multi-layer structure, including a terrain layer, a building layer, a vegetation layer, etc., and each layer has corresponding time series information. The system parses and loads these data into memory to construct a complete scene representation.

[0054] S202. Determine the spatial coordinates of the target starting position and the spatial coordinates of the target ending position in the three-dimensional space of the 4D scene model.

[0055] Spatial coordinates refer to the position information represented by (x, y, z) in a three-dimensional Cartesian coordinate system. The target starting position represents the initial position where the 4D target model starts to move, and the ending position represents its target position. These positions usually need to consider constraints such as terrain height and accessible areas in the scene.

[0056] In specific implementation, the system first establishes a three-dimensional coordinate system in the 4D scene model, usually with a fixed point in the scene as the origin (0, 0, 0). Then, based on the analysis of the accessible areas in the scene, it determines the legal starting coordinates, such as (100, 0, 50), which represents a position 100 meters east and 50 meters north of the origin on the ground. The ending coordinates are determined in the same way, such as (300, 0, 150). These coordinates will be converted into the internal data representation form of the scene for subsequent path planning.

[0057] S203. Based on a preset sampling density, perform random sampling in the area between the spatial coordinates of the target starting position and the spatial coordinates of the target ending position to obtain a set of candidate intermediate positions.

[0058] Sampling density refers to the number of points for random sampling in a unit space, usually expressed as points per cubic meter. Random sampling refers to the process of selecting spatial points in a specified spatial area based on a certain random distribution (such as uniform distribution or Gaussian distribution). The set of candidate intermediate positions refers to the set of all spatial points obtained through sampling.

[0059] During specific execution, the system first determines the sampling space range based on the starting and ending coordinates, such as forming an envelope box. Set the sampling density parameter, such as 10 sampling points per cubic meter. Then use methods such as Monte Carlo sampling to generate random coordinate points within this space range according to the set density. For example, if the volume of the envelope box is 100 cubic meters and the sampling density is 10 points per cubic meter, a total of 1000 candidate points are generated. The coordinates of these points are all stored in a data structure (such as an array or a list) to form a collection of candidate intermediate positions.

[0060] S204. Calculate the minimum distance between each of these candidate intermediate position points and the obstacles in the 4D scene model.

[0061] A candidate intermediate position point refers to a spatial coordinate point obtained through random sampling, represented in the form of (x, y, z). An obstacle refers to any object in the 4D scene model that may hinder the movement of the target, including static obstacles (such as walls, railings) and dynamic obstacles (such as other moving objects). The minimum distance refers to the shortest Euclidean distance from the candidate point to the surface of the obstacle, that is, the minimum value of the straight-line distance between two points.

[0062] The system uses methods of space partitioning and nearest point calculation to implement distance calculation. First, organize the obstacle data in the scene into an octree structure, and each node stores the bounding box information of the obstacle. For each candidate point, the system uses the bounding box hierarchical traversal algorithm to calculate the distance from this point to the surface of each obstacle. During specific calculation, first calculate the distance from the point to the bounding box of the obstacle as a preliminary screening, and then use the iterative closest point algorithm (ICP) to calculate the minimum distance from the point to the precise grid model of the obstacle, and finally obtain a distance value represented by a floating-point number.

[0063] S205. When the minimum distance is less than the preset safety distance threshold, determine whether the candidate intermediate position point is inside the obstacle or collides with the obstacle.

[0064] The preset safety distance threshold refers to the minimum allowable distance value predefined by the system, usually determined based on the size and motion characteristics of the target object. Inside the obstacle refers to the spatial area occupied by the obstacle. The collision state means that there is an overlap or contact between the spatial positions of the candidate point and the obstacle.

[0065] The specific process for the system to perform collision detection is: First, compare the calculated minimum distance with the preset safety distance threshold (such as 0.5 meters). When the distance is less than the threshold, the system uses the ray casting method to determine whether the point is inside the obstacle: emit rays from this point in all directions, count the number of intersections of the rays with the surface of the obstacle, and judge the inside and outside position of the point according to the parity. At the same time, use the separating axis theorem (SAT) to detect whether there is a collision between the bounding sphere of the point and the obstacle.

[0066] S206. If it is determined that the candidate intermediate position point is inside the obstacle or collides with the obstacle, mark the candidate intermediate position point as an unreachable point and delete it from the candidate intermediate position set.

[0067] An unreachable point refers to a position point that is not suitable as a path planning node due to insufficient safety distance or being inside an obstacle. The marking operation refers to adding a status identifier to the position point. The deletion operation refers to removing the point from the candidate set.

[0068] Specifically, the system performs the following operations in this step: For candidate points determined to be unsafe, the system adds a boolean - type unreachable flag to its data structure and sets it to true. At the same time, the system maintains a list of indexes of valid candidate points and removes the index of the unreachable point from this list. This method not only preserves the original sampling data for subsequent analysis but also ensures that path planning only uses safe position points. In implementation, the system uses a doubly - linked list or a dynamic array to store the candidate point set and achieves efficient deletion operations through pointer or index operations.

[0069] S207. After completing the reachability verification for all candidate intermediate position points, determine the remaining candidate intermediate position points as intermediate position points, and the intermediate position points are generated by random sampling.

[0070] Reachability verification refers to the process of confirming whether a spatial point meets the motion safety requirements through methods such as distance detection and collision detection. The remaining candidate intermediate position points refer to the spatial sampling points that still remain after safety distance and collision detection. The random sampling method refers to the method of using a random number generation algorithm to select coordinate points within a specified spatial range, including uniform random sampling, stratified random sampling, etc.

[0071] The system performs a complete verification and confirmation process in this step: First, organize the data of the candidate points that passed the previous screening, and collect all the points marked as reachable into a new data structure. The system uses a spatial index (such as a KD - tree or an octree) to organize these points for subsequent fast query and path planning. Each retained point contains its three - dimensional coordinate information (x, y, z) and related attributes (such as the distance from the obstacle, local environmental characteristics, etc.). These finally determined intermediate position points form the key node set for subsequent motion planning.

[0072] S208. Input the target start position in the 4D target model, the target end position in the 4D scene model, and the intermediate position points in the 4D scene model into the simulation environment, and generate multiple candidate motion trajectories through a kinematic solver. The candidate motion trajectory is the path for the 4D target model to reach the target end position from the target start position passing through at least one of the intermediate position points.

[0073] A simulation environment refers to a computer environment used to simulate the motion characteristics of the physical world, including a physics engine and a motion planning module. A kinematic solver is an algorithm module that calculates the kinematic characteristics (such as position, velocity, and acceleration) of an object. A candidate motion trajectory refers to a smooth curve connecting a starting point, intermediate points, and an ending point, which needs to satisfy kinematic constraints.

[0074] The system uses a hierarchical path planning method to generate a motion trajectory: First, the A* or RRT (Rapidly-Exploring Random Tree) algorithm is used to plan multiple rough paths in the set of intermediate position points. Then, each path is smoothed using B-spline or Bezier curves to generate a continuous trajectory that satisfies kinematic constraints. The kinematic solver calculates the velocity, acceleration, and attitude changes of the target model on the trajectory to ensure that the trajectory meets constraints such as maximum speed and maximum acceleration.

[0075] S209. Physically-constrained verification is performed on the candidate motion trajectory based on the physics engine to screen out the target motion trajectory that meets physical constraints, where the physical constraints include gravity, collision, and friction constraints.

[0076] A physics engine is a software system that simulates the basic laws of the physical world, including functions such as gravity, collision detection, and contact force calculation. Gravity constraint refers to the motion characteristics of an object under the action of gravity. Collision constraint refers to the spatial limitation that objects cannot penetrate each other. Friction constraint refers to the resistance force generated when objects come into contact.

[0077] The system performs the following verification process in the physics engine: For each candidate trajectory, the system performs physical simulation at discrete time steps. At each time step, the gravitational force acting on the target model is calculated, the collision state with other objects in the scene is detected, and the friction force at the contact point is calculated. The position and attitude of the object are updated through integral calculation to verify whether the motion conforms to physical laws. If a violation of physical constraints (such as penetrating an obstacle, the speed exceeding the maximum value that friction can provide, etc.) is detected at any time step, the trajectory is marked as invalid and removed from the candidate set. The finally retained trajectories form a set of target motion trajectories that meet physical constraints.

[0078] S210. The 4D target model, the 4D scene model, and the target motion trajectory are input into the rendering engine to generate pose sequences of multiple virtual sensors based on preset sensor parameters, where the sensor parameters include camera internal parameters and the scanning angle of the lidar.

[0079] A rendering engine refers to a software system used to generate images of virtual scenes, including functional modules such as geometric processing, lighting calculation, and material rendering. Sensor parameters include camera internal parameters (focal length, principal point coordinates, distortion coefficients, etc.) and lidar parameters (horizontal scanning angle range such as 360 degrees, vertical scanning angle range such as ±15 degrees, angular resolution such as 0.1 degree). A pose sequence refers to the combination of the position and pose of a sensor in a time series, represented by position coordinates (x, y, z) and Euler angles (α, β, γ).

[0080] The system configures virtual sensors based on preset parameters and calculates the observation sequence: First, set the camera internal parameter matrix and lidar parameters according to the calibration data. Then, the system plans multiple observation positions in the scene space and calculates the optimal observation pose for each position. Specifically, when implementing, the view entropy algorithm is used to evaluate the observation effect of each position and select the pose angle with the largest amount of information. The system organizes these poses into a sequence in chronological order, and each sequence element contains position coordinates, pose angles, and timestamp information.

[0081] S211. Render the 4D target model, the 4D scene model, and the target motion trajectory according to the pose sequence of the virtual sensor to generate target image data, scene image data, and motion data. The target image data and the scene image data include video data and point cloud data, and the motion data includes position and pose information.

[0082] This step specifically includes:

[0083] Successively obtain each pose in the pose sequence of the virtual sensor, and use each pose as the observation perspective.

[0084] In this step, the pose sequence refers to a set of sensor spatial position and orientation data arranged in chronological order. A pose contains six degrees of freedom parameters of spatial position coordinates (x, y, z) and rotation angles (roll, pitch, yaw). The observation perspective refers to the observation direction and field of view of the virtual sensor in a certain pose for the scene.

[0085] The system processes the pose sequence in an iterative manner: First, establish a time index structure and organize the pose data into an ordered sequence according to timestamps. For each time point, the system reads the corresponding pose data and constructs an observation matrix. Specifically, when implementing, the system uses a 4×4 homogeneous transformation matrix to represent the pose, where the 3×3 sub-matrix represents rotation and the 3×1 vector represents translation. The system converts the scene points in the world coordinate system to the sensor coordinate system through matrix operations to determine the observation perspective.

[0086] Generate an RGB image and a depth image based on the camera intrinsics for each such observation perspective, generate point cloud data based on the lidar scanning angles for each such observation perspective, the RGB image constitutes the target image data, and the depth image and the point cloud data constitute the scene image data.

[0087] In this step, the camera intrinsics is a set of parameters that describe the optical characteristics of the camera, including the focal length, the principal point coordinates, and the distortion coefficients. The RGB image is a two-dimensional matrix data that records the color information of the scene. The depth image records the distance from each pixel point in the scene to the camera. The lidar scanning angles define the emission direction range of the laser beam, usually including the horizontal and vertical scanning angles. The point cloud data is a data format that represents the geometric features of the object surface using a set of three-dimensional points.

[0088] The system performs a multi-stage rendering process: First, calculate the projection matrix based on the camera intrinsics and pose. Render the scene using a ray tracing or rasterization algorithm to generate a high-dynamic-range RGB image. At the same time, calculate the depth value for each pixel to construct the depth image. For the lidar data, the system simulates the scanning process of the laser beam within the set angle range, calculates the intersection coordinates and reflection intensity of the laser with the scene, and generates structured point cloud data. These data are stored in a standard format, such as the RGB image using the PNG format, the depth image using the 16-bit grayscale image format, and the point cloud data using the PCD format.

[0089] Record the three-dimensional spatial position coordinates and the Euler angle pose parameters when the 4D target model moves on the target motion trajectory, and use the three-dimensional spatial position coordinates and the Euler angle pose parameters as the motion data.

[0090] In this step, the three-dimensional spatial position coordinates use (x, y, z) to represent the position of the object in the world coordinate system. The Euler angle pose parameters use (roll, pitch, yaw) to represent the rotation angles of the object. The motion data is a set of time-series data that describes the motion state of the object.

[0091] The system implements the recording process of the motion data: Establish a sampling mechanism with a fixed time interval (such as 0.01 seconds), and obtain the state information of the target model at each sampling moment. The spatial position is represented by the world coordinates of the center point or feature points of the model, accurate to the millimeter level. The pose angles are represented by Euler angles, recording the rotation angles of the object around the three coordinate axes, accurate to 0.1 degrees. The system organizes these data into a time series, and each record contains a timestamp, position coordinates, and pose angles. Store these data in a binary format to ensure data compactness and reading efficiency. At the same time, establish an index structure to support fast time-based query and interpolation calculations.

[0092] S212. Based on the position and attitude information in the motion data, replay and verify the motion process of the 4D target model on the target motion trajectory to obtain a replay verification result.

[0093] Replay verification refers to a verification method that reapplies the recorded motion data to the target model to check whether the motion process meets expectations. The position and attitude information includes the spatial position and rotation angle data of the object at each moment. The verification result is an evaluation index for the legality of the motion trajectory.

[0094] Specific steps for the system to perform motion replay: First, read the position and attitude sequence in the motion data and construct an interpolation function with time as the index. Then, the system replays the motion at a fixed time step, and uses the interpolation function to calculate the exact position and attitude of the target model at each time point. The system checks multiple constraint conditions during the replay process: motion continuity (whether the changes in position and attitude are smooth), physical feasibility (whether the speed and acceleration are within a reasonable range), and interaction with the environment (whether there are collisions or penetrations). The system integrates these inspection results into a complete verification report, including the specific numerical values of each index and the time series data.

[0095] S213. If the replay verification result meets the preset motion constraint conditions, determine the target motion trajectory as the final motion trajectory.

[0096] The preset motion constraint conditions refer to a set of quantitative indicators used to evaluate the legality of the motion trajectory, including maximum speed limit (such as 3m / s), maximum acceleration limit (such as 2m / s²), maximum angular velocity limit (such as 45 degrees / s), etc. The replay verification result refers to the numerical values of each index obtained after performing a replay test on the motion trajectory. The final motion trajectory refers to the motion path that meets all constraint conditions and can be actually executed.

[0097] Specific process for the system to perform verification and confirmation: First, read the test data for replay verification, including time series data such as position sequence, speed sequence, acceleration sequence, angular velocity sequence, etc. For each sequence, the system calculates key indicators: calculate the maximum displacement and average speed through the position sequence, calculate the maximum speed and acceleration through the speed sequence, and calculate the maximum angular velocity and angular acceleration through the angle sequence. The system compares these calculated values with the preset thresholds to check for any over-limit situations. When all indicators meet the constraint conditions, the system copies the current trajectory data to a new data structure and marks it as the final trajectory. This final trajectory includes complete spatial path information, speed planning, and timestamp data.

[0098] S214. Add timestamp marks to the target image data, the scene image data, and the motion data, and establish a temporal correspondence relationship between the video data, the point cloud data, and the position and attitude information.

[0099] A timestamp marker refers to the time information added to data, usually in the Unix timestamp format (accurate to milliseconds). Video data is a sequence of images captured at a fixed frame rate (such as 30fps). Point cloud data is a set of three-dimensional points obtained by lidar scanning. The position and orientation information includes the spatial coordinates and rotation angles of an object. The temporal correspondence relationship refers to the mapping relationship of different types of data in the time dimension.

[0100] The system establishes a data synchronization mechanism in multiple steps: First, the system adds a capture timestamp to each frame of video data, with the timestamp accuracy reaching the millisecond level. For point cloud data, the start and end times of each scan are recorded. For position and orientation information, the exact moment of each sampling point is recorded. Then, the system creates a time index table, sorting all data by timestamp. A hash table is used to store the mapping relationship from timestamp to data, with the key value being the timestamp and the value being the corresponding data index or pointer. The system also creates an interpolation function to handle the problem of inconsistent sampling frequencies of different data types. For example, when the video frame rate is 30fps and the position data sampling rate is 100Hz, the system can calculate the accurate position at any moment through interpolation. This correspondence relationship is stored in a special data structure to support fast query and data synchronization operations.

[0101] The following describes the 4D scene editing and rendering system in the embodiments of the present invention application from the perspective of hardware processing. Please refer to Figure 3 , which is a schematic structural diagram of an entity device of the 4D scene editing and rendering system in the embodiments of the present application.

[0102] It should be noted that Figure 3 The structure of the 4D scene editing and rendering system shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention.

[0103] As Figure 3 shown, the 4D scene editing and rendering system includes a Central Processing Unit (CPU) 301, which can perform various appropriate actions and processes according to the program stored in the Read-Only Memory (ROM) 302 or the program loaded from the storage section 308 into the Random Access Memory (RAM) 303, such as executing the methods described in the above embodiments. In the RAM 303, various programs and data required for system operation are also stored. The CPU 301, ROM 302, and RAM 303 are connected to each other through a bus 304. The Input / Output (I / O) interface 305 is also connected to the bus 304.

[0104] The following components are connected to the I / O interface 305: an input section 306 including an audio input device, a button switch, etc.; an output section 307 including a liquid crystal display (LCD), an audio output device, an indicator light, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. The drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed so that a computer program read therefrom can be installed into the storage section 308 as needed.

[0105] Specifically, according to an embodiment of the present invention, the processes described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication section 309, and / or installed from the removable medium 311. When the computer program is executed by the central processing unit (CPU) 301, various functions defined in the present invention are executed.

[0106] It should be noted that specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or combined with an instruction execution system, device, or component.

[0107] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. Among them, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the block may occur in a different order from that marked in the accompanying drawings.

[0108] Specifically, the 4D scene editing and rendering system of this embodiment includes a processor and a memory. A computer program is stored on the memory. When the computer program is executed by the processor, the 4D scene editing and rendering method provided in the above embodiment is implemented.

[0109] On the other hand, the present invention also provides a computer-readable storage medium. This storage medium may be included in the 4D scene editing and rendering system described in the above embodiment; or it may exist alone and not be assembled into the 4D scene editing and rendering system. The above storage medium carries one or more computer programs. When the one or more computer programs are executed by a processor of the 4D scene editing and rendering system, the 4D scene editing and rendering system is enabled to implement the 4D scene editing and rendering method provided in the above embodiment.

[0110] As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the various embodiments of the present application.

[0111] As used in the above embodiments, depending on the context, the term "when..." may be interpreted to mean "if...", or "after...", or "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if detecting (the stated condition or event)" may be interpreted to mean "if determining...", or "in response to determining...", or "when detecting (the stated condition or event)", or "in response to detecting (the stated condition or event)".

[0112] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by hardware instructed by a computer program. This program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The aforementioned storage medium includes various media that can store program codes, such as ROM, random access memory (RAM), magnetic disks, or optical discs.

Claims

1. A 4D scene editing and rendering method, characterized in that: Applied to a 4D scene editing and rendering system, the method comprises: Inputting a 4D target model, a target starting position in a 4D scene model, a target end position in the 4D scene model, and an intermediate position point in the 4D scene model into a simulation environment, and generating a plurality of candidate motion trajectories through a kinematic solver, wherein the candidate motion trajectory is a path of the 4D target model from the target starting position through at least one of the intermediate position points to the target end position; Performing physical constraint verification on the candidate motion trajectories based on a physical engine to select target motion trajectories that meet the physical constraints, wherein the physical constraints include gravity, collision, and friction constraints; The 4D target model, the 4D scene model and the target motion trajectory are input into a rendering engine, and a pose sequence of multiple virtual sensors is generated based on preset sensor parameters, wherein the sensor parameters include camera intrinsic parameters and a scanning angle of a laser radar; The 4D target model, the 4D scene model and the target motion trajectory are rendered according to the posture sequence of the virtual sensor to generate target image data, scene image data and motion data, wherein the target image data and the scene image data include video data and point cloud data, and the motion data includes position and posture information.

2. The method according to claim 1, characterized in that Before the step of inputting the 4D target model, the target starting position in the 4D scene model, the target ending position in the 4D scene model and the intermediate position point in the 4D scene model into the simulation environment, the method further comprises: Acquire a 4D target model and a 4D scene model, wherein the 4D target model is a dynamic three-dimensional target with a timestamp, and the 4D scene model is a dynamic three-dimensional scene with a timestamp; A target starting position and a target end position are set in the 4D scene model, and a plurality of intermediate position points are generated between the target starting position and the target end position, wherein the intermediate position points are generated by random sampling.

3. The method according to claim 2, characterized in that The step of setting a target starting position and a target end position in the 4D scene model, and generating a plurality of intermediate position points between the target starting position and the target end position, specifically includes: Determine the spatial coordinates of the target starting position and the spatial coordinates of the target ending position in the three-dimensional space of the 4D scene model; Based on a preset sampling density, random sampling is performed in the area between the spatial coordinates of the target starting position and the spatial coordinates of the target end position to obtain a set of candidate intermediate positions; Unreachable candidate intermediate position points are eliminated from the candidate intermediate position collection to obtain the spatial coordinates of multiple intermediate position points.

4. The method according to claim 1, characterized in that: The step of eliminating unreachable candidate intermediate position points from the candidate intermediate position collection to obtain the spatial coordinates of multiple intermediate position points specifically includes: Calculating the minimum distance between each of the candidate intermediate position points and an obstacle in the 4D scene model; When the minimum distance is less than a preset safety distance threshold, determining whether the candidate intermediate position point is located inside the obstacle or collides with the obstacle; If it is determined that the candidate intermediate position point is located inside the obstacle or collides with the obstacle, the candidate intermediate position point is marked as an unreachable point and deleted from the candidate intermediate position collection; After completing the reachability verification for all candidate intermediate location points, the remaining candidate intermediate location points are determined as intermediate location points.

5. The method according to claim 1, characterized in that The step of rendering the 4D target model, the 4D scene model and the target motion trajectory according to the pose sequence of the virtual sensor to generate target image data, scene image data and motion data specifically includes: Obtaining each posture in the posture sequence of the virtual sensor in sequence, and using each posture as an observation perspective; Generate an RGB image and a depth image based on the camera intrinsic parameters of each observation angle, and generate point cloud data based on the laser radar scanning angle of each observation angle, wherein the RGB image constitutes the target image data, and the depth image and the point cloud data constitute the scene image data; The three-dimensional spatial position coordinates and Euler angle posture parameters of the 4D target model when it moves on the target motion trajectory are recorded, and the three-dimensional spatial position coordinates and the Euler angle posture parameters are used as the motion data.

6. The method according to claim 1, characterized in that After the step of rendering the 4D target model, the 4D scene model and the target motion trajectory according to the pose sequence of the virtual sensor to generate target image data, scene image data and motion data, the method further includes: Based on the position and posture information in the motion data, replay and verify the motion process of the 4D target model on the target motion trajectory to obtain a replay verification result; If the playback verification result meets the preset motion constraint condition, the target motion trajectory is determined as the final motion trajectory.

7. The method according to claim 1, characterized in that After the step of rendering the 4D target model, the 4D scene model and the target motion trajectory according to the pose sequence of the virtual sensor to generate target image data, scene image data and motion data, the method further includes: Timestamps are added to the target image data, the scene image data, and the motion data, and a time-series correspondence relationship is established between the video data, the point cloud data, and the position and posture information.

8. A 4D scene editing and rendering system, characterized in that: The 4D scene editing and rendering system comprises: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and the one or more processors call the computer instructions to enable the 4D scene editing and rendering system to perform the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on a 4D scene editing and rendering system, the 4D scene editing and rendering system is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is run on a 4D scene editing and rendering system, the 4D scene editing and rendering system is enabled to execute the method according to any one of claims 1 to 7.

Images