An automatic driving 4D Gaussian scene generation method and system based on a risk particle field

Abstract

The application discloses a kind of automatic driving 4D Gaussian field scene generation method and system based on risk particle field.It relates to the field of automatic driving and three-dimensional scene generation method, which comprises the following steps: obtaining automatic driving bird's-eye view space-time sequence, analyzing and constructing continuous space-time risk information density field; the density field is discretized into a set of meshless risk particles using the SPH kernel function, and the dynamics evolution is solved by the transport equation; the risk particles are bound to the 4D Gaussian primitives, and the life cycle index is constructed based on the risk gradient and the time change rate to dynamically control the reproduction, pruning, deformation and opacity attenuation of Gaussian primitives; the updated 4D Gaussian scene is rendered with multiple perspectives to output a panoramic video stream; the reverse evaluator is introduced to calculate the visual information capacity and risk residual, guiding the particle field to evolve in a closed loop with self-supervision.The application breaks the shackles of uniform distribution of spatial computing power, and realizes adaptive allocation of rendering resources and high-fidelity validation data generation in complex high-risk scenarios.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence, computer vision, autonomous driving simulation and 3D scene differentiable rendering technology, specifically involving a method and system for generating 4D Gaussian scenes for autonomous driving based on risk particle fields. Background Technology

[0002] With the rapid evolution of artificial intelligence and deep learning technologies, the core architecture of high-level autonomous driving systems is shifting towards a data-driven, end-to-end pure vision-based perception and control paradigm. In this paradigm, the system typically relies on multiple high-definition surround-view cameras mounted around the vehicle. Through spatial transformation networks, it upscales two-dimensional image features to a unified three-dimensional bird's-eye view or three-dimensional voxel space, thereby achieving accurate 3D reconstruction and trajectory prediction of surrounding traffic participants and static road network topology. However, this purely visual large-scale model, which heavily relies on massive amounts of data, faces a severe "long-tail data hunger" problem. In the real physical world, collecting natural driving data covering all complex interactions, extreme environments, and low-probability, high-risk scenarios not only increases exponentially in terms of time and economic costs but also poses significant safety risks. Therefore, using generative artificial intelligence to synthesize high-quality virtual autonomous driving video data has become an inevitable choice for bridging this data gap.

[0003] While existing autonomous driving video synthesis and generation technologies have alleviated data shortages to some extent, they still face insurmountable bottlenecks in adhering to physical laws, maintaining spatial consistency, and allocating underlying computing power. These bottlenecks manifest in the following technological development paths and inherent limitations:

[0004] First, there are video generation methods based on traditional two-dimensional image diffusion models. With the rise of diffusion models, researchers have attempted to directly generate multi-view driving videos using text prompts or two-dimensional bird's-eye views. Their fatal flaws lie in "cross-view spatial inconsistency" and "temporal physical discontinuities." Traditional independent two-dimensional diffusion models, lacking the absolute geometric constraints of a three-dimensional coordinate system, often result in severe misalignment and breaks in the generated images at the seams when generating adjacent views. Simultaneously, because two-dimensional models lack an understanding of vehicle rigid body kinematics and scene depth evolution, the generated videos are prone to the illusion of "background speed mismatch with vehicle speed" in the temporal dimension, failing to realistically reproduce the dynamic evolution of the physical world.

[0005] Second, there's the 3D scene generation method based on neural radiation fields. To address the multi-view consistency problem, the industry has introduced implicit neural representation technology, which uses a multilayer perceptron to fit the radiation field of a scene. However, NeRF's computational mechanism relies on a large number of dense spatial samples along light rays, resulting in extremely high training and rendering costs, making it difficult to meet the generation requirements of autonomous driving for long-term, large-scale dynamic scenes. Furthermore, NeRF treats the scene as a continuous implicit function, making it difficult to explicitly edit the physics and control the trajectory of independent dynamic objects in the scene (such as moving vehicles and pedestrians).

[0006] Third, dynamic scene generation methods based on traditional 3D Gaussian sputtering and 4D Gaussian sputtering. In recent years, 3D Gaussian sputtering technology has become a rising star in the field of 3D generation due to its explicit point cloud substrate and extremely high differentiable rasterization rendering speed. Existing 4DGS autonomous driving applications typically divide the scene into static background Gaussian and dynamic foreground Gaussian, and drive the motion of Gaussian primitives by learning a set of deformation fields or rigid body transformation matrices. However, this method has a hidden but fatal flaw: "blind allocation of computing power". During the initialization and optimization process, existing Gaussian generation models tend to allocate Gaussian primitives uniformly or simply by relying on image gradients throughout the entire 3D space (i.e., spatial representation computing power). In autonomous driving scenarios, the vast static sky and flat highway surfaces occupy the majority of the space. The information uncertainty in these areas is extremely low, but traditional methods assign millions of Gaussian primitives to them. Conversely, in areas with extremely high information uncertainty, such as emergency cuts directly in front of the vehicle or non-motorized vehicles severely obscured by heavy rain, which are of the greatest value to scene generation, the system does not adaptively allocate computing power. This results in blurry details and severely insufficient physical fidelity in the generation of these critical, complex, and high-risk areas. Summary of the Invention

[0007] This invention addresses the problems existing in the prior art by providing a method and system for generating 4D Gaussian scenes for autonomous driving based on risk particle fields. It breaks the constraints of blindly allocating computing power in traditional 3D generation models and no longer relies solely on black-box networks to "guess" the evolution of the scene. Instead, it introduces a continuous physical information potential field at the underlying level to explicitly guide the proliferation and extinction of Gaussian primitives. This allows the computing power of the generation system to adaptively concentrate on regions with high uncertainty and high dynamic interaction, solving a top-level technical problem that urgently needs to be overcome in the field of high-fidelity data generation for autonomous driving.

[0008] To address the above technical problems, this invention provides the following technical solution: a method for generating 4D Gaussian scenes for autonomous driving based on risk particle fields, comprising the following steps:

[0009] S1. Obtain the 4D bird's-eye view spatiotemporal layout tensor sequence of the autonomous driving environment, the vehicle motion state vector, the dynamic target state matrix and the sensor state degradation prior, and construct a continuous spatiotemporal risk information density field based on collision time, spatiotemporal occlusion visibility and environmental degradation intensity analysis.

[0010] S2. Using the Smooth Particle Hydrodynamics (SPH) kernel function, the continuous spatiotemporal risk information density field is discretized into a set of meshless risk information particles. Under the Lagrange manifold space, a particle transport state equation including spatial convection, isotropic diffusion and local risk yield is constructed, and the risk gradient, time change rate and adaptive local kernel radius of each particle are solved.

[0011] S3. Bind the meshless risk information particle set to the 4D Gaussian scene state in a multi-dimensional spatial feature. Solve the life cycle index jointly based on the risk information value, risk gradient and time change rate of the particles. Use the life cycle index as an explicit prior constraint to control the birth, splitting, pruning, spatial center point drift, covariance anisotropic deformation and opacity properties of the corresponding 4D Gaussian primitives.

[0012] S4. The updated 4D Gaussian primitive set is used to perform multi-view differentiable rasterization volume rendering based on the physical intrinsic and extrinsic parameter matrix of the surround-view multi-camera array. The multi-view panoramic continuous video stream is synchronously projected and decoded on a unified 3D scene state basis, and the aligned structured 3D annotation truth values ​​are extracted.

[0013] S5. Construct a built-in inverse risk estimator to inversely estimate the actual visual information capacity of a local area from the multi-view video stream generated by rendering, calculate the spatiotemporal gap loss between the theoretical risk requirement of the spatiotemporal risk information density field and the actual visual information capacity, and backpropagate the gradient of the spatiotemporal gap loss to adjust the yield of local risk particles and the life cycle parameters of 4D Gaussian primitives, so as to realize the self-supervised evolution of the generation system.

[0014] Furthermore, the analytical construction process of the spatiotemporal risk information density field described in step 1 above is specifically as follows:

[0015] Extract current spatial location Compact risk vector at the location The collision risk component Based on the reciprocal of the collision time, the occlusion visibility risk component is calculated. Sensor degradation risk components obtained through ray tracing. Obtained by mapping prior environmental strength;

[0016] Construct a weight function to represent the importance of the task:

[0017]

[0018] In the above formula, For position The relative distance vector relative to the center of the vehicle. To diagonalize the spatial scale attenuation matrix, This represents the gain coefficient for the vehicle's driving intention. The angle between the unit vector of the planned direction of the vehicle and the vector pointing to the target position;

[0019] Through linear weight vector With bias term The compact risk vector is nonlinearly activated and combined with the weighting function. The spatiotemporal risk information density field is obtained:

[0020]

[0021] in, This is the standard Sigmoid activation function.

[0022] Furthermore, the gridless risk information particle set mentioned in step 2 above, at any time step... Represented as a set of high-dimensional composite vectors , in which any particle Characterized as:

[0023] In the above formula, For spatial location, This is the intrinsic risk information value. For the risk gradient vector, For information flow velocity vector, To adapt to the local kernel radius, Information quality weight;

[0024] Any particle Local number density pass Continuous smooth kernel function Obtained by performing neighborhood interpolation:

[0025]

[0026] in, , For the set of neighborhood particle indices;

[0027] The adaptive local kernel radius Intrinsic risk information value Inversely proportional truncation mapping:

[0028]

[0029] in, Based on the radius, The sensitivity coefficient is used to ensure that particles are densely distributed in high-risk areas and sparsely distributed in low-risk areas.

[0030] The particle transport state equation is obtained by meshless SPH discretization of the partial differential control equation, merging convection and diffusion flows. Its compact analytical evolution form is as follows:

[0031] In the above formula, For relative information flow rates, the first term within the square brackets represents the spatiotemporal convection term, and the second term represents the dissipation coefficient. Controlled isotropic diffusion term, It is a small regularization quantity; The gradient of the smooth kernel function; For local risk yield; An adaptive Gaussian fluctuation term, following Langevin dynamics, is used to simulate the uncertainty of observations at the underlying level.

[0032] Furthermore, the multi-dimensional spatial feature binding and lifecycle control process described in step 3 above specifically includes:

[0033] For the first Gaussian element controlled by a risk information particle. Extracting particle features to calculate the local spatiotemporal life cycle index:

[0034]

[0035] in The dimension balance coefficient;

[0036] time step The number of Gaussian primitives grown within. Follow the discrete on-demand reallocation rule:

[0037]

[0038] Incremental items:

[0039]

[0040] in and These are the splitting threshold and the pruning threshold, respectively. and For the corresponding rate constant, Represents the non-negative truncation operator; when the exponent At that time, enforce the removal pruning of Gaussian primitives;

[0041] The spatial covariance matrix In the risk gradient vector Three-dimensional anisotropic stretching is performed under the induction of the following:

[0042]

[0043] In the above formula, The rotation matrix is ​​jointly constructed by the risk gradient direction and the vehicle's yaw angle; and Non-principal axis scale; For the principal axis stretching along the risk gradient direction, the following conditions must be met:

[0044]

[0045] in, The deformation coefficient enables the visual rendering to adaptively generate physical motion blur features in the direction of the risk mutation gradient.

[0046] The physical opacity Satisfies the logarithmic decay mapping:

[0047]

[0048] parameter It is a decay factor used to explicitly simulate partial visible physical decay in areas of drastic risk change.

[0049] Furthermore, the multi-view differential rasterization volume rendering process described in step 4 above specifically includes:

[0050] Acquire target surround view camera intrinsic parameter matrix and extrinsic transformation matrix Project the Gaussian center coordinates onto a two-dimensional pixel plane coordinate system. ;

[0051] The set of Gaussian primitives sorted by depth along the line of sight Perform differentiable transmittance synthesis and render the camera. Color tensor :

[0052] in The contribution value of the two-dimensional projection of the opacity of the Gaussian element. The color features are spherical harmonics; a structured panoramic depth map corresponding to the viewpoint is output synchronously using a depth cascade operator.

[0053] Furthermore, the self-supervised evolution of the generation system described in step 5 above specifically includes:

[0054] The multi-view tensor output from the rendering is input into the inverse risk assessor to reverse-calculate the actual visual information capacity representation of the current scene. ;

[0055] Define a high-risk spatiotemporal local set In the set Internally, a self-supervised spatiotemporal gap loss functional is constructed. :

[0056] In the above formula, The scaling factor is the normalization coefficient. To allow for error margin, It is a non-negative truncation operator; through calculation By using the analytical gradient of relative particle properties, the risk yield term and life cycle rate constant in high-risk regions are controlled in reverse to achieve closed-loop optimization within the system.

[0057] This invention also provides an autonomous driving 4D Gaussian scene generation system based on risk information particle fields, used to execute the above method, the system comprising:

[0058] The risk information field analysis module is used to obtain the spatiotemporal sequence of autonomous driving from a bird's-eye view and analyze and construct a continuous spatiotemporal risk information density field.

[0059] The meshless particle solver module is used to discretize the density field into a set of meshless risk information particles using a smooth particle hydrodynamic kernel function, and solve the dynamic transport equations of the particles.

[0060] The Gaussian lifecycle control module is used to bind the particle set to the 4D Gaussian scene state and dynamically adjust the proliferation, pruning, anisotropic deformation and opacity of Gaussian elements based on the lifecycle index.

[0061] The multi-view differentiable rendering module is used to combine physical intrinsic and extrinsic parameter matrices to perform rasterization volume rendering and synchronously project and output multi-view continuous video streams and 3D annotations.

[0062] The self-supervised closed-loop evolution module is used to back-calculate the actual visual information capacity through the inverse evaluator and update the system parameters based on the inverse gradient of the spatiotemporal gap loss to achieve adaptive resource allocation evolution.

[0063] The present invention also provides a computer device comprising a storage medium resource pool and a processor array having a deep learning acceleration core. The processor array is configured to execute the method described in any of the preceding claims. During execution, risk information field analysis, SPH particle neighborhood search, transport equation discretization operation, 4D Gaussian attribute lifecycle control, and multi-view differentiable rasterization parallel rendering are performed directly on the tensors in the memory pool to output high-fidelity panoramic video stream data for autonomous driving.

[0064] Compared with the prior art, the beneficial technical effects of the present invention using the above technical solution are as follows:

[0065] (1) It achieves adaptive allocation of generation computing power and maximization of information value. This invention completely abandons the drawback of the traditional 3D generation model which uniformly distributes the representation of computing power in the whole space, and for the first time introduces a risk information density field as an explicit prior for spatial resource allocation. By strongly binding the proliferation and pruning of Gaussian primitives with the uncertainty of space, the system can extract extremely valuable rendering computing power from the vast static background and accurately allocate it to high-value areas with intense dynamic interaction and severe field of view occlusion. While greatly improving the accuracy of local generation, it significantly reduces the video memory overhead of full-scene rendering.

[0066] (2) It endows the 3D generation process with physically continuous manifold properties. This invention innovatively introduces Smooth Particle Hydrodynamics (SPH) into the Gaussian generation pipeline. Instead of relying on black-box neural networks to blindly predict scene changes, it utilizes meshless Lagrange particles to carry out convection and diffusion equations, giving the propagation of uncertain information in the scene a strict physical manifold basis. This underlying physical constraint completely eliminates the spatiotemporal discontinuity problem during cross-frame generation, ensuring the consistency of rigid body dynamics of the generated video over long time.

[0067] (3) A fully internal closed-loop self-supervised evolution mechanism for the generative system was constructed. Traditional generative model optimization often relies on a large amount of manually calibrated data or joint adversarial training with externally fixed perceptual networks. This invention pioneered an inverse visual information capacity estimator, which directly compares the theoretical requirements of the generative framework with the actual output capacity to form a loss gradient, realizing full-link adaptive closed-loop updates from the bottom-level particle yield to the lifecycle of Gaussian primitives. This means that the system possesses the self-evolutionary attribute of continuously strengthening its ability to generate complex scenes without external data feeding. Attached Figure Description

[0068] Figure 1 This is an architecture diagram of the autonomous driving 4D Gaussian scene generation system based on risk particle fields of the present invention;

[0069] Figure 2 This is the data flow diagram of the meshless risk particle field evolution and Gaussian life cycle control of the present invention. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Those skilled in the art can make modifications or equivalent substitutions without departing from the spirit and scope of this invention.

[0071] This invention proposes a method and system for generating 4D Gaussian scenes for autonomous driving based on risk particle fields. Addressing the industry pain point of the extreme scarcity of long-tail, high-risk scene data in high-level autonomous driving simulation testing, this technical solution is the first to deeply integrate smooth particle hydrodynamics with explicit spatial representation using 4D Gaussian sputtering. Its core value lies in serving as a testing and evaluation platform for autonomous driving systems, adaptively allocating rendering computing power to synthesize high-quality long-tail boundary verification test cases. This method does not rely on blind counterfactual random trial and error, but rather on a learning-based risk screening mechanism and self-supervised closed-loop fine-tuning, ensuring that generated resources are precisely tilted towards high-value dynamic interaction regions.

[0072] Level 1 Hardware and Architecture Examples

[0073] In a preferred embodiment, the autonomous driving 4D Gaussian scene generation method and system based on risk particle fields of the present invention are deployed on special computer equipment with heterogeneous tensor acceleration computing power (such as GPU clusters or customized NPU acceleration arrays). The system includes the following core modules working in concert in terms of hardware and logical architecture:

[0074] 1. Risk Information Field Analysis Module: Used to obtain the bird's-eye view spatiotemporal sequence tensor of autonomous driving scenarios, and construct a continuous spatiotemporal risk information density field in Eulerian space through high-concurrency operator analysis.

[0075] 2. Meshless Particle Solver Module: The embedded processor's neighborhood search and particle adaptive discretization kernel is used to discretize the continuous risk field into a set of meshless risk information particles using a smooth particle hydrodynamic kernel function, and iteratively solve the particle transport state equation in the Lagrange manifold space.

[0076] 3. Gaussian Lifecycle Control Module: As the core unit for feature binding and computing power allocation, it is used to dynamically control the proliferation, pruning and extinction, spatial bias, covariance anisotropy deformation and opacity decay of the corresponding 4D Gaussian primitives by taking the risk particle attributes as explicit constraint priors.

[0077] 4. Multi-view Differentiable Rendering Module: Embedded hardware-level parallel rasterization volume rendering unit, combined with the physical intrinsic and extrinsic parameter matrix of the surround-view camera array, synchronously projects and decodes spatially consistent multi-view panoramic continuous video stream and 3D labeled ground truth.

[0078] 5. Self-supervised closed-loop evolution module: It includes a reverse risk assessment gateway and a self-supervised loss calculation unit, which is used to back-calculate the actual visual information capacity and solve the spatiotemporal gap loss. It fine-tunes the underlying parameters of the system through gradient backpropagation to achieve closed-loop evolution of adaptive resource allocation.

[0079] II. Detailed Explanation of Core Method Implementation Steps

[0080] Coordinate system, input data, and output object definition: Let the world coordinate system be... The local coordinate system of the vehicle is , No. Each camera coordinate system is Autonomous driving environment in continuous time The inputs include: 4D bird's-eye view spatiotemporal layout tensor Bicycle motion status Dynamic target state matrix Static Map Sensor degradation prior and multiple camera internal and external parameters .

[0081] The dynamic target state is represented as follows: In the formula The target center location, It is a three-dimensional dimension. Yaw angle For velocity components, For acceleration, For semantic categories.

[0082] The internal scene state of this invention is a set of 4D Gaussian primitives:

[0083] ,in As the center of space, For three-dimensional covariance, For opacity, For color or spherical harmonic coefficients, For rotation quaternions or rotation parameters, For the speed of movement, For semantic or instance properties, This refers to the lifecycle state.

[0084] The final output of this invention includes two categories: the first category is editable and reusable dynamic 4D Gaussian scene states. The second category is a pixel-level, multi-view continuous video stream obtained by physical projection of the scene state through multiple cameras. , as well as the depth map, semantic map, instance map, 3D bounding box and trajectory ground truth that are strictly aligned with it.

[0085] like Figure 1-2 As shown, Step 1: Analytical Construction of the Continuous Spatiotemporal Risk Information Density Field

[0086] To guide the precise allocation of spatial representation computing power, the system first needs to quantify the information uncertainty and testing value of each region within a continuous Eulerian space. This embodiment extracts the current spatial location... Compact risk vector at the location This is used to characterize the underlying environmental features. Among them, the collision risk component... It relies on the first-order dynamic boundary between the vehicle and traffic participants, which is specifically quantified by the inverse of the predicted time to collision (TTC):

[0087]

[0088] In the formula, This represents the total number of obstacles within a given scene. For the car and the first The expected collision time of each obstacle under the current trajectory trend. To prevent regularization of small quantities with a denominator of zero; occlusion visibility risk component. The percentage of occluded, hidden space volume is obtained by performing ray tracing along the sensor's line of sight in a bird's-eye view layout; sensor degradation risk component. Then, a linear mapping is performed based on the weather and light intensity prior extracted from environmental cue words to reflect the boundary region where the confidence of the perception system drops sharply.

[0089] To enable the generated scenarios to apply targeted pressure to specific control strategies, a weight function is constructed to characterize the importance of the tasks. :

[0090] in, For position The relative distance vector relative to the center of the vehicle; This is a diagonalized spatial scale attenuation matrix used to control the rate of risk diffusion with spatial distance; This represents the gain coefficient for the vehicle's driving intention. This is the unit vector representing the vehicle's current planned direction; For target location The unit direction vector pointing towards the center of the vehicle.

[0091] Through linear weight vector With bias term The compact risk vector is nonlinearly activated, combined with the weighting function. Finally, the spatiotemporal risk information density field is obtained. The analytical expression form:

[0092] In the formula, This represents the standard Sigmoid activation function. Indicates matrix transpose. As a bias term, the output density field value is strictly limited to... Within the range, a continuous scalar physical potential field is provided for the subsequent dense deployment of particles.

[0093] Step 2: Meshless discretization of the risk density field using SPH and transport evolution

[0094] After obtaining the continuous risk potential field in Eulerian space, traditional mesh generation schemes are prone to numerical dissipation and boundary fractures when dealing with highly dynamic traffic scenarios with large-scale topological changes. This step introduces the concept of Smooth Particle Hydrodynamics (SPH), discretizing the continuous density field into a set of high-dimensional composite vector particles in Lagrange manifold space. Any particle Expressed as Its corresponding spatial number density pass Continuous smooth kernel function Obtained by neighborhood interpolation:

[0095]

[0096] in, For neighborhood particles Information quality weight, For particles Adaptive local kernel radius A set of neighborhood particle indices within a defined limit. To encourage particles to spontaneously cluster towards high-risk, high-value regions, the adaptive local kernel radius... The intrinsic risk information value it carries Inversely proportional truncation mapping:

[0097] in, Based on the radius, The sensitivity coefficient is physically constrained to force an extremely dense particle distribution in high-risk areas and a sparse particle distribution in low-risk static background areas, thereby achieving a non-uniform evolution of the underlying rendering resolution. Let represent the minimum and maximum values ​​of the adaptive local kernel radius, respectively. This represents a truncation function, used to strictly restrict the value of a variable within a given upper and lower limit range.

[0098] To simulate the dynamic convection and diffusion of risk information with traffic flow, a particle transport state equation is constructed, and its specific partial differential meshless discretization evolution form is expressed as follows:

[0099] in, The relative information flow velocity between particles. and Particles With neighboring particles Information flow velocity vector and Particles With neighboring particles The intrinsic risk information value. The first term within the square brackets represents the spatiotemporal convection induced by rigid body motion, and the second term represents the dissipation coefficient. Controlled isotropic diffusion, It is a small regularization quantity; The spatial gradient of the smooth kernel function; This is a local risk yield term, used to cover the spontaneous risk increment caused by sudden lane changes or the emergence of blind spots; An adaptive Gaussian fluctuation stochastic term following Langevin dynamics is used to simulate the uncertainty of observations at the underlying level.

[0100] Step 3: Spatial binding and lifecycle control of risk particles and 4D Gaussian elements

[0101] To overcome the drawback of traditional 4D Gaussian sputtering, which distributes computational power across the entire space during initialization and deformation field optimization, this step directly cascades and binds discrete risk particle states to the underlying 4D Gaussian elements. For those affected by the first A Gaussian element controlled by a risk particle is first jointly solved for its local spatiotemporal life cycle index. :

[0102] In the formula, As a dimension balance coefficient, this index simultaneously constrains the absolute strength of the risk, the spatial jump rate, and the temporal deterioration rate.

[0103] The single-step time step of the simulation system. The number of Gaussian primitives grown within. Discrete evolution is performed following the on-demand reallocation rules:

[0104]

[0105] In the formula, and These are the preset splitting threshold and pruning threshold, respectively. and For the corresponding rate constant, This represents the non-negative truncation operator. These represent the minimum and maximum growth limits of the Gaussian primitive within a single step time, respectively. When the life cycle index of the region... At that time, the system forcibly executes the release and pruning of redundant Gaussian primitives in the space, thereby frequently relocating video memory and rendering computing power to the core area of ​​the danger evolution.

[0106] Meanwhile, the covariance matrix of Gaussian elements in space In particle risk gradient vector Three-dimensional anisotropic deformation is performed under the induction of the following:

[0107] in, This corresponds to an orthogonal rotation matrix jointly constructed by the risk gradient direction and the vehicle's yaw angle; and Non-principal axis scale; For the principal axis stretching scale along the risk gradient direction, it satisfies the following mapping:

[0108] In the above formula, As the initial isotropic scale, is the deformation stretching coefficient. The physical effect of this gradient-induced stretching is that, in the direction of abrupt risk changes, differentiable rendering automatically generates physical motion blur that conforms to the laws of rigid body dynamics, thereby fundamentally eliminating visual flicker during video frame stitching.

[0109] In addition, to explicitly simulate partial physical attenuation of visibility caused by heavy rain or line-of-sight obstruction, the primitive's physical opacity property... Constrained to a logarithmic decay mapping:

[0110] in, This is the maximum opacity limit. For the corresponding decay parameter, through the time degradation rate Optical transmittance of dynamically suppressed Gaussian projection.

[0111] Step 4: Multi-view miniaturized rasterization volume rendering and physical video output

[0112] After completing the physical adaptive tuning of the 4D Gaussian primitive states, the system introduces the homogeneous transformation matrix of the actual camera array extrinsic parameters of the test vehicle. With intrinsic parameter matrix The center point of the Gaussian space. Two-dimensional pixel coordinates projected onto a multi-view camera The projection equation is as follows:

[0113] in This is the depth value in the camera coordinate system. This is the intrinsic parameter matrix. This is the extrinsic parameter matrix. Further, using the homogeneous Jacobian matrix... The three-dimensional covariance matrix The projection is the covariance matrix in two-dimensional pixel space. :

[0114]

[0115] The set of Gaussian primitives sorted by depth along the line of sight of each camera Performs multi-dimensional rasterization volume rendering and simultaneously outputs the panoramic camera. Multi-view panoramic continuous video color tensor :

[0116] in, This is the set of Gaussian primitives sorted by depth along the line of sight. This is the Gaussian meta-index of the set currently to be rendered. To be ranked in terms of spatial depth The remaining Gaussian meta-indexes, and They are respectively Gaussian elements With primitive Combining two-dimensional covariance Then in pixels The projection contribution value at that location, It represents spherical harmonic color features. Because the volume rendering pipeline is fully differentiable, the scene can directly decode strictly spatially aligned structured 3D labeled truth boxes on a unified 3D substrate while synthesizing realistic video streams.

[0117] Step 5: System Self-Supervised Closed-Loop Evolution Based on Backward Evaluator

[0118] To achieve closed-loop self-evolution without human calibration bias, the system integrates an inverse risk estimator (preferably a visual analysis operator based on a pre-trained end-to-end multi-view feature extraction network). The rendered multi-view panoramic video stream is input into this estimator to inversely estimate the actual visual information capacity representation of each local region. .

[0119] Define a high-risk spatiotemporal local set ,in Activate the threshold for the dangerous boundary. In the set... Internally, the system automatically calculates the spatiotemporal gap loss function between theoretical risk requirements and actual visual capacity. : In the above formula, These are the scale normalization coefficients used for aligning scalars. This is the set error margin. It is a non-negative truncation operator. Indicates spatial location In time Numerical values ​​of continuous spatiotemporal risk information density field at a given location.

[0120] The spatiotemporal gap loss is solved using a differential control engine. Analyze the analytical gradient of the underlying particle properties and perform end-to-end parameter update control:

[0121]

[0122]

[0123] in, and The set adaptive learning rate. Backpropagation of this gradient enables the system to accurately and precisely screen weak spatial regions with blurred rendering or insufficient information carrying capacity, spontaneously increasing the yield of local risk particles. Alternatively, it can propagate parameters to enhance the local resolution of 4D Gaussian primitives. This mechanism completely avoids the limitations of conventional generative models that rely on blind guessing or counterfactual collision sampling, achieving closed-loop self-evolution of the system's long-tail scene data synthesis capabilities without external human intervention.

[0124] Based on the above method, this invention also provides a computer device hardware orchestration architecture capable of supporting the evolutionary computation of the risk particle field. The computer device includes a solid-state storage resource pool, a high-bandwidth memory pool, and a processor array equipped with deep learning acceleration cores. When configured to execute core code sequences, the processor array directly performs customized microarchitecture parallel computations on the tensors in the memory pool.

[0125] Parallel neighborhood search and SPH discrete computation: The processor array utilizes its internal dedicated matrix acceleration unit to perform rapid voxelization of the Eulerian space using block radix sorting, completing the distance matrix calculation and smoothing kernel gradient between tens of thousands of particle pairs within tens of milliseconds. Solving these equations allows for the online derivation of the partial differential equations of the transport state.

[0126] Hardware-level Gaussian deformation and rasterization rendering: Utilizing the processor's texture sampling and floating-point blending precision calculation channels to stretch covariance. The matrix multiplication and exponential decay control are fully allocated to parallel hardware thread blocks to achieve high-throughput numerical integration of multi-step time ordinary differential equations (ODEs) for multi-view perspective projection.

[0127] Furthermore, this invention also provides a non-volatile computer-readable storage medium (such as a high-speed NVMe solid-state drive array). When the server node of the cloud testing platform reads the instruction sequence in this storage medium, it triggers the construction of an analytical mapping from the underlying traffic uncertainty Eulerian scalar potential field to a concrete multi-camera two-dimensional pixel stream, outputting a panoramic high-fidelity synthetic dataset. The generated data stream is directly injected into the vehicle controller of the autonomous driving perception or control system through a high-speed PCIe bus or Ethernet interface for hardware-in-the-loop (HIL) extreme boundary robustness evaluation and blind testing.

[0128] While the present invention has been described above with reference to preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A method for generating 4D Gaussian scenes for autonomous driving based on risk particle fields, characterized in that, The steps are as follows: S1. Obtain the 4D bird's-eye view spatiotemporal layout tensor sequence of the autonomous driving environment, the vehicle motion state vector, the dynamic target state matrix and the sensor state degradation prior, and construct a continuous spatiotemporal risk information density field based on collision time, spatiotemporal occlusion visibility and environmental degradation intensity analysis. S2. Using the Smooth Particle Hydrodynamics (SPH) kernel function, the continuous spatiotemporal risk information density field is discretized into a set of meshless risk information particles. Under the Lagrange manifold space, a particle transport state equation including spatial convection, isotropic diffusion and local risk yield is constructed, and the risk gradient, time change rate and adaptive local kernel radius of each particle are solved. S3. Bind the meshless risk information particle set to the 4D Gaussian scene state in a multi-dimensional spatial feature. Solve the life cycle index jointly based on the risk information value, risk gradient and time change rate of the particles. Use the life cycle index as an explicit prior constraint to control the birth, splitting, pruning, spatial center point drift, covariance anisotropic deformation and opacity properties of the corresponding 4D Gaussian primitives. S4. The updated 4D Gaussian primitive set is used to perform multi-view differentiable rasterization volume rendering based on the physical intrinsic and extrinsic parameter matrix of the surround-view multi-camera array. The multi-view panoramic continuous video stream is synchronously projected and decoded on a unified 3D scene state basis, and the aligned structured 3D annotation truth values ​​are extracted. S5. Construct a built-in inverse risk estimator to inversely estimate the actual visual information capacity of a local area from the multi-view video stream generated by rendering, calculate the spatiotemporal gap loss between the theoretical risk requirement of the spatiotemporal risk information density field and the actual visual information capacity, and backpropagate the gradient of the spatiotemporal gap loss to adjust the yield of local risk particles and the life cycle parameters of 4D Gaussian primitives, so as to realize the self-supervised evolution of the generation system.

2. The method for generating 4D Gaussian scenes for autonomous driving in a risk particle field according to claim 1, characterized in that, In step S1, the spatiotemporal risk information density field The parsing and construction process includes the following sub-steps: S1.1 Extract the current spatial position Compact risk vector at the location It consists of collision risk components Risk component of obstructed visibility and sensor degradation risk component constitute; S1.2 Calculate the weight function used to characterize the importance of the task. : , In the formula, For position The relative distance vector relative to the center of the vehicle; This is the diagonalized spatial scale decay matrix; This represents the gain coefficient for the vehicle's driving intention. The angle between the unit vector of the planned direction of the vehicle and the vector pointing to the target position; S1.3, through linear weight vector With bias term The compact risk vector is nonlinearly activated and combined with the weighting function. Obtain the spatiotemporal risk information density field: , in, The standard sigmoid activation function is represented by T, which represents the transpose, and b represents the bias term.

3. The method for generating 4D Gaussian scenes for autonomous driving in a risk particle field according to claim 2, characterized in that, In step S2, the set of meshless risk information particles at any time step Represented as a set of high-dimensional composite vectors , in which any particle Characterized as: , In the formula, For spatial location, This is the intrinsic risk information value. For the risk gradient vector, For information flow velocity vector, To adapt to the local kernel radius, Information quality weight; Any particle Local number density pass Continuous smooth kernel function Obtained by performing neighborhood interpolation: ,in , For the set of neighborhood particle indices; For neighborhood particles Information quality weight, For particles With neighboring particles The relative spatial position vector between them For particles Adaptive local kernel radius The set of neighborhood particle indices within the constraints For neighborhood particles Spatial location; Adaptive local kernel radius Intrinsic risk information value The inversely proportional truncated mapping satisfies ,in Based on the radius, This is a sensitivity coefficient used to ensure that particles are densely distributed in high-risk areas and sparsely distributed in low-risk areas. Let represent the minimum and maximum values ​​of the adaptive local kernel radius, respectively. This represents a truncation function, used to strictly restrict the value of a variable within a given upper and lower limit range.

4. The method for generating 4D Gaussian scenes for autonomous driving in a risk particle field according to claim 3, characterized in that, In step S2, the particle transport state equation is obtained by meshless SPH discretization of the partial differential control equation, merging convection and diffusion flows. Its compact analytical evolution form is as follows: In the above formula, For relative information flow rate, For information flow velocity vector, Represents neighborhood particles Information flow velocity vector For neighborhood particles The intrinsic risk information value, Represents the spatiotemporal convection term. Indicated by the dissipation coefficient Controlled isotropic diffusion term, It is a small regularization quantity; The gradient of the smooth kernel function; For local risk yield; An adaptive Gaussian fluctuation term, following Langevin dynamics, is used to simulate the uncertainty of observations at the underlying level.

5. The method for generating 4D Gaussian scenes for autonomous driving in a risk particle field according to claim 4, characterized in that, The multidimensional spatial feature binding and lifecycle control process described in step S3 specifically includes: For the first Gaussian element controlled by a risk information particle. Extracting particle features to calculate the local spatiotemporal life cycle index : ， in The dimension balance coefficient; time step The number of Gaussian primitives grown within. Follow the discrete on-demand reallocation rule: ， Incremental Term ,in and These are the splitting threshold and the pruning threshold, respectively. and For the corresponding rate constant, Represents the non-negative truncation operator; when the exponent At that time, Gaussian primitive pruning is enforced. These represent the minimum and maximum limits for the number of Gaussian primitives to grow within a single time step, respectively.

6. The method for generating 4D Gaussian scenes for autonomous driving in a risk particle field according to claim 5, characterized in that, The update control of the spatial covariance matrix and opacity attribute in step 3 is specifically as follows: Spatial covariance matrix In the risk gradient vector Under the induction of the three-dimensional anisotropic stretching, as shown in the following equation: , In the above formula, The rotation matrix is ​​jointly constructed by the risk gradient direction and the vehicle's yaw angle; and Non-principal axis scale; To stretch the principal axis along the risk gradient direction, satisfying the following conditions: , The deformation coefficient enables the visual rendering to adaptively generate physical motion blur features in the direction of the risk mutation gradient. The initial fundamental isotropic scale of the Gaussian element; Physical Opacity Satisfies the logarithmic decay mapping: , In the formula, the parameters As an attenuation factor, it is used to explicitly simulate some visible physical attenuation in areas of drastic risk change. This indicates the maximum physical opacity limit.

7. The method for generating 4D Gaussian scenes for autonomous driving in a risk particle field according to claim 6, characterized in that, Step S4, the multi-view differential rasterization volume rendering process, includes the following sub-steps: S4.1 Acquire target surround view camera intrinsic parameter matrix and extrinsic transformation matrix Project the Gaussian center coordinates onto a two-dimensional pixel plane coordinate system. ; S4.2, The set of Gaussian primitives sorted by depth along the line of sight. Perform differentiable transmittance synthesis and render the camera. Color tensor : , in, The contribution value of the two-dimensional projection of the opacity of the Gaussian element. The color features are spherical harmonics; a structured panoramic depth map corresponding to the viewpoint is synchronously output using a depth cascade operator, where g represents the set of Gaussian primitives sorted by depth. The current Gaussian meta-index to be rendered is denoted by 'l', which represents the remaining Gaussian meta-index that precedes the current Gaussian meta-index 'g' in spatial depth along the camera's view direction. This represents the Gaussian pixel l that appears first along the line of sight in a two-dimensional pixel. The opacity projection contribution value at that location.

8. The method for generating 4D Gaussian scenes for autonomous driving in a risk particle field according to claim 7, characterized in that, In step S5, the self-supervised evolution of the generation system specifically involves: The multi-view tensor output from the rendering is input into the inverse risk assessor to reverse-calculate the actual visual information capacity representation of the current scene. ; Define a high-risk spatiotemporal local set In the set Internally, a self-supervised spatiotemporal gap loss functional is constructed. : In the formula, Indicates spatial location In time The numerical value of the continuous spatiotemporal risk information density field at that location. The scaling factor is the normalization coefficient. This is the tolerance margin; calculated... By using the analytical gradient of relative particle properties, the risk yield term and life cycle rate constant in high-risk regions are controlled in reverse to achieve closed-loop optimization within the system.

9. An autonomous driving scene generation system based on risk particle fields, used to execute the generation method as described in any one of claims 1 to 8, characterized in that, include: The risk information field analysis module is used to obtain the spatiotemporal sequence of autonomous driving from a bird's-eye view and analyze and construct a continuous spatiotemporal risk information density field. The meshless particle solver module is used to discretize the density field into a set of meshless risk information particles using a smooth particle hydrodynamic kernel function, and solve the dynamic transport equations of the particles. The Gaussian lifecycle control module is used to bind the particle set to the 4D Gaussian scene state and dynamically adjust the proliferation, pruning, anisotropic deformation and opacity of Gaussian elements based on the lifecycle index. The multi-view differentiable rendering module is used to combine physical intrinsic and extrinsic parameter matrices to perform rasterization volume rendering and synchronously project and output multi-view continuous video streams and 3D annotations. The self-supervised closed-loop evolution module is used to back-calculate the actual visual information capacity through the inverse risk assessor and update the system parameters based on the inverse gradient of the spatiotemporal gap loss to achieve adaptive resource allocation evolution.

10. A computer device comprising a storage medium resource pool and a processor array with deep learning acceleration cores, characterized in that: The processor array is configured to execute the method of any one of claims 1 to 8, which, during execution, directly performs risk information field analysis, SPH particle neighborhood search, transport equation discretization operation, 4D Gaussian attribute lifecycle control, and multi-view differentiable rasterization parallel rendering on tensors in the memory pool, and outputs high-fidelity panoramic video stream data for autonomous driving.

Images