Three-dimensional rendering method and electronic device

Abstract

The application discloses a three-dimensional rendering method and an electronic device, and relates to the technical field of data processing; comprising: in the iterative optimization process of the three-dimensional Gaussian splatting model of the dynamic scene, the resampling operation is adaptively triggered based on the scene cumulative transparency change amount, and the new Gaussian particle is initialized by using the optimizer state parameter of the stable Gaussian particle. Specifically, the resampling is triggered when the scene cumulative transparency change amount reaches a first preset threshold, ensuring that the resampling opportunity is highly matched with the actual dynamic characteristics of the scene; meanwhile, by inheriting the optimizer state parameter of the stable Gaussian particle, the new Gaussian particle directly inherits the local area gradient statistical prior information. The application effectively solves the training shock and interframe rendering fault caused by frequent resampling in the dynamic scene, and improves the convergence speed of the three-dimensional reconstruction of the complex dynamic scene and the space-time continuity of the rendering picture.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a three-dimensional rendering method and electronic device. Background Technology

[0002] Currently, in the field of 3D reconstruction and real-time rendering, 3D Gaussian Splatting (3DGS) technology is widely used due to its efficient explicit expression and high-quality rendering capabilities.

[0003] When constructing a 3D Gaussian splash model for a dynamic scene with continuous non-rigid deformation, existing solutions update Gaussian particles in batches according to a fixed time sequence. For new Gaussian particles generated during the update process, parameter configuration is completed by initially setting them to zero. The 3D Gaussian splash model obtained by the existing solution has poor operational stability, and the rendered image is prone to defects such as insufficient continuity and distortion, making it difficult to meet the requirements of dynamic scenes.

[0004] Therefore, there is an urgent need for a 3D rendering method suitable for non-rigid deformation to improve the stability of 3D Gaussian splash models. Summary of the Invention

[0005] In view of the above problems, this application provides a 3D rendering method and electronic device to improve the stability of 3D Gaussian splash models. The specific solution is as follows:

[0006] The first aspect of this application provides a 3D rendering method, including:

[0007] During the iterative optimization of the 3D Gaussian splash model in a dynamic scene, the cumulative change in scene transparency of the rendered image under the current rendering view is obtained. The dynamic scene includes a non-rigid deformation scene that changes continuously over time. The cumulative change in scene transparency represents the cumulative change in shape of the dynamic scene over time.

[0008] If the cumulative change in transparency of the scene is greater than or equal to the first preset threshold, a resampling operation is performed on each Gaussian particle in the Gaussian particle set of the three-dimensional Gaussian splash model to generate new Gaussian particles.

[0009] From the Gaussian particles associated with the new Gaussian particle, obtain the optimizer state parameters of the stable Gaussian particle, wherein the stable Gaussian particle is a Gaussian particle whose parameter fluctuation range is less than or equal to a second preset threshold.

[0010] Set the initial state of the optimizer for the new Gaussian particle to the optimizer state parameters, and perform a gradient update operation on the new Gaussian particle to obtain the updated three-dimensional Gaussian splash model.

[0011] A visualization of the dynamic scene is generated based on the updated 3D Gaussian splash model.

[0012] In one possible implementation, the iterative optimization process includes at least one iteration cycle, and obtaining the cumulative scene transparency change of the rendered image under the current rendering viewpoint includes:

[0013] The average value of the scene cumulative transparency of each pixel in the rendered image corresponding to the at least one iteration cycle is obtained. The scene cumulative transparency of the rendered image in each iteration cycle represents the shape structure and object occlusion degree of the dynamic scene in that iteration cycle.

[0014] Get the scene cumulative transparency of the rendered image from the current rendering viewpoint;

[0015] The difference between the average value and the cumulative transparency of the scene is determined as the change in cumulative transparency of the scene.

[0016] In one possible implementation, obtaining the optimizer state parameters of the stable Gaussian particle from the Gaussian particles associated with the new Gaussian particle includes:

[0017] Determine the neighborhood range centered on the coordinates of the new Gaussian particle, with a preset length as the radius;

[0018] Determine the particle type to which the new Gaussian particle belongs, wherein the particle type includes at least one of the following: rendering character, inherent attributes, and generation source;

[0019] From the set of Gaussian particles, select target Gaussian particles that are located within the neighborhood range and belong to the particle type;

[0020] From the plurality of target Gaussian particles, the stable Gaussian particle is selected;

[0021] The second-order moment parameter of the optimizer for the stable Gaussian particle is determined as the optimizer state parameter.

[0022] In one possible implementation, the iterative optimization process includes multiple iteration cycles. After obtaining the cumulative transparency change of the scene, and before performing a resampling operation on each Gaussian particle in the Gaussian particle set of the 3D Gaussian splash model, it further includes:

[0023] Obtain the coverage weight of each Gaussian particle in the Gaussian particle set. The coverage weight of the Gaussian particle represents the range of rendering pixels that the Gaussian particle can affect under the current rendering view.

[0024] The reconstruction error of each Gaussian particle in the Gaussian particle set is obtained. The reconstruction error of the Gaussian particle represents the pixel deviation between the image obtained after the Gaussian particle participates in the rendering and the standard ground truth image.

[0025] Based on the reconstruction error of each Gaussian particle and the coverage weight of each Gaussian particle, the basic importance value of each Gaussian particle is calculated.

[0026] Gaussian particles whose basic importance value is higher than or equal to a preset importance threshold are retained;

[0027] Obtain the temporal variance of the importance of the retained Gaussian particles in multiple iteration cycles, wherein the temporal variance of importance represents the variance of the basic importance values ​​of the retained Gaussian particles in multiple iteration cycles;

[0028] Gaussian particles whose importance time-series variance exceeds a preset variance threshold are identified as transient noise Gaussian particles.

[0029] Remove the transient noise Gaussian particles to obtain an updated set of Gaussian particles.

[0030] In one possible implementation, before performing a resampling operation on each Gaussian particle in the Gaussian particle set of the three-dimensional Gaussian splash model, the method further includes:

[0031] Obtain the projected area of ​​each Gaussian particle in the Gaussian particle set under the current rendering view;

[0032] Gaussian particles whose projected area exceeds a preset area threshold are identified as super-large Gaussian particles.

[0033] Establish the correlation between the super-large Gaussian particle and the constraint mask.

[0034] In one possible implementation, the resampling operation performed on each Gaussian particle in the Gaussian particle set of the three-dimensional Gaussian splash model to generate new Gaussian particles includes:

[0035] Splitting and cloning operations are prohibited on the Gaussian particles associated with the constraint mask.

[0036] The Gaussian particles that are not associated with the constraint mask are split and cloned to generate multiple Gaussian particles;

[0037] Pruning is performed on all the Gaussian particles to obtain pruned Gaussian particles.

[0038] In one possible implementation, performing the gradient update operation on the new Gaussian particle includes:

[0039] For all Gaussian particles after pruning, the rendering parameters of the Gaussian particles are iteratively updated based on their respective single parameter iteration update step size. The single parameter iteration update step size of Gaussian particles associated with constrained masks does not exceed a preset step size threshold.

[0040] In one possible implementation, after obtaining the cumulative transparency change of the scene, and before performing a resampling operation on each Gaussian particle in the Gaussian particle set of the 3D Gaussian splash model, the method further includes:

[0041] For each Gaussian particle, the fitting confidence of the Gaussian particle is calculated based on the horizontal and vertical gray-level gradients of the projected pixels of the Gaussian particle in the rendered image.

[0042] For each Gaussian particle, if the fitting confidence of the Gaussian particle is greater than or equal to a preset confidence threshold, the coefficient of the single parameter iteration update step size of the Gaussian particle is determined as the first learning rate.

[0043] For each Gaussian particle, if the fitting confidence of the Gaussian particle is less than the preset confidence threshold, the coefficient of the single parameter iteration update step size of the Gaussian particle is determined as the second learning rate.

[0044] Wherein, the first learning rate is greater than the second learning rate.

[0045] In one possible implementation, performing the gradient update operation on the new Gaussian particle includes:

[0046] For each Gaussian particle, the step size for a single parameter iteration update of the Gaussian particle is determined based on the learning rate of the Gaussian particle, wherein the learning rate is either the first learning rate or the second learning rate.

[0047] For each Gaussian particle, the rendering parameters of the Gaussian particle are iterated based on the step size of the single parameter iteration update corresponding to the Gaussian particle.

[0048] A second aspect of this application provides a three-dimensional rendering apparatus, comprising:

[0049] The change acquisition module is used to acquire the cumulative change in scene transparency of the rendered image under the current rendering view during the iterative optimization process of the three-dimensional Gaussian splash model of the dynamic scene. The dynamic scene includes a non-rigid deformation scene that continuously changes over time. The cumulative change in scene transparency represents the cumulative change in shape of the dynamic scene over time.

[0050] The resampling execution module is used to perform a resampling operation on each Gaussian particle in the Gaussian particle set of the three-dimensional Gaussian splash model if the cumulative change in the scene transparency is greater than or equal to a first preset threshold, so as to generate new Gaussian particles.

[0051] The state parameter acquisition module is used to acquire the optimizer state parameters of stable Gaussian particles from the Gaussian particles associated with the new Gaussian particles. The stable Gaussian particles are Gaussian particles whose own parameter fluctuation range is less than or equal to a second preset threshold.

[0052] The parameter update module is used to set the initial state of the optimizer for the new Gaussian particle to the optimizer state parameters, and to perform a gradient update operation on the new Gaussian particle to obtain the updated three-dimensional Gaussian splash model.

[0053] The rendering output module is used to generate a visualization of the dynamic scene based on the updated 3D Gaussian splash model.

[0054] A third aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the three-dimensional rendering method of the first aspect or any implementation thereof.

[0055] A fourth aspect of this application provides an electronic device, including at least one processor and a memory connected to the processor, wherein:

[0056] The memory is used to store computer programs;

[0057] The processor is used to execute the computer program so that the electronic device can implement the three-dimensional rendering method described in the first aspect or any implementation thereof.

[0058] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the three-dimensional rendering method described in the first aspect or any implementation thereof.

[0059] The sixth aspect of this application provides a vehicle including electronic equipment as described in the fourth aspect, or a three-dimensional rendering apparatus as described in the second aspect or any implementation thereof.

[0060] By employing the above technical solution, this application provides a 3D rendering method. During the iterative optimization of a 3D Gaussian splash model in a dynamic scene, a resampling operation is adaptively triggered based on the cumulative change in scene transparency, and new Gaussian particles are initialized using the optimizer state parameters of stable Gaussian particles. Specifically, the cumulative change in scene transparency characterizes the morphological changes of the dynamic scene over time. When this change reaches a first preset threshold, resampling is triggered, ensuring a high degree of matching between the resampling timing and the actual dynamic characteristics of the scene, avoiding fitting lag or invalid calculations caused by fixed-period resampling. Simultaneously, it abandons the conventional approach of directly initializing the optimizer state to zero for new Gaussian particles. By inheriting the optimizer state parameters of stable Gaussian particles, new Gaussian particles directly inherit prior information on local gradient statistics, avoiding cold-start oscillations in the early stages of parameter iteration. This linkage mechanism effectively solves the training oscillations and inter-frame rendering gaps caused by frequent resampling in dynamic scenes, improving the convergence speed of 3D reconstruction of complex dynamic scenes and the spatiotemporal continuity of the rendered image. Attached Figure Description

[0061] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0062] Figure 1 A schematic diagram of a system architecture is provided for this application;

[0063] Figure 2 A flowchart illustrating a three-dimensional rendering method provided in an embodiment of this application;

[0064] Figure 3 This is a schematic diagram of the structure of a three-dimensional rendering device provided in an embodiment of this application;

[0065] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0066] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0067] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0068] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0069] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0070] This application can be applied in the field of data processing. The following section will introduce several application scenarios that have been implemented in products, taking the non-rigid dynamic road surface rendering during vehicle movement as an example.

[0071] In practical applications such as autonomous driving simulation, real-time reconstruction of off-road conditions, and augmented reality navigation, the road surface where vehicles travel is often not a static, rigid plane, but a non-rigid, dynamic region containing mud, sand, gravel, and other materials that continuously deform as vehicles drive over them. The geometric topology and texture features of such scenes change drastically over time, placing extremely high demands on the real-time performance and stability of 3D rendering. Existing technologies typically employ a fixed-period resampling strategy to update the 3D Gaussian splash model. This approach cannot detect dynamic changes in the scene and easily leads to fitting lag in newly exposed areas. More importantly, existing technologies generally initialize the optimizer state parameters from zero when generating new Gaussian particles. Since dynamic scenes require high-frequency resampling to adapt to deformation, a large number of newly added Gaussian particles experience severe parameter oscillations in the early stages of iteration due to a lack of historical gradient information, resulting in frame discontinuities and flickering artifacts in the rendered image.

[0072] To address the aforementioned technical problems, this embodiment provides a three-dimensional rendering method and an electronic device, which will be described below.

[0073] See Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. The system may include an in-vehicle terminal 100 and a server 200. The server 200 can provide the methods provided in the embodiments of this application to one or more in-vehicle terminals.

[0074] The vehicle terminal 100 can be equipped with an application that has a 3D scene rendering function. The application can provide an interface on which the vehicle terminal 100 can receive relevant parameters input by the user, such as scene type, rendering precision, and rendering perspective, and send the parameters to the server 200. The server 200 can obtain the processing result based on the received parameters and return the processing result to the vehicle terminal 100.

[0075] It should be understood that in some optional implementations, the vehicle terminal 100 can also complete the action of obtaining the processing result based on the received parameters on its own, without the need for the server to cooperate. This application embodiment is not limited to this.

[0076] The following description Figure 1 The product form of the vehicle-mounted terminal 100;

[0077] The vehicle terminal 100 in this application embodiment can be a mobile phone, tablet computer, wearable device, vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., and this application embodiment does not impose any restrictions on it.

[0078] The vehicle-mounted terminal 100 may include a radio frequency unit, a memory, an input unit, a display unit, a camera (optional), an audio circuit (optional), a speaker (optional), a microphone (optional), a headphone jack (optional), a processor, an external interface, a power supply, and other components. Those skilled in the art will understand that the above-mentioned components are merely examples and do not constitute a limitation on the vehicle-mounted terminal or multi-functional device; it may include more or fewer components, or combine certain components, or use different components.

[0079] The input unit can be used to receive input numeric or character information, and to generate key signal inputs related to user settings and function control of the portable multi-functional device. Specifically, the input unit may include a touchscreen (optional) and / or other input devices. Other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0080] Among them, the input device can receive input data, etc.

[0081] The display unit can be used to display information input by the user or information provided to the user, various menus of the vehicle terminal, interactive interfaces, file display, and / or playback of any multimedia file. In the embodiments of this application, the display unit can be used to display the human-computer interaction interface for 3D scene rendering, the rendering configuration interface, and the final 3D visualization screen, processing results, etc.

[0082] The memory can be used to store software code related to the 3D rendering method, the processor can execute the steps of the 3D rendering method, and can also schedule other units (such as the above-mentioned input unit and display unit) to achieve the corresponding functions.

[0083] This radio frequency unit (optional) can be used to receive and send signals during information transmission or calls.

[0084] In this embodiment of the application, the radio frequency unit can send data to the server 200 and receive the processing results sent by the server 200.

[0085] It should be understood that this radio frequency unit is optional and can be replaced with other communication interfaces, such as a network port.

[0086] The vehicle terminal 100 also includes a power source (such as a battery) that supplies power to the various components.

[0087] The vehicle terminal 100 also includes an external interface, which can be a standard Micro USB interface or a multi-pin connector. It can be used to connect the vehicle terminal 100 to other devices for communication, or to connect a charger to charge the vehicle terminal 100.

[0088] Server 200 includes a bus, a processor, a communication interface, and memory. The processor, memory, and communication interface communicate with each other via the bus.

[0089] The memory can be used to store software code related to the 3D rendering method, the processor can execute the steps of the chip's 3D rendering method, and can also schedule other units to achieve the corresponding functions.

[0090] Reference Figure 2 , Figure 2 This is a flowchart illustrating a three-dimensional rendering method provided in an embodiment of this application, as shown below. Figure 2 As shown in the embodiment of this application, a three-dimensional rendering method may include steps S201 to S205, which are described in detail below.

[0091] Step S201: During the iterative optimization of the 3D Gaussian splash model in the dynamic scene, obtain the cumulative change in scene transparency of the rendered image under the current rendering view.

[0092] Dynamic scenes include non-rigid deformation scenes that continuously change over time. Examples of changes that occur over time include, but are not limited to, changes in scene geometry, texture, and occlusion relationships.

[0093] For example, dynamic scenarios can be off-road non-rigid road scenarios such as muddy roads, rut deformation, mud churning, mud splashing, and dynamic water pits during vehicle travel.

[0094] The 3D Gaussian splash model is an explicit model that uses a large number of Gaussian particles as basic expression units to realize 3D reconstruction and real-time rendering of scenes. Each Gaussian particle carries rendering parameters such as position, scale, rotation, transparency, and color.

[0095] In this embodiment, the iterative optimization process includes one or more iteration cycles. The iterative optimization process refers to the continuous updating of the parameters of each Gaussian particle in the three-dimensional Gaussian splash model in order to minimize the reconstruction error.

[0096] The rendering perspective refers to the spatial position, orientation, and field of view of a virtual camera in a 3D scene, which determines the viewing angle of the final rendered image. For example, the rendering perspective can be a vehicle-mounted front view, a side view, or a high-altitude overhead view.

[0097] The cumulative change in scene transparency represents the amount of morphological change that a dynamic scene undergoes over time.

[0098] The cumulative scene transparency of each pixel refers to the sum of the transparency of all superimposed Gaussian particles at that pixel location in the rendered image along the current rendering viewpoint. It is used to characterize the overall coverage of Gaussian particles at that pixel location and their relationship with dynamic scene occlusion.

[0099] The cumulative transparency change is a metric used to quantify the dynamics of a scene. Physically, it means that when the geometry of a non-rigid deformation scene (such as a mud pit caused by a car wheel) changes, previously occluded background areas may be exposed, or cracks may appear on previously continuous surfaces. These changes cause a significant alteration in the cumulative transparency of light passing through the dynamic scene. For example, when a vehicle drives over a muddy road, splashing mud and exposing the underlying soil, the cumulative transparency of the scene from the current rendering viewpoint will drop or rise sharply relative to historical cumulative transparency. This difference reflects the drastic degree of change in the dynamic scene's shape. By monitoring this change, the dynamic evolution of non-rigid regions can be accurately perceived, rather than relying on a fixed time step, thus providing objective data support for subsequent adaptive updates.

[0100] Step S202: If the cumulative change in scene transparency is greater than or equal to the first preset threshold, perform a resampling operation on each Gaussian particle in the Gaussian particle set of the 3D Gaussian splash model to generate new Gaussian particles.

[0101] The Gaussian particle set refers to the entire set of three-dimensional Gaussian particles that make up the whole three-dimensional Gaussian splash model. It is the basic unit set for scene representation.

[0102] Gaussian particles are the basic rendering unit of 3D Gaussian splashing technology, and have optimizable parameters such as position, scale, rotation, transparency, and spherical harmonic color.

[0103] The first preset threshold is an empirical value set based on the balance between rendering quality requirements and computing resources, used to distinguish between normal rendering noise and substantial scene deformation.

[0104] Resampling refers to the splitting, cloning, or pruning of Gaussian particles based on gradient information or error distribution during the topology adjustment process of a 3D Gaussian splash model.

[0105] Specifically, when the cumulative change in scene transparency exceeds a first preset threshold, it indicates that new geometric details or dynamic regions have appeared within the current view frustum that cannot be effectively covered by the original Gaussian particles. In this case, a resampling operation is immediately triggered. For example, when a vehicle rapidly passes through a flooded section of road, the ripples on the water surface cause drastic fluctuations in transparency. If the cumulative change in scene transparency is greater than or equal to the first preset threshold, a resampling operation is performed on each Gaussian particle in the Gaussian particle set of the 3D Gaussian splash model to generate new Gaussian particles. These new Gaussian particles are then split off in high-gradient regions to fill in the detail gaps.

[0106] The above-mentioned mechanism of triggering resampling operations on demand avoids invalid resampling calculations in static regions, and also prevents fitting delays caused by waiting for a fixed period in dynamically changing regions, thus achieving a match between computing resources and rendering timeliness.

[0107] Step S203: Obtain the optimizer state parameters of stable Gaussian particles from the Gaussian particles associated with the new Gaussian particles. Stable Gaussian particles are Gaussian particles whose parameter fluctuation range is less than or equal to the second preset threshold.

[0108] In traditional 3D Gaussian splashing algorithms, newly generated Gaussian particles are typically treated as entirely new, independent entities, with their optimizer state parameters forcibly initialized to zero. However, in dynamic scenarios, frequent resampling can cause a large number of Gaussian particles with their optimizer state parameters forcibly initialized to zero to simultaneously enter the optimization process, easily triggering severe oscillations in the global gradient.

[0109] To address the aforementioned issues, this embodiment introduces a state inheritance mechanism. A stable Gaussian particle refers to a historical Gaussian particle that has fully converged in previous iterations and exhibits minimal fluctuations in its position and attribute parameters. Stable Gaussian particles represent relatively reliable structural priors in dynamic scenarios.

[0110] The second preset threshold is a pre-set critical value for parameter fluctuation, used to determine whether a single Gaussian particle is a stable Gaussian particle.

[0111] The parameter fluctuation range of a Gaussian particle refers to the range of numerical changes in the core parameters such as position, transparency, and size of the Gaussian particle over multiple consecutive iteration cycles.

[0112] Optimizer state parameters refer to the internal cache parameters generated during the operation of gradient descent optimizers. They mainly include gradient accumulation information such as first moment and second moment, and record the gradient characteristics of Gaussian particles in historical iterations.

[0113] The optimizer state parameters can refer to internal variables used to adaptively adjust the learning rate. In this embodiment, the optimizer second moment parameters can be selected because they characterize the variance or local curvature information of the gradient, and are more reflective of the complexity of the dynamic scene than the optimizer first and second moment parameters.

[0114] Step S204: Set the initial state of the optimizer for the new Gaussian particle to the optimizer state parameters, and perform a gradient update operation on the new Gaussian particle to obtain the updated three-dimensional Gaussian splash model.

[0115] The optimizer initial state refers to the initial cache parameter state of the optimizer when the Gaussian particle is just generated.

[0116] The gradient update operation is the process of calculating the parameter gradient based on the rendering loss, combining the optimizer state parameters and the step size of the single parameter iteration update, and correcting the various rendering parameters of the Gaussian particle.

[0117] For example, rendering parameters include, but are not limited to: 3D coordinates, size parameters, rotation parameters, opacity, and spherical harmonics.

[0118] By directly assigning the inherited optimizer state parameters to the new Gaussian particles, the new Gaussian particles can be updated with an adaptive step size that matches the characteristics of the current scene during the first iteration, thus completely skipping the cold start oscillation phase caused by accumulating gradient statistics from scratch in traditional methods. For example, in high-curvature regions such as rut ​​edges, the inherited optimizer state parameters are larger, allowing the new Gaussian particles to make fine adjustments with smaller step sizes and avoid overshoot; while in flat mud regions, the inherited optimizer state parameters are smaller, allowing the new Gaussian particles to move into place quickly. This linkage mechanism ensures that even under high-frequency resampling conditions, the newly added Gaussian particles can be smoothly integrated into the 3D Gaussian splash model, fundamentally eliminating rendering artifacts caused by parameter abrupt changes.

[0119] For example, for a newly generated Gaussian particle due to mud churn, the optimizer state parameters of its associated stable Gaussian particle are extracted and used as the initial optimizer state for the new Gaussian particle. This approach is equivalent to allowing the new Gaussian particle to inherit the stable Gaussian particle's understanding of the local terrain complexity, giving it the ability to adapt to the current scene's gradient from the very beginning.

[0120] Step S205: Generate a visualization of the dynamic scene based on the updated 3D Gaussian splash model.

[0121] Visualized images refer to two-dimensional images generated by projecting a three-dimensional Gaussian splash model, such as the scene rendering output of an in-vehicle terminal that can be viewed by the user. For example, a display in a vehicle can show a simulated image of the vehicle, muddy road surface, tire tracks, and mud splash effects in real time.

[0122] For example, an updated set of Gaussian particles can be projected onto a two-dimensional screen space using a rasterization pipeline to synthesize a visual image, which can be an RGB (Red, Green, Blue) image.

[0123] Since steps S201 to S204 ensure the spatiotemporal continuity of the topology and parameters of the three-dimensional Gaussian splash model, the generated visualization can smoothly show the dynamic evolution process of the non-rigid road surface without obvious flickering or breakage.

[0124] This application's embodiments, on the one hand, utilize the cumulative transparency change of the scene as a trigger for dynamic perception, solving the fitting lag problem caused by fixed-period resampling; on the other hand, by inheriting the optimizer state parameters of stable Gaussian particles, it breaks the conventional practice of zero initialization of newly added Gaussian particles, effectively suppressing training oscillations caused by frequent resampling. The synergistic effect of these two aspects significantly improves the convergence speed of 3D reconstruction and the spatiotemporal continuity of the rendered image in complex dynamic scenes, providing robust technical support for real-time rendering of non-rigid road surfaces during vehicle movement.

[0125] This application provides a 3D rendering method. During the iterative optimization of a 3D Gaussian splash model in a dynamic scene, a resampling operation is adaptively triggered based on the cumulative change in scene transparency, and new Gaussian particles are initialized using the optimizer state parameters of stable Gaussian particles. Specifically, the cumulative change in scene transparency characterizes the morphological changes of the dynamic scene over time. When this change reaches a first preset threshold, resampling is triggered, ensuring a high degree of matching between the resampling timing and the actual dynamic characteristics of the scene, avoiding fitting lag or invalid calculations caused by fixed-period resampling. Simultaneously, it abandons the conventional approach of directly initializing the optimizer state to zero for new Gaussian particles. By inheriting the optimizer state parameters of stable Gaussian particles, new Gaussian particles directly inherit prior information on local gradient statistics, avoiding cold-start oscillations in the early stages of parameter iteration. This linkage mechanism effectively solves the training oscillations and inter-frame rendering gaps caused by frequent resampling in dynamic scenes, improving the convergence speed of 3D reconstruction of complex dynamic scenes and the spatiotemporal continuity of the rendered image.

[0126] To more accurately quantify the degree of morphological change in dynamic scenes and avoid false triggering of resampling due to subjective judgment or a single fixed threshold, this embodiment specifically defines the calculation method for the cumulative change in scene transparency. The iterative optimization process of this method includes at least one iteration cycle, and the above step S201 includes steps A1 to A3.

[0127] Step A1: Obtain the average value of the scene cumulative transparency of each pixel in the rendered image corresponding to at least one iteration cycle.

[0128] The cumulative scene transparency of the rendered image in the iteration cycle represents the shape structure and object occlusion degree of the dynamic scene within that iteration cycle.

[0129] External shape refers to the geometric contours and spatial forms of objects and surfaces in a dynamic scene. For example, the contours of ruts formed by vehicles or the raised shapes formed by mud accumulation.

[0130] Object occlusion refers to the degree to which a foreground object occludes or covers a background area in a dynamic scene. For example, splashed mud occludes the road surface behind the object, or a waterlogged area covers the ground.

[0131] The iteration cycle refers to the time unit for a complete round of parameter updates and rendering feedback for the 3D Gaussian splash model. For example, each time the 3D Gaussian splash model completes "rendering the image → calculating the error → updating the parameters of the Gaussian particles" is one iteration cycle.

[0132] A rendered image refers to a two-dimensional pixel image generated by the projection and superposition calculation of various Gaussian particles from the current rendering perspective.

[0133] A pixel is the smallest display unit that makes up a rendered image. Each pixel independently records information such as position, color, and transparency, and all pixels combine to present a complete scene.

[0134] Scene cumulative transparency refers to the transmittance value obtained by weighted accumulation of the opacities of all Gaussian particles along the line of sight during light projection. Its physical meaning directly characterizes the structural integrity of the dynamic scene and the degree of object occlusion from the current rendering perspective. For example, when the road surface is smooth and unobstructed, the scene cumulative transparency maintains a high steady-state value; however, when mud surges or the ground cracks, the medium distribution along the light path changes, and the scene cumulative transparency fluctuates. By statistically averaging the cumulative transparency of each scene within an iteration cycle, an adaptive baseline reflecting the steady-state characteristics of the dynamic scene can be established. This baseline updates over time and can effectively filter out interference from high-frequency rendering noise.

[0135] Step A2: Obtain the scene cumulative transparency of the rendered image from the current rendering viewpoint.

[0136] The current rendering perspective refers to the real-time spatial position, orientation, and field of view of the virtual camera at the time of this calculation; it is the perspective from which the 3D scene is observed at the moment.

[0137] For example, the scene cumulative transparency of the rendered image in the current rendering viewpoint refers to the cumulative value of the transparency corresponding to each pixel of the rendered image in the current rendering viewpoint.

[0138] The scene cumulative opacity of the rendered image from the current rendering perspective represents the instantaneous state of the dynamic scene at the latest moment. The current scene cumulative opacity is highly sensitive to sudden deformations in the dynamic scene. For example, in a vehicle-moving scene, the real-time scene cumulative opacity can characterize the abrupt changes in surface geometry caused by a vehicle's movement.

[0139] Step A3: Determine the difference between the average value and the cumulative scene transparency, which is the change in cumulative scene transparency.

[0140] Cumulative change in scene transparency The calculation formula is as follows: ,in, This is the average value. Accumulate transparency for the scene.

[0141] The cumulative change in scene transparency essentially constitutes a dynamic change discriminant factor, used to measure the degree to which the current scene (e.g., the road scene at the current time) deviates from the historical scene (the road scene 0.1 seconds ago). When the cumulative change in scene transparency is small, it indicates that the scene is in a relatively stable state or has only undergone minor perturbations; when the cumulative change in scene transparency increases significantly, it indicates that the scene has undergone a substantial change in topology.

[0142] The following example uses a dynamic scenario of a vehicle driving over a muddy road.

[0143] When a vehicle's tires drive over a section of road previously covered in loose mud, the mud is squeezed and peeled away, revealing a deeper, more textured, and harder layer of soil. In that instant, due to the removal of the surface obscuration and the exposure of the new, deeper area, the light penetration characteristics of the corresponding pixels in the current rendering viewpoint change drastically, causing a sharp drop in the current scene's cumulative transparency compared to the previous scene's cumulative transparency. At this point, the difference calculated in step A3 rapidly increases and exceeds a first preset threshold, precisely triggering a resampling operation. This guides the 3D Gaussian splash model to quickly generate Gaussian particles in the newly exposed area to fit the new terrain. Conversely, if the vehicle only travels at a constant speed on the reconstructed, flat road surface, the current scene's cumulative transparency remains roughly the same as the previous scene's, and the difference stays below the first preset threshold. In this case, no additional resampling is triggered, thus avoiding unnecessary computational overhead.

[0144] To further address the cold-start oscillation problem caused by the lack of prior information for new Gaussian particles, this embodiment specifically defines the process of extracting stable Gaussian particles from the associated particles of the new Gaussian particles and incorporating them into the optimizer state parameters. During dynamic scene 3D rendering, after resampling to generate new Gaussian particles, if their optimizer state is directly initialized to zero, the new Gaussian particles cannot effectively capture local gradient statistical features in the early stages of iteration, easily leading to drastic parameter fluctuations. To overcome this deficiency, step S203 specifically includes steps B1 to B5.

[0145] Step B1: Determine the neighborhood range centered on the coordinates of the new Gaussian particle and with a preset length as the radius.

[0146] The neighborhood range refers to the local search area defined in three-dimensional space, which is used to limit the spatial correlation boundary of parameter inheritance.

[0147] Coordinates refer to the spatial position coordinates of a new Gaussian particle in a three-dimensional scene coordinate system, including values ​​in the three dimensions of X, Y, and Z.

[0148] The preset length is a pre-set distance threshold. For example, the preset length can be dynamically set according to the average Gaussian particle density or the average scale of Gaussian particles in the dynamic scene. For instance, it can be set to a number of times the square root of the trace of the covariance matrix of the new Gaussian particle itself.

[0149] Understandably, while the overall topology of a dynamic scene changes over time, the geometry and gradient distribution of adjacent regions typically exhibit high continuity and similarity within a microscopic local scope. By restricting the search to the neighborhood, we can ensure that the inherited optimizer state parameters originate from samples sharing the same local terrain features as the new Gaussian particle, thus avoiding parameter interference from distant, irrelevant regions.

[0150] Step B2: Determine the particle type to which the new Gaussian particle belongs. The particle type includes at least one of the following: rendering character, inherent properties, and generation source.

[0151] Particle type is a classification identifier for the physical entity or semantic attribute represented by Gaussian particles.

[0152] Rendering roles refer to the functional roles that Gaussian particles play in dynamic scenes, such as road surfaces, splashing mud, vegetation, or rigid obstacles.

[0153] Inherent properties refer to the material characteristics of Gaussian particles, such as diffuse color, specular reflection coefficient, or opacity range.

[0154] The generation source records whether the Gaussian particles were generated through splitting, cloning, or initialization.

[0155] For example, in a dynamic scene where a vehicle is rolling over a muddy road, it is necessary to clearly distinguish between Gaussian particles representing "flowing mud" and Gaussian particles representing "stationary rocks".

[0156] This step ensures that parameter inheritance only occurs between semantically consistent Gaussian particles of the same type, preventing newly created mud particles from incorrectly inheriting the optimizer state parameters of adjacent stones, thereby avoiding rendering artifacts caused by material or motion mode mismatch.

[0157] Step B3: Select target Gaussian particles from the Gaussian particle set that are located in the neighborhood and belong to the particle type.

[0158] Specifically, the Gaussian particle set is traversed. First, Gaussian particles outside their neighborhood are removed. Then, from the remaining Gaussian particles, Gaussian particles whose particle type is inconsistent with the new Gaussian particle are further removed. For example, for a newly generated "wet soil" type Gaussian particle due to wheel rut compression, historical Gaussian particles within a certain radius that are also labeled "wet soil" are retained as a candidate set.

[0159] Step B4: Select stable Gaussian particles from multiple target Gaussian particles.

[0160] Among them, stable Gaussian particles refer to particles whose parameters converge and fluctuate very little during historical iterations, representing reliable structural information that has been fully learned in dynamic scenarios.

[0161] The criterion for determining whether a Gaussian particle is a stable Gaussian particle is that its parameter fluctuation range is less than or equal to a second preset threshold. Specifically, the parameter fluctuation range can be quantified by calculating the standard deviation or maximum change of key attributes such as position, scaling, or opacity of the Gaussian particle over the most recent several iterations.

[0162] The second preset threshold is a pre-set numerical criterion used to quantitatively determine whether the parameter fluctuations of the Gaussian particle are within a stable range.

[0163] The second preset threshold is an empirical value used to distinguish between "converged state" and "adjusted state". For example, if the root mean square error of the position offset of a Gaussian particle in the past 10 iterations is less than 0.001 meters and the rate of change of opacity is less than 1%, it is determined to be a stable Gaussian particle.

[0164] This step effectively prevents the transmission of erroneous gradient information from noise particles that have not yet converged to new Gaussian particles, ensuring the robustness of the inheritance process.

[0165] Step B5: Determine the second-order moment parameters of the optimizer for the stable Gaussian particle as the optimizer state parameters.

[0166] In this embodiment, the second-order moment parameter of the optimizer can be selected as the core object of inheritance, rather than the first-order moment parameter or other state variables.

[0167] The second moment estimate of an optimizer refers to the accumulated second-moment statistics obtained during the iteration process of a gradient descent-based optimizer (such as Adam). It belongs to the core internal state data of the optimizer and records the weighted cumulative result of historical gradient squares. The second moment estimate characterizes the variance of the gradient or the curvature information of the local loss function, reflecting the ease and uncertainty of updating parameters at that location.

[0168] In contrast, the first-order moment parameter primarily characterizes the direction of the gradient, exhibiting strong instantaneity and directionality. Direct inheritance can easily lead to new Gaussian particles accelerating along the incorrect old direction. However, in dynamic scenarios, the complexity of local terrain (such as the roughness of mud surfaces and the steepness of rut edges) is relatively stable in a short period of time, and the second-order moment parameter precisely encodes this local geometric complexity characteristic.

[0169] For example, in high-curvature regions such as rut ​​edges, the second-order moment parameter of stable Gaussian particles is relatively large, indicating that the optimizer should use a smaller step size to prevent overshoot; while in flat mud regions, the second-order moment parameter is smaller, allowing for larger step sizes and rapid convergence. By inheriting the second-order moment parameter, new Gaussian particles gain an understanding of the local terrain complexity at the moment of their creation, enabling them to participate in updates with an appropriate adaptive step size.

[0170] It should be understood that although the above steps describe the logic of obtaining parameters from a single stable Gaussian particle, in practical applications, the second-order moment parameters can also be obtained from multiple stable Gaussian particles that meet the conditions and then weighted and averaged. The weights can be inversely proportional to the spatial distance or stability index to obtain a more representative local prior estimate.

[0171] To further enhance the robustness of dynamic scene rendering, this embodiment adds a pre-cleaning step to the high-frequency adaptive resampling operation. If all rendered Gaussian particles are directly included in the update scope, short-term invalid dynamic noise such as mud splashes and transient reflections are easily misjudged as valid geometric structures. After splitting and cloning, a large number of invalid Gaussian particles are generated, contaminating the 3D Gaussian splash model data, thus causing the rendered image to flicker and produce artifacts. Therefore, before performing the resampling operation, this embodiment performs pre-cleaning on the Gaussian particle set through dual constraints of spatial weights and temporal variance to remove transient noise particles, ensuring that subsequent resampling only applies to the real scene deformation areas. The iterative optimization process of this method includes multiple iteration cycles. After step S201 and before step S202, it also specifically includes the following steps C1 to C7.

[0172] Step C1: Obtain the coverage weight of each Gaussian particle in the Gaussian particle set.

[0173] The coverage weight of a Gaussian particle represents the range of rendered pixels that the Gaussian particle can affect from the current rendering viewpoint.

[0174] The coverage weight reflects the spatial contribution of Gaussian particles to the final image.

[0175] In dynamic scenes, an effective Gaussian particle should typically cover a certain number of consecutive pixel areas. For example, Gaussian particles representing solid road surfaces often have a large projected area and high cumulative transparency, resulting in a higher coverage weight; while tiny Gaussian particles representing scattered mud in the distance, due to their greater distance from the camera or their extremely small size, cover very few pixels, resulting in a correspondingly lower coverage weight. This metric can be used to initially filter out stray noise that contributes almost no substantial contribution to the rendered image.

[0176] Step C2: Obtain the reconstruction error of each Gaussian particle in the Gaussian particle set.

[0177] The reconstruction error of Gaussian particles characterizes the pixel deviation between the image obtained after rendering with Gaussian particles and the standard ground truth image.

[0178] Reconstruction error is a core photometric metric for measuring the accuracy of Gaussian particle rendering. Specifically, it calculates the difference between the color value rendered by the current Gaussian particles and the corresponding pixel value in the acquired real image. For real-world static or quasi-static terrain, the reconstruction error should converge after several iterations; however, for randomly generated transient noise, due to its inconsistency with the real scene in terms of location and color, it is often accompanied by persistently high reconstruction errors.

[0179] This step provides a quantitative basis for distinguishing between non-convergent, effective structures and inherently flawed noise.

[0180] Step C3: Calculate the basic importance value of each Gaussian particle based on the reconstruction error and coverage weight of each Gaussian particle.

[0181] The basic importance value is a comprehensive score in the spatial dimension, used to characterize the confidence level of Gaussian particles as valid scene elements.

[0182] In this embodiment, the calculation logic for the basic importance value is designed to be positively correlated with the coverage weight and negatively correlated with the reconstruction error.

[0183] For example, a formula can be used. Calculate the fundamental importance value I of the Gaussian particle. base Where α is the coverage weight of the Gaussian particle, and L err This represents the reconstruction error of the Gaussian particle after normalization.

[0184] The above formula indicates that only Gaussian particles that occupy a significant position in space and closely fit the real image in terms of luminosity are assigned a high fundamental importance value. This design avoids misjudgments that may be caused by relying solely on errors (such as particles with low errors but no importance in the background), and also avoids the blurring of large Gaussian particles that may be introduced by relying solely on weights.

[0185] Step C4: Retain Gaussian particles whose basic importance value is higher than or equal to the preset importance threshold.

[0186] This step completes the first stage of coarse screening.

[0187] The preset importance threshold is a pre-defined numerical criterion used to distinguish between high-contribution effective particles and low-contribution particles.

[0188] For example, the preset importance threshold can be dynamically adjusted based on the complexity of the dynamic scene or set to a fixed empirical value. Gaussian particles below this threshold are considered low-mass redundancy or initial noise and are directly marked as objects to be pruned, thus no longer entering the subsequent temporal analysis process, thereby significantly reducing computational overhead. The retained Gaussian particles are considered as potentially valid components of the scene, and their temporal stability needs to be further verified.

[0189] Step C5: Obtain the temporal variance of the importance of the retained Gaussian particles in multiple iteration cycles. The temporal variance of importance represents the variance of the basic importance values ​​of the retained Gaussian particles in multiple iteration cycles.

[0190] Importance time series variance characterizes the degree of fluctuation of Gaussian particle properties from a time dimension.

[0191] It should be noted that, to ensure the rigor of the mathematical expression and the accuracy of the physical meaning, this embodiment uses the standard statistical variance formula to calculate the time-series variance of importance, that is: through the formula... Calculate the time series variance of importance.

[0192] in, The importance time series variance is given by N, where N is the total number of iterations, M is the total number of retained Gaussian particles, and I is the total number of Gaussian particles. i Let j be the fundamental importance value of the j-th Gaussian particle in the i-th iteration period. It represents the average value of the fundamental importance of each Gaussian particle in the i-th iteration period.

[0193] The above formula precisely quantifies the time stability of Gaussian particles. In contrast, using non-standard exponential weighting or other modified formulas may lead to distortion of sensitivity to short-term abrupt changes.

[0194] In the dynamic scenes of this application, the basic importance values ​​of real surface structures (such as tire tracks and rocks) are relatively stable across consecutive frames, with small temporal variances. However, transient phenomena such as mud splashes and water droplets appear only within a very short period of a few frames, and their basic importance values ​​undergo drastic jumps from zero to zero and back within a short time, resulting in significantly larger calculated temporal variances of importance. Therefore, the temporal variance of importance serves as a standard for distinguishing between steady-state structures and transient noise.

[0195] Step C6: Identify Gaussian particles whose importance time-series variance exceeds a preset variance threshold as transient noise Gaussian particles.

[0196] The preset variance threshold is a pre-defined fluctuation threshold used to distinguish between stable Gaussian particles and noise particles with fluctuation anomalies.

[0197] Transient noise Gaussian particles refer to Gaussian particles generated by short-term dynamic phenomena such as mud splashes, instantaneous puddle reflections, and temporary surface disturbances. They exist only briefly and are not part of the inherent geometric structure of the dynamic scene.

[0198] Step C7: Remove transient noise Gaussian particles to obtain an updated set of Gaussian particles.

[0199] This step constitutes the second stage of fine screening, ensuring that the final retained set of Gaussian particles is not only of high quality in space, but also consistent in time.

[0200] This embodiment, as a pre-processing purification method for adaptive resampling, effectively prevents dynamic artifacts amplified by high-frequency resampling. Especially when vehicles are driving on muddy roads, this embodiment can identify and remove transient noise Gaussian particles generated by mud splashes, water mist kicked up by tires, etc., avoiding the incorrect splitting and inheritance of these invalid particles. This ensures the purity of subsequent optimizer state inheritance and gradient updates, significantly improving the visual fidelity and spatiotemporal coherence of dynamic scene 3D rendering.

[0201] To further suppress cross-regional ultra-large Gaussian particles generated by local gradient anomalies and fitting strategy imbalances in dynamic scenes, and to ensure the integrity of local details and the clarity of the rendered image, this embodiment proposes an ultra-large Gaussian particle recognition and labeling mechanism based on mask constraints. During the iterative optimization of the 3D Gaussian splash model, in non-rigid dynamic areas such as mud splashes and water reflections, some Gaussian particles may excessively expand in scale to reduce reconstruction errors, forming ultra-large Gaussian particles whose size exceeds the normal range. These Gaussian particles tend to spread across multiple semantic regions, obscuring details such as tire tracks and stone edges, thus causing local blurring, light spot artifacts, and even distortion of the scene's topology.

[0202] If these Gaussian particles are directly deleted during the resampling stage, potentially valuable geometric information may be lost; without constraints, the rendering effect of the entire 3D Gaussian splash model will continue to be affected. Therefore, this embodiment identifies abnormal Gaussian particles and establishes mask relationships before the resampling operation, completing particle labeling to provide a basis for subsequent differentiated control. Specifically, before step S202 of the resampling operation, steps D1 to D3 are also included.

[0203] Step D1: Obtain the projected area of ​​each Gaussian particle in the Gaussian particle set under the current rendering view.

[0204] The projected area refers to the effective pixel range covered by the orthogonal or perspective projection of the three-dimensional Gaussian ellipsoid represented by a single Gaussian particle onto the current camera imaging plane.

[0205] In a practical implementation, the equivalent elliptical area of ​​the Gaussian particle in the two-dimensional screen space can be obtained by calculating the transformation of the covariance matrix of the Gaussian particle in the current view direction. This equivalent elliptical area is the projected area. Alternatively, the number of pixels that actually contribute non-zero transparency values ​​during the rasterization process can be directly counted as an approximation of the projected area.

[0206] The projected area directly reflects the spatial extent to which a Gaussian particle occupies the rendered image, and is a core geometric indicator for measuring whether its scale is abnormal. For example, in a standard vehicle driving road reconstruction scene, the projected area of ​​most Gaussian particles used to express road texture is usually concentrated between several and tens of pixels, while the Gaussian particles used to express distant backgrounds or large flat areas may be larger, but their shape and position usually conform to the scene priors.

[0207] Step D2: Determine that Gaussian particles whose projected area exceeds the preset area threshold are super-large Gaussian particles.

[0208] The preset area threshold is the boundary used to distinguish between normal-scale Gaussian particles and abnormal-scale Gaussian particles.

[0209] For example, the preset area threshold can be dynamically set according to the average Gaussian particle density of the dynamic scene, the camera resolution, or the statistical distribution of the Gaussian particle area in historical iterations. For example, it can be set to a number of times the median of the projected area of ​​all Gaussian particles in the Gaussian particle set, or set to a fixed pixel value (such as 100 pixels).

[0210] When the projected area of ​​a Gaussian particle exceeds a preset area threshold, it is identified as an ultra-large Gaussian particle. The physical significance of this determination is that in dynamic scenes, realistic geometric details (such as the edges of mud puddles or tire tracks) typically require medium-sized Gaussian particles for precise representation. Only when the optimization process gets stuck in a local minimum or attempts to forcibly fit complex textures with a single Gaussian particle will an ultra-large Gaussian particle be generated. For example, when a vehicle drives through a puddle, a 3D Gaussian splash model might attempt to cover the entire reflective area of ​​the water surface with a single, extremely large, flat Gaussian particle, rather than using multiple small Gaussian particles to fit the subtle undulations of the water waves. In this case, the projected area of ​​the Gaussian particle will significantly exceed the preset area threshold and be identified as an ultra-large Gaussian particle.

[0211] Step D3: Establish the correlation between the super-large Gaussian particle and the constraint mask.

[0212] The constraint mask is a data structure used to identify the state of a super-large Gaussian particle. It can take the form of a Boolean array of flags corresponding one-to-one with the super-large Gaussian particle, an integer state code tensor, or an independent list of sparse masks.

[0213] Establishing a correlation means that after detecting a super-large Gaussian particle, the flag position at the corresponding location in the constraint mask is set to an effective state (such as True or 1), or marked as "constrained update" type in the status code.

[0214] It is important to note that the constraint mask is not simply a deletion list, but rather a control signal carrier that runs throughout the entire process of subsequent resampling and gradient update operations. It allows for the imposition of specific constraint rules on the ultra-large Gaussian particle without immediately removing it. For example, during the resampling phase, Gaussian particles associated with the constraint mask are prohibited from splitting and cloning to prevent further spread of their anomalous scale; during the gradient update phase, their parameter update step size is limited to a safe range, allowing for gradual decay or correction rather than violent oscillations.

[0215] To illustrate the working principle of the above mechanism more clearly, the following example uses a dynamic scenario of mud splashing during vehicle movement.

[0216] When a wheel spins at high speed, kicking up a cloud of mud, the mud cloud changes shape rapidly in the air and becomes blurred. When fitting this transient object, the 3D Gaussian splash model tends to generate a huge, flat Gaussian particle to cover the area and reduce instantaneous loss. In step D1, the projected area of ​​this Gaussian particle is calculated to cover hundreds of pixels, far exceeding the average level of surrounding normal mud particles. In step D2, this projected area exceeds a preset area threshold and is identified as an ultra-large Gaussian particle. In step D3, the index position of this Gaussian particle is marked as True in the constraint mask. Subsequently, even if the reconstruction error of this Gaussian particle is low in the current frame, it will not be allowed to participate in regular splitting and cloning operations, thus preventing this fake mud from being copied and enlarged in subsequent frames, ultimately causing the entire wheel area to be rendered as a blurry patch.

[0217] This embodiment provides a structured solution to the problem of abnormal scale in dynamic scenes through the aforementioned mechanism of identifying and associating ultra-large Gaussian particles based on projected area. This mechanism not only locates problematic Gaussian particles that may cause image blurring and topological distortion, but more importantly, it decouples the perception of abnormal states from the subsequent processing logic through a constraint mask as an intermediate carrier. This allows for the application of refined soft constraints on abnormal Gaussian particles while preserving potentially valid geometric information. This approach effectively avoids the void flickering that may result from directly deleting large Gaussian particles, or the detail swallowing caused by directly retaining large Gaussian particles, which are common in traditional methods. It significantly improves the visual fidelity and geometric accuracy of 3D rendering in dynamic scenes.

[0218] For example, if steps C1 to C7 and steps D1 to D3 are executed in the same embodiment, steps C1 to C7 are executed first to complete the noise particle purification, and then steps D1 to D3 are executed to identify and mark ultra-large Gaussian particles; the execution order of the two can also be interchanged, and this application does not limit it to a single one.

[0219] To implement differentiated and precise control over identified anomalous ultra-large Gaussian particles and prevent them from continuously degrading and contaminating the overall 3D Gaussian splash model during subsequent iterative optimization, this embodiment further refines the resampling and gradient update execution logic based on mask constraints. In the iterative optimization of the 3D Gaussian splash model, simply identifying and marking ultra-large Gaussian particles cannot fundamentally solve the rendering anomaly problem. If conventional splitting, cloning, and full-step gradient update strategies are used for these anomalous Gaussian particles, they will cause these particles to continuously devour effective detail information around the scene. Simultaneously, drastic gradient fluctuations will induce artifacts such as light spots and flickering in the rendered image, severely affecting the stability of dynamic scene rendering.

[0220] Based on this, this embodiment transforms the pre-constructed constraint mask into executable model control signals, establishing a complete technical closed loop of "abnormal Gaussian particle detection → constraint mask marking → differentiated handling," thereby achieving targeted control of abnormal Gaussian particles. The corresponding step S202, which performs the resampling operation, specifically includes steps E1 to E3.

[0221] Step E1: Disable splitting and cloning operations on Gaussian particles with associated constrained masks.

[0222] The splitting operation refers to the process of replacing a Gaussian particle with two or more new Gaussian particles that are smaller in scale and slightly offset in position when the position gradient or reconstruction error of a Gaussian particle exceeds a threshold, with the aim of increasing the density of the local geometric representation.

[0223] Cloning refers to copying a Gaussian particle and moving it to a neighboring area to fill holes or cover insufficient areas in a dynamic scene.

[0224] Splitting and cloning operations are the core mechanisms by which 3D Gaussian splash models adapt to the growth of detail in dynamic scenes. However, the problem with ultra-large Gaussian particles already marked as constraint masks lies in their excessive size rather than insufficient quantity. Allowing them to continue splitting would generate even more large-scale Gaussian particles, causing anomalous regions to spread exponentially; allowing them to clone would replicate this erroneous geometric representation to the surrounding normal regions. Therefore, this step, by reading the state bits of the constraint mask, forcibly blocks the proliferation permission of such Gaussian particles, thus blocking the propagation path of anomalous topology at its source.

[0225] Step E2: Perform splitting and cloning operations on the unassociated Gaussian particles with constrained masks to generate multiple Gaussian particles.

[0226] For Gaussian particles not marked by the mask, the splitting and cloning operations are still triggered according to the standard gradient criterion or error distribution to ensure that realistic geometric details such as rut ​​edges and stone textures are adequately fitted and updated. This differentiated processing strategy enables the 3D Gaussian splash model to maintain a sensitive response to effective dynamic features while suppressing anomalies.

[0227] Step E3: Prune all Gaussian particles to obtain pruned Gaussian particles.

[0228] Pruning is the process of removing Gaussian particles that have an opacity below a threshold, are too small, or contribute very little to the rendering, in order to maintain the compactness of the 3D Gaussian splash model.

[0229] In this embodiment, the pruning operation is performed globally, regardless of whether it is associated with a constraint mask. This means that even very large Gaussian particles will be normally removed if their opacity gradually decreases below a threshold during subsequent updates, or if they are completely occluded by other newly generated valid Gaussian particles. This design does not provide permanent retention for abnormal Gaussian particles; very large Gaussian particles that have not contributed for a long time can still be naturally removed through pruning operations, ensuring the dynamic updating of the Gaussian particle set.

[0230] Accordingly, a gradient update operation is performed on the new Gaussian particles, including: for all pruned Gaussian particles, the rendering parameters of the Gaussian particles are iteratively updated based on their respective single parameter iteration update step size, wherein the single parameter iteration update step size of the Gaussian particles associated with constrained masks does not exceed a preset step size threshold.

[0231] The step size of a single parameter iteration update refers to the maximum amount of change that the attribute parameters (such as position, covariance, spherical harmonic coefficients, etc.) of a Gaussian particle are allowed to make in one iteration cycle, which is usually determined by the product of the learning rate and the gradient value.

[0232] The preset step size threshold is a safe upper limit that is much smaller than the normal update step size. For example, it can be set to 10% of the normal average step size or fixed to a very small constant value.

[0233] For normal Gaussian particles without associated masks, the update step size is adaptively calculated by the optimizer based on the current gradient to ensure convergence efficiency; while for super-large Gaussian particles with associated constrained masks, the calculated theoretical update step size is truncated to within a preset step size threshold.

[0234] It's important to explain why we only restrict the update step size for ultra-large Gaussian particles instead of deleting them outright. In dynamic scenes, some seemingly anomalous large Gaussian particles may contain valuable low-frequency geometric or ambient lighting information. Directly and abruptly deleting them could cause sudden holes or brightness changes in local areas, resulting in a visual flickering effect. By restricting the update step size, we are essentially applying a strong damping effect to these Gaussian particles, causing their parameters to adjust in the correct direction or gradually decay at an extremely slow pace. This gradual release mechanism prevents them from undergoing destructive and drastic deformation within a single frame while preserving their potential value as temporary transitional elements, allowing them to smoothly exit the scene or be corrected to a reasonable size.

[0235] To illustrate the working principle of the above mechanism more clearly, the following example uses a scenario of mud splashing during vehicle movement.

[0236] When a car wheel spins at high speed, kicking up a cloud of mud, the 3D Gaussian splash model might generate a flat, super-large Gaussian particle covering hundreds of pixels to represent this blurry mud in the initial fitting stage. This super-large Gaussian particle is associated with a constraint mask. In step E1 of this embodiment, although the reconstruction error of this super-large Gaussian particle may still be high, the presence of the constraint mask prohibits its splitting and cloning operations, preventing this "fake mud" from splitting into more small mud particles that contaminate the entire wheel area. In step E4, even if the gradient indicates that the super-large Gaussian particle needs to move significantly to match the real mud position in the next frame, its actual displacement is limited to a very small range. As the vehicle passes, the real mud disperses and disappears. Because the super-large Gaussian particle cannot quickly keep up with the changes in the real scene and its update ability is limited, its opacity will naturally decrease in subsequent iterations, and it will eventually be pruned and removed in step E3. The entire process results in a natural transition in the image, without any abrupt color block jumps or loss of detail.

[0237] This embodiment transforms static anomaly detection results into dynamic execution control logic through the aforementioned constraint mask-based differentiated resampling and gradient update mechanism.

[0238] To further improve the imbalance between rigid and non-rigid regions in dynamic scenarios and eliminate the underfitting of terrain edge details and overfitting of flat mud areas caused by the unified optimization strategy, this embodiment proposes a differentiated learning rate allocation strategy based on Gaussian particle fitting confidence.

[0239] In dynamic non-rigid road surface rendering scenarios involving vehicle movement, the local features of the dynamic scene exhibit significant differentiation: rigid fixed structures such as rut ​​edges, exposed rocks, and terrain boundaries contain a large number of high-frequency textures, requiring rapid iteration and updates of the 3D Gaussian splash model to accurately reproduce geometric details; non-rigid areas such as smooth mud and stagnant water surfaces have weak texture features and undergo continuous minor deformations. If a high-sensitivity parameter update mechanism is used, dynamic noise such as mud splashes and instantaneous reflections on the water surface may be easily identified as effective texture fitting, causing the rendered image to jitter and generating various visual artifacts.

[0240] Based on the above problems, this embodiment adaptively adjusts the Gaussian particle optimization intensity according to the local pixel gradient confidence of the dynamic scene, i.e., the fitting confidence of the Gaussian particles, to balance the scene detail restoration accuracy and dynamic rendering stability. After step S201 of obtaining the cumulative transparency change of the scene and before step S202 of performing the resampling operation, steps F1 to F3 are also included.

[0241] Step F1: For each Gaussian particle, calculate the fitting confidence of the Gaussian particle based on the horizontal and vertical gray-level gradients of the projected pixels in the rendered image.

[0242] The horizontal grayscale gradient refers to the rate of change of pixel brightness in the horizontal direction (x-axis) of the rendered image, while the vertical grayscale gradient refers to the rate of change of pixel brightness in the vertical direction (y-axis) of the rendered image. These two gradient components together describe the local texture structure features of the area currently covered by the Gaussian particle.

[0243] The fitting confidence of Gaussian particles is a scalar indicator used to quantify the saliency of the geometric structure or the complexity of the texture in the region where Gaussian particles are located.

[0244] In this embodiment, the standard Euclidean norm formula can be used to calculate the fit confidence score, i.e.:

[0245] Where C is the fit confidence level and H is the image grayscale value. For horizontal grayscale gradient, This represents the vertical grayscale gradient.

[0246] It is important to note that this formula uses the square root of the sum of squares to ensure that the calculation results accurately reflect the magnitude of the image gradient. The fit confidence is positively correlated with texture complexity: when the gradient magnitude within the projection region of a Gaussian particle is large, it indicates that the region contains clear edges, corners, or complex textures, resulting in a high fit confidence and suggesting the presence of effective high-frequency information requiring fine reconstruction. Conversely, when the gradient magnitude is small, it indicates that the region is a flat surface with uniform color or a blurred dynamic transition area, resulting in a low fit confidence and suggesting that maintaining overall smoothness should be prioritized over excessive pursuit of local details.

[0247] Step F2: For each Gaussian particle, if the fitting confidence of the Gaussian particle is greater than or equal to the preset confidence threshold, the coefficient of the single parameter iteration update step size of the Gaussian particle is determined as the first learning rate.

[0248] The preset confidence threshold is a criterion used to distinguish between high-frequency detail areas and low-frequency smooth areas. It can be dynamically set according to the overall contrast distribution of the dynamic scene or a fixed empirical value can be taken.

[0249] The first learning rate is a relatively large numerical coefficient used to control the aggressiveness of parameter updates. When the fitting confidence of the Gaussian particle is higher than the preset confidence threshold, it is determined that it may be located in key rigid feature regions such as rut ​​contours, obstacle edges, etc. At this time, assigning a higher first learning rate allows the optimizer to have a larger step size during gradient descent, thereby accelerating the convergence speed of these key features. This ensures that the 3D Gaussian splash model can quickly capture and lock onto subtle geometric structures, avoiding edge blurring or loss of details due to slow update pace.

[0250] Step F3: For each Gaussian particle, if the fitting confidence of the Gaussian particle is less than the preset confidence threshold, determine the coefficient of the single parameter iteration update step size of the Gaussian particle as the second learning rate.

[0251] The first learning rate is greater than the second learning rate.

[0252] The second learning rate is a relatively small numerical coefficient designed to apply a damping effect to parameter updates. When the fitting confidence of a Gaussian particle is lower than a preset confidence threshold, it is determined that it may be located in non-rigid, weakly textured regions such as mud surfaces, puddles, or blurred distant areas. Photometric errors in such regions are often caused primarily by sensor noise, lighting fluctuations, or transient deformations, rather than actual geometric differences. Applying a lower second learning rate in this case is equivalent to adding a low-pass filter to the optimization process, forcing the 3D Gaussian splash model to adjust parameters in a conservative and slow manner. This approach effectively suppresses parameter oscillations caused by overfitting dynamic noise, ensuring that the rendering results of non-rigid regions remain visually smooth and consistent, preventing speckled artifacts or flickering.

[0253] To illustrate the working principle of the above strategy more clearly, the following example uses a typical scenario during vehicle movement.

[0254] Suppose the current rendering view includes both a clear rut edge and a wet, muddy flat area. For the Gaussian particle forming the rut edge, because its projected pixels cross the light-dark boundary, both the horizontal and vertical grayscale gradients are large. The fitting confidence calculated in step F1 is significantly higher than the preset confidence threshold, so it is assigned a first learning rate (e.g., 0.01) in step F2. This allows the Gaussian particle to quickly move to the accurate edge position in subsequent iterations, sharply outlining the rut shape. For the Gaussian particle forming the muddy surface, the grayscale change within its projection area is gradual, and the gradient magnitude is extremely small. The calculated fitting confidence is lower than the preset confidence threshold, so it is assigned a second learning rate (e.g., 0.001) in step F3. Even if the instantaneous reconstruction error increases due to water reflection, the Gaussian particle will only fine-tune its position and opacity by a very small amount, without jumping drastically with noise, thus ensuring a uniform and soft texture on the muddy surface.

[0255] To implement the differentiated learning strategy in the parameter iteration process of Gaussian particles, and to enable the optimization algorithm to adaptively adjust the parameter update magnitude based on scene texture features, this embodiment refines the gradient update execution process for new Gaussian particles.

[0256] In the iterative optimization and fine-tuning of the 3D Gaussian splash model, the learning rate is only a single scalar coefficient. It needs to be combined with the gradient information obtained from backpropagation to be converted into an update amount that can be used to correct the rendering parameters of Gaussian particles. To this end, this embodiment designs a standardized update step size solution and parameter iteration mechanism. Specifically, in step S204, a gradient update operation is performed on the new Gaussian particles, which includes steps G1 to G2.

[0257] Step G1: For each Gaussian particle, determine the step size for a single parameter iteration update based on the learning rate of the Gaussian particle. The learning rate is either the first learning rate or the second learning rate.

[0258] The step size of a single parameter iteration update refers to the actual numerical range by which a specific property parameter of the Gaussian particle is allowed to change in the current iteration cycle.

[0259] In practical implementations, the step size for a single parameter iteration update is usually determined by the product of the learning rate and the current gradient value, or when using advanced optimizers such as adaptive moment estimation, it is calculated by the learning rate, gradient, and the first-order and second-order moment parameters maintained internally by the optimizer.

[0260] For example, for a Gaussian particle assigned a first learning rate, its calculated single parameter iteration update step size is relatively large, meaning that the Gaussian particle is allowed to make large positional movements or attribute adjustments in this iteration cycle to quickly respond to significant geometric errors; while for a Gaussian particle assigned a second learning rate, its single parameter iteration update step size is limited to a smaller range, which is equivalent to applying damping to the changes in parameters, allowing it to make only minor corrections.

[0261] Step G2: For each Gaussian particle, iterate the rendering parameters of the Gaussian particle based on the step size of the single parameter iteration update.

[0262] Gaussian particles are the smallest rendering units in a 3D Gaussian splash model, and their geometric kernel is a Gaussian ellipsoid.

[0263] Rendering parameters refer to the set of all learnable properties that make up a 3D Gaussian splash model.

[0264] In this embodiment, the rendering parameters include at least the following four core elements: position coordinates, used to define the center point of the Gaussian ellipsoid in three-dimensional space; covariance matrix, used to define the shape, scale, and rotation attitude of the Gaussian ellipsoid, directly determining the geometric coverage of the Gaussian particles; opacity, used to control the degree of occlusion and transmission of light by the Gaussian particles, affecting the rendering effect of realism and blending; and spherical harmonic coefficients, used to characterize the anisotropic characteristics of the Gaussian particle color as the viewing angle changes, determining the realism of highlights and diffuse reflections. During iterative updates, the step size of a single parameter iteration determined in step G1 is applied to each of the above-mentioned rendering parameters. For example, for position coordinates, the update formula can be expressed as P... new =P old -η×▽ P L, where η is the step size for a single parameter iteration update, determined by the learning rate, ▽ P L represents the gradient of the position coordinates. It should be understood that although position coordinates are used as an example here, the update logic for covariance, opacity, and spherical harmonic coefficients is similar, all controlled by the corresponding step size of a single parameter iteration update.

[0265] To illustrate the above execution process more clearly, we will continue with the example of rendering a non-rigid dynamic road surface during vehicle movement.

[0266] Suppose that in the rendered image of the current frame, there exists both a clear rut edge region and a blurry, wet mud region. For the Gaussian particle located at the rut edge, because it is considered to have a high fitting confidence and is assigned a first learning rate (e.g., 0.01), the single parameter iteration update step size for the position coordinates calculated in step G1 is relatively large (e.g., 0.05 meters). During the update in step G2, the Gaussian particle can quickly move towards the geometric boundary of the actual rut, accurately fitting the edge contour in just a few iterations, effectively avoiding edge blurring or slow convergence caused by too small a single parameter iteration update step size. Meanwhile, for the Gaussian particle located in the wet mud region, because its fitting confidence is low and it is assigned a second learning rate (e.g., 0.001), the single parameter iteration update step size for the position coordinates calculated in step G1 is extremely small (e.g., 0.0005 meters). Even if the instantaneous gradient of the Gaussian particle is large due to water reflection or sensor noise, its actual displacement in step G2 is strictly limited to the micrometer level. This minute adjustment ensures that Gaussian particles do not fluctuate wildly with transient noise, but gradually converge to a stable position with the accumulation of multiple frames of data. This guarantees the smoothness and visual coherence of the mud area rendering results and eliminates common flickering artifacts.

[0267] In an optional implementation, the step S204 of “performing a gradient update operation on the new Gaussian particle” can be combined with steps G1 to G2 and “for all pruned Gaussian particles, performing iterative updates of the rendering parameters of the Gaussian particles based on their respective single parameter iteration update step size, wherein the single parameter iteration update step size of the Gaussian particles associated with the constrained mask does not exceed a preset step size threshold”, specifically including the following steps H1 to H4.

[0268] Step H1: Determine that the Gaussian particles in the associated constraint mask are super-large Gaussian particles, and the Gaussian particles in the unassociated constraint mask are ordinary Gaussian particles.

[0269] Step H2: Set the learning rate of ordinary Gaussian particles to the first learning rate and the learning rate of super-large Gaussian particles to the second learning rate.

[0270] Step H3: For each ordinary Gaussian particle, determine the single parameter iteration update step size based on the first learning rate of the ordinary Gaussian particle. For each super-large Gaussian particle, determine the single parameter iteration update step size based on the second learning rate of the super-large Gaussian particle and a preset step size threshold, wherein the single parameter iteration update step size of the super-large Gaussian particle does not exceed the preset step size threshold.

[0271] Step H4: Perform gradient iterative update on the rendering parameters of all Gaussian particles according to the step size of the single parameter iteration update for each Gaussian particle.

[0272] The above describes a three-dimensional rendering method provided by the embodiments of this application. The following will describe the apparatus for performing the above three-dimensional rendering method.

[0273] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a three-dimensional rendering device provided in an embodiment of this application. Figure 3 As shown, the 3D rendering device includes:

[0274] The change acquisition module 301 is used to acquire the cumulative change in scene transparency of the rendered image under the current rendering view during the iterative optimization process of the three-dimensional Gaussian splash model of the dynamic scene. The dynamic scene includes a non-rigid deformation scene that changes continuously over time. The cumulative change in scene transparency represents the cumulative change in shape of the dynamic scene over time.

[0275] The resampling execution module 302 is used to perform a resampling operation on each Gaussian particle in the Gaussian particle set of the three-dimensional Gaussian splash model if the cumulative change in scene transparency is greater than or equal to a first preset threshold, so as to generate new Gaussian particles.

[0276] The state parameter acquisition module 303 is used to acquire the optimizer state parameters of stable Gaussian particles from Gaussian particles associated with new Gaussian particles. The stable Gaussian particles are Gaussian particles whose own parameter fluctuation range is less than or equal to a second preset threshold.

[0277] The parameter update module 304 is used to set the initial state of the optimizer of the new Gaussian particle to the optimizer state parameters, and to perform a gradient update operation on the new Gaussian particle to obtain the updated three-dimensional Gaussian splash model.

[0278] The rendering output module 305 is used to generate a visualization of the dynamic scene based on the updated 3D Gaussian splash model.

[0279] In one optional implementation, the iterative optimization process includes at least one iteration cycle, and the change acquisition module includes:

[0280] The mean value unit is used to obtain the average value of the scene cumulative transparency of each pixel in the rendered image corresponding to the at least one iteration cycle. The scene cumulative transparency of the rendered image in each iteration cycle represents the shape structure and object occlusion degree of the dynamic scene in that iteration cycle.

[0281] Get transparency unit, used to obtain the scene cumulative transparency of the rendered image from the current rendering viewpoint;

[0282] A change unit is used to determine the difference between the average value and the cumulative transparency of the scene, which is the change in the cumulative transparency of the scene.

[0283] In one optional implementation, the state parameter acquisition module includes:

[0284] Define the range unit, which is used to define the neighborhood range centered on the coordinates of the new Gaussian particle and with a preset length as the radius;

[0285] A type determination unit is used to determine the particle type to which a new Gaussian particle belongs, wherein the particle type includes at least one of rendering role, inherent attributes, and generation source;

[0286] The first particle screening unit is used to screen out target Gaussian particles that are located in the neighborhood range and belong to the particle type from the Gaussian particle set.

[0287] The second particle screening unit is used to screen out the stable Gaussian particles from the plurality of target Gaussian particles;

[0288] A parameter determination unit is used to determine the second-order moment parameter of the optimizer for the stable Gaussian particle as the optimizer state parameter.

[0289] In one alternative implementation, the iterative optimization process includes multiple iteration cycles.

[0290] In one alternative implementation, it also includes:

[0291] The weight acquisition module is used to acquire the coverage weight of each Gaussian particle in the Gaussian particle set. The coverage weight of the Gaussian particle represents the range of rendering pixels that the Gaussian particle can affect under the current rendering view.

[0292] An error acquisition module is used to acquire the reconstruction error of each Gaussian particle in the Gaussian particle set. The reconstruction error of the Gaussian particle represents the pixel deviation value between the image obtained after the Gaussian particle participates in the rendering and the standard ground truth image.

[0293] The importance calculation module is used to calculate the basic importance value of each Gaussian particle based on the reconstruction error of each Gaussian particle and the coverage weight of each Gaussian particle.

[0294] A particle retention module is used to retain Gaussian particles whose basic importance value is higher than or equal to a preset importance threshold.

[0295] The variance acquisition module is used to acquire the temporal variance of the importance of the retained Gaussian particles in multiple iteration cycles. The temporal variance of importance represents the variance of the basic importance values ​​of the retained Gaussian particles in multiple iteration cycles.

[0296] The noise particle identification module is used to identify Gaussian particles whose importance temporal variance exceeds a preset variance threshold as transient noise Gaussian particles.

[0297] The noise particle removal module is used to remove the transient noise Gaussian particles to obtain an updated set of Gaussian particles.

[0298] In one alternative implementation, it also includes:

[0299] The area acquisition module is used to acquire the projected area of ​​each Gaussian particle in the Gaussian particle set under the current rendering view.

[0300] The super-large particle module is used to identify Gaussian particles whose projected area exceeds a preset area threshold as super-large Gaussian particles.

[0301] The relationship establishment module is used to establish the association between the super-large Gaussian particle and the constraint mask.

[0302] In one optional implementation, the resampling execution module includes:

[0303] A module to prevent splitting and cloning is provided to prevent splitting and cloning operations on the Gaussian particles associated with the constraint mask.

[0304] The splitting and cloning module is used to perform splitting and cloning operations on the Gaussian particles that are not associated with the constraint mask to generate multiple Gaussian particles.

[0305] The pruning module is used to prune all the Gaussian particles to obtain pruned Gaussian particles.

[0306] In one alternative implementation, the parameter update module includes:

[0307] The parameter update unit is used to perform iterative updates of the rendering parameters of all Gaussian particles after pruning, based on their respective single parameter iteration update step size. The single parameter iteration update step size of Gaussian particles associated with constrained masks does not exceed a preset step size threshold.

[0308] In one alternative implementation, it also includes:

[0309] The confidence calculation module is used to calculate the fitting confidence of each Gaussian particle based on the horizontal and vertical gray-level gradients of the projected pixels of the Gaussian particle in the rendered image.

[0310] The first learning rate determination module is used to determine the coefficient of the single parameter iteration update step size of the Gaussian particle as the first learning rate if the fitting confidence of the Gaussian particle is greater than or equal to a preset confidence threshold for each Gaussian particle.

[0311] The second learning rate determination module is used to determine the coefficient of the single parameter iteration update step size of the Gaussian particle as the second learning rate if the fitting confidence of the Gaussian particle is less than the preset confidence threshold for each Gaussian particle.

[0312] Wherein, the first learning rate is greater than the second learning rate.

[0313] In one alternative implementation, the parameter update module includes:

[0314] The update step unit is configured to determine, for each Gaussian particle, the single parameter iteration update step size of the Gaussian particle based on the learning rate of the Gaussian particle, wherein the learning rate is either the first learning rate or the second learning rate.

[0315] The parameter iteration update unit is used to iterate the rendering parameters of each Gaussian particle based on the single parameter iteration update step size corresponding to the Gaussian particle.

[0316] This application also provides an electronic device in its embodiments. (See reference...) Figure 4 The diagram illustrates a structural schematic of an electronic device suitable for implementing the 3D rendering method in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed vehicle terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 4 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0317] like Figure 4 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. When the electronic device is powered on, the RAM 403 also stores various programs and data required for the operation of the electronic device. The processing unit 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0318] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, memory cards, hard drives, etc.; and communication devices 409. Communication device 409 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0319] This application also provides a computer program product, including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the three-dimensional rendering methods provided in this application.

[0320] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the three-dimensional rendering methods provided in this application.

[0321] This application also provides a vehicle, including the electronic device described in the above embodiments, or a 3D rendering device.

[0322] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0323] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0324] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0325] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A three-dimensional rendering method, characterized in that, include: During the iterative optimization of the 3D Gaussian splash model in a dynamic scene, the cumulative change in scene transparency of the rendered image under the current rendering view is obtained. The dynamic scene includes a non-rigid deformation scene that changes continuously over time. The cumulative change in scene transparency represents the cumulative change in shape of the dynamic scene over time. If the cumulative change in transparency of the scene is greater than or equal to the first preset threshold, a resampling operation is performed on each Gaussian particle in the Gaussian particle set of the three-dimensional Gaussian splash model to generate new Gaussian particles. From the Gaussian particles associated with the new Gaussian particle, obtain the optimizer state parameters of the stable Gaussian particle, wherein the stable Gaussian particle is a Gaussian particle whose parameter fluctuation range is less than or equal to a second preset threshold. Set the initial state of the optimizer for the new Gaussian particle to the optimizer state parameters, and perform a gradient update operation on the new Gaussian particle to obtain the updated three-dimensional Gaussian splash model. A visualization of the dynamic scene is generated based on the updated 3D Gaussian splash model.

2. The three-dimensional rendering method according to claim 1, characterized in that, The iterative optimization process includes at least one iteration cycle, and obtaining the cumulative scene transparency change of the rendered image under the current rendering view includes: The average value of the scene cumulative transparency of each pixel in the rendered image corresponding to the at least one iteration cycle is obtained. The scene cumulative transparency of the rendered image in each iteration cycle represents the shape structure and object occlusion degree of the dynamic scene in that iteration cycle. Get the scene cumulative transparency of the rendered image from the current rendering viewpoint; The difference between the average value and the cumulative transparency of the scene is determined as the change in cumulative transparency of the scene.

3. The three-dimensional rendering method according to claim 1, characterized in that, The step of obtaining the optimizer state parameters of the stable Gaussian particle from the Gaussian particles associated with the new Gaussian particle includes: Determine the neighborhood range centered on the coordinates of the new Gaussian particle, with a preset length as the radius; Determine the particle type to which the new Gaussian particle belongs, wherein the particle type includes at least one of the following: rendering character, inherent attributes, and generation source; From the set of Gaussian particles, select target Gaussian particles that are located within the neighborhood range and belong to the particle type; From the plurality of target Gaussian particles, the stable Gaussian particle is selected; The second-order moment parameter of the optimizer for the stable Gaussian particle is determined as the optimizer state parameter.

4. The three-dimensional rendering method according to claim 1, characterized in that, The iterative optimization process includes multiple iteration cycles. After obtaining the cumulative transparency change of the scene, and before performing a resampling operation on each Gaussian particle in the Gaussian particle set of the 3D Gaussian splash model, it also includes: Obtain the coverage weight of each Gaussian particle in the Gaussian particle set. The coverage weight of the Gaussian particle represents the range of rendering pixels that the Gaussian particle can affect under the current rendering view. The reconstruction error of each Gaussian particle in the Gaussian particle set is obtained. The reconstruction error of the Gaussian particle represents the pixel deviation between the image obtained after the Gaussian particle participates in the rendering and the standard ground truth image. Based on the reconstruction error of each Gaussian particle and the coverage weight of each Gaussian particle, the basic importance value of each Gaussian particle is calculated. Gaussian particles whose basic importance value is higher than or equal to a preset importance threshold are retained; Obtain the temporal variance of the importance of the retained Gaussian particles in multiple iteration cycles, wherein the temporal variance of importance represents the variance of the basic importance values ​​of the retained Gaussian particles in multiple iteration cycles; Gaussian particles whose importance time-series variance exceeds a preset variance threshold are identified as transient noise Gaussian particles. Remove the transient noise Gaussian particles to obtain an updated set of Gaussian particles.

5. The three-dimensional rendering method according to any one of claims 1 to 4, characterized in that, Before performing a resampling operation on each Gaussian particle in the Gaussian particle set of the three-dimensional Gaussian splash model, the method further includes: Obtain the projected area of ​​each Gaussian particle in the Gaussian particle set under the current rendering view; Gaussian particles whose projected area exceeds a preset area threshold are identified as super-large Gaussian particles. Establish the correlation between the super-large Gaussian particle and the constraint mask.

6. The three-dimensional rendering method according to claim 5, characterized in that, The resampling operation performed on each Gaussian particle in the Gaussian particle set of the three-dimensional Gaussian splash model to generate new Gaussian particles includes: Splitting and cloning operations are prohibited on the Gaussian particles associated with the constraint mask. The Gaussian particles that are not associated with the constraint mask are split and cloned to generate multiple Gaussian particles; Pruning is performed on all the Gaussian particles to obtain pruned Gaussian particles.

7. The three-dimensional rendering method according to claim 6, characterized in that, The gradient update operation on the new Gaussian particle includes: For all Gaussian particles after pruning, the rendering parameters of the Gaussian particles are iteratively updated based on their respective single parameter iteration update step size. The single parameter iteration update step size of the Gaussian particles associated with constrained masks does not exceed a preset step size threshold.

8. The three-dimensional rendering method according to any one of claims 1, 2, 3, 4 and 6, characterized in that, After obtaining the cumulative transparency change of the scene, and before performing a resampling operation on each Gaussian particle in the Gaussian particle set of the 3D Gaussian splash model, the method further includes: For each Gaussian particle, the fitting confidence of the Gaussian particle is calculated based on the horizontal and vertical gray-level gradients of the projected pixels of the Gaussian particle in the rendered image. For each Gaussian particle, if the fitting confidence of the Gaussian particle is greater than or equal to a preset confidence threshold, the coefficient of the single parameter iteration update step size of the Gaussian particle is determined as the first learning rate. For each Gaussian particle, if the fitting confidence of the Gaussian particle is less than the preset confidence threshold, the coefficient of the single parameter iteration update step size of the Gaussian particle is determined as the second learning rate. Wherein, the first learning rate is greater than the second learning rate.

9. The three-dimensional rendering method according to claim 8, characterized in that, The gradient update operation on the new Gaussian particle includes: For each Gaussian particle, the step size for a single parameter iteration update of the Gaussian particle is determined based on the learning rate of the Gaussian particle, wherein the learning rate is either the first learning rate or the second learning rate. For each Gaussian particle, the rendering parameters of the Gaussian particle are iterated based on the step size of the single parameter iteration update corresponding to the Gaussian particle.

10. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the three-dimensional rendering method as described in any one of claims 1 to 9.

Images