A physical perception snapshot compression imaging three-dimensional Gaussian splash reconstruction method and system

Abstract

The method of snapshot compressive imaging three-dimensional Gaussian splatting reconstruction belongs to the field of computer vision and computational photography. A reconstruction network based on the hybrid architecture of stochastic gradient Langevin dynamics (SGLD) and explicit three-dimensional Gaussian representation is built. Then, multiple physical perception modules are combined to achieve the deep decoupling and reconstruction of the spatiotemporal information of dynamic scenes. Driven by the embedded decoupling deformation field, the deformation field contains a coarse-grained trajectory prediction branch and a fine-grained integral fitting branch: the coarse-grained branch combines low-frequency time coding with latent embedding features, enabling Gaussian primitives to capture global rigid motion trajectories and serving as a temporal regularizer; the fine-grained branch uses high-frequency time coding to predict non-rigid deformation and physically fits the motion blur stripes generated by temporal integration through anisotropic scale stretching. An adaptive density control strategy compatible with physics is introduced. Using the physical integral imaging simulation module, high-fidelity and multi-view consistent high-speed dynamic three-dimensional scene reconstruction results are obtained.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision, computational photography, signal processing, and artificial intelligence. Specifically, it relates to a snapshot-compressed imaging method and system for three-dimensional Gaussian splash reconstruction of high-speed dynamic three-dimensional scenes. This method utilizes explicit three-dimensional Gaussian point cloud representation combined with stochastic dynamics optimization algorithms to recover the physical perception of high-speed dynamic three-dimensional scenes from single-exposure two-dimensional compressed measurements. This invention is particularly suitable for low-cost acquisition, three-dimensional reconstruction, and free-viewpoint video generation of high-speed moving scenes. Background Technology

[0002] Image and video data acquisition is the cornerstone of modern information society. With the increasing demand for high-speed dynamic scene observation in fields such as scientific research, industrial inspection, autonomous driving, sports broadcasting, and consumer electronics, traditional imaging systems face severe challenges. In the design architecture of traditional cameras, there is an irreconcilable physical contradiction between spatiotemporal resolution. To obtain high spatial resolution images, sensors typically need to integrate more pixel units, which leads to longer data readout times, thus limiting the imaging frame rate. Conversely, to capture rapidly changing high-speed motion, spatial resolution must be sacrificed or expensive storage hardware with extremely high bandwidth must be used, which greatly increases the system's cost and power consumption.

[0003] To overcome the limitations imposed by the Shannon-Nyquist Sampling Theorem, Video Snapshot Compressive Imaging (SCI) has emerged as a new computational imaging paradigm. SCI systems cleverly utilize the collaborative mechanism of optical encoding and computational reconstruction. At the optical front end, the system introduces a dynamic encoded aperture (such as a Digital Micromirror Array (DMD) or a Liquid Crystal CoS) into the optical path to perform pixel-by-pixel, time-varying light intensity modulation on multiple consecutive high-speed instantaneous frames within a single exposure time of the image sensor. At the sensor end, these modulated light signals are integrated over time, ultimately capturing a two-dimensional compressed measurement image on a low-speed two-dimensional sensor. From a data flow perspective, the SCI system transforms a high-dimensional spatiotemporal data cube (3D Video Cube) into a single image. The projection was compressed onto a two-dimensional plane using physical means. On, among which This is the compression ratio. This imaging method greatly reduces data transmission bandwidth and storage costs, theoretically enabling the performance of a high-speed camera to be achieved with the hardware of a low-speed, low-cost camera.

[0004] However, the practical application of SCI technology heavily relies on backend reconstruction algorithms, specifically how to faithfully recover the original high-speed video sequence from a single, severely aliased, and temporally lost two-dimensional measurement. This is a typical highly underdetermined (Ill-posed) inverse problem. Existing SCI reconstruction algorithms have mainly gone through two stages of development:

[0005] The first stage is based on traditional iterative optimization methods, such as GAP-TV and PnP-ADMM. These methods typically use total variation or sparse priors as regularization terms to recover the video by iteratively solving an optimization problem. Although the theory is relatively complete, the computational complexity is extremely high, the reconstruction time is long, and it is often difficult to handle complex non-rigid motion and texture details.

[0006] The second stage is based on deep learning-based end-to-end methods, such as BIRNAT, RevSCI, and EfficientSCI. These methods utilize convolutional neural networks (CNN) or Transformer architectures to significantly improve the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) of the reconstruction by learning the spatiotemporal prior distribution of massive amounts of data.

[0007] While the aforementioned methods have made significant progress in 2D image quality metrics, they share a common and fundamental flaw: they primarily treat SCI reconstruction as a denoising or deblurring task in the 2D image domain, outputting merely a sequence of 2D pixel arrays. This approach essentially ignores the inherent 3D geometry of the scene. Therefore, videos reconstructed using these methods cannot support novel view synthesis or guarantee multi-view consistency. In applications requiring 3D spatial perception, such as virtual reality (VR), holographic displays, and embody intelligent navigation, these depth-deficient 2D videos prove inadequate.

[0008] To address the limitations of 2D reconstruction, the academic community has recently begun exploring the direct recovery of 3D scene representations from SCI measurements. For example, the SCINeRF method attempts to introduce implicit neural radiation fields (NeRF) into SCI reconstruction, recovering video by jointly optimizing the camera trajectory and the scene's implicit field. However, NeRF uses a multilayer perceptron (MLP) as the scene representation, resulting in extremely slow training convergence (typically requiring hours or even days) and low inference rendering efficiency, making it difficult to meet the potential real-time processing requirements of SCI systems.

[0009] In contrast, 3D Gaussian Splatting (3DGS), as an emerging explicit radiation field representation technique, achieves high-quality, real-time rendering performance thanks to its anisotropic Gaussian rasterization pipeline, offering new possibilities for 3D reconstruction in SCI (Surface-Computer Interpretation). However, directly applying existing dynamic 3DGS frameworks (such as Deformable-3DGS or SCIGS) to SCI tasks faces significant physical and algorithmic challenges, mainly in the following three aspects:

[0010] First, the entanglement of motion and geometry. Existing dynamic 3DGS methods typically employ coordinate-based deformation networks, meaning they use spatial coordinates as input. and time To predict deformation. The inventors of this application point out that 3DGS is essentially a discrete, unstructured collection of point clouds. In high-speed imaging scenes of SCI, object edges often undergo drastic displacement or non-rigid deformation, resulting in spatially adjacent points (such as foreground object edges and background pixels) having drastically different motion trajectories. However, coordinate-based networks tend to learn spatially continuous and smooth deformation fields, which forces discrete foreground and background points to conform to similar motion laws, leading to severe motion artifacts and geometric adhesion in the reconstruction results.

[0011] Second, optimization instability and initialization dependency. Due to the integral effect of the time dimension, the loss landscape of the inverse problem of SCI exhibits high non-convexity and multimodality. Existing 3DGS optimization typically relies strictly on sparse point clouds generated by the Structure for Motion Reconstruction (SfM) algorithm for initialization. However, in the SCI task, the input is only a single blurred image, and the loss of temporal information causes the SfM algorithm to completely fail. In the absence of good initialization (i.e., only random initialization is possible), the use of standard deterministic gradient descent algorithms (such as Adam) is prone to getting trapped in local minima, leading to geometric collapse, incorrect depth estimation, or failure to converge.

[0012] Third, there is a conflict between scale regularization and physical integration properties. Existing dynamic 3DGS methods generally introduce scale regularization to prevent floating-point artifacts during optimization, forcing Gaussian primitives to maintain a small isotropic scale. This application points out that this artificial constraint directly conflicts with the physical imaging process of SCI. During the exposure time of SCI, high-speed moving objects form continuous motion blur stripes on the sensor, which is an inevitable physical result of time integration. If the scale of Gaussian primitives is forcibly restricted, the model will be forced to use hundreds or thousands of tiny spherical Gaussians to piece together a smooth stripe. This not only causes an explosive increase in the number of primitives and a waste of computational resources, but also introduces high-frequency "pearl necklace" aliasing artifacts due to the overlap of discrete primitives, severely degrading the reconstruction quality.

[0013] In summary, existing technologies lack a robust 3D reconstruction method that can fully respect the physical integral characteristics of SCI, effectively decouple motion and geometry, and does not rely on SfM initialization. Summary of the Invention

[0014] In view of the numerous shortcomings of the existing technologies, the purpose of this application is to provide a physically-aware snapshot compressed imaging (SCI) 3D Gaussian splash reconstruction method and system. This application re-examines the physical imaging mechanism of SCI and proposes a novel optimization paradigm and evolution strategy. Without requiring any prior geometric information (such as SfM point clouds), it can achieve high-quality, robust dynamic 3D scene reconstruction based solely on a single compressed measurement.

[0015] The technical solution adopted in this application is as follows:

[0016] A physically-aware snapshot compressed imaging method for 3D Gaussian splash reconstruction, comprising a data acquisition and random initialization stage, an embedded decoupled deformation field construction stage, a physically integral imaging simulation stage, a stochastic dynamics optimization stage, and an adaptive density control stage; the specific steps include:

[0017] Step S1: Obtain the single-exposure 2D compressed measurement value to be processed and the corresponding time-series physical coding mask; without any prior geometric information, perform a completely random initialization operation within the preset 3D scene bounding box to generate an initial sparse 3D Gaussian set; in addition to having conventional attributes such as position, covariance, color, and opacity, each Gaussian primitive is also given an independent, learnable latent embedding feature, which is used as the motion identity identifier of the primitive;

[0018] Step S2: Construct a hierarchical deformable network that does not depend on spatial coordinate index but only on the latent embedded features; the deformable network includes parallel coarse-grained trajectory prediction branches and fine-grained integral fitting branches; for any time step, the latent embedded features are respectively matched with the branches corresponding to low-frequency and high-frequency time-encoded inputs to predict the rigid motion residuals of Gaussian elements and the non-rigid deformation residuals including anisotropic stretching, and superimposed on the regularized state to generate the instantaneous dynamic Gaussian state;

[0019] Step S3: Based on the physical integration principle of snapshot compressed imaging, use a differentiable Gaussian rasterization engine to render instantaneous images at all time steps; simulate the optical modulation process by multiplying the instantaneous images element-wise with the corresponding physical coding mask; simulate the sensor integration process by accumulating all modulated image frames to synthesize simulated two-dimensional compressed measurement values ​​and establish a differentiable mapping from the four-dimensional spatiotemporal field to the two-dimensional measurement values.

[0020] Step S4: Calculate the photometric loss between the simulated and actual measurements; abandon the traditional deterministic gradient descent strategy and use the stochastic gradient Langevin dynamics (SGLD) algorithm to update the model parameters; during the update process, inject anisotropic random noise related to the geometry of the Gaussian primitives into their positional attributes, and the noise intensity is inversely gated by the opacity of the primitives to drive the primitives to explore in the non-convex solution space and escape local minima;

[0021] Step S5: Perform physically compatible adaptive density control; in the pruning operation, explicitly remove the regularization constraint on the Gaussian unit scale, allowing the Gaussian unit to undergo drastic anisotropic stretching along the motion direction to physically fit the motion blur fringes; in the growth operation, adopt a random residual inverse initialization strategy, and solve the regularization scale parameter of the new primitive by randomly sampling time steps to ensure that the new primitive covers the time integral envelope of the parent primitive in a statistical sense.

[0022] Furthermore, the random initialization operation in step S1 specifically involves: setting a three-dimensional view frustum bounding box based on the field of view angle of the two-dimensional compressed measurement value; and randomly sampling within this bounding box using a uniform or Gaussian distribution. Use a point as the initial Gaussian center; initialize the rotation of all Gaussians to unit quaternions, the scaling to isotropic small values, and the opacity to 0.01; initialize the latent embedding features of each Gaussian to random vectors that follow a standard normal distribution.

[0023] Furthermore, the hierarchical deformation network in step S2 utilizes the spectral bias characteristics of deep neural networks: the coarse-grained branch receives low-frequency time codes and focuses on learning smooth global displacement trajectories to achieve decoupling between motion and geometry; the fine-grained branch receives high-frequency time codes and focuses on learning high-frequency jitter and deformation details.

[0024] Furthermore, the noise injection mechanism of the SGLD algorithm in step S4 has geometrically perceptual characteristics: the injected noise vector is sampled from a standard normal distribution and transformed by the Choreski factor of the Gaussian element covariance matrix, so that the noise diffusion direction is consistent with the principal axis direction of the Gaussian ellipsoid; the noise amplitude decays synchronously with the learning rate and is multiplied by... The Sigmoid function value ensures that strong perturbations are applied only to primitives with uncertain fitting.

[0025] Furthermore, the random residual inverse initialization strategy in step S5 specifically includes: when splitting a parent primitive, the child primitive inherits the latent embedding features of the parent; and randomly and uniformly sampling a time point within the exposure time window [0, T]. ;Query the scale residual of the deformable network at this point in time. According to the desired target scale Reverse calculation of the reference scale of the offspring primitives in the canonical space .

[0026] The technical solution provided in this application has the following beneficial effects:

[0027] 1. This invention successfully solves the initialization problem caused by the lack of SfM point clouds in SCI 3D reconstruction tasks by introducing the SGLD sampling optimization mechanism. It can robustly converge to a fine 3D scene structure under extreme conditions where only random point clouds are given, avoiding the problem of deterministic optimization being prone to local optima.

[0028] 2. The deformation field based on latent embedded features proposed in this invention fundamentally abandons the assumption of continuous field based on coordinates, conforms to the discrete nature of 3DGS, and can effectively decouple motion trajectory from geometric structure. In particular, it can eliminate motion artifacts when dealing with complex dynamic scenes where foreground and background spaces overlap.

[0029] 3. This invention creatively utilizes the anisotropic properties of 3DGS to physically fit the integral motion blur of SCI by removing scale regularization and introducing random residual inverse initialization. This avoids the high-frequency aliasing problem caused by using a large number of tiny primitives to piece together the blur trajectory in traditional methods, and significantly improves the visual quality and parameter efficiency of the reconstruction.

[0030] 4. The method of the present invention can recover the complete time series and three-dimensional structure from a single compressed image, supports high-quality rendering at any time and any viewpoint, and greatly expands the application boundaries of SCI technology. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0032] Figure 1 This is a schematic diagram of the overall process of a physical sensing snapshot compressed imaging three-dimensional Gaussian splash reconstruction method according to the present invention. Detailed Implementation

[0033] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. The embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0034] This invention provides a PhysSCI-GS method for 3D Gaussian splash reconstruction using snapshot compressed imaging (hereinafter referred to as PhysSCI-GS), which aims to solve the core problems of existing technologies in handling the inverse SCI problem, such as initialization dependency, motion-geometry entanglement, and physical model mismatch. This method combines the explicit representation capability of 3DGS with the stochastic exploration capability of SGLD, and customizes deformation and evolution strategies specifically for the integral imaging characteristics of SCI.

[0035] like Figure 1 As shown, the method in this embodiment mainly includes the following steps:

[0036] Step S1: Data acquisition and random initialization of space;

[0037] Step S2: Construct an embedded decoupled deformation field based on individual identity;

[0038] Step S3: Simulation rendering of the physical integration imaging process;

[0039] Step S4: Stochastic dynamics optimization of physical perception;

[0040] Step S5: Physically compatible adaptive density control;

[0041] Step S6: Iterative convergence and output.

[0042] The following is a detailed technical breakdown and explanation of each step.

[0043] I. Data Acquisition and Spatial Random Initialization (Step S1)

[0044] Traditional 3DGS methods typically rely on sparse point clouds generated by SfM algorithms such as COLMAP as initial input to provide the initial geometry of the scene. However, in SCI tasks, the input is only a two-dimensional image, losing temporal information, leading to difficulties in feature point matching and causing SfM algorithms to fail. Existing methods (such as SCINERF) usually require complex pre-trained networks or extremely fragile heuristic initialization. This application proposes a completely "FromScratch" random initialization strategy, which, combined with subsequent SGLD optimization, exhibits extremely strong robustness.

[0045] The specific implementation is as follows:

[0046] 1.1 Input Data Parsing: Obtaining Two-Dimensional Compressed Measurements of a Single Exposure Where H and W are the image resolutions. Simultaneously, the corresponding time-series physical coding mask is obtained. ,in , Size of the time window. Mask. Corresponding to the first exposure time The modulation pattern at each instant. It should be noted that this method does not rely on any additional auxiliary data, such as multi-view images or depth maps.

[0047] 1.2 Spatial Bounding Box Definition: Based on the randomly initialized intrinsic parameter matrix of the camera. and randomly initialized scene depth range Define a view frustum in three-dimensional space, where, This represents the minimum depth value allowed for observation in the current scene. This represents the maximum depth value, which ranges from [0,1].

[0048] Axially Aligned Bounding Box (AABB) Construction: Construct a 3D axis-aligned bounding box that completely encloses the view frustum, with its coordinate range set to [value missing]. .

[0049] Coordinate value determination: To ensure that those skilled in the art can implement this, the values ​​of each boundary of the bounding box are determined through the following logic: and The values ​​are respectively taken as boundary values ​​of the depth range. and ; It is determined by calculating the extreme values ​​of the coordinates of the four vertices of the far plane of the view frustum in the three-dimensional coordinate system, ensuring that the three-dimensional axis-aligned bounding box can completely cover the projection area of ​​the view frustum in both the horizontal and vertical directions.

[0050] 1.3 Random Point Cloud Generation: Within the 3D bounding box, random point clouds are generated using a uniform probability distribution. Random sampling 1 coordinate point, as The initial mean position of each Gaussian element : Alternatively, a Gaussian distribution with the camera center as the origin can be used for sampling, so that the initial point cloud is more concentrated in front of the camera's field of view.

[0051] 1.4 Attribute Initialization: For each generated Gaussian element, initialize its attributes:

[0052] (1) Covariance property: the scaling factor of all Gaussian elements Initialized to isotropic logarithmic space values ,in The radius is randomly initialized; the rotation quaternion is... Initialize to a unit quaternion Or it may follow a random rotation that is uniformly distributed.

[0053] (2) Appearance attributes: spherical harmonic coefficient The DC component is initialized to a random value or a uniform gray value, and the higher-order coefficients are initialized to 0; the opacity is set to... It is initialized to 0.1 and mapped to the unconstrained optimization space through the inverse Sigmoid function.

[0054] (3) Key Innovation – Latent Embedding: The latent embedding features of each Gaussian unit Initialize as follows: follows a standard normal distribution A random vector; this vector serves as the unique motion identifier for this primitive in subsequent optimizations.

[0055] 1.5 Camera Pose Initialization: Since SCI cannot provide pose, the initial camera extrinsic parameters are set to the identity matrix or a coarse pose given by the experimental scenario. In subsequent optimizations, the camera pose will be adjusted jointly with Gaussian elements as an optimizable parameter.

[0056] II. Constructing an embedded decoupled deformation field based on individual identity (Step S2)

[0057] To address the "motion-geometric entanglement" problem in SCI imaging, namely the difficulty in distinguishing between a stationary background and a moving foreground, this application abandons the coordinate-based deformation network used in methods such as SCIGS and proposes a decoupled deformation field based on embedding features.

[0058] The design of this deformation field is based on two core physical insights: first, 3DGS is discrete, so motion should be "object-centric" rather than "space-centric"; second, neural networks have a spectral bias in the early stages of training, tending to learn low-frequency signals first.

[0059] like Figure 1 As shown (Note: This refers to the deformation field section in the overall flowchart), the deformation network is designed as a hierarchical structure, containing a coarse-grained deformation network. and fine-grained deformable networks .

[0060] The specific implementation process is as follows:

[0061] 2.1 Time code generation:

[0062] To achieve motion decoupling by utilizing spectral bias characteristics, this application designs two sets of time codes with different frequencies. A sinusoidal position coding function is used. scalar time It is mapped to a high-dimensional feature vector.

[0063] Low-frequency time coding Only use smaller ones Value (e.g.) This limits its bandwidth, forcing the network to only express smooth, low-frequency motion.

[0064] High-frequency time coding : Use larger Value (e.g.) This gives it the ability to express high-frequency jitter and drastic deformation.

[0065] 2.2 Coarse-grained trajectory prediction:

[0066] This branch aims to capture the overall rigid motion of an object. The input is the latent embedding features of Gaussian elements. With low-frequency time coding The concatenated vectors are mapped using a multilayer perceptron (MLP) to output coarse-grained translation residuals. and rotational residuals .

[0067] Its mathematical expression is: .

[0068] Because the input is limited to low-frequency encoding, this branch naturally ignores high-frequency noise and locks onto the main motion trajectory of the object. This is equivalent to applying an implicit low-pass filter in the time domain, ensuring the smoothness of the motion trajectory.

[0069] 2.3 Fine-grained integration fitting:

[0070] This branch aims to capture non-rigid deformations caused by time integration, particularly stretching due to motion blur. The input is latent embedded features. High-frequency time coding The concatenated vector. Through MLP mapping, fine-grained positional adjustment is output. Rotational fine adjustment amount And, crucially, anisotropic scale residuals. and color residual .

[0071] Its mathematical expression is: .

[0072] in Allowing Gaussian elements to deform over time during exposure is crucial for fitting motion blur in SCI. For example, a spherical Gaussian can... The image is stretched into an ellipsoid to simulate the motion blur produced by high-speed motion.

[0073] 2.4 Dynamic State Synthesis:

[0074] Ultimately, time step Gaussian state under From regular state Combined with the above residuals:

[0075] (1) Location update:

[0076] (2) Rotation Update: It is necessary to ensure that quaternions are normalized.

[0077] (3) Scale update: An exponential function is used to ensure that the scale is positive.

[0078] (4) Color update:

[0079] The technical benefits of this decoupling design are: It acts as an "anchor point," quickly locking in the general direction of movement in the early stages of optimization; and Based on this, it is responsible for "filling in" the blurred details caused by integral imaging. Even if two Gaussian elements overlap in spatial location (e.g., foreground occluding background), as long as their embedding features... Because they are different, they can exhibit completely different motion behaviors, thus perfectly resolving the entanglement between motion and geometry.

[0080] III. Simulation and rendering of the physical integral imaging process (step S3)

[0081] A dynamic Gaussian state with time-varying characteristics was constructed. Following this, the reconstruction process enters the crucial physical simulation phase. The physical integral imaging simulation module in this application is not a simple image renderer, but a differentiable simulator deeply coupled with the SCI imaging mechanism. Its core task is to establish a precise and differentiable mapping from a four-dimensional continuous spatiotemporal field to two-dimensional discrete compressed measurements, thereby allowing the error gradient to propagate backward from the two-dimensional sensor plane to every Gaussian cell in three-dimensional space. This process strictly follows the physical transport path of photons, simulating the entire process from scene radiation to sensor charge accumulation.

[0082] In practice, this step includes the following detailed processes and technical aspects:

[0083] 3.1 Time-Slice Rasterization: To simulate the time of a single exposure. The system first discretizes the time dimension into a continuous luminous flux integral. Each time step. For each time step ( The system invokes the differentiable Gaussian rasterization engine. The rasterization process first performs frustum culling, removing primitives outside the view frustum based on the current camera pose and the position and scale of the Gaussian primitives to reduce unnecessary computation. Subsequently, the valid 3D Gaussian primitives are projected onto the 2D image plane. The projection process involves converting the 3D covariance matrix... Transform into a two-dimensional covariance matrix This transformation is achieved through the Jacobian matrix. and view transformation matrix accomplish: This step achieves the geometric mapping from three-dimensional Euclidean space to two-dimensional screen space. For each pixel coordinate on the image plane... The system calculates the contribution of all Gaussian elements covering that pixel. Each Gaussian element in a pixel Opacity contribution The calculation formula is: in This represents the two-dimensional mean position after projection. This formula physically describes the attenuation characteristics of light passing through Gaussian elements. Finally, using a depth-sorted alpha blending technique, the instantaneous radiation image at the current moment is synthesized. : in This is a set of Gaussian elements sorted by depth from near to far. This process can accurately simulate the occlusion relationships and lighting distribution of a scene at a specific instant, generating instantaneous image frames that conform to physical laws.

[0084] 3.2 Physical Mask Modulation in the Generation of Instantaneous Images Next, the system needs to simulate the coded aperture modulation effect in the optical path of the SCI system. This involves obtaining the current time step... Corresponding physical coding mask The mask is typically loaded with a digital micromirror array (DMD) or liquid crystal on silicon (LCoS), and is represented as a binary matrix (0 or 1) or a grayscale matrix (transmittance between 0 and 1). The system performs a pixel-by-pixel Hadamard product operation, transforming the instantaneous image... With mask Modulation: This step physically corresponds to the on / off control or intensity attenuation of light as it passes through the coded aperture. For a binary mask, a pixel value of 0 indicates that light is blocked, and a pixel value of 1 indicates that light passes through. Through this high-speed temporal control, the temporal information of the scene is encoded into the spatial pattern.

[0085] 3.3 Sensor Integration: Simulates the photoelectric conversion and charge accumulation characteristics of image sensors (such as CCD or CMOS). The core characteristic of SCI cameras lies in their integration throughout the entire exposure time. No charge readout or reset is performed internally; the photon energies arriving at the photosensitive surface at all times are linearly superimposed in the potential well. Therefore, the system allocates a high-precision accumulation buffer in the video memory and stores all time steps... to Modulated image Accumulate to this buffer: Final output This refers to the simulated two-dimensional compressed measurement. This step realizes the projection compression of the 3D spatiotemporal volume along the time axis onto a two-dimensional plane, and is a crucial step in establishing the relationship between model parameters and observation data. Since the entire process consists of differentiable operators, the gradient of the observation error with respect to each Gaussian element property can be calculated through an automatic differentiation mechanism.

[0086] IV. Stochastic Dynamics Optimization of Physical Perception (Step S4)

[0087] In obtaining simulated measurement values Afterwards, the loss function needs to be constructed and the model parameters updated. The core challenge of the SCI inverse problem lies in its extreme ill-posedness: recovering a video sequence from a single image means an extremely large solution space, and the landscape of the loss function is full of local minima. Traditional deterministic gradient descent algorithms usually assume that the loss function is smooth and convex, which does not hold true in the SCI task. In particular, since this application adopts a random initialization strategy, the initial Gaussian point cloud is chaotic. If only deterministic gradients are used, the model is very prone to getting stuck in incorrect geometry (e.g., misinterpreting moving objects in the foreground as static textures in the background, or producing depth collapse).

[0088] To fundamentally solve this optimization problem, this application introduces the idea of ​​Langevin Dynamics from statistical physics, and reformulates the 3D reconstruction process as a sampling process in the posterior probability distribution, rather than finding a single maximum a posteriori estimate (MAP).

[0089] 4.1 Constructing the Physical Constraint Loss Function The loss function adopted in this application is... The design is a weighted sum of photometric consistency constraints and opacity sparsity constraints: Among them, photometric loss To ensure that the reconstruction results are visually consistent with the observed data, L1 norm and structural similarity (D-SSIM) are combined: ,in, =0.05 and =0.2 is a hyperparameter that balances the weights of each loss term. Opacity sparse loss. Defined as the sum of the L1 norms of the opacities of all Gaussian elements: The sparsity term not only serves as regularization but is also the key driver of primitive "death" during SGLD sampling. It forces primitives that contribute little to the reconstruction or are misplaced to gradually disappear. It is particularly important to note that this application strictly prohibits any regularization terms (such as scale penalty) targeting the Gaussian scale in the loss function. Existing techniques often use scale regularization to prevent excessively large Gaussians, but in SCI, the physical nature of motion blur requires Gaussian primitives to be able to stretch freely. Adding scale constraints would directly cause the model to fail to fit the physically real motion trajectory, which is one of the key design differences between this application and existing techniques.

[0090] 4.2 SGLD parameter update rules are in section [number missing]. In the next iteration, for the model parameters (especially the position of Gorski) The following stochastic gradient Langevin dynamics update rule is adopted: in The learning rate decays with the number of iterations. For deterministic gradient descent, the parameters are guided to move towards a low-energy (low-error) region. The injected random noise term is the fundamental driving force for this application to achieve global exploration, escape local minima, and recover the structure from random initialization.

[0091] 4.3 Anisotropic Noise Injection Mechanism for Geometric Awareness To ensure that the injected noise provides sufficient exploration capability without disrupting the already formed fine geometric structure (such as thin plates or edges), this application designs a noise construction method deeply coupled with the geometry. For the first... Each high-level element is injected into its position. noise The structure is as follows: ,in It is an isotropic noise vector that follows a standard normal distribution. The preconditioner matrix is ​​defined by combining the geometric features and confidence information of the Gaussian.

[0092] in This is the Cholesky factor of the Gausky element covariance matrix. This term's function is to reduce isotropic noise. The extrusion deformation is into an ellipsoid consistent with the shape of the Gaussian primitive. This means that noise primarily propagates along the major axis of the Gaussian primitive (i.e., the direction of greatest spatial uncertainty, which is also typically the direction of motion ambiguity), while being constrained along the minor axis (i.e., the direction of geometric determination, such as the normal to an object's surface). This design causes the primitive to tend to "slide" along the object's surface or motion trajectory rather than "jump" perpendicular to the surface, thus preserving the smoothness of the object's surface and the continuity of its structure.

[0093] (2) Opacity Reverse Gating: is the opacity gating factor, where =0.01 is the scaling factor; this factor establishes an inverse relationship between noise intensity and primitive confidence: when the primitive opacity is high... When it approaches 1, When noise injection stops, the algorithm degenerates into fine gradient descent to maintain structural stability; when When approaching 0, Injecting strong noise to drive the primitive to escape its current position.

[0094] V. Physically Compatible Adaptive Density Control (Step S5)

[0095] Density control strategies in 3DGS (i.e., primitive splitting, cloning, and pruning) are crucial for determining reconstruction quality and efficiency. Existing strategies are mostly based on gradient heuristics, and to suppress artifacts, scale regularization is commonly introduced to forcibly limit the size of Gaussian primitives. Through in-depth analysis, the inventors of this application discovered that this limitation fundamentally conflicts with the physical characteristics of SCI integral imaging. Therefore, this application proposes a physically compatible evolutionary strategy, which is one of the key innovations distinguishing this invention from existing technologies.

[0096] 5.1 Scale-Unconstrained Evolution Strategy In SCI imaging, high-speed moving objects within a single exposure time... Within the sensor, a continuous, elongated trajectory is left, known as a motion blur stripe. The problem with existing techniques is that if scale regularization is applied, forcing Gaussian primitives to maintain a small, round, isotropic shape, the model must use hundreds or thousands of tiny Gaussian primitives, arranged closely like a "pearl necklace," to fit this trajectory. This not only leads to an explosion in the number of primitives and increased memory usage, but more seriously, due to overlap and phase differences between discrete primitives, it generates high-frequency ripple artifacts and aliasing effects in the reconstructed image, severely impacting image quality. The solution proposed in this application is to completely remove the scale regularization constraint. This strategy allows Gaussian primitives to be used in fine-grained deformable networks. Driven by this, extreme anisotropic stretching occurs along the direction of motion. This allows a single (or very few) Gaussian primitives to be topologically deformed into "strips" or "noodles," thus physically perfectly covering the integral signal generated by motion blur. This strategy not only significantly reduces the number of primitives required and improves parameter efficiency, but also fundamentally eliminates high-frequency artifacts generated by continuous discrete fitting, achieving a natural and smooth fit to motion blur. This is a physically-aware inductive bias specifically designed for integral imaging.

[0097] 5.2 Stochastic Residual Inversion Strategy When the system determines that a parent Gaussian needs to be split or cloned to generate a child Gaussian, how to initialize the parameters of the child is a challenging problem. Since the parent Gaussian may have been stretched very long to fit a blurred trajectory, directly inheriting the parent's huge scale will result in an overly large child, failing to refine the texture; while simply resetting the scale will destroy the trajectory coverage, leading to training oscillations. This application proposes an initialization strategy based on temporal statistical sampling, specifically including the following logic: when the opacity... When it is less than 0.05, the first... The parent primitive is split or cloned to generate the second generation. For each new primitive, perform the following operations: Inheritance steps: The latent embedding features of the parent primitive are directly copied to the new primitive, i.e. Simultaneously initialize geometric properties. , Random time sampling: Uniformly and randomly sample a time point within the exposure time window [0, T]. This step utilizes the concept of Monte Carlo sampling, transforming the fitting of a continuous trajectory into the fitting of discrete time points.

[0098] Residual query steps: Inherited embedded features and sampling time Input-learnable fine-grained deformable networks Forward propagation obtains the prediction scale residual at that moment. ; Inverse kinematics steps: Set the new primitive in The expected total scale of time is ; Use inverse operations to solve its reference scale in regular space : ,in is the scaling activation function.

[0099] Physical significance: By randomly sampling and inversely solving along the time axis, this strategy statistically ensures that the resulting subsets of primitives are uniformly distributed within the temporal integral envelope covered by the parent primitives. This is equivalent to physically resampling the motion-blurred trajectory, replacing the original coarse parent primitives with multiple smaller subsets distributed along the trajectory. This maintains coverage of the motion trajectory while giving the subsets the ability to fit high-frequency texture details (such as texture variations in the blur), achieving a smooth transition from a "coarse trajectory" to a "fine texture."

[0100] 5.3 Based on Opacity-Based Lifecycle Management and SGLD Optimization, this application employs a dynamic "survival of the fittest" mechanism. Pruning: Due to the injection of noise, redundant Gaussian primitives located in incorrect positions, occluding the background, or contributing little to the reconstruction cannot stably accumulate gradients. In sparse loss... Under sustained suppression, its opacity It will decay rapidly. When Below the preset threshold When the value is 0.05, the primitive is considered "dead" and immediately removed. Growth: Simultaneously, the system monitors the magnitude of the positional gradient. In regions with large gradients (typically areas with rich textures, sharp edges, or underfitting), the system generates new primitives based on probability (through splitting or cloning). This dynamic life-and-death cycle based on opacity and gradient, combined with SGLD's exploration capabilities, ensures that Gaussian primitives can automatically migrate from blank or erroneous regions and concentrate in areas of complex structure and high information content within the scene, thus achieving optimal resource allocation.

[0101] VI. Iterative Convergence and Output (Step S6)

[0102] The above steps constitute a complete iterative cycle. During training, the system employs a phased annealing strategy: Initial Phase (Exploration Phase): A higher learning rate is set. and higher noise control parameters At this stage, the opacity gating factor is primarily open, driving Gaussian primitives to perform extensive Brownian motion-like exploration in 3D space. The goal of this stage is to quickly lock the coarse geometry and main motion trajectories of the scene and establish the correct global topology. In the refinement phase, as iterations proceed, the learning rate and noise parameters decay exponentially. The positions of the Gaussian primitives gradually lock, and noise injection ceases. At this point, the optimization focus shifts to fine-tuning the deformable network parameters and fitting appearance attributes (spherical harmonic coefficients). Using a stochastic residual inverse initialization strategy, the Gaussian primitives continuously split and refine, fitting high-frequency texture details and non-rigid deformations. When the loss function converges or reaches the preset maximum number of iterations, the system outputs the final optimized Gaussian point cloud model. This model contains complete geometric, appearance, and dynamic information of the scene and can be used to render reconstructed high-speed dynamic scenes at arbitrary frame rates (Temporal Super-resolution) and arbitrary viewpoints (Novel View Synthesis).

[0103] VII. System Device Implementation Examples

[0104] Based on the same inventive concept, this application also provides a physically-aware snapshot compressed imaging 3D Gaussian splash reconstruction system. This system can be deployed on high-performance workstations, cloud servers, or embedded computing platforms. Its logical structure includes:

[0105] 7.1 Data Acquisition and Random Initialization Module: This module is equipped with a high-speed data interface for reading raw 2D RAW data and synchronized mask data from the SCI camera. The module embeds a random number generation algorithm to generate random coordinate points following a uniform or Gaussian distribution within a 3D frustum bounding box set according to camera parameters, and assigns randomly initialized embedding vectors, rotation, scaling, and color attributes to each point. This module's design is completely independent of traditional SfM point cloud generation steps, ensuring the system's cold-start capability in the absence of prior information.

[0106] 7.2 Embedded Deformation Module: This module is the core dynamics engine of the system. It contains a large-capacity memory unit to store the potential embedding feature vectors of millions of Gaussian primitives. The computation unit is configured to perform forward propagation of a deep neural network and contains two parallel multilayer perceptron (MLP) logic blocks, corresponding to the coarse-grained deformation network and the fine-grained deformation network, respectively. This module is responsible for calculating and outputting the dynamic attribute residuals (displacement, rotation, stretching, etc.) of each primitive in real time at each rendering iteration, based on the input timestamp and the embedding features of each primitive.

[0107] 7.3 Integral Imaging Simulation Module: This module serves as a bridge between the three-dimensional digital world and two-dimensional observation data. It contains a highly optimized, tile-based, differentiable rasterization engine that supports cyclic calls along the time dimension. The module includes a high-precision accumulator register for execution... The module performs physical integration operations. It features a specially designed automatic differentiation interface that can accurately backpropagate photometric errors on the two-dimensional measurement plane back to the Gaussian elements and deformable network parameters in three-dimensional space.

[0108] 7.4 SGLD Sampling Optimization Module: This module is the system's optimization controller. It contains a loss calculation unit for real-time calculation of L1 loss and D-SSIM loss. The core component is an anisotropic noise generator, which reads the covariance matrix of each Gaussian primitive, performs real-time Chollesky decomposition, and transforms the generated standard normal noise into ellipsoidal noise matching the primitive shape. Furthermore, this module includes opacity gating logic to dynamically adjust the gain of injected noise based on the real-time opacity of the primitive, achieving a smooth transition from exploration to exploitation.

[0109] 7.5 Unconstrained Density Evolution Module: This module is responsible for the dynamic management of the Gaussian point cloud topology. It includes a state monitoring unit to detect whether the opacity of Gaussian primitives is below a threshold (triggering pruning) or whether the position gradient exceeds a threshold (triggering splitting / cloning). This module is specifically configured to shield against any scale-related regularization penalty calculations. When performing primitive splitting, this module calls its internal random time sampler, combining it with the output of the deformation module to perform random residual inverse initialization operations, ensuring a smooth transition between old and new primitives in the temporal dynamics.

[0110] VIII. Examples of Electronic Devices and Storage Media

[0111] This application also provides an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to, when executing the instructions, implement the physical perception snapshot compressed imaging three-dimensional Gaussian splash reconstruction method described in any one of the above embodiments. The electronic device may be a desktop computer, a laptop, a cloud server cluster, or an embedded edge computing device integrating a high-performance GPU / NPU.

[0112] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the physical perception snapshot compressed imaging three-dimensional Gaussian splash reconstruction method described in any of the above embodiments. The storage medium includes, but is not limited to, non-volatile storage media such as hard disks, optical disks, USB flash drives, and flash memory cards (SD / TF cards).

[0113] Summary of Beneficial Effects: Compared with existing technologies, the technical solution provided in this application has the following significant advantages: Extremely Strong Robustness: By introducing the SGLD sampling optimization mechanism, the convergence problem caused by the inability to use SfM initialization in SCI reconstruction tasks is successfully solved. Even starting from completely random noisy point clouds, it can stably converge to a fine 3D scene structure. Physically Realistic Motion Modeling: By creatively removing scale constraints and allowing Gaussian primitives to undergo extreme anisotropic stretching, this method perfectly matches the physical characteristics of continuous motion blur generated by SCI integral imaging, eliminating high-frequency aliasing artifacts caused by discrete primitive piecing together in traditional methods. Thorough Motion-Geometric Decoupling: Based on the deformation field design of latent embedding features, the binding relationship between spatial position and motion trajectory is broken, enabling the system to accurately distinguish spatially overlapping foreground moving objects and background stationary objects, eliminating motion artifacts. Efficient Evolution Strategy: The stochastic residual inverse initialization strategy not only ensures the consistency of new and old primitives in the temporal domain but also accelerates the convergence process of the model from coarse trajectories to fine textures, significantly improving training efficiency and reconstruction quality.

[0114] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for three-dimensional Gaussian splash reconstruction using physical sensing snapshot compressed imaging, characterized in that, Includes the following steps: Step S1: Data acquisition and spatial random initialization steps; Acquire two-dimensional compression measurements during a single exposure process and the corresponding time-series physical coding mask ;in This represents the size of the time window, which is also the number of instantaneous frames compressed within a single exposure time. Without relying on any external sparse point cloud or geometric priors, a 3D spatial bounding box covering the scene to be reconstructed is defined. Random sampling is performed within this 3D spatial bounding box to generate an initial 3D Gaussian set. The initial three-dimensional Gaussian set contains Each Gaussian element is assigned randomly initialized geometric properties, appearance properties, and potential embedding features to drive time-varying dynamics. Step S2: Constructing an embedded decoupled deformation field based on individual identity; A hierarchical deformable network is constructed that does not rely on spatial coordinate indexing but only on Gaussian element individual identity features; the deformable network is configured to include parallel coarse-grained trajectory prediction branches and fine-grained integral fitting branches; for exposure time windows... any time step within The latent embedding features of each Gaussian element and the temporal encoding at the current time are input into the deformable network; the temporal encoding of each Gaussian element at time step is calculated. The rigid motion residuals and non-rigid deformation residuals are superimposed onto the canonical state of the Gaussian element to generate the time step. Instantaneous dynamic Gaussian state The dynamic state includes time-varying position, rotation, anisotropic scale, and color. Step S3: Simulation rendering steps of the physical integral imaging process; Based on the dynamic Gaussian state The time step is obtained by rendering using a differentiable Gaussian rasterization engine. instantaneous radiation image ; Construct a forward physical integral model for a snapshot compressed imaging system; based on the model, simulate the optical modulation process, and incorporate time steps. to All instantaneous radiation images The physical coding mask corresponding to the time step. Perform element-wise modulation. The simulation sensor integration process involves accumulating photon energy across all modulated image frames on the simulated sensor plane to synthesize a simulated two-dimensional compressed measurement value. This establishes a differentiable mapping relationship from four-dimensional dynamic scenes to two-dimensional compressed measurement values; Step S4: Stochastic dynamics optimization step for physical perception; Construct a loss function to calculate the simulated two-dimensional compression measurement. Compared with the true two-dimensional compression measurement value The differences between them are as follows: abandoning the deterministic gradient descent strategy, the stochastic gradient Langevin dynamics (SGLD) algorithm is used to iteratively update the properties of Gaussian elements; during the parameter update process, anisotropic random noise related to the geometry of the Gaussian element is injected into the positional properties of the Gaussian element; the intensity of the noise is gated and adjusted by the current opacity of the Gaussian element to drive the Gaussian element to explore the non-convex solution space caused by the loss of time information and escape local minima; Step S5: Physically compatible scale-free density control steps; During the optimization process, the number and distribution of Gaussian primitives are dynamically adjusted; opacity-based pruning is performed, in which the regularization penalty on the scale of Gaussian primitives is explicitly removed, allowing Gaussian primitives to undergo anisotropic stretching along the motion trajectory direction to physically fit the time integral fuzzy trajectory (Motion Streaks); primitive growth is performed, and for Gaussian primitives to be split or cloned, a random residual inverse initialization strategy is used to calculate the regularization scale parameter of the new primitives to ensure that the new primitives statistically cover the time integral envelope of the parent primitives; Step S6: Iterative Convergence Step Repeat steps S2 to S5 to jointly optimize the regularization properties of Gaussian elements, latent embedding features, and deformable network parameters until more than 20,000 iterations are reached, and the reconstructed high-speed dynamic video sequence or dynamic 3D scene is output.

2. The method according to claim 1, characterized in that, The specific technical details of the random initialization operation in step S1 include: Step 1-1: Defining the scene space; Parameter analysis and definition: The resolution of the two-dimensional compressed measurement value Y in the analytical input. ;in, Defined as the height pixel value of the image. Defined as the width of the image in pixels; Camera Intrinsic Parameter Acquisition: Obtain the randomly initialized camera intrinsic parameter matrix. The camera intrinsic parameter matrix It is obtained through random initialization and includes focal length parameters. and principal point coordinate parameters It is used to establish the projection mapping relationship between three-dimensional spatial coordinates and two-dimensional pixel coordinates; View frustum space definition: based on a randomly initialized depth range Define a view frustum space; where, This represents the minimum depth value allowed for observation in the current scene. This represents the maximum depth value, which ranges from [0,1]. Axially Aligned Bounding Box (AABB) Construction: Construct a 3D axis-aligned bounding box that completely encloses the view frustum, with its coordinate range set to [value missing]. ; Coordinate values ​​are determined as follows: The values ​​of each boundary of the bounding box are determined by the following logic: and The values ​​are respectively taken as boundary values ​​of the depth range. and ; It is determined by calculating the extreme values ​​of the coordinates of the four vertices of the far plane of the view frustum in the three-dimensional coordinate system, ensuring that the three-dimensional axis-aligned bounding box can completely cover the projection area of ​​the view frustum in the horizontal and vertical directions; Step 1-2: Randomly sample locations; Inside the 3D bounding box, through a uniform probability distribution Random sampling 1 coordinate point, as The initial mean position of each Gaussian element : Alternatively, a Gaussian distribution with the camera center as the origin can be used for sampling to make the initial point cloud more concentrated in front of the camera's field of view; Steps 1-3: Randomly initialize attributes; Covariance property initialization: scaling of all Gaussian elements Initialized to isotropic logarithmic space values ,in The radius is randomly initialized; the rotation quaternion is... Initialize to a unit quaternion Or it may follow a random rotation with a uniform distribution; appearance attribute initialization: spherical harmonic coefficients The DC component is initialized to a random value or a uniform gray value, and the higher-order coefficients are initialized to 0; the opacity is set to... Initialized to 0.1 and mapped to the unconstrained optimization space via the inverse Sigmoid function; dynamic property initialization: the latent embedding features of each Gaussian unit are initialized. Initialize as follows: follows a standard normal distribution A random vector; this vector serves as the unique motion identifier for this primitive in subsequent optimizations.

3. The method according to claim 1, characterized in that, The specific steps of constructing and calculating the embedded decoupled deformation field in step S2 include: Step 2-1: Generation of multi-frequency time codes; Using a sinusoidal position encoding function to convert scalar time Mapped to high-dimensional feature vectors; two sets of time codes with different frequency ranges are generated, namely low-frequency time codes. and high-frequency time coding ; Step 2-2: Coarse-grained trajectory decoupling prediction; Using coarse-grained deformable networks Predict the rigid displacement of Gaussian elements; Input: latent embedding features of Gaussian elements With low-frequency time coding The data is stitched together; processing involves nonlinear mapping using a multilayer perceptron (MLP); output is a coarse-grained translation residual. and coarse-grained rotational residuals Mathematical expression: ; Steps 2-3: Fine-grained integral fitting prediction; Using fine-grained deformable networks Predict the non-rigid deformation and appearance changes of Gaussian elements; Input: latent embedding features of Gaussian elements High-frequency time coding The process involves stitching the data together; processing: nonlinear mapping using a multilayer perceptron; output: fine-grained position adjustment. Rotational fine adjustment amount Anisotropic Scale Residuals and color residual Mathematical expression: ; Steps 2-4: State overlay update; The residuals obtained from the above calculations are applied to the regularized state of the Gaussian element. Above, calculate the current time step. State parameters: Position update: Rotation Update: Scale update: Here, an exponential activation function is used to ensure the scale is positive; color update: .

4. The method according to claim 1, characterized in that, The specific implementation of the physical-aware stochastic dynamics optimization step in step S4 is as follows: In the k-th iteration, for the th Position parameters of each Gaussian element Execute the following update rules: in, The learning rate decays with the number of iterations. This represents the deterministic gradient of the total loss function with respect to location. The construction of the injected random noise term must satisfy physical and geometric constraints: in: This is an isotropic noise vector that follows a standard normal distribution; is the Choleski decomposition factor of the current covariance matrix of the Gaussian element or its approximate matrix; is the opacity gating factor, where This is a scaling factor; this factor establishes an inverse relationship between noise intensity and primitive confidence: when the primitive opacity is high... When it approaches 1, When noise injection stops, the algorithm degenerates into fine gradient descent to maintain structural stability; when When approaching 0, Injecting strong noise to drive the primitive to escape its current position.

5. The method according to claim 1, characterized in that, The physically compatible adaptive density control step in step S5 includes a scale-free evolutionary strategy, which is used to construct the total loss function. At that time, only photometric loss was included. and L1 norm loss of opacity , =0.001; By removing the scale constraint, individual Gaussian elements are allowed to deform in fine-grained networks. Driven by this, its scale parameters Extreme anisotropic stretching occurs along the direction of motion.

6. The method according to claim 1, characterized in that, The physically compatible adaptive density control step in step S5 further includes a random residual inverse initialization strategy, characterized in that: When opacity When it is less than 0.05, the first... The parent primitive is split or cloned to generate the second generation. For each new primitive, perform the following operations: Inheritance steps: The latent embedding features of the parent primitive are directly copied to the new primitive, i.e. Simultaneously initialize geometric properties. , ; Random sampling step: Uniformly sample a random time point within the exposure time interval [0, T]. ; Residual query steps: Inherited embedded features and sampling time Input-learnable fine-grained deformable networks Forward propagation obtains the prediction scale residual at that moment. ; Benchmark inverse kinematics steps: Set the new primitive in The expected total scale of time is ; Use inverse operations to solve its reference scale in regular space : ,in is the scaling activation function.

7. The method according to claim 1, characterized in that, The specific mathematical form for calculating photometric loss in step S4 is as follows: in, The photometric consistency loss is composed of a weighted average of L1 distance and structural similarity (D-SSIM) loss: For the opacity sparsity regularization term, it is defined as the sum of the absolute values ​​of the opacities of all Gaussian elements: in, =0.05 and =0.2 is the hyperparameter that balances the weights of each loss term; the... Its role is not only to sparsify the model, but also to drive the relocation cycle of MCMC: the noise injected by SGLD makes redundant primitives unable to stabilize at a certain position, causing their opacity to decay to 0 under the penalty of sparse regularization, and then they are recycled by the system and regenerated in the high error region.

8. The method according to claim 3, characterized in that, The latent embedding features It is a length of The learnable vector; this feature is randomly initialized in step S1 and is independent of the spatial coordinates of the Gaussian elements. ; During the SGLD optimization process, The motion properties of the object to which the Gaussian element belongs are learned and encoded by updating the algorithm through backpropagation.

9. The method according to claim 5, characterized in that, Suppression strategies for background inflation caused by lack of scale constraints: During the rendering process in step S3, the projected area of ​​each Gaussian pixel on the two-dimensional image plane is calculated; when the projected area of ​​a Gaussian pixel in the screen space exceeds 50% of the total number of pixels on the screen, or when its center projection coordinates move out of the view frustum range, its opacity is forcibly set to zero or it is forcibly split.

10. A physically-aware snapshot compressed imaging three-dimensional Gaussian splash reconstruction system, characterized in that, include: The random initialization module is configured to generate random point clouds that follow a uniform or Gaussian distribution within a set 3D space without any prior geometric information, and initialize the corresponding covariance, color, opacity and latent embedding features. An embedding-driven deformation module is configured to store and update the latent embedding features of each Gaussian cell and includes two multilayer perceptron networks for predicting low-frequency rigid motion trajectories and high-frequency non-rigid integral deformations based on the embedding features, respectively. Integral imaging simulation module, configured to perform time-slice rendering and physical mask modulation; This module contains a loop accumulator for... The rendered image at each time step is multiplied and accumulated pixel-wise with the mask to output simulated SCI measurements; the SGLD sampling optimization module is configured to perform stochastic gradient Langevin dynamics updates; this module integrates a noise generator to generate isotropic Gaussian noise and includes a covariance shaping unit to convert the isotropic noise into anisotropic noise that matches the shape of the Gaussian primitives; the unconstrained density evolution module is configured to manage the splitting and extinction of Gaussian primitives; this module is specifically configured to disable regularization constraints on the scale of Gaussian primitives and includes a stochastic time sampler to perform stochastic residual inverse initialization operations when generating new primitives.

Images