An object-oriented periodic dynamic motion 4D gaussian splash reconstruction method
By separating the foreground and background of multi-view dynamic videos, identifying and tracking dynamic objects, constructing periodic deformation fields and Gaussians, and optimizing object-level reconstruction, the problem of parameter redundancy and motion distortion in complex dynamic scenes of existing 4D Gaussian splashing methods is solved, and efficient object-level reconstruction and editing are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SICHUAN VOCATIONAL & TECHN COLLEGE OF COMM
- Filing Date
- 2026-04-07
- Publication Date
- 2026-06-12
AI Technical Summary
Existing 4D Gaussian splashing methods struggle to distinguish the independent motion patterns of different dynamic objects when dealing with complex dynamic scenes. They lack semantic boundaries and cannot perform differentiated modeling for periodic motions, resulting in parameter redundancy and temporal inconsistency.
By separating the foreground and background of multi-view dynamic videos, identifying and tracking dynamic objects, generating 3D mask regions and sparse point clouds, identifying periodically moving objects, constructing periodic deformation fields, optimizing normalized 3D Gaussian and global trajectory encoding, and performing adaptive density control, object-level reconstruction is achieved.
It achieves object-level decomposition and periodic motion abstraction, reduces the number of parameters, improves the realism and detail accuracy of the reconstruction, reduces memory usage and computational overhead, and supports independent editing and replacement of dynamic objects.
Smart Images
Figure CN121982187B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision and graphics technology, specifically to an object-oriented method for reconstructing 4D Gaussian splashes of periodic dynamic motion. Background Technology
[0002] 3D Gaussian splashing, as an explicit scene representation method, has been widely used in static scene reconstruction and new perspective compositing due to its efficient rendering speed and high-fidelity reconstruction results. With the increasing demand for dynamic scene reconstruction, researchers have proposed the 4D Gaussian splashing method, which achieves modeling of dynamic content by introducing a temporal dimension or a deformable network.
[0003] However, existing 4D Gaussian splashing methods still have the following shortcomings when dealing with complex dynamic scenes: modeling the entire scene as a whole makes it difficult to distinguish the independent motion patterns of different dynamic objects; Gaussian representation lacks semantic boundaries, making it difficult to achieve object-level motion abstraction and editing; it cannot adopt differentiated motion modeling strategies for different types of dynamic objects (such as pedestrians, animals, and vehicles); and it lacks dedicated modeling for periodic motion, resulting in parameter redundancy and temporal inconsistency.
[0004] Therefore, there is an urgent need in this field for a 4D Gaussian splash reconstruction method that can achieve object-level decomposition, periodic motion abstraction, and efficient compression. Summary of the Invention
[0005] To address the aforementioned shortcomings in the existing technology, this invention provides an object-oriented method for reconstructing 4D Gaussian splashing based on periodic dynamic motion.
[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:
[0007] An object-oriented method for reconstructing 4D Gaussian splashes of periodic dynamic motion includes the following steps:
[0008] The input multi-view dynamic video is separated into foreground and background. Dynamic objects in the foreground are identified and tracked. 3D mask regions of background and dynamic objects are generated. Sparse point clouds of background and dynamic objects are extracted based on the 3D mask regions of background and dynamic objects. The sparse point clouds of dynamic objects are temporally aligned to generate a 4D point cloud sequence of dynamic objects. Dynamic objects with periodic motion are identified based on the 4D point cloud sequence of dynamic objects, and a global trajectory code of dynamic objects is generated.
[0009] Based on the sparse point cloud of the background and dynamic objects, a normalized 3D Gaussian is initialized, a periodic deformation field is constructed for dynamic objects with periodic motion, a Gaussian for dynamic objects with periodic motion is constructed based on the global trajectory encoding of the dynamic objects, the normalized 3D Gaussian, and the periodic deformation field, a Gaussian for dynamic objects with periodic motion is constructed based on the global trajectory encoding of the dynamic objects and the normalized 3D Gaussian, and the normalized 3D Gaussian is determined as the Gaussian of the background.
[0010] Gaussian splatter rendering is performed on dynamic objects with periodic motion, dynamic objects with non-periodic motion, and the background to obtain reconstructed images. The reconstruction loss is calculated, and based on the reconstruction loss, the normalized 3D Gaussian splatter, periodic deformation field, and global trajectory encoding of dynamic objects are jointly optimized through backpropagation. During the optimization process, adaptive density control is performed at the object level according to the motion complexity of the dynamic objects and the reconstruction loss to dynamically adjust the Gaussian distribution, so as to complete the object-oriented periodic dynamic motion 4D Gaussian splatter reconstruction.
[0011] Furthermore, sparse point clouds of the background and dynamic objects are extracted based on 3D mask regions of the background and dynamic objects. The specific process is as follows:
[0012] Within the 3D mask region of the background and dynamic objects, the depth value of each mask pixel is determined, and a dense depth map of the background and dynamic objects is obtained through multi-view consistency constraints and depth map fusion.
[0013] Adaptive sampling based on scene structure is used for the dense depth map of the background to obtain the 2D coordinates of the background sampling points, and key point-guided adaptive sampling is used for the dense depth map of dynamic objects to obtain the 2D coordinates of the dynamic object sampling points.
[0014] Based on the dense depth maps of the background and dynamic objects, the 2D coordinates of the background sampling points and the dynamic object sampling points and their corresponding depth values are back-projected into the 3D space through the camera intrinsic and extrinsic parameters to generate the initial sparse point cloud of the background and dynamic objects. Then, the initial sparse point cloud of the background and dynamic objects is denoised and outlier removed to extract the sparse point cloud of the background and dynamic objects.
[0015] Furthermore, based on the 4D point cloud sequence of dynamic objects, dynamic objects with periodic motion are identified, and a global trajectory code for the dynamic objects is generated. The specific process is as follows: a local coordinate system is established for each dynamic object, the motion trajectory of key parts is extracted from the 4D point cloud sequence of the dynamic object, and the motion trajectory of key parts is determined by the autocorrelation function method, Fourier spectrum analysis method, or periodic intensity evaluation method to determine whether the motion trajectory of key parts is periodic; if so, it is determined to be a dynamic object with periodic motion; otherwise, it is determined to be a dynamic object with non-periodic motion, and the position sequence of the center of the dynamic object in the world coordinate system is recorded as the global trajectory of the dynamic object. The global trajectory of the dynamic object is encoded by B-spline or parametric curve to generate the global trajectory code of the dynamic object.
[0016] Furthermore, the periodic intensity assessment method is used to determine whether the motion trajectory of key parts is periodic. The specific process is as follows:
[0017] The motion trajectory of the key parts is obtained by fast Fourier transform to obtain its corresponding spectrum, and the energy of the fundamental frequency and its first three harmonics in the spectrum and the total energy in the spectrum are determined.
[0018] The periodic intensity index value of the motion trajectory of key parts is calculated based on the energy of the fundamental frequency and its three preceding harmonics in the spectrum and the total energy in the spectrum. Its expression is as follows:
[0019]
[0020] in: This refers to the periodic intensity index value of the motion trajectory of key components. For when Time to take The maximum value, Given the current trajectory vector and delay time The mean of the dot product of the subsequent trajectory vectors, To delay time, The mean of the dot product between the current trajectory vector and its own trajectory vector. This represents the energy of the fundamental frequency and its first three harmonics in the frequency spectrum. This represents the total energy in the spectrum.
[0021] Determine whether the periodicity intensity index value of the motion trajectory of the key part is greater than 0.5; if so, it is determined to be a dynamic object with periodic motion; otherwise, it is determined to be a dynamic object with non-periodic motion.
[0022] Furthermore, a periodic deformation field is constructed for a dynamic object with periodic motion. The specific process is as follows:
[0023] A dynamic object with periodic motion is divided into multiple local rigid bodies, and the joints between two adjacent local rigid bodies are determined.
[0024] By performing time-series analysis on the rotation angle and relative displacement in the local coordinate system of the joint, the period length, phase and amplitude of each joint are obtained. Based on the period length, phase and amplitude of each joint, a lightweight MLP network is used with the position and periodic phase of a normalized 3D Gaussian as input and the local displacement as output to model the periodic motion of the joint and form a joint-level periodic deformation field to complete the construction of the periodic deformation field.
[0025] Furthermore, the reconstruction loss includes L1 loss and SSIM loss, and periodic regularization loss and motion smoothing loss are introduced;
[0026] Periodic regularization loss is used to constrain the output consistency of a periodic deformation field over a complete cycle;
[0027] Motion smoothing loss is used to constrain the temporal derivatives of the global trajectory and joint rotations.
[0028] Furthermore, the expression for the periodic regularized loss is:
[0029]
[0030] in: This is the periodic regularized loss value. The input is the position of a normalized 3D Gaussian. and periodic phase The periodic deformation field function value at time, The input is the position of a normalized 3D Gaussian. and periodic phase The periodic deformation field function value at time, L2 norm operations;
[0031] The expression for motion smoothing loss is:
[0032]
[0033] in: This represents the motion smoothing loss value. For all time Summation, For the first Gauss in time The world location coordinates.
[0034] The beneficial effects of this invention are as follows:
[0035] (1) This invention separates the foreground and background of the input multi-view dynamic video, identifies and tracks dynamic objects in the foreground, generates 3D mask regions of the background and dynamic objects, extracts sparse point clouds of the background and dynamic objects based on the 3D mask regions of the background and dynamic objects, and performs temporal alignment on the sparse point clouds of the dynamic objects to generate a 4D point cloud sequence of the dynamic objects. Based on the 4D point cloud sequence of the dynamic objects, it identifies dynamic objects with periodic motion and generates global trajectory codes for the dynamic objects. This process decomposes the multi-view dynamic video into independent dynamic objects and static backgrounds, and learns a normalized 3D Gaussian group for each independent dynamic object and static background. It also abstracts the repetitive motion of dynamic objects with periodic motion through periodic deformation fields, avoiding the redundancy brought by global unified modeling. It does not require learning a single deformation field for the entire scene, significantly reducing the number of parameters, thereby reducing memory usage and storage costs. Furthermore, users can independently edit, replace, or delete any object without affecting other elements in the scene, realizing true "modular" scene editing, which greatly facilitates content creation in applications such as virtual reality and film and television production.
[0036] (2) This invention initializes a normalized 3D Gaussian based on sparse point clouds of background and dynamic objects, constructs a periodic deformation field for dynamic objects with periodic motion, constructs a Gaussian for dynamic objects with periodic motion based on the global trajectory encoding of dynamic objects, normalized 3D Gaussian and periodic deformation field, constructs a Gaussian for dynamic objects with periodic motion based on the global trajectory encoding of dynamic objects and normalized 3D Gaussian, and determines the normalized 3D Gaussian as the Gaussian of the background; this process strictly follows the physical motion law of the object, avoids the motion distortion caused by treating the whole as a continuous deformation form in traditional methods, and maintains geometric consistency of motion between parts through joint chain transmission, which can accurately restore the alternating swing of limbs and coordinated movement of the torso in walking, running and other actions, significantly improving the realism and detail accuracy of the reconstruction;
[0037] (3) This invention renders Gaussian images of dynamic objects with periodic motion, dynamic objects with non-periodic motion, and background, and calculates the reconstruction loss. Based on the reconstruction loss, it jointly optimizes the normal state 3D Gaussian, periodic deformation field, and global trajectory encoding of dynamic objects through backpropagation. During the optimization process, adaptive density control is performed at the object level according to the motion complexity of the dynamic objects and the reconstruction loss, and the Gaussian distribution is dynamically adjusted to complete the object-oriented periodic dynamic motion 4D Gaussian splash reconstruction. This process dynamically adjusts the number of Gaussian objects according to the motion complexity, reconstruction error, and geometric details of each object, allocates the limited Gaussian resources to the areas that need the most detail, avoids the computational waste caused by global uniform distribution, and reduces training and rendering overhead while maintaining visual quality. Attached Figure Description
[0038] Figure 1 This is a schematic diagram of the process of an object-oriented 4D Gaussian splash reconstruction method for periodic dynamic motion. Detailed Implementation
[0039] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.
[0040] like Figure 1 As shown, an object-oriented method for reconstructing 4D Gaussian splashes of periodic dynamic motion includes steps S1-S3, as detailed below:
[0041] S1. Separate the foreground and background of the input multi-view dynamic video, identify and track dynamic objects in the foreground, generate 3D mask regions for the background and dynamic objects, extract sparse point clouds of the background and dynamic objects based on the 3D mask regions of the background and dynamic objects, and perform temporal alignment on the sparse point clouds of the dynamic objects to generate a 4D point cloud sequence of the dynamic objects. Based on the 4D point cloud sequence of the dynamic objects, identify dynamic objects with periodic motion and generate global trajectory codes for the dynamic objects.
[0042] In an optional embodiment of the present invention, the present invention performs foreground and background separation on the input multi-view dynamic video, identifies and tracks dynamic objects in the foreground, and generates 3D mask regions for the background and dynamic objects. The specific process is as follows: First, the input multi-view dynamic video is separated into foreground and background (including sky and environment) frame by frame through a pre-trained semantic segmentation model, and independent dynamic objects (including pedestrians, vehicles and animals) in the foreground are identified. An instance tracking algorithm is used to assign a unique ID to each dynamic object to ensure consistency across frames. Subsequently, based on multi-view geometric constraints, the 2D mask projections of each view are fused into 3D space to generate 3D mask regions for each dynamic object and background.
[0043] This invention extracts sparse point clouds of background and dynamic objects based on 3D mask regions of background and dynamic objects. The specific process is as follows: Within the 3D mask regions of background and dynamic objects, the depth value of each mask pixel is determined, and dense depth maps of background and dynamic objects are obtained through multi-view consistency constraints and depth map fusion; adaptive sampling based on scene structure is used in the dense depth map of the background to obtain the 2D coordinates of the background sampling points, and adaptive sampling guided by key points is used in the dense depth map of the dynamic objects to obtain the 2D coordinates of the dynamic object sampling points; based on the dense depth maps of background and dynamic objects, the 2D coordinates of the background sampling points and dynamic object sampling points and the corresponding depth values are back-projected into 3D space through camera intrinsic and extrinsic parameters to generate initial sparse point clouds of background and dynamic objects, and the initial sparse point clouds of background and dynamic objects are denoised and outlier removed to extract the sparse point clouds of background and dynamic objects.
[0044] This invention identifies dynamic objects with periodic motion based on 4D point cloud sequences of dynamic objects and generates a global trajectory code for the dynamic objects. The specific process is as follows: a local coordinate system is established for each dynamic object; the motion trajectory of key parts is extracted from the 4D point cloud sequence of the dynamic object; the motion trajectory of key parts is determined by autocorrelation function method, Fourier spectrum analysis method, or periodic intensity evaluation method to determine whether the motion trajectory of key parts is periodic; if so, it is determined to be a dynamic object with periodic motion; otherwise, it is determined to be a dynamic object with non-periodic motion. The position sequence of the center of the dynamic object in the world coordinate system is recorded as the global trajectory of the dynamic object. The global trajectory of the dynamic object is encoded using B-spline or parametric curve to generate a global trajectory code for the dynamic object.
[0045] Specifically, the present invention identifies different key parts for different dynamic objects, and determines the movable parts according to the dynamic type, and identifies them as key parts.
[0046] This invention uses the autocorrelation function method to determine whether the motion trajectory of key parts has periodicity. The specific process is as follows:
[0047] Calculate the current trajectory vector and delay time based on the motion trajectory of key parts. The mean of the dot product of the subsequent trajectory vectors is expressed as:
[0048]
[0049] in: Given the current trajectory vector and delay time The mean of the dot product of the subsequent trajectory vectors, To observe the length of the time window, To delay time, For key parts in time The trajectory of movement, For key parts in time The trajectory of movement;
[0050] Determine if there exists a delay time that makes the current trajectory vector equal to the delay time. The mean of the dot product of the subsequent trajectory vectors is greater than 0.6; if so, it is determined to be a dynamic object with periodic motion; otherwise, it is determined to be a dynamic object with non-periodic motion.
[0051] This invention uses Fourier spectrum analysis to determine whether the motion trajectory of a key part is periodic. The specific process is as follows: the motion trajectory of the key part is subjected to fast Fourier transform to obtain its corresponding spectrum, and the peak energy at the fundamental frequency and its harmonics in the spectrum is determined; it is determined whether the proportion of the peak energy at the fundamental frequency and its harmonics in the spectrum to the total energy in the spectrum exceeds 0.3; if so, it is determined to be a dynamic object with periodic motion, otherwise it is determined to be a dynamic object with non-periodic motion.
[0052] This invention uses a periodic intensity assessment method to determine whether the motion trajectory of a key component is periodic. The specific process is as follows:
[0053] The motion trajectory of the key parts is obtained by fast Fourier transform to obtain its corresponding spectrum, and the energy of the fundamental frequency and its first three harmonics in the spectrum and the total energy in the spectrum are determined.
[0054] The periodic intensity index value of the motion trajectory of key parts is calculated based on the energy of the fundamental frequency and its three preceding harmonics in the spectrum and the total energy in the spectrum. Its expression is as follows:
[0055]
[0056] in: This refers to the periodic intensity index value of the motion trajectory of key components. For when Time to take The maximum value, Given the current trajectory vector and delay time The mean of the dot product of the subsequent trajectory vectors, To delay time, The mean of the dot product between the current trajectory vector and its own trajectory vector. This represents the energy of the fundamental frequency and its first three harmonics in the frequency spectrum. This represents the total energy in the spectrum.
[0057] Determine whether the periodicity intensity index value of the motion trajectory of the key part is greater than 0.5; if so, it is determined to be a dynamic object with periodic motion; otherwise, it is determined to be a dynamic object with non-periodic motion.
[0058] S2. Based on the sparse point cloud of the background and dynamic objects, initialize the normalized 3D Gaussian, construct the periodic deformation field for the dynamic objects with periodic motion, construct the Gaussian of the dynamic objects with periodic motion based on the global trajectory encoding of the dynamic objects, the normalized 3D Gaussian, and the periodic deformation field, construct the Gaussian of the dynamic objects with periodic motion based on the global trajectory encoding of the dynamic objects and the normalized 3D Gaussian, and determine the normalized 3D Gaussian as the Gaussian of the background.
[0059] In an optional embodiment of the present invention, the present invention initializes a normalized 3D Gaussian based on the sparse point cloud of the background and dynamic objects. The specific process is as follows: based on the sparse point cloud of the background and dynamic objects, a normalized 3D Gaussian is initialized for each point. Specifically, the point cloud coordinates are used as the center position of the Gaussian. The set of its neighboring points is determined by K-nearest neighbor search and the covariance matrix of the local point cloud distribution is calculated. The square root of the eigenvalue of the covariance matrix is taken as the initial scale of the Gaussian in the three axes. The rotation is initialized to a unit quaternion. The spherical harmonic coefficients are obtained by averaging the pixel colors of the corresponding viewpoints. The opacity is set to a fixed initial value, thereby generating a set of geometrically aligned normalized 3D Gaussians for the background and each dynamic object.
[0060] This invention constructs a periodic deformation field for dynamic objects with periodic motion. The specific process is as follows: the dynamic object with periodic motion is divided into multiple local rigid bodies, and the joints between two adjacent local rigid bodies are determined; by performing time-series analysis on the rotation angle and relative displacement in the local coordinate system of the joint, the period length, phase and amplitude of each joint are obtained; based on the period length, phase and amplitude of each joint, a lightweight MLP network is used with the position and periodic phase of a normalized 3D Gaussian as input and the local displacement as output to model the periodic motion of the joint, forming a joint-level periodic deformation field, thus completing the construction of the periodic deformation field.
[0061] This invention constructs a Gaussian for dynamic objects with periodic motion based on global trajectory encoding, normalized 3D Gaussian, and periodic deformation fields. It also constructs a Gaussian for dynamic objects with non-periodic motion based on the same global trajectory encoding and normalized 3D Gaussian, and uses the normalized 3D Gaussian as the background Gaussian. Specifically, for a dynamic object with periodic motion, based on its normalized 3D Gaussian and global trajectory encoding, along the joint chain, the Gaussian position in each local rigid body coordinate system is gradually transformed to the object's local coordinate system using a periodic deformation field, thus obtaining the object's position in the local coordinate system. Finally, the global Gaussian is superimposed... Global trajectory encoding generates the final position and covariance of the Gaussian in the world coordinate system to construct the Gaussian of dynamic objects with periodic motion. For dynamic objects with non-periodic motion, the modeling method is selected according to their type. If it is a rigid body object, only global trajectory encoding is applied to achieve the overall transformation. If it is a non-rigid body object, the mapping from the normal state 3D Gaussian to each time step is learned through a general deformation field. After superimposing the global trajectory encoding, the world coordinates are obtained to construct the Gaussian of the dynamic object with non-periodic motion. For static backgrounds, the normal state 3D Gaussian is directly determined as the final representation, which remains unchanged in the time dimension without any motion transformation.
[0062] S3. Render the Gaussian of the dynamic object with periodic motion, the dynamic object with non-periodic motion, and the background to obtain the reconstructed image and calculate the reconstruction loss. Based on the reconstruction loss, optimize the normalized 3D Gaussian, the periodic deformation field, and the global trajectory encoding of the dynamic object through backpropagation. During the optimization process, according to the motion complexity of the dynamic object and the reconstruction loss, perform adaptive density control at the object level to dynamically adjust the Gaussian distribution to complete the object-oriented periodic dynamic motion 4D Gaussian splash reconstruction.
[0063] In an optional embodiment of the present invention, the present invention renders a Gaussian image of a dynamic object with periodic motion, a dynamic object with non-periodic motion, and a background. Specifically, the Gaussian image of the dynamic object with periodic motion, the dynamic object with non-periodic motion, and the background is input into a differentiable rasterizer, and the reconstructed image is rendered by α-mixing and depth sorting.
[0064] The reconstruction loss includes L1 loss and SSIM loss, and introduces periodic regularization loss and motion smoothing loss. Periodic regularization loss is used to constrain the output consistency of the periodic deformation field within a complete cycle; motion smoothing loss is used to constrain the temporal derivatives of the global trajectory and joint rotation. Specifically, this invention obtains the reconstruction loss by weighted summation of L1 loss, SSIM loss, periodic regularization loss, and motion smoothing loss.
[0065] The expression for periodic regularized loss is:
[0066]
[0067] in: This is the periodic regularized loss value. The input is the position of a normalized 3D Gaussian. and periodic phase The periodic deformation field function value at time, The input is the position of a normalized 3D Gaussian. and periodic phase The periodic deformation field function value at time, This is an L2 norm operation.
[0068] The expression for motion smoothing loss is:
[0069]
[0070] in: This represents the motion smoothing loss value. For all time Summation, For the first Gauss in time The world location coordinates.
[0071] This invention utilizes backpropagation to jointly optimize the normalized 3D Gaussian properties, periodic deformation field parameters, and global trajectory encoding of dynamic objects based on reconstruction loss. During optimization, adaptive density control is performed at the object level based on the motion complexity of the dynamic objects and the reconstruction loss, dynamically adjusting the Gaussian distribution to achieve object-oriented 4D Gaussian splash reconstruction of periodic dynamic motion. Specifically, based on the reconstruction loss, the normalized 3D Gaussian properties, periodic deformation field parameters, and global trajectory encoding of dynamic objects are jointly optimized via backpropagation. Then, during optimization, adaptive density control is performed at the object level based on the motion complexity and reconstruction error distribution of each dynamic object. This involves increasing the number of Gaussians in areas of intense motion or rich detail, and decreasing the number of Gaussians in redundant or flat areas, thereby dynamically adjusting the Gaussian distribution of each object to precisely match the representation capacity with the scene complexity, ultimately completing object-oriented 4D Gaussian splash reconstruction of periodic dynamic motion.
[0072] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.
Claims
1. An object-oriented method for reconstructing 4D Gaussian splashing of periodic dynamic motion, characterized in that, Includes the following steps: The input multi-view dynamic video is separated into foreground and background. Dynamic objects in the foreground are identified and tracked. 3D mask regions of background and dynamic objects are generated. Sparse point clouds of background and dynamic objects are extracted based on the 3D mask regions of background and dynamic objects. The sparse point clouds of dynamic objects are temporally aligned to generate a 4D point cloud sequence of dynamic objects. Dynamic objects with periodic motion are identified based on the 4D point cloud sequence of dynamic objects, and a global trajectory code of dynamic objects is generated. Based on the sparse point cloud of the background and dynamic objects, a normalized 3D Gaussian is initialized, a periodic deformation field is constructed for dynamic objects with periodic motion, a Gaussian for dynamic objects with periodic motion is constructed based on the global trajectory encoding of the dynamic objects, the normalized 3D Gaussian, and the periodic deformation field, a Gaussian for dynamic objects with periodic motion is constructed based on the global trajectory encoding of the dynamic objects and the normalized 3D Gaussian, and the normalized 3D Gaussian is determined as the Gaussian of the background. The specific process for constructing a periodic deformation field for a dynamic object with periodic motion is as follows: the dynamic object with periodic motion is divided into multiple local rigid bodies, and the joints between two adjacent local rigid bodies are determined; by performing time-series analysis on the rotation angle and relative displacement in the local coordinate system of the joint, the period length, phase and amplitude of each joint are obtained; based on the period length, phase and amplitude of each joint, a lightweight MLP network is used with the position and periodic phase of a normalized 3D Gaussian as input and the local displacement as output to model the periodic motion of the joint, forming a joint-level periodic deformation field to complete the construction of the periodic deformation field; Gaussian splatter rendering is performed on dynamic objects with periodic motion, dynamic objects with non-periodic motion, and the background to obtain reconstructed images. The reconstruction loss is calculated, and the normalized 3D Gaussian splatter, periodic deformation field, and global trajectory encoding of dynamic objects are jointly optimized through backpropagation based on the reconstruction loss. During the optimization process, adaptive density control is performed at the object level according to the motion complexity of the dynamic objects and the reconstruction loss to dynamically adjust the Gaussian distribution, so as to complete the object-oriented periodic dynamic motion 4D Gaussian splatter reconstruction. The reconstruction loss includes L1 loss and SSIM loss, and introduces periodic regularization loss and motion smoothing loss; periodic regularization loss is used to constrain the output consistency of the periodic deformation field within a complete cycle; motion smoothing loss is used to constrain the temporal derivatives of the global trajectory and joint rotation. The expression for periodic regularized loss is: in: This is the periodic regularized loss value. The input is the position of a normalized 3D Gaussian. and periodic phase The periodic deformation field function value at time, The input is the position of a normalized 3D Gaussian. and periodic phase The periodic deformation field function value at time, L2 norm operations; The expression for motion smoothing loss is: in: This represents the motion smoothing loss value. For all time Summation, For the first Gauss in time The world location coordinates.
2. The object-oriented periodic dynamic motion 4D Gaussian splash reconstruction method according to claim 1, characterized in that, Extracting sparse point clouds of background and dynamic objects from 3D mask regions based on background and dynamic objects, the specific process is as follows: Within the 3D mask region of the background and dynamic objects, the depth value of each mask pixel is determined, and a dense depth map of the background and dynamic objects is obtained through multi-view consistency constraints and depth map fusion. Adaptive sampling based on scene structure is used for the dense depth map of the background to obtain the 2D coordinates of the background sampling points, and key point-guided adaptive sampling is used for the dense depth map of dynamic objects to obtain the 2D coordinates of the dynamic object sampling points. Based on the dense depth maps of the background and dynamic objects, the 2D coordinates of the background sampling points and the dynamic object sampling points and their corresponding depth values are back-projected into the 3D space through the camera intrinsic and extrinsic parameters to generate the initial sparse point cloud of the background and dynamic objects. Then, the initial sparse point cloud of the background and dynamic objects is denoised and outlier removed to extract the sparse point cloud of the background and dynamic objects.
3. The object-oriented periodic dynamic motion 4D Gaussian splash reconstruction method according to claim 1, characterized in that, The process of identifying dynamic objects with periodic motion based on 4D point cloud sequences and generating global trajectory codes for these objects is as follows: A local coordinate system is established for each dynamic object. The motion trajectory of key parts is extracted from the 4D point cloud sequence of the dynamic object. The periodicity of the motion trajectory of key parts is determined by the autocorrelation function method, Fourier spectrum analysis method, or periodic intensity evaluation method. If it is periodic, the object is determined to be a dynamic object with periodic motion; otherwise, it is determined to be a dynamic object with non-periodic motion. The position sequence of the center of the dynamic object in the world coordinate system is recorded as the global trajectory of the dynamic object. The global trajectory of the dynamic object is encoded using B-splines or parametric curves to generate the global trajectory code of the dynamic object.
4. The object-oriented periodic dynamic motion 4D Gaussian splash reconstruction method according to claim 3, characterized in that, The process of determining whether the motion trajectory of key components is periodic by using the periodic intensity assessment method is as follows: The motion trajectory of the key parts is obtained by fast Fourier transform to obtain its corresponding spectrum, and the energy of the fundamental frequency and its first three harmonics in the spectrum and the total energy in the spectrum are determined. The periodic intensity index value of the motion trajectory of key parts is calculated based on the energy of the fundamental frequency and its three preceding harmonics in the spectrum and the total energy in the spectrum. Its expression is as follows: in: This refers to the periodic intensity index value of the motion trajectory of key components. For when Time to take The maximum value, Given the current trajectory vector and delay time The mean of the dot product of the subsequent trajectory vectors, To delay time, The mean of the dot product between the current trajectory vector and its own trajectory vector. This represents the energy of the fundamental frequency and its first three harmonics in the frequency spectrum. This represents the total energy in the spectrum. Determine whether the periodicity intensity index value of the motion trajectory of the key part is greater than 0.5; if so, it is determined to be a dynamic object with periodic motion; otherwise, it is determined to be a dynamic object with non-periodic motion.