An adaptive three-dimensional Gaussian splash dynamic scene reconstruction method based on local time slice anchor points

Abstract

This application discloses an adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points, belonging to the field of 3D reconstruction technology. The method includes: acquiring multi-view video and initial point cloud data; initializing standardized anchor points based on the initial point cloud; deriving 3D Gaussian primitives from the standardized anchor points to construct an initial representation of the dynamic scene; constructing time slice anchor points by introducing time parameters based on the standardized anchor points; limiting the effective interval through a lifecycle window function; comparing the model-rendered image with the real image; iteratively updating the model parameters; periodically calculating the image residual; locating the error region; and determining candidate optimization positions through back projection; selecting the anchor point to be optimized through an effective nearest neighbor matching method; executing a coarse-to-fine time segmentation strategy; modulating the effective opacity of the Gaussian primitives using relevant sets; performing screen-space Gaussian splash rendering; iteratively updating the parameters; and outputting a new perspective dynamic rendering video. This application can achieve high-fidelity reconstruction of complex dynamic scenes.

Description

Technical Field

[0001] This application relates to the field of 3D reconstruction technology, and in particular to an adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points. Background Technology

[0002] With the rapid development of applications such as virtual reality, augmented reality, free-viewpoint video, and digital twins, multi-view dynamic 3D scene modeling and novel perspective synthesis have become important research directions in computer vision and computer graphics. Existing dynamic 3D Gaussian splash modeling methods mainly follow two temporal modeling routes: The first route adopts explicit temporal evolution or trajectory modeling (such as SpaceTimeGS and FreeTimeGS), treating Gaussians as the basic units of temporal evolution. It describes the motion and attribute changes of Gaussian elements by introducing time-related parameters or trajectory functions, usually relying on global joint optimization to optimize shared Gaussian parameters across the entire time series; The second route combines normalized space modeling with temporally conditionalized mapping (such as 4DGaussian and LocalDyGS). It first defines Gaussian elements in normalized space, and then maps them to observation spaces at different times through temporally conditionalized deformation fields to establish a long-range correspondence between normalized space and observation space, maintaining the temporal consistency of Gaussian elements.

[0003] However, both of these technical approaches have significant drawbacks: the explicit temporal evolution approach, by granting Gaussian units a high degree of temporal freedom, significantly expands the parameter space, leading to a complex optimization process. In long sequences or fast dynamic scenarios, it is prone to training instability and convergence difficulties, and lacks explicit constraints and decomposition mechanisms for the temporal degrees of freedom, making it difficult to balance dynamic adaptability and optimization stability. The canonical space and temporal conditional mapping approach, while maintaining temporal consistency, relies on long-range temporal coupling in its modeling method. In dynamic and complex scenarios, it is prone to underfitting of local details, or it may need to sacrifice dynamic expressiveness for stability. Overall, existing technologies generally lack modeling mechanisms for explicit decomposition and fine control of the temporal dimension, making it difficult to achieve an effective balance between dynamic expressiveness, optimization stability, and model controllability, and thus failing to meet the demand for high-precision modeling in complex dynamic scenarios.

[0004] Therefore, there is an urgent need for a dynamic 3D Gaussian splash modeling method that can explicitly decompose and finely control the time dimension. This method can adaptively adjust the temporal modeling granularity according to the local dynamic characteristics of the scene, provide fine temporal segmentation representation in rapidly changing regions, and maintain a simple model structure in slowly changing regions. This effectively decouples long-range temporal dependencies while ensuring temporal continuity, improves dynamic adaptability and optimization stability, and achieves high-fidelity reconstruction of complex dynamic scenes. Summary of the Invention

[0005] The main purpose of this application is to provide an adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points. It aims to effectively decouple long-range time dependencies while ensuring temporal continuity, improve dynamic adaptability and optimization stability, and achieve high-fidelity reconstruction of complex dynamic scenes.

[0006] To achieve the above objectives, this application proposes an adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points, the method comprising: Acquire multi-view video and initial point cloud data corresponding to the scene to be reconstructed, initialize a set of standard anchor points based on the initial point cloud, and derive a set of three-dimensional Gaussian primitives from the standard anchor points. The standard anchor points control a set of three-dimensional Gaussian primitives and decode and generate the position, shape and color attributes of the three-dimensional Gaussian primitives. Based on the set of standard anchor points, a time center and a time width are introduced to construct time slice anchor points, and a set of time slice anchor points with time constraints is output. The time slice anchor points inherit the spatial attributes of the standard anchor points and the effective time interval of the time slice anchor points is limited by a preset life cycle window function. A model rendering image is generated based on the three-dimensional Gaussian primitive set, and the model rendering image is compared with the real image in the multi-view video. The model parameters are continuously iteratively optimized and updated according to the comparison results, and the image residual between the model rendering image and the real image is periodically calculated. After locating the modeling error area, the error information is mapped to the three-dimensional scene space to determine the candidate optimization position. Based on the candidate optimization positions, the anchor point to be optimized is selected by the effective nearest neighbor matching method, and a coarse-to-fine time segmentation strategy is performed on the anchor point to be optimized to output a set of time slice anchor points after hierarchical refinement. By combining the hierarchically refined time slice anchor point set, the three-dimensional Gaussian primitive set, and the lifecycle window function, the effective opacity of the three-dimensional Gaussian primitive is modulated, and scene rendering is completed through screen-space Gaussian splashing. All learnable parameters are iteratively updated with real images as supervision, and a new perspective dynamic rendering video of the scene to be reconstructed is output.

[0007] In one possible implementation, the initialization of a set of canonical anchor points based on the initial point cloud data, and the generation of a set of three-dimensional Gaussian primitives from the canonical anchor points, wherein the canonical anchor points manage a set of three-dimensional Gaussian primitives and decode and generate the position, shape, and color attributes of the three-dimensional Gaussian primitives, including: The spatial location information in the initial point cloud data is obtained, and the center position of the standard anchor point in the three-dimensional scene space is initialized based on the spatial location information; the standard anchor point includes spatial location, scale parameters and learnable static feature vectors, which are used as basic expression units to characterize the local spatial structure. Based on the decoding of the canonical anchor point, multiple three-dimensional Gaussian elements are derived, and the position offset, morphological parameters, opacity, and appearance attribute representation of each three-dimensional Gaussian element relative to the canonical anchor point are predicted, so as to form an expressive Gaussian set in the local spatial domain corresponding to each canonical anchor point.

[0008] In one possible implementation, based on the set of canonical anchor points, a time center and a time width are introduced to construct time slice anchor points, and a set of time slice anchor points with time constraints is output. These time slice anchor points inherit the spatial attributes of the canonical anchor points, and their effective time intervals are limited by a preset lifecycle window function, including: Obtain the spatial attributes of each standard anchor point in the set of standard anchor points. The spatial attributes include spatial location, scale and static feature vector. Retain the spatial attributes as the basic attributes of the time slice anchor points. Configure a time center and time width for each time slice anchor point, and introduce a preset life cycle window function. The time center corresponds to the core moment when the anchor point takes effect, the time width corresponds to the time range during which the anchor point takes effect, and the life cycle window function is constructed based on the time center and the time width. By integrating the basic attributes, the time center, the time width, and the lifecycle window function, a complete set of time slice anchor points is constructed, which are then aggregated to form a time slice anchor point set with time constraints.

[0009] In one possible implementation, the process involves generating a model rendering image based on the three-dimensional Gaussian meta-set, comparing the model rendering image with real images in the multi-view video, continuously iteratively optimizing and updating the model parameters based on the comparison results, and periodically calculating the image residual between the model rendering image and the real image. This is done to map the error information to the three-dimensional scene space after locating the modeling error region, thereby determining candidate optimization positions. This includes: Based on the three-dimensional Gaussian primitive set, screen-space Gaussian splash rendering is performed to generate a model rendering image; The structural similarity residuals of the real images corresponding to the time sequence of the model-rendered images in the multi-view video are calculated with those of the model-rendered images. The residual mask is then constructed by combining the edge response intensity and the residual mask is used to locate the error regions where the modeling accuracy is insufficient. The pixel coordinate information of the error region is obtained, and the pixel coordinates are back-projected into three-dimensional space in combination with the multi-view camera parameters to obtain the error region in three-dimensional space, which serves as a candidate optimization position.

[0010] In one possible implementation, the step involves selecting an anchor point to be optimized based on the candidate optimization positions using an effective nearest neighbor matching method, applying a coarse-to-fine time segmentation strategy to the anchor point to be optimized, and outputting a set of time slice anchor points with refined hierarchical structure, including: The three-dimensional coordinates of the candidate optimization position are obtained, and the position information and expressive power parameters of the existing standard anchor points and time slice anchor points are collected to calculate the spatial distance between the candidate optimization position and each anchor point. The expressive power parameters of each anchor point are combined for weighted evaluation, and the anchor point with the closest spatial distance and the best expressive power is selected as the anchor point to be optimized. For anchor points of the standard type to be optimized, coarse-grained time intervals are divided within the full time range covered by the anchor points to be optimized to generate initial time slice anchor points; For time-slice type anchor points to be optimized, fine-grained sub-time intervals are further divided within the original effective time interval of the anchor point to be optimized, generating sub-anchor points that inherit the spatial attributes of the parent anchor point and only adjust the time parameters. The initial time slice anchor points and the sub-anchor points are combined to form a hierarchically refined set of time slice anchor points.

[0011] In one possible implementation, the effective opacity of the 3D Gaussian primitives is modulated by combining the hierarchically refined set of time slice anchor points, the set of 3D Gaussian primitives, and the lifecycle window function. Scene rendering is then performed using screen-space Gaussian splashing. All learnable parameters are iteratively updated with real images as supervision, and a new perspective dynamically rendered video of the scene to be reconstructed is output, including: Obtain the set of time slice anchor points after hierarchical refinement, and decode the position, shape, and color attributes of the three-dimensional Gaussian primitives; The activation weight of the 3D Gaussian element at each time slice anchor point is calculated using the lifecycle window function, and the activation weight is multiplied by the opacity parameter of the Gaussian element itself to obtain the effective opacity. Based on the aforementioned effective opacity and screen-space Gaussian splashing technology, the visual information of all three-dimensional Gaussian primitives is integrated to complete scene rendering; Using the real images in the multi-view video as the supervision benchmark, the pixel-level reconstruction loss, structural similarity loss and edge enhancement loss between the rendered image and the real image are calculated; The model parameters are iteratively updated based on the loss function to optimize the scene representation; Based on the optimized model, a new perspective dynamic rendering video of the scene to be reconstructed is generated.

[0012] Furthermore, to achieve the above objectives, this application also proposes an adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points, wherein the adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points includes: The acquisition module is used to acquire multi-view video and initial point cloud data corresponding to the scene to be reconstructed, initialize a set of standard anchor points based on the initial point cloud data, and derive a set of three-dimensional Gaussian primitives from the standard anchor points. The standard anchor points control a set of three-dimensional Gaussian primitives and decode and generate the position, shape and color attributes of the three-dimensional Gaussian primitives. The construction module is used to construct time slice anchors based on the set of standard anchors, introduce time center and time width, and output a set of time slice anchors with time constraints. The time slice anchors inherit the spatial attributes of the standard anchors and limit the effective time interval of the time slice anchors through a preset life cycle window function. The activation module is used to generate a model rendering image based on the three-dimensional Gaussian primitive set, compare the model rendering image with the real image in the multi-view video, continuously iterate and optimize the model parameters according to the comparison results, and periodically calculate the image residual between the model rendering image and the real image so as to map the error information to the three-dimensional scene space after locating the modeling error area and determine the candidate optimization position. The segmentation module is used to select the anchor point to be optimized based on the candidate optimization position by an effective nearest neighbor matching method, perform a coarse-to-fine time segmentation strategy on the anchor point to be optimized, and output a set of time slice anchor points after hierarchical refinement. The rendering module combines the hierarchically refined time slice anchor point set, the three-dimensional Gaussian primitive set, and the lifecycle window function to modulate the effective opacity of the three-dimensional Gaussian primitives. It then completes scene rendering through screen-space Gaussian splashing, iteratively updates all learnable parameters with real images as supervision, and outputs a new perspective dynamic rendering video of the scene to be reconstructed.

[0013] Furthermore, to achieve the above objectives, this application also proposes an adaptive three-dimensional Gaussian splash dynamic scene reconstruction device based on local time slice anchor points. The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the adaptive three-dimensional Gaussian splash dynamic scene reconstruction method based on local time slice anchor points as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the adaptive three-dimensional Gaussian splash dynamic scene reconstruction method based on local time slice anchor points as described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of an adaptive three-dimensional Gaussian splash dynamic scene reconstruction method based on local time slice anchor points as described above.

[0016] This application provides an adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points. This method acquires multi-view video and initial point cloud data corresponding to the scene to be reconstructed. Based on the initial point cloud data, a set of standardized anchor points is initialized, and a set of 3D Gaussian primitives is derived from these standardized anchor points. Each standardized anchor point manages a set of 3D Gaussian primitives and decodes their position, shape, and color attributes. Then, based on the standardized anchor point set, a time center and time width are introduced to construct time slice anchor points, outputting a time slice anchor point set with time constraints. The time slice anchor points inherit the spatial attributes of the standardized anchor points, and their effective time interval is limited by a preset lifecycle window function. A model rendering image is then generated based on the 3D Gaussian primitive set. This model rendering image is compared with the real image in the multi-view video. Based on the comparison results, the model parameters are continuously iteratively optimized and updated. The image residual between the model rendering image and the real image is periodically calculated to locate the modeling error region and reduce the error signal. Information is mapped to a 3D scene space to determine candidate optimization positions. Then, based on these candidate positions, an anchor point to be optimized is selected using an effective nearest neighbor matching method. A coarse-to-fine time segmentation strategy is applied to this anchor point, outputting a hierarchically refined set of time slice anchor points. This set, combined with the hierarchically refined time slice anchor point set, the 3D Gaussian primitive set, and the lifecycle window function, modulates the effective opacity of the 3D Gaussian primitives. Scene rendering is then completed using screen-space Gaussian splashing. All learnable parameters are iteratively updated with real-world images as supervision, outputting a new perspective dynamic rendering of the scene to be reconstructed. By rendering video, long-term dynamic modeling is broken down into short-term segmented tasks, reducing temporal coupling complexity and improving optimization stability. Among these, the time slice anchor points are precisely adapted to the dynamic speed of the scene, enhancing the ability to express local fast motion details. The residual-driven and coarse-to-fine segmentation strategy can adaptively optimize the modeling granularity, reduce redundancy, and improve model controllability. At the same time, the lifecycle window and opacity modulation are combined to ensure temporal continuity and rendering consistency. Thus, while ensuring temporal continuity, long-term temporal dependencies are effectively decoupled, dynamic adaptability and optimization stability are improved, and high-fidelity reconstruction of complex dynamic scenes is achieved. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an embodiment of an adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points according to this application. Figure 2 This is a schematic diagram of the module structure of an adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points according to an embodiment of this application; Figure 3 This is a schematic diagram of the device structure of the hardware operating environment involved in the adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points in an embodiment of this application.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone; or an electronic device capable of performing the above functions, a big data service platform, or an adaptive 3D Gaussian splash dynamic scene reconstruction system based on local time slice anchor points. The following description uses an adaptive 3D Gaussian splash dynamic scene reconstruction system based on local time slice anchor points as an example to illustrate this embodiment and the subsequent embodiments.

[0024] Based on this, embodiments of this application provide an adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points, referring to... Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of an adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points according to this application.

[0025] In this embodiment, the adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points includes steps S11~S15: Step S11: Obtain multi-view video and initial point cloud data corresponding to the scene to be reconstructed; initialize a set of standard anchor points based on the initial point cloud data; and derive a set of three-dimensional Gaussian primitives from the standard anchor points. The standard anchor points control a set of three-dimensional Gaussian primitives and decode and generate the position, shape and color attributes of the three-dimensional Gaussian primitives. It should be noted that the scene to be reconstructed refers to a real scene whose dynamic changes need to be restored using 3D modeling technology, including but not limited to moving objects, changing environments, and other scenes with dynamic features in the time dimension; multi-view video refers to a continuous video sequence obtained by synchronously shooting the same scene to be reconstructed from multiple different spatial perspectives, which can provide rich spatial and temporal dynamic information of the scene; initial point cloud data refers to a data set extracted from multi-view video, consisting of a large number of discrete 3D coordinate points, each point containing spatial position information of the scene surface; 3D Gaussian primitives refer to basic rendering units with ellipsoidal shapes used to fit and stitch together to restore the scene structure and object appearance, which have attributes such as position, shape, color, and opacity; canonical anchor points refer to structured units used to manage a set of 3D Gaussian primitives and provide the basis for Gaussian primitive attribute decoding, each canonical anchor point corresponds to a local spatial region, containing information such as spatial position, scale, and static feature vectors; a canonical anchor point set refers to a whole formed by multiple canonical anchor points organized according to preset rules, used to fully cover the spatial range of the scene to be reconstructed; a 3D Gaussian primitive set refers to a whole composed of multiple 3D Gaussian primitives, which is the core foundation for constructing scene representation.

[0026] The core objective of this step is to initialize the center position of the canonical anchor points in the 3D scene space using the spatial location information provided by the initial point cloud data, and derive multiple 3D Gaussian primitives by decoding the canonical anchor points. This allows the construction of an expressive Gaussian representation structure within the local spatial domain corresponding to each anchor point, thus providing a structured initial scene representation for subsequent dynamic modeling and optimization. The implementation approach is based on the spatial distribution characteristics of the point cloud data, rationally dividing local spatial regions and configuring canonical anchor points, and then assigning specific attributes to the Gaussian primitives through the feature decoding capabilities of the anchor points, ensuring that the initial scene representation can cover the overall scene structure.

[0027] Specifically, in one possible implementation, the initial point cloud data can be obtained by calculating it using a 3D reconstruction algorithm (such as a motion recovery structure algorithm) from frame images of multi-view video, or it can be directly acquired by devices such as LiDAR. Further details of the subsequent steps can be found in steps S21-S22, and will not be repeated here.

[0028] Step S12: Based on the set of standard anchor points, introduce time center and time width to construct time slice anchor points, and output a set of time slice anchor points with time constraints. The time slice anchor points inherit the spatial attributes of the standard anchor points and limit the effective time interval of the time slice anchor points through a preset life cycle window function. It should be noted that the time center refers to the core moment when the time slice anchor takes effect, which is the central reference point of the time slice anchor's effective time interval; the time width refers to the range of time periods during which the time slice anchor is effective, used to limit the time span for the time slice anchor to take effect; the time slice anchor refers to a structured unit with an explicit time effective interval built based on the standard anchor, used to optimize the modeling of scene dynamic details within a specific time slice; the time-constrained time slice anchor set refers to a whole composed of multiple time slice anchors with different effective time intervals, used to cover the dynamic changes of the scene to be reconstructed at different times; the spatial attributes refer to the relevant features of the standard anchor in three-dimensional space, including spatial position, scale, and static feature vectors, which are the basis for the time slice anchor to accurately correspond to the scene spatial region; the preset life cycle window function refers to a function built based on the time center and time width, used to control the activation and decay state of the time slice anchor, which can achieve a smooth transition of the anchor's effective interval.

[0029] The core purpose of this step is to provide a time-dimensional control mechanism for dynamic detail modeling of the scene. By introducing time parameters on the basis of the standard anchor points to construct time slice anchor points, each time slice anchor point is only effective within a specific time interval. This breaks down long-term dynamic modeling into multiple short-term segmented modeling tasks, avoiding the problem of underfitting dynamic details caused by long-term modeling. The implementation idea is to inherit the spatial attributes of the standard anchor points to ensure spatial correspondence, limit the effective time interval through time center and time width, and then use the life cycle window function to achieve smooth control of the effective state, ensuring that the time slice anchor points can accurately adapt to the dynamic time characteristics of the scene.

[0030] For details, please refer to steps S31-S33, which will not be repeated here.

[0031] Step S13: Generate a model rendering image based on the three-dimensional Gaussian set, and compare the model rendering image with the real image in the multi-view video. Based on the comparison results, continuously iterate and optimize the model parameters, and periodically calculate the image residual between the model rendering image and the real image. After locating the modeling error area, map the error information to the three-dimensional scene space to determine the candidate optimization position. It should be noted that the model-rendered image refers to a two-dimensional image generated by a rendering algorithm based on the current three-dimensional Gaussian tuple, which can reflect the current scene representation; the real image refers to the original frame image in the multi-view video that corresponds to the model-rendered image in time sequence, and is the benchmark for measuring the model rendering effect and modeling accuracy; the comparison result refers to the differences between the model-rendered image and the real image in terms of pixel values, structural features, edge information, etc.; the model parameters refer to various parameters involved in the construction of scene representation, including attribute parameters such as static feature vectors of canonical anchor points, multi-resolution hash table parameters, and weight parameters of the decoding network, etc., which are not limited here.

[0032] Furthermore, iterative optimization and updating refers to the process of continuously adjusting model parameters to reduce the difference between the model-rendered image and the real image based on the comparison results; periodic calculation refers to calculating the image residual at preset iteration intervals or time intervals to avoid wasting computing resources due to frequent calculations, while ensuring timely capture of modeling errors; image residual refers to the quantified value of the difference between the model-rendered image and the real image, which can reflect the degree of inadequacy in modeling accuracy; modeling error region refers to the area in the model-rendered image that differs significantly from the real image and whose modeling accuracy does not meet the requirements; error information refers to relevant data such as the location, magnitude, and type of the modeling error region; 3D scene space refers to the 3D spatial coordinate system corresponding to the scene to be reconstructed; candidate optimization position refers to the core position selected in the 3D scene space that needs to be optimized for modeling accuracy through subsequent time segmentation strategies.

[0033] The core purpose of this step is to locate modeling errors and determine optimization directions. By generating model rendering images and comparing them with real images, the model parameters are iteratively optimized to improve the accuracy of the initial scene representation. At the same time, the image residuals are periodically calculated to locate error regions and mapped to three-dimensional space, providing clear optimization targets for the subsequent refinement of time slice anchor points. The implementation idea is to generate rendering images based on the current scene representation, measure the differences with real images as a benchmark, reduce the overall differences by iteratively updating parameters, and then focus on error regions with large differences, extracting their three-dimensional spatial positions as candidate optimization positions to ensure that the optimization process is highly targeted and efficient.

[0034] Specifically, at the beginning of this step, relying on the set of three-dimensional Gaussian primitives controlled only by the normative anchor points in the initial iteration stage, screen-space Gaussian splash rendering technology is used, combined with the intrinsic and extrinsic parameters of the multi-view video camera, to generate a model rendering image that perfectly matches the time sequence and perspective of the real video frames. In this stage, no time slice anchor points participate in modeling and rendering. The rendering result is determined only by the position, shape, color and other attributes of the Gaussian primitives generated by the normative anchor points through the decoding network. It can only achieve basic fitting of low-frequency static features of the scene. This rendering image will serve as the initial benchmark for subsequent residual analysis.

[0035] Subsequently, the rendered image of the model and real images from the same time and perspective in multi-view videos were used as supervision benchmarks. Pixel-level L1 reconstruction loss, structural similarity loss (SSIM) loss, and Sobel edge enhancement loss were calculated sequentially. At the same time, volume regularization loss was introduced to constrain the geometry of the 3D Gaussian elements to avoid modeling distortion caused by scale anomalies. The four types of losses were then fused into a total loss function according to preset weights. The Adam gradient descent optimization algorithm was used to perform the first round of iterative updates on all learnable model parameters, such as the attributes of the canonical anchors, the parameters of the multi-resolution hash table, and the weight parameters of the decoding network. This initially improved the accuracy of the basic scene modeling and completed the initial optimization of the model.

[0036] During the continuous iterative update of model parameters, a residual-driven triggering mechanism is introduced to adaptively determine the spatial location and temporal range of new time slice anchor points to be generated. Specifically, by comparing and analyzing the model rendering results with the corresponding real images, edge response information is extracted to highlight areas with significant structural changes, thereby enhancing the perception of high-frequency dynamic details and object contour errors. Based on this, a structural similarity residual evaluation map is further constructed to quantitatively characterize areas that are difficult for the model to fit accurately. To avoid interference from large areas of low-frequency regions in error evaluation, a joint statistical thresholding strategy is adopted, retaining only high-error regions with significantly concentrated residuals as candidate triggering regions, thus focusing the expansion of the model's expressive power on locations with real-world complex dynamic changes. Subsequently, a density gating mechanism is introduced to impose spatial continuity and minimum size constraints on candidate regions to suppress scattered error responses caused by noise fluctuations or locally unstable optimization processes, retaining only high-residual regions with stable spatial support to form a reliable residual mask. The residual mask is used to mark the local dynamic structures and areas lacking details that are difficult to fully express by the current canonical anchor points. It also serves as the trigger for subsequent time slice anchor point activation and progressive temporal refinement processes, enabling the model to perform adaptive densification modeling only in the necessary spatial locations and time windows. This effectively controls the model size and optimization stability while improving the accuracy of dynamic representation.

[0037] After completing the 2D residual region screening, high residual pixels are mapped from image space to 3D scene space to generate candidate refinement positions. Specifically, for each high residual pixel retained in the residual mask, its 2D pixel coordinates are first converted to a normalized viewing direction in the camera coordinate system based on the camera imaging model and camera intrinsic parameters, thus obtaining the corresponding spatial ray representation. Then, combined with the camera's pose information in the world coordinate system, the ray is transformed from the camera coordinate system to the 3D scene coordinate system to obtain a spatial ray in the world coordinate system. On this spatial ray, the 3D position matching the pixel observation is determined, thus generating the corresponding 3D candidate point.

[0038] By performing the aforementioned backprojection process on all high residual pixels, a set of 3D candidate locations distributed near the true error region is constructed. This set is used to indicate locations where the model consistently exhibits underfitting within local spatial and temporal ranges, and serves as the spatial basis for subsequent effective nearest neighbor anchor matching and time-slice refinement operations, achieving a direct mapping from the 2D residual signal to the 3D candidate refinement locations.

[0039] Step S14: Based on the candidate optimization positions, select the anchor point to be optimized using the effective nearest neighbor matching method, perform a coarse-to-fine time segmentation strategy on the anchor point to be optimized, and output a set of time slice anchor points after hierarchical refinement. It should be noted that the effective nearest neighbor matching method refers to a matching algorithm that combines the expressive power of anchor points with spatial distance, used to select the anchor point that best matches the candidate optimization position from existing anchor points; the anchor point to be optimized refers to the anchor point selected by the effective nearest neighbor matching method that needs to be time-segmented to improve modeling accuracy, including standard anchor points and existing time slice anchor points; the coarse-to-fine time segmentation strategy refers to a hierarchical time partitioning method, first performing coarse-grained time segmentation on standard anchor points, and then performing fine-grained time segmentation on existing time slice anchor points, gradually improving the granularity of time modeling; the set of time slice anchor points after hierarchical refinement refers to the whole set of time slice anchor points after time segmentation, including coarse-grained, fine-grained and other different levels of time slice anchor points, which can more accurately adapt to the dynamic time characteristics of the scene.

[0040] For details, please refer to steps S51-S54, which will not be repeated here.

[0041] Step S15: Combining the hierarchically refined time slice anchor point set, the three-dimensional Gaussian primitive set, and the lifecycle window function, modulate the effective opacity of the three-dimensional Gaussian primitives, complete scene rendering through screen-space Gaussian splashing, iteratively update all learnable parameters with real images as supervision, and output a new perspective dynamic rendering video of the scene to be reconstructed.

[0042] It should be noted that effective opacity refers to the actual opacity value of the 3D Gaussian primitives participating in rendering at the current time sequence, which is the product of the Gaussian primitive's own opacity and the activation weight of the lifecycle window function; screen-space Gaussian splashing refers to an efficient rendering technique that can map the attribute information (position, shape, color, effective opacity) of 3D Gaussian primitives to a 2D image plane to generate high-quality scene rendering images; scene rendering refers to the process of converting a 3D scene representation into a 2D visual image, which is the core link in presenting the modeling results; all anchor points refer to all anchor points in the set of normalized anchor points and the set of time slice anchor points after hierarchical refinement; iterative update refers to the process of continuously adjusting all anchor points and 3D Gaussian primitive parameters with real images as supervision to reduce the difference between the rendered image and the real image; new perspective dynamic rendering video refers to a video that can show the dynamic changes of the scene to be reconstructed from any unshot perspective, which is the final output of dynamic scene reconstruction and can meet the application needs of virtual reality, free viewpoint video, etc.

[0043] The core purpose of this step is to integrate all modeling units to complete the final scene rendering, and improve the modeling accuracy through iterative optimization, ultimately outputting a high-quality dynamic rendering video from a new perspective. The implementation idea is to combine the time slice anchor point set after hierarchical refinement, the three-dimensional Gaussian primitive set, and the life cycle window function to accurately modulate the effective opacity of each three-dimensional Gaussian primitive, and then complete the scene rendering through screen space Gaussian splashing technology. The parameters are continuously updated iteratively with real images as supervision to ensure that the rendering results are highly consistent with the real scene in terms of dynamic details, structural accuracy, etc.

[0044] For details, please refer to steps S61-S66, which will not be repeated here.

[0045] This embodiment reduces temporal coupling complexity and improves optimization stability by splitting long-term dynamic modeling into short-term segmented tasks. Specifically, the time slice anchor points are precisely adapted to the dynamic speed of the scene, enhancing the ability to express details of local rapid motion. The residual-driven and coarse-to-fine segmentation strategy can adaptively optimize the modeling granularity, reduce redundancy, and improve model controllability. At the same time, the lifecycle window and opacity modulation are combined to ensure temporal continuity and rendering consistency. Thus, while ensuring temporal continuity, long-term temporal dependencies are effectively decoupled, dynamic adaptability and optimization stability are improved, and high-fidelity reconstruction of complex dynamic scenes is achieved.

[0046] Understandably, this solution is based on a 3D Gaussian splash dynamic scene reconstruction framework, using multi-view video as the core training and supervision data. In the initial stage, a set of standardized anchor points is initialized based on the scene point cloud positions, and a set of 3D Gaussian primitives is derived from these anchor points. These Gaussian primitives exist in an ellipsoidal form, possessing different orientation, shape, and color attributes, used to fit and stitch together to reconstruct objects and the overall structure of the scene. The anchor points set in this solution are not independent modeling units, but rather act as managers, managing a group of 3D Gaussian primitives. The position, shape, color, and other attributes of the Gaussian primitives are all decoded and generated by the anchor points, thus forming the basic modeling structure of the scene. Building upon this, the solution introduces time-slice anchor points as a core improvement. These anchor points inherit the spatial attributes of the standardized anchor points while adding time dimension parameters such as time center and time width. Activation control within a specific time interval is achieved through a lifecycle window function, replacing the original method of a single anchor point managing a group of Gaussian primitives over a long period with segmented time management. This allows the dynamic details of different time periods to be independently managed by the corresponding time-slice anchor points, improving the modeling accuracy of rapidly changing areas.

[0047] During the modeling and optimization process, a residual-driven coarse-to-fine time segmentation mechanism is adopted. First, the model-rendered image is compared with real multi-view images. The residual region is obtained through edge perception and similarity calculation. The error region is back-projected onto the three-dimensional space to determine the candidate position. Then, the anchor point to be optimized is selected based on the effective nearest neighbor matching rule. Coarse-grained time slice anchor points are generated for the standard anchor points to cover the entire dynamic interval. The existing time slice anchor points are further subdivided into sub-anchor points for shorter time intervals, so as to achieve adaptive refinement of the dynamic modeling granularity.

[0048] Finally, in the rendering stage, by combining the Gaussian meta-attributes of anchor point management with the lifecycle control of time slice anchor points, scene rendering is completed through Gaussian effective opacity modulation and screen space splashing. Multi-view video frames are used as supervision signals to continuously iterate and optimize model parameters, correct modeling errors, and ultimately achieve a new perspective for continuous and high-quality rendering of dynamic scenes, outputting corresponding dynamic videos. The overall solution, based on the existing 3D Gaussian reconstruction framework, solves the problems of insufficient dynamic detail expression and limited modeling accuracy caused by traditional long-term single anchor point management through time slice anchor points and segmented time management mechanisms, making the modeling and rendering of dynamic scenes more refined, stable, and adaptable to complex and changing scenes.

[0049] In one feasible implementation, the initialization of a set of standardized anchor points based on the initial point cloud data, and the generation of a set of three-dimensional Gaussian primitives from the standardized anchor points, wherein the standardized anchor points manage a set of three-dimensional Gaussian primitives and decode and generate the position, shape, and color attributes of the three-dimensional Gaussian primitives, including: Step S21: Obtain spatial location information from the initial point cloud data, and initialize the center position of the standard anchor point in the three-dimensional scene space based on the spatial location information; the standard anchor point includes spatial location, scale parameters and learnable static feature vectors, which are used as basic expression units to characterize the local spatial structure. It should be noted that the initial point cloud data refers to the sparse 3D reconstruction results obtained from motion reconstruction methods based on multi-view images, used to provide initial geometric support and camera pose reference for dynamic scene modeling. This point cloud, in discrete 3D coordinates, characterizes the locations of key structures in the scene with stable observability, serving as the geometric starting point for subsequent canonical anchor initialization and scene representation optimization. Spatial location information corresponds to the spatial coordinates of each discrete point in the point cloud in the 3D coordinate system, used to determine the initial center position of the canonical anchor in the scene space. 3D Gaussian primitives are derived from the canonical anchors, their spatial centers determined by the anchor positions and predicted offsets, and geometrically modeled using an ellipsoidal shape. Their shape and orientation are controlled by scale and orientation parameters, thus adaptively characterizing the local spatial distribution features of different objects and structures in the scene.

[0050] A canonical anchor point is a structured unit with attributes such as spatial location, scale, and static feature vectors. It does not directly participate in rendering but acts as a control center, associating a group of 3D Gaussian primitives and providing a foundation for Gaussian primitive attribute decoding. The preset quantity refers to the number of 3D Gaussian primitives associated with each canonical anchor point, which is a pre-defined fixed value or a dynamically adjustable range, adjustable according to scene complexity and modeling accuracy requirements. A control group refers to a set consisting of a canonical anchor point and all 3D Gaussian primitives associated with it. The canonical anchor point implements unified control and attribute decoding for the 3D Gaussian primitives within the control group. Spatial location refers to the coordinates of the canonical anchor point in the 3D scene space, used to determine the local spatial region corresponding to the canonical anchor point. Scale refers to the size of the local spatial region corresponding to the canonical anchor point, used to limit the spatial distribution range of associated 3D Gaussian primitives. Static feature vectors refer to the learnable feature vectors inherent to the canonical anchor point, whose dimensions can be set according to modeling requirements (e.g., 256-dimensional, 512-dimensional), serving as the basis for decoding and generating 3D Gaussian primitive attributes.

[0051] The core objective of this step is to construct a structured set of canonical anchor points based on the spatial distribution of the initial point cloud data, establishing the association between these canonical anchor points and 3D Gaussian primitives. This provides a foundation for subsequent attribute optimization. The approach involves using the spatial location information of the initial point cloud data to reflect the spatial density distribution of the scene. More canonical anchor points are generated in dense point cloud regions to ensure local modeling accuracy, while the number of canonical anchor points is appropriately reduced in sparse point cloud regions to control computational costs. This ensures that the set of canonical anchor points can efficiently and comprehensively cover the scene to be reconstructed. Each canonical anchor point is associated with 3D Gaussian primitives within its spatial range to form a control group. The control range is limited by the scale parameters of the canonical anchor points, ensuring that each 3D Gaussian primitive can be controlled by its corresponding canonical anchor point, achieving structured partitioning and efficient management of the scene space.

[0052] Specifically, in one embodiment, ScaffoldGS proposes a 3D Gaussian splashing modeling framework based on "anchors" to improve the structure and efficiency of the 3D Gaussian splashing method in the modeling and rendering process. In this framework, the scene is first organized by a set of sparse canonical anchors, each describing a small spatial region around it. The canonical anchors themselves do not directly serve as rendering primitives, but rather as structural centers used to generate a set of associated 3D Gaussian primitives. Each anchor contains spatial location, scale, and additional feature information. The position and attributes of the 3D Gaussian primitives are obtained by decoding the anchor parameters through a network, thereby achieving efficient representation of local regions.

[0053] Furthermore, building upon ScaffoldGS, LocalDyGS extends this anchor-point structure to dynamic scene modeling. While still dividing the scene space around anchor points, LocalDyGS further divides the scene into multiple anchor-dominated local regions, modeling dynamic changes within each region. To describe time-varying dynamic information, LocalDyGS introduces time-dependent dynamic features, modeled using a multi-resolution hash table, and fuses them with the static features of the anchor points to generate Gaussian properties at different times. Simultaneously, to balance the influence of static and dynamic information across different regions and times, LocalDyGS uses a weighting mechanism to adjust both, enabling the model to express local dynamic changes while maintaining overall stability.

[0054] Overall, ScaffoldGS provides a structured anchor-Gaussian generation framework for efficiently organizing 3D Gaussian representations; while LocalDyGS introduces the idea of ​​local dynamic modeling on this basis, enabling each anchor point to adapt to scene changes in the time dimension, providing a basic structure for subsequent more refined time modeling methods.

[0055] Step S22: Based on the decoding of the canonical anchor point, derive multiple three-dimensional Gaussian elements, and predict their position offset, morphological parameters, opacity and appearance attribute representations relative to the canonical anchor point, thereby forming an expressive Gaussian set in the local spatial domain corresponding to each canonical anchor point.

[0056] It should be noted that: position offset refers to the offset coordinate value of the 3D Gaussian primitive relative to the standard anchor point in space, used to precisely adjust the spatial position of the 3D Gaussian primitive so that it can fit the local shape of the object; shape adjustment parameters refer to the parameters used to adjust the ellipsoidal shape of the 3D Gaussian primitive, controlling the scale and spatial orientation of the Gaussian distribution along each principal axis, enabling the 3D Gaussian primitive to adapt to the local geometric features of different objects; color mapping information refers to the information used to determine the color attributes of the 3D Gaussian primitive, including RGB color values ​​or color feature vectors, enabling the 3D Gaussian primitive to present colors consistent with the real scene; final position refers to the final coordinate value of the 3D Gaussian primitive in the 3D scene space after position offset adjustment; final shape refers to the ellipsoidal shape presented by the 3D Gaussian primitive after shape adjustment parameter optimization; final color attribute refers to the RGB color value determined by the 3D Gaussian primitive based on color mapping information, used to achieve realistic rendering of the scene.

[0057] The core objective of this step is to assign precise position, shape, and color attributes to each 3D Gaussian primitive within the control group by standardizing anchor points, so that the 3D Gaussian primitive set can initially present the geometric shape and appearance features of the scene to be reconstructed. The idea is to use the nonlinear mapping capability of the decoding network to transform the static feature vector of the standard anchor points and the dynamic features decoded from the corresponding multi-resolution hash grid into specific attribute parameters of the 3D Gaussian primitives, and establish the relationship between the standard anchor points and the attributes of the 3D Gaussian primitives.

[0058] Specifically, for any canonical anchor point, its static feature vector and the dynamic features output by the corresponding multi-resolution hash grid are first fused and input into a pre-defined lightweight decoding network. This decoding network consists of multiple fully connected layers and nonlinear activation functions, used to map the high-dimensional feature information carried by the anchor point into a set of attribute parameters for three-dimensional Gaussian primitives. The decoding network predicts the positional offset of each three-dimensional Gaussian primitive relative to the center position of the canonical anchor point, as well as the scale and orientation parameters that characterize its spatial expansion shape and orientation, while generating corresponding opacity parameters and appearance attribute representations. The final spatial position of each three-dimensional Gaussian primitive is obtained by vector superposition of its offset and the spatial position of the canonical anchor point, ensuring that the three-dimensional Gaussian primitives can be accurately distributed within the local spatial region corresponding to the canonical anchor point; the scale and orientation parameters jointly control the ellipsoidal shape of the Gaussian primitive in three-dimensional space, so that the final shape of the three-dimensional Gaussian primitive can fit the geometric features of the object in the corresponding local region; the appearance attributes are used to restore the color and visual detail representation consistent with the real scene during the rendering process. Throughout the modeling process, the weight parameters of the decoding network, the parameters of the multi-resolution hash grid, and the static feature vectors of the canonical anchors are all used as learnable variables in end-to-end training. The network is continuously iterated and updated through backpropagation of the rendering error-related loss function, so that the generated 3D Gaussian primitives gradually approximate the real dynamic scene in terms of spatial distribution, geometric shape, and appearance, thus achieving high-quality 3D scene reconstruction and representation optimization.

[0059] In one feasible implementation, based on the set of canonical anchor points, a time center and a time width are introduced to construct time slice anchor points, and a set of time slice anchor points with time constraints is output. The time slice anchor points inherit the spatial attributes of the canonical anchor points, and the effective time interval of the time slice anchor points is limited by a preset lifecycle window function, including: Step S31: Obtain the spatial attributes of each standard anchor point in the standard anchor point set. The spatial attributes include spatial location, scale, and static feature vector. The spatial attributes are retained as the basic attributes of the time slice anchor points. It should be noted that in this embodiment, the construction of time slice anchors is based on the initial canonical anchor framework of the iteration. Specifically, the dynamic scene modeling iterative training of this scheme starts with a basic framework containing only canonical anchors as the initial state. At this time, no time slice anchors have been constructed yet. All three-dimensional Gaussian primitives are only controlled by canonical anchors. Canonical anchors have no explicit time-effective parameters. They rely solely on spatial attributes and static feature vectors to achieve low-frequency static modeling and basic dynamic fitting of the scene. This is also the core basis for subsequent time slice anchor generation, activation, and hierarchical refinement. After iterative training begins, time slice anchors are not generated globally directly. Instead, the initial model rendering image is obtained periodically through screen-space Gaussian splash rendering. The high-frequency dynamic regions with insufficient modeling accuracy are located by combining residual analysis of the real image. Then, based on the residual-driven activation mechanism, time slice anchors are derived from the normal anchors, completing the first generation of time slice anchors. Subsequently, the time slices are further refined through a coarse-to-fine time segmentation strategy, gradually building a hierarchical set of time slice anchors. This achieves iterative growth from "no time slice anchors" to "coarse-grained time slice anchors" and then to "fine-grained time slice anchors." The entire process only generates and refines anchors in high residual regions, ensuring the sparsity and efficiency of the modeling.

[0060] Furthermore, the canonical anchor set refers to a collection of multiple canonical anchors organized according to the spatial distribution rules of the scene. Each canonical anchor corresponds to a local spatial region of the scene and is used to control the 3D Gaussian primitives within that region. Each canonical anchor possesses unique spatial attributes to adapt to the modeling needs of the corresponding local region. Spatial attributes refer to the set of attributes of the canonical anchor in the 3D scene space related to its position, range, and features. These attributes are the core foundation for the canonical anchor to achieve local spatial control and Gaussian primitive attribute decoding. Temporal slice anchors refer to structured units built based on canonical anchors that have explicit temporal effective intervals. These are used to optimize dynamic detail modeling within a specific time slice. Basic attributes refer to the spatial attributes inherited from the canonical anchors. These attributes are the prerequisite for the temporal slice anchors to accurately correspond to the scene spatial region and maintain collaborative modeling with the canonical anchors.

[0061] The core purpose of this step is to provide a spatial dimension foundation for time slice anchors. By inheriting the spatial attributes of the standard anchors, it ensures that the time slice anchors can be precisely aligned with the standard anchors in space, jointly covering local areas of the scene, and avoiding modeling gaps caused by inconsistent spatial attributes. The implementation idea is to extract the core spatial attributes of each standard anchor in the standard anchor set and use them directly as the basic attributes of the time slice anchors. There is no need to redesign spatial parameters, which not only ensures the consistency of spatial modeling, but also simplifies the construction process of time slice anchors.

[0062] Specifically, the spatial attributes of the standard anchor points will be defined as the basic attributes of the time slice anchor points. Subsequently, only time-related parameters need to be added to complete the construction of the time slice anchor points, ensuring that the time slice anchor points and the standard anchor points maintain a high degree of consistency in the spatial dimension, thus laying the foundation for collaborative modeling.

[0063] Step S32: Configure a time center and time width for each time slice anchor point, and introduce a preset lifecycle window function. The time center corresponds to the core moment when the anchor point takes effect, the time width corresponds to the time range during which the anchor point takes effect, and the lifecycle window function is constructed based on the time center and the time width. It should be noted that configuring time-related parameters for each time slice anchor point allows each anchor point to correspond to different time effective intervals, adapting to the dynamically changing spatiotemporal characteristics of the scene; the time center refers to the core moment when the time slice anchor point takes effect, which is the midpoint of the time interval in which the anchor point takes effect, used to clarify the core time reference for the anchor point to play its role; the time width refers to the range of time periods in which the time slice anchor point takes effect, used to limit the time boundary of the anchor point's effect; the preset lifecycle window function refers to a pre-designed mathematical function used to control the activation and decay states of the time slice anchor point, enabling a smooth transition of the anchor point's effective interval and avoiding dynamic modeling discontinuities caused by hard cuts; anchor point The core moment of effectiveness refers to the critical time point corresponding to the time center of the time slice anchor point. At this moment, its activation weight reaches its maximum value, making the 3D Gaussian primitives derived from the anchor point most expressive of local dynamic details. The effective time range of the anchor point refers to the time interval during which the time slice anchor point can adjust the attributes of the 3D Gaussian primitives. Within this interval, the anchor point is in an active state, and outside the interval, it is in a decaying or inactive state. Construction based on time center and time width refers to using the time center as the core parameter of the function and the time width as the interval parameter of the function, and substituting them into the preset function formula to form a lifecycle window function that can precisely control the effective state of the anchor point time.

[0064] The core purpose of this step is to give time slice anchors control over the time dimension. By configuring the time center and time width, the effective time interval of the anchor is defined. Then, the effective state is smoothly controlled through the lifecycle window function, so that the time slice anchors can accurately adapt to the dynamic time characteristics of the scene. The implementation idea is to configure appropriate time parameters for the dynamic characteristics of the spatial region corresponding to each time slice anchor, and then associate the time parameters with the anchor activation state through a smooth lifecycle window function to ensure that the anchor is effective only within a specific time interval, thus providing support for segmented dynamic modeling.

[0065] Step S33: Integrate the basic attributes, the time center, the time width, and the lifecycle window function to construct a complete time slice anchor point, and summarize them to form a set of time slice anchor points with time constraints.

[0066] It should be noted that fusion refers to integrating parameters from different dimensions, such as basic attributes, time center, time width, and lifecycle window function, to form a unified parameter set that can fully describe the characteristics of time slice anchors. Basic attributes refer to the spatial location, scale, and static feature vectors inherited from the standard anchor, which are the spatial basis of the time slice anchor. Time center refers to the core moment when the time slice anchor takes effect, which is the core reference parameter of the time dimension. Time width refers to the time range during which the time slice anchor is effective, used to limit the boundary of time action. Lifecycle window function refers to the mathematical function that controls the activation and decay state of the time slice anchor, which is the core of time dimension control. A complete time slice anchor refers to a structured unit with spatial attributes, time parameters, and time control functions, which can play a dynamic modeling and optimization role in a specific spatial region and time interval. Aggregation refers to organizing all independent time slices into a unified set. A time slice anchor set with time constraints refers to a whole composed of multiple time slice anchors with different time effective intervals. Each anchor is only effective within its corresponding time interval, collectively covering the dynamic changes of different time periods in the scene.

[0067] The core objective of this step is to construct a complete set of time slice anchors, achieving an organic combination of spatial attributes and temporal parameters. This enables each time slice anchor to accurately optimize dynamic detail modeling within a specific spatiotemporal range. The approach involves combining the spatial support provided by basic attributes with the temporal control capabilities provided by the time center, time width, and lifecycle window function, forming time slice anchors that combine spatial positioning and temporal constraints. These anchors are then aggregated to form a set of anchors covering the entire spatiotemporal range of the scene, providing complete structured support for segmented dynamic modeling.

[0068] Specifically, to enhance temporal representation capabilities, this invention proposes Time-Sliced ​​Anchors (TSA). TSA maintains consistency with canonical anchors in its local modeling form but further introduces explicit lifetime parameters. In scene modeling, canonical anchors effectively describe relatively slowly changing scene components, while TSA, as a complementary branch, enhances rapidly changing details that canonical anchors struggle to accurately fit, thus forming a collaborative representation. Based on the representation of canonical anchors, TSA's attributes are defined... for:

[0069] in, Indicates the center position in three-dimensional space. Represents a static eigenvector. This indicates the scope or local scale of influence in space, used to define the size of the spatial area affected. , These are newly added lifecycle parameters, representing the time center and time width, respectively.

[0070] The parameters described above explicitly define the activation range of TSA on the normalized time axis, thus ensuring that it is effective only within specific time slices. In this way, TSA significantly enhances its ability to model fine-grained temporal dynamics while inheriting the canonical anchor representation framework.

[0071] Furthermore, in order to explicitly control the activation range of TSA, this paper defines the following lifecycle window function. :

[0072] Where t represents the normalized time. Indicates the center of time. Half the width of the lifecycle, This is the smoothing coefficient.

[0073] This function can be viewed as a smooth approximation of a rectangular time gate function: it remains active within the interval and rapidly decays to zero outside the interval. Compared to a rigid rectangular window, this smooth form provides a more stable transition at the boundaries, while its approximately square shape provides TSA with clear and interpretable time-on / off control capabilities.

[0074] Furthermore, in the TSA modeling process, appearance and geometric parameters are determined through static features. With dynamic characteristics The fusion generation of static features. For learnable vectors attached to each TSA, and dynamic features This is obtained by performing multi-resolution hash encoding on the spatiotemporal coordinates:

[0075]

[0076] in, To form a spatiotemporal joint feature representation by inputting the spatial location of anchor points and their corresponding temporal information into a multi-layer hash coding network, This represents the feature vector output by the L-th layer hash coding module under given spatial location and time conditions.

[0077] Subsequently, static and dynamic features are fused, and corresponding Gaussian parameters are generated through a lightweight decoding network. During rendering, the activation state of the Gaussian parameter obtained by TSA decoding is simultaneously affected by its opacity. With lifecycle window Joint control.

[0078] Specifically, its effective opacity Defined as:

[0079] This embodiment realizes the entire process of establishing temporal slice anchors, from spatial foundation construction to temporal parameter configuration and the formation of a complete set. By inheriting the spatial attributes of standard anchors, it ensures spatial consistency between the temporal slice anchors and the original scene modeling framework; by configuring the time center, time width, and lifecycle window function, it endows the anchors with precise temporal control capabilities; and by aggregating them into a time-constrained set of anchors, it achieves comprehensive coverage of dynamic changes in different spatiotemporal regions of the scene. This design breaks down long-term dynamic modeling into multiple short-term segmented tasks, effectively reducing the modeling difficulty caused by long-term temporal coupling, and ultimately improving the detail fidelity and training stability of dynamic scene reconstruction.

[0080] In one feasible implementation, the process involves generating a model rendering image based on the three-dimensional Gaussian meta-set, comparing the model rendering image with real images in the multi-view video, continuously iteratively optimizing and updating the model parameters based on the comparison results, and periodically calculating the image residual between the model rendering image and the real image. This is done to map the error information to the three-dimensional scene space after locating the modeling error region and determining candidate optimization positions. The process includes: Step S41: Perform screen-space Gaussian splash rendering based on the three-dimensional Gaussian meta-set to generate a model rendering image; It should be noted that screen-space Gaussian splash rendering refers to a highly efficient 3D rendering technique that maps the attribute information of 3D Gaussian primitives to a 2D screen space. This involves projecting, weighting, and blending each Gaussian primitive to render it as a pixel. It is also the core rendering implementation method of 3D Gaussian splashing. The core purpose of this step is to transform the 3D scene representation into a 2D visual image, providing an intuitive comparison for subsequent residual calculations and error localization. The implementation relies on the efficiency and high fidelity of screen-space Gaussian splash rendering, fully utilizing the spatial, color, and other attribute information of the 3D Gaussian primitive set. The 3D Gaussian primitives are projected onto a 2D screen and pixel-level blended to generate a model rendering image corresponding to the multi-view video sequence, ensuring that the rendering result accurately reflects the current scene modeling state.

[0081] Specifically, in one embodiment, firstly, all attribute data of the 3D Gaussian primitive set is loaded, including the spatial position, ellipsoidal shape parameters, RGB color values, and opacity parameters of each 3D Gaussian primitive. Simultaneously, the camera intrinsic and extrinsic parameters corresponding to the viewpoint to be rendered are determined. The spatial position of each 3D Gaussian primitive is converted from the world coordinate system to the camera coordinate system, and then converted to the screen pixel coordinate system using the camera's perspective projection matrix. The pixel coordinates of each Gaussian primitive in screen space are calculated, and the geometric parameters of its projection ellipse in screen space are calculated in conjunction with the shape parameters of the Gaussian primitive. For each pixel in screen space, all 3D Gaussian primitives with non-zero projection response at that pixel are traversed. The corresponding 2D Gaussian response intensity is calculated based on the distribution of its projection ellipse in screen space, and the color contribution of each Gaussian primitive is weighted and forward-combined with the opacity parameter and the transmittance generated by depth sorting. By accumulating and fusing the color contributions of all relevant Gaussian primitives, the final rendered color value of the pixel is obtained. After traversing all pixels in screen space and completing the pixel color calculation, a model rendering image corresponding to the time sequence of the multi-view video is output, ensuring that the viewpoint and resolution of this image are consistent with the real image used for subsequent comparison.

[0082] Step S42: Calculate the structural similarity residual between the real image in the multi-view video that corresponds to the time sequence of the model rendering image and the model rendering image, and construct a residual mask by combining the edge response intensity. Use the residual mask to locate the error area where the modeling accuracy is insufficient. It should be noted that the real image corresponding to the time sequence refers to the original frame image in the multi-view video that is at the same time point and from the same shooting angle as the model-rendered image, and serves as the true benchmark for measuring modeling accuracy; the structural similarity residual refers to the quantitative value of the difference between the model-rendered image and the real image calculated based on the structural similarity index (SSIM), which can comprehensively reflect the differences between the two images in terms of brightness, contrast, and structural features; the edge response intensity refers to the pixel gradient magnitude of the image edge region extracted by the edge detection algorithm, which can reflect the clarity of the object contour and detail boundaries in the image; the residual mask refers to the binary or grayscale mask image constructed based on the structural similarity residual and the edge response intensity, used to mark the areas in the model-rendered image that differ significantly from the real image; the error area with insufficient modeling accuracy refers to the area in the model-rendered image marked by the residual mask that has significant differences from the real image in terms of structure, edges, brightness, etc., and is the core area that needs to be optimized for modeling in the future.

[0083] The core purpose of this step is to accurately locate the areas in the rendered image where the modeling accuracy is substandard. This provides a two-dimensional image-level error basis for mapping the error to three-dimensional space and determining the optimization position. The approach is to quantify the image differences from two dimensions: structural similarity and edge details. By constructing a residual mask, areas with significant differences are selected, thereby achieving accurate location of the modeling error area and ensuring that the subsequent optimization process is highly targeted.

[0084] Specifically, during training, this invention introduces a residual-based adaptive mechanism to determine when and where new temporal slice anchors (TSAs) need to be generated. Specifically, the Sobel edge responses are first calculated for both the model-predicted image and the real image, and an overall edge mask is constructed based on the joint quantile threshold of edge intensity. Subsequently, structural similarity (SSIM) residuals are calculated under the constraints of this mask, and the region with the highest residual values ​​is selected as a candidate high-error region.

[0085] To avoid unnecessary anchor point generation caused by noise or sporadic errors, this invention further introduces a density constraint step, retaining only high residual regions that meet connectivity and minimum area requirements, thereby ensuring that the triggering region has real structural significance.

[0086] Step S43: Obtain the pixel coordinate information of the error region, and combine the multi-view camera parameters to back-project the pixel coordinates to three-dimensional space to obtain the error region in three-dimensional space as a candidate optimization position.

[0087] It should be noted that pixel coordinate information refers to the x and y coordinates of all pixels within the error region in the two-dimensional pixel coordinate system of the rendered image, which is the basis for mapping two-dimensional errors to three-dimensional space; multi-view camera parameters refer to the intrinsic and extrinsic parameters of the camera that captures multi-view videos. The camera intrinsic parameters include focal length, principal point coordinates, pixel size, etc., used to describe the optical characteristics of the camera, while the camera extrinsic parameters include rotation matrix and translation vector, used to describe the spatial position and attitude of the camera in the world coordinate system; back projection refers to the process of mapping pixels in the two-dimensional screen pixel coordinate system back to the three-dimensional world coordinate system, which is the core operation for realizing the transformation of the two-dimensional error region to three-dimensional space; three-dimensional space refers to the three-dimensional world coordinate system corresponding to the dynamic scene to be reconstructed, which contains the spatial position information of all objects in the scene; the range of the error region in three-dimensional space refers to the three-dimensional space region formed after back-projecting all pixels of the two-dimensional error region to the three-dimensional space, which corresponds to the physical spatial position of insufficient modeling accuracy in the scene.

[0088] The core objective of this step is to transform the modeling error region at the 2D image level into an error region in the 3D scene space, providing a spatial basis for subsequently determining 3D candidate optimization positions. The approach involves extracting the pixel coordinates of the error region, combining the camera's intrinsic and extrinsic parameters, and using a back-projection algorithm to map each high residual pixel into a spatial ray in the 3D scene space. This transforms the 2D error signal into corresponding 3D spatial constraint information, indicating the location where the model may be underfitting in a local area. This achieves the error localization transformation from 2D to 3D, ensuring that subsequent optimizations can accurately apply to the 3D physical space of the scene.

[0089] Specifically, firstly, all pixels in the residual mask are traversed, and the two-dimensional pixel coordinates marked as high residual regions are extracted and organized into an error pixel coordinate set. Then, camera intrinsic and extrinsic parameters consistent with the model rendering perspective are loaded, and back projection calculation is performed on each two-dimensional pixel in the pixel coordinate set to convert it into a normalized viewing direction in the camera coordinate system. Furthermore, it is mapped to a spatial ray in the world coordinate system through camera pose transformation. On the spatial ray, the corresponding three-dimensional candidate position is determined according to the rendering consistency relationship corresponding to the current three-dimensional Gaussian scene representation, thereby mapping the two-dimensional error signal into a set of candidate refinement points in the three-dimensional scene space, which is used to indicate the position where the model has underfitting in a local area.

[0090] This embodiment implements a complete workflow from 3D scene representation to 2D rendered image, then to 2D error localization, and finally 3D error mapping. This workflow achieves precise error localization from 2D to 3D, providing clear and targeted optimization targets for subsequent effective nearest neighbor matching and coarse-to-fine time segmentation strategies. It avoids blind global optimization, significantly improving the efficiency and accuracy of dynamic scene reconstruction optimization. At the same time, relying on the residual-driven localization method, it ensures that the optimization process only applies to areas with insufficient modeling accuracy, achieving efficient utilization of modeling resources.

[0091] In one feasible implementation, the step of selecting the anchor point to be optimized based on the candidate optimization position using an effective nearest neighbor matching method, and performing a coarse-to-fine time segmentation strategy on the anchor point to be optimized to output a set of time slice anchor points with refined hierarchical structure includes: Step S51: Obtain the three-dimensional coordinates of the candidate optimization position, and collect the position information and expressive power parameters of the existing standard anchor points and time slice anchor points to calculate the spatial distance between the candidate optimization position and each anchor point. Combine the expressive power parameters of each anchor point for weighted evaluation, and select the anchor point with the closest spatial distance and the best expressive power as the anchor point to be optimized. It should be noted that 3D coordinates refer to the X, Y, and Z coordinates of the candidate optimization position in the 3D world coordinate system, which is the core basis for locating the spatial position; existing standard anchor points refer to the anchor points in the set of standard anchor points constructed in the initial representation of dynamic scene reconstruction without explicit time effective intervals; time slice anchor points refer to the anchor points in the set of time slice anchor points with explicit lifecycle windows and time constraints that have been constructed; position information refers to the spatial position coordinates of existing standard anchor points and time slice anchor points in the 3D world coordinate system; expressive power parameter refers to the quantitative parameter that measures the expressive ability of anchor points to model local spatial regions, reflecting the modeling effect of 3D Gaussian primitives under anchor point control; spatial distance refers to the Euclidean distance between the candidate optimization position and the spatial position of each anchor point, which is the spatial basis for anchor point matching; weighted evaluation refers to the process of constructing a cost function by combining spatial distance and expressive power parameter to comprehensively score each anchor point; anchor points to be optimized refer to the anchor points with the highest matching degree with the candidate optimization position after weighted evaluation, which are the objects of subsequent time segmentation operations.

[0092] The core objective of this step is to accurately match the corresponding anchor point to be optimized for each candidate optimization position, so that subsequent time segmentation operations can be applied to the most suitable anchor point, thereby improving the targeting and effectiveness of time slice anchor point refinement. The approach is to extract the spatial information of the candidate optimization positions and the position and expressive power information of each anchor point, and to perform a weighted evaluation by calculating spatial distance and expressive power, and to select the anchor point that is spatially closest and has the best expressive power as the anchor point to be optimized, ensuring that the anchor point optimization is highly matched with the modeling requirements of the candidate positions.

[0093] Specifically, for each candidate optimization position, the most suitable anchor point for time segmentation needs to be found among the existing anchor points. During the matching process, this invention distinguishes between canonical anchor points and time-slice anchor points, and performs a weighted evaluation based on the anchor point's effective expressive power at the current time. Specifically, the expressive power of a canonical anchor point can be understood as the interpretability strength of its derived Gaussian units for a local region, determined by its opacity; the expressive power of a time-slice anchor point is determined by introducing a lifecycle window time gating on top of the aforementioned interpretability, causing its expressive strength to change with time position. Based on these expressive powers, the spatial distance between the candidate optimization position and each anchor point is weighted, thereby selecting the anchor point with the most expressive power and the closest spatial proximity at the current time point as the matching result, providing a basis for subsequent time segmentation.

[0094] Step S52: For anchor points of the standard type to be optimized, divide the entire time range covered by the anchor points to be optimized into coarse-grained time intervals to generate initial time slice anchor points. It should be noted that the standard type of anchor points to be optimized refers to the anchor points to be optimized that belong to the standard anchor point set after being screened in step S51. This type of anchor point has no explicit time effective interval and covers the entire time range of scene modeling. The entire time range refers to the overall time span of the dynamic scene to be reconstructed, that is, the shooting time range of multi-view videos, which is the time range of the standard anchor points. The coarse-grained time interval refers to the large-span time division of the entire time range. Each interval has a large time width and is the initial level of time segmentation. The initial time slice anchor points refer to the first batch of time slice anchor points generated after coarse-grained time segmentation of the standard type anchor points to be optimized. They inherit the spatial attributes of the standard anchor points and are given an explicit time effective interval.

[0095] The core purpose of this step is to assign coarse-grained time effective intervals to unconstrained standard anchor points, generate initial time slice anchor points, and achieve the initial transformation from timeless modeling to time-segmented modeling. The implementation idea is to set a reasonable number of coarse-grained time divisions for the entire time range of the standard type anchor points to be optimized based on the dynamic features of the scene, and evenly divide the entire time range into multiple continuous coarse-grained time intervals. For each interval, a corresponding initial time slice anchor point is generated, so that each initial anchor point is only effective within the corresponding coarse-grained interval, thus initially solving the underfitting problem of long-term dynamic modeling.

[0096] Specifically, for a standard anchor point, its lifecycle covers the entire time range. When it cannot fully fit dynamic changes within certain time periods, several initial time slice anchor points will be generated on the full time axis.

[0097] Step S53: For time slice type anchor points to be optimized, further subdivide the original effective time interval of the anchor point to be optimized into fine-grained sub-time intervals to generate sub-anchor points that inherit the spatial attributes of the parent anchor point and only adjust the time parameters. It should be noted that the time slice type of anchor point to be optimized refers to the anchor point to be optimized that belongs to the already constructed time slice anchor point set and is selected through step S51. This type of anchor point has an explicit life cycle window and an effective time interval. The original effective time interval refers to the activation time interval defined by the life cycle window of the time slice type anchor point itself. The fine-grained sub-time interval refers to the small-span time division of the original effective time interval. The time width of each sub-interval is much smaller than the original interval, which is a finer level of time segmentation. The parent anchor point refers to the time slice type anchor point to be optimized that has undergone fine-grained time segmentation and is the basis for the generation of sub-anchor points. The sub-anchor point refers to the new time slice anchor point generated after segmenting the original effective time interval of the parent anchor point. Inheriting the spatial attributes of the parent anchor point means that the sub-anchor point completely reuses all spatially related attributes of the parent anchor point, such as spatial location, scale, and static feature vector. Adjusting only the time parameters means that the sub-anchor point only modifies the two time parameters of time center and time width, and the other parameters remain consistent with the parent anchor point.

[0098] The core purpose of this step is to refine the time dimension of existing time slice anchor points, generate finer-grained sub-anchor points, and improve the modeling accuracy of fast-moving dynamic scenes. The idea is to evenly divide the original effective time interval of the parent anchor point into multiple fine-grained sub-time intervals, generate a corresponding sub-anchor point for each sub-interval, and the sub-anchor point inherits all spatial attributes of the parent anchor point. Only the time parameters of the sub-interval are reconfigured to adapt to the sub-interval, so that each sub-anchor point is effective in a smaller time interval, and achieves accurate fitting of dynamic details.

[0099] Specifically, for existing time slice anchors, further subdivision is only performed within their own lifecycle, uniformly dividing the original longer time window into multiple shorter sub-time intervals and generating corresponding sub-anchors. This segmentation process employs a hierarchical strategy from coarse to fine: the standard anchor first generates coarser-grained time slices in the time dimension, while existing time slice anchors are only further refined within local time intervals when necessary. Newly generated sub-anchors inherit the spatial location and appearance features of their parent anchors, adjusting only time-related lifecycle parameters to ensure the stability of the modeling process. Through this recursive and progressive time segmentation method, the model can adaptively adjust the granularity of time modeling according to dynamic complexity, providing a finer representation in rapidly changing time periods while maintaining a simpler representation structure in less changing time periods, avoiding unnecessary model growth.

[0100] Step S54: Summarize the initial time slice anchor points and the sub-anchor points to form a set of time slice anchor points after hierarchical refinement.

[0101] It should be noted that the initial time slice anchor point refers to the anchor point generated after coarse-grained time segmentation of the anchor point to be optimized of the specification type, which is the first level of time slice anchor point refinement; the sub-anchor point refers to the anchor point generated after fine-grained time segmentation of the anchor point to be optimized of the time slice type, which is a higher level of time slice anchor point refinement; the time slice anchor point set after refinement refers to the unified anchor point set formed by integrating time slice anchor points of different levels and different segmentation sources, including coarse-grained initial time slice anchor points and fine-grained sub-anchor points, and has multi-level time effective intervals.

[0102] The core purpose of this step is to integrate all the new anchor points generated by coarse and fine-grained time segmentation to form a structured set of hierarchical refined time slice anchor points. This provides unified time slice anchor point support for subsequent dynamic scene rendering and parameter optimization, forming a hierarchical refined anchor point set covering different time granularities and different spatial regions, ensuring that the anchor point set can adapt to dynamic changes at different scales of the scene.

[0103] This embodiment implements a complete workflow from candidate optimization positions to anchor point matching, then to coarse-grained and fine-grained time segmentation, and finally to the formation of a hierarchical refined time slice anchor point set. This workflow employs a coarse-to-fine hierarchical time segmentation strategy, decomposing long-term dynamic modeling into multiple controllable short-term segmented modeling tasks. This effectively reduces the modeling difficulty caused by long-distance temporal coupling. Simultaneously, through residual-driven anchor point matching and segmentation, time granularity refinement is performed only in areas with insufficient modeling accuracy, avoiding the waste of computational resources caused by globally indiscriminate refinement, thus achieving a balance between modeling accuracy and computational efficiency. Furthermore, the hierarchical design of anchor points and the complete inheritance of spatial attributes ensure the spatiotemporal consistency of dynamic scene modeling, allowing the generated hierarchical refined time slice anchor point set to accurately adapt to dynamic changes at different scales and rates in the scene. This lays a solid anchor point foundation for subsequent effective opacity modulation and high-quality scene rendering.

[0104] In one feasible implementation, the effective opacity of the 3D Gaussian primitives is modulated by combining the hierarchically refined set of time slice anchor points, the set of 3D Gaussian primitives, and the lifecycle window function. Scene rendering is then performed using screen-space Gaussian splashing. All learnable parameters are iteratively updated with real images as supervision, and a new perspective dynamic rendering video of the scene to be reconstructed is output, including: Step S61: Obtain the set of time slice anchor points after hierarchical refinement, and decode the position, shape, and color attributes of the three-dimensional Gaussian primitives; It should be noted that the set of time slice anchor points after hierarchical refinement refers to the collection of anchor points with different time granularities formed after coarse-to-fine time segmentation. Each anchor point has both spatial attributes and explicit time parameters; the time parameters refer to the core time features of the time slice anchor points, including the time center u and the time half-width δ, which are used to define the effective time interval of the anchor point [u δ,u+δ.

[0105] The core purpose of this step is to integrate the spatiotemporal attributes of time slice anchor points with the basic visual attributes of 3D Gaussian primitives, providing complete parameter support for subsequent calculations of effective opacity and scene rendering. The approach is to extract the temporal and spatial attributes of all time slice anchor points after hierarchical refinement, while decoding the position, shape, and color core attributes of all 3D Gaussian primitives, and establishing an attribute association mapping between anchor points and the control Gaussian primitives to ensure that the temporal control capability of anchor points can accurately act on the corresponding visual attributes of Gaussian primitives.

[0106] Specifically, firstly, the hierarchically refined set of time-slice anchor points is loaded, and for each time-slice anchor point, its temporal parameters (time center and lifecycle window range) and corresponding spatial and feature attributes (anchor point center position and anchor point feature representation) are extracted to calculate its temporal activation weight at a given query time. Then, the set of canonical anchor points and their static features are loaded, and combined with the dynamic features output from the multi-hash table network, a decoding network is used to generate three-dimensional Gaussian element attributes derived from the canonical anchor points and time-slice anchor points at a specified query time t. These attributes include the spatial position of each Gaussian element (obtained by superimposing the anchor point center and the predicted offset), morphological parameters, opacity, and appearance attribute representation.

[0107] Step S62: Calculate the activation weight of the 3D Gaussian element at each time slice anchor point in the current time series using the lifecycle window function, and multiply the activation weight by the opacity parameter of the Gaussian element itself to obtain the effective opacity. It should be noted that the current time sequence refers to the normalized query time t during scene rendering, with a value range of [0,1], corresponding to the specific frame moment of the multi-view video; the activation weight refers to the calculated value of the lifecycle window function at the current time sequence t, reflecting the degree of activation of the control Gaussian primitives by the time slice anchor at that moment, with a value range of [0,1], and the closer the value is to 1, the higher the degree of activation; the opacity parameter σ of the 3D Gaussian primitive itself refers to the inherent opacity of the Gaussian primitive, generated by anchor feature decoding, reflecting the visual visibility of the Gaussian primitive itself, with a value range of [0,1]; the effective opacity α(t) refers to the opacity of the Gaussian primitive actually participating in rendering at the current time sequence, which is the product of the activation weight and the inherent opacity, and is the core indicator for measuring the color contribution of Gaussian primitives in screen-space Gaussian splash rendering.

[0108] The core purpose of this step is to assign temporally effective opacity to the 3D Gaussian primitives under each time slice anchor point, thereby achieving temporal dimension activation control of the Gaussian primitives. This allows the Gaussian primitives to participate in rendering only within the effective time interval of the corresponding anchor point. The implementation idea is to calculate the activation weight of the 3D Gaussian primitives under each time slice anchor point based on the current query time sequence t through the lifecycle window function, and then multiply the weight by the inherent opacity of the Gaussian primitive to obtain the effective opacity that changes dynamically over time, thus achieving temporal modulation of the rendering contribution of the Gaussian primitives.

[0109] Specifically, please refer to the description of effective opacity in step S33, which will not be repeated here.

[0110] Step S63: Based on the effective opacity and screen space Gaussian splashing technology, integrate the visual information of all three-dimensional Gaussian primitives to complete scene rendering; It should be noted that visual information refers to the position, shape, color attributes, and effective opacity of the three-dimensional Gaussian primitives, which are the foundation of the scene's visual representation; integration refers to the process of Gaussian projection and weighted fusion of the visual information of all three-dimensional Gaussian primitives according to screen pixel coordinates; scene rendering refers to the process of converting the digital representation of a three-dimensional dynamic scene into a two-dimensional visual image corresponding to the time sequence of multi-view video, which is an intuitive presentation of the reconstruction result.

[0111] The core purpose of this step is to use screen-space Gaussian splashing technology to convert the visual information of 3D Gaussian primitives with effective opacity into high-quality 2D scene rendering images, thereby realizing the visualization of dynamic scenes. The implementation idea is to project the 3D Gaussian primitives from the world coordinate system to the camera coordinate system and then to the screen pixel coordinate system, calculate the projection ellipse parameters of each Gaussian primitive in screen space, and then calculate the color contribution weight of each Gaussian primitive to the corresponding pixel based on the effective opacity. The color contributions of all Gaussian primitives are weighted and fused to obtain the final screen pixel color, thus completing the scene rendering.

[0112] Specifically, firstly, the camera parameters corresponding to the current rendering viewpoint are loaded, including the camera intrinsic matrix K (focal length, principal point coordinates) and extrinsic parameters (rotation matrix R, translation vector T), to determine the resolution of the rendered image and the screen pixel coordinate system range. Then, the set of three-dimensional Gaussian primitives is traversed, and coordinate projection is performed on each Gaussian primitive: first, its 3D position in the world coordinate system is converted to coordinates in the camera coordinate system using the extrinsic parameters; then, the camera coordinates are projected to the screen pixel coordinate system using the intrinsic matrix to obtain the pixel coordinates of the Gaussian primitive on the screen. Simultaneously, the geometric parameters (major axis, minor axis, rotation angle) of its projection ellipse in screen space are calculated based on the shape parameters of the Gaussian primitive. Based on the effective opacity α(t) and depth information of the Gaussian primitive, its color contribution weight to the screen pixel is calculated; the weight is positively correlated with the effective opacity and negatively correlated with the depth value. For each pixel in screen space, all Gaussian primitives whose projection ellipses cover that pixel are retrieved, and the RGB color values ​​and corresponding contribution weights of these Gaussian primitives are collected. Finally, the color contribution of each pixel is weighted and fused, and the color values ​​of all Gaussian primitives are summed according to their weights and normalized to obtain the final RGB value of the pixel. Color values; after traversing all pixels on the screen and calculating the color values ​​of all pixels, post-processing is performed on the rendered image, including brightness adjustment, anti-aliasing, and other operations to optimize the visual effect of the image and finally complete the scene rendering in the current time sequence.

[0113] Step S64: Using the real images in the multi-view video as the supervision benchmark, calculate the pixel-level reconstruction loss, structural similarity loss, and edge enhancement loss between the rendered image and the real image; It should be noted that the real image in multi-view video refers to the original video frame image at the same time sequence and shooting angle as the rendered image. It serves as the true benchmark for measuring the rendering effect and contains the real visual information of the scene. The supervised benchmark refers to using the real image as a reference standard to quantify the difference between the rendered image and the real scene, guiding the iterative optimization of model parameters. The rendered image refers to the scene rendered image at the current time sequence generated by screen-space Gaussian splashing technology. The pixel-level reconstruction loss refers to the L1 loss, which is the mean absolute error of the pixel-level RGB color values ​​between the rendered image and the real image, used to measure the overall color deviation between the two images. The structural similarity loss refers to the L1 loss. SSIM The loss is a value calculated based on the Structural Similarity Index (SSIM), which comprehensively measures the differences between two images in terms of brightness, contrast, and structural features. The closer the SSIM value is to 1, the smaller the loss value. Edge enhancement loss refers to L... Sobel The loss is a loss value based on Sobel edge detection. It measures the difference in edge detail between the two images by calculating the difference in edge response intensity between the rendered image and the real image, and is used to improve the edge sharpness and structural detail of the rendered image.

[0114] The core purpose of this step is to quantify the difference between the rendered image and the real image from three dimensions: pixel color, overall structure, and edge details. This provides a basis for loss calculation for model parameter optimization. The approach is to calculate pixel-level reconstruction loss, structural similarity loss, and edge enhancement loss separately to characterize the rendering error from different dimensions. This allows subsequent parameter optimization to simultaneously consider the overall color consistency, structural similarity, and edge detail fidelity of the image.

[0115] Specifically, the pixel-level reconstruction loss (L1) is calculated as follows: For the two normalized images, the absolute error of the RGB three-channel color values ​​is calculated pixel by pixel. The errors of the three channels are summed to obtain the total error for each pixel. Then, the average of the total errors of all pixels is calculated to obtain the final pixel-level reconstruction loss. The structural similarity loss (L1) is also calculated. SSIM The aligned two images are divided into multiple non-overlapping fixed-size image blocks. For each image block, brightness, contrast, and structural similarity are calculated separately. The combined SSIM value of this image block is then obtained. The SSIM is calculated by averaging the SSIM values ​​of all image blocks. mean Then through L SSIM =1 SSIM mean Obtain the structural similarity loss; calculate the edge enhancement loss (L Sobel The Sobel operator is used to perform edge detection on the rendered image and the real image respectively. The gradient magnitudes of the two images in the horizontal and vertical directions are calculated to obtain their respective edge response intensity maps. Then, the error between the two intensity maps is calculated as the final edge enhancement loss.

[0116] Step S65: Iteratively update the model parameters based on the loss function to optimize the scene representation; It should be noted that the loss function refers to the total loss function obtained by weighted fusion of pixel-level reconstruction loss, structural similarity loss, edge enhancement loss, and volume regularization loss; model parameters refer to all learnable parameters involved in dynamic scene modeling, including static feature vectors of time slice anchors, scale factors, and weight parameters of the decoding network; scene representation refers to the digital representation of the dynamic scene composed of a set of canonical anchors, a set of time slice anchors, a set of three-dimensional Gaussian primitives, and their relationships, and its accuracy is continuously improved with the optimization of model parameters.

[0117] The core purpose of this step is to iteratively optimize all learnable parameters of the model based on the back gradient of the multi-dimensional loss function, improve the accuracy of dynamic scene representation, and make the subsequently rendered images closer to the real scene.

[0118] Specifically, in terms of loss function design, this invention adopts the training objective commonly used in the 3D Gaussian splashing framework, and supplements and enhances it. The overall loss consists of pixel-level reconstruction loss, structural similarity loss, volume regularization term, and edge enhancement loss. The above losses are jointly optimized through weighted summation, thereby achieving a balance between overall reconstruction accuracy, structural consistency, model compactness, and detail fidelity.

[0119] Step S66: Based on the optimized model, generate a new perspective dynamic rendering video of the scene to be reconstructed.

[0120] It should be noted that the optimized model refers to the dynamic scene modeling model after multiple rounds of parameter iteration and updates, including the optimized set of standard anchor points, the set of time slice anchor points, the set of three-dimensional Gaussian primitives and their relationships; the new perspective refers to any perspective not within the shooting range of the multi-view video, which is a virtual observation perspective that breaks through the limitations of the shooting perspective; the dynamic rendering video refers to the scene video that is continuously rendered in time sequence from the new perspective, which can present the dynamic changes of the scene to be reconstructed at different times.

[0121] The core objective of this step is to synthesize new perspectives of dynamic scenes based on the optimized model, generate high-quality dynamic rendering videos from new perspectives, and achieve the ultimate goal of dynamic scene reconstruction. The implementation idea is to first define the camera parameters (intrinsic and extrinsic parameters) of the new perspective, then traverse all time sequences of the scene according to the time sequence, perform screen-space Gaussian splash rendering from the new perspective in each time sequence to obtain a single frame image, and finally combine the single frame rendering images of all time sequences into a continuous dynamic rendering video according to the frame rate, so as to realize immersive observation of dynamic scenes from any new perspective.

[0122] Specifically, firstly, based on the intrinsic and extrinsic parameters of the camera at the new perspective, the observation position, pose, and output image resolution are determined. Then, within the normalized time axis, the query time series is continuously sampled and queried at a preset time step to describe the state evolution of the dynamic scene at different times. For each query time point, the time parameters are input into the dynamic scene representation model, the activation weights of the time slice anchors are calculated, and the corresponding three-dimensional Gaussian primitive attributes are derived by combining the normalized anchors and the time slice anchors, including spatial location, morphological parameters, opacity, and appearance feature representations. Based on this, screen-space Gaussian splash rendering is performed based on the new perspective camera parameters, mapping the continuous spatiotemporal three-dimensional Gaussian representation to the corresponding new perspective image frames. By continuously rendering each query time on the time axis, the new perspective image sequence constituting the dynamic scene is directly obtained, forming a smooth new perspective dynamic rendering result.

[0123] This embodiment implements a complete dynamic scene reconstruction process, from integrating anchor points and Gaussian meta-attributes to effective opacity calculation, scene rendering, loss quantization, parameter optimization, and finally generating a dynamic video with a new perspective. This process combines temporal control of time-slice anchor points with efficient rendering depth using screen-space Gaussian splashing. Through residual-driven parameter optimization, it achieves accurate fitting of dynamic scene details while breaking through the limitations of shooting angles, enabling free-viewpoint synthesis of dynamic scenes.

[0124] Compared to traditional methods, this workflow decomposes long-term dynamic modeling into short-term segmented fitting using a coarse-to-fine temporal segmentation strategy. This effectively reduces the modeling difficulty caused by long-distance temporal coupling and improves the fidelity of dynamic details. Simultaneously, through effective opacity temporal modulation, Gaussian elements participate in rendering only within their corresponding time intervals, improving computational efficiency while maintaining rendering quality. The resulting new-perspective dynamic rendering video combines rich detail, smooth dynamics, and flexible viewpoints, meeting the dynamic scene reconstruction needs of various fields such as sports live streaming, AR / VR, and film production.

[0125] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0126] This application also provides an adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points. Please refer to [link / reference]. Figure 2 The adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points includes: The acquisition module is used to acquire multi-view video and initial point cloud data corresponding to the scene to be reconstructed, initialize a set of standard anchor points based on the initial point cloud data, and derive a set of three-dimensional Gaussian primitives from the standard anchor points. The standard anchor points control a set of three-dimensional Gaussian primitives and decode and generate the position, shape and color attributes of the three-dimensional Gaussian primitives. The construction module is used to construct time slice anchors based on the set of standard anchors, introduce time center and time width, and output a set of time slice anchors with time constraints. The time slice anchors inherit the spatial attributes of the standard anchors and limit the effective time interval of the time slice anchors through a preset life cycle window function. The activation module is used to generate a model rendering image based on the three-dimensional Gaussian primitive set, compare the model rendering image with the real image in the multi-view video, continuously iterate and optimize the model parameters according to the comparison results, and periodically calculate the image residual between the model rendering image and the real image so as to map the error information to the three-dimensional scene space after locating the modeling error area and determine the candidate optimization position. The segmentation module is used to select the anchor point to be optimized based on the candidate optimization position by an effective nearest neighbor matching method, perform a coarse-to-fine time segmentation strategy on the anchor point to be optimized, and output a set of time slice anchor points after hierarchical refinement. The rendering module combines the hierarchically refined time slice anchor point set, the three-dimensional Gaussian primitive set, and the lifecycle window function to modulate the effective opacity of the three-dimensional Gaussian primitives. It then completes scene rendering through screen-space Gaussian splashing, iteratively updates all learnable parameters with real images as supervision, and outputs a new perspective dynamic rendering video of the scene to be reconstructed.

[0127] This application provides an adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points, which employs an adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points as described in the above embodiments, and can solve the technical problems in the background art. Compared with the prior art, the beneficial effects of the adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points provided in this application are the same as the beneficial effects of the adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points provided in the above embodiments, and other technical features in the adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points are the same as the features disclosed in the methods of the above embodiments, and will not be repeated here.

[0128] This application provides an adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points. The adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points in the above embodiment 1.

[0129] The following is for reference. Figure 3This document illustrates a structural schematic diagram of an adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points, suitable for implementing embodiments of this application. The adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points in this application embodiment can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 3 The adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0130] like Figure 3 As shown, an adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows an adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points to wirelessly or wiredly communicate with other devices to exchange data. Although an adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0131] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0132] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0133] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements an adaptive three-dimensional Gaussian splash dynamic scene reconstruction method based on local time slice anchor points provided by the above methods.

[0134] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0135] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0136] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. An adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points, characterized in that, include: Acquire multi-view video and initial point cloud data corresponding to the scene to be reconstructed. Initialize a set of standard anchor points based on the initial point cloud data, and derive a set of three-dimensional Gaussian primitives from the standard anchor points. The standard anchor points control a set of three-dimensional Gaussian primitives and decode and generate the position, shape and color attributes of the three-dimensional Gaussian primitives. Based on the set of standard anchor points, a time center and a time width are introduced to construct time slice anchor points, and a set of time slice anchor points with time constraints is output. The time slice anchor points inherit the spatial attributes of the standard anchor points and the effective time interval of the time slice anchor points is limited by a preset life cycle window function. A model rendering image is generated based on the three-dimensional Gaussian primitive set, and the model rendering image is compared with the real image in the multi-view video. The model parameters are continuously iteratively optimized and updated according to the comparison results, and the image residual between the model rendering image and the real image is periodically calculated. After locating the modeling error area, the error information is mapped to the three-dimensional scene space to determine the candidate optimization position. Based on the candidate optimization positions, the anchor point to be optimized is selected by the effective nearest neighbor matching method, and a coarse-to-fine time segmentation strategy is performed on the anchor point to be optimized to output a set of time slice anchor points after hierarchical refinement. By combining the hierarchically refined time slice anchor point set, the three-dimensional Gaussian primitive set, and the lifecycle window function, the effective opacity of the three-dimensional Gaussian primitive is modulated, and scene rendering is completed through screen-space Gaussian splashing. All learnable parameters are iteratively updated with real images as supervision, and a new perspective dynamic rendering video of the scene to be reconstructed is output.

2. The adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points as described in claim 1, characterized in that, The process involves initializing a set of standardized anchor points based on the initial point cloud, and deriving a set of 3D Gaussian primitives from these standardized anchor points. Each standardized anchor point manages a set of 3D Gaussian primitives and decodes them to generate the position, shape, and color attributes of these primitives, including: The spatial location information in the initial point cloud data is obtained, and the center position of the standard anchor point in the three-dimensional scene space is initialized based on the spatial location information; the standard anchor point includes spatial location, scale parameters and learnable static feature vectors, which are used as basic expression units to characterize the local spatial structure. Based on the decoding of the canonical anchor point, multiple three-dimensional Gaussian elements are derived, and the position offset, morphological parameters, opacity, and appearance attribute representation of each three-dimensional Gaussian element relative to the canonical anchor point are predicted, so as to form an expressive Gaussian set in the local spatial domain corresponding to each canonical anchor point.

3. The adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points as described in claim 1, characterized in that, Based on the set of standard anchor points, a time center and time width are introduced to construct time slice anchor points, outputting a set of time slice anchor points with time constraints. These time slice anchor points inherit the spatial attributes of the standard anchor points and are limited to their effective time intervals by a preset lifecycle window function, including: Obtain the spatial attributes of each standard anchor point in the set of standard anchor points. The spatial attributes include spatial location, scale and static feature vector. Retain the spatial attributes as the basic attributes of the time slice anchor points. Configure a time center and time width for each time slice anchor point, and introduce a preset life cycle window function. The time center corresponds to the core moment when the anchor point takes effect, the time width corresponds to the time range during which the anchor point takes effect, and the life cycle window function is constructed based on the time center and the time width. By integrating the basic attributes, the time center, the time width, and the lifecycle window function, a complete set of time slice anchor points is constructed, which are then aggregated to form a time slice anchor point set with time constraints.

4. The adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points as described in claim 1, characterized in that, The process involves generating a model rendering image based on the 3D Gaussian primitive set, comparing the model rendering image with real images in the multi-view video, continuously iteratively optimizing and updating the model parameters based on the comparison results, and periodically calculating the image residual between the model rendering image and the real image. This is done to map the error information to the 3D scene space after locating the modeling error region and determining candidate optimization positions. The process includes: Based on the three-dimensional Gaussian primitive set, screen-space Gaussian splash rendering is performed to generate a model rendering image; The structural similarity residuals of the real images corresponding to the time sequence of the model-rendered images in the multi-view video are calculated with those of the model-rendered images. The residual mask is then constructed by combining the edge response intensity and the residual mask is used to locate the error regions where the modeling accuracy is insufficient. The pixel coordinate information of the error region is obtained, and the pixel coordinates are back-projected into three-dimensional space in combination with the multi-view camera parameters to obtain the error region in three-dimensional space, which serves as a candidate optimization position.

5. The adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points as described in claim 1, characterized in that, The step involves selecting anchor points to be optimized based on the candidate optimization positions using an effective nearest neighbor matching method, applying a coarse-to-fine time segmentation strategy to the anchor points to be optimized, and outputting a set of time slice anchor points with refined hierarchical structure, including: The three-dimensional coordinates of the candidate optimization position are obtained, and the position information and expressive power parameters of the existing standard anchor points and time slice anchor points are collected to calculate the spatial distance between the candidate optimization position and each anchor point. The expressive power parameters of each anchor point are combined for weighted evaluation, and the anchor point with the closest spatial distance and the best expressive power is selected as the anchor point to be optimized. For anchor points of the standard type to be optimized, coarse-grained time intervals are divided within the full time range covered by the anchor points to be optimized to generate initial time slice anchor points; For time-slice type anchor points to be optimized, fine-grained sub-time intervals are further divided within the original effective time interval of the anchor point to be optimized, generating sub-anchor points that inherit the spatial attributes of the parent anchor point and only adjust the time parameters. The initial time slice anchor points and the sub-anchor points are combined to form a hierarchically refined set of time slice anchor points.

6. The adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points as described in claim 1, characterized in that, The method combines the hierarchically refined time slice anchor point set, the 3D Gaussian primitive set, and the lifecycle window function to modulate the effective opacity of the 3D Gaussian primitives. Scene rendering is completed through screen-space Gaussian splashing. All learnable parameters are iteratively updated with real images as supervision, and a new perspective dynamic rendering video of the scene to be reconstructed is output, including: Obtain the set of time slice anchor points after hierarchical refinement, and decode the position, shape, and color attributes of the three-dimensional Gaussian primitives; The activation weight of the 3D Gaussian element at each time slice anchor point is calculated using the lifecycle window function, and the activation weight is multiplied by the opacity parameter of the Gaussian element itself to obtain the effective opacity. Based on the aforementioned effective opacity and screen-space Gaussian splashing technology, the visual information of all three-dimensional Gaussian primitives is integrated to complete scene rendering; Using the real images in the multi-view video as the supervision benchmark, the pixel-level reconstruction loss, structural similarity loss and edge enhancement loss between the rendered image and the real image are calculated; The model parameters are iteratively updated based on the loss function to optimize the scene representation; Based on the optimized model, a new perspective dynamic rendering video of the scene to be reconstructed is generated.

7. An adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points, characterized in that, include: The acquisition module is used to acquire multi-view video and initial point cloud data corresponding to the scene to be reconstructed, initialize a set of standard anchor points based on the initial point cloud data, and derive a set of three-dimensional Gaussian primitives from the standard anchor points. The standard anchor points control a set of three-dimensional Gaussian primitives and decode and generate the position, shape and color attributes of the three-dimensional Gaussian primitives. The construction module is used to construct time slice anchors based on the set of standard anchors, introduce time center and time width, and output a set of time slice anchors with time constraints. The time slice anchors inherit the spatial attributes of the standard anchors and limit the effective time interval of the time slice anchors through a preset life cycle window function. The activation module is used to generate a model rendering image based on the three-dimensional Gaussian primitive set, compare the model rendering image with the real image in the multi-view video, continuously iterate and optimize the model parameters according to the comparison results, and periodically calculate the image residual between the model rendering image and the real image so as to map the error information to the three-dimensional scene space after locating the modeling error area and determine the candidate optimization position. The segmentation module is used to select the anchor point to be optimized based on the candidate optimization position by an effective nearest neighbor matching method, perform a coarse-to-fine time segmentation strategy on the anchor point to be optimized, and output a set of time slice anchor points after hierarchical refinement. The rendering module combines the hierarchically refined time slice anchor point set, the three-dimensional Gaussian primitive set, and the lifecycle window function to modulate the effective opacity of the three-dimensional Gaussian primitives. It then completes scene rendering through screen-space Gaussian splashing, iteratively updates all learnable parameters with real images as supervision, and outputs a new perspective dynamic rendering video of the scene to be reconstructed.

8. An adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points, characterized in that, The adaptive 3D Gaussian splash dynamic scene reconstruction device based on local time slice anchor points includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the adaptive three-dimensional Gaussian splash dynamic scene reconstruction method based on local time slice anchor points as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the adaptive 3D Gaussian splash dynamic scene reconstruction method based on local time slice anchor points as described in any one of claims 1 to 6.

Images