Reconstruction method and device of dynamic gaussian scene, electronic equipment and storage medium

Abstract

The application provides a dynamic Gaussian scene reconstruction method and device, electronic equipment and storage medium, and relates to the technical field of computer vision and graphics, and the method comprises the following steps: obtaining image information to be reconstructed; inputting the image information to be reconstructed into a dynamic Gaussian scene reconstruction model to obtain a dynamic Gaussian scene reconstruction result output by the dynamic Gaussian scene reconstruction model; wherein the dynamic Gaussian scene reconstruction model is trained based on the correlation of the spatial feature overlap degree between different views, the view difference degree between different views and the constraint relationship between adjacent frames of the same view determined by image samples. The technical scheme of the application is that the dynamic Gaussian scene reconstruction model trained based on the correlation of the spatial feature overlap degree between different views, the view difference degree between different views and the constraint relationship between adjacent frames of the same view determined by image samples is used to reconstruct the dynamic Gaussian scene of the image information to be reconstructed, so that the reconstruction accuracy and rendering fidelity of the dynamic Gaussian scene are improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision and graphics technology, and in particular to a method, apparatus, electronic device and storage medium for reconstructing dynamic Gaussian scenes. Background Technology

[0002] The synthesis of novel perspectives in dynamic scenes is one of the core challenges in computer vision and graphics. Its core objective is to reconstruct spatiotemporally consistent 3D dynamic scenes from multi-view videos to support rendering and interaction from any perspective at any time. This technology promises to provide users with a highly realistic dynamic environment experience and is a key technology supporting the development of immersive interactive systems such as virtual reality and the metaverse.

[0003] Currently, traditional dynamic scene reconstruction methods are mostly based on Structure from Motion (SfM) and Multi-View Stereo (MVS) techniques. These methods face significant challenges in handling complex dynamic changes, such as non-rigid deformation of objects, changes in light and shadow over time, and changes in topology (such as object splitting or merging). They are time-consuming and costly, and more importantly, the fidelity of images rendered from new perspectives is low. Although Neural Radiation Field (NeRF) has performed well in static scenes, and many works have attempted to extend it to the field of 4D reconstruction, this method relies on ray sampling and neural network inference. When dealing with large-scale complex dynamic scenes, it often has long training times and low rendering frame rates, making it difficult to meet user needs. At the same time, it does not fully utilize the spatiotemporal correlation between different perspectives and between adjacent frames of the same perspective, resulting in suboptimal optimization results in dynamic scene reconstruction—such as local artifacts, motion discontinuities, and insufficient temporal coherence. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for reconstructing dynamic Gaussian scenes. It addresses the shortcomings of existing technologies that fail to fully utilize the spatiotemporal correlations between different viewpoints and between adjacent frames within the same viewpoint, leading to suboptimal results in dynamic scene reconstruction—such as local artifacts, motion discontinuities, and insufficient temporal coherence. The invention achieves dynamic Gaussian scene reconstruction by training a dynamic Gaussian scene reconstruction model based on the correlations between spatial feature overlaps between different viewpoints, viewpoint differences between different viewpoints, and constraint relationships between adjacent frames within the same viewpoint, as determined by image samples. This improves the reconstruction accuracy and rendering fidelity of dynamic Gaussian scenes, effectively addressing issues such as insufficient reconstruction consistency between different viewpoints, local reconstruction artifacts, and limited detail fidelity.

[0005] This invention provides a method for reconstructing a dynamic Gaussian scene, comprising the following steps.

[0006] Obtain the image information to be reconstructed; The image information to be reconstructed is input into the dynamic Gaussian scene reconstruction model to obtain the dynamic Gaussian scene reconstruction result output by the dynamic Gaussian scene reconstruction model. The dynamic Gaussian scene reconstruction model is trained based on the spatial feature overlap between different viewpoints determined by image samples, the viewpoint difference between different viewpoints, and the correlation of constraint relationships between adjacent frames of the same viewpoint.

[0007] According to the present invention, a method for reconstructing a dynamic Gaussian scene is provided, wherein the dynamic Gaussian scene reconstruction model is trained based on the following steps: Obtain image samples; Determine the spatial feature overlap based on image samples, and establish the viewpoint difference between different viewpoints of image samples based on the spatial feature overlap. Construct the constraint relationship between adjacent frames of the same viewpoint in image samples; The basic dynamic Gaussian scene reconstruction model is trained based on the correlation between image samples, the difference in viewpoints between different viewpoints, and the constraint relationship between adjacent frames of the same viewpoint, to obtain the dynamic Gaussian scene reconstruction model.

[0008] According to the present invention, a method for reconstructing a dynamic Gaussian scene determines the spatial feature overlap based on image samples, including: Generate an inverted index table based on image samples; The number of shared feature points is calculated based on the inverted index table; where the number of shared feature points represents the number of shared feature points between any two viewpoints in the same frame; Spatial feature overlap is constructed based on the number of shared feature points.

[0009] According to the present invention, a method for reconstructing a dynamic Gaussian scene establishes the viewpoint difference between different viewpoints of image samples based on the spatial feature overlap, including: Viewpoint pairs are determined based on the degree of overlap of spatial features; a viewpoint pair consists of any two different viewpoints at the same time. The degree of difference in viewpoints between different viewpoints is determined by the optical axis direction vectors corresponding to each viewpoint in the viewpoint alignment.

[0010] According to the present invention, a method for reconstructing a dynamic Gaussian scene includes a trajectory smoothing loss function and a deformation regularization loss function for the constraint relationship between adjacent frames from the same viewpoint; constructing the constraint relationship between adjacent frames from the same viewpoint for image samples includes: Obtain the actual location of the image sample; The trajectory smoothing loss function is determined based on the actual location and the smoothed location; where the smoothed location is determined based on a pre-established short time window. Obtain the rate of change of eigenvalues ​​of the covariance matrix; The deformation regularization loss function is determined based on the rate of change of the eigenvalues ​​of the covariance matrix.

[0011] According to the present invention, a method for reconstructing a dynamic Gaussian scene is provided, which trains a basic dynamic Gaussian scene reconstruction model based on image samples, the degree of viewpoint difference between different viewpoints, and the correlation of constraint relationships between adjacent frames of the same viewpoint, to obtain a dynamic Gaussian scene reconstruction model, including: Based on the image samples and the basic loss function, the first stage of basic optimization training is performed on the basic training dynamic Gaussian scene reconstruction model to obtain the first training dynamic Gaussian scene reconstruction model. Based on the perspective differences between different perspectives, the constraint relationship between adjacent frames of the same perspective, and the total loss of iterative optimization, the second stage of spatiotemporal detail optimization training is carried out on the first training dynamic Gaussian scene reconstruction model to obtain the second training dynamic Gaussian scene reconstruction model. The third stage of global accuracy calibration training is performed on the second training dynamic Gaussian scene reconstruction model based on the total target loss, resulting in the dynamic Gaussian scene reconstruction model.

[0012] According to the present invention, a method for reconstructing a dynamic Gaussian scene is characterized by performing a third-stage global accuracy calibration training on a second-trained dynamic Gaussian scene reconstruction model based on the target total loss to obtain a dynamic Gaussian scene reconstruction model, comprising: Based on the total target loss, the second training dynamic Gaussian scene reconstruction model is trained in the third stage of global accuracy calibration to obtain the candidate dynamic Gaussian scene reconstruction model. Obtain the distributed training architecture; The candidate dynamic Gaussian scene reconstruction model is trained in a distributed manner using a distributed training architecture to obtain the dynamic Gaussian scene reconstruction model.

[0013] The present invention also provides a device for reconstructing dynamic Gaussian scenes, comprising the following modules: The information acquisition module is used to acquire information about the image to be reconstructed. The scene reconstruction module is used to input the image information to be reconstructed into the dynamic Gaussian scene reconstruction model and obtain the dynamic Gaussian scene reconstruction result output by the dynamic Gaussian scene reconstruction model. The dynamic Gaussian scene reconstruction model is trained based on the spatial feature overlap between different viewpoints determined by image samples, the viewpoint difference between different viewpoints, and the correlation of the constraint relationship between adjacent frames of the same viewpoint.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described methods for reconstructing dynamic Gaussian scenes.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for reconstructing any of the dynamic Gaussian scenes described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a method for reconstructing any of the dynamic Gaussian scenes described above.

[0017] This invention provides a method, apparatus, electronic device, and storage medium for reconstructing dynamic Gaussian scenes. The method involves acquiring image information to be reconstructed; inputting the image information into a dynamic Gaussian scene reconstruction model to obtain the dynamic Gaussian scene reconstruction result output by the model; wherein the dynamic Gaussian scene reconstruction model is trained based on the correlation of spatial feature overlap between different viewpoints, viewpoint differences between different viewpoints, and constraints between adjacent frames within the same viewpoint, determined from image samples. This invention addresses the shortcomings of existing technologies that fail to fully utilize the spatiotemporal correlation between different viewpoints and between adjacent frames within the same viewpoint, leading to suboptimal optimization results in dynamic scene reconstruction—such as local artifacts, motion discontinuities, and insufficient temporal coherence. It achieves the reconstruction of dynamic Gaussian scenes from image information using a dynamic Gaussian scene reconstruction model trained based on the correlation of spatial feature overlap between different viewpoints, viewpoint differences between different viewpoints, and constraints between adjacent frames within the same viewpoint, determined from image samples. This improves the reconstruction accuracy and rendering fidelity of dynamic Gaussian scenes, effectively addressing problems such as insufficient reconstruction consistency between different viewpoints, local reconstruction artifacts, and limited detail fidelity. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the dynamic Gaussian scene reconstruction method provided by the present invention.

[0020] Figure 2 This is a schematic diagram of multi-view relationships provided by the present invention.

[0021] Figure 3 This is a schematic diagram of the training process for the dynamic Gaussian scene reconstruction model provided by the present invention.

[0022] Figure 4 This is a schematic diagram of the distributed loading and training process provided by the present invention.

[0023] Figure 5 This is a schematic diagram of the structure of the dynamic Gaussian scene reconstruction device provided by the present invention.

[0024] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0026] The following is combined with Figure 1 The present invention describes a method for reconstructing dynamic Gaussian scenes. This method is applicable to the reconstruction and enhancement of dynamic Gaussian scenes based on viewpoint correlation. The execution subject of this method can be an electronic device or a dynamic Gaussian scene reconstruction device installed in the electronic device. The dynamic Gaussian scene reconstruction device can be implemented by software, hardware, or a combination of both. Figure 1 This is one of the flowcharts illustrating the dynamic Gaussian scene reconstruction method provided by the present invention, such as... Figure 1 As shown, the method includes the following steps: 101 and 102.

[0027] Step 101: Obtain the image information to be reconstructed.

[0028] In this step, the image information to be reconstructed is information about a dynamic scene that needs to be reconstructed, which is acquired using an image acquisition device. The image acquisition device can be, for example, a camera, and there can be at least one camera. Different cameras can acquire information from different angles, and this embodiment does not limit this.

[0029] Specifically, it acquires the image information to be reconstructed from the dynamic scene that needs to be reconstructed.

[0030] Step 102: Input the image information to be reconstructed into the dynamic Gaussian scene reconstruction model to obtain the dynamic Gaussian scene reconstruction result output by the dynamic Gaussian scene reconstruction model; wherein, the dynamic Gaussian scene reconstruction model is trained based on the spatial feature overlap between different viewpoints determined by image samples, the viewpoint difference between different viewpoints, and the correlation of the constraint relationship between adjacent frames of the same viewpoint.

[0031] Specifically, the image information to be reconstructed is input into the dynamic Gaussian scene reconstruction model to obtain the dynamic Gaussian scene reconstruction result output by the dynamic Gaussian scene reconstruction model.

[0032] In one specific embodiment, the dynamic Gaussian scene reconstruction model is trained based on the following steps: acquiring image samples; determining the spatial feature overlap based on the image samples, and establishing the viewpoint difference between different viewpoints of the image samples based on the spatial feature overlap; constructing the inter-frame constraint relationship of adjacent frames of the same viewpoint of the image samples; and training the basic dynamic Gaussian scene reconstruction model based on the correlation between the image samples, the viewpoint difference between different viewpoints, and the inter-frame constraint relationship of adjacent frames of the same viewpoint to obtain the dynamic Gaussian scene reconstruction model.

[0033] In this step, the image samples are used to train the dynamic Gaussian scene reconstruction model. The image samples can be samples obtained from different viewpoints.

[0034] Specifically, the spatial feature overlap is determined based on the image samples, and the perspective difference between different viewpoints of the image samples is established based on the spatial feature overlap. The constraint relationship between adjacent frames of the same viewpoint of the image samples is constructed. The basic dynamic Gaussian scene reconstruction model is trained based on the correlation between the image samples, the perspective difference between different viewpoints and the constraint relationship between adjacent frames of the same viewpoint to obtain the dynamic Gaussian scene reconstruction model.

[0035] In one specific embodiment, determining the spatial feature overlap based on image samples includes: generating an inverted index table based on the image samples; calculating the number of shared feature points based on the inverted index table; wherein the number of shared feature points represents the number of shared feature points between any two viewpoints in the same frame; and constructing the spatial feature overlap based on the number of shared feature points.

[0036] Specifically, generating an inverted index table based on image samples primarily utilizes Structure-from-Motion (SFM) reconstruction technology to generate images.bin or images.txt files (images.bin and images.txt are two different output file formats generated by the SFM software after feature matching and sparse reconstruction; images.bin is a binary file and cannot be read directly; images.txt is a plain text file and can be opened and viewed with a text editor). Then, the set of identifiers (IDs) of all 3D feature points observed in each frame of the image samples is extracted. By traversing image samples from all viewpoints, an inverted index table is constructed, with 3D point IDs as keys and a list of viewpoint indices that observed that point as values. The specific form of the inverted index table is as follows: ,in, This represents the ID of a 3D point in a sparse point cloud. This indicates the number of the image sample observed at that point. Indicates the total number of viewpoints.

[0037] The advantage of this setup is that by extracting the identity markers of all 3D feature points from the image samples using SFM, an inverted index table can be obtained, which can effectively avoid the high computational overhead of pairwise image matching.

[0038] Specifically, calculating the number of shared feature points based on the inverted index table mainly involves filling the common view matrix M by traversing all 3D points in the inverted index table after obtaining it. The common view matrix M is a symmetric matrix, where the elements in the common view matrix M... Indicates perspective Perspective The number of shared feature points between them. For each 3D point P in the inverted index table, extract the view index list of observations of that point. In the view index list In the pairwise combination, for any two view indexes , Indicates and For any given viewpoint, an increment operation is performed at the position corresponding to the common view matrix M. . This is a constant coefficient, for example, C can be 1, but this embodiment does not limit this.

[0039] The advantage of this setup is that this method establishes viewpoint relationships directly through 3D points, determines the number of shared feature points, and only processes viewpoint pairs with co-view relationships during the process of determining the number of shared feature points, which greatly reduces computational redundancy.

[0040] Specifically, constructing spatial feature overlap based on the number of shared feature points mainly involves, after determining the number of shared feature points, treating the common-view matrix M filled with the number of shared feature points as an adjacency matrix of a weighted undirected graph, with the viewpoint being the node and the elements of the shared feature points as the viewpoint. As edge weights. For each target perspective. , search target perspective All associated perspectives with a value greater than 0 in the common-view matrix M , will be related perspective According to the right of the border Sort the views in descending order from largest to smallest. Select the top G views (for example, G=10, but this embodiment does not limit this) as the target views. The optimal associated neighbors are determined, and the final association list generated based on all the optimal associated neighbors is the spatial feature overlap. The spatial feature overlap is then used as the input candidate space for the mechanism of viewpoint difference between different viewpoints.

[0041] In one specific embodiment, establishing the viewpoint difference degree between different viewpoints of image samples based on the spatial feature overlap degree includes: determining viewpoint pairs based on the spatial feature overlap degree; a viewpoint pair consists of any two different viewpoints at the same time; and determining the viewpoint difference degree between different viewpoints based on the optical axis direction vectors corresponding to each viewpoint in the viewpoint pair.

[0042] In this step, the viewpoint can be, for example, , and This embodiment does not limit the scope of the two different perspectives.

[0043] The optical axis direction vector includes a first optical axis direction vector and a second optical axis direction vector, which are direction vectors corresponding to different viewpoints.

[0044] Specifically, after determining the spatial feature overlap, the determination of spatial feature overlap... Every perspective at any given moment Then, based on the perspective The optical axis direction vectors corresponding to each viewpoint determine the degree of viewpoint difference between different viewpoints. Perspective from the moment and perspective The difference in perspective between corresponding different viewpoints The calculation is shown in formula (1).

[0045] (1) In formula (1), express Perspective from the moment The corresponding first optical axis direction vector, express Perspective from the moment The corresponding second optical axis direction vector. express The absolute value, express The absolute value of.

[0046] The advantage of this setup is that, during the subsequent training of the dynamic Gaussian scene reconstruction model, the obtained spatial feature overlap is first sorted in descending order of angular difference. Then, the angular difference between different viewpoints is determined. Based on the determined angular difference between different viewpoints, a camera association graph corresponding to the dynamic scene is formed. The camera association graph can well take into account the overlap and difference between cameras, providing geometric prior guidance for adaptive viewpoint selection and targeted optimization of highly overlapping visible Gaussian regions during subsequent training. This ensures that viewpoint pairs with large angular differences but still feature overlap are prioritized during iteration, fully exploring the geometric constraints between multiple viewpoints to improve multi-view consistency and rendering quality.

[0047] In one specific embodiment, Figure 2 This is a schematic diagram of multi-view relationships provided by the present invention, such as... Figure 2 As shown, Figure 2 In the diagram, (a) represents (a) based on feature overlap association and ranking. Here, id:1, id:2, id:3, id:4, id:5, id:6, etc., represent different cameras. For the purple camera id:1, it is the selected target camera and can observe purple feature points (the first feature point). Yellow cameras id:2, id:3, id:6, etc., have multi-view overlap with the purple camera id:1, and are therefore identified as matching cameras for feature matching, able to observe most purple feature points (the first feature point, i.e., the number of shared feature points). White cameras id:4, id:5, etc., have no view overlap with the selected purple camera id:1, and are therefore identified as feature mismatched cameras. The light blue feature points are unobservable feature points. The spatial feature overlap is constructed based on the number of shared feature points. This involves treating the shared view matrix M, filled with the number of shared feature points, as a weighted undirected graph adjacency matrix after determining the number of shared feature points. The viewpoints are the nodes, and the elements of the shared feature points are the elements of the shared feature points. As edge weights. For each target perspective. , search target perspective All associated perspectives with a value greater than 0 in the common-view matrix M , will be related perspective According to the right of the border Sort in descending order from largest to smallest. Figure 2 (b) represents angle calculation and sorting. Specifically, it involves calculating the angles of the associated cameras of the purple camera id:1, i.e., determining the viewpoint pair based on the spatial overlap. A viewpoint pair consists of any two different viewpoints at the same time (e.g., id:1 and id:6, where the angle between the viewpoints of id:1 and id:6 is...). ).like Figure 2As shown in (c), this represents the maximum angle association list for each camera. The list includes the camera ID and the associated camera ID corresponding to the viewpoint ID of each camera. The list is generated by extracting the top G viewpoints (e.g., G=2, etc.). Figure 2 (As shown in the green box in section (c), the top G viewpoints selected in the cropping sort correspond to the target-associated camera IDs. This embodiment does not limit this selection) as the target viewpoints. The optimal associated neighbors are determined, and the final association list generated based on all the optimal associated neighbors is the spatial feature overlap. The target associated camera IDs extracted from the associated camera IDs are used as the batch size, which is used in the model training process.

[0048] In one specific embodiment, after determining the spatial feature overlap based on image samples and establishing the viewpoint difference between different viewpoints of the image samples based on the spatial feature overlap, the method further includes: constructing spatial geometric constraints. During the model training phase, the camera association graph constructed from the viewpoint difference between different viewpoints is transformed into multi-view geometric consistency constraints, and a spatial constraint loss term is adaptively constructed. Spatial constraint loss term This is used to improve the reconstruction stability of dynamic Gaussian scene reconstruction models under complex lighting and sparse viewpoints. Spatial constraint loss term. The calculation is shown in formula (2).

[0049] (2) In formula (2), This represents the loss defined by native 3D Gaussian sputtering between the real image and the rendered image from the current training perspective (primary perspective); This represents the spatial constraint weight coefficient, used to adjust the strength of the auxiliary viewpoint constraint; This indicates that the target associated camera ID extracted from the associated camera IDs is used as the batch size during model training. Indicates the first Real observation images from a related perspective, Indicates the first Rendered and predicted images from a related viewpoint; The calculation is shown in formula (3).

[0050] (3) In formula (3), It represents the mean absolute error between the ground truth image and the rendered image, and is responsible for constraining the absolute accuracy of pixel values; 1-SSIM represents the structural similarity metric; 1-SSIM represents the structural distortion, which is responsible for constraining high-frequency structural features of the image such as edges and textures. This represents the hyperparameter coefficients used to balance the weights of pixel-level errors and structural differences.

[0051] The advantage of this setup is that, during backpropagation, the spatial constraint loss term... The dynamic Gaussian scene reconstruction model is required to optimize the current viewpoint while also ensuring the rendering accuracy of its associated viewpoints. By incorporating associated viewpoints with good geometric gradients into the loss calculation, ambiguities in depth estimation or feature representation of the primary viewpoint can be corrected.

[0052] In one specific embodiment, the inter-frame constraint relationship of adjacent frames at the same viewpoint includes a trajectory smoothing loss function and a deformation regularization loss function; constructing the inter-frame constraint relationship of adjacent frames at the same viewpoint for image samples includes: obtaining the actual position of the image sample; determining the trajectory smoothing loss function based on the actual position and the smoothed position; wherein, the smoothed position is determined based on a pre-established short temporal window; obtaining the rate of change of eigenvalues ​​of the covariance matrix; and determining the deformation regularization loss function based on the rate of change of eigenvalues ​​of the covariance matrix.

[0053] Specifically, the actual positions of image samples are obtained; the trajectory smoothing loss function is determined based on the actual and smoothed positions, mainly by establishing a short temporal window. This short temporal window maintains the motion trajectory of each Gaussian element within N consecutive frames. Since independent frame-by-frame optimization can easily lead to unreasonable jumps in Gaussian element positions between adjacent frames, trajectory smoothing constraints are introduced to ensure temporal consistency of the motion. The trajectory smoothing loss function... The calculation is shown in formula (4).

[0054] (4) In formula (4), Indicates the weighting factor. The first image sample represents the... One perspective The actual location at that moment The first image sample represents the... Smooth position from each viewpoint. Smooth position The calculation is shown in formula (5).

[0055] (5) In formula (5), Indicates a short time window. Indicates Gaussian weights, The first image sample represents the... One perspective The actual location at any given moment.

[0056] Specifically, obtain the rate of change of the eigenvalues ​​of the covariance matrix; determine the deformation regularization loss function based on the rate of change of the eigenvalues ​​of the covariance matrix, and the deformation regularization loss function... The calculation is shown in formula (6).

[0057] (6) In formula (6), Indicates the weighting factor. The first image sample represents the... One perspective The rate of change of the eigenvalues ​​of the covariance matrix at time t. This represents the threshold for abnormal deformation.

[0058] The advantage of this setup is that it further makes full use of the spatiotemporal relationship between adjacent frames from the same camera perspective, designs a short temporal window and deformation accumulation constraint strategy to effectively suppress motion drift caused by the independence of frame optimization, determines the deformation regularization loss function, and enhances the geometric consistency and motion continuity of the Gaussian sphere in the time dimension based on the deformation regularization loss function.

[0059] In one specific embodiment, when determining the trajectory smoothing loss function and deformation regularization loss function Then, based on the trajectory smoothing loss function and deformation regularization loss function Determine the global timing loss Global timing loss The calculation is shown in formula (7).

[0060] (7) In formula (7), Indicates in Real-time observation images; Indicates in Rendering the observed image at any given moment; Indicates in Real-time observation images; Indicates in Rendering observation images at any given moment.

[0061] The advantage of this setup is that it results in global timing loss. Not only must the image rendering before and after the current time step meet the basic reconstruction quality, but it also requires a trajectory smoothing loss function. and deformation regularization loss function This provides a strong geometric prior guide for the dynamic evolution of Gaussian elements, ensuring the visual temporal coherence and physical motion consistency of the generated videos.

[0062] In one specific embodiment, further, in determining the global timing loss... Next, determine the total loss of iterative optimization. Total loss during iterative optimization The calculation is shown in formula (8).

[0063] (8) In formula (8), Represents the spatial constraint loss term. The calculation is shown in formula (2); The weighting factor representing the timing constraints; Represents the global timing loss. The calculation is shown in formula (7).

[0064] In one specific embodiment, a basic dynamic Gaussian scene reconstruction model is trained based on image samples, the correlation between viewpoint differences and the constraint relationships between adjacent frames at the same viewpoint, to obtain a dynamic Gaussian scene reconstruction model. This includes: performing a first-stage basic optimization training on the basic dynamic Gaussian scene reconstruction model based on image samples and a basic loss function to obtain a first-trained dynamic Gaussian scene reconstruction model; performing a second-stage spatiotemporal detail optimization training on the first-trained dynamic Gaussian scene reconstruction model based on the viewpoint differences between different viewpoints, the constraint relationships between adjacent frames at the same viewpoint, and the total loss of iterative optimization to obtain a second-trained dynamic Gaussian scene reconstruction model; and performing a third-stage global accuracy calibration training on the second-trained dynamic Gaussian scene reconstruction model based on the target total loss to obtain a dynamic Gaussian scene reconstruction model.

[0065] Specifically, based on image samples and the fundamental loss function The first stage of basic optimization training is performed on the basic dynamic Gaussian scene reconstruction model to obtain the first trained dynamic Gaussian scene reconstruction model. The core objective of this first stage is to achieve global parameter initialization through the original dynamic Gaussian training paradigm, laying an unbiased foundation for subsequent optimizations. This stage adopts a random single-view training mode: in each iteration, a viewpoint (without a fixed reference frame) is randomly selected from all training images in the image samples. The parameters of the dynamic Gaussian set are updated based on the error between the rendered observation image and the real observation image from the randomly selected viewpoint. The loss function at this point is the basic loss function. .

[0066] The advantage of this setup is that it preserves the integrity of the scene's global structure at this stage, while avoiding overfitting to specific viewpoints through viewpoint randomness, ensuring global consistency in Gaussian parameter initialization, and providing a reliable starting point for subsequent fine-tuning optimization.

[0067] Specifically, based on the difference in viewpoints between different viewpoints, the constraint relationship between adjacent frames within the same viewpoint, and the total loss of iterative optimization. The first training dynamic Gaussian scene reconstruction model is subjected to a second stage of spatiotemporal detail optimization training to obtain the second training dynamic Gaussian scene reconstruction model. This mainly involves introducing a joint optimization mechanism that combines the perspective difference between different viewpoints and the constraint relationship between adjacent frames of the same viewpoint. The focus is on strengthening the spatiotemporal correlation of Gaussian parameters to achieve synchronous improvement of dynamic details and multi-view consistency.

[0068] The advantage of this setup is that, based on the difference in viewpoints between different perspectives and the constraint relationships between adjacent frames within the same viewpoint, a spatiotemporal correlation graph G is constructed. G stores the degree of spatiotemporal correlation between different frames. In each iteration, the Top-K images (e.g., K=2 means selecting the two images with the highest correlation to the current reference frame each time; this embodiment does not impose this limitation) with the highest correlation to the current reference frame are selected from G to form a training group. This group includes one high spatial correlation frame (to enhance spatial consistency) and one high temporal correlation frame (to enhance temporal coherence). In summary, through this stage of refined optimization, the suboptimal Gaussian sphere results caused by single-viewpoint optimization can be reduced, and its ability to represent the details of dynamic regions can be significantly improved.

[0069] In one specific embodiment, the second training dynamic Gaussian scene reconstruction model is subjected to a third-stage global accuracy calibration training based on the target total loss to obtain a dynamic Gaussian scene reconstruction model. This includes: performing a third-stage global accuracy calibration training on the second training dynamic Gaussian scene reconstruction model based on the target total loss to obtain a candidate dynamic Gaussian scene reconstruction model; obtaining a distributed training architecture; and performing distributed accelerated training on the candidate dynamic Gaussian scene reconstruction model based on the distributed training architecture to obtain a dynamic Gaussian scene reconstruction model.

[0070] In this step, the total target loss is calculated as shown in formula (9).

[0071] (9) In formula (9), This represents the total number of different viewpoints for the image samples. This full-cumulative calculation method aims to achieve a balance in the overall quality of dynamic scenes: it forces the optimization of all Gaussian sphere properties to minimize the sum of errors in the reconstruction results at different viewpoints and frames, further improving the geometric consistency and motion continuity of the Gaussian sphere.

[0072] Specifically, the second-stage global accuracy calibration training is performed on the second-stage dynamic Gaussian scene reconstruction model based on the target total loss to obtain the candidate dynamic Gaussian scene reconstruction model. The third-stage global accuracy calibration training mainly focuses on eliminating error accumulation or overfitting caused by local optimization, so that the entire dynamic scene has better global consistency. This stage adopts a full image loss accumulation strategy, that is, the rendering loss of all view images (including all time frames) in the training set is accumulated and then backpropagated.

[0073] In one specific embodiment, Figure 3 This is a schematic diagram of the training process of the dynamic Gaussian scene reconstruction model provided by the present invention, as shown below. Figure 3 As shown, the training steps of the dynamic Gaussian scene reconstruction model include steps 301, 302 and 303.

[0074] Step 301: Perform the first stage of basic optimization training on the basic dynamic Gaussian scene reconstruction model based on image samples and basic loss function to obtain the first training dynamic Gaussian scene reconstruction model.

[0075] Specifically, the image samples are input into the basic dynamic Gaussian scene reconstruction model, and the basic dynamic Gaussian scene reconstruction model is subjected to the first stage of basic optimization training. Specifically, a random single-view training mode is adopted. In each iteration, a real image (without a fixed reference frame) from a viewpoint is randomly selected from all training images in the image samples. The error of the rendered image is used to train the dynamic Gaussian set and the basic loss function in the first stage of basic optimization training to obtain the first training dynamic Gaussian scene reconstruction model.

[0076] The advantage of this setup is that it preserves the global structural integrity of the dynamic scene while avoiding overfitting to specific viewpoints through viewpoint randomness, ensuring global consistency of the error in the initialization of Gaussian parameters in the dynamic Gaussian set, and providing a reliable starting point for subsequent fine-tuning optimization.

[0077] Step 302: Based on the perspective difference between different perspectives, the constraint relationship between adjacent frames of the same perspective, and the total loss of iterative optimization, perform the second stage of spatiotemporal detail optimization training on the first training dynamic Gaussian scene reconstruction model to obtain the second training dynamic Gaussian scene reconstruction model.

[0078] Step 303: Perform a third-stage global accuracy calibration training on the second training dynamic Gaussian scene reconstruction model based on the target total loss to obtain the dynamic Gaussian scene reconstruction model.

[0079] In one specific embodiment, after determining the candidate dynamic Gaussian scene reconstruction model after training, a distributed training architecture is further obtained; the candidate dynamic Gaussian scene reconstruction model is then trained in a distributed accelerated manner based on the distributed training architecture to obtain the dynamic Gaussian scene reconstruction model.

[0080] Distributed training architectures can be, for example, using the Distributed Data Parallel (DDP) framework of PyTorch (a widely used deep learning framework) to wrap multiple Graphics Processing Units (GPUs) architectures for Gaussian rendering, ensuring parameter synchronization among the GPUs. Further distributed accelerated training is then applied to the candidate dynamic Gaussian scene reconstruction model. During training, each training frame is evenly divided according to the number of distributed training architectures, generating a corresponding number of image segments. Each distributed training architecture is responsible for rendering and optimizing its assigned image segment region. This approach not only achieves a balanced distribution of computational load but also maintains the local continuity of the image, which is beneficial for the local optimization of Gaussian primitives.

[0081] The advantage of this setup is that it introduces a spatial partitioning strategy, dividing the complete sparse point cloud of the dynamic scene into multiple sub-blocks based on spatial location. Each sub-block is allocated to different distributed training architecture devices for storage and processing. This distributed storage scheme effectively alleviates the limitations of single-GPU memory, enabling the system to handle larger-scale scene data. When a GPU needs to access the sparse point cloud stored on other GPUs, only the Gaussian byte subset necessary for rendering the current image segment is transferred, minimizing communication overhead. This distributed training architecture is fully compatible with the aforementioned mechanism for fusing viewpoint differences between different viewpoints and constraints between adjacent frames within the same viewpoint. Through this distributed design, multi-GPU computing resources can be fully utilized, significantly improving the training efficiency of the dynamic Gaussian scene reconstruction model while maintaining the integrity and consistency of the algorithm.

[0082] In one specific embodiment, Figure 4 This is a schematic diagram of the distributed loading and training process provided by the present invention, as follows: Figure 4 As shown, the distributed training architecture performs distributed accelerated training on the candidate dynamic Gaussian scene reconstruction model to obtain the specific accelerated training steps of the dynamic Gaussian scene reconstruction model, including steps 401, 402, 403, 404, 405 and 406.

[0083] Step 401: Initialize the distributed training architecture.

[0084] Specifically, the distributed training architecture is initialized to establish the communication foundation between the GPUs in the distributed training architecture.

[0085] Step 402: Load the complete dataset of the dynamic scene.

[0086] Specifically, a complete dynamic scene dataset is loaded from image samples. The complete dynamic scene dataset includes a sequence image dataset and an initial point cloud dataset.

[0087] Step 403: Perform segmentation processing on the sequence image dataset to determine the target segmentation region set.

[0088] Specifically, image segmentation processing is performed on each image sequence in the image sequence dataset to determine a set of target segmentation regions. The set of target segmentation regions includes at least one target segmentation region, but this embodiment does not limit this.

[0089] Step 404: Spatially divide the initial point cloud dataset to determine the target block set.

[0090] Specifically, the initial point clouds in the initial point cloud dataset are spatially partitioned to determine the target block set. The target block set includes at least one target block, but this embodiment does not limit this.

[0091] In one specific embodiment, the execution of steps 403 and 404 does not have a specific order. For example, step 403 can be executed first, followed by step 404; or step 404 can be executed first, followed by step 403; or steps 403 and 404 can be executed simultaneously. This embodiment does not limit this.

[0092] In one specific embodiment, the image segmentation processing of each sequence image in the sequence image dataset and the spatial block processing of each initial point cloud in the initial point cloud dataset are performed based on the number of GPUs for uniform segmentation. Then, each GPU, each target segmentation region and each target block are numbered. This embodiment does not limit this process.

[0093] Step 405: Input the target segmentation region and target block into the GPU in the distributed training architecture for local rendering, and determine the gradient calculation results.

[0094] Specifically, based on the numbering of each GPU, each target segmentation region, and each target block, the GPUs and target segmentation regions with the same number are input into the GPUs with the same number for local rendering and gradient calculation. Furthermore, during the parallel execution of local rendering and gradient calculation tasks by each GPU, "point cloud transfer" can be performed between GPUs as needed. This method significantly reduces the bandwidth load on the GPU mechanism.

[0095] Step 406: Update the parameters of the candidate dynamic Gaussian scene reconstruction model based on all gradient calculation results to obtain the dynamic Gaussian scene reconstruction model.

[0096] Specifically, after obtaining the gradient calculation results from the GPU output, the target gradient calculation result is determined based on the average of all gradient calculation results. Then, the parameters of the candidate dynamic Gaussian scene reconstruction model are updated synchronously based on the target gradient calculation result, thereby achieving distributed accelerated training of the candidate dynamic Gaussian scene reconstruction model and obtaining the dynamic Gaussian scene reconstruction model.

[0097] The advantage of this setup is that, while maintaining the spatiotemporal constraints of the candidate dynamic Gaussian scene reconstruction model, the parameters of the candidate dynamic Gaussian scene reconstruction model are updated synchronously based on the target gradient calculation results, thus obtaining the dynamic Gaussian scene reconstruction model, which significantly shortens the overall training cycle of complex dynamic scenes.

[0098] In summary, the dynamic Gaussian scene reconstruction method proposed in this invention, after establishing the viewpoint difference between different viewpoints and the constraint relationship between adjacent frames of the same viewpoint, has the advantage of a three-stage progressive training approach designed to address the complexity of Gaussian parameter optimization in dynamic scenes. Through a coarse-to-fine parameter iteration method, it achieves a smooth transition from the initial construction to precise optimization of the dynamic Gaussian set. Combined with a distributed training architecture, it improves the scene reconstruction accuracy and temporal coherence of the dynamic Gaussian scene reconstruction model while ensuring training efficiency.

[0099] This invention provides a method for reconstructing dynamic Gaussian scenes. The method involves acquiring image information to be reconstructed, inputting this image information into a dynamic Gaussian scene reconstruction model, and obtaining the dynamic Gaussian scene reconstruction result output by the model. The dynamic Gaussian scene reconstruction model is trained based on the spatial feature overlap between different viewpoints, the viewpoint difference between different viewpoints, and the correlation of constraints between adjacent frames within the same viewpoint, all determined from image samples. This invention addresses the shortcomings of existing technologies that fail to fully utilize the spatiotemporal correlation between different viewpoints and between adjacent frames within the same viewpoint, leading to suboptimal optimization results in dynamic scene reconstruction—such as local artifacts, motion discontinuities, and insufficient temporal coherence. The method utilizes a dynamic Gaussian scene reconstruction model trained based on the spatial feature overlap between different viewpoints, the viewpoint difference between different viewpoints, and the correlation of constraints between adjacent frames within the same viewpoint, all determined from image samples. This allows for the reconstruction of dynamic Gaussian scenes from the image information, improving reconstruction accuracy and rendering fidelity, and effectively addressing issues such as insufficient consistency in reconstruction between different viewpoints, local reconstruction artifacts, and limited detail fidelity.

[0100] The following describes the dynamic Gaussian scene reconstruction apparatus provided by the present invention. The dynamic Gaussian scene reconstruction apparatus described below and the dynamic Gaussian scene reconstruction method described above can be referred to in correspondence.

[0101] Figure 5This is a schematic diagram of the structure of the dynamic Gaussian scene reconstruction device provided by the present invention, with reference to... Figure 5 As shown, the dynamic Gaussian scene reconstruction device 500 includes: an information acquisition module 501 and a scene reconstruction module 502; wherein, The information acquisition module 501 is used to acquire information about the image to be reconstructed.

[0102] The scene reconstruction module 502 is used to input the image information to be reconstructed into the dynamic Gaussian scene reconstruction model and obtain the dynamic Gaussian scene reconstruction result output by the dynamic Gaussian scene reconstruction model. The dynamic Gaussian scene reconstruction model is trained based on the spatial feature overlap between different viewpoints determined by the image samples, the viewpoint difference between different viewpoints, and the correlation of the constraint relationship between adjacent frames of the same viewpoint.

[0103] In one example embodiment, the device further includes a model training module. The model training module is configured to: acquire image samples; determine spatial feature overlap based on the image samples, and establish viewpoint differences between different viewpoints of the image samples based on the spatial feature overlap; construct inter-frame constraint relationships between adjacent frames of the same viewpoint of the image samples; and train a basic dynamic Gaussian scene reconstruction model based on the correlation between the image samples, the viewpoint differences between different viewpoints, and the inter-frame constraint relationships between adjacent frames of the same viewpoint, to obtain a dynamic Gaussian scene reconstruction model.

[0104] In one example embodiment, the model training module determines the spatial feature overlap based on image samples, specifically by: generating an inverted index table based on the image samples; calculating the number of shared feature points based on the inverted index table; wherein the number of shared feature points represents the number of shared feature points between any two viewpoints in the same frame; and constructing the spatial feature overlap based on the number of shared feature points.

[0105] In one example embodiment, the model training module establishes the viewpoint difference between different viewpoints of image samples based on the spatial feature overlap. Specifically, it is used to: determine viewpoint pairs based on the spatial feature overlap; a viewpoint pair consists of any two different viewpoints at the same time; and determine the viewpoint difference between different viewpoints based on the optical axis direction vectors corresponding to each viewpoint in the viewpoint pair.

[0106] In one example embodiment, the inter-frame constraint relationship from the same viewpoint includes a trajectory smoothing loss function and a deformation regularization loss function.

[0107] In one example embodiment, the model training module constructs inter-frame constraint relationships between adjacent frames of image samples from the same viewpoint, specifically for: obtaining the actual position of the image samples; determining a trajectory smoothing loss function based on the actual position and the smoothed position; wherein the smoothed position is determined based on a pre-established short temporal window; obtaining the rate of change of eigenvalues ​​of the covariance matrix; and determining a deformation regularization loss function based on the rate of change of eigenvalues ​​of the covariance matrix.

[0108] In one example embodiment, the model training module trains a basic dynamic Gaussian scene reconstruction model based on image samples, the correlation between viewpoint differences and the constraints between adjacent frames at the same viewpoint, to obtain a dynamic Gaussian scene reconstruction model. Specifically, this involves: performing a first-stage basic optimization training on the basic dynamic Gaussian scene reconstruction model based on image samples and a basic loss function to obtain a first-stage trained dynamic Gaussian scene reconstruction model; performing a second-stage spatiotemporal detail optimization training on the first-stage trained dynamic Gaussian scene reconstruction model based on the viewpoint differences between different viewpoints, the constraints between adjacent frames at the same viewpoint, and the total iterative optimization loss to obtain a second-stage trained dynamic Gaussian scene reconstruction model; and performing a third-stage global accuracy calibration training on the second-stage trained dynamic Gaussian scene reconstruction model based on the target total loss to obtain a dynamic Gaussian scene reconstruction model.

[0109] In one example embodiment, the model training module performs a third-stage global accuracy calibration training on the second trained dynamic Gaussian scene reconstruction model based on the target total loss to obtain a dynamic Gaussian scene reconstruction model. Specifically, this involves: performing a third-stage global accuracy calibration training on the second trained dynamic Gaussian scene reconstruction model based on the target total loss to obtain a candidate dynamic Gaussian scene reconstruction model; obtaining a distributed training architecture; and performing distributed accelerated training on the candidate dynamic Gaussian scene reconstruction model based on the distributed training architecture to obtain the dynamic Gaussian scene reconstruction model.

[0110] The apparatus of this embodiment can be used to execute the method of any embodiment in the side embodiment of the method for reconstructing dynamic Gaussian scenes. Its specific implementation process and technical effects are similar to those in the side embodiment of the method for reconstructing dynamic Gaussian scenes. For details, please refer to the detailed description in the side embodiment of the method for reconstructing dynamic Gaussian scenes, which will not be repeated here.

[0111] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 6As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640. The processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a dynamic Gaussian scene reconstruction method. This method includes: acquiring image information to be reconstructed; inputting the image information to be reconstructed into a dynamic Gaussian scene reconstruction model to obtain the dynamic Gaussian scene reconstruction result output by the dynamic Gaussian scene reconstruction model; wherein the dynamic Gaussian scene reconstruction model is trained based on the spatial feature overlap between different viewpoints determined by image samples, the viewpoint difference between different viewpoints, and the correlation of constraints between adjacent frames of the same viewpoint.

[0112] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0113] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the dynamic Gaussian scene reconstruction method provided by the above methods. The method includes: acquiring image information to be reconstructed; inputting the image information to be reconstructed into a dynamic Gaussian scene reconstruction model to obtain the dynamic Gaussian scene reconstruction result output by the dynamic Gaussian scene reconstruction model; wherein the dynamic Gaussian scene reconstruction model is trained based on the spatial feature overlap between different viewpoints determined by image samples, the viewpoint difference between different viewpoints, and the correlation of constraint relationships between adjacent frames of the same viewpoint.

[0114] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a method for reconstructing a dynamic Gaussian scene provided by the above methods. The method includes: acquiring image information to be reconstructed; inputting the image information to be reconstructed into a dynamic Gaussian scene reconstruction model to obtain a dynamic Gaussian scene reconstruction result output by the dynamic Gaussian scene reconstruction model; wherein the dynamic Gaussian scene reconstruction model is trained based on the spatial feature overlap between different viewpoints determined by image samples, the viewpoint difference between different viewpoints, and the correlation of constraint relationships between adjacent frames of the same viewpoint.

[0115] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0116] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for reconstructing a dynamic Gaussian scene, characterized in that, include: Obtain the image information to be reconstructed; The image information to be reconstructed is input into the dynamic Gaussian scene reconstruction model to obtain the dynamic Gaussian scene reconstruction result output by the dynamic Gaussian scene reconstruction model; wherein, the dynamic Gaussian scene reconstruction model is trained based on the spatial feature overlap between different viewpoints determined by image samples, the viewpoint difference between different viewpoints, and the correlation of constraint relationships between adjacent frames of the same viewpoint.

2. The method for reconstructing a dynamic Gaussian scene according to claim 1, characterized in that, The dynamic Gaussian scene reconstruction model is trained based on the following steps: Obtain the image sample; The spatial feature overlap is determined based on the image samples, and the viewpoint difference between the different viewpoints of the image samples is established based on the spatial feature overlap. Construct the inter-frame constraint relationship of the image samples from the same viewpoint; The basic dynamic Gaussian scene reconstruction model is trained based on the image samples, the perspective difference between different perspectives, and the correlation between the constraints between adjacent frames of the same perspective, to obtain the dynamic Gaussian scene reconstruction model.

3. The method for reconstructing a dynamic Gaussian scene according to claim 2, characterized in that, Determining the spatial feature overlap based on the image samples includes: Generate an inverted index table based on the image samples; The number of shared feature points is calculated based on the inverted index table; wherein, the number of shared feature points represents the number of shared feature points between any two viewpoints in the same frame; The spatial feature overlap is constructed based on the number of shared feature points.

4. The method for reconstructing a dynamic Gaussian scene according to claim 2, characterized in that, The step of establishing the viewpoint difference degree between different viewpoints of the image samples based on the spatial feature overlap includes: Viewpoint pairs are determined based on the overlap of the spatial features; each viewpoint pair consists of any two different viewpoints at the same time. The degree of difference in viewpoints between different viewpoints is determined based on the optical axis direction vectors corresponding to each viewpoint in the viewpoint pair.

5. The method for reconstructing a dynamic Gaussian scene according to claim 2, characterized in that, The constraint relationship between adjacent frames from the same viewpoint includes a trajectory smoothing loss function and a deformation regularization loss function; The construction of the inter-frame constraint relationship of the same viewpoint for the image samples includes: Obtain the actual location of the image sample; The trajectory smoothing loss function is determined based on the actual position and the smoothed position; wherein the smoothed position is determined based on a pre-established short time window; Obtain the rate of change of eigenvalues ​​of the covariance matrix; The deformation regularization loss function is determined based on the rate of change of the eigenvalues ​​of the covariance matrix.

6. The method for reconstructing a dynamic Gaussian scene according to claim 2, characterized in that, The step of training a basic dynamic Gaussian scene reconstruction model based on the correlation between the image samples, the viewpoint differences between different viewpoints, and the constraint relationships between adjacent frames of the same viewpoint to obtain the dynamic Gaussian scene reconstruction model includes: Based on the image samples and the basic loss function, the basic dynamic Gaussian scene reconstruction model is subjected to the first stage of basic optimization training to obtain the first training dynamic Gaussian scene reconstruction model. The second stage of spatiotemporal detail optimization training is performed on the first training dynamic Gaussian scene reconstruction model based on the perspective difference between different perspectives, the constraint relationship between adjacent frames of the same perspective, and the total loss of iterative optimization, to obtain the second training dynamic Gaussian scene reconstruction model; wherein, the total loss of iterative optimization is determined based on the constraint relationship between adjacent frames of the same perspective; The second training dynamic Gaussian scene reconstruction model is subjected to a third-stage global accuracy calibration training based on the target total loss to obtain the dynamic Gaussian scene reconstruction model.

7. The method for reconstructing a dynamic Gaussian scene according to claim 6, characterized in that, The step of performing a third-stage global accuracy calibration training on the second trained dynamic Gaussian scene reconstruction model based on the target total loss to obtain the dynamic Gaussian scene reconstruction model includes: Based on the target total loss, the second training dynamic Gaussian scene reconstruction model is subjected to a third-stage global accuracy calibration training to obtain a candidate dynamic Gaussian scene reconstruction model. Obtain the distributed training architecture; The candidate dynamic Gaussian scene reconstruction model is trained in a distributed accelerated manner based on the distributed training architecture to obtain the dynamic Gaussian scene reconstruction model.

8. A device for reconstructing a dynamic Gaussian scene, characterized in that, include: The information acquisition module is used to acquire information about the image to be reconstructed. The scene reconstruction module is used to input the image information to be reconstructed into the dynamic Gaussian scene reconstruction model to obtain the dynamic Gaussian scene reconstruction result output by the dynamic Gaussian scene reconstruction model; wherein, the dynamic Gaussian scene reconstruction model is trained based on the spatial feature overlap between different viewpoints determined by image samples, the viewpoint difference between different viewpoints, and the correlation of the constraint relationship between adjacent frames of the same viewpoint.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method for reconstructing a dynamic Gaussian scene as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method for reconstructing a dynamic Gaussian scene as described in any one of claims 1 to 7.

Images