A dynamic fuzzy scene three-dimensional reconstruction method based on 3DGS technology

Abstract

The application discloses a dynamic fuzzy scene three-dimensional reconstruction method based on a 3DGS technology, which comprises the following steps: initializing a three-dimensional reconstruction system; data input and preprocessing; 3DGS-based differentiable rendering prediction; initial estimation of a camera pose; calculation of pixel-level residuals of an RGB camera image; reverse distribution of the pixel-level residuals to Gaussian-level residuals to form photometric residuals and depth residuals describing Gaussian dynamics and staticity; calculation of dynamic auxiliary evidence; dynamic evidence fusion and dynamic confidence update; Gaussian body soft shielding and weight adjustment based on the dynamic confidence; selection of a local Gaussian set for camera pose optimization; addition of new scene information in the form of Gaussians to a global map, Gaussian merging and migration, and acquisition of a dense renderable map; and output of visualized camera poses and the dense renderable map according to the static Gaussian set and the camera pose in the above steps.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision, and relates to a dynamic fuzzy scene three-dimensional reconstruction method based on a 3DGS technology. BACKGROUND

[0002] Three-dimensional reconstruction of outdoor scenes is an important research direction of computer vision and graphics, and is also a key component of current embodied intelligence technology, which is crucial for many applications such as autonomous driving, augmented / virtual reality, robot navigation, simultaneous localization and mapping, and the core goal is to efficiently construct an accurate and complete three-dimensional digital representation of the real scene.

[0003] In recent years, implicit neural representation methods such as Neural Radiance Fields (NeRF) have made breakthroughs in view synthesis quality, but their training and rendering speed is slow, and the model expression is not intuitive enough.

[0004] Three-dimensional Gaussian (3D Gaussian Splatting, 3DGS) is a brand new display scene representation method. It uses a large number of three-dimensional Gaussian ellipsoids to simulate the scene, which can achieve high-quality real-time rendering. Its core advantage lies in its ability to model continuous surfaces, reduce memory requirements, improve noise and outlier handling, enhance hole filling and scene repair. And it has significant advantages in training efficiency and editability compared to NeRF, showing great application potential.

[0005] Traditional three-dimensional Gaussian still faces many challenges in high-quality three-dimensional reconstruction when dealing with complex, dynamic, large-scale outdoor scenes due to data acquisition and the scene itself. Severe lighting changes, bad weather, and the ubiquitous weak texture, repetitive texture, shiny or transparent surfaces in the scene continuously interfere with the process of inferring three-dimensional information from two-dimensional images. Whether it is the accuracy of pose estimation in Structure from Motion (SfM) or the learning of Gaussian parameters (position, shape, color, opacity) in the three-dimensional Gaussian optimization process, it will be severely affected. Dynamic objects in the scene (such as pedestrians and vehicles) not only can cause SfM to fail, but also can produce suspended artifacts or blurred areas in the three-dimensional Gaussian reconstruction results, destroying the static representation of the scene. SUMMARY

[0006] To solve the above technical problems, the purpose of the present application is to provide a dynamic fuzzy scene three-dimensional reconstruction method based on 3DGS technology, which can mark and remove interference in dynamic scenes and solve the problem of artifacts and blurring in the reconstruction process.

[0007] The dynamic fuzzy scene three-dimensional reconstruction method based on the 3DGS technology of the present application comprises:

[0008] Step 1: initialize the three-dimensional reconstruction system, define a three-dimensional Gaussian set to represent the scene;

[0009] Step 2: input the RGB camera image by frame and perform de-distortion, color correction, and depth filtering processing;

[0010] Step 3: for the continuous frame RGB camera image, make a 3DGS-based differentiable rendering prediction to predict the color and depth on the current camera image plane from the current camera pose, and obtain the color prediction value and depth prediction value for each pixel;

[0011] Step 4: perform initial estimation on the camera pose;

[0012] Step 5: calculate the pixel-level residual error of the RGB camera image, and inversely assign the pixel-level residual error to the Gaussian-level residual error to form photometric residual error and depth residual error describing the dynamic and static nature of the Gaussian;

[0013] Step 6: calculate other dynamic auxiliary evidence of the RGB camera image, input the optical flow data and semantic segmentation information of the RGB camera image, and obtain the optical flow inconsistency and semantic prior information;

[0014] Step 7: fuse the multi-modal dynamic evidence to obtain instantaneous dynamic evidence; and then perform time smoothing with the historical dynamic confidence to obtain stable dynamic confidence;

[0015] Step 8: according to the dynamic confidence of each Gaussian, divide the static Gaussian set and the dynamic Gaussian set, and perform soft shielding on the Gaussian in the dynamic region; and then adjust the weight of each Gaussian according to its dynamic confidence;

[0016] Step 9: based on the camera observation information of the current frame, select the relevant local Gaussian set to use an optimization algorithm to optimize the camera pose, and obtain the optimized camera pose;

[0017] Step 10: add new scene information in the form of Gaussian to the global map, and perform Gaussian merging and migration to obtain a dense renderable map;

[0018] Step 11: output the visualized camera pose and the dense renderable map according to the static Gaussian set and the camera pose in the above steps.

[0019] The application proposes a dynamic fuzzy scene three-dimensional reconstruction method based on 3DGS technology, which has at least the following beneficial effects:

[0020] 1. Improved accuracy in dynamic scene processing: By introducing dynamic segmentation and confidence update mechanisms directly in the 3D Gaussian space, this method can accurately identify and suppress the interference of dynamic objects, thereby improving the stability of SLAM in dynamic environments. Compared to traditional methods that rely on feature point rejection or optical flow analysis, this system combines geometric and semantic information from a global perspective, significantly improving the robustness and deployability of dynamic mapping and localization.

[0021] 2. Enhanced scene reconstruction quality: With high-resolution 3D Gaussian point cloud rendering technology, this system achieves a dense and realistic scene reconstruction effect, especially in dynamic and low-texture environments, while still maintaining excellent detail restoration capabilities. At the same time, it avoids artifacts and drift problems, and handles dynamic objects more accurately, ensuring the consistency of reconstruction quality.

[0022] 3. Support for real-time tasks: The combination of fast projection, dynamic confidence update, and local optimization modules enables efficient algorithms, improving the ability to process dynamic scenes in real time. The system can quickly respond to changing dynamic environments (such as traffic flow, crowds, etc.) without interrupting SLAM functions, making it suitable for real-time navigation and robot task execution.

[0023] 4. Reduced computational resource consumption: Through data compression strategies and local optimization techniques for Gaussian point clouds, this method significantly reduces memory and computational resource consumption compared to traditional dense mapping algorithms. This method is particularly suitable for embedded devices and resource-constrained environments.

[0024] 5. Easy operation and scalability: This method builds a lightweight SLAM system that does not require complex configuration through automated dynamic decision-making and optimization strategies, making it deployable without professional technical personnel. At the same time, the system is modular in design, allowing easy extension to different sensor inputs (such as RGB-D, lidar, etc.) and diverse scene requirements.

[0025] 6. Improved stability of robots and systems: In dynamic interference scenarios, the Gaussian confidence soft shielding mechanism ensures that the system can respond in real time to changes, avoiding localization and mapping failures. In addition, the separation modeling of dynamic targets and static scenes reduces the interruption rate of the system, enhancing overall stability.

[0026] 7. Improved overall task efficiency: Quickly complete localization and correct map drift in dynamic environments, avoiding additional repair costs. The system improves overall task efficiency from three aspects: efficient reconstruction, high robustness, and low resource consumption, and lays the foundation for subsequent complex tasks such as dynamic tracking and target segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1is a flow chart of a dynamic blurred scene three-dimensional reconstruction method based on 3DGS technology. DETAILED DESCRIPTION

[0028] As shown in the figure, the dynamic blurred scene three-dimensional reconstruction method based on 3DGS technology comprises the following steps: Figure 1

[0029] Step 1: initialize the three-dimensional reconstruction system, define a three-dimensional Gaussian set to represent the scene, specifically:

[0030] Step 1.1: define a three-dimensional Gaussian set Each Gaussian has the following parameters:

[0031] represents the center point of the scene where each Gaussian is located.

[0032] where , , represents the variance of each axis direction, , , , , , represents the covariance between different axes, the overall 3D space covariance matrix is used to describe the extension direction and scale of the three-dimensional Gaussian ellipsoid, and control its influence range.

[0033] represents the color information of the Gaussian.

[0034] represents the Gaussian weight, which is used to describe the contribution degree of the Gaussian in the rendering process.

[0035] represents the dynamic confidence of the Gaussian, the initial value is 0, which is used for subsequent dynamic object segmentation.

[0036] Step 1.2: define the camera intrinsic matrix , which represents the perspective projection model of the camera, and is used to describe the imaging characteristics of the camera to ensure the accuracy of the projection;

[0037]

[0038] wherein, and represent the focal length of the camera, and represent the principal point of the camera.

[0039] Step 1.3: define the camera pose​ ; wherein denotes a rotation of the camera, denotes a translation of the camera.

[0040] Step 2: Data input and pre-processing. The RGB camera image is input frame by frame and de-distortion, color correction, depth filtering processing is performed, specifically:

[0041] Step 2.1: Read sensor data per frame: RGB image .

[0042] Step 2.2: Pre-process the input, de-distort the image, correct the color, and filter the depth. This step can prevent dynamic misjudgment caused by sensor noise and improve the credibility of subsequent data.

[0043] Step 3: Differentiable rendering prediction based on 3DGS. For consecutive frame RGB camera images, do differentiable rendering prediction based on 3DGS, predict the color and depth on the current camera image plane from the current camera pose, get the color prediction value and depth prediction value for each pixel, specifically:

[0044] Step 3.1: According to the camera observation information of the current frame, including the camera's RGB image and depth image , project each pixel to the camera coordinate system to obtain the 3D point cloud; then fit the 3D point cloud into a Gaussian .

[0045] Step 3.2: Project each Gaussian to the current camera image plane under the current camera pose and camera intrinsic parameters , and transform to the camera coordinate system; calculate the coordinates of the Gaussian center in the camera coordinate system and the distance of the Gaussian along the camera Z axis :

[0046]

[0047]

[0048] wherein, is the z-axis coordinate of the Gaussian center in the camera coordinate system.

[0049] Step 3.3: Project the point in the camera coordinate system to the current camera image plane according to the following formula, to get the pixel coordinates and :

[0050]

[0051]

[0052] Step 3.4: Project the 3D covariance to the current camera image plane to get the shape of the Gaussians in that image plane:

[0053]

[0054] where, is the covariance matrix projected to the current camera image plane, is the projection Jacobian from the current camera image plane to 3D space.

[0055] Step 3.5: Render weight computation, to let each Gaussian affect multiple pixel neighborhoods in the current camera image plane, compute the raw weight of a Gaussian to any pixel:

[0056]

[0057] where the exponential term represents the Gaussian's influence on the neighborhood it lies in; the disparity term decays the Gaussian's influence on pixels far away; a very small regularization term prevents numerical instability, represents the pixel center of the Gaussian's projection.

[0058] Step 3.6: Weight normalization, to avoid the mutual accumulation of Gaussian weights, normalize each pixel based on the shared weight of all the affecting pixels, the formula is as follows:

[0059]

[0060] Step 3.7: Based on the normalized weight of the Gaussian , finally synthesize the color prediction value and depth prediction value of each pixel;

[0061] Weighted average all Gaussian color weights to get the pixel color prediction value:

[0062]

[0063] Weighted average all Gaussian depth weights to get the pixel depth prediction value:

[0064]

[0065] where, is the pixel color prediction value, This is the pixel depth prediction value.

[0066] Step 4: Perform initial estimation of camera pose, specifically as follows:

[0067] The camera observation information for the current frame includes the camera's RGB image. and depth images The initial Gaussian set, the estimated color and estimated depth obtained in step 3, are used for rapid optimization at the front end based on the principle of photometric consistency to obtain the optimized initial estimated camera pose. Optimizing the pose quickly at the front end helps obtain initial values ​​for convergence and reduces the computational burden of subsequent joint optimization.

[0068] Step 5: Calculate the pixel-level residuals of the RGB camera image, and then inversely distribute these pixel-level residuals to Gaussian-level residuals to form photometric and depth residuals that describe the dynamic and static properties of the Gaussian image. This provides data support for subsequent dynamic judgment and scene updates. Specifically:

[0069] Step 5.1: Calculate the pixel-level residual of the RGB camera image using the following formula:

[0070]

[0071] in, Represents pixels luminous residual, This represents the pixel color value observed by the sensor. This represents the pixel color prediction based on the current pose and the Gaussian set.

[0072]

[0073] in, Represents pixels depth residual, This represents the pixel color value observed by the sensor. This represents the pixel color prediction based on the current pose and the Gaussian set.

[0074] Step 5.2: Normalize the pixel residuals using Gaussian weights. Reverse assignment to Gaussian residuals:

[0075]

[0076] in, Represents the luminous residual of the i-th Gaussian;

[0077]

[0078] in, Let represent the depth residual of the i-th Gaussian.

[0079] Step 6: Calculate dynamic auxiliary evidence. Calculate other dynamic auxiliary evidence of RGB camera image, input optical flow data and semantic segmentation information of RGB camera image, get optical flow inconsistency and semantic prior information. Multi-modal dynamic auxiliary evidence can effectively reduce the misjudgment rate when relying only on photometric residual.

[0080] Optical flow inconsistency : Mark the dynamic possibility of Gauss based on the average error of forward and backward optical flow in the Gaussian projection area.

[0081] Semantic prior information : Based on semantic segmentation, identify dynamic objects through pre-trained semantic network, and improve the judgment accuracy of large objects (such as cars and pedestrians).

[0082] Step 7: Dynamic evidence fusion and dynamic confidence update. Fuse multi-modal dynamic evidence to get instantaneous dynamic evidence; then time smooth with historical dynamic confidence to get stable dynamic confidence, which is:

[0083] Step 7.1: Fusion to calculate instantaneous dynamic evidence , the formula is as follows:

[0084]

[0085] Among them, represents the instantaneous dynamic evidence, represents the photometric residual, represents the depth residual, represents the optical flow inconsistency, represents the semantic prior information, , , , respectively represent the weight of each evidence, used to balance the importance of each dynamic evidence.

[0086] Step 7.2: Based on the instantaneous dynamic evidence of the current frame and the historical dynamic confidence , update the dynamic confidence of each Gauss, realize the time smooth iteration of dynamic confidence, the formula is as follows:

[0087]

[0088] Among them, is the time smoothing factor, which controls the weight of the historical dynamic confidence ; function ensures that the dynamic evidence is continuously differentiable; is used to adjust sensitivity of the function; is the minimum threshold value of dynamic determination.

[0089] Step 8: Gaussian body soft shielding and weight adjustment based on dynamic confidence. According to the dynamic confidence of each Gaussian, divide the static Gaussian set and the dynamic Gaussian set, and perform soft shielding on the Gaussian in the dynamic region; then, according to the dynamic confidence of each Gaussian, re-adjust the weight of the Gaussian, specifically:

[0090] Step 8.1: Construct a soft shielding function, the main goal of which is to shield the dynamic Gaussians of ; and make the static Gaussians of fully participate in the subsequent process. The final soft shielding function is determined as follows:

[0091]

[0092] wherein, controls the strength of soft shielding.

[0093] Step 8.2: For each Gaussian , apply the soft shielding function to re-adjust the weight of the Gaussian, and the formula is as follows:

[0094]

[0095] wherein, represents the latest weight of the i-th Gaussian, represents the original weight of the i-th Gaussian.

[0096] Step 8.3: After soft shielding, the Gaussian set can be directly divided according to the value of the dynamic confidence :

[0097] Static Gaussian set , wherein represents the static Gaussian dynamic confidence threshold, and the Gaussian elements in the static Gaussian set normally participate in the subsequent optimization.

[0098] Dynamic Gaussian set , wherein represents the static Gaussian dynamic confidence threshold, and the Gaussian elements in the dynamic Gaussian set are excluded in the subsequent process.

[0099] Step 9: Select a local Gaussian set for camera pose optimization. Based on the camera observation information of the current frame, select the relevant local Gaussian set to use the optimization algorithm for camera pose optimization to obtain the optimized camera pose, specifically:

[0100] Step 9.1: Select local Gaussians for camera pose optimization, based on the static Gaussians filtered in step 8, further filter and select the static Gaussians within the current camera view as local Gaussians Local filtering can significantly reduce the computation scale and ensure that the local Gaussians are strongly related to the subsequent optimization.

[0101] Step 9.2: Camera pose optimization, the optimization goal is to adjust the camera pose so that each Gaussian projected onto the camera pixel plane is consistent with the current camera observation; use Gauss-Newton optimization algorithm to calculate the gradient loss and perform iterative update, finally get the optimized camera pose The loss function calculation process is as follows:

[0102]

[0103] wherein, represents the photometric loss, represents the color observation value of pixel , represents the color prediction value of pixel , the photometric loss is used to measure the color matching degree.

[0104]

[0105] wherein, represents the depth loss, represents the depth observation value of pixel , represents the depth prediction value of pixel , the depth loss is used to measure the geometric consistency.

[0106]

[0107] wherein, represents the regularization loss, represents the optimized camera pose, represents the initial camera pose, the regularization loss is used to limit the large deviation of camera pose in the optimization process.

[0108] wherein, is the total loss, , , respectively represents the weight of each loss term.

[0109] Step 10: Add new scene information in Gaussian form to the global map, and perform Gaussian merging and migration to obtain a dense renderable map, specifically:

[0110] Step 10.1: New Gaussian insertion: Add the new Gaussian obtained by Gaussian fitting in step 3 to the static map to gradually enrich the map representation.

[0111] Step 10.2: Gaussian merging (sparsification): Perform merging on Gaussians that are spatially adjacent and have similar color information or covariance matrices, control the maximum number of Gaussians; avoid merging dynamic Gaussians with static backgrounds, and control the scale of the representation and rendering cost.

[0112] Step 10.3: Pruning (removing dynamic or inefficient Gaussians): For dynamic Gaussians that satisfy , remove them from the static map, where represents the dynamic confidence threshold of the static Gaussian; for Gaussians that satisfy , remove them directly, where represents the minimum weight threshold. Removing Gaussians that are continuously dynamic can avoid long-term artifacts; short-term dynamic Gaussians are only soft-screened to retain recovery possibilities.

[0113] Step 11: Output the visual camera pose and dense renderable map based on the static Gaussian set and camera pose in the above steps, specifically:

[0114] Step 11.1: Insert the camera pose of each frame into the camera pose sequence, connect the camera pose sequence according to time, and draw a visual camera movement path.

[0115] Step 11.2: Project the static Gaussian set onto the screen to display the geometry and color information of the entire scene.

[0116] The present application introduces a dynamic object determination mechanism, a soft screening and dynamic penalty strategy, and an accurate map maintenance method by introducing a Gaussian-level dynamic confidence, and realizes a three-dimensional reconstruction method in a dynamic fuzzy scene, with the following key points:

[0117] 1. Dynamic object determination mechanism: A mechanism of "pixel residual - aggregated Gaussian - multi-modal evidence fusion - temporal smoothing" is used for dynamic determination. Pixel-level rendering residuals, flow inconsistencies, semantic priors, etc. are aggregated into Gaussian-level evidence, and dynamic confidence is reflected in the Gaussian.

[0118] 2. Soft screening and dynamic penalty strategy: Use soft screening of pixels affected by dynamics in the loss, and impose a penalty on long-term dynamic Gaussians, which can avoid false deletion of static structures and gradually eliminate dynamic components.

[0119] 3. Local joint optimization strategy: only optimize the pose in the front end, which helps to obtain the initial value of convergence and reduce the computational burden of subsequent joint optimization. Correct the pose in the local map in the back end to avoid loop drift problem, and balance the accuracy and real-time performance.

[0120] 4. Clear map maintenance strategy: insert, merge, and prune at the Gaussian level, and avoid false merging of dynamic and static parts through dynamic confidence .

[0121] The above only describes the preferred embodiments of the present application and is not intended to limit the idea of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A dynamic blurred scene three-dimensional reconstruction method based on 3DGS technology, characterized in that, The method comprises the following steps: Step 1: initialize a three-dimensional reconstruction system, and define a three-dimensional Gaussian set to represent a scene; Step 2: input an RGB camera image frame by frame, and perform de-distortion, color correction and depth filtering processing; Step 3: for the RGB camera images of consecutive frames, perform 3DGS-based differentiable rendering prediction to predict the color and depth on the current camera image plane from the current camera pose, and obtain the color prediction value and the depth prediction value of each pixel; Step 4: perform initial estimation on the camera pose; Step 5: calculate the pixel-level residual error of the RGB camera image, and inversely distribute the pixel-level residual error to the Gaussian-level residual error to form photometric residual error and depth residual error for describing the dynamic and static characteristics of the Gaussian; Step 6: calculate other dynamic auxiliary evidence of the RGB camera image, input the optical flow data and semantic segmentation information of the RGB camera image, and obtain optical flow inconsistency and semantic prior information; Step 7: fuse the multi-modal dynamic evidence to obtain instantaneous dynamic evidence; and then perform time smoothing on the historical dynamic confidence to obtain stable dynamic confidence; Step 8: according to the dynamic confidence of each Gaussian, divide the static Gaussian set and the dynamic Gaussian set, and perform soft shielding on the Gaussian in the dynamic region; and then re-adjust the weight of each Gaussian according to the dynamic confidence of the Gaussian; Step 9: based on the camera observation information of the current frame, select the relevant local Gaussian set to perform camera pose optimization by using an optimization algorithm, and obtain the optimized camera pose; Step 10: add new scene information in the form of Gaussian to the global map, and perform Gaussian merging and migration to obtain a dense renderable map; Step 11: output the visualized camera pose and the dense renderable map according to the static Gaussian set and the camera pose in the above steps.

2. The dynamic blurred scene 3D reconstruction method based on 3D GS technology according to claim 1, characterized in that, The step 1 specifically comprises: Step 1.1: Define a set of three-dimensional Gaussians Each Gaussian with the following parameters: represents the center point of the scene where each Gaussian is located; wherein , , denotes the variance in each axis direction, , , , , , denotes the covariance between different axes, Overall, it represents a 3D space covariance matrix, which is used to describe the extension direction and scale of a three-dimensional Gaussian ellipsoid, and controls its influence range; representing color information of the Gaussian; Gauss weight representing the contribution of the Gauss in the rendering process; Dynamic confidence of Gaussian, initial value is 0, used for subsequent dynamic object segmentation; Step 1.2: Define camera intrinsic matrix , represents the perspective projection model of the camera, used to describe the imaging characteristics of the camera, ensuring the accuracy of the projection; wherein and denotes the camera focal length, and denotes the camera principal point; Step 1.3: Define camera pose ; where denotes the rotation of the camera, denotes the translation of the camera. 3.The dynamic blurred scene 3D reconstruction method based on 3D GS technology according to claim 2, characterized in that, The step 3 specifically comprises: Step 3.1: Project each pixel to camera coordinate system to obtain 3D point cloud according to camera observation information of current frame, including RGB image and depth image of camera Step 3.2: Fit Gaussian representation according to 3D point cloud​​​ Step 3.2: Project each Gaussian into the current camera image plane, transform to camera coordinate system; compute the Gaussian center according to the current camera pose and camera intrinsics ; in camera coordinate system and the distance of the Gaussian along the camera Z-axis : wherein is the Gaussian center z-coordinate in the camera coordinate system; Step 3.3: Project the point in the camera coordinate system to the current camera image plane to get the pixel coordinates corresponding to the Gaussian center According to the following formula And : Step 3.4: Project the 3D covariance onto the current camera image plane, obtaining the shape of the Gaussian in that image plane: wherein, is the covariance matrix projected to the current camera image plane, is the projection Jacobian from the current camera image plane to 3D space; Step 3.5: rendering weight calculation, in order to let each Gaussian affect multiple pixel neighborhoods on the current camera image plane, the original weight of the Gaussian to any pixel is calculated: where the exponential term represents the influence of the Gaussian on the neighborhood it lies in; the disparity term dampens the influence of the Gaussian on distant pixels; a very small regularization term prevents numerical instability, represents the center of the pixel of the Gaussian projection; Step 3.6: weight normalization processing, in order to avoid mutual accumulation and superposition between Gaussian weights, normalization processing is performed on each pixel based on the shared weight of all affected pixels, and the formula is as follows: Step 3.7: Normalization weight based on Gaussian Finally, the color prediction value and the depth prediction value of each pixel are synthesized. all Gaussian color weights are weighted and averaged to obtain the pixel color prediction value: all Gaussian depth weights are weighted and averaged to obtain the pixel depth prediction value: wherein, is a pixel color prediction value, is a pixel depth prediction value.

4. The dynamic blurred scene 3D reconstruction method based on 3D GS technology according to claim 1, characterized in that, The step 4 specifically comprises: The camera observation information of the current frame includes an RGB image of the camera and a depth image , an initial Gaussian set, the estimated color and depth obtained in step 3, based on the photometric consistency principle, a fast optimization is carried out in the front end to obtain the optimized initial estimated camera pose .

5. The dynamic blurred scene 3D reconstruction method based on 3D GS technology according to claim 3, characterized in that, The step 5 specifically comprises: Step 5.1: calculate the pixel-level residual error of the RGB camera image according to the following formula: wherein, represents a photometric residual of a pixel represents a pixel color value observed by a sensor, represents a pixel color prediction value based on the current pose and the set of Gaussians;​ wherein, represents a depth residual of a pixel represents a pixel color value observed by a sensor represents a pixel color prediction value based on the current pose and the set of Gaussians;​ Step 5.2: Pixel residuals are passed through a Gaussian normalization weight Reverse assignment to Gaussian level residuals: wherein, represents the photometric residual of the i-th Gaussian; wherein, represents the depth residual of the i-th Gaussian.

6. The dynamic blurred scene 3D reconstruction method based on 3D GS technology according to claim 5, characterized in that, The step 7 specifically comprises: Step 7.1: Fusing computed instantaneous dynamic evidence The formula is as follows: wherein, represents instantaneous dynamic evidence, represents photometric residual, represents depth residual, represents optical flow inconsistency, represents semantic prior information, , , , respectively represent the weight of each evidence, for balancing the importance of each dynamic evidence. Step 7.2: Update the instantaneous dynamic evidence based on the current frame and the historical dynamic confidence , update the dynamic confidence of each Gaussian , implement a temporal smoothing iteration of the dynamic confidence, formula as follows: wherein, is a time smoothing factor, controlling the weight of the historical dynamic confidence ; function ensures that the dynamic evidence is continuously differentiable; used to adjust the sensitivity of the function; is a minimum threshold for the dynamicity determination.

7. The dynamic blurred scene 3D reconstruction method based on 3D GS technology according to claim 2, characterized in that, The step 8 specifically comprises: Step 8.1: Constructing the soft mask function, the main goal is to mask the dynamic gauss of ; make the static gauss of participate in the subsequent process completely, and the final soft mask function is determined as: wherein, controlling the strength of the soft mask; Step 8.2: For each Gaussian , apply the soft mask function , re-adjust the weight of the Gaussian, formula as follows: wherein, represents the latest weight of the i-th Gaussian, represents the original weight of the i-th Gaussian; Step 8.3: After soft masking, the Gaussian set can be directly partitioned according to the values of the dynamic confidence scores: static gausian set wherein represents static gausian dynamic confidence threshold, gausian elements in static gausian set normally participate in subsequent optimization; dynamic gausss wherein represents a static gausss dynamic confidence threshold, gausss in the dynamic gausss are eliminated in a subsequent process. 8.The dynamic blurred scene 3D reconstruction method based on 3D GS technology of claim 1, wherein, The step 9 specifically comprises: Step 9.1: Select local Gauss set for camera pose optimization, based on the static Gauss set filtered in step 8, further filtering is performed to select the static Gauss within the camera view of the current frame as the local Gauss set ; Step 9.2: camera pose optimization, the optimization target is to adjust the camera pose so that each Gaussian projected to the camera pixel plane is consistent with the observation value of the current camera; The Gauss-Newton optimization algorithm is used to obtain the gradient loss and perform iterative updating, and finally the optimized camera pose is obtained The loss function calculation process is as follows: wherein, represents a luminosity loss, represents a color observation value of a pixel of a pixel, represents a color prediction value of a pixel of a pixel, a luminosity loss for measuring the degree of color matching; wherein, represents a depth loss, represents a depth observation value of a pixel at a pixel location, represents a depth prediction value of a pixel at a pixel location, the depth loss is used to measure geometric consistency; wherein, denotes the regularization loss, denotes the optimized camera poses, denotes the initial camera poses, the regularization loss for limiting large shifts of the camera poses during the optimization process; ​ wherein, is the total loss, , , respectively represent the weight of each loss term. 9.The dynamic blurred scene 3D reconstruction method based on 3D GS technology of claim 1, wherein, The step 10 specifically comprises: Step 10.1: add the new Gaussian obtained by Gaussian fitting in step 3 to the static map to gradually enrich the map representation; Step 10.2: perform merging on the Gaussians that are spatially adjacent and similar in color information or covariance matrix, control the maximum number of Gaussians, and avoid merging dynamic Gaussians with static backgrounds, and control the representation scale and rendering cost in merging; Step 10.3: Remove from the static map the dynamic Gaussians that satisfy where denotes the dynamic confidence threshold for static Gaussians; remove directly the Gaussians that satisfy where denotes the weight minimum threshold.

10. The dynamic blurred scene 3D reconstruction method based on 3D GS technology according to claim 1, characterized in that, The step 11 specifically comprises: Step 11.1: Inserting the camera poses of each frame into the camera pose sequence, connecting the camera pose sequence by time, and drawing a visualization camera movement path. Inserting the camera pose sequence into the camera pose sequence, connecting the camera pose sequence by time, and drawing a visualization camera movement path. Step 11.2: Project the static set of Gaussians onto the screen, revealing the geometry and color information of the entire scene.

Images