Depth constraint-based three-dimensional Gaussian camera motion blur removal method

Abstract

The invention relates to the field of three-dimensional reconstruction, and discloses a depth constraint-based three-dimensional Gaussian camera motion blur removal method, which comprises the following steps of: S1, initializing three-dimensional Gaussian parameters by using a group of multi-view pictures with camera motion blur; s2, performing edge processing on the depth map obtained by three-dimensional Gaussian rasterization rendering; s3, evaluating the scene fuzzy degree through an image difference detection algorithm, and adaptively adjusting a camera track simulation method; s4, the number of key frames is adjusted according to the scene fuzzy degree, and weighted averaging is carried out according to the time weight; and S5, scene representation of the three-dimensional Gaussian model is optimized by using depth constraint, and the method provided by the invention can accurately recover complex details and realize real-time rendering under the condition that serious camera motion blur exists in an input image.

Description

Technical Field

[0001] The invention relates to the field of three-dimensional reconstruction, and in particular to a method for removing motion blur of a three-dimensional Gaussian camera based on depth constraints. Background Art

[0002] In the field of new perspective synthesis and three-dimensional reconstruction, the emergence of three-dimensional Gaussian solves the problems of long training time and slow rendering speed of neural radiation field. However, the reconstruction quality of three-dimensional Gaussian depends on the quality of the input image and the initialization of the camera pose. When the input image is blurred, the camera pose is inevitably not accurate enough, resulting in a large number of artifacts and blurs in the reconstruction result, which seriously affects the quality of the synthesized new perspective picture. In the actual data acquisition of three-dimensional reconstruction tasks, it is often affected by camera jitter and shutter time, resulting in camera motion blur in the original image obtained. Commonly used motion structure recovery methods such as COLMAP will produce many errors when processing blurred images, and the camera pose cannot be accurately calculated, resulting in low-quality reconstruction. Therefore, the present invention proposes a three-dimensional Gaussian camera motion blur removal method based on depth constraints. In the case of severe camera motion blur in the input image, the scene representation is jointly learned based on the depth map constraint and the three-dimensional Gaussian model. Through the multi-terminal loss function, the camera trajectory is optimized, and the more accurate camera pose and Gaussian model parameters are trained to restore the complex details of the original blurred image and improve the quality of three-dimensional reconstruction. Summary of the invention

[0003] The purpose of the present invention is to provide a three-dimensional Gaussian camera motion blur removal method based on depth constraint to solve the problems raised in the above background technology.

[0004] To achieve the above object, the present invention provides the following technical solution: a three-dimensional Gaussian camera motion blur removal method based on depth constraint, comprising the following steps:

[0005] Step S1, initializing three-dimensional Gaussian parameters using a set of multi-view pictures with camera motion blur;

[0006] Step S2, performing edge processing on the depth map obtained by three-dimensional Gaussian rasterization rendering;

[0007] Step S3, evaluating the blur level of the scene through an image difference detection algorithm, and adaptively adjusting the camera trajectory simulation method;

[0008] Step S4, adjusting the number of key frames according to the blur level of the scene, and performing weighted averaging according to the time weight;

[0009] Step S5, optimizing the scene representation of the three-dimensional Gaussian model using the depth constraint.

[0010] As a preferred embodiment, the step S1 uses a set of multi-view pictures with camera motion blur to initialize the three-dimensional Gaussian parameters, specifically comprising the following steps:

[0011] S11: Use GOLMAP to estimate the internal and external parameters and point positions of the input image, complete motion structure recovery, and obtain sparse point cloud data;

[0012] S12: Initialize three-dimensional Gaussian parameters using the sparse point cloud data processed in S11, including point position coordinates, spherical harmonic coefficients, covariance matrix, etc.;

[0013] S13: Initialize the camera parameters of the Gaussian model using the internal and external parameters of the camera processed in S11, including focal length, principal point coordinates, distortion degree, posture parameters, etc.;

[0014] S14: Count the number of cameras, the number of pictures, and the number of points in the point cloud, and initialize the model scene parameters.

[0015] As a preferred embodiment: the specific process of step S2 is:

[0016] S21: Modify the rasterization part of the 3D Gaussian, add a single-channel depth map to the rendered output, and synthesize the virtual blurred depth map D virtual can be decomposed into a combination of depth maps of k key frames within the exposure time;

[0017] S22: Perform edge detection on each depth map generated by rasterization rendering, and estimate the points located at the edge;

[0018] S23: For each rendered depth map D i , 1≤i≤k, let a point located at the edge be in different depth maps D i The depth in is d i , use the minimum constraint to process the above depth values ​​to synthesize the virtual blurred depth map D virtual , where the depth of each point is expressed as:

[0019] d=min{d1, d2, d3,…,d k )

[0020] The depth values ​​of the remaining non-edge parts are represented by the median, and a synthetic blurred depth map is obtained through the above method.

[0021] As a preferred embodiment: the specific process of step S3 is as follows:

[0022] S31: Analyze the blur degree of the scene and calculate its blur degree measurement value Deg using an image difference detection algorithm;

[0023] S32: setting a threshold for each blur level, and in the early stage of training, when the blur level of the scene reaches a certain threshold, selecting a camera trajectory simulation strategy corresponding to the threshold.

[0024] Preferably, the step S31 analyzes the blur level of the scene, specifically using an image difference detection algorithm to calculate the blur level measurement value Deg, including the following steps:

[0025] S311: During the training process, an iteration round threshold is set. When the number of training rounds reaches the corresponding threshold, the camera trajectory simulation method is adjusted. Specifically, a group of continuous images I1, I2, ..., I k , where 1≤i≤k, the image is normalized;

[0026] S312: For each pair of adjacent continuous images, calculate their difference according to the following formula:

[0027] V i =|I i+1 -I i |

[0028] After obtaining the difference value of each pair of images, the blur degree measure is calculated using the following formula:

[0029]

[0030] Where M is the total number of pixels in the image (i.e. H×W), H and w represent the height and width of the image respectively, and V i (j, k) represents the value of pixel (j, k) in the difference image.

[0031] As a preferred embodiment, the step S32 includes the following steps:

[0032] S321: Divide the blur range into multiple thresholds τ1, τ2, ..., τ according to the blur degree n , for each threshold τ, a corresponding camera trajectory simulation method is assigned to it; where τ1 corresponds to linear interpolation simulation, τ2 corresponds to third-order Bezier curve simulation, and τ n Corresponding to n+1 order Bezier curve simulation;

[0033] S322: Under the initial conditions, the camera trajectory simulation mode is set to linear interpolation simulation by default;

[0034] S323: Determine the camera trajectory simulation method to be used by monitoring the blur degree Deg in real time.

[0035] As a preferred embodiment, the step S4 comprises the following steps:

[0036] S41: setting the number of key frames N according to the scene blur degree measurement value Deg calculated in S312;

[0037] S42: for the linear interpolation simulation of the camera trajectory in the default state, the number of key frames is initialized to 5, and for the n-th order Bezier curve, n+1 key frames are selected as camera pose control points to simulate the camera trajectory;

[0038] S43: For each key frame k i Assign time weight ω i :

[0039]

[0040] Where T start and T end Respectively represent the start and end of the exposure time,

[0041] Normalize the calculated time weight;

[0042] S44: Perform weighted averaging on the selected key frame images to synthesize a virtual RGB image B(u) with camera motion blur:

[0043]

[0044] Among them, C i Represents a clear key frame image obtained by rasterization rendering of a three-dimensional Gaussian model.

[0045] u represents the pixel position in the image with height H and width W.

[0046] Preferably, in step S5, optimizing the scene representation of the three-dimensional Gaussian model using the depth constraint comprises the following steps:

[0047] S51: Use the depth prediction model to estimate the depth of each input image Compare it with the blurred depth map simulated in step S23 For comparison, construct the loss function:

[0048] L depth =(N(D virtual )-N(D input )) 2

[0049]

[0050] Among them, D mid represents the median of D, avg represents the average value, and the average value of the absolute difference between each point in the depth map and the median is calculated, ∈ is a minimum value;

[0051] S52: The image B with camera motion blur synthesized in step S44 virtual Compare with the input blurred image and construct the loss function:

[0052]

[0053] Where λ is a weight factor, L D-SSIM is a loss based on the structural similarity index;

[0054] S53: Modify the total variation loss by adding edge-aware constraints and constructing the edge-preserving total variation loss function:

[0055]

[0056] Where D is the synthesized virtual depth image D virtual ,λ d is the weight coefficient, ε is the regularization constant;

[0057] S54: The final loss function is defined as follows:

[0058] L=L depth +L rgb +L eptv

[0059] Through the above method, the three-dimensional Gaussian parameters and camera trajectory control parameters are jointly optimized during the training process. After the training is completed, a relatively accurate camera pose and optimized three-dimensional scene representation are obtained. Rasterization rendering is used in real time within a certain range to obtain clear and sharp images.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] When the input image has severe motion blur, it can jointly learn the 3D scene representation through depth map constraints, optimize the camera pose parameters, greatly reduce the artifacts and blur in the original result, and significantly improve the reconstruction quality;

[0062] During the training process, the camera trajectory is adaptively adjusted and the camera pose is optimized. At the same time, the synthesized virtual depth map is subjected to minimum constraint edge processing, the number of key frames is dynamically adjusted, and the synthesized blurred image is weighted averaged to make it more consistent with physical laws and the rendered new perspective image is clearer. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a flow chart of the three-dimensional Gaussian camera motion blur removal technology based on depth constraint of the present invention;

[0064] Figure 2 It is a diagram of the training reasoning process of the three-dimensional Gaussian camera motion blur removal technology based on depth constraints of the present invention;

[0065] Figure 3 It is a schematic diagram comparing the results of the three-dimensional Gaussian camera motion blur removal technology based on depth constraints of the present invention. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] Example

[0068] See also Figure 1 The three-dimensional Gaussian camera motion blur removal method based on depth constraint shown in the figure includes the following steps:

[0069] Step S1, using a set of multi-view pictures with camera motion blur to initialize three-dimensional Gaussian parameters; specifically: input a set of three-channel RGB images with camera motion blur, the camera motion blur is an image distortion phenomenon caused by the relative motion between the camera and the subject or the camera's own shaking, when the shutter speed is not enough to freeze the motion, the light is continuously exposed on the photosensitive element, thereby forming a blurred track.

[0070] Furthermore, GOLMAP is used to estimate the internal and external parameters and point positions of the camera for the input image, thereby completing the motion structure recovery. GLOMAP is a general global motion structure reconstruction pipeline used for image-based reconstruction. GLOMAP requires the COLMAP database as input and outputs COLMAP sparse reconstruction. Compared with COLMAP, GLOMAP reconstruction is faster and of higher quality. The sparse point cloud data and camera pose data obtained after processing in this step are used to initialize the three-dimensional Gaussian model parameters and camera parameters respectively. In the three-dimensional Gaussian model, for each Gaussian point G, there is a 3D center point position μ and a 3D covariance matrix ∑. Each Gaussian point can be represented as:

[0071]

[0072] Among them, the 3D covariance matrix ∑ is decomposed into ∑ = RSS T R T; R is the rotation matrix, represented by the quaternion q∈SO(3); S is the scaling matrix; For each Gaussian, there is an opacity value σ, which is used for subsequent rasterization rendering, and the spherical harmonic coefficient SH is used to represent this parameter and the RGB color value.

[0073] During the training process, the camera internal parameters remain unchanged. The camera pose T is a transformation matrix, which is expressed by SE(3):

[0074]

[0075] Since the initialization image has camera motion blur, this parameter is not accurate and will be continuously iterated and optimized during subsequent training.

[0076] Step S2: perform edge processing on the depth map obtained by three-dimensional Gaussian rasterization rendering to make it conform to physical laws; the specific implementation is: refer to Figure 2 , based on the initial 3D Gaussian, modify its rasterization part according to the transparency α and the depth value d of each point i Rendering to get the depth map D:

[0077]

[0078] The depth map is a single-channel image, and the synthesized virtual blurred depth map D virtual It can be decomposed into a combination of depth maps of k key frames within the exposure time. After obtaining the depth map generated by rasterization rendering, Canny edge detection is used to roughly estimate the points located at the edge of the object, and these points are marked for subsequent processing.

[0079] In the original RGB image, the depth value of the foreground is low, while the depth value of the background is high. For each rendered depth map D i , 1≤i≤k, assuming that a point located at the edge is in different depth maps D i The depth in is d i , in order to synthesize the virtual blurred depth map D virtual , use the minimum constraint to process these depth values, and the depth of the specific corresponding points is expressed as:

[0080] d=min{d1, d2, d3,…,d k )

[0081] For the depth values ​​of the remaining non-edge parts, the median is directly used to represent them. In this way, a synthetic blurred depth map can be obtained.

[0082] Step S3: Evaluate the blur degree of the scene by using the image difference detection algorithm, and adaptively adjust the camera trajectory simulation method; the specific implementation is as follows: first, use the image difference detection algorithm to evaluate the blur degree of the scene. During the training process, set the iteration round threshold to 1000. When the number of training rounds reaches the corresponding threshold, adjust the camera trajectory simulation method. Specifically, randomly select a group of continuous images I1, I2, ..., I k , where 1≤i≤k, the image is normalized, and for each pair of adjacent continuous images, the difference is calculated according to the following formula:

[0083] V i =|I i+1 -I i |

[0084] After obtaining the difference value of each pair of images, use the following formula to calculate its blur degree measure value Deg:

[0085]

[0086] Where M is the total number of pixels in the image (i.e. H×W), H and W represent the height and width of the image respectively, and V i (j, k) represents the value of pixel (j, k) in the difference image. The scene blur metric value calculated in the above manner will be used to adjust the subsequent camera trajectory simulation strategy.

[0087] Furthermore, a threshold is set for each blur level. In the early stage of training, when the scene blur level reaches a certain threshold, the camera trajectory simulation strategy corresponding to the threshold is selected; specifically, according to the blur level, the blur range is divided into multiple thresholds τ1, τ2, ..., τ n , for each threshold τ, a corresponding camera trajectory simulation method is assigned to it; where τ1 corresponds to linear interpolation simulation, τ2 corresponds to third-order Bezier curve simulation, and τ n Corresponding to n+1 order Bezier curve simulation.

[0088] Among them, during initialization, the camera trajectory simulation method is set to linear interpolation simulation. This strategy is suitable for scenes with low blur. As the blur level of the scene increases, the camera trajectory simulation method to be used is determined by real-time monitoring of the blur level B.

[0089] Step S4: According to the degree of scene blur, adjust the number of key frames, and perform weighted averaging according to the time weight; specifically, the camera trajectory simulation method used is determined according to the degree of scene blur obtained in the previous step, and the number of key frames used is determined accordingly. Specifically, for the camera trajectory linear interpolation simulation in the default state, the number of key frames is initialized to 5. As the degree of scene blur increases, the camera trajectory simulation method changes. For the n-order Bezier curve, n+1 key frames are selected as camera pose control points to simulate the camera trajectory.

[0090] Furthermore, for each key frame k i Assign time weight ω i :

[0091]

[0092] Where T start and T end Respectively represent the start and end of the exposure time,

[0093] The calculated time weights are normalized; finally, the rendered key frame images are weighted averaged to simulate the process of camera motion blur and synthesize a virtual RGB image with camera motion blur:

[0094]

[0095] Among them, C i Represents a clear key frame image obtained by rasterization rendering of a three-dimensional Gaussian model.

[0096] Represents the pixel position in the image with height H and width W.

[0097] Step S5: Use the scene representation of the depth-constrained 3D Gaussian model; specifically, use the depth prediction model Depth-Anything model to predict and estimate the depth of each input image. Depth-Anything is a monocular depth estimation model that can handle the details of objects well. Experiments have shown that it performs better than MiDaS in the above tasks.

[0098] like Figure 2 As shown, the depth map predicted by the model is compared with the blurred depth map simulated in the above steps. For comparison, construct the loss function:

[0099] L depth =(N(D virtual )-N(D input )) 2

[0100]

[0101] Among them, D mid represents the median of D, avg represents the average value, and the average value of the absolute difference between each point in the depth map and the median is calculated. ∈ is a minimum value to prevent division by zero errors.

[0102] The above synthesized image B with camera motion blur virtual Compare with the input blurred image and construct the loss function:

[0103]

[0104] Where λ is a weight factor used to balance the impact of the two parts of the loss, L D-SSIM It is a loss based on the Structural Similarity Index (SSIM) that can better capture the visual similarity of images.

[0105] In order to prevent the depth from changing suddenly in the non-edge part, the total variation loss is modified by adding edge-aware constraints to construct the edge-preserving total variation (Edge-PreservingTVLoss) loss function:

[0106]

[0107] Where D is the synthesized virtual depth image D virtual ,λ d is the weight coefficient, ε is the regularization constant, which introduces a minimum threshold for small gradients to prevent instability caused by too small gradient values ​​during the calculation process. This loss can effectively suppress the mutation of the depth image in non-edge areas, while retaining more details at the boundary of the object, thereby improving the robustness of the depth constraint.

[0108] The final loss function is defined as follows:

[0109] L=L depth +L rgb +L eptv

[0110] Through the above method, the three-dimensional Gaussian parameters and camera trajectory control parameters are jointly optimized during the training process. After the training is completed, a relatively accurate camera pose and optimized three-dimensional scene representation are obtained. Rasterization rendering is used in real time within a certain range to obtain clear and sharp images.

[0111] exist Figure 3In the figure, the first row of pictures is a clear image of the real scene and the corresponding depth map of the image, the second row is the input image with camera motion blur and the corresponding depth map, and the third row is the result after deblurring using this method. It can be seen that the deblurring effect of this method is significant, and the result is very close to the real clear image.

[0112] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0113] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A three-dimensional Gaussian camera motion blur removal method based on depth constraints, characterized in that: The steps include: Step S1, initializing three-dimensional Gaussian parameters using a set of multi-view pictures with camera motion blur; Step S2, performing edge processing on the depth map obtained by three-dimensional Gaussian rasterization rendering; Step S3, evaluating the blur level of the scene through an image difference detection algorithm, and adaptively adjusting the camera trajectory simulation method; Step S4, adjusting the number of key frames according to the blur level of the scene, and performing weighted averaging according to the time weight; Step S5, optimizing the scene representation of the three-dimensional Gaussian model using the depth constraint.

2. The method for removing 3D Gaussian camera motion blur based on depth constraint according to claim 1, characterized in that: The step S1 uses a set of multi-view pictures with camera motion blur to initialize the three-dimensional Gaussian parameters, which specifically includes the following steps: S11: Use GOLMAP to estimate the internal and external parameters and point positions of the input image, complete motion structure recovery, and obtain sparse point cloud data; S12: Initialize three-dimensional Gaussian parameters using the sparse point cloud data processed in S11, including point position coordinates, spherical harmonic coefficients, covariance matrix, etc.; S13: Initialize the camera parameters of the Gaussian model using the internal and external parameters of the camera processed in S11, including focal length, principal point coordinates, distortion degree, posture parameters, etc.; S14: Count the number of cameras, the number of pictures, and the number of points in the point cloud, and initialize the model scene parameters.

3. The method for removing 3D Gaussian camera motion blur based on depth constraint according to claim 2, characterized in that: The specific process of step S2 is: S21: Modify the rasterization part of the 3D Gaussian, add a single-channel depth map to the rendered output, and synthesize the virtual blurred depth map D virtual It can be decomposed into a combination of depth maps of k key frames within the exposure time; S22: Perform edge detection on each depth map generated by rasterization rendering, and estimate the points located at the edge; S23: For each rendered depth map D i , 1≤i≤k, let a point located at the edge be in different depth maps D i The depth in is d i , use the minimum constraint to process the above depth values ​​to synthesize the virtual blurred depth map D virtual , where the depth of each point is expressed as: <h2 style=";text-align:left;direction:ltr">d = min {d1, d2, d3,…, d}<h2 style=";text-align:left;direction:ltr"> k <h2 style=";text-align:left;direction:ltr"> ) The depth values ​​of the remaining non-edge parts are represented by the median, and a synthetic blurred depth map is obtained through the above method.

4. The method for removing 3D Gaussian camera motion blur based on depth constraint according to claim 3, characterized in that: The specific process of step S3 is: S31: Analyze the blur degree of the scene and calculate its blur degree measurement value Deg using an image difference detection algorithm; S32: setting a threshold for each blur level, and in the early stage of training, when the blur level of the scene reaches a certain threshold, selecting a camera trajectory simulation strategy corresponding to the threshold.

5. The method for removing 3D Gaussian camera motion blur based on depth constraint according to claim 4, characterized in that: The step S31 analyzes the blur level of the scene, specifically using an image difference detection algorithm to calculate the blur level measurement value Deg, including the following steps: S311: During the training process, an iteration round threshold is set. When the number of training rounds reaches the corresponding threshold, the camera trajectory simulation method is adjusted. Specifically, a group of continuous images I1, I2, ..., I k , where 1≤i≤k, the image is normalized; S312: For each pair of adjacent continuous images, calculate their difference according to the following formula: V i =|I i+1 -I i | After obtaining the difference value of each pair of images, the blur degree measure is calculated using the following formula: Where M is the total number of pixels in the image (i.e. H×W), H and W represent the height and width of the image respectively, and V i (j, k) represents the value of pixel (j, k) in the difference image.

6. The method for removing 3D Gaussian camera motion blur based on depth constraint according to claim 5, characterized in that: The step S32 comprises the following steps: S321: Divide the blur range into multiple thresholds τ1, τ2, ..., τ according to the blur degree n , for each threshold τ, a corresponding camera trajectory simulation method is assigned to it; where τ1 corresponds to linear interpolation simulation, τ2 corresponds to third-order Bezier curve simulation, and τ n Corresponding to n+1 order Bezier curve simulation; S322: Under the initial conditions, the camera trajectory simulation mode is set to linear interpolation simulation by default; S323: Determine the camera trajectory simulation method to be used by monitoring the blur degree Deg in real time.

7. The method for removing 3D Gaussian camera motion blur based on depth constraint according to claim 6, characterized in that: The step S4 comprises the following steps: S41: setting the number of key frames N according to the scene blur degree measurement value Deg calculated in S312; S42: for the linear interpolation simulation of the camera trajectory in the default state, the number of key frames is initialized to 5, and for the n-th order Bezier curve, n+1 key frames are selected as camera pose control points to simulate the camera trajectory; S43: For each key frame k i Assign time weight ω i : Where T start and T end Respectively represent the start and end of the exposure time, Normalize the calculated time weight; S44: Perform weighted averaging on the selected key frame images to synthesize a virtual RGB image B(u) with camera motion blur: Among them, C i Represents a clear key frame image obtained by rasterization rendering of a three-dimensional Gaussian model. u represents the pixel position in the image with height H and width W.

8. The method for removing 3D Gaussian camera motion blur based on depth constraint according to claim 7, characterized in that: In step S5, optimizing the scene representation of the three-dimensional Gaussian model using the depth constraint includes the following steps: S51: Use the depth prediction model to estimate the depth of each input image Compare it with the blurred depth map simulated in step S23 For comparison, construct the loss function: L aepth =(N(D virtual )-N(D input )) 2 Among them, D mid represents the median of D, avg represents the average value, and the average value of the absolute difference between each point in the depth map and the median is calculated, ∈ is a minimum value; S52: The image B with camera motion blur synthesized in step S44 virtual Compare with the input blurred image and construct the loss function: Where λ is a weight factor, L D-SSIM is a loss based on the structural similarity index; S53: Modify the total variation loss by adding edge-aware constraints and constructing the edge-preserving total variation loss function: Where D is the synthesized virtual depth image D virtual ,λ d is the weight coefficient, ε is the regularization constant; S54: The final loss function is defined as follows: L=L depth +L rgb +L eptv Through the above method, the three-dimensional Gaussian parameters and camera trajectory control parameters are jointly optimized during the training process. After the training is completed, a relatively accurate camera pose and optimized three-dimensional scene representation are obtained. Rasterization rendering is used in real time within a certain range to obtain clear and sharp images.

Images