A dynamic 3D reconstruction method based on incremental updating

Abstract

The present invention discloses a dynamic three-dimensional reconstruction method based on incremental updating, which belongs to the field of image data processing, comprising the steps of: obtaining multi-view images of a forestry scene to be reconstructed to form an initial data set D0; constructing an initial three-dimensional scene model of the forestry scene based on D0 and the 3DGS method; monitoring the forestry scene to construct an incremental data set D0 of the t-th change; t During each incremental learning phase, the previous incremental model is divided into changing and non-changing regions. By optimizing the changing and non-changing regions and constructing a new scene model, the incremental model for the tth incremental learning phase is obtained and used for image rendering. This invention avoids full scene reconstruction, significantly improving the efficiency of 3D reconstruction in dynamic environments while ensuring the accuracy of the 3D model. Furthermore, only a small number of new perspective images are needed to update the scene, significantly reducing the cost and time of data acquisition. It is particularly suitable for large-scale monitoring such as forestry, as well as scenarios requiring long-term monitoring and real-time updates.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a dynamic three-dimensional reconstruction method based on incremental updating. Background Art

[0002] With the rapid development of incremental 3D reconstruction technology, methods based on 3DGS (3D Gaussian Splatting) have shown great potential in fields such as forestry monitoring, urban planning, and environmental change monitoring. For example, in forestry monitoring, factors such as plant growth and small-scale natural disasters can cause changes in the forest scene, requiring regular updates to the 3D model to meet long-term monitoring needs.

[0003] Currently, common update methods, such as the patent application with application number 202410637023.7, have the following problems: (1) The entire scene needs to be reconstructed. Even in the face of dynamically changing scenes, a new model still needs to be recalculated and constructed, which leads to large computational workload, high computational cost, and low update efficiency. (2) This patented method requires the collection of a large number of multi-view images and detailed grid division and processing of each view. This will increase the cost and time of data collection for large-scale scenes (such as forest monitoring). (3) This patented method is mainly suitable for single modeling and static scene analysis. It cannot effectively cope with long-term changes in the scene and time-series dynamic updates, and cannot support long-term monitoring applications such as forest growth and vegetation changes.

[0004] How to ensure low data collection costs and high rendering accuracy in forestry monitoring to meet the needs of long-term monitoring and dynamic updates is an urgent problem to be solved.

[0005] 3D Gaussian Splatting (3DGS) is a 3D scene reconstruction and rendering technology. 3DGS uses a large number of 3D Gaussian distributions to represent the scene. A 3D Gaussian distribution (also called a 3D Gaussian sphere, Gaussian sphere, or Gaussian ellipsoid) represents a point in the scene, and these 3D Gaussian distributions are distributed on a 2D plane for rendering. 3DGS mainly includes the following steps: (1) Collect multi-view images of the three-dimensional scene and construct a point cloud of the three-dimensional scene from a set of images. This step can be achieved through the Structure from Motion (SfM) method; (2) Convert each point in the point cloud into a three-dimensional Gaussian distribution, which contains parameters such as position information, color information, and covariance matrix, to obtain a set of three-dimensional Gaussian distributions. For the convenience of description, this set of three-dimensional Gaussian distributions (Gaussian spheres) is also called Gaussian point cloud model or 3DGS model; (3) Train the parameters of the Gaussian distribution. Usually, a neural network is used to learn the optimal parameters of each three-dimensional Gaussian distribution, including position, covariance matrix, color, and opacity. The training process is also an adjustment process. By adjusting each three-dimensional Gaussian distribution, a set of adjusted three-dimensional Gaussian distributions is finally obtained, which is generally called a three-dimensional scene model; (4) Render a new view image of the scene based on the three-dimensional scene model.

[0006] During the training process, the loss function of 3DGS is generally to compare the difference between the real image and the rendered image, generally including L1 loss and structural similarity difference D-SSIM.

[0007] For each three-dimensional Gaussian distribution, the position refers to the center coordinates of the Gaussian distribution; the covariance matrix is ​​used to control the shape of the Gaussian distribution (the size and direction of the ellipsoid); the color is the RGB value, indicating the color of the area; and the opacity is used to control the contribution weight of the Gaussian distribution to the final image. Summary of the Invention

[0008] The purpose of the present invention is to provide a solution to the above problems, which can meet the needs of low data acquisition cost and high rendering accuracy in forestry scene monitoring, and is based on long-term monitoring and dynamic updating, and a dynamic three-dimensional reconstruction method based on incremental updating.

[0009] In order to achieve the above object, the technical solution adopted by the present invention is as follows: a dynamic three-dimensional reconstruction method based on incremental updating, comprising the following steps;

[0010] S1, obtain multi-view images of the forestry scene to be reconstructed to form the initial data set D0;

[0011] , where N is the total number of samples, (I j ,K j ,Rj ,t j ) is the jth sample in D0, I j is the jth image, K j is the camera intrinsic parameter matrix, R j is the camera's rotation matrix, t j is the camera's translation vector;

[0012] S2, constructing the initial 3D scene model of the forestry scene based on D0 and 3DGS method , Includes a set of Gaussian spheres;

[0013] S3, monitor the forestry scene. If there is a change, obtain multi-view images of the changed scene to form an incremental data set. The incremental data set corresponding to the t-th change is D t , t≥1, D t Same sample format as D0;

[0014] S4, based on D t And residual calculation, the incremental model obtained by the t-1th incremental learning Divided into a changing area and a non-changing area, including steps S41 to S45;

[0015] S41, create an empty set GS t , D t The image of each sample in is taken as the real image, and Generate the corresponding rendering, D t There are K samples in total, where the image of the kth sample is The rendering is ;

[0016] S42, calculation and The residual value of the corresponding pixel in the , the residual value of pixel p is ;

[0017] S43, select The pixels of A set of changing pixels , where 、 They are The median and standard deviation of all pixel residuals in ;

[0018] S44, according to the following formula The Gaussian spheres are stored in GS t ;

[0019] ,

[0020] Where ξ is the preset contribution threshold, w i (p) is The contribution of the i-th Gaussian ball to pixel p;

[0021] S45, process D in sequence according to S42~S44 t For each sample, store the Gaussian sphere filtered by each sample into GS t ,Will Medium GS t The Gaussian sphere is regarded as the changing area, and the rest are the non-changing areas;

[0022] S5, set the objective function of the change area , according to the following formula Parameters of the changing area Adjust to get the optimal parameters , and the adjusted Marked as the first incremental model ;

[0023] ,

[0024] ,

[0025] Where, for The increment of the parameter in , arg min (∙) is the argmin function, λ loc is a regularization term for local optimization;

[0026] S6, optimize the non-changing area, including S61~S62, and convert the optimized Marked as the second incremental model ;

[0027] S61, calculate the confidence of each Gaussian ball in the non-changing area, where the i-th Gaussian ball g i The confidence level is γ i ;

[0028] S62, preset lower confidence limit γ min , if γ i <γ min , then delete or fade out g i ;

[0029] S7, construct a new scene model, including S71~S72;

[0030] S71, construct a new scene set D zt , D zt There are K samples in total, where the kth sample is Relative to The pixel area related to the newly added scene;

[0031] S72, based on D zt and 3DGS method to construct a new scene model, and After fusion, we get the incremental model of the tth incremental learning ;

[0032] S8, use Render the image.

[0033] Preferably, the image is an RGB image, I j The size is H×W×3, where H and W are the height and width of the image respectively, and 3 is the number of color channels.

[0034] Preferably, the S2 is specifically:

[0035] S21, generates a 3D point cloud of the 3D scene using the SfM method for D0;

[0036] S22, using the 3DGS method to transform each point of the 3D point cloud into a Gaussian sphere, and all Gaussian spheres constitute a 3DGS model;

[0037] S23, taking the image in the sample as the real image, and using the 3DGS model to generate the rendering of each real image, when the 3DGS model parameters are When I j The rendering is , and calculate the loss function ;

[0038] ,

[0039] Where, , I j (p) are and I j The color value of pixel p in , is the square of the L2 norm, To avoid overfitting of the model, for The weight of

[0040] S24, to minimize Adjust the parameters of the 3DGS model Until the iteration ends, the initial three-dimensional scene model is obtained .

[0041] As a preference, in S42, the residual value of pixel p Calculate according to the following formula;

[0042] ,

[0043] Where, 、 They are and The color value of pixel p in , is the L2 norm.

[0044] Preferably, in S61, g i The confidence level is γ i Calculated according to the following formula:

[0045] ,

[0046] Where, is the number of pixels covered by the Gaussian sphere projection and whose contribution weight is greater than ξ, and p is The pixels in the image are denoted by , e is a natural constant, and β is the exponential decay parameter of the residual.

[0047] As a preference, in S62, fade out g i Specifically: If γ i <γ min , then use (1-η)⋅α i Update g i Opacity α i , where η is the attenuation factor, used to control each fade-out α i rate of decrease.

[0048] As a preferred embodiment, in S71, D zt The kth sample is A set of changed pixels .

[0049] As a preference, S72, based on D zt The new scene model is constructed using the 3DGS method, specifically:

[0050] Generate the initial Gaussian sphere of the pixels in the change area, where the initial Gaussian sphere corresponding to the mth pixel is , , where 、 、 、 They are 3D coordinates, initial covariance, color, and initial opacity of ; Acquired through the depth camera, from Obtained from, 0.2≤α0≤0.5.

[0051] As a preference, when rendering in S8, the color of pixel p in the rendered image is According to the following formula:

[0052] ,

[0053] ,

[0054] Where L is the total number of Gaussian balls used to render pixel p, and for the lth Gaussian ball g l , α l g l Opacity, w l (p) is g l Contribution to pixel p, u p is the two-dimensional coordinate of pixel p in the image, g l The two-dimensional coordinates of the projection center projected onto the imaging plane under the rendering perspective, T is the transposition operation, , is g l The covariance matrix on the imaging plane, exp(⋅) is the exp function.

[0055] About datasets D0 and D t :For the sample of the jth sample in D0 (I j ,K j ,R j ,t j ), , which contains the color information of the forest scene. , is the camera intrinsic parameter matrix, which includes parameters such as focal length and principal point, and controls the projection relationship of the image. , is the rotation matrix in the camera extrinsic parameter, which represents the rotation of the camera. : The translation vector in the camera external parameter, indicating the position of the camera. t The number t in the t-th change varies with the number of forestry scene changes during the monitoring process. t ,Only multi-view images of the changing scenes are needed to form an incremental dataset, without images of the entire forestry scene, thereby reducing the amount of data collected and the collection cost.

[0056] The idea of ​​the present invention is: (1) first construct the initial three-dimensional scene model of the forestry scene using the initial data set D0 and the 3DGS method ; (2) Monitor the forestry scene. When the forestry scene changes for the first time, construct an incremental data set D1, and then Perform the first incremental learning according to steps S4 to S7 of the invention to obtain the incremental model of the first incremental learning ; (3) Similarly, every time a change in the forestry scene is detected, an incremental data set is constructed and an incremental model is trained, where the incremental data set D t , learn the incremental model .

[0057] The present invention is performing incremental learning from Update to When, it is divided into the following steps:

[0058] (1) Press S4 first The Gaussian sphere is divided into a changing region and a non-changing region;

[0059] (2) According to S5, the parameters of the Gaussian sphere in the change area are optimized to obtain the first incremental model , = optimized change area + unoptimized non-change area. This step mainly processes the visible change areas with large residuals in the model.

[0060] (3) Optimize the non-changing area according to S6 to obtain the second incremental model ; = optimized changed area + optimized non-changed area; this step mainly processes invalid or invisible Gaussian spheres in the non-changed area.

[0061] (4) If there are new scenes, such as plant growth, etc., model the new scenes to obtain a new scene model and compare it with Fusion obtains the incremental model of the tth incremental learning .

[0062] Compared with the prior art, the advantages of the present invention are:

[0063] (1) Innovative updating method: The present invention optimizes and incrementally updates the local changed areas, deletes useless Gaussian spheres in the unchanged areas, and adds Gaussian spheres corresponding to the newly added scenes, thus avoiding the reconstruction of the entire scene. This significantly improves the efficiency of 3D reconstruction in dynamic environments while ensuring the accuracy of the 3D model.

[0064] (2) Small amount of data dependence: The present invention only requires a small amount of new perspective images to complete the scene update, which greatly reduces the cost and time of data collection. It is particularly suitable for large-scale monitoring such as forestry and scenes that require long-term monitoring and real-time updates.

[0065] (3) High-quality reconstruction: Through residual-driven Gaussian sphere updates, this method optimizes the image rendering quality in each incremental update, ensuring that the reconstruction results are highly consistent with the real scene.

[0066] (4) Dynamic adaptability: This method has strong temporal adaptability and can effectively cope with seasonal changes and dynamic environmental changes in forestry scenarios. It is suitable for long-term monitoring and real-time updating. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0068] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0069] Example 1: See Figure 1 ,A dynamic three-dimensional reconstruction method based on incremental ,update, includes the following steps;

[0070] S1, obtain multi-view images of the forestry scene to be reconstructed to form the initial data set D0;

[0071] , where N is the total number of samples, (I j ,K j ,R j ,t j ) is the jth sample in D0, I j is the jth image, K j is the camera intrinsic parameter matrix, R j is the camera's rotation matrix, t j is the camera's translation vector;

[0072] S2, constructing the initial 3D scene model of the forestry scene based on D0 and 3DGS method , Includes a set of Gaussian spheres;

[0073] S3, monitor the forestry scene. If there is a change, obtain multi-view images of the changed scene to form an incremental data set. The incremental data set corresponding to the t-th change is D t , t≥1, D t Same sample format as D0;

[0074] S4, based on D t And residual calculation, the incremental model obtained by the t-1th incremental learning Divided into a changing area and a non-changing area, including steps S41 to S45;

[0075] S41, create an empty set GS t , D t The image of each sample in is taken as the real image, and Generate the corresponding rendering, D t There are K samples in total, where the image of the kth sample is The rendering is ;

[0076] S42, calculation and The residual value of the corresponding pixel in the , the residual value of pixel p is ;

[0077] S43, select The pixels of A set of changing pixels , where 、 They are The median and standard deviation of all pixel residuals in ;

[0078] S44, according to the following formula The Gaussian spheres are stored in GS t ;

[0079] ,

[0080] Where ξ is the preset contribution threshold, w i (p) is The contribution of the i-th Gaussian ball to pixel p;

[0081] S45, process D in sequence according to S42~S44 t For each sample, store the Gaussian sphere filtered by each sample into GS t ,Will Medium GS t The Gaussian sphere is regarded as the changing area, and the rest are the non-changing areas;

[0082] S5, set the objective function of the change area , according to the following formula Parameters of the changing area Adjust to get the optimal parameters , and the adjusted Marked as the first incremental model ;

[0083] ,

[0084] ,

[0085] Where, for The increment of the parameter in , arg min (∙) is the argmin function, λ loc is a regularization term for local optimization;

[0086] S6, optimize the non-changing area, including S61~S62, and convert the optimized Marked as the second incremental model ;

[0087] S61, calculate the confidence of each Gaussian ball in the non-changing area, where the i-th Gaussian ball g i The confidence level is γ i;

[0088] S62, preset lower confidence limit γ min , if γ i <γ min , then delete or fade out g i ;

[0089] S7, construct a new scene model, including S71~S72;

[0090] S71, construct a new scene set D zt , D zt There are K samples in total, where the kth sample is Relative to The pixel area related to the newly added scene;

[0091] S72, based on D zt and 3DGS method to construct a new scene model, and After fusion, we get the incremental model of the tth incremental learning ;

[0092] S8, use Render the image.

[0093] In S2 of this embodiment, by formula To train the initial 3D scene model The purpose is to minimize the error between the image rendered by the model and the actual image, thereby ensuring that the generated three-dimensional model restores the real scene as accurately as possible.

[0094] The contribution degree of the present invention, such as w in S44 i (p), is The contribution of the i-th Gaussian ball to pixel p is usually determined by the projection of the Gaussian ball in three-dimensional space and the pixel distance. shown.

[0095] Example 2: See Figure 1 Based on Example 1, we provide more specific operations as follows:

[0096] Regarding the image in S1, the image is an RGB image, I j The size is H×W×3, where H and W are the height and width of the image respectively, and 3 is the number of color channels.

[0097] Regarding S2, it specifically includes the following steps S21 to S24:

[0098] S21, generates a 3D point cloud of the 3D scene using the SfM method for D0;

[0099] S22, using the 3DGS method to transform each point of the 3D point cloud into a Gaussian sphere, and all Gaussian spheres constitute a 3DGS model;

[0100] S23, taking the image in the sample as the real image, and using the 3DGS model to generate the rendering of each real image, when the 3DGS model parameters are When I j The rendering is , and calculate the loss function ;

[0101] ,

[0102] Where, , I j (p) are and I j The color value of pixel p in , is the square of the L2 norm, To avoid overfitting of the model, for The weight of

[0103] S24, to minimize Adjust the parameters of the 3DGS model Until the iteration ends, the initial three-dimensional scene model is obtained .

[0104] In step S42, the residual value of pixel p Calculate according to the following formula;

[0105] ,

[0106] Where, 、 They are and The color value of pixel p in , is the L2 norm.

[0107] In step S61, g i The confidence level is γ i Calculated according to the following formula:

[0108] ,

[0109] Where, is the number of pixels covered by the Gaussian sphere projection and whose contribution weight is greater than ξ, and p is The pixels in the image are denoted by , e is a natural constant, and β is the exponential decay parameter of the residual.

[0110] In step S62, fade out g i Specifically: If γi <γ min , then use (1-η)⋅α i Update g i Opacity α i , where η is the attenuation factor, used to control each fade-out α i rate of decrease.

[0111] In step S71, D zt The kth sample is A set of changing pixels .

[0112] Step S72, based on D zt The new scene model is constructed using the 3DGS method, specifically:

[0113] Generate the initial Gaussian sphere of the pixels in the change area, where the initial Gaussian sphere corresponding to the mth pixel is , , where 、 、 、 They are 3D coordinates, initial covariance, color, and initial opacity of ; Acquired through the depth camera, from Obtained from, 0.2≤α0≤0.5.

[0114] When rendering in step S8, the color of pixel p in the rendered image is According to the following formula:

[0115] ,

[0116] ,

[0117] Where L is the total number of Gaussian balls used to render pixel p, and for the lth Gaussian ball g l , α l g l Opacity, w l (p) is g l Contribution to pixel p, u p is the two-dimensional coordinate of pixel p in the image, g l The two-dimensional coordinates of the projection center projected onto the imaging plane under the rendering perspective, T is the transposition operation, , is g l The covariance matrix on the imaging plane, exp(⋅) is the exp function.

[0118] And about the color of pixel p in the rendering , assuming that pixel p is affected by three Gaussian balls, as shown in Table 1 below:

[0119] Table 1, Gaussian sphere parameter table

[0120] l=1~3 <![CDATA[α l ]]> <![CDATA[w l (p)]]> <![CDATA[c l ]]> 1 1.0 0.6 (1.0,0.0,0.0) 2 0.8 0.3 (0.0,1.0,0.0) 3 0.2 0.1 (0.0,0.0,1.0)

[0121] According to Table 1 and The calculation formula is as follows:

[0122] ,

[0123] The result is an RGB color value, which is the color effect of the three Gaussian spheres mixed together. Through the above formula, dynamic scenes can be rendered efficiently and with high quality, maintaining the continuity and authenticity of the rendering.

[0124] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A dynamic 3D reconstruction method based on incremental updating, characterized in that: The following steps are included: S1, obtain multi-view images of the forestry scene to be reconstructed to form the initial data set D0; , where N is the total number of samples, (I j ,K j ,R j ,t j ) is the jth sample in D0, I j is the jth image, K j is the camera intrinsic parameter matrix, R j is the camera's rotation matrix, t j is the camera's translation vector; S2, constructing the initial 3D scene model of the forestry scene based on D0 and 3DGS method , Includes a set of Gaussian spheres; S3, monitor the forestry scene. If there is a change, obtain multi-view images of the changed scene to form an incremental data set. The incremental data set corresponding to the t-th change is D t , t≥1, D t Same sample format as D0; S4, based on D t And residual calculation, the incremental model obtained by the t-1th incremental learning Divided into a changing area and a non-changing area, including steps S41 to S45; S41, create an empty set GS t , D t The image of each sample in is taken as the real image, and Generate the corresponding rendering, D t There are K samples in total, where the image of the kth sample is The rendering is ; S42, calculation and The residual value of the corresponding pixel in the , the residual value of pixel p is ; S43, select The pixels of A set of changing pixels , where 、 They are The median and standard deviation of all pixel residuals in ; S44, according to the following formula The Gaussian spheres are stored in GS t ; ， Where ξ is the preset contribution threshold, w i (p) is The contribution of the i-th Gaussian ball to pixel p; S45, process D in sequence according to S42~S44 t For each sample, store the Gaussian sphere filtered by each sample into GS t ,Will Medium GS t The Gaussian sphere is regarded as the changing area, and the rest are the non-changing areas; S5, set the objective function of the change area , according to the following formula Parameters of the changing area Adjust to get the optimal parameters , and the adjusted Marked as the first incremental model ; ， ， Where, for The increment of the parameter in , arg min (∙) is the argmin function, λ loc is a regularization term for local optimization; S6, optimize the non-changing area, including S61~S62, and convert the optimized Marked as the second incremental model ; S61, calculate the confidence of each Gaussian ball in the non-changing area, where the i-th Gaussian ball g i The confidence level is γ i ; S62, preset lower confidence limit γ min , if γ i <γ min , then delete or fade out g i ; S7, construct a new scene model, including S71~S72; S71, construct a new scene set D zt , D zt There are K samples in total, where the kth sample is Relative to The pixel area corresponding to the newly added scene; S72, based on D zt and 3DGS method to construct a new scene model, and After fusion, we get the incremental model of the tth incremental learning ; S8, use Render the image.

2. The dynamic 3D reconstruction method based on incremental updating according to claim 1, characterized in that: The image is an RGB image, I j The size is H×W×3, where H and W are the height and width of the image respectively, and 3 is the number of color channels.

3. The dynamic 3D reconstruction method based on incremental updating according to claim 1, characterized in that: The S2 is specifically: S21, generates a 3D point cloud of the 3D scene using the SfM method for D0; S22, using the 3DGS method to transform each point of the 3D point cloud into a Gaussian sphere, and all Gaussian spheres constitute a 3DGS model; S23, taking the image in the sample as the real image, and using the 3DGS model to generate the rendering of each real image, when the 3DGS model parameters are When I j The rendering is , and calculate the loss function ; ， Where, , I j (p) are and I j The color value of pixel p in , is the square of the L2 norm, To avoid overfitting of the model, for The weight of S24, to minimize Adjust the parameters of the 3DGS model Until the iteration ends, the initial three-dimensional scene model is obtained .

4. The dynamic 3D reconstruction method based on incremental updating according to claim 1, characterized in that: In S42, the residual value of pixel p Calculate according to the following formula; ， Where, 、 They are and The color value of pixel p in , is the L2 norm.

5. The dynamic 3D reconstruction method based on incremental updating according to claim 1, characterized in that: In S61, g i The confidence level is γ i Calculated according to the following formula: ， Where, is the number of pixels covered by the Gaussian sphere projection and whose contribution weight is greater than ξ, and p is The pixel in the image is , e is a natural constant, and β is the exponential decay parameter of the residual.

6. The dynamic 3D reconstruction method based on incremental updating according to claim 1, characterized in that: In S62, fade out g i Specifically: If γ i <γ min , then use (1-η)⋅α i Update g i Opacity α i , where η is the attenuation factor, used to control each fade-out α i rate of decrease.

7. The dynamic 3D reconstruction method based on incremental updating according to claim 1, characterized in that: In S71, D zt The kth sample is A set of changing pixels .

8. The dynamic 3D reconstruction method based on incremental updating according to claim 1, characterized in that: S72, based on D zt The new scene model is constructed using the 3DGS method, specifically: Generate the initial Gaussian sphere of the pixels in the change area, where the initial Gaussian sphere corresponding to the mth pixel is , , where 、 、 、 They are 3D coordinates, initial covariance, color, and initial opacity of ; Acquired through the depth camera, from Obtained from, 0.2≤α0≤0.

5.

9. The dynamic 3D reconstruction method based on incremental updating according to claim 1, characterized in that: When rendering in S8, the color of pixel p in the rendered image According to the following formula: ， ， Where L is the total number of Gaussian balls used to render pixel p, and for the lth Gaussian ball g l , α l g l Opacity, w l (p) is g l Contribution to pixel p, u p is the two-dimensional coordinate of pixel p in the image, g l The two-dimensional coordinates of the projection center projected onto the imaging plane under the rendering perspective, T is the transposition operation, , is g l The covariance matrix on the imaging plane, exp(⋅) is the exp function.

Images