A method for sputtering based on uncertainty and view supporting dual module gating

Abstract

The present application relates to a kind of Gaussian sputtering method based on uncertainty and view support dual module gating, belong to computer vision, three-dimensional reconstruction, neural rendering and explicit scene representation technical field, for the stubborn ghost problem caused by dynamic interference in multi-view image reconstruction, establish the dual evidence state division mechanism with the uncertainty module and view support module of Gaussian cell as parallel core module, Gaussian cell is divided into multiple states, according to the cloning and splitting operation of state to Gaussian cell Differentiated gating, first obtain the coarse training model that main body and background can be restored, dynamic interference is not refined, then utilize the semantic difference mask constrained by state to carry out two-stage scene fine reconstruction, to reduce the influence of dynamic interference and error Gaussian on subsequent reconstruction, to suppress stubborn ghost and obtain fine static three-dimensional scene expression, improve the stability and robustness of three-dimensional reconstruction under complex scene.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, 3D reconstruction, neural rendering, and explicit scene representation, specifically to a Gaussian sputtering method based on dual-module gating of uncertainty and view support. Background Technology

[0002] 3D Gaussian sputtering (3DGS) is a technique that uses explicit Gaussian primitives to represent scenes and rasterization to achieve novel perspective synthesis. It boasts high training efficiency, extremely fast inference speed, and very high scene reconstruction quality. However, it heavily relies on the "static world" assumption. In uncontrolled scenes containing dynamic disturbances such as moving pedestrians and vehicles, existing Gaussian sputtering methods are prone to generating erroneous Gaussian structures during training. These erroneous structures typically manifest as persistent ghosting, inconsistent local clumps across perspectives, or incorrect occlusion structures, severely impacting the desired static scene representation.

[0003] Existing anti-interference methods (such as WildGaussians) introduce uncertainty prediction, but suffer from heavy online computational burdens and are limited to two dimensions, failing to address the fundamental problem of erroneous Gaussian unit (EGU) densification corresponding to interference objects in three dimensions. Some schemes delay or uniformly control the EGU densification process as a whole. While this one-size-fits-all approach can suppress the growth of erroneous EGUs in the early training stages to some extent, it does not further distinguish between cloning and splitting operations, nor does it establish an independent module to differentiate between "high-support conflict regions" and "low-support background regions." Therefore, it still struggles to simultaneously address dynamic suppression and background recovery in the early training stages. Thus, a Gaussian sputtering method based on dual-module gating of uncertainty and view support is urgently needed to solve these problems. Summary of the Invention

[0004] The purpose of this invention is to provide a Gaussian sputtering method based on dual-module gating of uncertainty and view support to solve the problems mentioned in the background art.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: a Gaussian sputtering method based on uncertainty and view support dual-module gating, comprising the following steps:

[0006] Obtain a multi-view image sequence of the scene to be reconstructed, and initialize the basic Gaussian set in the scene;

[0007] A dual-evidence state partitioning mechanism is established with the uncertainty module and view support module of Gaussian elements as parallel core modules. The Gaussian elements are divided into multiple states, and differentiated gating is applied to cloning and splitting operations based on the states. A coarse model is output, in which the main body and background structures can be recovered and dynamic disturbances are not refined.

[0008] Based on the coarse model rendering training view, the corresponding coarse model rendering image is obtained, and the coarse model rendering image and the corresponding original multi-view image are input into the pre-trained visual feature extraction model. The Gaussian state constraint is combined to generate a semantic difference mask. The semantic difference mask uses a quantile threshold to generate an interference region mask.

[0009] The interference region mask is incorporated into the weighted calculation of the reconstruction loss to further optimize the scene representation, resulting in the final 3D Gaussian scene representation.

[0010] Preferably, the multi-view image sequence is processed to obtain camera intrinsic and extrinsic parameters, and distortion correction is performed on the multi-view image sequence; a sparse point cloud is generated as a basic Gaussian element set, each Gaussian element including position parameter, scale parameter, rotation parameter, opacity parameter and color / spherical harmonic parameter.

[0011] Preferably, the uncertainty module addresses the positional parameters in the Gaussian element set. Scale parameters and opacity parameter Variational modeling is performed to obtain parametric uncertainty information characterizing the uncertainty of each Gaussian element. :

[0012] ;

[0013] in, For the corresponding weighting coefficients, The standard deviation is denoted as .

[0014] Preferably, the view support module determines the support information through calculation of any of the following feature dimensions:

[0015] Each Gaussian element is determined to be a valid number of times it is visible in all training iterations;

[0016] The effective projected area or effective projected contribution of each Gaussian element in each training view;

[0017] The perspective distribution of each Gaussian element is diverse.

[0018] Preferably, the division rule of the dual-evidence state division mechanism is as follows:

[0019] First state: , is defined as a stable Gaussian element;

[0020] Second state: , is defined as a weakly supported Gaussian element;

[0021] Third state: , defined as conflicting high-ski elements;

[0022] Fourth state: , defined as the unconfirmed Gaussian element;

[0023] in, For the first Support information for each Gaussian element; For the first The relative uncertainty of each Gaussian element, Support thresholds and uncertainty thresholds for a given view.

[0024] Preferably, for each Gaussian element, its parameter uncertainty is combined. and support information By dynamically adjusting the uncertainty, the first Splitting gradient threshold of Gaussian units With clone trigger probability :

[0025] ;

[0026] ;

[0027] in, Based on the splitting threshold, and For control coefficients, for function;

[0028] Based on split gradient threshold With clone trigger probability The differentiated gating is as follows:

[0029] For the conflicting Gaussian units in the third state, prioritize restricting splitting operations and increase their cloning threshold, decrease their opacity growth rate, or increase their pruning priority.

[0030] For the weakly supported Gaussian unit in the second state, the restricted cloning operation is retained while the splitting operation is restricted to maintain background recovery;

[0031] For the stable Gaussian units in the first state, cloning and splitting operations are performed according to a preset densification strategy;

[0032] For the unconfirmed Gaussian units in the fourth state, their splitting operation is delayed and they are kept pending observation and updates.

[0033] Preferably, for the conflicting Gaussian element in the third state, its opacity cap and pruning priority score are set as follows:

[0034] ;

[0035] ;

[0036] in This represents the upper limit of opacity currently allowed by the Gaussian element. This is the default upper limit. For the current pruning rating of Gaussian elements, The default rating is set. The conflict score of the Gaussian element in this conflict. These are the inhibition coefficient and the enhancement coefficient, respectively.

[0037] For weakly supported Gaussian elements in the second state, the splitting operation is restricted by the following constraints:

[0038] ;

[0039] in, This is the constraint coefficient.

[0040] Preferably, the generation of the interference region mask includes:

[0041] Render the current view using a coarse model to obtain the result compared to each training view. Corresponding rough model rendering image ;

[0042] And the rough model rendering By inputting the corresponding original multi-view image into the pre-trained visual feature extraction model, semantic feature vectors are obtained. and ;

[0043] The semantic difference between the two images is calculated using cosine similarity, and then bilinear interpolation is used to upsample back to the original image resolution to form a semantic difference map. :

[0044] ;

[0045] Based on semantic difference maps Construct the composite mask fraction matrix Its joint energy function can be expressed as:

[0046] ;

[0047] in, This represents the normalized absolute photometric error. This represents the aggregated two-dimensional projection of the Gaussian element parameter uncertainty at the current viewpoint. For the corresponding fusion weight coefficients, Represents pixel coordinates;

[0048] The semantic difference mask uses a quantile threshold. Generate initial interference region mask:

[0049] ;

[0050] Morphological purification of the initial interference mask is performed using information on the area or structural continuity of connected components; the filtered area is smaller than... Scattered, isolated noise areas The minimum area threshold of the connected components is used, and the mask holes are filled by performing a closing operation using the structuring element radius to output the interference region mask. .

[0051] Preferably, a weighted calculation of the image reconstruction loss is introduced. for:

[0052] ;

[0053] in, It is the set of all pixel coordinates in a two-dimensional plane. This is the soft mask suppression coefficient.

[0054] Preferably, the projection correspondence from a two-dimensional mask to three-dimensional space is used to satisfy... The Gaussian element imposes explicit additional constraints on its... Add an opacity decay regularization term:

[0055] ;

[0056] in, Indicates the first The projection coordinates of the center of a three-dimensional Gaussian element onto a two-dimensional image. Its opacity parameter, For regularization weights.

[0057] Beneficial effects: This invention establishes a dual-evidence state division mechanism with the uncertainty module and view support module of Gaussian elements as parallel core modules, no longer relying on a single uncertainty signal, thereby improving the ability to distinguish between dynamic interference conflict regions and static weak support regions; and based on this mechanism, Gaussian elements are divided into multiple states, and differentiated gating is implemented for cloning and splitting operations according to the states, preventing the excessive refinement of dynamic interference regions from the source and effectively suppressing ghosting generation.

[0058] Furthermore, this invention employs offline mask generation and module decoupling design, reducing computational overhead and enhancing training stability. It improves real-time reconstruction performance under dynamic interference without sacrificing reconstruction quality. The fusion of semantic and texture features significantly improves interference recognition accuracy and reduces false background detections. The final output is a standard 3D Gaussian primitive set, maintaining real-time rendering speed and compatibility with the 3DGS standard rasterization renderer. Attached Figure Description

[0059] Figure 1 This is a flowchart of the Gaussian sputtering three-dimensional reconstruction method of the present invention;

[0060] Figure 2 This is a comparison chart of the reconstruction results of this invention and the original 3DGS reconstruction results. Detailed Implementation

[0061] To make the objectives and advantages of this invention clearer, the invention will be specifically described below with reference to embodiments. It should be understood that the following text is merely used to describe one or more specific embodiments of the invention and does not strictly limit the scope of protection specifically claimed by the invention.

[0062] This addresses a typical scenario: the input data contains dynamic disturbances, while the goal is to recover the 3D Gaussian primitive representations of the static subject and background. Existing 3DGS methods directly use conventional densification mechanisms, which easily trigger erroneous 3D Gaussian primitive growth in regions of dynamic disturbances, causing dynamic objects to evolve into clear ghosting or floating artifacts during subsequent training. This paper proposes a Gaussian sputtering method based on a dual-module gated approach of uncertainty and view support. The specific process is as follows: Figure 1 As shown, it includes the following steps:

[0063] S1. Obtain the multi-view image sequence of the scene to be reconstructed, and initialize the basic Gaussian set in the scene;

[0064] In one embodiment, the multi-view image is processed using Structure for Motion Restoration (SFM) to obtain camera intrinsic and extrinsic parameters, while the multi-view image is distorted; and a sparse point cloud is generated as the initial set of three-dimensional Gaussian elements for iteration, each of which includes at least position parameters, scale parameters, rotation parameters, opacity parameters, and color / spherical harmonic parameters.

[0065] S2. Establish the uncertainty module and view support module of Gaussian elements as parallel core modules; among which:

[0066] S21, the Gaussian element uncertainty module quantifies the uncertainty of each Gaussian element in its spatial, geometric, and photometric properties; its core functional logic is achieved through an uncertainty evaluation function. express:

[0067] ;

[0068] in, For the first The overall uncertainty score of each Gaussian element; This is the set of fundamental physical parameters for this primitive. This is the set of auxiliary variables learned by this module regarding the parameter distribution characteristics; the module ultimately outputs the relative parameter uncertainty information of each primitive as the first evidence for subsequent state division.

[0069] In one embodiment, the Gaussian uncertainty module is implemented through variational migration modeling, and specifically includes the following sub-modules;

[0070] Position uncertainty submodule: for Gaussian position parameters Perform distribution modeling; define the base location as By learning the offset mean with standard deviation The actual sampling location is obtained:

[0071] ;

[0072] in, For random disturbance factors, It reflects the uncertainty of spatial location.

[0073] Scale uncertainty submodule: for Gaussian scale parameters Interval perturbation modeling is adopted; basic scale parameters are defined. By learning interval boundaries With width The sampling scale is obtained as follows:

[0074] ;

[0075] in The scale perturbation factor, the interval width This characterizes the uncertainty in the geometric coverage of the primitive.

[0076] Opacity offset distribution: Modeling opacity in the transformation space; defining the basic opacity parameter. By learning its distribution standard deviation and mean Then through a temperature coefficient The function maps back to probability space; its actual sampling opacity for:

[0077] ;

[0078] in, for Function, Opacity Perturbation Factor , It follows a standard normal distribution. Temperature hyperparameters for controlling mapping smoothness;

[0079] In this embodiment, a Kullback-Leibler (KL) divergence regularization term is introduced for the three distributions mentioned above to pull the posterior distribution back to the preset prior distribution. After obtaining a stable variational offset distribution, the parameter uncertainty information of each Gaussian element is further calculated. The position offset dispersion is normalized relative to the Gaussian element's own scale. The scale offset interval width is normalized relative to the base scale. The opacity offset dispersion is normalized relative to the base opacity. Relative uncertainty information is used, and its calculation method includes the following combination:

[0080] ;

[0081] in, These are the corresponding weighting coefficients; this relative uncertainty This will serve as the core control signal for subsequent gating densification. In this embodiment, the number of uncertainty preheating iteration steps is set. The value is 1500. After the number of iterations reaches the warm-up step count, the uncertainty assessment is initiated.

[0082] S22. The View Support Module aims to quantify the strength and stability of objective evidence regarding the existence of Gaussian units in a scene from the perspective of multi-view observation. The View Support Module is updated during each training view rendering and records at least whether each Gaussian unit is effectively observed by the current view, whether it produces an effective projection in the image plane, and whether it makes an effective contribution to the pixels. Its core functional logic is achieved through the View Support Evaluation Function. express:

[0083] ;

[0084] in, For the first Support information for each Gaussian unit, including support information Visibility frequency Diverse perspectives Projective geometric contribution Each feature dimension is uniquely determined.

[0085] Let the first A Gaussian unit in the training view set The valid visibility indicator on is The effective projected area is The unit vector for the view direction is The view support module determines support information using any of the following feature dimensions:

[0086] Visibility frequency This metric represents the total number of times the Gaussian primitive was deemed validly visible across all training iterations. It reflects the observational stability of the primitive during training. The calculation formula is as follows:

[0087] ;

[0088] Diversity of perspectives The number of different camera perspectives that can be observed for this primitive and the breadth of its spatial distribution are statistically analyzed; the more comprehensive the perspective coverage, the more sufficient the evidence for the consistency of the primitive across multiple perspectives.

[0089] ;

[0090] in q represents the set of training views that the Gaussian element is effectively observed; p and q are indices in the summation formula;

[0091] Projective geometric contribution Calculate the projected area, weight contribution, and cross-view overlap of the primitive in each view. This metric reflects the strength of the spatial constraints imposed on the primitive.

[0092] ;

[0093] in, To prevent extremely small constants with a denominator of 0.

[0094] S3. Construct a dual-evidence state division mechanism with the uncertainty module and view support module of Gaussian elements as parallel core modules, and divide Gaussian elements into multiple states;

[0095] In this embodiment, the division rules of the dual-evidence state division mechanism are as follows:

[0096] First state: , defined as a stable high-level primitive; this state corresponds to a view with high support and low uncertainty, usually representing a static subject, static background or converged structure, and can be used as the main object for normal densification;

[0097] Second state: Defined as a weakly supported high-level primitive; this state corresponds to low view support and high uncertainty, typically representing distant backgrounds, areas that have been occluded for a long time, or areas with insufficient support. This state should not be simply regarded as an interference area, but a certain recovery path should be reserved;

[0098] Third state: , defined as conflicting Gaussian elements; this state corresponds to high view support and high uncertainty, usually representing a region that is repeatedly observed by multiple views but is still unstable, and most likely corresponds to dynamic disturbances or multi-view conflict regions.

[0099] Fourth state: This is defined as a Gaussian element to be confirmed; this state corresponds to a view with low support and low uncertainty, typically representing a region that has just been initialized or has not yet been fully optimized. The Gaussian splitting operation can be delayed for this state, and observation can continue.

[0100] in, and The first The overall support and relative uncertainty of each Gaussian element. Support thresholds and uncertainty thresholds for a given view.

[0101] S4. First-stage differentiated density gating: Differentiated gating is applied to cloning and splitting operations based on the state; and a coarse model is output where the main body and background structures can be recovered and dynamic disturbances are not refined.

[0102] 3D Gaussian densification includes at least two operations: 3D Gaussian cloning and 3D Gaussian splitting. 3D Gaussian cloning typically replicates small-scale 3D Gaussian units and gradually differentiates them during subsequent training, suitable for supplementing coverage and representational capacity. 3D Gaussian splitting typically divides a larger 3D Gaussian unit into multiple smaller 3D Gaussian units, suitable for improving local resolution and high-frequency detail representation. For each 3D Gaussian unit, this embodiment combines its parameter uncertainty. and support information The control strategy is as follows: dynamically adjust the first... Splitting gradient threshold of a 3D Gaussian unit With clone trigger probability :

[0103] ;

[0104] ;

[0105] in, Based on the splitting threshold, and For control coefficients;

[0106] Based on split gradient threshold With clone trigger probability Differentiated gating is as follows:

[0107] For conflicting Gaussian units in the third state, prioritize restricting splitting operations and increase their cloning threshold, decrease their opacity growth rate, or increase their pruning priority.

[0108] For weakly supported Gaussian units in the second state, restricted cloning operations are retained while splitting operations are restricted to maintain background recovery.

[0109] For stable Gaussian units in the first state, cloning and splitting operations are performed according to a preset densification strategy;

[0110] For the unconfirmed Gaussian unit in the fourth state, delay its splitting operation and keep it pending observation and update.

[0111] In one embodiment, for the conflicting Gaussian unit in the third state, its opacity cap and pruning priority score are set as follows:

[0112] ;

[0113] ;

[0114] in This represents the upper limit of opacity currently allowed by the Gaussian element. This is the default upper limit. For the current pruning rating of Gaussian elements, The default rating is set. The conflict score of the Gaussian element in this conflict. These are the inhibition coefficient and the enhancement coefficient, respectively; these are used to suppress the growth of its opacity and increase its priority in pruning operations.

[0115] The formula for calculating the conflict score is as follows: ,in and This is the slope coefficient.

[0116] For weakly supporting Gaussian elements in the second state, the following formula can be used to impose additional constraints on the splitting gradient threshold, while maintaining the original cloning trigger frequency:

[0117]

[0118] in, This is a constraint coefficient. The practical effect of this setting is that the difficulty of triggering a split increases with increasing uncertainty, but the cloning operation remains... Continuous adjustment allows for the preservation of the ability to restore static backgrounds;

[0119] For the fourth-state unconfirmed Gaussian unit, set its splitting operation delay step number. Set its split threshold before reaching the delay step number. for This is done to delay the splitting operation.

[0120] For first-state stable Gaussian units, without additional restrictions, a densification strategy is performed based on the original values ​​of the split gradient threshold and the cloning trigger probability.

[0121] In one embodiment, the training process is divided into stages. If each step is divided into a statistical window, then the termination condition for the first stage of gated dense training is at least one of the following: reaching the preset number of iterations. ;continuous Rate of change of Gaussian state distribution within a statistical window If this condition is met, the state distribution can be considered to have converged. , For the first When the window ends, the first The proportion of Gaussian states in the class of states Preset rate of change threshold; continuous Split trigger frequency within a statistical window If this condition is met, the denserization operation can be considered to have converged; where , The number of steps to perform a density check within the window. This represents the number of steps within the window that trigger the split operation. The preset frequency threshold; conflicting Gaussian cell splitting ratio If this condition is met, it can be considered that the refinement of dynamic interference has been effectively suppressed. , The Gaussian average of the conflict states within the window. This represents the number of times the split operation was triggered. This is a preset ratio. The first stage ends when any one of the above conditions is met, or they can be combined as needed.

[0122] After the first stage of gated densification training, the geometric evolution of dynamic objects can be automatically filtered, thereby outputting a clear coarse model in which the main body and background structure can be recovered and dynamic disturbances are not refined. After the first stage, at least the following information is output: coarse model Gaussian parameters, uncertainty of each Gaussian, view support information of each Gaussian, state label of each Gaussian, and coarse model rendering map and residual information corresponding to the training view.

[0123] S5. Second-stage semantic difference mask generation under state constraints: Obtain the corresponding coarse-model rendering image based on the coarse-model rendering training view, and input the coarse-model rendering image and the corresponding original multi-view image into the pre-trained visual feature extraction model to extract the semantic difference mask; In this embodiment: Perform the following operations for each training view:

[0124] Render the current view using a coarse model to obtain the result compared to each training view. Corresponding rough model rendering image ;

[0125] The coarse model rendering and the original input image are cropped in 14×14 units and then fed into a pre-trained visual feature extraction model to obtain semantic feature vectors. and This embodiment uses the DINOv2-Vits14-reg4 model;

[0126] The semantic difference between the two images is calculated using cosine similarity, and then bilinear interpolation is used to upsample back to the original image resolution to form a semantic difference map. :

[0127] ;

[0128] By combining the semantic difference map with the uncertainty map, view support map, residual map, and Gaussian state map output from the first stage, a synthetic mask score matrix is ​​constructed. Its joint energy function can be expressed as:

[0129] ;

[0130] in, This represents the normalized absolute photometric error. This is the aggregated two-dimensional projection of the uncertainty of the Gaussian element parameters estimated in three-dimensional space in the first stage to the current viewpoint. For the corresponding fusion weight coefficients, Represents pixel coordinates.

[0131] S6. Generate a semantic difference mask based on the mask fraction map and state constraints. In this embodiment, the semantic difference mask uses a quantile threshold. That is, the score matrix for the current view. Take the first The quantile is used as a threshold to generate the initial interference mask:

[0132] ;

[0133] Morphological purification of the initial mask is performed using information about the area or structural continuity of connected components; areas smaller than [a certain size] are filtered out. Scattered, isolated noise areas This is the minimum area threshold for connected components, expressed in pixels. In a preferred embodiment... The percentage is set as a certain proportion of the total number of pixels in the input image, ranging from 0.01% to 0.5%; and the radius of the structuring element is used. Perform a closing operation on the mask holes to fill them, thereby outputting a spatially smooth and accurately boundaryed interference region mask. This mask will be directly incorporated into the weighted calculation of the reconstruction loss in the next step.

[0134] S7. Introduce the interference region mask into the weighted calculation of the reconstruction loss to further optimize the scene representation and obtain the final 3D Gaussian scene representation.

[0135] In this embodiment, the loss function is constructed using mean absolute error and structural similarity, and the mask image is incorporated into the two-stage weighted loss. For two images, their mean is calculated. , Standard deviation , and covariance The formula for calculating its structural similarity is:

[0136]

[0137] Given a real image from the current perspective and rendering images Its mask-weighted image reconstruction loss Defined as:

[0138] ;

[0139] in, It is the set of all pixel coordinates in a two-dimensional plane. represents the soft mask suppression coefficient; fine-tuning training of the model is performed under soft mask weighting, using the projection correspondence from the two-dimensional mask to the three-dimensional space to satisfy... Explicit additional constraints are imposed on the 3D Gaussian primitives by adding an opacity decay regularization term to their loss function:

[0140] ;

[0141] in, Indicates the first The projection coordinates of the center of a three-dimensional Gaussian element onto a two-dimensional image. Its opacity parameter, The regularization weights are used to force the 3D Gaussian elements within the interference region to become transparent, thus promoting the gradual fading of incomplete ghost images.

[0142] In one embodiment, the variational sampling intensity or variational regularization intensity is gradually reduced in the later stage of fine reconstruction, and the Gaussian scene representation is folded into a deterministic representation. Deterministic fine training is then performed to restore the detail and sharpness of the final result. This embodiment uses annealing-based deterministic constraints; specifically, as the number of training iterations increases... The increase is based on the learning rate scheduling function. Adjusting the weights of the KL divergence regularization term forces the variational offset distributions of position, scale, and opacity to converge toward zero variance.

[0143] In the later stages of the second phase of training, a folding operation on the variational distribution is performed:

[0144] ;

[0145] ;

[0146] ;

[0147] The expected value of the distribution is permanently folded into the basic 3D Gaussian metaparameters. After parameter folding, the scene model sheds all probability distribution attributes and reverts to a purely deterministic 3D Gaussian scene representation. In this state, execution continues under mask constraints. Step-by-step deterministic refinement training, in which The preset positive integer; this process enables the lighting and texture of static scenes to converge quickly, greatly improving the sharpness and detail fidelity of the final rendering result.

[0148] In one specific case, a scene randomly collected outdoors was selected, including the static reconstructed subject and pedestrians dynamically interspersed within the scene, totaling 267 images with a resolution of 1008×756; the first stage coarse model iteration steps... The starting step number for dense gating is 7000. The density operation interval is 2500. The value is 100; the number of fine-tuning training iterations is... The value is 30,000; this yields a final, interference-free, refined model, with specific effects such as... Figure 2 As shown, taking a pose in the training set containing a handheld doll and a standing pedestrian occluding interference as an example, it can be seen that the 3DGS algorithm reconstructs an incomplete doll and pedestrian in this image pose, resulting in poor visual effects; while the method of this invention removes such interference in the reconstruction result, restores the real static scene behind the occlusion, and at the same time better preserves the details of the scene objects, resulting in good visual effects.

[0149] The embodiments of the present invention have been described in detail above with reference to the examples. However, the present invention is not limited to the above embodiments. For those skilled in the art, after learning the contents described in the present invention, several equivalent changes and substitutions can be made without departing from the principle of the present invention. These equivalent changes and substitutions should also be considered to fall within the protection scope of the present invention.

Claims

1. A Gaussian sputtering method based on dual-module gating of uncertainty and view support, characterized in that: Includes the following steps: Obtain a multi-view image sequence of the scene to be reconstructed, and initialize the basic Gaussian set in the scene; A dual-evidence state partitioning mechanism is established with the uncertainty module and view support module of Gaussian elements as parallel core modules. The Gaussian elements are divided into multiple states, and differentiated gating is applied to cloning and splitting operations based on the states. A coarse model is output, in which the main body and background structures can be recovered and dynamic disturbances are not refined. Based on the coarse model rendering training view, the corresponding coarse model rendering image is obtained, and the coarse model rendering image and the corresponding original multi-view image are input into the pre-trained visual feature extraction model. The Gaussian state constraint is combined to generate a semantic difference mask. The semantic difference mask uses a quantile threshold to generate an interference region mask. The interference region mask is incorporated into the weighted calculation of the reconstruction loss to further optimize the scene representation, resulting in the final 3D Gaussian scene representation. The uncertainty module determines the position parameters in the Gaussian element set. Scale parameters and opacity parameter Variational modeling is performed to obtain parametric uncertainty information characterizing the uncertainty of each Gaussian element. : ； in, For the corresponding weighting coefficients, Standard deviation The standard deviation of the distribution; The view support module determines support information by calculating any of the following feature dimensions: Each Gaussian element is determined to be a valid visible number throughout all training iterations; The effective projected area or effective projected contribution of each Gaussian element in each training view; The diversity of perspective distribution among the Gaussian elements; The division rules of the dual-evidence state division mechanism are as follows: First state: , is defined as a stable Gaussian element; Second state: , is defined as a weakly supported Gaussian element; Third state: , defined as conflicting high-ski elements; Fourth state: , defined as the unconfirmed Gaussian element; in, For the first Support information for each Gaussian element; The first The relative uncertainty of each Gaussian element, Support thresholds and uncertainty thresholds for a given view.

2. The Gaussian sputtering method based on dual-module gating of uncertainty and view support according to claim 1, characterized in that: The multi-view image sequence is processed to obtain camera intrinsic and extrinsic parameters, and distortion correction is performed on the multi-view image sequence. A sparse point cloud is generated as a basic set of Gaussian elements, each of which includes position parameters, scale parameters, rotation parameters, opacity parameters, and color / spherical harmonic parameters.

3. The Gaussian sputtering method based on dual-module gating of uncertainty and view support according to claim 1, characterized in that: For each Gaussian element, the combined parameter uncertainty is... and support information By dynamically adjusting the uncertainty, the first Splitting gradient threshold of Gaussian units With clone trigger probability : ； ； in, Based on the splitting threshold, and For control coefficients, for function; Based on split gradient threshold With clone trigger probability The differentiated gating is as follows: For the conflicting Gaussian units in the third state, prioritize restricting splitting operations and increase their cloning threshold, decrease their opacity growth rate, or increase their pruning priority. For the weakly supported Gaussian unit in the second state, the restricted cloning operation is retained while the splitting operation is restricted to maintain background recovery; For the stable Gaussian units in the first state, cloning and splitting operations are performed according to a preset densification strategy; For the unconfirmed Gaussian units in the fourth state, their splitting operation is delayed and they are kept pending observation and updates.

4. The Gaussian sputtering method based on dual-module gating of uncertainty and view support according to claim 3, characterized in that: For the conflicting Gaussian element in the third state, its opacity cap and pruning priority score are set as follows: ； ； in This represents the upper limit of opacity currently allowed by the Gaussian element. This is the default upper limit. For the current pruning rating of Gaussian elements, The default rating is set. The conflict score of the Gaussian element in this conflict. These are the inhibition coefficient and the enhancement coefficient, respectively. For weakly supported Gaussian elements in the second state, the splitting operation is restricted by the following constraints: ； in, This is the constraint coefficient.

5. The Gaussian sputtering method based on dual-module gating of uncertainty and view support according to claim 1, characterized in that: The generation of the interference region mask includes: Render the current view using a coarse model to obtain the result compared to each training view. Corresponding rough model rendering image ; And the rough model rendering By inputting the corresponding original multi-view image into the pre-trained visual feature extraction model, semantic feature vectors are obtained. and ; The semantic difference between the two images is calculated using cosine similarity, and bilinear interpolation is used to upsample back to the original image resolution to form a semantic difference map. : ； Based on semantic difference maps Construct the composite mask score matrix Its joint energy function can be expressed as: ； in, This represents the normalized absolute photometric error. This represents the aggregated two-dimensional projection of the Gaussian element parameter uncertainty at the current viewpoint. For the corresponding fusion weight coefficients, Represents pixel coordinates; The semantic difference mask uses a quantile threshold. Generate initial interference region mask: ； Morphological purification of the initial interference mask is performed using information on the area or structural continuity of connected components; the filtered area is smaller than... Scattered, isolated noise areas The minimum area threshold of the connected components is used, and the mask holes are filled by performing a closing operation using the structuring element radius to output the interference region mask. .

6. The Gaussian sputtering method based on dual-module gating of uncertainty and view support according to claim 5, characterized in that: Introducing a weighted calculation of image reconstruction loss for: ； in, It is the set of all pixel coordinates in a two-dimensional plane. This is the soft mask suppression coefficient.

7. The Gaussian sputtering method based on dual-module gating of uncertainty and view support according to claim 6, characterized in that: Through the projection correspondence from a two-dimensional mask to three-dimensional space, for satisfying The Gaussian element imposes explicit additional constraints on its... Add an opacity decay regularization term: ； in, Indicates the first The projection coordinates of the center of a three-dimensional Gaussian element onto a two-dimensional image. Its opacity parameter, For regularization weights.

Images