A method for re-distribution of gaussian point information
By acquiring a set of Gaussian points, constructing an approximate model and an association matrix, the redistribution of Gaussian point information is achieved, solving the overfitting and redundancy problems in sparse viewpoint 3D reconstruction and improving rendering quality and efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NAT UNIV OF DEFENSE TECH
- Filing Date
- 2026-04-09
- Publication Date
- 2026-06-09
AI Technical Summary
In 3D reconstruction tasks with sparse perspectives, 3DGS technology suffers from overfitting, leading to a decline in rendering quality. Existing methods also suffer from information loss and Gaussian point redundancy, increasing computational and storage burdens.
By acquiring a set of visible Gaussian points, randomly removing a preset number of Gaussian points, constructing an approximate model of pixel color with respect to opacity, calculating gradient terms, constructing an association matrix and information allocation coefficients, and progressively adjusting the masking rate of Gaussian points, the redistribution of Gaussian point information is achieved.
The Gaussian point size was reduced, information loss was avoided, rendering quality and reconstruction efficiency were improved, system complexity was simplified, and overfitting was alleviated.
Smart Images
Figure CN121999145B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of computer vision and 3D reconstruction technology, and in particular to a Gaussian point information redistribution method. Background Technology
[0002] Three-dimensional Gaussian sputtering (3DGS) technology has become a research hotspot in the field of 3D reconstruction due to its high rendering quality and efficient training performance, and is widely used in scenarios such as robot perception, autonomous driving, and virtual reality.
[0003] In 3D reconstruction tasks with sparse perspectives, 3DGS technology faces the problem of overfitting. During training, due to the limited input perspectives, floating artifacts are prone to occur, leading to a decline in rendering quality for new perspectives. Existing solutions often employ the method of randomly masking Gaussian points to improve the visibility of the remaining Gaussian points. However, this approach suffers from information loss and does not directly reduce the size of the Gaussian points, resulting in Gaussian point redundancy during the inference phase. This introduces additional computational and storage burdens, impacting reconstruction efficiency and quality. Summary of the Invention
[0004] Therefore, it is necessary to provide a Gaussian point information redistribution method that can compensate for information loss, alleviate overfitting, and reduce computational burden in response to the above-mentioned technical problems.
[0005] A Gaussian point information redistribution method, the method comprising:
[0006] Based on the 3D Gaussian sputtering framework, a set of visible Gaussian points covering the pixels is obtained, and the attribute parameters of each Gaussian point are extracted from the Gaussian point set to form a set of Gaussian point attribute parameters.
[0007] Randomly remove a preset number of Gaussian points from the set of visible Gaussian points as Gaussian points to be removed, and determine the neighborhood Gaussian point set corresponding to each Gaussian point to be removed in the set of visible Gaussian points;
[0008] Based on the Gaussian point attribute parameter set, an approximate model of pixel color with respect to Gaussian point opacity is constructed, and the gradient term of pixel color with respect to the opacity of each Gaussian point is calculated to form a gradient term set.
[0009] Based on the gradient term set, an association matrix is constructed between each Gaussian point to be removed and the corresponding Gaussian points in the neighborhood Gaussian point set. The information allocation coefficient of each neighborhood Gaussian point is calculated based on the association matrix and the information allocation factor, forming an information allocation coefficient set.
[0010] Based on the information allocation coefficient set, the opacity and color information of each Gaussian point to be removed are compensated to the Gaussian points in the corresponding neighborhood Gaussian point set. The opacity and color attribute parameters of the neighborhood Gaussian points are updated to obtain the updated Gaussian point attribute parameter set.
[0011] During the training iteration of 3D reconstruction, the Gaussian point masking rate is progressively and dynamically adjusted. Based on the adjusted Gaussian point masking rate, the associated parameters of Gaussian point opacity are modified to maintain the physical constraint range of Gaussian point opacity. At the same time, the redistribution of Gaussian point information in this round is completed based on the updated Gaussian point attribute parameter set until all training iterations of 3D reconstruction are completed, and the final Gaussian point attribute parameter set is obtained, realizing the redistribution of Gaussian point information.
[0012] The aforementioned Gaussian point information redistribution method first obtains a set of visible Gaussian points and extracts attribute parameters, providing complete basic data support for information redistribution and ensuring that subsequent calculations can accurately associate the core attributes of Gaussian points. Randomly removing a preset number of Gaussian points directly reduces the overall size of Gaussian points, reducing the computational and storage burden from the source and avoiding the inefficiency caused by Gaussian point redundancy in traditional methods. Second, it constructs an approximate model of pixel color with respect to opacity and calculates the gradient term, accurately quantifying the impact of the opacity of each Gaussian point on pixel color, providing a scientific basis for information allocation. Based on the gradient term, it constructs an association matrix and calculates information allocation coefficients, which can determine a reasonable information allocation ratio according to the correlation strength between Gaussian points, ensuring that the opacity and color information of the Gaussian points to be removed are accurately compensated to neighboring Gaussian points, completely avoiding the information loss caused by simply removing Gaussian points. The complete preservation of information allows the model to make full use of effective data during training, reducing floating artifacts caused by limited input viewpoints and significantly alleviating overfitting. Finally, the masking rate was progressively adjusted and the opacity-related parameters were modified during the training iterations. This maintained the physical constraints on opacity, ensuring the stability of rendering quality, and further improved the reconstruction effect by making the allocation of Gaussian point information more accurate through multiple rounds of iterative optimization. The entire process does not require the introduction of additional complex data augmentation modules or reliance on high-precision depth maps. While simplifying the system complexity, it also alleviates the overfitting problem and improves the overall rendering quality by reducing the size of Gaussian points and making efficient use of information. Attached Figure Description
[0013] Figure 1 This is a flowchart illustrating a Gaussian point information redistribution method in one embodiment;
[0014] Figure 2 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0015] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0016] In one embodiment, such as Figure 1 As shown, a Gaussian point information redistribution method is provided, including:
[0017] Step 102: Obtain the set of visible Gaussian points covering the pixels based on the three-dimensional Gaussian sputtering framework, and extract the attribute parameters of each Gaussian point from the Gaussian point set to form a set of Gaussian point attribute parameters.
[0018] 3D Gaussian sputtering technology represents the target visually using a set of Gaussian primitives (i.e., Gaussian points), where the attribute parameters of each Gaussian primitive are determined by three-dimensional coordinates. The covariance matrix of the primitives Color characteristics and opacity Composition, expression is The visible Gaussian point set refers to the collection of Gaussian points that can cover the pixels in the target scene and participate in rendering calculations. The aforementioned attribute parameters of each Gaussian point are extracted from this set, and after being organized according to the correspondence between Gaussian points and parameters, a Gaussian point attribute parameter set is formed, providing basic data for subsequent information redistribution calculations.
[0019] Step 104: Randomly remove a preset number of Gaussian points from the set of visible Gaussian points as Gaussian points to be removed, and determine the neighborhood Gaussian point set corresponding to each Gaussian point to be removed in the set of visible Gaussian points.
[0020] The preset number can be set according to actual training needs and scenario complexity, and random elimination is possible. For a given number of visible Gaussian points, removing these points allows the remaining Gaussian points to have a larger gradient and higher visibility. To reduce the impact of the removed points, the information from the removed points is distributed among the surrounding Gaussian points.
[0021] Step 106: Based on the Gaussian point attribute parameter set, construct an approximate model of pixel color with respect to Gaussian point opacity, calculate the gradient term of pixel color with respect to the opacity of each Gaussian point, and form a gradient term set.
[0022] First, based on parameters such as opacity and color features from the Gaussian point attribute parameter set, a calculation expression for pixel color is constructed in conjunction with transmittance. Then, through approximation, a relationship between pixel color and Gaussian point opacity is established, i.e., an approximate model. The gradient term is used to quantify the influence of Gaussian point opacity changes on pixel color. Each Gaussian point corresponds to a gradient term at each pixel it covers. Integrating all gradient terms according to the correspondence between Gaussian points and pixels forms a gradient term set, providing support for subsequent correlation matrix construction and information allocation coefficient calculation.
[0023] Specifically:
[0024] ;
[0025] in, Indicates the Gaussian point at the pixel The projection weights at the specified points are expressed as follows:
[0026] ;
[0027] in, Indicates the first The pixel coordinates of the projection center of a Gaussian point from the current viewpoint. Represents pixels Pixel coordinates The scaling parameter is obtained by approximating the projected covariance. To reduce computational complexity and enhance numerical stability, The approximation of the original two-dimensional anisotropic Gaussian kernel by using an isotropic Gaussian kernel can be understood as a spatial weighting function of the contribution intensity of Gaussian points to pixels.
[0028] Color changes can be represented in the following form:
[0029] ;
[0030] If Gaussian points are removed The resulting impact is as follows:
[0031] ;
[0032] The goal is to change the surrounding Gaussian points to reduce the loss of Gaussian points. The resulting impact, namely:
[0033] ;
[0034] in, This represents the set of neighboring points around the Gaussian point that were removed from the rendering process. The compact form is as follows:
[0035] ;
[0036] in, This represents the change in opacity of Gaussian points within a neighborhood set. This represents the change in opacity of the Gaussian point to be removed. and They are and The concatenated form. Solving the objective function defined by the compact form of the formula yields:
[0037] ;
[0038] in accordance with The structure is constructed to approximate its form. For the Gaussian point... and :
[0039] .
[0040] Step 108: Based on the gradient term set, construct the correlation matrix between each Gaussian point to be removed and the corresponding neighborhood Gaussian point set. Calculate the information allocation coefficient of each neighborhood Gaussian point based on the correlation matrix and the information allocation factor to form an information allocation coefficient set.
[0041] The correlation matrix is used to characterize the correlation strength between the Gaussian point to be removed and its neighboring Gaussian points. It is constructed based on the transmittance, projection weights, and color feature information of the Gaussian points in the gradient term set, and is obtained through cumulative calculation in the pixel space. Information allocation factor. The hyperparameter is a preset parameter used to adjust the proportion and intensity of information allocation. By combining the correlation matrix and its inverse matrix, the proportion of information to be removed Gaussian points that each neighboring Gaussian point should receive can be calculated, which is the information allocation coefficient. The information allocation coefficients of all neighboring Gaussian points are integrated to form the information allocation coefficient set.
[0042] Specifically, this includes: the cumulative value in pixel space is:
[0043] ;
[0044] also
[0045] ;
[0046] so
[0047] ;
[0048] make ,in Information allocation factors can be represented using... For a single neighbor Redistribute the opacity and color:
[0049] ;
[0050] .
[0051] Step 110: Based on the information allocation coefficient set, compensate the opacity and color information of each Gaussian point to be removed to the corresponding Gaussian points in the neighborhood Gaussian point set, update the opacity and color attribute parameters of the neighborhood Gaussian points, and obtain the updated Gaussian point attribute parameter set.
[0052] Based on the allocation coefficients corresponding to each neighboring Gaussian point in the information allocation coefficient set, the opacity and color information of the Gaussian point to be removed are proportionally compensated to the neighboring Gaussian points. A specific update formula is then used to correct the opacity and color attribute parameters of the neighboring Gaussian points, ensuring that the effective information of the Gaussian point to be removed is preserved. After all the updated attribute parameters of the Gaussian points are compiled, an updated Gaussian point attribute parameter set is formed, which is used for subsequent reconstruction calculations in this round.
[0053] Step 112: During the training iteration of 3D reconstruction, the Gaussian point masking rate is progressively and dynamically adjusted. Based on the adjusted Gaussian point masking rate, the associated parameters of Gaussian point opacity are modified to maintain the physical constraint range of Gaussian point opacity. At the same time, the redistribution of Gaussian point information in this round is completed based on the updated Gaussian point attribute parameter set until all training iterations of 3D reconstruction are completed, and the final Gaussian point attribute parameter set is obtained, thus realizing the redistribution of Gaussian point information.
[0054] The Gaussian point masking ratio refers to the proportion of Gaussian points temporarily masked in each training iteration. Masking is temporary and only applies to the Gaussian point set of the current training iteration. The random removal and masking of Gaussian points must be repeated in the next training iteration. Progressive dynamic adjustment refers to setting an initial masking ratio at the beginning of the training iteration and gradually increasing the Gaussian point masking ratio as the number of training iterations increases until a preset maximum masking ratio is reached and then remains constant. This is achieved by modifying the associated parameters of Gaussian point opacity. Instead of directly modifying the opacity It can maintain opacity The physical constraint interval ∈ [0,1] is used to ensure the stability of the expected transmittance and avoid numerical fluctuations. After completing the information redistribution in this round based on the updated Gaussian point attribute parameter set, steps 104 to 112 are repeated until all preset training iterations are completed. The final Gaussian point attribute parameter set can be used for the 3D reconstruction of the target scene to achieve effective redistribution of Gaussian point information.
[0055] The aforementioned Gaussian point information redistribution method first obtains a set of visible Gaussian points and extracts attribute parameters, providing complete basic data support for information redistribution and ensuring that subsequent calculations can accurately associate the core attributes of Gaussian points. Randomly removing a preset number of Gaussian points directly reduces the overall size of Gaussian points, reducing the computational and storage burden from the source and avoiding the inefficiency caused by Gaussian point redundancy in traditional methods. Second, it constructs an approximate model of pixel color with respect to opacity and calculates the gradient term, accurately quantifying the impact of the opacity of each Gaussian point on pixel color, providing a scientific basis for information allocation. Based on the gradient term, it constructs an association matrix and calculates information allocation coefficients, which can determine a reasonable information allocation ratio according to the correlation strength between Gaussian points, ensuring that the opacity and color information of the Gaussian points to be removed are accurately compensated to neighboring Gaussian points, completely avoiding the information loss caused by simply removing Gaussian points. The complete preservation of information allows the model to make full use of effective data during training, reducing floating artifacts caused by limited input viewpoints and significantly alleviating overfitting. Finally, the masking rate was progressively adjusted and the opacity-related parameters were modified during the training iterations. This maintained the physical constraints on opacity, ensuring the stability of rendering quality, and further improved the reconstruction effect by making the allocation of Gaussian point information more accurate through multiple rounds of iterative optimization. The entire process does not require the introduction of additional complex data augmentation modules or reliance on high-precision depth maps. While simplifying the system complexity, it also alleviates the overfitting problem and improves the overall rendering quality by reducing the size of Gaussian points and making efficient use of information.
[0056] In one embodiment, the Gaussian point attribute parameter set includes the color of the pixel; the color of the pixel is:
[0057] ;
[0058] in, Indicates the first i A Gaussian point at a pixel p Opacity at the location, Indicates the Gaussian point at the pixel Projection weights at that location, Indicates the first The pixel coordinates of the projection center of a Gaussian point from the current viewpoint. The scale parameter is obtained by approximating the projected covariance. For the first i A Gaussian point at a pixel p Color characteristics at the location, Represented as the first i A Gaussian point at a pixel p Transmittance at that location K Indicates the number of covered pixels p The number of Gaussian points.
[0059] Specifically, transmittance The calculation formula is: , indicating the first i The complementary product of the opacities of all Gaussian points preceding the current Gaussian point is used to represent the occlusion effect of the preceding Gaussian points on the color contribution of the current Gaussian point. This pixel color expression can more clearly reflect the relationship between the various attribute parameters of the Gaussian point and the pixel color, providing a concise and accurate basic formula for subsequent approximation model construction and gradient term calculation. By clarifying the calculation logic of pixel color, the influence of opacity on color can be quantified, providing a reliable basis for the compensation calculation of opacity and color information during information redistribution.
[0060] In one embodiment, based on the Gaussian point attribute parameter set, an approximate model of pixel color with respect to Gaussian point opacity is constructed, including:
[0061] Based on the Gaussian point attribute parameter set, an approximate model of pixel color with respect to Gaussian point opacity is constructed as follows:
[0062] ;
[0063] in, Represents the color of a pixel. Indicates the first i A Gaussian point at a pixel p Opacity at the location, Indicates the Gaussian point at the pixel Projection weights at that location, Indicates the first The pixel coordinates of the projection center of a Gaussian point from the current viewpoint. Represents pixels Pixel coordinates The scale parameter is obtained by approximating the projected covariance. For the first i A Gaussian point at a pixel p Color characteristics at the location, Indicates the first i A Gaussian point at a pixel p Transmittance at that location Represents pixels p The color of the first i The gradient term is opaque at a Gaussian point.
[0064] Specifically, the approximate model obtains an approximate relationship between pixel color and Gaussian opacity by ignoring higher-order quantities, thus enabling rapid calculation of the gradient of the effect of opacity changes on color. This simplifies the complex relationship between color and opacity, making gradient calculation more efficient and feasible, and providing key quantitative indicators for subsequent minimization of information loss and calculation of information allocation coefficients.
[0065] In one embodiment, the gradient term of the pixel color with respect to the opacity of each Gaussian point is calculated, forming a set of gradient terms, including:
[0066] By combining the projection weights of Gaussian points at the pixel points, the gradient term of the pixel color with respect to the opacity of each Gaussian point is calculated through an approximate model.
[0067] The gradient terms of all Gaussian points are integrated according to the correspondence between Gaussian points and pixels to form a gradient term set.
[0068] Specifically, firstly, for each Gaussian point, at each pixel it covers, the transmittance, color features, and projection weight of that Gaussian point are substituted into the formula of the approximate model to calculate the gradient term at that pixel. Then, according to the correspondence between Gaussian point number, pixel coordinates, and gradient term values, all calculated gradient terms are organized and integrated to form a gradient term set. This gradient term set contains information about the influence of each Gaussian point on the color of the pixels it covers, and is the core foundational data for subsequently constructing the objective function and the correlation matrix. Integrating the gradient term information ensures that subsequent information redistribution calculations can accurately correlate with each Gaussian point and its corresponding pixel, guaranteeing the accuracy of information compensation.
[0069] In one embodiment, the gradient term set provides the foundational data for subsequently constructing the objective function, which minimizes the impact of Gaussian point removal on pixel color. The objective function is expressed as:
[0070] ;
[0071] in, For the removed Gaussian points j The set of neighboring points involved in this rendering. For pixels p The color of the Gaussian point was removed j Opaque gradient term, For pixels p Gaussian points that contribute to color rendering are culled. j The amount of change in opacity For pixels p Gaussian points that contribute to color rendering i The amount of change in opacity For pixelsp Color at Gaussian point i Opaque gradient term.
[0072] Specifically, the core idea of this objective function is to compensate for the impact of removing Gaussian points by adjusting the change in opacity of each Gaussian point in the neighborhood Gaussian point set. j The resulting pixel color changes minimize color loss. By optimizing this objective function, the amount of opacity change that needs to be adjusted for neighboring Gaussian points can be determined, providing an optimization target for the calculation of information redistribution coefficients and ensuring that information redistribution can preserve the original rendering effect to the greatest extent.
[0073] In one embodiment, the compact form of the objective function is:
[0074] ;
[0075] in, Let represent the change in opacity of Gaussian points in the neighborhood point set. Let represent the change in opacity of the Gaussian point to be removed. and These are the concatenation forms of the gradient terms corresponding to the neighborhood point set and the Gaussian point to be removed, respectively.
[0076] Specifically, this compact form facilitates solving optimization problems through matrix operations, reducing computational complexity. It simplifies the objective function solution process, improves computational efficiency, and enables efficient information redistribution even in large-scale Gaussian point scenarios.
[0077] In one embodiment, based on the gradient term set, an association matrix is constructed between each Gaussian point to be removed and the corresponding Gaussian points in its neighborhood set, including:
[0078] Based on the gradient term set, the correlation matrix between each Gaussian point to be removed and the corresponding Gaussian points in its neighborhood set is constructed as follows:
[0079] ;
[0080] in, Indicates the first i A Gaussian point at a pixel p Transmittance at that location Indicates the first i A Gaussian point at pixel Projection weights at that location, Indicates the first j A Gaussian point at a pixel p Transmittance at that location Indicates the first j A Gaussian point at pixel Projection weights at that location, Indicates the first i The color characteristics of a Gaussian point Indicates the first j The color characteristics of a Gaussian point.
[0081] Specifically, the correlation matrix Used to measure the Gaussian points to be removed. j With neighboring Gaussian points i The correlation strength between them is calculated by examining all pixels affected by both. p The cumulative total is calculated above. Among them, Represents the Gaussian point i with Gauss point j Inner product in color space; and Representing the two Gaussian points at the pixel level p The rendering contribution intensity at each point is calculated. The overall correlation strength between the two points is obtained by summing the cumulative values in the pixel space, forming the elements of the correlation matrix. The correlation matrix can accurately quantify the correlation between the Gaussian point to be removed and its neighboring Gaussian points, providing a scientific basis for calculating the information allocation coefficients and ensuring that information is allocated to the neighboring Gaussian points with the highest correlation and the most suitable values.
[0082] In one embodiment, the information allocation coefficient of each neighborhood Gaussian point is calculated based on the correlation matrix and the information allocation factor, including:
[0083] Based on the correlation matrix and information allocation factor, the information allocation coefficient of each neighborhood Gaussian point is calculated as follows:
[0084] ;
[0085] in, Indicates information allocation factor, Represents the correlation matrix. This represents the inverse of the incidence matrix.
[0086] Specifically, the information allocation factor is a preset hyperparameter that can be adjusted based on experimental results to control the overall intensity of information allocation. Through the combined calculation of the correlation matrix and its inverse matrix, the information allocation coefficient accurately reflects the correlation strength between neighboring Gaussian points and the Gaussian points to be removed, achieving a reasonable allocation of information and avoiding rendering quality degradation caused by information imbalance.
[0087] In one embodiment, based on the information allocation coefficient set, the opacity and color information of each Gaussian point to be removed are compensated to the corresponding Gaussian points in the neighborhood Gaussian point set. The opacity and color attribute parameters of the neighborhood Gaussian points are then updated to obtain the updated Gaussian point attribute parameter set, including:
[0088] Based on the information allocation coefficient set, the opacity and color information of each Gaussian point to be removed are compensated to the corresponding Gaussian points in the neighborhood Gaussian point set. The opacity and color attribute parameters of the neighborhood Gaussian points are then updated, resulting in the following updated Gaussian point attribute parameters:
[0089] ;
[0090] ;
[0091] in, , These are the Gaussian points in the neighborhood before the update. i Opacity and color characteristics, , These are the Gaussian points to be removed. j Opacity and color characteristics, For neighborhood Gaussian points i Information allocation coefficient, , For neighborhood Gaussian points i The updated opacity and color characteristics.
[0092] Specifically, for each Gaussian point to be removed j Its opacity and color features According to information allocation coefficient The proportion is compensated to each Gaussian point in its neighborhood Gaussian point set. i Opacity updates use a direct accumulation method, proportionally adding the opacity of the Gaussian point to be removed to the original opacity of the neighboring Gaussian points; color feature updates use a weighted average method, combining the original opacity of the neighboring Gaussian points with the compensated opacity. ) as weights, for the original color features and the compensated color features ( A weighted average is applied to ensure natural color transitions. After all neighboring Gaussian points are updated, the attribute parameters of the Gaussian points that were not culled and those that were not updated are combined to form an updated set of Gaussian point attribute parameters. Through precise information compensation and parameter updates, the effective information of the Gaussian points to be culled is fully preserved, while maintaining the rationality of the attribute parameters of neighboring Gaussian points, thus avoiding rendering distortion caused by information compensation.
[0093] In one embodiment, the associated parameter of Gaussian point opacity is modified based on the adjusted Gaussian point masking rate, including:
[0094] Gaussian point opacity Associated parameters The correspondence is as follows:
[0095] ;
[0096] Based on Gaussian point shielding rate Modify associated parameters The process is as follows:
[0097] ;
[0098] ;
[0099] in, The original association parameters. This is the associated parameter for the modified Gaussian point opacity.
[0100] Specifically, Gaussian point opacity Associated parameters A correspondence is established between them through an exponential function to ensure... The value of always satisfies the physical constraint range ∈[0,1]. In each training iteration, based on the adjusted Gaussian point masking ratio... First through Calculate the current correlation parameters Then multiply it by 1 / (1- Obtain the modified association parameters. Finally passed Inversely, new opacity is derived. This is different from directly multiplying the opacity by 1 / (1-). Unlike the previous method, this approach, which adjusts opacity by modifying correlation parameters, better maintains the desired transmittance, ensuring numerical stability and physical consistency during optimization. It avoids the problem of values exceeding physical constraints that might occur when directly modifying opacity, thus ensuring the stability of the training process and the reliability of the reconstruction results.
[0101] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.
[0102] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 2 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a Gaussian point information redistribution method. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.
[0103] Those skilled in the art will understand that Figure 2 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0104] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0105] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0106] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A Gaussian point information redistribution method, characterized in that, The method includes: Based on the three-dimensional Gaussian sputtering framework, a set of visible Gaussian points covering the pixels is obtained, and the attribute parameters of each Gaussian point are extracted from the set of Gaussian points to form a set of Gaussian point attribute parameters. A predetermined number of Gaussian points are randomly removed from the set of visible Gaussian points as Gaussian points to be removed, and the neighborhood Gaussian point set corresponding to each Gaussian point to be removed in the set of visible Gaussian points is determined. Based on the Gaussian point attribute parameter set, an approximate model of pixel color with respect to Gaussian point opacity is constructed, and the gradient term of pixel color with respect to the opacity of each Gaussian point is calculated to form a gradient term set. Based on the gradient term set, an association matrix is constructed between each Gaussian point to be removed and the corresponding neighborhood Gaussian points. The information allocation coefficient of each neighborhood Gaussian point is calculated based on the association matrix and the information allocation factor, forming an information allocation coefficient set. Based on the information allocation coefficient set, the opacity and color information of each Gaussian point to be removed are compensated to the Gaussian points in the corresponding neighborhood Gaussian point set, and the opacity and color attribute parameters of the neighborhood Gaussian points are updated to obtain the updated Gaussian point attribute parameter set. During the training iteration of 3D reconstruction, the Gaussian point masking rate is progressively and dynamically adjusted. Based on the adjusted Gaussian point masking rate, the associated parameters of Gaussian point opacity are modified to maintain the physical constraint range of Gaussian point opacity. At the same time, the redistribution of Gaussian point information in this round is completed based on the updated Gaussian point attribute parameter set until all training iterations of 3D reconstruction are completed, and the final Gaussian point attribute parameter set is obtained, realizing the redistribution of Gaussian point information.
2. The method according to claim 1, characterized in that, The Gaussian point attribute parameter set includes the color of the pixel; the color of the pixel is: in, Indicates the first i A Gaussian point at a pixel p Opacity at the location, Indicates the Gaussian point at the pixel Projection weights at that location Indicates the first The pixel coordinates of the projection center of a Gaussian point from the current viewpoint. Represents pixels Pixel coordinates The scale parameter is obtained by approximating the projected covariance. For the first i A Gaussian point at a pixel p Color characteristics at the location, Represented as the first i A Gaussian point at a pixel p Transmittance at that location K Indicates the number of covered pixels p The number of Gaussian points.
3. The method according to claim 1, characterized in that, Based on the Gaussian point attribute parameter set, an approximate model of pixel color with respect to Gaussian point opacity is constructed, including: Based on the Gaussian point attribute parameter set, an approximate model of pixel color with respect to Gaussian point opacity is constructed as follows: in, Represents the color of a pixel. Indicates the first i A Gaussian point at a pixel p Opacity at the location, Indicates the Gaussian point at the pixel Projection weights at that location Indicates the first The pixel coordinates of the projection center of a Gaussian point from the current viewpoint. Represents pixels Pixel coordinates The scale parameter is obtained by approximating the projected covariance. For the first i A Gaussian point at a pixel p Color characteristics at the location, Indicates the first i A Gaussian point at a pixel p Transmittance at that location Represents pixels p The color of the first i The gradient term is opaque at a Gaussian point.
4. The method according to claim 1, characterized in that, The gradient term of the pixel color with respect to the opacity of each Gaussian point is calculated, forming a set of gradient terms, including: By combining the projection weights of Gaussian points at pixel points, the gradient term of pixel color with respect to the opacity of each Gaussian point is calculated through the approximate model. The gradient terms of all Gaussian points are integrated according to the correspondence between Gaussian points and pixels to form the gradient term set.
5. The method according to claim 4, characterized in that, The gradient term set provides the basic data for constructing the objective function, which minimizes the impact of removing Gaussian points on pixel color. Its expression is: in, For the removed Gaussian points j The set of neighboring points involved in this rendering. For pixels p The color of the Gaussian point was removed j Opaque gradient term, For pixels p Gaussian points that contribute to color rendering are culled. j The amount of change in opacity For pixels p Gaussian points that contribute to color rendering i The amount of change in opacity For pixels p Color at Gaussian point i Opaque gradient term.
6. The method according to claim 5, characterized in that, The compact form of the objective function is: in, Let represent the change in opacity of Gaussian points in the neighborhood point set. and These are the concatenation forms of the gradient terms corresponding to the neighborhood point set and the Gaussian point to be removed, respectively.
7. The method according to claim 1, characterized in that, Based on the gradient term set, construct the association matrix between each Gaussian point to be removed and the corresponding Gaussian points in its neighborhood set, including: Based on the set of gradient terms, the correlation matrix between each Gaussian point to be removed and the corresponding Gaussian points in its neighborhood set is constructed as follows: in, Indicates the first i A Gaussian point at a pixel p Transmittance at that location Indicates the first i A Gaussian point at pixel Projection weights at that location Indicates the first j A Gaussian point at a pixel p Transmittance at that location Indicates the first j A Gaussian point at pixel Projection weights at that location Indicates the first i The color characteristics of a Gaussian point Indicates the first j The color characteristics of a Gaussian point.
8. The method according to claim 1, characterized in that, The information allocation coefficients of each neighborhood Gaussian point are calculated based on the correlation matrix and information allocation factor, including: Based on the correlation matrix and information allocation factor, the information allocation coefficient of each neighborhood Gaussian point is calculated as follows: in, Indicates information allocation factor, Represents the correlation matrix. This represents the inverse of the incidence matrix.
9. The method according to claim 1, characterized in that, Based on the information allocation coefficient set, the opacity and color information of each Gaussian point to be removed are compensated to the corresponding Gaussian points in the neighborhood Gaussian point set. The opacity and color attribute parameters of the neighborhood Gaussian points are then updated to obtain the updated Gaussian point attribute parameter set, including: Based on the information allocation coefficient set, the opacity and color information of each Gaussian point to be removed are compensated to the corresponding Gaussian points in the neighborhood Gaussian point set. The opacity and color attribute parameters of the neighborhood Gaussian points are then updated, resulting in the following updated Gaussian point attribute parameters: in, , These are the Gaussian points in the neighborhood before the update. i Opacity and color characteristics, , These are the Gaussian points to be removed. j Opacity and color characteristics, For neighborhood Gaussian points i Information allocation coefficient, , For neighborhood Gaussian points i The updated opacity and color characteristics.
10. The method according to claim 1, characterized in that, The associated parameters for modifying the Gaussian point opacity based on the adjusted Gaussian point masking rate include: Gaussian point opacity Associated parameters The correspondence is as follows: Based on Gaussian point shielding rate Modify associated parameters The process is as follows: in, The original association parameters. This is the associated parameter for the modified Gaussian point opacity.