Multi-scale three-dimensional Gaussian splashing method and system
By adopting a local scale adaptive mechanism, the optimization inconsistency problem of the 3D Gaussian splashing method under multi-scale observation conditions is solved, and stable position updates and detail representation of Gaussian elements are achieved, thereby improving the stability and detail fidelity of 3D reconstruction.
Patent Information
- Application Number
- CN202611605822.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-16
- Publication Date
- 2026-11-17
AI Technical Summary
Existing 3D Gaussian splashing methods struggle to balance optimization stability and convergence efficiency across different regions under multi-scale observation conditions. Unifying global scale parameter settings leads to divergent or slow convergence of Gaussian element position updates, making it difficult to adapt to significant spatial sampling scale differences.
A visibility-guided local scale adaptation mechanism is adopted. By constructing a coarse prior representation of the scene, analyzing the color contribution weight of Gaussian elements, selecting the optimal observation viewpoint, calculating the local scale factor, and performing clustering, the learning rate of the position parameters of Gaussian elements and the adaptive densification splitting threshold are adjusted.
It improves the stability and detail fidelity of 3D reconstruction, adapts to the optimization parameters of different regions, avoids positional divergence in small-scale regions and redundancy in large-scale regions, and improves the reconstruction quality of complex scenes.
Smart Images

Figure CN123118930A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer graphics, and in particular relates to a Gaussian splashing method suitable for multi-scale image observation. Background Technology
[0002] 3D Gaussian Splatting (3DGS), as an explicit 3D representation method, has attracted widespread attention due to its efficient point-based modeling capabilities and real-time rendering characteristics for 3D scenes. This method uses a set of parametric Gaussian primitives (including position, scale, rotation, and appearance attributes) to continuously represent a 3D scene, and combines this with a differentiable rendering mechanism to achieve end-to-end optimization. While maintaining high training efficiency, it can achieve rendering quality and detail representation capabilities highly consistent with real images.
[0003] Standard 3DGS methods typically use a uniform global scale parameter to control Gaussian hyperparameters during optimization. This parameter simultaneously affects the learning rate for position updates and the splitting threshold during adaptive densification. This design implicitly assumes that the input multi-view images have a relatively consistent spatial sampling scale or imaging resolution. Although some studies have attempted to introduce scale adaptation mechanisms at the rendering level, the optimizer level (learning rate of the position parameter and splitting threshold for adaptive densification) still uses a globally uniform setting, which is difficult to adapt to the differentiated optimization needs of different regions under multi-scale observation. When there are significant scale differences in the observation scene, such as spatial resolution or ground sampling scale differences spanning orders of magnitude between images from different viewpoints, under the condition of observation across orders of magnitude, if the scale parameter is set large, the position updates of Gaussian hyperparameters in small-scale regions are prone to divergence; if the scale parameter is set small, the convergence of Gaussian hyperparameters in large-scale regions is slow and prone to redundancy. A uniform global hyperparameter setting is difficult to balance optimization stability and convergence efficiency at different scales.
[0004] Therefore, under multi-scale observation conditions, how to achieve local scale adaptive modeling of the Gaussian meta-optimization process and coordinate the learning rate adjustment and adaptive densification mechanism is the key to improving the reconstruction performance of 3D Gaussian splashing in complex scenes. Summary of the Invention
[0005] This invention provides a multi-scale three-dimensional Gaussian splashing method, which aims to solve the problem of inconsistent Gaussian element optimization caused by unifying global scale parameters when there are significant differences in spatial sampling scale in multi-view observations.
[0006] To achieve the above objectives, this invention proposes a visibility-guided local scale adaptation mechanism. This mechanism constructs a coarse prior representation of the scene and analyzes its effective visibility using the color contribution weights of Gaussian elements, thereby accurately matching the optimal observation scale for Gaussian elements in different regions. Based on this, through scale factor clustering and grouping, the position update step size and densification splitting threshold of Gaussian elements in corresponding regions are adaptively adjusted, thereby suppressing position divergence in small-scale regions while avoiding redundant elements in large-scale regions, significantly improving the reconstruction stability and detail fidelity of 3D Gaussian splashes in complex multi-scale scenes.
[0007] According to one aspect of the present invention, a multi-scale 3D Gaussian splashing method is provided, comprising: acquiring multi-view images of a target scene, calculating camera parameters and sparse point clouds of the scene corresponding to each view, and initializing a 3D Gaussian model using the sparse point clouds, the 3D Gaussian model containing multiple Gaussian primitives; constructing a coarse prior representation of the scene based on the multi-view images, freezing the position parameters and adaptive densification operation of the Gaussian primitives during training, optimizing the scale parameters, rotation parameters, and appearance parameters of the Gaussian primitives, and recording the color contribution weight of each Gaussian primitive under different views; constructing a set of reliable observation views corresponding to each Gaussian primitive based on the color contribution weight; selecting an optimal observation view for each Gaussian primitive based on the set of reliable observation views, and calculating the corresponding local scale factor based on the optimal observation view; clustering the local scale factors corresponding to all Gaussian primitives to obtain multiple scale groups and a unified local scale factor corresponding to each scale group; adjusting the learning rate of the position parameters and the adaptive densification splitting threshold of the Gaussian primitives according to the unified local scale factor; and iteratively optimizing the 3D Gaussian model based on the adjusted learning rate of the position parameters and the adaptive densification splitting threshold to obtain a 3D reconstruction result of the target scene.
[0008] As a further technical solution, a reliable observation view set is constructed based on the color contribution weight, including: calculating the effective visibility metric of each Gaussian pixel under each view based on the color contribution weight, filtering observation views that meet the conditions through a preset visibility threshold, and forming the reliable observation view set; the effective visibility metric is the maximum transparency mixing weight of the Gaussian pixel among all affected pixels under the corresponding view.
[0009] As a further technical solution, the optimal observation angle is selected for each Gaussian element, including: selecting the observation angle with the largest physical resolution from the set of reliable observation angles as the optimal observation angle, and the local scale factor is calculated based on the focal length of the optimal observation angle and the depth of the Gaussian element.
[0010] As a further technical solution, clustering is performed on the local scale factors corresponding to all Gaussian units, including: clustering according to the order of magnitude of the local scale factors, grouping local scale factors with differences of less than one order of magnitude into the same scale group, and using the mean within the group as the unified local scale factor of the scale group.
[0011] As a further technical solution, the learning rate of the position parameters of Gaussian elements is adjusted, including multiplying the original global learning rate by a unified local scale factor so that Gaussian elements in different scale regions can obtain a position optimization step size that is adapted to their local spatial sampling scale.
[0012] As a further technical solution, the adaptive densification splitting threshold is adjusted, including: setting the splitting threshold in the adaptive densification process to be proportional to the unified local scale factor, so that Gaussian units in different scale regions adopt different densification splitting triggering conditions; the Gaussian units generated by new splitting or cloning inherit the unified local scale factor of the scale group to which their parent unit belongs.
[0013] According to one aspect of the present invention, a multi-scale three-dimensional Gaussian splashing system is provided, comprising: an initialization module, configured to acquire multi-view images of a target scene, calculate camera parameters and sparse point clouds of the scene corresponding to each view, and initialize a three-dimensional Gaussian model using the sparse point clouds; a coarse prior construction module, configured to construct a coarse prior representation of the scene based on the multi-view images, freeze the position parameters and adaptive densification operation of Gaussian elements during training, optimize the scale parameters, rotation parameters, and appearance parameters of the Gaussian elements, and record the color contribution weight of each Gaussian element under different viewpoints; and a visibility analysis module, configured to construct a reliable representation corresponding to each Gaussian element based on the color contribution weight. The system comprises: a set of observation viewpoints; a scale calculation module, used to select the optimal observation viewpoint for each Gaussian element based on the set of reliable observation viewpoints, and to calculate the corresponding local scale factor based on the optimal observation viewpoint; a scale clustering module, used to cluster the local scale factors corresponding to all Gaussian elements to obtain multiple scale groups and a unified local scale factor corresponding to each scale group; and an optimization and control module, used to control the learning rate of the position parameters and the adaptive densification splitting threshold of the Gaussian elements based on the unified local scale factor, and to iteratively optimize the three-dimensional Gaussian model based on the controlled learning rate of the position parameters and the adaptive densification splitting threshold to obtain the three-dimensional reconstruction result of the target scene.
[0014] As a further technical solution, the optimization and control module includes a learning rate control unit and a split threshold control unit. The learning rate control unit is used to multiply the original global learning rate by a unified local scale factor, and the split threshold control unit is used to set the split threshold in the adaptive densification process to be proportional to the unified local scale factor.
[0015] According to one aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the multi-scale three-dimensional Gaussian splashing method.
[0016] According to one aspect of the present invention, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the multi-scale three-dimensional Gaussian splashing method.
[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention constructs a local scale adaptive mechanism to match optimized parameters that are adapted to the observation scale of Gaussian elements in different scale regions, thereby achieving adaptive adjustment of the position update step size and densification splitting threshold of Gaussian elements in different scale regions, thus improving the stability and detail representation capability of 3D reconstruction in complex scenes.
[0018] 2. For the cracked areas on the dam surface, a smaller unified local scale factor allows the learning rate to be adapted for fine-tuning of local details; for the reservoir topography, a larger unified local scale factor allows the learning rate to be adapted for rapid convergence of large-scale morphology. For the near-view observation area, a lower unified local scale factor makes primitives more likely to trigger splitting, allowing for detail enhancement using smaller Gaussian primitives; for the far-view global region, a higher unified local scale factor suppresses excessive splitting, maintaining computational efficiency. This adaptive mechanism ensures that different regions obtain optimized parameters that match their observation scale.
[0019] 3. This invention clusters local scale factors according to their order of magnitude, grouping local scale factors with differences of less than one order of magnitude into the same scale group, and using the mean within the group as the unified local scale factor for that scale group. This significantly reduces memory consumption and improves the consistency and smoothness of optimization within local regions, avoiding uncoordinated updates of adjacent primitives due to small differences in scale factors. Attached Figure Description
[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 A flowchart illustrating the multi-scale three-dimensional Gaussian splashing method provided in an embodiment of the present invention; Figure 2 This is a flowchart of multi-scale image data acquisition provided in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the effective observation angle selection provided in an embodiment of the present invention. Figure 4 This is a schematic diagram of local scale factor clustering provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the overall three-dimensional reconstruction result when the present invention is applied to a hydraulic structure scenario; Figure 6 This is a schematic diagram comparing the rendering results from different perspectives with real images, provided in an embodiment of the present invention. Detailed Implementation
[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined to form new technical solutions. Such combinations are not bound by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.
[0023] In one embodiment, the multi-scale 3D Gaussian splashing method of the present invention can be applied to multi-scale image-driven 3D reconstruction tasks. Taking a reservoir dam operation and maintenance project as an example, its input data includes a set of multi-view images with different spatial resolutions or different ground sampling scales. Figure 1 As shown, the specific steps include: Step (1): Image Acquisition. This step acquires a multi-view image set of the target scene. The multi-view image set includes image data with different spatial resolutions or different ground sampling scales, and the image data exhibits significant differences in spatial sampling scale. The images are acquired from different shooting distances, different focal lengths, different sensors, or different acquisition platforms.
[0024] In this embodiment, to meet the inspection needs of concrete dam operation and maintenance, a DJI Matrice 350 RTK drone equipped with a Zenmuse H20 dual-mode camera is used for image acquisition. Figure 2 As shown, the collected content includes: a. Long-distance global aerial photography: Flying at an altitude of approximately 300 meters, using a wide-angle module (focal length 4.5mm, resolution 4056×3040), the overall outline of the dam and the topography of the reservoir area were captured, and 324 images were collected; b. Mid-range rapid inspection: At a distance of approximately 35 meters from the dam surface, using a wide-angle module, inspect areas that may be damaged and collect 319 images; c. Close-up detailed inspection: Flying at a distance of about 6 meters close to the dam surface, using a zoom module (focal length 87mm, resolution 5184×3888), detailed images were taken of the damaged areas to obtain millimeter-level details such as cracks, peeling, and leakage traces, collecting 1147 images.
[0025] Step (2): Preprocessing and sparse reconstruction. This step estimates the camera's intrinsic and extrinsic parameters based on the structure-reconstruction-motion algorithm and reconstructs the sparse point cloud. The sparse point cloud is then used to initialize a set of three-dimensional Gaussian elements, each parameterized by position, scale, rotation, opacity, and color.
[0026] This embodiment performs camera intrinsic parameter calibration and distortion correction on the acquired images. The COLMAP software is used to execute the structure-on-motion algorithm to estimate the intrinsic and extrinsic parameters of 1790 camera views and reconstruct the sparse point cloud of the scene. To balance computational efficiency and reconstruction accuracy, the image width is scaled to 1000 pixels while maintaining the aspect ratio.
[0027] Step (3): Construction of coarse prior scene representation. This step takes the multi-view image set as input and uses the 3D Gaussian splashing algorithm for forward rendering training. During the training process, position parameter updates and adaptive densification operations are disabled, only appearance-related parameters are optimized, and the color contribution weight of each Gaussian element under different views is recorded.
[0028] In this embodiment, a 3D Gaussian set is initialized based on sparse point clouds. Each Gaussian unit By location ,scale Rotation Quaternions Opacity Colors represented by spherical harmonic coefficients Parameterization. Using all 1790 images as input, a standard 3DGS algorithm is employed for forward rendering training. The core objective of this stage is to obtain the visibility information of each Gaussian primitive from various viewpoints; therefore, position parameter updates and adaptive point encryption are disabled, and only scale, rotation, color, and opacity parameters are optimized. After training, each Gaussian primitive is recorded according to the opacity blending formula used in the rendering process. From various perspectives Color contribution weight ,in, Gausky element The effective opacity when projected onto pixel p from the corresponding viewing angle; This represents the observation ray along pixel p in Gaussian units. The cumulative transmittance formed by the previous Gaussian elements.
[0029] Step (4): Effective visibility metric calculation and candidate view selection. This step calculates the effective visibility metric for each Gaussian element based on the color contribution weights. By introducing a visibility threshold Valid observation perspectives are selected, and a set of reliable observation perspectives for each Gaussian element is constructed, such as... Figure 3 As shown.
[0030] Gao Siyuan From the perspective The effective visibility metric is defined as the maximum color contribution weight of the primitive across all affected pixels at that viewpoint: , in From the perspective Lower The set of all affected pixels. A visibility threshold is introduced. Filter out those that meet the requirements From the perspective of constructing a reliable candidate set of visible viewpoints: This candidate set excludes unstable viewpoints that are occluded or too far away, resulting in weak contributions, thus ensuring the reliability of subsequent local scale calculations.
[0031] Step (5): Optimal Viewpoint Selection and Local Scale Factor Calculation. In this step, the optimal viewpoint with the highest physical resolution is selected from the set of reliable observation viewpoints, and the local scale factor of each Gaussian element is calculated based on the Gaussian element depth and focal length corresponding to this optimal viewpoint. The local scale factor is used to characterize the local spatial sampling scale corresponding to the Gaussian element.
[0032] In the candidate set In the middle, select the one that makes the physical resolution Maximizing the perspective .in, This represents the focal length of the camera corresponding to the k-th observation viewpoint. Gausky element The center of the depth corresponds to the camera coordinate system at the k-th observation viewpoint. This viewpoint generates the largest position gradient during backpropagation, dominating the position gradient. The optimization process. Local scale factor. Defined as the reciprocal of the physical resolution of this optimal viewpoint, and a constant K is introduced for numerical compensation: , in This is used to compensate for the order-of-magnitude shift introduced by the focal length. For high-resolution images of the dam crest region, the optimal viewing angle is typically a long-range aerial photograph. Larger The value is relatively large; for high-resolution elements in the cracked area of the dam surface, the optimal viewing angle is usually a close-up inspection. Smaller The values are relatively small. This adaptive mechanism ensures that different regions obtain optimized parameters that match their observation scale.
[0033] Step (6): Generation of local unified scale factor. This step performs order-of-magnitude clustering on the local scale factors of all Gaussian units, groups local scale factors with differences of less than one order of magnitude into the same scale group, and uses the mean of the group as the unified local scale factor of the scale group.
[0034] Hydraulic structure reconstruction typically involves tens of millions of Gaussian units; if each is stored independently... This will result in huge memory overhead. Therefore, a scale-based clustering strategy is adopted. Figure 4 ): (The difference is less than an order of magnitude) Grouping them into the same scale group and using the within-group mean As a unified local scale factor for this group, S scale groups are ultimately obtained. This clustering strategy not only significantly reduces memory consumption but also improves the consistency and smoothness of optimization within local regions, avoiding inconsistent updates between adjacent primitives due to small differences in scale factors.
[0035] Step (7): Scale-adaptive optimization reconstruction. This step utilizes the unified local scale factor to adjust the learning rate of the Gaussian unit's position parameters and the splitting threshold of adaptive densification, so that Gaussian units in different spatial information density regions can obtain optimized step sizes that adapt to their local scales, thereby achieving stable reconstruction of multi-scale 3D scenes.
[0036] In this embodiment, a unified local scale factor is used. Instead of the original global scale factor camera_extent, the following two core optimization mechanisms are controlled respectively: (a) Location parameter learning rate: location parameters learning rate multiplied by For the cracked areas on the dam surface, The learning rate is relatively small, allowing for fine-tuning of local details; for reservoir terrain, The learning rate is relatively large, allowing for rapid convergence of large-scale patterns.
[0037] (b) Adaptive densification: Setting the size threshold for Gaussian cell splitting and Proportional. For close-up viewing areas, the lower This makes it easier for primitives to trigger splits, allowing for detail enhancement using smaller Gaussian primitives; for the global region with a long field of view, higher... To suppress excessive splitting and maintain computational efficiency, new Gaussian primitives generated by cloning or splitting inherit the local scale factor of their parent primitives, ensuring scale consistency throughout the optimization process. Figure 5 This demonstrates the overall 3D reconstruction results when the present invention is applied to a hydraulic structure scenario. Figure 6 The paper further compares the rendering results with the real images from different perspectives. It shows that, from a long-distance global observation perspective, both the original Gaussian splashing method and the method of this invention can recover the overall structure of the dam. However, as the observation distance decreases, the original 3D Gaussian splashing method exhibits varying degrees of texture blurring and loss of detail in local detail areas such as the dam surface structure and cracks. In contrast, the method of this invention maintains a rendering result consistent with the reference image. This demonstrates that the present invention can balance global structural representation and local detail reconstruction under multi-view observation conditions with significant spatial sampling scale differences, improving the stability and detail fidelity of 3D reconstruction across scale regions.
[0038] In one embodiment, the present invention also provides a multi-scale three-dimensional Gaussian splashing system, including an initialization module, a coarse prior construction module, a visibility analysis module, a scale calculation module, a scale clustering module, and an optimization and control module.
[0039] The initialization module is used to acquire multi-view images of the target scene, calculate the camera parameters and sparse point cloud corresponding to each view, and initialize a 3D Gaussian model using the sparse point cloud. Taking a reservoir dam operation and maintenance project as an example, the initialization module acquires 1790 multi-view images, estimates the intrinsic and extrinsic parameters of each view using COLMAP software, reconstructs the sparse point cloud, and initializes a 3D Gaussian element set using the sparse point cloud. Each Gaussian element is parameterized by position, scale, rotation, opacity, and spherical harmonic coefficients.
[0040] The coarse prior construction module is used to construct a coarse prior representation of the scene based on the multi-view image. During the training process, the position parameters of Gaussian elements and the adaptive densification operation are frozen, and only the spherical harmonic coefficients and opacity parameters of Gaussian elements are optimized. The color contribution weight of each Gaussian element under different viewpoints is recorded.
[0041] The visibility analysis module is used to construct a set of reliable observation viewpoints for each Gaussian pixel based on the color contribution weight. Specifically, the visibility analysis module calculates the effective visibility metric for each Gaussian pixel at each viewpoint based on the transparency mixing formula. The effective visibility metric is the maximum transparency mixing weight of the Gaussian pixel among all affected pixels at the corresponding viewpoint. The module then filters observation viewpoints that meet the conditions through a preset visibility threshold to form a set of reliable observation viewpoints for that Gaussian pixel.
[0042] The scaling module is used to select the optimal observation angle for each Gaussian element based on the set of reliable observation angles, and to calculate the corresponding local scaling factor based on the optimal observation angle. Specifically, the scaling module selects the observation angle with the highest physical resolution from the set of reliable observation angles as the optimal observation angle, and the local scaling factor is calculated based on the focal length of the optimal observation angle and the depth of the Gaussian element.
[0043] The scale clustering module is used to cluster the local scale factors corresponding to all Gaussian units, obtaining multiple scale groups and the unified local scale factor corresponding to each scale group. Specifically, the scale clustering module clusters according to the order of magnitude of the local scale factors, grouping local scale factors with differences of less than one order of magnitude into the same scale group, and using the mean within the group as the unified local scale factor for that scale group.
[0044] The optimization and control module is used to adjust the learning rate of the Gaussian unit's position parameters and the adaptive densification splitting threshold according to the unified local scale factor, and to iteratively optimize the three-dimensional Gaussian model based on the adjusted learning rate of the position parameters and the adaptive densification splitting threshold to obtain the three-dimensional reconstruction result of the target scene.
[0045] Furthermore, the optimization and control module includes a learning rate control unit and a splitting threshold control unit. The learning rate control unit multiplies the original global learning rate by the unified local scale factor, enabling Gaussian primitives in different scale regions to obtain position optimization step sizes adapted to their local spatial sampling scales. For dam surface crack regions, a smaller unified local scale factor allows for fine-tuning of local details; for reservoir topography, a larger unified local scale factor allows for rapid convergence of large-scale morphologies. The splitting threshold control unit sets the splitting threshold during the adaptive densification process to be proportional to the unified local scale factor. For near-view observation regions, a lower unified local scale factor makes it easier for primitives to trigger splits, achieving detail enhancement with smaller Gaussian primitives; for far-view global regions, a higher unified local scale factor suppresses excessive splitting, maintaining computational efficiency. Newly split or cloned Gaussian primitives inherit the unified local scale factor of their parent primitive's scale group.
[0046] In one embodiment, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described multi-scale three-dimensional Gaussian splashing method.
[0047] The electronic device can be a server, workstation, personal computer, embedded device, or dedicated 3D reconstruction equipment. Taking the operation and maintenance scenario of a reservoir dam as an example, the electronic device can be a portable workstation deployed on the engineering site or a cloud server, used to receive multi-view images collected by drones, execute a multi-scale 3D Gaussian splashing method, and generate 3D reconstruction results of the dam structure for subsequent structural safety monitoring and defect analysis.
[0048] In one embodiment, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described multi-scale three-dimensional Gaussian splashing method.
[0049] The storage medium can be a read-only memory (ROM), random access memory (RAM), USB flash drive, portable hard drive, magnetic disk, optical disk, or other media capable of storing program code. When the computer program is executed by the processor, it implements a method including the following steps: acquiring multi-view images of the target scene and initializing a 3D Gaussian model; constructing a coarse prior representation of the scene and recording color contribution weights; constructing a reliable set of observation viewpoints based on the color contribution weights; selecting the optimal observation viewpoint and calculating the local scale factor; clustering the local scale factors to obtain scale groups and a unified local scale factor; adjusting the learning rate of the position parameters and the adaptive densification splitting threshold according to the unified local scale factor; and iteratively optimizing the 3D Gaussian model based on the adjusted parameters to obtain the 3D reconstruction result.
[0050] In summary, this invention discloses a multi-scale 3D Gaussian splashing method. Addressing the optimization inconsistency problem caused by significant spatial sampling scale differences in multi-view images, a local scale adaptive mechanism is proposed: a reliable set of observation views for Gaussian elements is constructed based on coarse prior visibility analysis. On this basis, the view with the highest physical resolution dominating optimization is selected to calculate the local scale factor, and stable scale representation is obtained through order-of-magnitude clustering. This stable scale representation is then used to uniformly control the position update step size and adaptive densification splitting threshold of Gaussian elements, thereby achieving collaborative optimization and consistent reconstruction of regions with different spatial information density. This invention can improve the convergence stability and detail representation capability of the 3D Gaussian splashing model under multi-scale observation conditions.
[0051] The terms “comprising” and “having”, and any variations thereof, in the specification, claims, and drawings of this invention are intended to cover a non-exclusive inclusion, such as a process, method, system, product, or apparatus that includes a series of steps or units, not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.
Claims
1. A multi-scale three-dimensional Gaussian splashing method, characterized in that, include: Acquire multi-view images of the target scene, calculate the camera parameters and sparse point cloud of the scene corresponding to each view, and initialize a three-dimensional Gaussian model using the sparse point cloud. The three-dimensional Gaussian model contains multiple Gaussian elements. Based on the multi-view images, a coarse prior representation of the scene is constructed. During the training process, the position parameters and adaptive densification operation of Gaussian elements are frozen, the scale parameters, rotation parameters and appearance parameters of Gaussian elements are optimized, and the color contribution weight of each Gaussian element under different viewpoints is recorded. Based on the color contribution weights, construct a set of reliable observation perspectives corresponding to each Gaussian element; Based on the set of reliable observation perspectives, the optimal observation perspective is selected for each Gaussian element, and the corresponding local scale factor is calculated based on the optimal observation perspective. Cluster the local scale factors corresponding to all Gaussian units to obtain multiple scale groups and the unified local scale factor corresponding to each scale group. Based on the unified local scale factor, the learning rate of the position parameters of the Gaussian unit and the adaptive dense splitting threshold are adjusted respectively; Based on the adjusted learning rate of the position parameters and the adaptive dense splitting threshold, the three-dimensional Gaussian model is iteratively optimized to obtain the three-dimensional reconstruction result of the target scene.
2. The multi-scale three-dimensional Gaussian splashing method according to claim 1, characterized in that, Constructing a reliable set of observation viewpoints based on the color contribution weights includes: calculating the effective visibility metric of each Gaussian pixel under each viewpoint based on the color contribution weights, filtering observation viewpoints that meet the conditions through a preset visibility threshold, and forming the reliable set of observation viewpoints; the effective visibility metric is the maximum transparency mixing weight of the Gaussian pixel among all affected pixels under the corresponding viewpoint.
3. The multi-scale three-dimensional Gaussian splashing method according to claim 1, characterized in that, Selecting the optimal observation angle for each Gaussian element includes: selecting the observation angle with the highest physical resolution from the set of reliable observation angles as the optimal observation angle, wherein the local scale factor is calculated based on the focal length of the optimal observation angle and the depth of the Gaussian element.
4. The multi-scale three-dimensional Gaussian splashing method according to claim 1, characterized in that, Clustering is performed on the local scale factors corresponding to all Gaussian units, including: clustering according to the order of magnitude of the local scale factors, grouping local scale factors with differences of less than one order of magnitude into the same scale group, and using the mean of the group as the unified local scale factor of the scale group.
5. The multi-scale three-dimensional Gaussian splashing method according to claim 1, characterized in that, The learning rate of the position parameters of Gaussian elements is adjusted by multiplying the original global learning rate by a uniform local scale factor, so that Gaussian elements in different scale regions can obtain a position optimization step size that is adapted to their local spatial sampling scale.
6. The multi-scale three-dimensional Gaussian splashing method according to claim 1, characterized in that, The adaptive densification splitting threshold is adjusted by: setting the splitting threshold in the adaptive densification process to be proportional to the unified local scale factor, so that Gaussian units in different scale regions adopt different densification splitting triggering conditions; and the Gaussian units generated by new splitting or cloning inherit the unified local scale factor of the scale group to which their parent unit belongs.
7. A multi-scale three-dimensional Gaussian splashing system, characterized in that, include: An initialization module is used to acquire multi-view images of the target scene, calculate the camera parameters and sparse point cloud of the scene corresponding to each view, and initialize a three-dimensional Gaussian model using the sparse point cloud. The coarse prior construction module is used to construct a coarse prior representation of the scene based on the multi-view image. During the training process, the position parameters and adaptive densification operation of Gaussian elements are frozen, the scale parameters, rotation parameters and appearance parameters of Gaussian elements are optimized, and the color contribution weight of each Gaussian element under different views is recorded. The visibility analysis module is used to construct a set of reliable observation viewpoints corresponding to each Gaussian element based on the color contribution weight; The scale calculation module is used to select the optimal observation view for each Gaussian element based on the set of reliable observation views, and to calculate the corresponding local scale factor based on the optimal observation view. The scale clustering module is used to cluster the local scale factors corresponding to all Gaussian units to obtain multiple scale groups and the unified local scale factor corresponding to each scale group. The optimization and control module is used to adjust the learning rate of the Gaussian unit's position parameters and the adaptive densification splitting threshold according to the unified local scale factor, and to iteratively optimize the three-dimensional Gaussian model based on the adjusted learning rate of the position parameters and the adaptive densification splitting threshold to obtain the three-dimensional reconstruction result of the target scene.
8. The multi-scale three-dimensional Gaussian splashing system according to claim 7, characterized in that, The optimization and control module includes a learning rate control unit and a split threshold control unit. The learning rate control unit is used to multiply the original global learning rate by a unified local scale factor, and the split threshold control unit is used to set the split threshold in the adaptive densification process to be proportional to the unified local scale factor.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the multi-scale three-dimensional Gaussian splashing method as described in any one of claims 1 to 6.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the multi-scale three-dimensional Gaussian splashing method as described in any one of claims 1 to 6.
