Low-computing-power fast three-dimensional modeling method and system based on 3DGS

Abstract

The present application relates to the technical field of three-dimensional modeling, and more particularly to a low-computing-power fast three-dimensional modeling method and system based on 3DGS, which proposes the following scheme: in each modeling cycle, the Gaussian field of the previous cycle is updated as the starting model to perform forward rendering, and the rendering image is obtained and residual information is calculated. The residual is robustly aggregated as a statistical unit according to a preset tile, and a projection statistical spectrum is constructed in combination with the Gaussian projection contribution. The structure change degree is obtained by cross-cycle comparison, and the complexity score is formed by residual gating to divide the complex region and the flat region. According to the density scheduling state, the tile-to-Gaussian object mapping is established, the complex region is directionally split and encrypted, the intermediate region is limited to fine-tune the appearance and transparency, and the flat region is absorbed and merged to be sparse, so that fast convergence and stable modeling are realized under the condition of limited computing power, while the proportion of invalid calculation is significantly reduced while maintaining the reconstruction accuracy and structural stability.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling technology, and in particular to a low-computing-power, fast 3D modeling method and system based on 3DGS. Background Technology

[0002] Fast reconstruction methods based on 3D Gaussian representation have been widely used in continuous viewpoint acquisition and online modeling scenarios. They achieve efficient fitting of complex geometry and textures by arranging a large number of Gaussian objects in space and combining viewpoint-related color representations. However, in practical engineering deployments, especially on mobile terminals or embedded low-computing-power devices, existing technologies typically employ a globally uniform update strategy. This means that all Gaussian objects are optimized or their density adjusted in each modeling cycle without differentiating the degree of structural change in different image regions. This approach easily leads to two problems in continuous modeling: firstly, flat or stable regions are repeatedly subjected to high-cost computations, resulting in wasted computing power and redundant growth in the number of objects; secondly, regions with rapidly changing local structures or frequently rearranged occlusion relationships lack targeted density scheduling, causing residuals to be concentrated on a small number of Gaussian objects, resulting in insufficient detail representation or decreased convergence speed. Furthermore, existing methods often rely on a single residual threshold or object-level statistics when splitting or merging objects, lacking a mechanism for comparing the degree of structural change across modeling cycles, making it difficult to achieve adaptive density allocation while maintaining model stability. Therefore, in low-computing-power continuous modeling scenarios, how to achieve differentiated updates for different regions, suppress invalid computations, and improve model convergence efficiency while ensuring expression accuracy has become an urgent technical problem to be solved.

[0003] To address the above issues, this application presents a low-computing-power, rapid 3D modeling method and system based on 3DGS. Summary of the Invention

[0004] The technical problem this invention aims to solve is to address the shortcomings of existing technologies by providing a low-computing-power, fast 3D modeling method and system based on 3DGS. In each modeling cycle, the Gaussian field updated in the previous cycle is used as the starting model for forward rendering to obtain the rendered image and calculate residual information. The residuals are robustly aggregated using preset tiles as statistical units, and a projection statistical spectrum is constructed by combining Gaussian projection contributions. Cross-cycle comparisons yield the degree of structural change, and residual gating generates a complexity score to classify complex and flat regions. A tile-to-Gaussian object mapping is established based on density scheduling states. Complex regions are directionally split and densified, intermediate regions undergo restricted fine-tuning of appearance and transparency, and flat regions are absorbed, merged, and sparsified, thereby achieving rapid convergence and stable modeling under computationally limited conditions.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A low-computing-power, fast 3D modeling method based on 3DGS is applied to a graphics processing unit (GPU). The GPU is configured with a current Gaussian field model, which is the updated Gaussian field model from the previous modeling cycle. Each modeling cycle corresponds to image data from one viewpoint of the object to be modeled. The method includes:

[0007] Obtain the image data of the object to be modeled in the current modeling cycle;

[0008] Based on the image data and the current Gaussian field model, forward rendering is performed to obtain a rendered image, and residual information is determined based on the rendered image and the image data;

[0009] Complexity information is determined based on the residual information and the Gaussian object set corresponding to the current Gaussian field model. The complexity information is used to characterize the local geometric detail complexity of the object to be modeled in the current modeling cycle and to divide the complex region into a flat region. Each Gaussian object in the Gaussian object set is associated with at least spatial position parameters, shape parameters, transparency parameters, and color representation parameters.

[0010] Based on the complexity information, an adaptive Gaussian density allocation is performed on the current Gaussian field model to obtain an updated Gaussian field model for the current modeling cycle, and 3D modeling is performed based on the updated Gaussian field model.

[0011] If the current modeling cycle is the first modeling cycle, the construction process of the current Gaussian field model includes:

[0012] Acquire image data from the viewpoint corresponding to the first modeling cycle, and perform feature extraction based on the image data to obtain a set of feature points;

[0013] Based on the set of feature points, camera pose is solved to determine the camera parameters for the first modeling cycle, and an initial geometric reference is constructed based on the camera parameters;

[0014] An initial Gaussian object set is generated based on the initial geometric reference, wherein the spatial position parameters of each Gaussian object in the initial Gaussian object set are determined by the initial geometric reference, the shape parameters are determined by a preset covariance initialization strategy, the transparency parameters are determined by a preset initial opacity value, and the color representation parameters are determined by the pixel color of the image data at the corresponding projection position.

[0015] Perform forward rendering based on the image data and the current Gaussian field model, including:

[0016] In the current modeling cycle, obtain the camera parameters corresponding to the image data, and based on the camera parameters, project each Gaussian object in the Gaussian object set from three-dimensional space to the image plane to obtain the projection ellipse parameters of each Gaussian object on the image plane.

[0017] For the pixels on the image plane, the projected ellipses are sorted according to the depth information of each Gaussian object, and pixel-by-pixel alpha blending is performed based on the transparency parameter to generate the rendered image.

[0018] The determination of the residual information includes:

[0019] During the generation of the rendered image, the color contribution of each Gaussian object under the current viewpoint is determined based on the color representation parameters, wherein the color contribution includes at least spherical harmonic coefficients used to characterize the color change of the viewpoint;

[0020] After generating the rendered image, pixel-level residuals are calculated based on the rendered image and the image data, and the pixel-level residuals are aggregated using the color contribution to obtain the residual information.

[0021] Complexity information is determined based on the residual information and the set of Gaussian objects corresponding to the current Gaussian field model, including:

[0022] The residual information between the rendered image and the image data is aggregated according to a preset tile division rule to obtain the tile residual statistics of each tile, wherein the tile residual statistics include at least residual energy statistics and residual gradient statistics.

[0023] The Gaussian projection contributions of each tile are aggregated based on the Gaussian object set to obtain the projection statistical spectrum of each tile. The projection statistical spectrum includes at least the coverage weight statistic, the depth mixing dispersion statistic, and the anisotropic direction consistency statistic.

[0024] The projection statistical spectrum of each tile in the current modeling cycle is compared with the projection statistical spectrum of each tile in the previous modeling cycle to obtain the structural variability of each tile. Based on the residual statistics of the tile, the structural variability is weighted by residual gate to obtain the complexity score of each tile.

[0025] Based on the complexity score, complex regions and flat regions are divided. The complex region includes the region corresponding to the tile whose complexity score meets the first threshold condition, and the flat region includes the region corresponding to the tile whose complexity score meets the second threshold condition, wherein the first threshold is greater than the second threshold.

[0026] The residual information between the rendered image and the image data is aggregated according to a preset tile division rule, including:

[0027] Based on the residual information and the image data, a pixel residual map is calculated.

[0028] Based on the preset tile division rules, the pixel residual map is divided into multiple tile regions, wherein each tile region corresponds to a set of pixels of a preset size;

[0029] Within each tile region, pixel residuals are calculated for the pixel set, and robust aggregation is performed on the pixel residuals to obtain the residual energy statistics of the tile region. The robust aggregation includes at least one of mean aggregation, quantile aggregation, and truncated mean aggregation.

[0030] Within each tile region, a gradient operator is performed on the pixel residual to obtain a residual gradient map, and the residual gradient maps are aggregated to obtain a residual gradient statistic for the tile region. The residual gradient statistic is used to characterize the spatial drasticness of the residual change within the tile region.

[0031] The residual energy statistic and the residual gradient statistic are weighted to obtain the tile residual statistic.

[0032] The Gaussian projection contributions of each tile are aggregated based on the Gaussian object set to obtain the projection statistical spectrum of each tile, including:

[0033] Determine the target tile region for each Gaussian object;

[0034] Within each target tile region, for the Gaussian object belonging to the target tile region, the coverage weight statistic is accumulated based on the transparency parameter and the projection ellipse coverage weight;

[0035] Within each target tile region, based on the depth information and pixel-by-pixel alpha mixing order of the Gaussian object belonging to the target tile region, the effective mixing weight distribution of the Gaussian object in the preset depth layer is statistically analyzed, and the depth mixing dispersion statistic is calculated based on the effective mixing weight distribution.

[0036] Within each target tile region, extract the principal axis direction information of the projection ellipse of the Gaussian object belonging to the target tile region, and calculate the anisotropic direction consistency statistic based on the principal axis direction information;

[0037] The coverage weight statistic, the depth mixing dispersion statistic, and the anisotropic direction consistency statistic are combined to form the projection statistical spectrum.

[0038] Based on the complexity information, perform adaptive Gaussian density assignment on the current Gaussian field model, including:

[0039] Based on the complexity score, the density scheduling state of each tile region is determined, and the density scheduling state includes at least an encrypted state, a fine-tuning state, and a sparse state.

[0040] A mapping relationship between tiles and Gaussian objects is established based on the density scheduling state to determine the set of encrypted candidate Gaussian objects, the set of fine-tuned candidate Gaussian objects, and the set of sparse candidate Gaussian objects. The mapping relationship is determined at least based on the overlap between the projected ellipse of the Gaussian object and the tile region.

[0041] For the set of encrypted candidate Gaussian objects, a directional splitting operation is performed to generate at least one child Gaussian object. The directional splitting operation includes: determining the splitting direction based on the principal direction of the residual gradient and the principal axis direction of the projection ellipse, and perturbing the spatial position parameter and the shape parameter in the splitting direction to obtain the initialization parameters of the child Gaussian object, while making the child Gaussian object inherit the prior values ​​of the color representation parameter and the transparency parameter.

[0042] For the set of fine-tuning candidate Gaussian objects, perform parameter-constrained updates, wherein the parameter-constrained updates include at least: updating the color representation parameter and the transparency parameter, while keeping the spatial position parameter and the shape parameter unchanged;

[0043] For the sparse candidate Gaussian object set, an absorption and merging operation is performed, wherein the absorption and merging operation includes: determining a cluster of Gaussian objects that satisfy similarity constraints within the same tile region, the similarity constraints including at least color similarity constraints, principal axis direction similarity constraints, and depth consistency constraints, and merging the Gaussian object clusters into a merged Gaussian object, wherein the spatial position parameters, shape parameters, transparency parameters, and color representation parameters of the merged Gaussian object are obtained by weighting the parameters of the Gaussian object cluster according to the effective mixing weights.

[0044] The encrypted state corresponds to a tile region whose complexity score meets the first threshold condition; the sparse state corresponds to a tile region whose complexity score meets the second threshold condition; and the fine-tuned state corresponds to a tile region whose complexity score is between the first threshold and the second threshold.

[0045] A low-computing-power, rapid 3D modeling system based on 3DGS, the system comprising:

[0046] The periodic scheduling module is used to acquire image data of the object to be modeled from the corresponding viewpoint of each modeling cycle, and maintain the current Gaussian field model on the graphics processing unit. The current Gaussian field model is the updated Gaussian field model of the previous modeling cycle.

[0047] The complexity evaluation module performs forward rendering based on the image data and the current Gaussian field model to obtain a rendered image, determines residual information based on the rendered image and the image data, and determines complexity information based on the residual information and the Gaussian object set corresponding to the current Gaussian field model, so as to divide complex regions and flat regions.

[0048] The modeling output module is used to perform adaptive Gaussian density allocation on the current Gaussian field model according to the complexity information to obtain the updated Gaussian field model for the current modeling cycle, and to complete the 3D modeling output based on the updated Gaussian field model.

[0049] Compared with the prior art, the beneficial effects of the present invention are:

[0050] This invention uses the image plane as a unified statistical coordinate system to perform cross-period comparisons of local structural changes and combines residual gating to form a complexity score, thereby achieving differentiated density scheduling for Gaussian objects. Compared with existing globally unified update methods, this scheme can perform targeted densification in complex regions to enhance detail representation, implement restricted parameter updates in intermediate regions to suppress geometric drift, and perform absorption merging in flat regions to compress object size, thus significantly reducing the proportion of invalid computations while maintaining reconstruction accuracy and structural stability. Attached Figure Description

[0051] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0052] Figure 1 An exemplary application scenario diagram provided for an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the structure of a graphics processing unit provided in an embodiment of the present invention;

[0054] Figure 3 This is a flowchart illustrating a low-computing-power, fast 3D modeling method based on 3DGS, as provided in an embodiment of the present invention. Detailed Implementation

[0055] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0056] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0057] This application's method is applicable to imaging modeling scenarios requiring continuous multi-view acquisition, continuous updating, and rapid acquisition of usable 3D models under limited computing power. Limited computing power refers not only to low hardware specifications but also to situations where effective available computing power is limited due to deployment configuration, power consumption constraints, latency requirements, or data flow limitations. For example, while mobile acquisition devices, portable detection terminals, edge-side graphics processing units, embedded industrial control terminals, or integrated on-site modeling devices can complete image acquisition and basic rendering, they struggle to handle the computational overhead of high-frequency, full-scale iterative updates of large-scale Gaussian objects as in traditional 3DGS solutions. 3DGS is a modeling and rendering method that represents a 3D scene using explicit Gaussian object sets. It does not use traditional triangular meshes for closed surface reconstruction, nor does it differ from volume rendering based on implicit neural fields. Instead, it assigns 3D Gaussian objects with spatial location, shape distribution, transparency, and color representation parameters to local structures in the scene, projects these 3D Gaussian objects onto the image plane at a given viewpoint to form an elliptical projection, and then combines depth sorting and transparency mixing to generate the rendering result for the corresponding viewpoint. In such scenarios, the modeling system usually needs to provide interim 3D results during the acquisition process so that the operator can promptly determine whether the acquisition coverage is sufficient, whether there are blind spots, and whether reshooting is needed, thereby avoiding offline rework. This makes the modeling method of acquiring and updating simultaneously have significant engineering value.

[0058] It is understood that the application scenarios applicable to this application include, but are not limited to: First, the target object has rich local geometric details, occlusion relationships, or texture / reflection differences, and needs to gradually complete the 3D information through multiple perspectives, such as cultural relics, industrial parts, complex curved surface components, and precision parts with grooves or holes; Second, the acquisition process is limited by on-site conditions, making it impossible to acquire a full-coverage view at once, but rather to gradually input image data in a single-view, one-cycle manner; Third, the modeling results need to be displayed in real time or near real time on the acquisition terminal side to serve on-site verification, interactive preview, or subsequent task connection. In application scenarios, those skilled in the art generally encounter a practical contradiction: if the full training approach of high-precision 3DGS is followed, then for each new perspective image introduced, the existing Gaussian field needs to be re-optimized on a large scale, and the computational load increases rapidly with the number of Gaussians, resulting in unacceptable latency in on-site modeling; if the number of iterations is simply reduced or the number of Gaussians is directly limited, it is easy for the local details just exposed under the new perspective to not be expressed in time, manifested as blurred edges, unstable occlusion transitions, local structural drift, or insufficient texture / lighting fitting.

[0059] It's important to note that the reason for continuously updating the Gaussian field model under different viewpoints is not simply because more images have been captured, but because the scene representation mechanism of 3DGS itself dictates that new viewpoints introduce new constraint information. Specifically, each Gaussian object in the Gaussian field model is associated with at least spatial position parameters, shape parameters, transparency parameters, and color representation parameters. Its projection position, projection ellipse shape, occlusion order, and color contribution under the current viewpoint are all related to the camera viewpoint. When the acquisition viewpoint changes, surface areas that were not fully exposed in the previous modeling cycle, occluded detail areas, or viewpoint-related color change areas will become visible again. At the same time, the coverage and depth blending relationships of existing Gaussian objects in the image plane will also change. In other words, even if the target object itself does not change, the geometric constraints, occlusion constraints, and photometric constraints imposed on the model under different viewpoints will still change. These changes need to be absorbed by updating the Gaussian field parameters and performing necessary density redistribution; otherwise, the model will struggle to simultaneously maintain consistency across multiple viewpoints and render realism.

[0060] In existing technical practices, common approaches typically involve accumulating a certain number of multi-view images and then performing batch training, or performing approximately uniform iterative updates on all Gaussian objects after each new viewpoint is added. While the former avoids frequent updates, it fails to meet the need for rapid on-site feedback and lacks real-time assessment of the current model's quality during acquisition. The latter, while enabling continuous updates, has computational costs highly dependent on the total number of Gaussian objects and consumes significant computing power on regions that contribute little to the current viewpoint or exhibit minimal variation in adjacent views. This can lead to low iteration efficiency, high memory pressure, and accumulated training latency on low-performance graphics processing units. Those skilled in the art often recognize that not all regions deserve equal updates in engineering implementation. However, quickly identifying the regions that truly require encryption and focused updates from the current viewpoint without costly comparisons of each Gaussian object, and ensuring that this identification directly drives Gaussian density allocation and parameter update strategies, remains a key bottleneck hindering the balance between accuracy and efficiency in edge 3DGS modeling.

[0061] The core concept of this application stems from the aforementioned engineering dilemma. In low-computing-power scenarios, the truly scarce resource is not the ability to perform a single forward rendering, but rather the ability to repeatedly optimize a massive number of Gaussian objects. Therefore, this application does not follow the path of comparing objects one by one and iteratively updating uniformly. Instead, it shifts the focus of complexity determination from the Gaussian object domain to the tile statistical domain of the image plane. On the one hand, by extracting residual energy and residual gradient statistics for each tile region through the residual information between the rendered image and the image data of the current modeling cycle, it reflects which regions do indeed have fitting errors from the current perspective. On the other hand, by combining the projection contribution of the current Gaussian field model on the image plane, it constructs the projection statistical spectrum of each tile and compares it with the projection statistical spectrum of the corresponding tile from the previous modeling cycle to characterize the degree of local structural change caused by the change in perspective. By jointly gating structural changes and residual magnitude, this application can quickly obtain a more computationally efficient complexity score without establishing a one-to-one correspondence between Gaussian objects across cycles, and accordingly divide the region into different state regions suitable for encryption, fine-tuning, or sparseness.

[0062] This application further develops an adaptive Gaussian density allocation mechanism that matches low computing power conditions. Specifically, in regions with high complexity scores and significant residuals, targeted splitting or high-frequency updates are prioritized to improve the representation of newly exposed details, occluded boundaries, and regions sensitive to viewpoint changes. In regions with small structural changes but still some residuals, constrained parameter fine-tuning is prioritized to avoid unnecessary growth in the number of Gaussians. In regions with low residuals and stable structures, absorption merging, deletion, or reduced-frequency updates are performed to recover computing power and suppress the disorderly expansion of Gaussian object sizes. Therefore, this application does not simply aim to reduce the number of Gaussians or the number of iterations, but rather establishes a computing power allocation mechanism adapted to the current viewpoint information increment during multi-view continuous modeling. This ensures that the computing resources of the graphics processing unit prioritize serving the regions that truly affect the quality of 3D modeling, thus maintaining good geometric detail restoration capabilities and modeling update efficiency even under limited computing power conditions.

[0063] It should also be noted that the applicability of this application does not depend on the target object having fixed structural partitions, clear functional area divisions, or pre-defined regions of interest, nor does it require semantic segmentation or manual annotation of the target object before modeling. As long as the 3D modeling process of the target object meets the basic conditions of multi-view progressive input, continuous edge-side updates, and limited computing power budget, and there are local visibility changes, occlusion relationship changes, or view-related appearance changes under different views, the technical solution of this application can be used to quickly update the Gaussian field model with low computing power. In other words, this application focuses on a type of modeling and computational constraint problem that is common in the edge-side 3DGS engineering process. The complexity assessment and density scheduling mechanism proposed has universality for scene expression methods, rather than a specific adaptation design for a certain type of object or a certain type of acquisition device.

[0064] refer to Figure 1 , Figure 1 This is an exemplary application scenario diagram provided for an embodiment of this application.

[0065] like Figure 1 As shown, the object to be modeled can be a solid target with complex curved surface structures, subtle geometric details, or viewpoint-dependent appearance characteristics. During the acquisition process, the object is in a relatively static or controllable posture change state. An image acquisition device is used to acquire image data of the object to be modeled from different viewpoints. This image acquisition device can be an industrial camera, a mobile terminal camera module, an array camera system, or other imaging device with multi-view acquisition capabilities. In practical applications, the image acquisition device typically outputs image data in a single-view, frame-by-frame acquisition manner, with each frame corresponding to one modeling cycle.

[0066] The graphics processing unit (GPU) is communicatively connected to the image acquisition device to receive image data corresponding to each modeling cycle and maintain the current Gaussian field model locally. The GPU can be a discrete graphics card, an embedded graphics acceleration chip, an edge GPU module, or other processors with graphics rendering and parallel computing capabilities. In low-computing-power deployment scenarios, the GPU's memory capacity, the number of parallel computing cores, or power consumption budget are all constrained; therefore, continuous modeling and updating of multi-view images needs to be completed under limited computing power.

[0067] Therefore, in Figure 1 In the application scenario shown, the 3D modeling result is not generated offline all at once, but rather evolves gradually as multi-view image data is continuously input into the image acquisition device. This 3D modeling result can be rendered and displayed in real time on the graphics processing unit, or output to an external display terminal or storage system for subsequent analysis, detection, or interactive applications.

[0068] It should be noted that, Figure 1 This illustration only demonstrates a typical application architecture of the technical solution of this application, and its structural connections and module divisions do not constitute a limitation on the technical solution of this application. Those skilled in the art can make equivalent substitutions or extensions to the physical form, communication method, and output path of the image acquisition device and graphics processing unit according to the actual deployment environment, without departing from the core concept of this application.

[0069] refer to Figure 2 , Figure 2 This is a schematic diagram of the structure of the graphics processing unit provided in an embodiment of this application.

[0070] like Figure 2 As shown, the graphics processing unit may include a memory, a processor, and a display. The functional components are connected through a bus or other data interaction channels to collaboratively complete the low-computing-power fast 3D modeling process based on 3DGS.

[0071] The memory is used to store program instructions and data, including but not limited to: current Gaussian field model data, Gaussian object set parameters, tile projection statistical spectra over multiple modeling cycles, tile residual statistics, complexity score data, and control parameters for executing adaptive density allocation and lightweight iteration strategies. The memory can be volatile memory, non-volatile memory, or a combination of both. In low-computing-power deployment environments, memory capacity is often constrained. Therefore, this application reduces the reliance on historical Gaussian object-level data through tile-level statistics and residual gating mechanisms to lower storage overhead.

[0072] The processor is communicatively connected to the memory and is used to invoke program instructions. Specifically, in each modeling cycle, the processor receives image data transmitted by the image acquisition device, performs forward rendering based on the current Gaussian field model, generates a rendered image, and calculates residual information; then, it constructs a tile projection statistical spectrum by combining the projection statistics of the Gaussian object set, compares it with the corresponding statistical spectrum of the previous modeling cycle, and determines the degree of structural variability; based on this, it uses the tile residual statistics to perform residual-gated weighting on the degree of structural variability to obtain a complexity score, and triggers adaptive Gaussian density allocation operations such as directional splitting, parameter-constrained updates, or absorption merging, thereby completing the update of the Gaussian field model within a limited computing power budget.

[0073] The display is connected to the processor and is used to output the 3D modeling results or intermediate rendering results of the current modeling cycle. During implementation, the display can show a 3D view rendered based on the updated Gaussian field model in real time, so that the operator can judge the modeling coverage and detail expression effect from the current acquisition viewpoint. Because this application adopts a differentiated density scheduling and lightweight iteration strategy, the processor can still maintain a relatively stable rendering frame rate under limited computing power, making the output on the display continuous and interactive.

[0074] It should be noted that, Figure 2 The structure shown is merely an exemplary form of the graphics processing unit. The physical form of the memory, processor, and display can be discrete devices or integrated into the same chip or the same terminal device. Those skilled in the art can make equivalent substitutions or functional reorganizations of the graphics processing unit structure according to specific application requirements without departing from the technical concept of this application, without affecting the implementation principle of the low-computing-power rapid 3D modeling scheme disclosed in this application.

[0075] Next, with reference to the accompanying drawings, the low-computing-power fast 3D modeling method based on 3DGS provided in the embodiments of this application will be further elaborated. Figure 3 The method shown is applied to a graphics processing unit (GPU) configured with a current Gaussian field model, which is the updated Gaussian field model of the GPU in the previous modeling cycle. Each modeling cycle corresponds to image data of the object to be modeled from one viewpoint. The method includes:

[0076] S1: Obtain the image data of the object to be modeled in the current modeling cycle;

[0077] In this embodiment, the image data can be acquired by an industrial camera, a mobile terminal camera module, or an array camera device. Each modeling cycle corresponds to one or more frames of image data under a single viewpoint or a small range of pose changes. Unlike offline batch modeling, this embodiment adopts a single-viewpoint cycle-by-cycle input method, enabling the graphics processing unit to perform local updates after receiving each new viewpoint data, thereby forming a continuously evolving 3D model. The purpose of this method is to reduce the computational burden per cycle, distribute the computing power consumption across each modeling cycle, and provide phased 3D results during the acquisition phase, avoiding the peak computing power pressure and time delay caused by centralized training after all multi-view acquisitions are completed. Those skilled in the art will understand that the resolution, frame rate, and acquisition path of the image data can be adjusted according to the actual scenario, as long as it can provide new viewpoint constraint information for the current Gaussian field model; this application does not limit this.

[0078] S2: Perform forward rendering based on the image data and the current Gaussian field model to obtain a rendered image, and determine residual information based on the rendered image and the image data;

[0079] Specifically, each Gaussian object in the current Gaussian field model is projected onto the image plane according to the camera parameters corresponding to the current modeling cycle. Then, pixel-by-pixel alpha blending is performed according to depth sorting and transparency parameters to generate a rendered image from the current viewpoint. Subsequently, the rendered image is compared pixel-by-pixel with the actual acquired image to obtain pixel-level residuals, and the residual energy distribution and residual gradient distribution are further constructed. This step is necessary because different viewpoints introduce new visible regions and new occlusion relationships. Without explicitly measuring the error from the current viewpoint, it is impossible to determine which regions truly need updating. Introducing residual information quantifies the fit of the current Gaussian field model to that viewpoint, providing a foundation for subsequent complexity assessment and computational scheduling. By employing a forward rendering combined with residual calculation, error analysis can be completed using only the existing Gaussian objects and image data, eliminating the need to establish a one-to-one mapping relationship between Gaussian objects across cycles, thus reducing computational complexity.

[0080] S3: Determine the complexity information based on the residual information and the set of Gaussian objects corresponding to the current Gaussian field model;

[0081] The complexity information is used to characterize the local geometric detail complexity of the object to be modeled in the current modeling cycle, and to divide the complex region into a flat region. Each Gaussian object in the Gaussian object set is associated with at least spatial position parameters, shape parameters, transparency parameters, and color representation parameters.

[0082] In this embodiment, the image plane is first divided into blocks according to a preset tile partitioning rule. The residual information is aggregated into tile residual statistics, including residual energy and residual gradient statistics. Simultaneously, during the forward rendering process, the projection contribution of each Gaussian object on the image plane is statistically analyzed to construct the projection statistical spectrum of each tile. This spectrum is then compared with the statistical spectrum of the corresponding tile from the previous modeling cycle to obtain the structural variability. Based on this, the structural variability is weighted using residual gating to form a tile-level complexity score, which is used to divide complex regions into complex and flat regions.

[0083] Understandably, as the viewing angle changes, some regions may experience structural changes with minimal error, while others may exhibit significant errors with little structural change. Relying solely on structure or residuals could lead to an imbalance in computational resources. By jointly evaluating structural changes and residual magnitude, the complexity assessment is aligned with real-world modeling needs, thus avoiding ineffective updates to numerous stable regions under low computational power conditions. Replacing Gaussian-by-Gaussian comparisons with tile-level statistics makes the complexity calculation dependent on the number of image blocks, rather than linearly dependent on the number of Gaussian objects, significantly reducing the computational burden.

[0084] S4: Perform adaptive Gaussian density allocation on the current Gaussian field model according to the complexity information to obtain the updated Gaussian field model for the current modeling cycle, and perform 3D modeling based on the updated Gaussian field model;

[0085] In this embodiment, directional splitting or high-frequency parameter updates are triggered in complex regions to enhance the Gaussian density and parameter expressive power in areas with significant errors. In flat regions, absorption merging, deletion, or frequency reduction updates are performed to reduce the number of Gaussian objects or lower their update frequency. In regions with high residuals but limited structural changes, restricted parameter fine-tuning is prioritized, updating only color representation parameters or transparency parameters while maintaining the basic stability of spatial position and shape parameters. This application prioritizes the allocation of limited computing power to regions with the greatest impact on modeling quality, while suppressing the disorderly growth of the number of Gaussian objects and avoiding a continuous increase in memory usage due to full iteration. By using the updated Gaussian field model as the current model for the next modeling cycle at the end of each modeling cycle, progressive optimization of the model is achieved, allowing the 3D modeling results to gradually converge during multi-view input.

[0086] Before detailing the specific technical aspects corresponding to the steps, this application's embodiments need to reiterate:

[0087] Unlike offline modeling, which involves capturing all viewpoints before unified training, this embodiment employs a continuous input and correction Gaussian field update process. Each modeling cycle only introduces image constraints corresponding to the current viewpoint, while the current Gaussian field model continues to participate in fitting, building upon the update results from the previous cycle. In this mode, the model is not built all at once but converges gradually over multiple viewpoint switching processes. Therefore, the role of the same Gaussian object is not constant across different modeling cycles; its projection coverage, occlusion participation, and color contribution weight all shift with viewpoint changes. Those skilled in the art will understand that this "cross-cycle parameter domain change" is not an abnormal parameter drift but a normal evolutionary phenomenon of 3DGS under multi-viewpoint constraints. Therefore, the unified update strategy used in static model optimization cannot be simply applied; instead, the update value of local regions needs to be reassessed within each modeling cycle.

[0088] Furthermore, this embodiment does not directly use the existence of errors in Gaussian objects as the sole basis for updates. Instead, it introduces a method to determine whether a local region is worth investing computational resources in within the current cycle. This is because, during continuous modeling, errors in the current image do not necessarily correspond to missing model structures; they may also originate from temporary occlusion rearrangement, color response differences, or changes in projection coverage caused by changes in local viewpoints. Conversely, some local regions, while not prominent in pixel errors, may still require increased Gaussian density to maintain modeling stability in subsequent cycles because the new viewpoint exposes previously unfitted geometric edges or thin-walled structures. Based on this, this embodiment divides the image plane into statistical units, comprehensively examining error performance and changes in projection behavior at the regional scale. This gives complexity assessment the meaning of modeling resource scheduling, rather than just error numerical judgment, thus providing a more engineering-compliant basis for subsequent density allocation.

[0089] It should also be noted that this embodiment does not employ an object-by-object tracking approach to analyze the changes in Gaussian objects across different periods. While theoretically it is possible to compare the parameter changes, projection changes, or contribution changes of each Gaussian object individually, this approach introduces a significant computational and storage burden when the number of Gaussian objects is large. Furthermore, in areas where occlusion relationships change frequently, object-level comparisons are easily affected by visibility switching, leading to instability in determining the importance of changes. Therefore, this embodiment instead uses a projection statistical representation method that is inherently consistent with the rendering process. That is, during the forward rendering stage, statistics such as coverage, depth blending, and orientation consistency within the local area are extracted simultaneously and then compared with corresponding statistics from historical periods. This approach does not require establishing strict cross-period object correspondences but can still stably reflect the changing trends of the local structural expression state, making it particularly suitable for rapid complexity determination under limited computing power conditions.

[0090] Furthermore, the complex and flat regions in this embodiment are functional divisions oriented towards the modeling update strategy, and do not correspond to fixed physical partitions of the object to be modeled, nor are they required to be consistent with the geometric semantic structure of the object. For example, in a certain modeling cycle, a region that was originally a smooth surface may be judged as a complex region due to strong occlusion boundaries or specular changes caused by a change in viewpoint; while in subsequent modeling cycles, the region may statistically revert to a flat state. This dynamic partitioning mechanism helps to keep the Gaussian density allocation synchronized with the current viewpoint information increment, avoiding the problems of over-encryption and long-term computational resource consumption commonly encountered when using preset partitions or artificial regions of interest. Accordingly, the splitting, merging, fine-tuning, or frequency reduction updates involved in subsequent steps are all carried out around the results of this dynamic region partitioning, rather than performing preset processing on fixed object parts.

[0091] It is important to emphasize that the computational pressure in Gaussian field modeling does not solely stem from the number of Gaussians themselves, but is also directly related to the proportion of Gaussians participating in high-frequency updates and the dimensionality of Gaussian parameter updates. In continuous modeling scenarios, if all Gaussian objects are continuously updated at the same frequency and in the same dimension, even if the total number is controlled within a certain range, the modeling cycle time may still increase due to the ineffective iteration of a large number of low-contribution objects. Therefore, this embodiment considers density allocation and update intensity allocation in a coordinated manner in its overall design. That is, the region state is first determined through complexity assessment, and then it is decided whether Gaussian objects in that region should undergo directional splitting, parameter fine-tuning, absorption and merging, or low-frequency maintenance. This ensures that computational pressure control is reflected not only in the adjustment of the number of Gaussians but also in the selection of update paths.

[0092] Next, we will further elaborate on the technical content of the method in this application regarding the current Gaussian field model.

[0093] It is understood that this application outputs an updated Gaussian field model for each modeling cycle. The updated Gaussian field model can be understood as the staged Gaussian field representation result obtained by performing local parameter correction and / or density redistribution on the Gaussian field model of the previous modeling cycle after introducing the current viewpoint image constraint in the modeling cycle. This result is not required to reach the final convergence state in a single modeling cycle, but is used to characterize the usable model state under the fused viewpoint information at the current moment. When entering the next modeling cycle, the updated Gaussian field model of the previous modeling cycle is used as the current Gaussian field model, that is, the updated Gaussian field model of the previous modeling cycle is used as the starting model of the next modeling cycle, so that the subsequent modeling cycles can continue to absorb new viewpoint information and perform progressive optimization on the basis of existing geometric and photometric representations.

[0094] It should be noted that if the current modeling cycle is the first modeling cycle, there is obviously no updated Gaussian field model passed from the previous modeling cycle in the graphics processing unit. Therefore, the current Gaussian field model used in this modeling cycle is the initial Gaussian field model obtained by initialization. This initial Gaussian field model can be constructed based on the image data of the viewpoint corresponding to the first modeling cycle, the set of feature points extracted from the image data, and the camera pose solution results, and generate a Gaussian field representation containing the initial Gaussian object set as the starting basis for continuous updates in subsequent modeling cycles.

[0095] In one example, the construction process of the current Gaussian field model includes:

[0096] Acquire image data from the viewpoint corresponding to the first modeling cycle, and perform feature extraction based on the image data to obtain a set of feature points;

[0097] Based on the set of feature points, camera pose is solved to determine the camera parameters for the first modeling cycle, and an initial geometric reference is constructed based on the camera parameters;

[0098] An initial Gaussian object set is generated based on the initial geometric reference, wherein the spatial position parameters of each Gaussian object in the initial Gaussian object set are determined by the initial geometric reference, the shape parameters are determined by a preset covariance initialization strategy, the transparency parameters are determined by a preset initial opacity value, and the color representation parameters are determined by the pixel color of the image data at the corresponding projection position.

[0099] Specifically, for the image data in the first modeling cycle, scale-invariant feature extraction or corner detection is first performed to obtain a spatially stable set of feature points. Feature point extraction can employ gradient-based corner response algorithms, scale-space feature extraction algorithms, or equivalent implementations thereof, as long as the pixel locations with local structural saliency can be identified in the image. Those skilled in the art will understand that the number and distribution of feature points can be adjusted according to the image resolution and texture richness; in low-texture areas, the threshold can be appropriately reduced to ensure the integrity of the underlying geometric reference.

[0100] Furthermore, since the first modeling cycle typically only contains a single-view image, the position and attitude of the current camera in the world coordinate system can be determined during implementation by combining device calibration parameters or extrinsic parameter initialization information, using PnP solving, least squares attitude estimation, or other equivalent pose recovery methods. If known intrinsic parameter calibration results exist, the intrinsic parameter matrix can be directly used for back-projection calculation; if not, a fast calibration can be performed during the initialization phase to obtain approximate intrinsic parameters. In this embodiment, the pixel coordinates of feature points in the image plane are back-projected to three-dimensional space, and an initial geometric reference is constructed by combining the camera pose information. This geometric reference can be in the form of a sparse point cloud, a regular depth plane, or an approximately enclosing structure, as long as it can provide an initial spatial position distribution for the Gaussian object.

[0101] Furthermore, when generating the initial Gaussian object set, the spatial position parameters of each Gaussian object are directly derived from the three-dimensional coordinates of the initial geometric reference point; the shape parameters are set according to a preset covariance initialization strategy, such as estimating the local scale based on the spatial distance between adjacent feature points, or adaptively shrinking in subsequent modeling cycles after setting a unified initial scale; the transparency parameter can be set to a unified intermediate value to ensure that each Gaussian object can participate in the blending operation in the forward rendering in the initial stage without excessive occlusion; the color representation parameter is assigned by sampling the color value of the corresponding pixel by projecting the initial geometric reference point back to the image plane, and when it is necessary to support view-related color changes, the color can be further extended to a low-order spherical harmonic representation to store basic lighting information.

[0102] Next, we will further elaborate on the technical aspects of forward rendering in this application.

[0103] It should be noted that the forward rendering in this application is not only used to generate the visual image from the current viewpoint. In this embodiment, it also undertakes the responsibility of generating intermediate quantities required for subsequent complexity evaluation. That is, while completing the Gaussian object projection and pixel blending, it retains process quantities that can be used for residual statistics and projection statistical spectrum construction. Therefore, this embodiment focuses not only on the visual consistency of the rendering result itself in the forward rendering stage, but also on the extractability of the contribution relationship, occlusion participation relationship, and directional distribution information of each Gaussian object to the local region of the image plane during the rendering process.

[0104] In its implementation, the forward rendering stage first maps each Gaussian object in the current Gaussian field model from 3D space to the image plane based on the camera parameters corresponding to the current modeling cycle. Each mapped Gaussian object forms a projection ellipse with directional and scale information on the image plane. This projection ellipse is determined by spatial position parameters, shape parameters, and the current camera viewpoint. Since the shape parameters of a Gaussian object can characterize the stretching direction and scale range of its local spatial distribution, its projection result not only reflects whether the Gaussian object covers a pixel but also carries the directional features of its local geometric structure.

[0105] After projection, pixel-by-pixel alpha blending based on depth order and transparency parameters is performed on the pixels on the image plane to obtain the rendered image for the current modeling cycle. This blending process serves two purposes in this embodiment: firstly, it utilizes the accumulation of transparency and the superposition of color contributions to generate a rendering result corresponding to the actual acquired image, used for subsequent residual information calculation; secondly, the blending order itself reflects the occlusion participation of Gaussian objects in the current viewpoint. Especially in regions with complex local structures, overlapping layers, or thin-walled occlusion, the effective weight distribution of different Gaussian objects during the blending process will show significant differences. These differences, after being aggregated using depth blending dispersion statistics, can stably characterize the structural complexity of the region in the current viewpoint.

[0106] Furthermore, the forward rendering stage in this embodiment also involves the expansion and calculation of color representation parameters under the current viewpoint. For Gaussian objects represented using spherical harmonic coefficients or equivalent viewpoint-dependent colors, their color contribution is not a fixed constant but varies with the viewing direction. This design allows the forward rendering stage to more accurately express the appearance differences of reflections, diffuse reflections, and local illumination changes under the current viewpoint, thereby improving the consistency between the rendered image and the actual acquired image. At the same time, the viewpoint-dependent nature of color contribution also provides a basis for subsequent residual interpretation. That is, when a significant residual appears in a certain area, the color contribution changes of Gaussian objects in that area can be combined to determine whether the residual is more inclined towards geometric underfitting or appearance underfitting. Thus, the forward rendering stage not only provides visualization results for the current cycle of 3D modeling but also provides a discriminative analytical basis for subsequent residual gating and differentiated update strategies, enabling subsequent update processes to allocate computing resources more effectively.

[0107] In one example, determining the residual information includes:

[0108] During the generation of the rendered image, the color contribution of each Gaussian object under the current viewpoint is determined based on the color representation parameters, wherein the color contribution includes at least spherical harmonic coefficients used to characterize the color change of the viewpoint;

[0109] After generating the rendered image, pixel-level residuals are calculated based on the rendered image and the image data, and the pixel-level residuals are aggregated using the color contribution to obtain the residual information.

[0110] Those skilled in the art will understand that the projection, sorting, blending, and color unrolling processes described above in the forward rendering process can all be implemented using conventional parallel graphics computing methods, such as tile-based parallel processing, pixel-based parallel blending, or batch Gaussian projection calculations based on graphics processing units. As long as the required projection coverage, depth blending, and orientation information can be obtained while generating the rendered image, the processing requirements of the subsequent steps in this embodiment can be met. This application does not limit the specific form of parallel scheduling implementation.

[0111] In some optional implementations, this application provides a 3DGS forward rendering implementation for a single modeling cycle, used to illustrate the specific process of forward rendering in this application. This implementation uses a single frame of image data input in the current modeling cycle as the target view, generates a corresponding rendered image based on the current Gaussian field model, and synchronously outputs intermediate quantities usable in subsequent steps during the rendering process. It should be noted that this implementation is mainly used to explain the engineering implementation logic of the forward rendering chain; those skilled in the art can make equivalent substitutions for the data structure, parallel scheduling method, and cache organization form without changing the processing principle.

[0112] In this implementation, the camera parameters corresponding to the current modeling cycle are first read, including intrinsic parameters, extrinsic parameters, and the observation direction information of the current viewpoint. Then, view transformation and projection calculations are performed on each Gaussian object in the current Gaussian field model. Specifically, the following processing path can be adopted: first, the spatial position parameters of the Gaussian object are transformed from the world coordinate system to the camera coordinate system to obtain the center position in the camera coordinate system; then, combined with the shape parameters, a linear transformation is performed on the local spatial distribution of the Gaussian object in the current viewpoint to obtain its two-dimensional projection ellipse parameters on the image plane, including the ellipse center, principal axis direction, principal axis scale, and coverage area. For Gaussian objects that fall behind the camera or whose projection range exceeds the image boundary, they can be directly marked as invalid objects in the current viewpoint and skipped in subsequent blending to reduce invalid calculations. Through the above processing, each Gaussian object corresponds to a projection ellipse description that can participate in rendering in the current modeling cycle, and this description simultaneously retains directional and scale information for subsequent analysis.

[0113] After calculating the projection parameters, pixel coverage relationships are established for the valid Gaussian objects under the current viewpoint, and depth-ordered blending is performed. Specifically, Gaussian objects within the coverage area of ​​each pixel are collected by scanning pixels or traversing the projection coverage area, and sorted according to the depth values ​​in the camera coordinate system. For a given pixel, the transparency parameters and color representation parameters of the corresponding Gaussian object are read sequentially according to the sorting results. The coverage weight and color contribution of the Gaussian object at that pixel are calculated, and alpha blending is performed. The color contribution can be calculated by directly assigning constant colors, or by expanding the spherical harmonic coefficients based on the viewing direction to obtain the direction-related color value under the current viewpoint. At the engineering implementation level, the forward rendering process in this embodiment can be executed in batch parallel mode on the graphics processing unit. For example, the projection ellipse parameters are calculated in batch parallel by Gaussian object in the projection stage, and depth sorting and alpha blending are performed in parallel by pixel or by coverage block in the blending stage. For the case where the color representation parameter is a spherical harmonic coefficient, the basis function value corresponding to the current viewpoint can be pre-calculated and reused in the same modeling cycle to reduce redundant calculations.

[0114] Next, we will further elaborate on the technical aspects of the complexity information in the method of this application.

[0115] It should be noted that the tiles in this application can be understood as statistical calculation units constructed according to preset rules on the image plane corresponding to the current modeling cycle. Their essence is not merely for image display segmentation, nor is it equivalent to artificial region division oriented towards the semantic structure of the target object. Rather, they serve as intermediate statistical carriers for residual information aggregation, projection contribution aggregation, and cross-modeling cycle variation comparison. Specifically, a tile corresponds to a set of continuous pixels in the image coordinate system, and each tile in this embodiment participates in subsequent processing as an independent complexity evaluation unit. By establishing complexity evaluation at the tile level, rather than directly at the single pixel or single Gaussian object level, complexity calculation can simultaneously consider local spatial resolution and computational power controllability, thus making it more suitable for low-computing-power deployment requirements in continuous modeling scenarios.

[0116] In one optional implementation, the tile division rules of this application specifically include:

[0117] The image plane is first divided into several tile regions according to a preset grid rule, where each tile covers a certain number of pixels, such as 32×32, 64×64, or 128×128 pixels. The size of these tile divisions can be flexibly adjusted according to the actual image resolution, the scale of the target object, and the computing power budget of the graphics processing unit. Each tile region is not dependent on the geometric partitioning of the object, but is uniformly distributed based on the pixel distribution in the image plane. Therefore, this tile division method is applicable to all types of objects to be modeled, regardless of their geometric structure.

[0118] In practical applications, if certain areas of the target object or the scene being captured are rich in detail or undergo significant changes when the viewpoint shifts, the information differences within some tiles may become excessively large. In such cases, using fixed-size tiles may fail to reflect the complexity within the region. Therefore, in some situations, the tile size can be adaptively adjusted. Smaller tiles can be used for more refined analysis of geometric edges or complex textured areas in the image, while larger tiles can be used for coarse-grained aggregation analysis of flat, simple structures or areas with minimal texture variation. This adaptive strategy can flexibly adjust the computational granularity according to the changing needs of the target region, avoiding over-refinement of low-complexity areas, thereby further reducing the computational burden.

[0119] In one example, complexity information is determined based on the residual information and the set of Gaussian objects corresponding to the current Gaussian field model, including:

[0120] S3.1: Aggregate the residual information between the rendered image and the image data according to the preset tile division rules to obtain the tile residual statistics of each tile, wherein the tile residual statistics include at least the residual energy statistics and the residual gradient statistics;

[0121] Specifically, the residuals are calculated by comparing each pixel individually to obtain pixel-level residuals, which are then aggregated. Furthermore, the entire image plane is divided into multiple tile regions using tile partitioning rules, with each tile corresponding to a portion of the image. The purpose of this is to reduce computational complexity through the aggregation of local regions, avoiding lengthy pixel-by-pixel calculations across the entire image.

[0122] In one example, the residual information between the rendered image and the image data is aggregated according to a preset tile partitioning rule, including:

[0123] S3.1.1: Calculate the pixel residual map based on the residual information and the image data;

[0124] Specifically, in the current modeling cycle, after forward rendering is completed, a rendered image corresponding to the current viewpoint has been obtained, while process quantities describing the color contribution of Gaussian objects are also retained. The reason for constructing a pixel residual map in this step, instead of just retaining an overall error value, is that subsequent complexity evaluation requires determining the spatial distribution and local variation of the error on the image plane; in other words, simply knowing that the error is large is insufficient to support density scheduling, it is also necessary to know where the error is located and how it is distributed. The pixel residual map can be understood as a two-dimensional data map that assigns a residual value to each pixel position in the current image coordinate system, used to record the degree of fitting deviation of that pixel under the current Gaussian field model. This map is aligned pixel-by-pixel with the image data of the current modeling cycle, therefore no additional registration processing is required during subsequent tile division.

[0125] In this embodiment, the construction of the pixel residual map can be achieved as follows: First, the size consistency check and coordinate alignment confirmation of the rendered image and image data are performed. When the acquired image has undergone cropping, scaling, or distortion correction, it is preferable to enter the residual calculation after the same preprocessing link to ensure that the rendered image and image data are comparable at the pixel level. Then, the color values ​​of the rendered image and image data are read according to the pixel position to generate the corresponding pixel residual. The pixel residual can be in the form of brightness difference, three-channel color difference, or perceptual difference after color space transformation. For example, in industrial imaging scenarios, grayscale difference can be directly used to reduce the amount of calculation. In scenarios with obvious color changes, three-channel difference can be used and synthesized into a single-value residual through preset weights. Those skilled in the art will understand that if the viewpoint-related color contribution information is retained in the forward rendering stage, the pixel residual can also be additionally labeled or weighted according to the color contribution amplitude in this step so as to distinguish the error caused by insufficient geometric coverage and the error caused by color response deviation in subsequent steps. As long as a pixel residual map that can be used for spatial aggregation can be formed in the end, this application does not limit the specific data type of the residual map.

[0126] S3.1.2: Based on the preset tile division rules, the pixel residual map is divided into multiple tile regions, wherein each tile region corresponds to a set of pixels of a preset size;

[0127] In this embodiment, tile partitioning can be performed according to a preset grid rule in the pixel residual map coordinate system. Specifically, the image resolution of the current modeling cycle can be read, and the image plane can be divided into several rectangular tile regions according to the preset tile size. When the image width and height cannot be divided evenly by the tile size, the edge regions can be processed using padding, truncation, or boundary adaptive tile methods. For example, for an image with a resolution of 1920×1080, if the preset tile size is 32×32 pixels, then 60 columns of tiles can be divided horizontally, and 34 rows of tiles can be divided vertically. Edge portions that are not fully sized are truncated to form edge tiles. Those skilled in the art will understand that the tile size can be configured based on the computing power budget and the target detail scale: smaller tile sizes result in finer local positioning but a greater number of statistical units; larger tile sizes result in higher statistical stability but reduced detail positioning capability. To ensure a one-to-one correspondence between the subsequent projection statistical spectrum and the residual statistics, the tile partitioning rule is preferably kept fixed within the same modeling cycle and continues with the same partitioning benchmark in adjacent modeling cycles.

[0128] S3.1.3: Within each tile region, the pixel residual is calculated for the pixel set, and the pixel residual is robustly aggregated to obtain the residual energy statistics of the tile region, wherein the robust aggregation includes at least one of mean aggregation, quantile aggregation and truncated mean aggregation;

[0129] Specifically, residual energy statistics describe the overall fitting bias of a tile region in the current modeling cycle, and are one of the core inputs used in subsequent residual gating to determine whether the region is worth investing update computing power in. Robust aggregation is used instead of directly calculating the ordinary mean of pixel residuals within the tile because tiles often contain normal error pixels, locally reflective pixels, edge transition pixels, and a small number of anomalous pixels. If a few anomalously high residual pixels dominate the statistical results, subsequent steps will misclassify the tile as a persistently high-complexity region, leading to unnecessary Gaussian splits or high-frequency updates. Robust aggregation weakens the influence of outliers while preserving the overall error trend of the tile, making the tile residual energy statistics more suitable as a basis for cross-cycle comparisons and gating weighting.

[0130] In this embodiment, for each tile region, the effective pixel residual set within the tile is first extracted based on the effective pixel mask established in S3.1.2, and invalid pixels or abnormal pixels marked by cropping are removed; then, a tile residual energy statistic is generated according to a preset aggregation strategy. The robust aggregation strategy can be implemented in different ways depending on the device's computing power and the scene's noise level: when the image noise is low and the computational overhead is sensitive, mean aggregation can be used; when there are local highlights or significant sampling noise, quantile aggregation can be used, for example, taking the middle and high quantile values ​​of the pixel residuals within the tile to reflect the local error level; when it is desirable to consider both overall error and outlier suppression, truncated mean aggregation can be used, that is, first sorting the pixel residuals within the tile, removing the highest and / or lowest proportions of pixel residuals, and then calculating the mean. For example, for a 32×32 pixel tile with 900 effective pixels, the pixel residuals can be sorted from smallest to largest. After removing the top 5% and bottom 5% of the pixel residuals, the average of the remaining 810 pixel residuals is calculated, and this average is used as the residual energy statistic for the tile. Those skilled in the art will understand that the removal ratio can be adjusted according to the image noise level, and a common range is 3% to 10%.

[0131] S3.1.4: Within each tile region, a gradient operator is performed on the pixel residual to calculate a residual gradient map, and the residual gradient maps are aggregated to obtain a residual gradient statistic for the tile region, wherein the residual gradient statistic is used to characterize the spatial drasticness of the residual change within the tile region.

[0132] Specifically, residual energy statistics alone are insufficient to fully reflect the error characteristics within a tile. Some tiles may have high overall residuals but uniform distribution, often corresponding to color deviations or inconsistent slow lighting. Other tiles, even with relatively low average residuals, exhibit strong spatial boundary variations, typically indicating underfitting of geometric edges, shifts in occlusion relationships, or inadequate representation of detailed structures. By calculating residual gradients within tiles and generating residual gradient statistics, complexity assessment can identify whether errors exhibit spatially concentrated variations, thus providing higher resolution for subsequent gating weighting. In other words, residual energy characterizes the magnitude of the error, while residual gradient characterizes the sharpness of error changes; combining both is necessary to accurately distinguish between parameter fine-tuning requirements and density enhancement requirements.

[0133] In this embodiment, for each tile region, the pixel residual sub-map corresponding to that tile is first read from the pixel residual map, and boundary pixels are padded if necessary to ensure that the gradient operator can be stably calculated at the tile boundaries. The gradient operator can be a conventional local difference method, a Sobel-type operator, a Prewitt-type operator, or an equivalent discrete gradient operator, as long as the changes in the pixel residual in the horizontal and vertical directions can be obtained. Subsequently, the residual gradient map of the tile is generated based on the gradient response, and aggregation processing is performed on the residual gradient map. The aggregation method can be similar to S3.1.3, using a robust aggregation strategy, such as performing truncated mean aggregation or taking the middle and high quantile values ​​for the gradient magnitude, to avoid the peak of a single boundary pixel dominating the statistical results. For example, the residual gradient magnitudes can be sorted in each tile, and the top 80% quantile values ​​can be taken as the residual gradient statistics of that tile, so that the statistics are more sensitive to local edge errors and less affected by extreme noise. Those skilled in the art will understand that if it is necessary to further distinguish directional errors, a gradient principal direction histogram can be added to this step. However, in this example, it is sufficient to output gradient statistics that can characterize the severity of the space to meet the needs of subsequent steps.

[0134] S3.1.5: Weight the residual energy statistic and the residual gradient statistic to obtain the tile residual statistic;

[0135] In this embodiment, the weights can be configured in a fixed way or dynamically adjusted according to the modeling stage. For example, in the early stage of modeling, when the model as a whole is not yet stable and local errors are generally high, the weight of the residual energy statistic can be appropriately increased, for example, the residual energy weight can be set to 0.6 to 0.7 and the residual gradient weight can be set to 0.3 to 0.4, so as to ensure that the overall fitting deviation can be identified first. In the middle and later stages of modeling, when the overall error decreases and local detail boundaries are more worthy of attention, the weight of the residual gradient statistic can be increased, for example, the residual energy weight can be adjusted to 0.4 to 0.5 and the residual gradient weight can be adjusted to 0.5 to 0.6.

[0136] S3.2: Aggregate the Gaussian projection contributions of each tile based on the Gaussian object set to obtain the projection statistical spectrum of each tile, wherein the projection statistical spectrum includes at least the coverage weight statistic, the depth mixing dispersion statistic, and the anisotropic direction consistency statistic.

[0137] Specifically, for each tile region, the contribution of each Gaussian object to that tile region is calculated based on its spatial location, shape, transparency, and color. Since the projection shape, size, coverage area, and depth information of each Gaussian object under the current viewpoint all affect the final rendering result of that tile region, the construction of the projection statistical spectrum includes several statistical components, primarily: Coverage weight statistics, representing the total effective coverage area of ​​Gaussian objects within each tile region under the current viewpoint, reflecting the geometric complexity of the region. If there are many Gaussian objects within a tile region and their projection areas are large, the coverage weight statistics of that tile are high, indicating high geometric complexity. Depth mixing dispersion statistics, reflecting the differences in depth ranking among Gaussian objects within the tile region. If multiple Gaussian objects within the region have different depth values, and the occlusion relationships among these Gaussian objects are complex, the depth mixing dispersion is high, indicating complex occlusion and depth variations in the region. Anisotropic direction consistency statistics, used to measure the consistency of the principal axis directions of Gaussian objects within the tile region. If the projection principal axes of multiple Gaussian objects are relatively consistent, the directional consistency of the tile region is strong, which usually indicates that the region has a relatively regular geometric structure or texture information.

[0138] In one example, the Gaussian projection contributions of each tile are aggregated based on the Gaussian object set to obtain the projection statistical spectrum of each tile, including:

[0139] S3.2.1: Determine the target tile region for each Gaussian object;

[0140] Specifically, a Gaussian object corresponds to a projected ellipse on the image plane from the current viewpoint, and its coverage area typically spans multiple pixels, or even multiple tiles. If the attribution or association between Gaussian objects and tiles is not established beforehand, subsequent statistics would require a backward search of all Gaussian objects during tile traversal, resulting in high computational costs and difficulty in maintaining consistency with the forward rendering stage. By determining the target tile region for each Gaussian object immediately after forward rendering, the participation range of each Gaussian object from the current viewpoint can be pre-registered, thus transforming the subsequent statistical process into a forward accumulation process of directly reading the corresponding Gaussian object set by tile index. Here, the target tile region does not require the Gaussian object to belong to only a single tile, but rather refers to one or more tile regions that the Gaussian object actually covers and contributes to after projection from the current viewpoint.

[0141] In this embodiment, the determination of the target tile region can be combined with the projection ellipse parameters obtained in the forward rendering stage. Specifically, the center, principal axis length, principal axis direction, and two-dimensional coverage boundary of the projection ellipse of each Gaussian object in the current view are first read; then, according to the tile division rules, the bounding rectangle of the projection ellipse is mapped to the tile grid coordinates to obtain a set of candidate tiles. Since the bounding rectangle usually introduces some tiles that are not actually covered by the projection ellipse, to avoid overestimation in subsequent statistics, a secondary screening can be performed on the candidate tiles: a preset number of sampling points are extracted from the candidate tiles. For example, three to five sampling points can be selected to determine whether the sampling points fall within the effective coverage area of ​​the projection ellipse, or to determine whether the tile is a valid target tile based on the geometric intersection relationship between the projection ellipse and the tile boundary. If the intersection area or effective coverage ratio is lower than the preset minimum coverage ratio, the tile can be removed from the target tile region. For example, for a 32×32 pixel tile, the minimum coverage ratio can be set to 5% to 10% of the tile area. When the Gaussian object projection only glides over the corner of the tile and the effective coverage is insufficient to this ratio, the tile is not included in the target tile area to reduce statistical noise caused by low contribution relationships.

[0142] S3.2.2: Within each target tile region, for the Gaussian object belonging to the target tile region, the coverage weight statistics are accumulated based on the transparency parameter and the projection ellipse coverage weight;

[0143] Specifically, the coverage weight statistic does not merely reflect the number of Gaussian objects within a tile, but rather describes the degree to which the tile is effectively occupied and represented by Gaussian objects from the current viewpoint. Simply using the number of Gaussian objects ignores differences in their transparency and projected coverage. For example, two Gaussian objects might have vastly different coverage areas and high transparency within the tile, contributing significantly to rendering; another might only have minimal coverage at the tile's edge and low transparency, having almost no impact on the rendered image. Equating these two factors would lead to misjudgments of the tile's structural strength. Therefore, this step incorporates both the transparency parameter and the projected ellipse coverage weight into the accumulation process, ensuring that the coverage weight statistic more closely reflects the actual representation intensity of the tile from the current viewpoint.

[0144] In this embodiment, the calculation of the coverage weight statistic can be performed tile by tile. For a given tile, the list of Gaussian objects belonging to that tile is first read through the mapping structure established in S3.2.1; for each Gaussian object in the list, its projected coverage weight within that tile is calculated. The projected coverage weight can be determined based on the degree of overlap between the Gaussian object's projected ellipse and the tile region, specifically by using the effective coverage area ratio, the sampling point coverage ratio, or by directly calculating the pixel coverage mask generated in the forward rendering stage. Subsequently, the projected coverage weight is combined with the Gaussian object's transparency parameter to form the Gaussian object's coverage contribution value to the tile. Specifically, the combination method can be the product of the two, and the coverage contribution values ​​of all Gaussian objects belonging to that tile are accumulated to obtain the coverage weight statistic.

[0145] S3.2.3: Within each target tile region, based on the depth information of the Gaussian object belonging to the target tile region and the pixel-wise Alpha mixing order, the effective mixing weight distribution of the Gaussian object in the preset depth layer is statistically analyzed, and the depth mixing dispersion statistic is calculated based on the effective mixing weight distribution.

[0146] Specifically, coverage weight statistics alone cannot reflect the complexity of occlusion relationships within a tile. This is because multiple Gaussian objects may have high coverage within the same tile, but if these Gaussian objects are of similar depth and their blending order is stable, the region may be a smooth surface. Conversely, if Gaussian objects participate in blending across multiple depth layers, it usually corresponds to overlapping structures, thin-walled structures, hole boundaries, or occlusion rearrangements caused by viewpoint changes. This type of information is crucial in low-computational-power continuous modeling because such regions often require more detailed parameter updates or density refinement.

[0147] In this embodiment, the calculation of the depth blending dispersion statistic can be carried out as follows. For a certain tile, firstly, the list of Gaussian objects belonging to that tile and their center depth values ​​under the current view are read; then, a preset depth layer is established based on the depth range of the Gaussian objects within the current tile. The depth layer can adopt a fixed number of layers, for example, dividing the minimum to maximum depth range within the tile into 4, 6, or 8 layers, or an adaptive layering method based on depth sample density can be used. To facilitate cross-period comparison, this embodiment uses a fixed number of layers in the same modeling stage, and generates layer boundaries in each tile by linear mapping according to the current depth range. Then, combining the pixel-by-pixel alpha blending order and effective blending weight recorded in the forward rendering stage, the Gaussian objects participating in the blending at each pixel are assigned to the corresponding layer according to their depth layer, and their effective blending weights are accumulated into that depth layer to form the effective blending weight distribution of the depth layer of the tile. Effective blending weights can be understood as the weights by which Gaussian objects actually affect the final pixel color during pixel-by-pixel alpha blending. Therefore, they naturally include information on transparency and occlusion relationships, and can better reflect the actual degree of imaging participation than simple depth counting.

[0148] S3.2.4: Within each target tile region, extract the principal axis direction information of the projection ellipse of the Gaussian object belonging to the target tile region, and calculate the anisotropic direction consistency statistic based on the principal axis direction information;

[0149] Specifically, in the continuous modeling process of 3DGS, the edges of local structures, slender components, hole edges, or striped textures usually exhibit a relatively stable directional expression on the projection plane. This directionality is reflected in the distribution of the principal axis directions of the Gaussian object's projection ellipse. If the principal axis directions belonging to the Gaussian object within a tile are highly consistent, it often means that the area is composed of a relatively well-defined edge orientation or a regular surface. If the directional distribution is scattered or shows significant changes in adjacent periods, it may correspond to the exposure of new edges, structural rearrangement, or unstable fitting regions under viewpoint switching.

[0150] In this embodiment, for a specific tile, the list of Gaussian objects belonging to that tile and their projection ellipse principal axis orientation angles are first read through the mapping structure in S3.2.1. To avoid interference from low-contribution Gaussian objects in the orientation statistics, only Gaussian objects with coverage contribution values ​​higher than a preset lower limit within the tile are selected to participate in the orientation consistency calculation. This coverage contribution lower limit can be determined according to the quantile values ​​of the coverage contribution values ​​of all Gaussian objects belonging to the tile; for example, objects below the top 30% quantile values ​​are considered weak-contribution objects and ignored. Subsequently, the principal axis orientation angles participating in the statistics are divided into several orientation intervals according to a preset angular resolution, forming a tile-level orientation distribution. The orientation intervals can be divided at a granularity of 15 degrees, 22.5 degrees, or 30 degrees, depending on the object structure complexity and computing power budget. Since orientation has periodicity, orientations differing by 180 degrees can be considered equivalent orientations in the steps to avoid orientation reversal leading to statistical discontinuity. Afterward, the anisotropic orientation consistency statistics are calculated based on the concentration of the orientation distribution; the more concentrated the orientations, the higher the consistency, and the more dispersed the orientations, the lower the consistency.

[0151] S3.2.5: Combine the coverage weight statistic, the depth mixing dispersion statistic, and the anisotropic direction consistency statistic into the projection statistical spectrum;

[0152] In this embodiment, the combination process of the projection statistical spectrum can be performed as follows: First, the three types of statistics are normalized within the same period. The coverage weight statistics can be normalized based on the quantile normalization of the coverage weight distribution of all valid tiles in the current period; the depth mixing dispersion statistics can be normalized based on the quantile normalization of the dispersion distribution of all valid tiles in the current period; and the anisotropic direction consistency statistics can be normalized based on their own defined range or the current period distribution. Then, the normalized three types of statistics are written into the projection statistical spectrum structure corresponding to the tile in a fixed order, and a validity flag field, tile number, modeling period number, and a quality flag for subsequent comparison are added. The quality flag can be used to record the reliability of the statistics for the current period of the tile. For example, if the effective pixel ratio is insufficient, the number of participating Gaussian objects is too small, or the effective weight of the depth layer is insufficient, the tile can be marked as a low-reliability tile, reducing its structural variability contribution during the comparison in S3.3. For example, the threshold for the number of valid Gaussian objects participating in the direction statistics can be set to 3. When there are fewer than 3 Gaussian objects participating in the direction statistics for a certain tile, the direction consistency component is still recorded, but a low confidence mark is attached. The change of the direction component is attenuated during subsequent comparisons.

[0153] S3.3: Compare the projection statistical spectrum of each tile in the current modeling cycle with the projection statistical spectrum of each tile in the previous modeling cycle to obtain the structural variability of each tile, and perform residual gating weighting on the structural variability based on the tile residual statistics to obtain the complexity score of each tile.

[0154] Specifically, this application does not simply compare the differences between the statistics of two modeling periods. Instead, under the same tile coordinate reference, it makes the local projection expression state of the current period comparable to that of the previous period before determining the structural change.

[0155] It is understandable that in continuous modeling scenarios, although the same tile corresponds to the same statistical position on the image plane in adjacent modeling cycles, the set of Gaussian objects involved in the mixing, the order of occlusion, and the distribution of the projection ellipse direction may be replaced or rearranged. If the original statistical values ​​are directly compared, fluctuations caused by brightness fluctuations, local sampling jitter, or a small number of low-contribution Gaussian objects are easily misjudged as structural changes. Based on this, in this step, the projection statistical spectra of the current modeling cycle and the previous modeling cycle are first normalized at the same scale and the components are aligned, so that the coverage weight statistics, depth mixing dispersion statistics, and anisotropic direction consistency statistics are in a jointly comparable state. Then, the component change is established for each component and synthesized into a tile-level structural change degree according to a preset weight. Those skilled in the art will understand that the component alignment processing can adopt a unified dimension mapping, historical statistical range normalization, or dynamic normalization based on a sliding window, as long as the stability of the component comparison between adjacent modeling cycles can be guaranteed. This application does not limit this.

[0156] In this embodiment, the calculation process of structural variability can be carried out as follows: For each tile, firstly, the projection statistical spectrum of the current modeling cycle and the corresponding tile projection statistical spectrum saved in the previous modeling cycle are read; then, the changes in coverage weight statistics, depth mixing dispersion, and anisotropic direction consistency are calculated respectively, and noise suppression processing is performed on each component change to weaken the local spike effects caused by fluctuations in a small number of edge pixels. Noise suppression processing can adopt methods such as segmented truncation, sliding smoothing, or lower limit filtering of changes. For example, component changes below a preset lower limit of change are recorded as invalid changes to avoid continuous updates triggered by small statistical drifts in flat areas. Afterwards, the effective component changes are weighted and synthesized according to the component weights to obtain the structural variability of the tile, wherein the component weights can be preset according to the modeling object type and acquisition characteristics, or can be adaptively adjusted according to the statistical performance of historical modeling cycles. For example, when the object to be modeled has many thin walls, holes, or stacked structures, the weight of the depth mixing dispersion component can be increased; when the surface of the object to be modeled has a large number of slender structures with clear edge directions, the weight of the anisotropic direction consistency component can be increased. Through the above processing, the structural variability no longer only represents whether the statistical value has changed, but can represent whether the tile has undergone local structural reconstruction or occlusion relationship changes that are worth including in density scheduling from the current perspective, thus providing a basic quantity with engineering significance for subsequent complexity scoring.

[0157] Furthermore, to ensure the continuity of cross-cycle comparisons, at the end of each modeling cycle, the current tile projection statistical spectrum can be written into the historical cache as a reference value for the next modeling cycle, and the change trend of the tile in the most recent few cycles can be recorded simultaneously for subsequent stabilization processing of periodic jitter regions. For example, the sensitivity to change can be reduced for tiles whose directional components fluctuate back and forth in multiple consecutive cycles but whose residuals remain low, so as to reduce invalid encryption triggers.

[0158] Furthermore, when applying residual gating weights to the degree of structural variability based on tile residual statistics, this embodiment does not simply multiply the residual magnitude directly by the degree of structural variability. Instead, it first decomposes the tile residual statistics into two gating components: residual energy response and residual gradient response. Then, it modulates the degree of structural variability in layers according to gating rules to obtain a more stable complexity score. Specifically, the following processing logic can be adopted: When a tile has high structural variability but low residual energy and low residual gradient, it indicates that although the tile has undergone statistical changes in projection, the current rendering result still maintains good consistency with the image data. In this case, the structural variability of the tile is attenuated with a low gating weight to avoid prematurely triggering high-cost splitting. When a tile has high structural variability, high residual energy, and a high residual gradient, it usually corresponds to newly exposed local edges, occlusion switching, or areas with insufficient detail expression. In this case, a high gating weight is applied to the structural variability to improve the complexity score. When the structural variability is moderate but the residual energy is high and the residual gradient is low, it often corresponds to areas with insufficient color fitting or inconsistent local lighting. A moderate gating weight can be applied to the structural variability to make this area more inclined to fine-tune parameters rather than directly encrypt them in subsequent steps. For ease of engineering implementation, the gating weight can be obtained through threshold interval mapping. The threshold can be determined based on the tile residual distribution statistically obtained during the initialization phase or the first few modeling cycles. For example, the residual energy statistics of all valid tiles in the current modeling cycle can be sorted from smallest to largest, and the top 30% quantiles can be used as the low residual threshold and the top 80% quantiles as the high residual threshold. The residual gradient statistics can also be determined using quantiles to set the low and high gradient thresholds. Then, corresponding gating weights can be selected based on the interval the tile falls into; for example, low gating weights could be 0.3 to 0.5, medium gating weights 0.6 to 0.8, and high gating weights 0.9 to 1.2. Using quantiles instead of fixed absolute values ​​helps adapt to variations in the residual statistical range under different image resolutions, exposure conditions, and object scales.

[0159] S3.4: Divide complex regions and flat regions based on the complexity score, wherein the complex region includes the tile region corresponding to the tile whose complexity score meets the first threshold condition, and the flat region includes the tile region corresponding to the tile whose complexity score meets the second threshold condition, and the first threshold is greater than the second threshold.

[0160] In this embodiment, the division between complex and flat regions can be achieved as follows: After calculating the complexity scores of all tiles in the current modeling cycle, the complexity scores of all valid tiles are first statistically analyzed to form a complexity distribution sequence for the current cycle. Then, a first threshold and a second threshold are determined using a preset threshold strategy, and a state determination is performed for each tile. The threshold strategy can be a fixed threshold or an adaptive threshold that adapts to the current cycle's score distribution. To adapt to changes in the magnitude of complexity scores under different objects, perspectives, and residual levels, this embodiment uses an adaptive threshold strategy, i.e., determining the threshold range based on the quantile values ​​of the current cycle's complexity score distribution.

[0161] For example, all valid tile complexity scores can be sorted from smallest to largest. A quantile between the top 75th and top 90th quantiles can be used as the first threshold candidate interval, and a quantile between the top 20th and top 40th quantiles can be used as the second threshold candidate interval. This can then be fine-tuned by combining the region proportion constraints from the previous modeling cycle to avoid excessively large abrupt changes in the proportion of complex or flat regions within a single cycle. For instance, in a modeling cycle with 500 valid tiles, if the 425th tile has a score of 0.78 and the 450th tile has a score of 0.84 after sorting, the first threshold can be set to 0.80; if the 125th tile has a score of 0.32 and the 200th tile has a score of 0.41, the second threshold can be set to 0.36. At this point, tiles with a complexity score of 0.80 or higher are classified into complex regions, tiles with a complexity score of 0.36 or higher are classified into flat regions, and tiles with a complexity score between 0.36 and 0.80 are retained as intermediate regions. By using quantile values ​​combined with region proportion constraints, the threshold can be automatically adjusted according to the current image content and modeling progress, avoiding segmentation distortion caused by using the same threshold in the initial model building stage and the later convergence stage. Those skilled in the art will understand that the above quantile ratios and example values ​​are only for illustrating the threshold determination method; in practical applications, they can be configured according to image resolution, acquisition noise level, and computing power budget.

[0162] For example, the following numerical example is provided to illustrate the calculation process of complexity information in the embodiments of this application. This example is only used to explain the computational link and value relationship between tile residual statistics, projection statistical spectrum, structural variability, residual gating, and complexity score. The selected parameters and values ​​are illustrative and do not represent actual calibration results or engineering recommendations.

[0163] This example assumes the current modeling cycle corresponds to an image resolution of 640×480, using a 32×32 pixel tile partitioning rule. The image plane can then form 20 columns and 15 rows, totaling 300 tiles. For ease of demonstration, only three consecutive tiles are selected as examples, denoted as tile A, tile B, and tile C. Tile A is located in the transition region of the target object's edge, tile B is located in the smooth surface region, and tile C is located in the region where the hole edge overlaps with the thin wall. It should be noted that the current modeling cycle has completed forward rendering and retained the pixel-by-pixel blending process, while the previous modeling cycle has cached the projection statistical spectrum of the corresponding tiles. Therefore, this example begins from S3.1.

[0164] Specifically, in the pixel residual map construction stage, the rendered image output from the forward rendering and the current viewpoint image data are already in the same coordinate system. The residual is calculated pixel by pixel to form the pixel residual map. Taking tile A as an example, this tile is 32×32 pixels in size, with a total of 1024 pixels, of which 980 are effective pixels. After cropping the pixel residuals, the residual values ​​are mainly distributed between 0.02 and 0.85. Tile B has 1010 effective pixels, and the pixel residuals are mainly distributed between 0.01 and 0.32. Tile C has 944 effective pixels, and the pixel residuals are mainly distributed between 0.03 and 0.96, with many high residual pixels near the hole edge. A fixed mesh method is used for tile division. The current modeling cycle and the previous modeling cycle maintain the same tile coordinate reference. Therefore, tiles A, B, and C correspond to the same image plane position in both modeling cycles, requiring no additional registration. When calculating the residual energy statistics for tiles, robust aggregation is performed on the effective pixel residuals within each tile. This example uses truncated mean aggregation, with a truncation ratio of 5% at the top and bottom. Taking tile A as an example, after sorting the 980 effective pixel residuals, the lowest and highest 49 are removed, and the average of the remaining 882 pixel residuals is calculated, yielding a residual energy statistics of 0.41. For tile B, after removing the top and bottom 5%, the residual energy statistics are 0.12; the corresponding value for tile C is 0.53.

[0165] Furthermore, in the residual gradient statistics calculation stage, discrete gradient operators are performed on the pixel residual sub-images of each tile, and robust aggregation of gradient magnitudes is conducted. This example uses conventional local difference gradients and takes the top 80% quantiles as the residual gradient statistics. Taking tile A as an example, after sorting the residual gradient magnitudes, the top 80% quantile corresponds to 0.36, indicating that this tile has relatively obvious edge-type error variations; tile B corresponds to 0.09, indicating that the error distribution of this tile is relatively gentle; tile C corresponds to 0.48, indicating that the error within this tile varies drastically in space, typically corresponding to hole edges or occlusion switching areas. Subsequently, the residual energy statistics and residual gradient statistics are normalized for the same period. Assuming that among all 300 tiles in the current modeling period, the 5th quantile of the residual energy statistics is 0.05 and the 95th quantile is 0.60, and the 5th quantile of the residual gradient statistics is 0.04 and the 95th quantile is 0.55, then the normalized residual energy for tile A is approximately 0.65, and the normalized residual gradient is approximately 0.63; for tile B, approximately 0.13 and 0.10 respectively; and for tile C, approximately 0.87 and 0.86 respectively. Since the current modeling stage is in the middle, this example uses a residual energy weight of 0.45 and a residual gradient weight of 0.55. Therefore, the residual statistics of the fused tiles are approximately: tile A approximately 0.64, tile B approximately 0.11, and tile C approximately 0.86. Tile C was identified as a high residual tile, tile B as a low residual tile, and tile A as a medium-to-high residual tile. During the projection statistical spectrum construction phase, the projection contribution of Gaussian objects to the tiles from the current perspective needs to be transformed into a statistically comparable representation across periods. Taking tile A as an example, based on the intersection relationship between the projection ellipse and the tile, 14 Gaussian objects were identified as belonging to this tile; 9 for tile B; and 21 for tile C. After minimum coverage ratio filtering, 12 valid Gaussian objects were retained for tile A, 7 for tile B, and 18 for tile C.

[0166] Taking tile A as an example, the cumulative coverage contribution of 12 Gaussian objects within this tile is 6.8; for tile B it is 3.1; and for tile C it is 11.4. Assuming the 5th quantile of the coverage weight statistics for all valid tiles in the current period is 1.2 and the 95th quantile is 12.0, then the normalized coverage weight statistics for tile A are approximately 0.52, for tile B approximately 0.18, and for tile C approximately 0.95. This result reflects that tile C is represented more densely by Gaussian objects from the current perspective and is more likely to be located in a structurally complex region.

[0167] Furthermore, when calculating the depth blending dispersion statistics, the Gaussian objects within each tile are layered according to the current view depth. This example uniformly uses 6 layers of depth layering. Taking tile A as an example, based on the pixel-by-pixel alpha blending process statistics, the effective blending weight distribution of the 6 layers is 0.44, 0.27, 0.16, 0.08, 0.03, and 0.02. It can be seen that the weights are mainly concentrated in the first 3 layers, with a moderate degree of dispersion. The distribution of tile B is 0.71, 0.18, 0.07, 0.03, 0.01, and 0.00, with the weights highly concentrated in the first two layers and a low degree of dispersion. The distribution of tile C is 0.23, 0.20, 0.17, 0.15, 0.14, and 0.11, with high participation from multiple layers and a high degree of dispersion. After normalizing the quantiles of the depth-mixed dispersion statistics of all tiles in the current period, we assume that tile A corresponds to 0.46, tile B corresponds to 0.12, and tile C corresponds to 0.91.

[0168] Furthermore, the anisotropic directional consistency statistic is used to describe whether the dominant structural orientation within a tile is concentrated. Taking tile A as an example, there are 9 Gaussian objects participating in the directional statistics, while the remaining 3 are ignored because their coverage contribution is below the 30% quantile lower limit of the tile. The directional distribution is concentrated in two adjacent angular intervals, resulting in high directional consistency, which is 0.74 after normalization. Tile B has 4 Gaussian objects participating in the directional statistics, with a more dispersed directional distribution, resulting in 0.33 after normalization. Tile C has 15 Gaussian objects participating in the directional statistics, with the directional distribution spanning multiple intervals and overlapping of structural orientations between layers, resulting in 0.41 after normalization. It should be noted that high directional consistency does not directly equate to high complexity; its significance lies in providing a basis for determining the stability of the directional structure in cross-period comparisons. For example, the high directional consistency of tile A indicates the presence of a relatively clear edge orientation within the tile in the current period; if this component changes significantly in the next period, it can be determined that the structural exposure relationship has been adjusted.

[0169] Furthermore, the coverage weight statistics, deep mixing dispersion statistics, and anisotropic direction consistency statistics are written into the projection statistical spectrum in a fixed order, and a quality label is attached. Assume the projection statistical spectra of tiles A, B, and C in the current period are: Tile A = [0.52, 0.46, 0.74], Tile B = [0.18, 0.12, 0.33], Tile C = [0.95, 0.91, 0.41]. The corresponding projection statistical spectra cached in the previous modeling period are: Tile A = [0.39, 0.31, 0.81], Tile B = [0.16, 0.10, 0.35], Tile C = [0.73, 0.62, 0.57]. Tile A shows improvements in both coverage weight and depth mixing dispersion, while directional consistency decreases slightly; tile B shows little overall change; tile C shows significant improvements in both coverage weight and depth mixing dispersion, while directional consistency decreases, suggesting that new occlusion may have occurred and local structural rearrangement may have occurred in this tile.

[0170] During the structural variability calculation phase, the projected statistical spectrum components of the current period and the previous period are compared one by one, and variation denoising processing is performed. In this example, the lower limit for the variation of the coverage weight component is set to 0.03, the lower limit for the variation of the depth mixing dispersion is 0.04, and the lower limit for the variation of the orientation consistency is 0.05. Variations below these lower limits are considered invalid. Therefore, the component variations of tile A are 0.13, 0.15, and 0.07, all above the lower limit; the component variations of tile B are 0.02, 0.02, and 0.02, all below the lower limit, and are recorded as 0 after denoising; the component variations of tile C are 0.22, 0.29, and 0.16, all above the lower limit. Considering that the current object includes thin walls and hole edge structures, this example sets the structural variability component weights as follows: coverage weight 0.30, depth mixing dispersion 0.45, and orientation consistency 0.25. After being synthesized according to this weight, the structural variability of tile A is approximately 0.12, that of tile B is 0.00, and that of tile C is approximately 0.23. This result indicates that the local projection representation of tile C under the current viewpoint changes more significantly compared to the previous period, tile A exhibits moderate change, and tile B remains relatively stable. The structural variability is not directly used as a complexity score but is incorporated into the residual gating weighting stage. This example requires determining the residual gating threshold first. Assuming that the residual statistics of all 300 valid tiles in the current period are sorted from smallest to largest, the 90th value is 0.24 (top 30th percentile), and the 240th value is 0.68 (top 80th percentile), then the low residual threshold is set to 0.24, and the high residual threshold is set to 0.68; simultaneously, it is assumed that the low threshold for the residual gradient normalization statistics is 0.22, and the high threshold is 0.70. Based on the previous results of this example, the residual energy of tile A is normalized to 0.65 and the residual gradient is normalized to 0.63, both of which are in the medium-high range but do not exceed the high threshold simultaneously. Therefore, the gating level is determined to be medium gating. Both of the residual energy and residual gradient of tile B are below the low threshold, so it is determined to be low gating. The residual energy and residual gradient of tile C both exceed the high threshold, so it is determined to be high gating. In this example, the weight of low gating is set to 0.4, the weight of medium gating is 0.7, and the weight of high gating is 1.1. Then the complexity scores after gating weighting are as follows: tile A is 0.12×0.7≈0.084, tile B is 0.00×0.4=0, and tile C is 0.23×1.1≈0.253. Tile C has a large structural change, and the residuals support this change, so it has the highest complexity score. Tile A has a structural change, but the residuals support it to a moderate degree, so its complexity score is in the middle range. Even though tile B has slight statistical fluctuations, it will not enter the high complexity region because the degree of structural change is suppressed and the residuals are low.

[0171] It should be noted that the above numerical examples only demonstrate the calculation process of three tiles in a single modeling cycle. In practical applications, the same processing chain can be executed in parallel for all tiles, and the projection statistics spectrum and complexity score results of the current cycle can be written to a cache at the end of each modeling cycle for cross-cycle comparison and threshold fine-tuning in the next modeling cycle. Those skilled in the art will understand that the tile size, quantile threshold, gating weight, number of depth layers, and weights of each component in the examples can all be configured according to image resolution, object structure scale, acquisition noise level, and computing power budget. As long as the processing logic of residual statistics, projection statistics, cross-cycle change determination, and residual gating is completed at the tile level, the complexity information construction process described in the embodiments of this application can be realized.

[0172] Next, we will further elaborate on the technical content of the Gaussian density allocation method in this application.

[0173] In one example, performing adaptive Gaussian density assignment on the current Gaussian field model based on the complexity information includes:

[0174] S4.1: Determine the density scheduling state for each tile region based on the complexity score. The density scheduling state includes at least an encrypted state, a fine-tuning state, and a sparse state. The encrypted state corresponds to a tile region whose complexity score meets the first threshold condition. The sparse state corresponds to a tile region whose complexity score meets the second threshold condition. The fine-tuning state corresponds to a tile region whose complexity score is between the first threshold and the second threshold.

[0175] S4.2: Establish a mapping relationship between tiles and Gaussian objects based on the density scheduling state to determine the set of encrypted candidate Gaussian objects, the set of fine-tuned candidate Gaussian objects, and the set of sparse candidate Gaussian objects, wherein the mapping relationship is determined at least based on the overlap between the projected ellipse of the Gaussian object and the tile region.

[0176] In this embodiment, the mapping relationship from tiles to Gaussian objects can reuse the target tile region mapping structure established in stage S3.2.1, avoiding redundant calculations while maintaining coordinate consistency between preceding and subsequent steps. Specifically, the current modeling cycle's tile state diagram and the mapping cache from tile indices to the Gaussian object list are first read. Then, for each Gaussian object, the state labels of its assigned tiles and the projection overlap in each tile are summarized to generate a state attribution score for that Gaussian object. Since the same Gaussian object may simultaneously cover encrypted state tiles, fine-tuned state tiles, and sparse state tiles, to avoid conflicts, this step does not use a simple "any hit equals attribution" approach, but instead makes a joint decision based on overlap and effective mixing weights. For example, the overlap of a Gaussian object in encrypted state tiles, fine-tuned state tiles, and sparse state tiles can be accumulated, and then corrected according to the effective mixing weight of the Gaussian object in each tile to finally obtain three types of state assignment scores. The class with the highest score is taken as the primary assignment state of the Gaussian object. The effective mixing weight refers to the characterization parameter of the Gaussian object's actual imaging contribution to the corresponding tile area. It can be determined by at least one or a combination of the Gaussian object's transparency contribution, color mixing contribution, visibility ratio after depth sorting, and residual contribution in the cumulative transmittance of rendering. It is used to reflect whether the Gaussian object has an effective impact on the tile in the final rendering result, even though it projects and covers the tile. For example, when a Gaussian object is heavily occluded by a foreground Gaussian object within a corresponding tile, or its contribution to pixel color is lower than a preset threshold after its transparency decays, its effective blending weight can be set to a lower value; when a Gaussian object has high visibility, high opacity contribution, or large cumulative color contribution within a corresponding tile, its effective blending weight can be set to a higher value. The effective blending weight can be obtained by directly reusing the Gaussian object contribution parameters recorded during the tile rendering or tile attribution analysis in stage S3.2.1, or it can be obtained by further normalizing the pixel contribution statistics of the Gaussian object within the target tile.

[0177] The overlap refers to the degree of spatial coverage association between the projection ellipse of the Gaussian object and the tile region, used to characterize the strength of geometric coincidence between the Gaussian object and the tile on the two-dimensional projection plane. The overlap can be determined by the proportion of the intersection area between the two-dimensional projection ellipse formed by the Gaussian object after being projected by the camera and the corresponding tile boundary. For example, the intersection area between the projection ellipse and the rectangular area of ​​the tile can be divided by the area of ​​the projection ellipse and the area of ​​the tile, or by the union area of ​​the two, to obtain the corresponding normalized overlap index; alternatively, the proportion of the number of Gaussian object pixels effectively covered by the object that falls within the tile to the total number of Gaussian object pixels covered by the object can be used as the overlap. The overlap can be derived from the statistical results of the Gaussian object projection bounding range, ellipse coverage range, and the set of tiles hit when establishing the target tile region mapping structure in stage S3.2.1, thus eliminating the need to repeatedly perform projection intersection calculations.

[0178] Finally, for each Gaussian object, the tile overlap degree and effective mixing weight corresponding to the Gaussian object are multiplied in the encrypted state tile set, the fine-tuned state tile set, and the sparse state tile set, respectively, to calculate the Gaussian object's attribution score to each state.

[0179] S4.3: For the set of encrypted candidate Gaussian objects, perform a directional splitting operation to generate at least one child Gaussian object, wherein the directional splitting operation includes: determining the splitting direction based on the principal direction of the residual gradient and the principal axis direction of the projection ellipse, and perturbing the spatial position parameter and the shape parameter in the splitting direction to obtain the initialization parameters of the child Gaussian object, while making the child Gaussian object inherit the prior values ​​of the color representation parameter and the transparency parameter;

[0180] Specifically, the set of encrypted candidate Gaussian objects corresponds to high-complexity regions where structural changes and residual gating both point to the current modeling cycle. These regions typically exhibit newly exposed local edges, switching of aperture occlusion, overlapping of thin-walled structures, or insufficient local detail representation. If only parameter fine-tuning is performed on existing Gaussian objects in these regions, a single Gaussian object may simultaneously represent multiple local structural directions or depth layers, resulting in rendering residuals remaining at a medium-to-high level for an extended period. Therefore, this step employs directional splitting rather than random splitting to divide the encrypted candidate Gaussian objects into sub-Gaussian objects along directions that better align with the current error pattern, thereby improving local representation resolution. This application uses the principal direction of the residual gradient and the principal axis direction of the projection ellipse to jointly determine the splitting direction because the principal direction of the residual gradient reflects the main direction of error change in the image plane from the current viewpoint, while the principal axis direction of the projection ellipse reflects the geometric stretching direction of the Gaussian object from the current viewpoint. Combining these two aspects helps prevent the splitting direction from deviating from the actual error trajectory.

[0181] In this embodiment, the directional splitting operation can be performed on an object-by-object basis. Specifically, a candidate Gaussian object is first read from the encrypted candidate Gaussian object set, and its primary and second-highest contributing tiles are obtained. Then, the residual gradient map is read within the primary tile of the Gaussian object, and the local residual gradient direction distribution corresponding to the effective coverage area of ​​the Gaussian object is extracted. The principal direction of the residual gradient of the object is obtained using a coverage contribution weighting method. Simultaneously, the principal axis direction information of the projection ellipse of the Gaussian object under the current viewpoint is read, and the splitting direction is determined by combining it with a preset direction fusion rule. The direction fusion rule can adopt a consistency-first strategy: when the angle between the principal direction of the residual gradient and the principal axis direction of the projection ellipse is small, for example, less than 30 degrees, the splitting direction is determined along the common direction of both; when the angle is large, the principal direction of the residual gradient is preferred, and the splitting amplitude is limited to avoid excessive splitting when the error direction is unclear. After determining the splitting direction, the spatial position parameters and shape parameters are perturbed in this direction to generate initialization parameters for at least one sub-Gaussian object. For example, the original Gaussian object can be divided into two sub-Gaussian objects along the splitting direction in the spatial direction corresponding to the projection. One of them is offset in the positive direction and the other is offset in the negative direction. The offset amount can be determined by the scale range of the original Gaussian object in this direction, the gradient intensity of the master tile residual, and the current modeling stage. The shape parameters are appropriately shrunk in the splitting direction and maintained or slightly expanded in the vertical direction so that the sub-Gaussian objects can respectively take on the expression of different sub-regions of the original object in the local structure.

[0182] Furthermore, the offset can be adaptively controlled in conjunction with the current modeling stage. In the early stages of modeling or during rapid structural growth, the overall scene geometry is not yet fully unfolded, and highly complex regions often require more aggressive capacity replenishment. In this case, a slightly larger offset is permissible to enhance the unfolding capability of the new structure. In the later stages of modeling or during the optimization and stabilization phase, the overall structure is basically formed, and the purpose of splitting is more to compensate for local details and boundary errors. At this point, the offset should be relatively convergent to prevent excessive disturbance to the already stable regions. In other words, the offset can be embodied as a control strategy that gradually refines as the modeling process progresses.

[0183] S4.4: For the set of fine-tuning candidate Gaussian objects, perform parameter-constrained updates, wherein the parameter-constrained updates include at least: updating the color representation parameter and the transparency parameter, while keeping the spatial position parameter and the shape parameter unchanged;

[0184] Specifically, the set of candidate Gaussian objects for fine-tuning corresponds to tile regions with complexity scores in the middle range. These regions typically exhibit some errors or slight structural changes in the current modeling cycle, but these are insufficient to support object splitting; they are also unsuitable for merging and compression, as this could compromise the existing fitting quality. Updating the spatial location and shape parameters of these objects could cause the geometry to drift repeatedly between adjacent modeling cycles, especially in scenarios with rapid viewpoint changes or fluctuating lighting conditions, leading to incorrect mapping of appearance errors to geometric changes. Therefore, this step employs a parameter-constrained update strategy for candidate Gaussian objects, prioritizing the updating of color representation and transparency parameters while keeping the spatial location and shape parameters unchanged. This allows these objects to primarily handle appearance and visibility corrections without introducing additional geometric perturbations. This approach helps control the degrees of freedom in optimization under low computational power conditions and reduces ineffective iteration overhead.

[0185] In this embodiment, the parameter-constrained update can be performed directly after the forward rendering and residual calculation of the current modeling cycle. Specifically, the set of candidate Gaussian objects for fine-tuning and their master tile information are first read, and the color contribution, effective blending weights, and pixel residual correlation information of these Gaussian objects under the current viewpoint are extracted from the forward rendering process. Subsequently, a constrained update term is constructed for each candidate Gaussian object for fine-tuning. When updating the color representation parameter, the spherical harmonic coefficients or equivalent viewpoint-related color parameters can be locally corrected based on the color residual distribution and viewing direction information within the pixels covered by the Gaussian object. When updating the transparency parameter, adjustments are made based on the effective blending weight deviation of the object in pixel-by-pixel alpha blending, making its visibility contribution under the current viewpoint closer to the actual image response. To prevent the fine-tuned object from accumulating excessive deviations over multiple modeling cycles, this step can set an upper limit on the parameter update amplitude for each cycle. For example, component-level truncation can be used for color representation parameter updates, and variation amplitude limits and effective range clipping can be used for transparency parameter updates. For example, in the later stages of modeling, the single-cycle transparency update amplitude can be limited to 10% to 20% of the current value to maintain a smooth transparency evolution.

[0186] S4.5: For the sparse candidate Gaussian object set, perform an absorption and merging operation, wherein the absorption and merging operation includes: determining a cluster of Gaussian objects that satisfy similarity constraints within the same tile region, the similarity constraints including at least color similarity constraints, principal axis direction similarity constraints, and depth consistency constraints, and merging the Gaussian object clusters into a merged Gaussian object, wherein the spatial position parameters, shape parameters, transparency parameters, and color representation parameters of the merged Gaussian object are obtained by weighting the parameters of the Gaussian object cluster according to the effective mixing weights;

[0187] Specifically, the sparse candidate Gaussian object set corresponds to tile regions with low complexity scores and typically weak residual support. These regions are often flat surfaces, areas with weak variations in repetitive textures, or relatively stable fitting regions. Retaining a large number of Gaussian objects in such regions for an extended period not only consumes GPU memory but also continuously generates computational burden during the projection, sorting, blending, and statistical stages of each subsequent modeling cycle, conflicting with the goal of low-computing-power deployment. Therefore, this step introduces an absorption-merge operation. Under the premise of satisfying similarity constraints, multiple Gaussian objects with similar expressions, consistent depths, and similar orientations within the same tile region are merged into a single merged Gaussian object to compress the object size and reduce the computational pressure in subsequent modeling cycles. Absorption-merge is used instead of direct deletion to preserve the main expressive power of the original region as much as possible while reducing the number of objects.

[0188] In this embodiment, the absorption and merging operation is performed on a tile region basis, and candidate clustering is prioritized within sparse state tiles. Specifically, the list of sparse state tiles and their associated Gaussian object sets are first read. For each sparse state tile, Gaussian objects whose primary state is sparse and whose coverage contribution exceeds the minimum participation threshold are selected as merging candidates. Subsequently, candidate object clusters are constructed based on color similarity constraints, principal axis similarity constraints, and depth consistency constraints. Color similarity constraints prevent objects with significant appearance differences from being incorrectly merged; specifically, the dominant color contribution after the color representation parameters are expanded from the current viewpoint can be compared. Principal axis similarity constraints ensure that merged objects retain their original regional directional structural features; the directional angle threshold can be configured from 15 to 30 degrees. Depth consistency constraints prevent objects from being merged across layers; the depth difference threshold can be set based on the current tile depth layer width, for example, not exceeding 0.5 to 1 times the single-layer depth span of the tile. Objects satisfying the above constraints can form the same Gaussian object cluster. If an object can be assigned to multiple object clusters simultaneously, it is preferentially assigned to the cluster with the smaller color difference and the smaller depth difference.

[0189] In yet another example, embodiments of this application provide a low-computing-power, fast 3D modeling system based on 3DGS, the system comprising:

[0190] The periodic scheduling module is used to acquire image data of the object to be modeled from the corresponding viewpoint of each modeling cycle, and maintain the current Gaussian field model on the graphics processing unit. The current Gaussian field model is the updated Gaussian field model of the previous modeling cycle.

[0191] The complexity evaluation module performs forward rendering based on the image data and the current Gaussian field model to obtain a rendered image, determines residual information based on the rendered image and the image data, and determines complexity information based on the residual information and the Gaussian object set corresponding to the current Gaussian field model, so as to divide complex regions and flat regions.

[0192] The modeling output module is used to perform adaptive Gaussian density allocation on the current Gaussian field model according to the complexity information to obtain the updated Gaussian field model for the current modeling cycle, and to complete the 3D modeling output based on the updated Gaussian field model.

[0193] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A low-computing-power, fast 3D modeling method based on 3DGS, applied to a graphics processing unit, characterized in that: The graphics processing unit is configured with a current Gaussian field model, which is the updated Gaussian field model of the graphics processing unit in the previous modeling cycle. Each modeling cycle corresponds to image data of the object to be modeled from one viewpoint. The method includes: Obtain the image data of the object to be modeled in the current modeling cycle; Based on the image data and the current Gaussian field model, forward rendering is performed to obtain a rendered image, and residual information is determined based on the rendered image and the image data; Complexity information is determined based on the residual information and the Gaussian object set corresponding to the current Gaussian field model. The complexity information is used to characterize the local geometric detail complexity of the object to be modeled in the current modeling cycle and to divide the complex region into a flat region. Each Gaussian object in the Gaussian object set is associated with at least spatial position parameters, shape parameters, transparency parameters, and color representation parameters. Based on the complexity information, an adaptive Gaussian density allocation is performed on the current Gaussian field model to obtain an updated Gaussian field model for the current modeling cycle, and 3D modeling is performed based on the updated Gaussian field model.

2. The low-computing-power rapid 3D modeling method based on 3DGS according to claim 1, characterized in that, If the current modeling cycle is the first modeling cycle, the construction process of the current Gaussian field model includes: Acquire image data from the viewpoint corresponding to the first modeling cycle, and perform feature extraction based on the image data to obtain a set of feature points; Based on the set of feature points, camera pose is solved to determine the camera parameters for the first modeling cycle, and an initial geometric reference is constructed based on the camera parameters; An initial Gaussian object set is generated based on the initial geometric reference, wherein the spatial position parameters of each Gaussian object in the initial Gaussian object set are determined by the initial geometric reference, the shape parameters are determined by a preset covariance initialization strategy, the transparency parameters are determined by a preset initial opacity value, and the color representation parameters are determined by the pixel color of the image data at the corresponding projection position.

3. The low-computing-power rapid 3D modeling method based on 3DGS according to claim 1, characterized in that, Perform forward rendering based on the image data and the current Gaussian field model, including: In the current modeling cycle, obtain the camera parameters corresponding to the image data, and based on the camera parameters, project each Gaussian object in the Gaussian object set from three-dimensional space to the image plane to obtain the projection ellipse parameters of each Gaussian object on the image plane. For the pixels on the image plane, the projected ellipses are sorted according to the depth information of each Gaussian object, and pixel-by-pixel alpha blending is performed based on the transparency parameter to generate the rendered image.

4. The low-computing-power rapid 3D modeling method based on 3DGS according to claim 3, characterized in that, The determination of the residual information includes: During the generation of the rendered image, the color contribution of each Gaussian object under the current viewpoint is determined based on the color representation parameters, wherein the color contribution includes at least spherical harmonic coefficients used to characterize the color change of the viewpoint; After generating the rendered image, pixel-level residuals are calculated based on the rendered image and the image data, and the pixel-level residuals are aggregated using the color contribution to obtain the residual information.

5. The low-computing-power rapid 3D modeling method based on 3DGS according to claim 1, characterized in that, Complexity information is determined based on the residual information and the set of Gaussian objects corresponding to the current Gaussian field model, including: The residual information between the rendered image and the image data is aggregated according to a preset tile division rule to obtain the tile residual statistics of each tile, wherein the tile residual statistics include at least residual energy statistics and residual gradient statistics. The Gaussian projection contributions of each tile are aggregated based on the Gaussian object set to obtain the projection statistical spectrum of each tile. The projection statistical spectrum includes at least the coverage weight statistic, the depth mixing dispersion statistic, and the anisotropic direction consistency statistic. The projection statistical spectrum of each tile in the current modeling cycle is compared with the projection statistical spectrum of each tile in the previous modeling cycle to obtain the structural variability of each tile. Based on the residual statistics of the tile, the structural variability is weighted by residual gate to obtain the complexity score of each tile. Based on the complexity score, complex regions and flat regions are divided. The complex region includes the region corresponding to the tile whose complexity score meets the first threshold condition, and the flat region includes the region corresponding to the tile whose complexity score meets the second threshold condition, wherein the first threshold is greater than the second threshold.

6. The low-computing-power rapid 3D modeling method based on 3DGS according to claim 5, characterized in that, The residual information between the rendered image and the image data is aggregated according to a preset tile division rule, including: Based on the residual information and the image data, a pixel residual map is calculated. Based on the preset tile division rules, the pixel residual map is divided into multiple tile regions, wherein each tile region corresponds to a set of pixels of a preset size; Within each tile region, pixel residuals are calculated for the pixel set, and robust aggregation is performed on the pixel residuals to obtain the residual energy statistics of the tile region. The robust aggregation includes at least one of mean aggregation, quantile aggregation, and truncated mean aggregation. Within each tile region, a gradient operator is performed on the pixel residual to obtain a residual gradient map, and the residual gradient maps are aggregated to obtain a residual gradient statistic for the tile region. The residual gradient statistic is used to characterize the spatial drasticness of the residual change within the tile region. The residual energy statistic and the residual gradient statistic are weighted to obtain the tile residual statistic.

7. The low-computing-power rapid 3D modeling method based on 3DGS according to claim 5, characterized in that, The Gaussian projection contributions of each tile are aggregated based on the Gaussian object set to obtain the projection statistical spectrum of each tile, including: Determine the target tile region for each Gaussian object; Within each target tile region, for the Gaussian object belonging to the target tile region, the coverage weight statistic is accumulated based on the transparency parameter and the projection ellipse coverage weight; Within each target tile region, based on the depth information and pixel-by-pixel alpha mixing order of the Gaussian object belonging to the target tile region, the effective mixing weight distribution of the Gaussian object in the preset depth layer is statistically analyzed, and the depth mixing dispersion statistic is calculated based on the effective mixing weight distribution. Within each target tile region, extract the principal axis direction information of the projection ellipse of the Gaussian object belonging to the target tile region, and calculate the anisotropic direction consistency statistic based on the principal axis direction information; The coverage weight statistic, the depth mixing dispersion statistic, and the anisotropic direction consistency statistic are combined to form the projection statistical spectrum.

8. The low-computing-power rapid 3D modeling method based on 3DGS according to claim 5, characterized in that, Based on the complexity information, perform adaptive Gaussian density assignment on the current Gaussian field model, including: Based on the complexity score, the density scheduling state of each tile region is determined, and the density scheduling state includes at least an encrypted state, a fine-tuning state, and a sparse state. A mapping relationship between tiles and Gaussian objects is established based on the density scheduling state to determine the set of encrypted candidate Gaussian objects, the set of fine-tuned candidate Gaussian objects, and the set of sparse candidate Gaussian objects. The mapping relationship is determined at least based on the overlap between the projected ellipse of the Gaussian object and the tile region. For the set of encrypted candidate Gaussian objects, a directional splitting operation is performed to generate at least one child Gaussian object. The directional splitting operation includes: determining the splitting direction based on the principal direction of the residual gradient and the principal axis direction of the projection ellipse, and perturbing the spatial position parameter and the shape parameter in the splitting direction to obtain the initialization parameters of the child Gaussian object, while making the child Gaussian object inherit the prior values ​​of the color representation parameter and the transparency parameter. For the set of fine-tuning candidate Gaussian objects, perform parameter-constrained updates, wherein the parameter-constrained updates include at least: updating the color representation parameter and the transparency parameter, while keeping the spatial position parameter and the shape parameter unchanged; For the sparse candidate Gaussian object set, an absorption and merging operation is performed, wherein the absorption and merging operation includes: determining a cluster of Gaussian objects that satisfy similarity constraints within the same tile region, the similarity constraints including at least color similarity constraints, principal axis direction similarity constraints, and depth consistency constraints, and merging the Gaussian object clusters into a merged Gaussian object, wherein the spatial position parameters, shape parameters, transparency parameters, and color representation parameters of the merged Gaussian object are obtained by weighting the parameters of the Gaussian object cluster according to the effective mixing weights.

9. The low-computing-power rapid 3D modeling method based on 3DGS according to claim 8, characterized in that, The encrypted state corresponds to a tile region whose complexity score meets the first threshold condition; the sparse state corresponds to a tile region whose complexity score meets the second threshold condition; and the fine-tuned state corresponds to a tile region whose complexity score is between the first threshold and the second threshold.

10. A low-computing-power rapid 3D modeling system based on 3DGS, used to implement the low-computing-power rapid 3D modeling method based on 3DGS as described in any one of claims 1-9, characterized in that, The system includes: The periodic scheduling module is used to acquire image data of the object to be modeled from the corresponding viewpoint of each modeling cycle, and maintain the current Gaussian field model on the graphics processing unit. The current Gaussian field model is the updated Gaussian field model of the previous modeling cycle. The complexity evaluation module performs forward rendering based on the image data and the current Gaussian field model to obtain a rendered image, determines residual information based on the rendered image and the image data, and determines complexity information based on the residual information and the Gaussian object set corresponding to the current Gaussian field model, so as to divide complex regions and flat regions. The modeling output module is used to perform adaptive Gaussian density allocation on the current Gaussian field model according to the complexity information to obtain the updated Gaussian field model for the current modeling cycle, and to complete the 3D modeling output based on the updated Gaussian field model.

Images