Method and device for rendering three-dimensional gaussian point cloud, electronic equipment and storage medium

Abstract

This invention provides a rendering method, apparatus, electronic device, and storage medium for 3D Gaussian point clouds, belonging to the field of 3D graphics rendering technology. In this method, by evaluating the primitive validity of multiple Gaussian points based on spatial distribution parameters and eliminating invalid Gaussian points, redundant data located outside the view frustum or occluded can be effectively filtered out before rendering calculations, avoiding unnecessary computational waste. Simultaneously, by spatially aggregating adjacent Gaussian points that meet aggregation conditions to generate shared rendering primitives, and using a shader that supports batch primitive generation to perform a single core rendering calculation on unaggregated Gaussian points and shared rendering primitives, the computational load on the GPU is significantly reduced. Furthermore, based on the baseline rendering parameters obtained from the single calculation, vertex data for rendering patches is generated in batches, achieving efficient pipeline processing from data culling and aggregation to batch rendering, supporting high-frame-rate real-time rendering of large-scale 3D Gaussian point cloud scenes.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional graphics rendering technology, and in particular to a rendering method, apparatus, electronic device, and storage medium for three-dimensional Gaussian point clouds. Background Technology

[0002] Currently, in the field of 3D Gaussian point cloud rendering, vertex shaders are typically used to perform per-vertex calculations on the rendering facet corresponding to each Gaussian point. Since a single Gaussian point requires the generation of a rendering facet containing multiple vertices, the vertex shader repeatedly performs core calculations such as coordinate transformations and projection mappings for the same Gaussian point, resulting in a waste of GPU computing resources. Furthermore, existing rendering workflows typically perform pruning only after all vertex calculations are completed, causing a large number of invalid Gaussian points located outside the view frustum or occluded to still participate in the calculations, further exacerbating computational consumption. These issues make it difficult to achieve high-frame-rate real-time rendering of large-scale 3D Gaussian point cloud scenes.

[0003] In summary, existing 3D Gaussian point cloud rendering methods suffer from wasted GPU resources and difficulty in achieving high frame rate real-time rendering of large-scale scenes due to repetitive core computations and delayed invalid point removal. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a rendering method, apparatus, electronic device and storage medium for three-dimensional Gaussian point clouds, so as to alleviate the technical problems of existing three-dimensional Gaussian point cloud rendering methods, such as GPU resource waste and difficulty in achieving high frame rate real-time rendering of large-scale scenes.

[0005] In a first aspect, the present invention provides a method for rendering three-dimensional Gaussian point clouds, the method comprising: Acquire three-dimensional Gaussian point cloud data, which includes spatial distribution parameters and visual attribute parameters of multiple Gaussian points; Based on the spatial distribution parameters, the primitive validity of the multiple Gaussian points is evaluated, and invalid Gaussian points are eliminated to obtain a set of candidate Gaussian points. Based on the spatial distribution parameters and the visual attribute parameters, adjacent Gaussian points in the candidate Gaussian point set that meet the preset aggregation conditions are spatially aggregated to generate shared rendering primitives. By using a shader that supports batch primitive generation, single core rendering calculations are performed on candidate Gaussian points that did not participate in the spatial aggregation process and the shared rendering primitives to obtain the corresponding baseline rendering parameters. Based on the aforementioned baseline rendering parameters, multiple vertex data for the corresponding rendering facets are generated in batches for pixel rendering. Specifically, based on the spatial distribution parameters, the primitive validity of the plurality of Gaussian points is evaluated, invalid Gaussian points are eliminated, and a candidate Gaussian point set is obtained, including: Based on the spatial distribution parameters, calculate the projection range of each Gaussian point in the screen space; When the projection range is outside the preset screen boundary, or the area of ​​the projection range is less than the first area threshold, the corresponding Gaussian point is determined as the invalid Gaussian point and discarded. Get the depth buffer information of the previous frame's rendered image; Based on the depth buffer information, the remaining Gaussian points after projection range elimination are subjected to occlusion test, and the Gaussian points that are completely occluded are eliminated to obtain the candidate Gaussian point set.

[0006] Further, based on the spatial distribution parameters and the visual attribute parameters, adjacent Gaussian points in the candidate Gaussian point set that meet the preset aggregation conditions are spatially aggregated to generate shared rendering primitives, including: Calculate the spatial distance and visual feature difference between any two Gaussian points in the candidate Gaussian point set; When the spatial distance is less than a preset distance threshold and the visual feature difference is less than a preset difference threshold, the two corresponding Gaussian points are determined as adjacent Gaussian points and satisfy the preset aggregation condition. Adjacent Gaussian points that meet the preset aggregation conditions are merged into the shared rendering primitives.

[0007] Furthermore, the shader that supports batch primitive generation is a mesh shader; By using a shader that supports batch primitive generation, single-core rendering calculations are performed on candidate Gaussian points that did not participate in the spatial aggregation process and the shared rendering primitives to obtain the corresponding baseline rendering parameters, including: The reference rendering parameters are obtained by performing a coordinate matrix transformation and camera projection mapping on a single candidate Gaussian point or a single shared rendering primitive using the mesh shader. Based on the aforementioned baseline rendering parameters, multiple vertex data for the corresponding rendering facets are generated in batches, including: Based on the baseline rendering parameters and the preset fixed offset compensation rules, the four vertex data of the rendered facet are generated.

[0008] Furthermore, the spatial distribution parameters include a three-dimensional covariance matrix, and the visual attribute parameters include color parameters and transparency parameters; Merging adjacent Gaussian points that meet the preset aggregation conditions into the shared rendering primitive includes: Based on the spatial positions of the adjacent Gaussian points and the three-dimensional covariance matrix, calculate the spatial position parameters and the fused covariance matrix of the merged large Gaussian ellipsoid. Based on the color parameters and transparency parameters of the adjacent Gaussian points, the blended color parameters and blended transparency parameters of the large Gaussian ellipsoid are calculated. The large Gaussian ellipsoid is used as the shared rendering primitive.

[0009] Furthermore, before performing a single core rendering calculation on the candidate Gaussian points that did not participate in the spatial aggregation process and the shared rendering primitives respectively using a shader that supports batch primitive generation, the method further includes: Estimate the projected area of ​​each candidate Gaussian point and each shared rendering primitive in screen space; When the projected area is greater than or equal to the second area threshold, the corresponding Gaussian point or shared rendering primitive is assigned to the batch generation shader path, so that the candidate Gaussian point and the shared rendering primitive that did not participate in the spatial aggregation process can be processed by the shader that supports batch primitive generation. When the projected area is less than the second area threshold, the corresponding Gaussian point or shared rendering primitive is assigned to the single-point instantiation shader path so that rendering calculations can be performed through the single-point instantiation shader.

[0010] Furthermore, the method also includes: The occupancy rate of each computing unit in the graphics processing unit is monitored in real time, and the second area threshold is dynamically adjusted based on the occupancy rate.

[0011] Secondly, the present invention also provides a rendering apparatus for three-dimensional Gaussian point clouds, the apparatus comprising: An acquisition unit is used to acquire three-dimensional Gaussian point cloud data, wherein the three-dimensional Gaussian point cloud data includes spatial distribution parameters and visual attribute parameters of multiple Gaussian points; The evaluation unit is used to evaluate the primitive validity of the multiple Gaussian points based on the spatial distribution parameters, remove invalid Gaussian points, and obtain a set of candidate Gaussian points. An aggregation unit is used to perform spatial aggregation processing on adjacent Gaussian points in the candidate Gaussian point set that meet preset aggregation conditions based on the spatial distribution parameters and the visual attribute parameters, and generate shared rendering primitives. The rendering calculation unit is used to perform a single core rendering calculation on the candidate Gaussian points that did not participate in the spatial aggregation process and the shared rendering primitives through a shader that supports batch primitive generation, so as to obtain the corresponding baseline rendering parameters. The generation unit is used to generate multiple vertex data of the corresponding rendering facets in batches based on the reference rendering parameters for pixel rendering. The evaluation unit is further configured to: calculate the projection range of each Gaussian point in the screen space based on the spatial distribution parameters; When the projection range is outside the preset screen boundary, or the area of ​​the projection range is less than the first area threshold, the corresponding Gaussian point is determined as the invalid Gaussian point and discarded. Get the depth buffer information of the previous frame's rendered image; Based on the depth buffer information, the remaining Gaussian points after projection range elimination are subjected to occlusion test, and the Gaussian points that are completely occluded are eliminated to obtain the candidate Gaussian point set.

[0012] Thirdly, the present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the method described in the first aspect.

[0013] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the method described in the first aspect.

[0014] This invention provides a rendering method for 3D Gaussian point clouds, comprising: acquiring 3D Gaussian point cloud data, the 3D Gaussian point cloud data containing spatial distribution parameters and visual attribute parameters of multiple Gaussian points; evaluating the primitive validity of multiple Gaussian points based on the spatial distribution parameters, removing invalid Gaussian points, and obtaining a candidate Gaussian point set; performing spatial aggregation processing on adjacent Gaussian points in the candidate Gaussian point set that meet preset aggregation conditions based on the spatial distribution parameters and visual attribute parameters, generating shared rendering primitives; and performing single core rendering calculations on the candidate Gaussian points that did not participate in the spatial aggregation processing and the shared rendering primitives respectively using a shader that supports batch primitive generation, to obtain corresponding baseline rendering parameters; based on... Based on baseline rendering parameters, multiple vertex data for corresponding rendering patches are generated in batches for pixel rendering. Specifically, based on spatial distribution parameters, the primitive validity of multiple Gaussian points is evaluated, invalid Gaussian points are removed, and a candidate Gaussian point set is obtained. This includes: calculating the projection range of each Gaussian point in screen space based on spatial distribution parameters; when the projection range is outside a preset screen boundary, or the area of ​​the projection range is less than a first area threshold, the corresponding Gaussian point is identified as an invalid Gaussian point and removed; obtaining the depth buffer information of the previous frame's rendering image; and performing occlusion testing on the remaining Gaussian points after projection range removal based on the depth buffer information, removing completely occluded Gaussian points to obtain the candidate Gaussian point set. As described above, the rendering method for 3D Gaussian point clouds of the present invention effectively filters out redundant data located outside the view frustum or occluded before rendering by evaluating the primitive validity of multiple Gaussian points based on spatial distribution parameters and removing invalid Gaussian points, thus avoiding unnecessary computational consumption. Simultaneously, by spatially aggregating adjacent Gaussian points that meet the aggregation conditions to generate shared rendering primitives, and using a shader that supports batch primitive generation to perform a single core rendering calculation on unaggregated Gaussian points and shared rendering primitives, the method avoids the repeated execution of core calculations such as coordinate transformation and projection mapping for the same Gaussian point in traditional methods, significantly reducing the computational load on the GPU. Furthermore, based on the baseline rendering parameters obtained from the single calculation, vertex data for rendering patches is generated in batches, achieving efficient pipeline processing from data culling and aggregation to batch rendering. This ultimately solves the problem of GPU resource waste caused by repeated core calculations and delayed invalid point removal in existing technologies, thereby supporting high-frame-rate real-time rendering of large-scale 3D Gaussian point cloud scenes. Attached Figure Description

[0015] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating a method for rendering a three-dimensional Gaussian point cloud, provided as an embodiment of the present invention; Figure 2 A schematic diagram comparing the computational logic of a conventional vertex shader and the mesh shader of the present invention is provided for embodiments of the present invention. Figure 3 A schematic diagram of a rendering device for a three-dimensional Gaussian point cloud provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] To facilitate understanding of this embodiment, a detailed description of a three-dimensional Gaussian point cloud rendering method disclosed in this embodiment of the invention will be provided first.

[0019] Example 1: like Figure 1 As shown, this embodiment of the invention provides a rendering method for 3D Gaussian point clouds. This method achieves efficient real-time rendering of 3D Gaussian point clouds through collaborative processing of primitive filtering and aggregation with batch calculation. Specifically, the rendering method includes the following steps: Step S100: Obtain three-dimensional Gaussian point cloud data. The three-dimensional Gaussian point cloud data includes spatial distribution parameters and visual attribute parameters of multiple Gaussian points.

[0020] Specifically, 3D Gaussian point cloud data is the fundamental data source for describing the geometric structure and appearance attributes of a 3D scene. Spatial distribution parameters characterize the position and shape features of Gaussian points in the 3D world coordinate system. These parameters may include 3D coordinate positions and covariance matrices or scaling and rotation parameters describing the shape and orientation of the Gaussian ellipsoid. Visual attribute parameters characterize the optical performance of Gaussian points, and may include RGB color values, spherical harmonic coefficients, and transparency parameters. It should be understood that although this embodiment uses covariance matrices and RGB color as examples, in other embodiments, spatial distribution parameters can also use quaternions, Euler angles, or other equivalent geometric descriptions, and visual attribute parameters can also use the YUV color space or other color encoding methods, as long as they can fully describe the spatial state and visual characteristics of the Gaussian points. Obtaining these parameters provides a complete data foundation for subsequent primitive evaluation and rendering calculations.

[0021] Step S200: Evaluate the primitive validity of multiple Gaussian points based on spatial distribution parameters, remove invalid Gaussian points, and obtain a set of candidate Gaussian points.

[0022] Specifically, primitive validity assessment is a preprocessing step performed before generating vertex data for rendered patches, aiming to reduce the number of primitives involved in subsequent high-overhead rendering calculations from the outset. The assessment process primarily relies on the spatial distribution parameters of Gaussian points to determine their visibility or contribution to the current viewpoint. For example, the projection range of a Gaussian point in screen space can be calculated based on the spatial distribution parameters. If the projection range is completely outside the view frustum or the projection area is less than a preset small threshold, the Gaussian point is deemed to have no substantial contribution to the final image and is identified as an invalid Gaussian point and removed. The remaining Gaussian points after this removal process constitute a candidate Gaussian point set. This pre-assessment mechanism differs from the pruning operation performed after the vertex shader calculations are completed in traditional rendering workflows. It effectively avoids performing invalid coordinate transformations and projection calculations on invisible primitives, thereby reducing the waste of computing power in the graphics processing unit.

[0023] As one implementation method, the process of evaluating the primitive validity of multiple Gaussian points based on spatial distribution parameters, eliminating invalid Gaussian points, and obtaining a set of candidate Gaussian points can be further refined to more accurately reduce redundant calculations in the early stages of the rendering pipeline.

[0024] Specifically, the evaluation process first calculates the projection range of each Gaussian point in screen space based on spatial distribution parameters. In 3D Gaussian point cloud rendering, spatial distribution parameters typically include a 3D covariance matrix describing the shape and orientation of the Gaussian ellipsoid. To determine whether a Gaussian point is visible, this 3D covariance matrix needs to be transformed to 2D screen space using the current view projection matrix, resulting in a 2D projection ellipse. The bounding box or approximately rectangular region of this projection ellipse constitutes the projection range of the Gaussian point in screen space. This calculation process involves only lightweight matrix operations, far lower than the computational overhead of a full vertex shader.

[0025] When the projection area is outside the preset screen boundary, or the area of ​​the projection area is less than the first area threshold, the corresponding Gaussian point is identified as an invalid Gaussian point and discarded. The preset screen boundary typically refers to the effective display area of ​​the current rendering viewport. If the projection area falls entirely outside this area, it means that the Gaussian point contributes nothing to the imaging of the current viewpoint and is considered an invisible primitive outside the view frustum. The first area threshold is used to filter sub-pixel level primitives. For example, when the projection area is less than 1 pixel or 0.5 pixels, the Gaussian point's visual contribution to the final image is negligible, and it may even be discarded due to rasterization rules. By performing this geometric culling before vertex generation, the expensive subsequent coordinate transformations and attribute interpolation calculations for a large number of invisible or non-contributing Gaussian points can be avoided. This pre-processing culling mechanism, compared to post-processing filtering performed during the rasterization stage in traditional rendering workflows, effectively reduces the load on the graphics processing unit.

[0026] Furthermore, the evaluation process includes acquiring the depth buffer information of the previous frame's rendered image and performing occlusion tests on the remaining Gaussian points after projection range culling based on the depth buffer information. Completely occluded Gaussian points are then removed, resulting in a candidate Gaussian point set. In static scenes with minimal camera viewpoint changes (or when the camera is stationary), the depth buffer information of the previous frame can serve as a conservative estimate of the current frame's scene depth, as the scene content between consecutive frames typically exhibits high temporal coherence. In practice, the depth value of the center of the projection range or the sampling point of the current Gaussian point can be compared with the stored value at the corresponding position in the depth buffer of the previous frame. If the depth value of the current Gaussian point is greater than the depth value at the corresponding position in the previous frame, and considering a certain error tolerance, it can be determined that the Gaussian point is completely occluded by a foreground object. It should be understood that this occlusion test based on historical frames is a heuristic and rapid prediction method. Although there may be a small number of misjudgments due to drastic scene changes, in most real-time roaming or interactive scenarios, the computational savings far outweigh the potential risk of image quality loss. By combining projection range culling and depth occlusion testing, this embodiment constructs a multi-level primitive screening mechanism to ensure that the candidate Gaussian point set entering the subsequent core rendering calculation only contains effective primitives that make a substantial contribution to the current image.

[0027] Step S300: Based on spatial distribution parameters and visual attribute parameters, perform spatial aggregation processing on adjacent Gaussian points in the candidate Gaussian point set that meet the preset aggregation conditions to generate shared rendering primitives.

[0028] Specifically, spatial aggregation is a merging operation performed on multiple spatially adjacent and visually similar Gaussian points in a candidate Gaussian point set. Preset aggregation conditions typically include spatial distance thresholds and visual feature difference thresholds. When the Euclidean distance between two or more Gaussian points is less than the distance threshold, and the differences in their visual attributes such as color and transparency are less than the difference threshold, the aggregation conditions are met. Adjacent Gaussian points that meet the conditions are merged into a new entity, i.e., a shared rendering primitive. It is important to emphasize that a shared rendering primitive is not a simple logical grouping, but rather a single rendering object with independent, merged spatial distribution parameters and visual attribute parameters. For example, multiple small Gaussian ellipsoids can be merged into a large Gaussian ellipsoid with approximately the same coverage area. Through spatial aggregation, the total number of primitives requiring independent processing can be effectively reduced while maintaining a generally consistent visual effect, creating a multiplier-level performance optimization effect compared to subsequent single-core calculations.

[0029] In step S400, a single core rendering calculation is performed on candidate Gaussian points and shared rendering primitives that did not participate in spatial aggregation processing using a shader that supports batch primitive generation, to obtain the corresponding baseline rendering parameters.

[0030] Specifically, a shader supporting batch primitive generation refers to a graphics processing unit capable of processing a primitive within a single thread group and outputting the corresponding vertex data for that primitive. Examples include a mesh shader or other programmable shader stages with equivalent functionality. In this embodiment, for each candidate Gaussian point not involved in aggregation or each shared rendering primitive, the shader performs a single core rendering calculation, rather than repeatedly performing it for each vertex of the rendered face. The content of a single core rendering calculation includes, but is not limited to, the world-view-projection matrix transformation of 3D coordinates, the projection mapping of the covariance matrix, and preprocessing of color and transparency. The calculated result is called the baseline rendering parameters. The baseline rendering parameters are key intermediate data connecting the single core calculation and batch vertex generation; they include basic information shared by all vertices of the primitive. For example, the baseline rendering parameters may include the transformed center point screen coordinates, the projected 2D covariance matrix, and preprocessed primary color values. Since all vertices of the same primitive share the same baseline rendering parameters, the redundant overhead of repeatedly performing four or more source matrix operations and parameter parsing on the same Gaussian point under the traditional per-vertex rendering architecture is fundamentally eliminated.

[0031] Step S500: Based on the baseline rendering parameters, generate multiple vertex data for the corresponding rendering facet in batches for pixel rendering.

[0032] Specifically, after obtaining the baseline rendering parameters, the shader supporting batch primitive generation uses these parameters to quickly derive the vertex data required to generate the rendering patch. Since the baseline rendering parameters already contain the core geometry and attribute information of the primitives, the vertex data generation process only requires lightweight offset compensation or local interpolation operations. For example, for a quadrilateral rendering patch, the screen space positions of the four corner points can be directly calculated based on the center point coordinates and a preset fixed offset vector in the baseline rendering parameters; at the same time, the texture coordinates, color attributes, etc. of each vertex can also be directly assigned or simply calculated based on the corresponding fields in the baseline parameters, without needing to access the original Gaussian point data again or repeatedly perform projection transformations. The generated vertex data is then encapsulated into primitives and output to the rasterization stage and fragment shader for final pixel rendering. It should be understood that although this embodiment uses a quadrilateral patch and four vertices as an example, in other embodiments, the rendering patch can also be a triangle, hexagon, or other polygon, with the number of vertices adjusted accordingly, as long as the vertex data is generated in batches based on the same baseline rendering parameters.

[0033] Through the coordinated execution of steps S100 to S500, this embodiment optimizes rendering efficiency in two dimensions: Firstly, by evaluating the validity of primitives in step S200 and spatial aggregation processing in step S300, the number of invalid and redundant primitives is reduced before vertex generation, thus lowering the primitive processing load on the GPU. Secondly, by utilizing shaders that support batch primitive generation to perform single-core calculations and batch generate vertices based on baseline rendering parameters in steps S400 and S500, the inherent defect of repetitive calculations of multiple vertices of the same origin for a single primitive is eliminated. This dual optimization mechanism of reducing workload and improving efficiency effectively improves the real-time rendering frame rate in large-scale 3D Gaussian point cloud scenes while maintaining consistent rendering quality.

[0034] As one implementation method, the process of generating shared rendering primitives by spatially aggregating adjacent Gaussian points that meet preset aggregation conditions in the candidate Gaussian point set based on spatial distribution parameters and visual attribute parameters can be further specified to further compress the number of primitives while ensuring image quality.

[0035] Specifically, the process first calculates the spatial distance and visual feature difference between any two Gaussian points in the candidate Gaussian point set. Spatial distance is determined by calculating the Euclidean distance between the three-dimensional coordinates of the two Gaussian points, measuring their geometric proximity. Visual feature difference is calculated based on visual attribute parameters, such as the difference in the vector magnitude of the RGB color values ​​of the two Gaussian points, the absolute difference in their transparency parameters, or a combination of factors including the difference in spherical harmonic coefficients, to quantify their similarity in appearance. This dual-dimensional measurement mechanism aims to ensure that only spatially adjacent and visually highly similar points are considered for merging, avoiding the erroneous merging of spatially adjacent but color-different edge points that would result in blurred image details.

[0036] When the spatial distance is less than a preset distance threshold and the visual feature difference is less than a preset difference threshold, the corresponding two Gaussian points are determined to be adjacent Gaussian points and satisfy the preset aggregation conditions. The preset distance threshold and preset difference threshold can be configured according to rendering quality requirements and performance goals. For example, in near-field areas with high image quality requirements, a smaller threshold can be set to retain more details; while in far-field or textured smooth areas, the threshold can be appropriately relaxed to improve the merging rate. In practical applications, an adaptive strategy can also be adopted to dynamically adjust the threshold based on the local density or gradient changes in the region where the Gaussian point is located.

[0037] Adjacent Gaussian points that meet preset aggregation conditions are merged into a shared rendering primitive. It's important to note that this shared rendering primitive doesn't simply mean grouping multiple Gaussian points into the same rendering batch or logical group; rather, it refers to fusing multiple independent Gaussian points into a single rendering entity with entirely new parameter definitions. This newly generated large Gaussian ellipsoid possesses independently calculated spatial positions, a fused covariance matrix, and integrated color and transparency attributes. For example, when rendering a uniformly textured wall or dense foliage, numerous small, similarly colored Gaussian points can be merged into fewer but larger-coverage Gaussian ellipsoids. This entity-level merging directly reduces the total number of primitives that need to be fed into the rendering pipeline, creating a synergistic effect with the aforementioned single-core computation optimization, improving rendering efficiency from both the number of primitives and the computational cost per primitive. It should be understood that although this embodiment uses the merging of two Gaussian points as an example, in actual processing, the aggregation operation can be recursive or clustered; that is, multiple Gaussian points can be gradually merged into a single shared rendering primitive, as long as they meet the preset aggregation conditions.

[0038] As one implementation method, the shader supporting batch primitive generation is a mesh shader. Specifically, a mesh shader is a programmable graphics processing stage. Its core characteristic, which distinguishes it from traditional vertex shaders, is that it can run in thread groups and has the ability to directly output a variable number of primitives. In this embodiment, this hardware feature of the mesh shader is utilized to allocate the processing task of a single candidate Gaussian point or a single shared rendering primitive to an independent thread group for execution. This allows multiple threads within the thread group to share the same calculation result, thereby providing a hardware-level data reuse foundation for subsequent batch vertex generation. It should be understood that although this embodiment uses a mesh shader as an example, in other implementations, computational shaders or other programmable units with similar primitive generation and data sharing capabilities can also be used to achieve the same function, as long as they can meet the requirement of performing a single core calculation on a single primitive and batch outputting multiple vertices.

[0039] Based on this, a coordinate matrix transformation and camera projection mapping are performed on a single candidate Gaussian point or a single shared rendering primitive using a mesh shader to obtain the baseline rendering parameters. For example... Figure 2 As shown in the logic on the right, with Figure 2 Unlike the traditional approach on the left, where the vertex shader performs four independent coordinate transformations and projection calculations for each of the four vertices of the same Gaussian point, the mesh shader in this embodiment triggers only one complete geometric transformation pipeline when processing a single candidate Gaussian point or a shared rendering primitive. This calculation encompasses the entire chain of matrix operations from the world coordinate system to the view coordinate system and then to screen space, as well as the transformation process from the 3D covariance matrix to the 2D projected ellipse. The calculated results are encapsulated as baseline rendering parameters, which contain all the core information needed to determine the position and shape of the rendered facet in screen space, such as the transformed center point coordinates, the eigenvalues ​​of the projected 2D covariance matrix, and the preprocessed color and transparency base values. Since this process is executed uniformly at the thread group level, this core computational overhead occurs only once, regardless of how many vertices need to be generated subsequently, mathematically eliminating the redundant computations caused by per-vertex processing in the traditional architecture.

[0040] Furthermore, based on the baseline rendering parameters and the preset fixed offset compensation rules, the four vertex data of the rendered face are generated. Specifically, the fixed offset compensation rules define the geometric relationship between the standard topology of the rendered face and its center point. For quadrilateral rendered faces, this rule typically includes four sets of predefined two-dimensional offset vectors, corresponding to the relative positions of the four corner points of the quadrilateral with respect to the center point, such as unit offsets in the upper left, upper right, lower left, and lower right directions. Inside the mesh shader, after obtaining the baseline rendering parameters, each thread only needs to read the center point coordinates from the shared baseline rendering parameters and perform simple addition or scaling operations with the corresponding fixed offset vectors to quickly calculate the final screen space position of each vertex. Similarly, the texture coordinates, color interpolation weights, and other attributes of the vertices can also be obtained based on the corresponding fields in the baseline parameters through similar lightweight compensation operations. This generation method transforms the originally complex matrix multiplication and projection operations into extremely low-overhead vector addition and subtraction, and all vertices reuse the same baseline data, without needing to access the original Gaussian point data again or repeatedly execute transformation instructions.

[0041] To more intuitively demonstrate the technical advantages of this embodiment, combined with Figure 2 A comparative explanation will be provided. In traditional rendering methods ( Figure 2 In the left-hand representation, rendering a quadrilateral face corresponding to a Gaussian point requires the vertex shader to perform four complete coordinate transformations, projection calculations, and Gaussian parameter parsings, resulting in a linear increase in GPU instruction scheduling and ALU load. However, in the scheme of this embodiment (…),… Figure 2 In the right-hand diagram, the mesh shader performs only one core calculation to obtain the baseline parameters, and then generates four vertex data in batches through fixed offset compensation. Assuming the computational cost of a single core calculation is C, and the computational cost of a single offset compensation is c (where c is much smaller than C), the computational cost per primitive in the traditional scheme is approximately 4C, while the computational cost in this embodiment is approximately C+4c. In large-scale 3D Gaussian point cloud rendering scenarios, this reduction in computational complexity effectively alleviates the GPU's computational bottleneck and improves the real-time rendering frame rate. Simultaneously, since all vertex data originates from the deterministic derivation of the same baseline parameters, the generated rendered patches are completely equivalent to the traditional per-vertex calculation scheme in terms of geometric accuracy and attribute consistency, avoiding potential image artifacts introduced by differences in computational paths.

[0042] As one implementation method, spatial distribution parameters include a three-dimensional covariance matrix, and visual attribute parameters include color parameters and transparency parameters. Specifically, the three-dimensional covariance matrix mathematically describes the shape, size, and rotational orientation of the Gaussian ellipsoid in three-dimensional space, and is a key geometric factor determining the projection range and blur degree of the Gaussian point; color parameters typically use RGB three-channel values ​​or spherical harmonic function coefficients to characterize the color representation of the Gaussian point, while transparency parameters define the opacity or weight of the Gaussian point to light. These parameters together constitute the complete rendering state of the Gaussian point, and are also objects that must be accurately processed during subsequent spatial aggregation.

[0043] Based on this, the process of merging adjacent Gaussian points that meet preset aggregation conditions into a shared rendering primitive includes calculating the spatial position parameters and fusion covariance matrix of the merged large Gaussian ellipsoid based on the spatial position and 3D covariance matrix of each adjacent Gaussian point. This process is not a simple arithmetic average, but aims to generate a new entity that can approximately wrap around or equivalently replace the original multiple small Gaussian points in terms of geometric coverage. For example, the spatial position parameters of the large Gaussian ellipsoid can be obtained by weighted averaging of the 3D coordinates of adjacent Gaussian points, where the weighting coefficient can be proportional to the transparency or volume of each Gaussian point to ensure that points with higher visual contribution have a greater influence on the new center position. For the calculation of the fusion covariance matrix, it is necessary to comprehensively consider the covariance of each original Gaussian point and their dispersion relative to the new center position. In one feasible implementation, the principle of moment matching can be used to transform the second moments of each original Gaussian point to a unified coordinate system and accumulate them to derive the fusion covariance matrix. This processing method ensures that the projection profile of the merged large Gaussian ellipsoid in screen space can cover the overall distribution characteristics of the original point group, avoiding coverage shrinkage or shape distortion caused by simple averaging.

[0044] Furthermore, the merging process also includes calculating the blended color and blended transparency parameters of the large Gaussian ellipsoid based on the color and transparency parameters of each adjacent Gaussian point. Since human color perception is non-linear, and Gaussian points with different transparency levels follow specific optical mixing rules when superimposed, the fusion of visual attributes needs to follow the principles of energy conservation or alpha mixing. For example, the blended color parameter should not be a direct average of the original colors, but rather a weighted blend using the transparency of each point as a weight, or calculated using premultiplied alpha to prevent the low-transparency background color from incorrectly diluting the high-transparency foreground color. The blended transparency parameter can be calculated based on the cumulative probability model of the original transparency levels, making the overall opacity of the large Gaussian ellipsoid approximate the visual effect of the original multiple Gaussian points superimposed. Through the collaborative fusion of the above geometric and visual attributes, the final generated large Gaussian ellipsoid is used as a shared rendering primitive in subsequent rendering.

[0045] It should be understood that although the above embodiments describe a fusion strategy based on transparency and moment matching, other fusion algorithms can be adopted in other implementations depending on specific rendering quality requirements or computational constraints. For example, in near-field areas with extremely high image quality requirements, more complex rotation and projection fusion of spherical harmonic coefficients can be introduced; while in far-field or performance-sensitive scenes, simplified bounding box approximations or principal component analysis (PCA) can be used to quickly estimate the fusion covariance. Regardless of the specific formula used, the core purpose is to make the projection coverage and color representation of shared rendering primitives in screen space as close as possible to the superposition effect of multiple adjacent Gaussian points, thereby effectively reducing the number of primitives and GPU load while maintaining the visual fidelity of the rendered image and avoiding obvious image defects or loss of detail caused by primitive merging.

[0046] As one implementation, the method includes a path distribution step before performing a single core rendering computation on candidate Gaussian points and shared rendering primitives that are not involved in spatial aggregation, using a shader that supports batch primitive generation. Specifically, this step first estimates the projected area in screen space for each candidate Gaussian point and each shared rendering primitive. This estimation process is typically performed after primitive validity evaluation and before proceeding to the expensive core rendering computation, aiming to dynamically select the optimal rendering path based on the primitive's screen-space scale. The estimated projected area can employ lightweight approximation algorithms, such as a fast estimation based on the ratio of the trace of the 3D covariance matrix in the spatial distribution parameters to the current view distance, or by calculating the bounding box area of ​​the 2D projected ellipse, thereby avoiding the need to pre-perform complete vertex transformation and rasterization calculations to determine the path.

[0047] Based on the prediction results, the system executes dynamic routing logic: when the projected area is greater than or equal to the second area threshold, the corresponding Gaussian point or shared rendering primitive is assigned to the batch generation shader path, so that the candidate Gaussian point and shared rendering primitive that did not participate in the spatial aggregation process can be processed by a single core rendering calculation through a shader that supports batch primitive generation; when the projected area is less than the second area threshold, the corresponding Gaussian point or shared rendering primitive is assigned to the single-point instantiation shader path, so that the rendering calculation can be performed by a single-point instantiation shader. Among them, the batch generation shader path is the path described in the previous embodiment that uses units with primitive generation capabilities such as mesh shaders for processing. Its advantage is that it can output multiple vertices in batches through a single core calculation, eliminating redundant calculations from the same source; while the single-point instantiation shader path refers to the use of traditional vertex shaders or simplified point sprite rendering methods to directly generate predefined geometry for a single primitive, without involving complex thread group primitive generation logic.

[0048] The reason for introducing the aforementioned adaptive hybrid scheduling strategy is that while shaders supporting batch primitive generation have computational efficiency advantages when processing medium to large primitives, the startup and synchronization overhead of their thread groups may outweigh the computational gains from batch generation when processing primitives with extremely small screen-space projection areas (such as distant, edge regions, or sub-pixel levels), leading to performance degradation. The setting of the second area threshold is precisely to define this performance crossover point. For example, in a practical implementation, the second area threshold can be set to 4 pixels squared or other empirical values. When the primitive projection area is greater than this threshold, the amount of repetitive computation saved by batch generation is sufficient to cover the thread group scheduling overhead; conversely, when the primitive degenerates into tiny dots, the single-point instantiation path can complete rendering with lower fixed overhead. It should be understood that the second area threshold is not a fixed constant; it can be configured or dynamically adjusted according to specific GPU architecture characteristics, scene density, or image quality requirements, as long as it can achieve the function of distinguishing primitive size and matching the corresponding rendering path.

[0049] Through this dynamic path distribution mechanism, this embodiment constructs a hybrid rendering pipeline, enabling different primitives in a large-scale 3D Gaussian point cloud scene to be allocated appropriately: large Gaussian points in the foreground or with rich details utilize batch generation shader paths to eliminate computational redundancy, while small Gaussian points in the background or sparse areas avoid scheduling waste by using single-point instantiation shader paths. This strategy is an adaptive supplement to, rather than a rejection of, the core optimization scheme based on shaders supporting batch primitive generation in the aforementioned embodiments. It effectively overcomes the performance bottleneck of a single shader architecture in extreme scenarios and helps improve the robustness and frame rate stability of the overall rendering pipeline under different view distances and load conditions.

[0050] As one implementation method, the method further includes: real-time monitoring of the occupancy rate of each computing unit of the graphics processing unit, and dynamically adjusting the second area threshold based on the occupancy rate.

[0051] Specifically, based on the adaptive hybrid shader scheduling strategy, a dynamic adjustment mechanism based on hardware load feedback is introduced to address the performance adaptation lag that may occur when using a fixed threshold to handle complex and variable 3D Gaussian point cloud scenes. In actual rendering, the geometric complexity and texture density of the scene often fluctuate drastically with viewpoint movement. If the second area threshold remains constant, it may lead to overload of the batch shader generation path in dense scenes and insufficient utilization of the single-point instantiation shader path in sparse scenes. By monitoring the occupancy of each computing unit (such as stream processor clusters and texture mapping units) in real time, a feedback signal reflecting the current rendering pipeline pressure status can be constructed. When the system detects that the compute unit occupancy rate exceeds the preset high-load threshold, it can automatically increase the second area threshold, allowing more medium-sized primitives to be routed to the relatively low-cost single-point instantiation shader path, thereby alleviating congestion in the batch shader generation path. Conversely, when the occupancy rate is below the low-load threshold, the second area threshold can be appropriately decreased to include more primitives in the batch shader generation path, fully utilizing the hardware's parallel computing power and improving rendering detail. This closed-loop control logic helps maintain the throughput ratio of the two rendering paths within an optimal range, ensuring the stability and smoothness of the overall rendering frame rate. It should be understood that the monitoring frequency of the occupancy rate and the adjustment step size of the threshold can be configured according to the response characteristics of the specific hardware architecture. For example, filtering algorithms such as sliding window averaging or exponentially weighted moving average can be used to smooth instantaneous noise and avoid rendering jitter caused by frequent threshold jumps.

[0052] Furthermore, to further improve the overall efficiency of the rendering pipeline, occlusion-aware optimization can be introduced as an auxiliary culling method before path distribution. Specifically, before performing projected area estimation and path allocation, the depth buffer information of the previous frame's rendered image can be used to perform a quick, coarse occlusion test on the Gaussian points to be processed. Unlike the precise occlusion culling performed on the candidate Gaussian point set in the previous embodiment, the occlusion test here mainly serves the path scheduling decision, and its judgment criteria can be more lenient and have lower computational overhead. For example, it is only necessary to compare the depth value of the primitive center point in screen space with the depth value of the corresponding pixel position in the previous frame. If it is determined that it is likely to be occluded, it can be directly assigned to the single-point instantiation shader path or even skipped, without entering the computationally expensive batch generation shader path. Since the batch generation shader usually involves the startup and synchronization overhead of thread groups, avoiding sending invisible or low-contribution primitives to this path can save valuable computational resources. This optimization step is independent of, yet complementary to, the aforementioned dynamic threshold adjustment mechanism. The former optimizes the processing efficiency of visible primitives from a load balancing perspective, while the latter reduces the overall load by minimizing invalid computations. Their synergistic effect helps further improve real-time rendering performance in complex scenes. It should be noted that, because this occlusion awareness optimization utilizes historical frame data for prediction, it is a heuristic strategy and may lead to misjudgments in a very small number of fast-moving or suddenly changing viewpoint scenarios. However, in most conventional roaming and interactive applications, the performance gains usually far outweigh the potential risk of visual artifacts, and therefore it can be integrated into the rendering system as an optional enhancement.

[0053] To further verify the technical effectiveness of the 3D Gaussian point cloud rendering method provided by this invention in practical applications, a large-scale urban 3D roaming scene is used as an example for illustration. In this application scenario, the 3D Gaussian point cloud data typically contains millions or even tens of millions of Gaussian points to accurately reproduce complex details such as urban buildings, vegetation, and roads. When users roam in real-time in the virtual environment, frequent viewpoint movements cause drastic fluctuations in the number of visible Gaussian points that need to be processed in each frame, which places stringent requirements on the real-time performance and stability of the rendering system.

[0054] In traditional rendering architectures that do not employ the solution of this invention, the vertex shader performs coordinate transformations and projection calculations on a vertex-by-vertex basis on the quadrilateral facet corresponding to each Gaussian point. This results in a linear increase in the overhead of repetitive computation of the same source for a single primitive with the scale of the point cloud. Simultaneously, a large number of Gaussian points located outside the view frustum or occluded by the foreground are only culled after all vertex calculations have been completed, leading to inefficient consumption of the graphics processing unit's computing power. At point cloud scales of millions, this double redundancy often makes it difficult to maintain a smooth interactive rendering frame rate, and the GPU utilization remains saturated for extended periods, limiting the deployment of this technology on mobile or low-power devices.

[0055] After applying the rendering method provided in this embodiment of the invention, the system first evaluates the primitive validity of Gaussian points at the current viewpoint based on spatial distribution parameters before performing pixel rendering. Specifically, by calculating the projection range and combining it with the depth buffer information of the previous frame, approximately 30% to 50% of invisible or non-contributing Gaussian points can be eliminated before the vertex generation stage, thereby effectively avoiding subsequent expensive calculations for these invalid primitives. Based on this, the system further performs spatial aggregation processing on spatially adjacent and visually similar points in the candidate Gaussian point set, merging multiple tiny Gaussian ellipsoids into a shared rendering primitive with a larger coverage area. In the distant areas of the city scene or in the uniformly textured building facade areas, spatial aggregation processing can further reduce the total number of primitives by approximately 20% to 30%, resulting in a substantial compression of the number of primitives entering the core rendering stage.

[0056] For the filtered and aggregated primitives, the system performs a single core rendering calculation through the mesh shader to obtain baseline rendering parameters, and then generates multiple vertex data for rendering patches in batches based on these parameters. Compared to the traditional approach where each Gaussian point requires four complete matrix transformations and parameter analyses, this invention reduces the core computation of a single primitive to about one-quarter of the original. This reconstruction of computational logic, together with the aforementioned primitive reduction mechanism, creates a synergistic effect, alleviating the load pressure on the graphics processing unit from both the perspectives of single primitive computational complexity and the total number of primitives.

[0057] In a real-world test environment, a 3D urban scene dataset containing one million Gaussian points was loaded, and the rendering performance of the traditional vertex shader scheme and the scheme of this invention were compared under the same hardware configuration. Test results show that after adopting the scheme of this invention, the average rendering frame rate increased by 10% to 20%, and the computing power utilization of the graphics processing unit decreased by more than 20%. This measurable performance improvement makes the previously stuttering roaming experience at high resolutions smooth and stable, while also reserving computing power for higher precision image quality settings or more complex post-processing effects. It should be understood that the above test data is only used to exemplify the technical effects of the scheme of this invention, and the actual performance gains may vary depending on the hardware architecture, scene complexity, and parameter configuration.

[0058] Furthermore, although this embodiment uses a 3D urban real-scene roaming as an example for illustration, the rendering method of the present invention is also applicable to other application fields involving real-time rendering of large-scale 3D Gaussian point clouds, such as virtual reality headset rendering, augmented reality scene overlay, industrial digital twin visualization, and autonomous driving simulation. In these scenarios, the solution of the present invention can effectively improve rendering efficiency and interactive response speed by eliminating computational redundancy and reducing invalid primitive processing, meeting the comprehensive requirements of real-time performance and image quality for different business scenarios.

[0059] Example 2: This invention also provides a rendering apparatus for three-dimensional Gaussian point clouds. This three-dimensional Gaussian point cloud rendering apparatus is mainly used to execute the three-dimensional Gaussian point cloud rendering method provided in Embodiment 1 of this invention. The rendering apparatus for three-dimensional Gaussian point clouds provided in this invention will be described in detail below.

[0060] Figure 3 This is a schematic diagram of a rendering apparatus for a three-dimensional Gaussian point cloud according to an embodiment of the present invention, as shown below. Figure 3 As shown, the device mainly includes: an acquisition unit 10, an evaluation unit 20, an aggregation unit 30, a rendering and calculation unit 40, and a generation unit 50, wherein: Acquisition unit 10 is used to acquire three-dimensional Gaussian point cloud data, which includes spatial distribution parameters and visual attribute parameters of multiple Gaussian points; Evaluation unit 20 is used to evaluate the primitive validity of multiple Gaussian points based on spatial distribution parameters, remove invalid Gaussian points, and obtain a set of candidate Gaussian points. The evaluation unit is also used to: calculate the projection range of each Gaussian point in the screen space based on the spatial distribution parameters; when the projection range is outside the preset screen boundary, or the area of ​​the projection range is less than a first area threshold, the corresponding Gaussian point is identified as an invalid Gaussian point and removed; obtain the depth buffer information of the previous frame rendering; and perform occlusion testing on the remaining Gaussian points after projection range removal based on the depth buffer information, remove the Gaussian points that are completely occluded, and obtain a set of candidate Gaussian points. Aggregation unit 30 is used to perform spatial aggregation processing on adjacent Gaussian points in the candidate Gaussian point set that meet the preset aggregation conditions based on spatial distribution parameters and visual attribute parameters, and generate shared rendering primitives. The rendering calculation unit 40 is used to perform a single core rendering calculation on candidate Gaussian points and shared rendering primitives that do not participate in spatial aggregation processing, respectively, through a shader that supports batch primitive generation, to obtain the corresponding baseline rendering parameters. The generation unit 50 is used to generate multiple vertex data of the corresponding rendering facets in batches based on the baseline rendering parameters for pixel rendering.

[0061] Optionally, the aggregation unit is also used to: calculate the spatial distance and visual feature difference between any two Gaussian points in the candidate Gaussian point set; when the spatial distance is less than a preset distance threshold and the visual feature difference is less than a preset difference threshold, determine the corresponding two Gaussian points as adjacent Gaussian points and satisfy the preset aggregation condition; and merge the adjacent Gaussian points that satisfy the preset aggregation condition into a shared rendering primitive.

[0062] Optionally, the shader supporting batch primitive generation is a mesh shader; the rendering calculation unit is also used to: perform a coordinate matrix transformation and camera projection mapping on a single candidate Gaussian point or a single shared rendering primitive through the mesh shader to obtain the reference rendering parameters; the generation unit is also used to: generate the four vertex data of the rendering patch based on the reference rendering parameters and the preset fixed offset compensation rules.

[0063] Optionally, the spatial distribution parameters include a three-dimensional covariance matrix, and the visual attribute parameters include color parameters and transparency parameters; the aggregation unit is also used to: calculate the spatial position parameters and fusion covariance matrix of the merged large Gaussian ellipsoid based on the spatial position and three-dimensional covariance matrix of each adjacent Gaussian point; calculate the fusion color parameters and fusion transparency parameters of the large Gaussian ellipsoid based on the color parameters and transparency parameters of each adjacent Gaussian point; and use the large Gaussian ellipsoid as a shared rendering primitive.

[0064] Optionally, the device is further configured to: estimate the projected area of ​​each candidate Gaussian point and each shared rendering primitive in screen space; when the projected area is greater than or equal to a second area threshold, assign the corresponding Gaussian point or shared rendering primitive to the batch generation shader path, so as to perform a single core rendering calculation on the candidate Gaussian point and shared rendering primitive that did not participate in the spatial aggregation process through the shader that supports batch primitive generation; when the projected area is less than the second area threshold, assign the corresponding Gaussian point or shared rendering primitive to the single-point instantiation shader path, so as to perform rendering calculation through the single-point instantiation shader.

[0065] Optionally, the device is also used to: monitor the occupancy rate of each computing unit of the graphics processing unit in real time, and dynamically adjust the second area threshold based on the occupancy rate.

[0066] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0067] like Figure 4 As shown in the embodiment of this application, an electronic device 600 includes a processor 601, a memory 602, and a bus. The memory 602 stores machine-readable instructions that can be executed by the processor 601. When the electronic device is running, the processor 601 communicates with the memory 602 via the bus. The processor 601 executes the machine-readable instructions to perform the steps of the rendering method for three-dimensional Gaussian point clouds described above.

[0068] Specifically, the memory 602 and processor 601 mentioned above can be general-purpose memory and processor, without any specific limitations. When the processor 601 runs the computer program stored in the memory 602, it can execute the above-mentioned three-dimensional Gaussian point cloud rendering method.

[0069] The processor 601 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 601 or by instructions in software form. The processor 601 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 602, and processor 601 reads the information from memory 602 and, in conjunction with its hardware, completes the steps of the above method.

[0070] Corresponding to the above-described rendering method for 3D Gaussian point clouds, this application embodiment also provides a computer-readable storage medium storing machine-executable instructions. When the machine-executable instructions are called and run by a processor, the machine-executable instructions cause the processor to perform the steps of the above-described rendering method for 3D Gaussian point clouds.

[0071] The 3D Gaussian point cloud rendering device provided in this application embodiment can be specific hardware on a device or software or firmware installed on the device. The device provided in this application embodiment has the same implementation principle and produced the same technical effects as the foregoing method embodiments. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0072] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0073] For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0074] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0075] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0076] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the three-dimensional Gaussian point cloud rendering method described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0077] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0078] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for rendering three-dimensional Gaussian point clouds, characterized in that, The method includes: Acquire three-dimensional Gaussian point cloud data, which includes spatial distribution parameters and visual attribute parameters of multiple Gaussian points; Based on the spatial distribution parameters, the primitive validity of the multiple Gaussian points is evaluated, and invalid Gaussian points are eliminated to obtain a set of candidate Gaussian points. Based on the spatial distribution parameters and the visual attribute parameters, adjacent Gaussian points in the candidate Gaussian point set that meet the preset aggregation conditions are spatially aggregated to generate shared rendering primitives. By using a shader that supports batch primitive generation, single core rendering calculations are performed on candidate Gaussian points that did not participate in the spatial aggregation process and the shared rendering primitives to obtain the corresponding baseline rendering parameters. Based on the aforementioned baseline rendering parameters, multiple vertex data for the corresponding rendering facets are generated in batches for pixel rendering. Specifically, based on the spatial distribution parameters, the primitive validity of the plurality of Gaussian points is evaluated, invalid Gaussian points are eliminated, and a candidate Gaussian point set is obtained, including: Based on the spatial distribution parameters, calculate the projection range of each Gaussian point in the screen space; When the projection range is outside the preset screen boundary, or the area of ​​the projection range is less than the first area threshold, the corresponding Gaussian point is determined as the invalid Gaussian point and discarded. Get the depth buffer information of the previous frame's rendered image; Based on the depth buffer information, the remaining Gaussian points after projection range elimination are subjected to occlusion test, and the Gaussian points that are completely occluded are eliminated to obtain the candidate Gaussian point set. Specifically, based on the spatial distribution parameters and the visual attribute parameters, adjacent Gaussian points in the candidate Gaussian point set that meet preset aggregation conditions are spatially aggregated to generate shared rendering primitives, including: Calculate the spatial distance and visual feature difference between any two Gaussian points in the candidate Gaussian point set; When the spatial distance is less than a preset distance threshold and the visual feature difference is less than a preset difference threshold, the two corresponding Gaussian points are determined as adjacent Gaussian points and satisfy the preset aggregation condition. Adjacent Gaussian points that meet the preset aggregation conditions are merged into the shared rendering primitives.

2. The method according to claim 1, characterized in that, The shader that supports batch primitive generation is a mesh shader; By using a shader that supports batch primitive generation, single-core rendering calculations are performed on candidate Gaussian points that did not participate in the spatial aggregation process and the shared rendering primitives to obtain the corresponding baseline rendering parameters, including: The baseline rendering parameters are obtained by performing a coordinate matrix transformation and camera projection mapping on a single candidate Gaussian point or a single shared rendering primitive using the mesh shader. Based on the aforementioned baseline rendering parameters, multiple vertex data for the corresponding rendering facets are generated in batches, including: Based on the baseline rendering parameters and the preset fixed offset compensation rules, the four vertex data of the rendered patch are generated.

3. The method according to claim 1, characterized in that, The spatial distribution parameters include a three-dimensional covariance matrix, and the visual attribute parameters include color parameters and transparency parameters; Merging adjacent Gaussian points that meet the preset aggregation conditions into the shared rendering primitive includes: Based on the spatial positions of the adjacent Gaussian points and the three-dimensional covariance matrix, calculate the spatial position parameters and the fused covariance matrix of the merged large Gaussian ellipsoid. Based on the color parameters and transparency parameters of the adjacent Gaussian points, the blended color parameters and blended transparency parameters of the large Gaussian ellipsoid are calculated. The large Gaussian ellipsoid is used as the shared rendering primitive.

4. The method according to claim 1, characterized in that, Before performing a single core rendering calculation on candidate Gaussian points that did not participate in the spatial aggregation process and the shared rendering primitives using a shader that supports batch primitive generation, the method further includes: Estimate the projected area of ​​each candidate Gaussian point and each shared rendering primitive in screen space; When the projected area is greater than or equal to the second area threshold, the corresponding Gaussian point or shared rendering primitive is assigned to the batch generation shader path, so that the candidate Gaussian point and the shared rendering primitive that did not participate in the spatial aggregation process can be processed by the shader that supports batch primitive generation. When the projected area is less than the second area threshold, the corresponding Gaussian point or shared rendering primitive is assigned to the single-point instantiation shader path so that rendering calculations can be performed through the single-point instantiation shader.

5. The method according to claim 4, characterized in that, The method further includes: The occupancy rate of each computing unit in the graphics processing unit is monitored in real time, and the second area threshold is dynamically adjusted based on the occupancy rate.

6. A rendering device for three-dimensional Gaussian point clouds, characterized in that, The device includes: An acquisition unit is used to acquire three-dimensional Gaussian point cloud data, wherein the three-dimensional Gaussian point cloud data includes spatial distribution parameters and visual attribute parameters of multiple Gaussian points; The evaluation unit is used to evaluate the primitive validity of the multiple Gaussian points based on the spatial distribution parameters, remove invalid Gaussian points, and obtain a set of candidate Gaussian points. An aggregation unit is used to perform spatial aggregation processing on adjacent Gaussian points in the candidate Gaussian point set that meet preset aggregation conditions based on the spatial distribution parameters and the visual attribute parameters, and generate shared rendering primitives. The rendering calculation unit is used to perform a single core rendering calculation on the candidate Gaussian points that did not participate in the spatial aggregation process and the shared rendering primitives respectively through a shader that supports batch primitive generation, to obtain the corresponding baseline rendering parameters. The generation unit is used to generate multiple vertex data of the corresponding rendering facets in batches based on the reference rendering parameters for pixel rendering. The evaluation unit is further configured to: calculate the projection range of each Gaussian point in the screen space based on the spatial distribution parameters; When the projection range is outside the preset screen boundary, or the area of ​​the projection range is less than the first area threshold, the corresponding Gaussian point is determined as the invalid Gaussian point and discarded. Get the depth buffer information of the previous frame's rendered image; Based on the depth buffer information, the remaining Gaussian points after projection range elimination are subjected to occlusion test, and the Gaussian points that are completely occluded are eliminated to obtain the candidate Gaussian point set. The aggregation unit is further configured to: calculate the spatial distance and visual feature difference between any two Gaussian points in the candidate Gaussian point set; When the spatial distance is less than a preset distance threshold and the visual feature difference is less than a preset difference threshold, the two corresponding Gaussian points are determined as adjacent Gaussian points and satisfy the preset aggregation condition. Adjacent Gaussian points that meet the preset aggregation conditions are merged into the shared rendering primitives.

7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program is executed by the processor to perform the method of any one of claims 1 to 5.

Images