A three-dimensional gaussian splotch forward rendering method

Abstract

The application provides a three-dimensional Gaussian splash forward rendering method, and relates to the technical field of image processing, and the method comprises the steps of obtaining anisotropic three-dimensional Gaussian models to be rendered; simplifying the screen space projection form of the three-dimensional Gaussian model from a two-dimensional elliptical projection to a two-dimensional circular projection, and generating a compact load field; generating Gaussian records according to the image block range covered by Gaussian primitives; reading a first target Gaussian primitive in a current image block; performing coarse screening processing on the first target Gaussian primitive to determine a second target Gaussian primitive; performing fine screening processing on the second target Gaussian primitive based on a current pixel to determine a third target Gaussian primitive; reading the compact load field corresponding to the third target Gaussian primitive; and completing pixel color accumulation and transmittance updating to generate a rendered image, so as to solve the problems of complex calculation link, many intermediate variables, and large amount of data needing to be read and transmitted in the current three-dimensional Gaussian splash forward rendering process, thereby reducing the calculation efficiency.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a three-dimensional Gaussian splash forward rendering method. Background Technology

[0002] 3D Gaussian Splatting (3DGS) is an explicit rendering method for real-time new perspective compositing and 3D scene representation. In this method, the scene is explicitly represented by a large number of 3D Gaussian primitives, each containing attributes such as spatial position, scale, rotation, color, and opacity. Compared to methods such as implicit neural radiation fields, 3DGS does not require extensive sampling of every camera ray. Instead, it projects 3D Gaussians onto the image plane and performs forward blending based on the relationship between depth and transparency, making it more suitable for real-time rendering and large-scale parallel computing by graphics processing units (GPUs).

[0003] Currently, in the forward rendering process of 3DGS: First, the Gaussian plane is transformed from the world coordinate system to the camera coordinate system and then projected onto the screen space. Next, its 2D projection shape on the screen is calculated based on the Gaussian plane's scale and rotation, generating the area of ​​the image tile covered by the Gaussian plane. Then, the Gaussian plane and the tiles it covers are recorded and sorted according to tile number and depth relationship. Finally, within each tile, the rendering stage calculates the Gaussian contribution of each pixel in depth order according to the sorting results, completing color accumulation and transmittance updates to obtain the final image. To accurately represent the anisotropic 3D Gaussian projection shape, the original scheme needs to calculate the 2D covariance matrix and its inverse matrix in screen space. These parameters are used to describe the elliptical shape after Gaussian projection and participate in the pixel-level Gaussian attenuation weight calculation. Meanwhile, visual attributes such as color and opacity are usually stored in complete floating-point form and still need to be frequently read and used in calculations during the sorted rendering stage.

[0004] However, calculating the screen-space two-dimensional covariance matrix and its inverse involves numerous matrix operations, square root operations, and reciprocal operations, and requires passing a large number of intermediate variables. As the number of Gaussians and the image resolution increase in the scene, the computational overhead rises sharply, becoming one of the main performance bottlenecks of forward rendering. On the other hand, visual attributes such as color and opacity are passed between rendering stages in full floating-point form, resulting in a large data scale and cross-stage memory access overhead. Especially in the post-sorting rendering stage, random access and parsing of multiple fields are still required for each Gaussian record, making it difficult to achieve sequential data consumption and further restricting the utilization of GPU parallel efficiency. Summary of the Invention

[0005] This application provides a three-dimensional Gaussian splash forward rendering method to solve the technical problems of low computational efficiency caused by the complex calculation chain, numerous intermediate variables, and large amount of data to be read and transmitted in the current three-dimensional Gaussian splash forward rendering process.

[0006] This application provides a three-dimensional Gaussian splash forward rendering method, including: Obtain the isotropic 3D Gaussian model to be rendered; Through projection shape transformation processing, the screen space projection shape of the three-dimensional Gaussian model is simplified from a two-dimensional elliptical projection to a two-dimensional circular projection, and a compact load field containing screen coordinates, target curvature parameters, and compact visual fields is generated; the projection shape transformation processing includes: three-dimensional Gaussian isotropic simplification processing, radial focal length consistency approximation processing, and projection shape isotropic approximation processing; the compact visual field includes: color labels and opacity labels. Gaussian records are generated according to the image patch range covered by each Gaussian primitive in the three-dimensional Gaussian model; the Gaussian records are sorted according to the image patch number and depth value order covered by each Gaussian primitive; the sorting stage is completed together with the compact load field; During the rendering phase, for each pixel within the current image block, the first target Gaussian pixel in the current image block is read according to the Gaussian record; Based on the current pixel, an axis-aligned bounding box is constructed according to the effective influence radius of the first target Gaussian element, and the first target Gaussian element is coarsely screened to determine the second target Gaussian element. Based on the current pixel, a preset pixel region is determined; the preset pixel region is a circular region with the current pixel as the center and a preset length as the radius. Based on the preset pixel region, the second target high-resolution pixel is finely screened to determine the third target high-resolution pixel; Read the compact load field corresponding to the third target Gaussian primitive, and obtain the corresponding screen coordinates, target curvature parameters, and compact visual field; after all image blocks have been traversed, complete pixel color accumulation and transmittance update to generate the rendered image.

[0007] In some embodiments, the Gaussian elements in the three-dimensional Gaussian model are three-dimensional ellipsoids with different directional scales and rotational attitudes; the spatial shape of the Gaussian elements is determined by the three-dimensional covariance. The step of determining screen coordinates and simplifying the screen space projection form of the 3D Gaussian model from a 2D elliptical projection to a 2D circular projection through projection shape transformation includes: The three-dimensional Gaussian model is simplified isotropically, and the three-dimensional covariance is approximated from a direction-dependent ellipsoidal expression to a uniform-scale spherical expression; the three-dimensional covariance is: ; In the formula, The radius of the Gaussian element is represented as approximately the size of a sphere. It is the identity matrix; By using the radial focal length consistency approximation, the lateral focal length of the camera is made equal to the longitudinal focal length. Based on the horizontal and vertical focal lengths, calculate the normalized projection factor; Based on the normalized projection factor, the screen coordinates are determined and the isotropic approximation of the projection shape is used to degenerate the two-dimensional covariance of the screen space into a diagonal matrix, thus simplifying the screen space projection shape of the three-dimensional Gaussian model from a two-dimensional elliptical projection to a two-dimensional circular projection.

[0008] In some embodiments, the step of generating the target curvature parameter includes: Based on the screen space projection under the retained off-axis perspective stretching term, determine the major axis matching compensation coefficients corresponding to each Gaussian element. Obtain the original curvature parameter; the original curvature parameter is the reciprocal of the screen space scale radius; Based on the major axis matching compensation coefficient, the initial curvature parameter is corrected to generate the target curvature parameter; the target curvature parameter is: ; In the formula, Let be the major axis matching compensation coefficient of the i-th Gaussian element; These are the original curvature parameters.

[0009] In some embodiments, the step of determining the major axis matching compensation coefficients corresponding to each Gaussian element based on the screen space projection under the retained off-axis perspective stretching term includes: Based on the major and minor semi-axis of the two-dimensional elliptical projection, a major axis matching compensation coefficient is calculated to match the major semi-axis; the major axis matching compensation coefficient makes the radius of the two-dimensional circular projection match the major semi-axis of the two-dimensional elliptical projection. Based on the major axis matching compensation coefficient, and according to the normalized projection position of each Gaussian element in the current viewpoint, the major axis matching compensation coefficient corresponding to each Gaussian element is determined: The major axis matching compensation coefficient is: ; In the formula, and It is represented as the normalized coordinates of the Gaussian metacenter from the current perspective.

[0010] In some embodiments, the step of generating the compact visual field includes: K-Means clustering is performed on the color components of all Gaussian elements in the three-dimensional Gaussian model to group similar colors into the same category and assign color labels; each Gaussian element stores its corresponding color label. The opacity is divided into a finite number of opacity levels in the range of 0 to 1, and the opacity levels are stored using opacity labels; The color label and opacity label are merged into a compact visual field.

[0011] In some embodiments, the compact visual field uses a 16-bit fixed-length encoding; the high 12 bits of the fixed-length encoding are used to store color labels, and the low 4 bits are used to store opacity labels; the color labels are configured with 4096 color categories, and the opacity labels are configured with 16 levels.

[0012] In some embodiments, the step of reading the first target Gaussian cell in the current image block according to the Gaussian record includes: The compact load field corresponding to the first target Gaussian element in the current image block is read sequentially according to the sorting in the Gaussian record.

[0013] In some embodiments, the method further includes: The sorted Gaussian records are cached in a cache.

[0014] In some embodiments, the step of constructing an axis-aligned bounding box based on the effective influence radius of the first target Hoskiy cell according to the current pixel, performing coarse screening on the first target Hoskiy cell, and determining the second target Hoskiy cell includes: The effective influence radius of the first target Gaussian element is determined based on the screen coordinates and curvature parameters corresponding to the first target Gaussian element. Based on the effective radius of influence, construct an axis-aligned bounding box; If the current pixel is outside the axis-aligned bounding box, then skip the corresponding first high-order pixel; After the first target Gaussian element has been traversed, the second target Gaussian element is determined.

[0015] In some embodiments, the step of performing fine screening on the second target Gaussian pixels based on the preset pixel region to determine the third target Gaussian pixels includes: Calculate the screen space distance between the current pixel center and the two-dimensional circular projection center corresponding to the second target Gaussian element; If the screen space distance is greater than the sum of the preset length and the effective influence radius of the second target Gaussian element, then skip the corresponding second target Gaussian element; After the second target Gaussian element has been traversed, the third target Gaussian element is determined.

[0016] This application provides a three-dimensional Gaussian splash forward rendering method, comprising: acquiring an isotropic three-dimensional Gaussian model to be rendered; simplifying the screen space projection shape of the three-dimensional Gaussian model from a two-dimensional elliptical projection to a two-dimensional circular projection through projection shape transformation processing, and generating a compact load field containing screen coordinates, target curvature parameters, and a compact visual field; the projection shape transformation processing includes: three-dimensional Gaussian isotropic simplification processing, radial focal length consistency approximation processing, and projection shape isotropic approximation processing; the compact visual field includes: color labels and opacity labels; generating Gaussian records according to the image patch range covered by each Gaussian primitive in the three-dimensional Gaussian model; the Gaussian records are sorted according to the image patch number and depth value order covered by each Gaussian primitive; the sorting stage carries the compact load field together to complete the sorting; during the rendering stage, for the current image patch... For each pixel, the first target Gaussian primitive in the current image block is read according to the Gaussian record; based on the current pixel, an axis-aligned bounding box is constructed according to the effective influence radius of the first target Gaussian primitive, and the first target Gaussian primitive is coarsely screened to determine the second target Gaussian primitive; based on the current pixel, a preset pixel region is determined; the preset pixel region is a circular region with the current pixel as the center and a preset length as the radius; based on the preset pixel region, the second target Gaussian primitive is finely screened to determine the third target Gaussian primitive; the compact load field corresponding to the third target Gaussian primitive is read, and the corresponding screen coordinates, target curvature parameters, and compact visual field are obtained; after all image blocks have been traversed, pixel color accumulation and transmittance update are completed, and the rendered image is generated to reduce the complexity of projection calculation and Gaussian attenuation calculation in the three-dimensional Gaussian splash forward rendering process. Attached Figure Description

[0017] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the 3D Gaussian splash forward rendering method in this application; Figure 2 This is a schematic diagram of the two-level isotropic approximation in this application; Figure 3 This is a schematic diagram of curvature parameter compensation in this application; Figure 4 This is a schematic diagram of the compact visual field compression in this application; Figure 5 This is a schematic diagram illustrating the unfolding of multiple image block coverage relationships and compact load copying in this application; Figure 6 This application provides a schematic diagram of the composition and sorting of the compact payload fields, as well as a schematic diagram of the sequential carrying, rendering, and consumption. Figure 7 This is the current index lookup graph; Figure 8 This is a back-lookup diagram of the "portable delivery" index in this application; Figure 9 This is a schematic diagram of the fine screening process in this application; Figure 10 This is a flowchart of the fine screening process in this application; Figure 11 This is a schematic diagram of the curvature-based pixel-Gaussian early screening in this application; Figure 12 This is a flowchart of the curvature-based pixel-Gaussian early screening process in this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0020] For example, the current 3D Gaussian splash forward rendering process has the following problems: 1. Anisotropic projection calculation is complex with many intermediate variables: From a methodological perspective, existing forward rendering schemes typically retain a relatively complete anisotropic Gaussian projection calculation. That is, each Gaussian not only needs to complete coordinate transformation and screen projection, but also needs to calculate the two-dimensional covariance based on the 3D scale, rotation, and camera parameters, and then inverse it to obtain the conic parameters used for pixel contribution calculation. The advantage of this scheme is its strong expressive power, which can describe the elliptical shape after Gaussian projection relatively completely; however, for real-time rendering scenarios, its calculation chain is long, with many intermediate variables, and the amount of data that needs to be read and transmitted in subsequent stages is also large.

[0021] 2. The sorting results mainly carry Gaussian numbers, but the rendering stage still needs to look up attributes and intermediate variables: Besides the complexity of projection calculation itself, existing solutions also have shortcomings in cross-stage data organization. Existing solutions typically use Gaussian numbers as the main load after sorting during the sorting stage. The sorting results can tell the rendering stage which Gaussians should be processed in which order within a given image patch, but the rendering stage still needs to look up screen coordinates, cone parameters, color, opacity, and other attributes or intermediate variables based on the Gaussian numbers. Since the access order after sorting is different from the storage order of the original Gaussian array, this "sorting index combined with attribute lookup" method is prone to random memory access, decreased cache locality, and repeated reads. In cases where a Gaussian covers multiple image patches, the same Gaussian attribute may be repeatedly accessed during the rendering of multiple image patches.

[0022] 3. The large size of visual attribute data makes it unsuitable as a direct load for the sorting stage: In existing solutions, visual attributes such as color and opacity are typically stored in relatively complete floating-point or multi-component forms. Color usually includes three RGB components, and opacity requires an additional component. If it is desired to carry more information needed for the rendering stage simultaneously during the sorting stage, these visual attributes will significantly increase the sorting load width, memory transfer volume, and sorting pressure. Therefore, existing solutions lack a compact visual attribute representation suitable for synchronous transmission during the sorting stage, forcing the rendering stage to rely on sorted Gaussian numbers to look up the original attributes.

[0023] 4. The skipping of invalid Gaussian contributions in the pixel blending stage is not advanced enough, resulting in redundant calculations: Current rendering stages typically have pixel threads within an image patch traverse the corresponding Gaussian sequence and decide whether to accumulate color based on the magnitude of the Gaussian contribution to the pixel. For Gaussians that are far from the current pixel and have a small actual contribution, it may still be necessary to calculate the distance, attenuation, or opacity contribution before determining whether to skip them. This approach generates some invalid calculations when the Gaussian coverage is large and the number of Gaussians within the image patch is large, especially increasing the computational burden on the pixel blending stage.

[0024] In summary, the main shortcomings of the current 3D Gaussian splash forward rendering process are as follows: First, the calculation of the screen space projection shape is relatively complex, requiring the generation and transmission of many intermediate variables; second, the sorting stage mainly transmits Gaussian numbers, while the rendering stage still needs to look up the original attributes or intermediate variables, resulting in discontinuous memory access and repeated readings; third, the large scale of visual attribute data such as color and opacity is not conducive to serving as a compact load that can be synchronously transmitted during the sorting stage; and fourth, the skipping of invalid Gaussian contributions in the pixel blending stage is not advanced enough, resulting in some redundant calculations.

[0025] To address the technical problems of low computational efficiency caused by the complex computational chain, numerous intermediate variables, and large amount of data to be read and transmitted in the aforementioned 3D Gaussian splash forward rendering process, this application provides a 3D Gaussian splash forward rendering method, which is described below: like Figure 1 The diagram shown is a flowchart of the three-dimensional Gaussian splash forward rendering method in this application.

[0026] It is worth noting that this application pertains to a 3D Gaussian forward rendering pipeline. Input data includes a 3D Gaussian model, camera extrinsic parameters, camera intrinsic parameters, image size, and tile (image patch) size. The 3D Gaussian model contains information such as the center position, scale, color, and opacity of the Gaussian elements; the camera extrinsic parameters are used to transform the Gaussian elements from the world coordinate system to the camera coordinate system; the camera intrinsic parameters are used to project the camera coordinates onto the image plane; and the image size and tile size are used to determine the screen partitioning method and the tile extent covered by each Gaussian element.

[0027] The overall process can be divided into three stages: preprocessing, sorting, and rendering. The preprocessing stage treats each Gaussian as a single object, performing geometric information unpacking, coordinate transformation, visibility filtering, screen projection, covering tile range generation, depth quantization, and compact load packing. After preprocessing, each visible Gaussian receives its screen coordinates, screen space scale or curvature parameters, covering tile range, quantization depth, and the compact load required for the rendering stage; invisible or invalidally projected Gaussians will not proceed to the subsequent sorting and rendering processes.

[0028] The sorting phase takes the preprocessed output as input and expands the relationship of a Gaussian covering multiple tiles into multiple tile-Gaussian records, each corresponding to the relationship of "a certain Gaussian affects a certain tile". Subsequently, these records are sorted according to tile number and depth order, ensuring that Gaussian records within the same tile are stored contiguously and that the rendering order within the tile is satisfied from front to back. After sorting, a sorted compact load sequence is generated, and the start and end positions of each tile within this sequence are recorded.

[0029] The rendering phase processes tiles as objects. Based on the tile record range generated in the sorting phase, the sorted compact load corresponding to the current tile is read sequentially. For each pixel within a tile, the relevant Gaussian spheres are traversed in depth order, the contribution of the Gaussian spheres to the pixel is calculated, and color accumulation and transmittance updates are performed to finally obtain the output image.

[0030] Currently, 3D Gaussian forward rendering typically includes Gaussian numbers during the sorting stage. After sorting, although the rendering stage knows the processing order of Gaussians within each tile, it still needs to look up data such as screen coordinates, cone parameters, color, and opacity based on the Gaussian numbers. This application, however, generates a compact payload required for rendering in advance during the preprocessing stage, sorts this payload along with the tile-Gaussian records during the sorting stage, and sequentially reads the sorted compact payloads and directly performs blending during the rendering stage.

[0031] Through the aforementioned data flow, this application replaces the traditional "sorting Gaussian numbers and back-looking up attributes during the rendering stage" with "sorting with compact load and sequentially consuming load during the rendering stage." This data organization method reduces random back-looking up of the original Gaussian attribute table and intermediate variable table during the rendering stage after sorting, and provides a unified data transfer foundation for subsequent algorithm simplification, visual field compression, and pixel-Gaussian early screening.

[0032] The specific plan is as follows: For example, in the original 3D Gaussian forward rendering, the Gaussian projection onto the image plane typically manifests as a two-dimensional ellipse. To describe the coverage and pixel contribution of this two-dimensional ellipse, the original scheme usually needs to first calculate the screen space two-dimensional covariance from the 3D covariance, and then calculate the screen coverage scale and conic parameters from the two-dimensional covariance. This process involves 3D covariance construction, projection Jacobian matrix transformation, two-dimensional covariance eigenvalue calculation, matrix inversion, and elliptic quadratic form calculation, resulting in a long computational chain and many intermediate variables.

[0033] The algorithm simplification in this application proceeds in two progressive layers. The first layer, under the simplification of three-dimensional Gaussian isotropic simplification and radial focal length consistency approximation, reorganizes the screen space scale and conic parameters in the original scheme into regular expressions concerning normalized coordinates, normalized Gaussian scale, and uniform focal length. This step still retains the off-axis perspective stretching term, so the screen projection shape can still be represented as a two-dimensional ellipse. The second layer, based on the above regular expressions, further adopts the isotropic approximation of the projection shape, ignoring the off-axis perspective stretching term when calculating the screen space scale and conic parameters, approximating the two-dimensional elliptical projection as a two-dimensional circular projection. Through these two simplification layers, the complex two-dimensional covariance inversion and elliptic quadratic form calculation in the original scheme can ultimately be simplified to the calculation of circular distance based on screen coordinates, screen space scale radius, and curvature parameters.

[0034] It is important to note that while the isotropic approximation of the projection shape can significantly reduce the computational complexity of the two-dimensional covariance and conic parameters, this approximation replaces the original two-dimensional elliptical projection with a circular projection, compressing the original broadening along the major axis. Therefore, after obtaining the basic curvature parameters, this application further introduces curvature parameter compensation based on major axis matching to correct the basic curvature parameters to the compensated curvature parameters, which are then used for subsequent compact load transfer, pixel contribution calculation, and pixel-Gaussian early screening.

[0035] Specifically, this application provides a three-dimensional Gaussian splash forward rendering method, including: S100: Obtain the isotropic 3D Gaussian model to be rendered; in the 3D Gaussian model, Gaussian primitives are 3D ellipsoids with different directional scales and rotational orientations; the spatial shape of the Gaussian primitives is determined by the 3D covariance. It is worth noting that in the original 3D Gaussian representation, a Gaussian is usually not a simple sphere, but a 3D ellipsoid with different directional scales and rotational orientations. This spatial shape is usually described by the 3D covariance, that is, determined by the scale matrix and the Gaussian rotation matrix. ; In the formula, R represents the Gaussian rotation matrix, and S represents the scaling matrix. T S T These are their transpose matrices, respectively. Their advantage is that they can express the directionality and anisotropy of Gaussians in 3D space, but during forward rendering, they also make subsequent projection calculations dependent on the complete covariance matrix.

[0036] The three-dimensional covariance is combined with the projection Jacobian matrix and the camera transformation matrix to further calculate the two-dimensional covariance in screen space: ; In the formula, J represents the projection Jacobian matrix, and W represents the rotation component in the camera transformation. After obtaining the two-dimensional covariance, the original scheme usually needs to estimate the screen coverage radius based on its eigenvalues. If the two-dimensional covariance is written as: .

[0037] Then its eigenvalue λ can be expressed as: .

[0038] Screen space major axis σ major The larger eigenvalue is used, i.e., the above formula is plused; the screen space scale R is also calculated from the larger eigenvalue. .

[0039] Simultaneously, to calculate the pixel contribution, the original scheme also requires finding the inverse matrix of the two-dimensional covariance, i.e., the cone parameters. Let: .

[0040] The parameters of the cone are: .

[0041] Pixel-level Gaussian decay during the rendering phase requires the use of a complete quadratic form: .

[0042] As can be seen, the original scheme requires several steps from 3D Gaussian to pixel contribution, including 3D covariance construction, 2D covariance projection, eigenvalue calculation, matrix inversion, and elliptic quadratic form calculation. For scenarios with a large number of Gaussian parallel renderings, this process incurs high computational cost and intermediate variable transfer overhead.

[0043] S200: Through projection shape transformation processing, the screen space projection shape of the three-dimensional Gaussian model is simplified from a two-dimensional elliptical projection to a two-dimensional circular projection, and a compact load field containing screen coordinates, target curvature parameters, and compact visual fields is generated; the projection shape transformation processing includes: three-dimensional Gaussian isotropic simplification processing, radial focal length consistency approximation processing, and projection shape isotropic approximation processing; the compact visual field includes: color labels and opacity labels.

[0044] The step of determining the screen coordinates and simplifying the screen space projection of the 3D Gaussian model from a 2D elliptical projection to a 2D circular projection through projection shape transformation includes the following sub-steps: S210: The three-dimensional Gaussian model is simplified isotropically, approximating the three-dimensional covariance from a direction-dependent ellipsoidal expression to a sphere of uniform scale. This application first simplifies the three-dimensional Gaussian model isotropically, approximating the Gaussian ellipsoid in three-dimensional space as a Gaussian sphere, no longer distinguishing between different scales and rotations, but using a uniform scale or radius r to describe the size of the Gaussian. The three-dimensional covariance can be approximated as: ; In the formula, The radius of the Gaussian element is represented as approximately the size of a sphere. Let be the identity matrix; through this approximation, the spatial form of the three-dimensional Gaussian is simplified from a "direction-dependent ellipsoid" to a "direction-independent sphere". However, the simplification of the three-dimensional Gaussian isotropically is not enough to obtain the final simplified formula, because the different horizontal and vertical focal lengths of the camera will still lead to inconsistencies in the horizontal and vertical scale expressions after projection.

[0045] S220: Utilizing radial focal length consistency approximation, the camera's lateral focal length is made equal to its longitudinal focal length. To ensure a unified and simplifiable form for subsequent screen space scales and conic parameters, this scheme further employs radial focal length consistency approximation, assuming that the lateral and longitudinal focal lengths are approximately equal. .

[0046] S230: Calculate the normalized projection factor based on the horizontal and vertical focal lengths.

[0047] Specifically, let the center of Gauss in the camera coordinate system be... And define normalized coordinates and normalized Gaussian scale: .

[0048] Under the three-dimensional Gaussian isotropic and radial focal length uniformity approximation, the screen space major axis scale originally calculated from the complete two-dimensional covariance eigenvalues ​​can be reorganized into the following regular expression: .

[0049] This means that the screen scale, which originally required two-dimensional covariance, eigenvalues, and square roots, can be reorganized into a scalar expression related to Gaussian normalized scale, unified focal length, and normalized horizontal and vertical coordinates.

[0050] Similarly, the inverse covariance matrix of two dimensions can also be rearranged into a more regular form. Define intermediate quantities: .

[0051] The two-dimensional covariance inverse matrix can then be written as: .

[0052] Thus, the parameters of the cone are obtained: .

[0053] After the 3D shape is simplified to a uniform scale and the horizontal and vertical focal lengths are approximately the same, the projection scale of the Gaussian on the screen in both directions can be described using the same projection factor. At this point, the screen spatial scale, conic parameters, and screen center coordinates can all be organized around the camera coordinates, depth, and uniform focal length. The regularized expressions for the screen scale and conic parameters have already been explained; below, a simplified form of the screen coordinates is given to illustrate how the final screen position parameters needed for the preprocessing stage are obtained.

[0054] The original algorithm projection process typically involves several intermediate steps, including world coordinates to camera coordinates, clip space transformation, normalized device coordinates, and pixel coordinates. First, the Gaussian center is transformed from the world coordinate system to the camera coordinate system, i.e., by performing a matrix multiplication between the view matrix and the world coordinate vector to obtain the camera coordinate vector. .

[0055] Subsequently, the camera coordinates are mapped to the clip space using a projection matrix. Clip space coordinates can be understood as the homogeneous projection result before perspective division; their horizontal coordinates are simultaneously influenced by the camera focal length, image width, principal point position, and depth term. Taking the horizontal coordinate as an example, expanding the matrix-vector multiplication yields the clip space coordinates: ; In the formula, W is the image width. Next, perspective division is performed to convert the crop space coordinates to normalized device coordinates: .

[0056] Then map the standardized device coordinates to pixel coordinates: .

[0057] Substituting the clip space coordinates and normalized device coordinates into the above formula and rearranging, we can eliminate the intermediate clip space and normalized device coordinates, directly obtaining the simplified form from camera coordinates to pixel coordinates: .

[0058] Based on the radial focal length consistency approximation, common terms can be further merged in actual calculations. This is because both screen coordinates and subsequent screen space scale calculations include f / z. c First, calculate the normalized projection factor: .

[0059] In summary, the above process completes the first layer of algorithm simplification. First, in terms of the 3D Gaussian shape, the original anisotropic ellipsoid is approximated as a Gaussian sphere represented by a uniform scale. Second, in terms of camera projection parameters, a radial focal length consistency approximation is introduced, allowing the screen space scale and conic parameters to be organized into regular expressions about normalized coordinates, normalized Gaussian scale, and uniform focal length. Simultaneously, the screen coordinates can be simplified around the camera coordinates and the normalized projection factor. It should be noted that the above process still retains the off-axis perspective stretching term caused by normalized coordinates; therefore, the 2D projection shape can still be represented as an ellipse. The following process will further simplify the 2D elliptical projection into a 2D circular projection by using an isotropic approximation of the projection shape.

[0060] For example, after the above process of 3D Gaussian isotropic simplification and radial focal length uniformity approximation, the Gaussian can be described using a uniform scale. However, the off-axis perspective stretching term after projection is still retained, and its screen space projection may still appear as an ellipse. Specifically, the farther away from the camera's optical axis, the more obvious the perspective stretching caused by the normalized horizontal and vertical coordinates, and the screen's 2D covariance will show a term related to x. norm y norm Related items, which can cause inconsistencies in horizontal and vertical scales and overlapping items.

[0061] This application further employs an isotropic approximation of the projection shape, such as Figure 2 As shown. This approximation does not involve setting x when calculating the coordinates of the Gaussian screen center. norm and y norm The value is 0, but the x-axis is ignored when calculating the screen space scale and cone parameters. norm y norm The resulting off-axis perspective stretching term. Its physical meaning is: the Gaussian center is still calculated using the real position, but the two-dimensional shape of the Gaussian after being projected onto the image plane is approximately circular.

[0062] S240: Based on the normalized projection factor, determine the screen coordinates and use the isotropic approximation of the projection shape to degenerate the two-dimensional covariance of the screen space into a diagonal matrix, thereby simplifying the screen space projection shape of the three-dimensional Gaussian model from a two-dimensional elliptical projection to a two-dimensional circular projection.

[0063] Specifically, the corrected principal point coordinates are defined as follows: .

[0064] The screen coordinates can then be written as: ; .

[0065] Under isotropic Gaussian and focal length uniformity approximations, if the off-axis term is retained, the screen size will still be affected by x. norm y norm The effect; when these terms are ignored in the calculation of scale and conic parameters, σ major It degenerates into a basic screen space scale. This scale is denoted as the screen space scale radius r. p : .

[0066] Under this approximation, the two-dimensional covariance in screen space degenerates into a form with two equal directions and zero cross term: .

[0067] Its inverse matrix is: .

[0068] Therefore, the original cone parameters become: .

[0069] Furthermore, this application defines the reciprocal of the screen space scale radius as the original curvature parameter: .

[0070] During the rendering phase, for the current pixel, let its offset from the center of the Gaussian screen be: .

[0071] Substituting the above conclusions into... The final Gaussian decay formula simplifies to: .

[0072] Therefore, the three cone parameters A, B, and C that needed to be passed and used in the original scheme are replaced by a single curvature parameter in this application. Before simplification, the rendering stage needed to read the screen coordinates and the complete cone parameters, and calculate the elliptic quadratic form with intersection terms; after simplification, the rendering stage only needs to read the screen coordinates and the curvature parameter, and calculate the normalized distance under the circular projection.

[0073] The above steps are key to the algorithm simplification in this application. They not only reduce the overhead of two-dimensional covariance inversion and cone parameter generation in the preprocessing stage, but also reduce the number of intermediate variables that need to be passed in subsequent stages, providing a direct basis for compact load packing and "portable transfer".

[0074] For example, after isotropic approximation of the projection shape, the original 2D elliptical projection is simplified to a circular projection. This simplification allows the rendering stage to perform Gaussian decay calculations using only screen coordinates and curvature parameters, thus avoiding explicit calculation and transmission of the complete 2D covariance and conic parameters. However, this approximation also introduces a problem: the simplified circle is not the equivalent central circle of the original ellipse, but rather the concentric inscribed circle corresponding to the minor semi-axis of the original ellipse, thus losing the broadening of the original ellipse along its major axis.

[0075] like Figure 3 The figure shown is a schematic diagram of curvature parameter compensation in this application.

[0076] The steps for generating the target curvature parameter include the following: S250: Obtain the two-dimensional elliptical projection of the three-dimensional Gaussian model.

[0077] S260: Determine the major axis matching compensation coefficients corresponding to each Gaussian element based on the screen space projection under the retained off-axis perspective stretching item.

[0078] The step of determining the major axis matching compensation coefficients corresponding to each Gaussian element based on the screen space projection under the retained off-axis perspective stretching term includes the following sub-steps: S261: Calculate the major axis matching compensation coefficient that matches the major axis based on the major and minor axes of the two-dimensional elliptical projection; the major axis matching compensation coefficient that matches the major axis makes the radius of the two-dimensional circular projection match the major axis of the two-dimensional elliptical projection.

[0079] Specifically, the original two-dimensional covariance matrix is: .

[0080] This matrix can be further written as: .

[0081] Therefore, the variances of the two principal axes of the original elliptical projection are as follows: .

[0082] Correspondingly, the minor and major semi-axis of the original ellipse are: .

[0083] Therefore, the simplified circular projection radius is equal to the minor semi-axis of the original ellipse, not the midpoint between the major and minor semi-axis. In other words, this simplification replaces the original ellipse with a concentric inscribed circle, thus making the Gaussian kernel narrower along its major axis. To compensate for this narrowing, this application introduces curvature parameter compensation based on major axis matching. Let the compensated curvature parameter be: ; In the formula, Let be the curvature compensation coefficient. Since a smaller curvature parameter corresponds to a larger effective radius of influence, the reasonable compensation direction should be to let . <1 means that the curvature parameter is appropriately reduced to expand the coverage of the simplified circular Gaussian.

[0084] If we want the compensated circular Gaussian radius to match the major semi-axis of the original ellipse, then we have: .

[0085] Therefore, the major axis matching compensation coefficient can be obtained: .

[0086] S262: Based on the major axis matching compensation coefficient, determine the major axis matching compensation coefficient corresponding to each Gaussian element according to the normalized projection position of each Gaussian element in the current view: The major axis matching compensation coefficient is: ; In the formula, and It is represented as the normalized coordinates of the Gaussian metacenter from the current perspective. Let represent the compensation coefficient of the i-th Gaussian.

[0087] S270: Obtain the original curvature parameter; the original curvature parameter is the reciprocal of the screen space scale radius.

[0088] S280: Based on the major axis matching compensation coefficient, the initial curvature parameter is corrected to generate the target curvature parameter; the target curvature parameter is: ; In the formula, Let be the major axis matching compensation coefficient of the i-th Gaussian element; The original curvature parameters are used. Target curvature parameter compensation occurs after the isotropic approximation of the projected shape in the preprocessing stage and before compact load packaging. In other words, the compensated curvature parameters are used in subsequent compact load packaging, sorting, and rendering stages, not the uncompensated basic curvature parameters.

[0089] For example, after algorithm simplification, the amount of intermediate geometry required in the rendering stage has been significantly reduced, but color and opacity can still be the main burden of "carry-through transmission". Currently, 3D Gaussian forward rendering typically saves and reads the color and opacity of each Gaussian element directly. Color generally includes three components: red, green, and blue, and opacity requires an additional component. If these attributes participate in cross-stage transmission in full numerical form, even if the geometric parameters have been compressed, the overall load is still relatively large.

[0090] like Figure 4 The diagram shown is a schematic of the compact visual field compression in this application.

[0091] The steps for generating the compact visual field include the following: S290: Perform K-Means clustering on the color components of all Gaussian elements in the three-dimensional Gaussian model to group similar colors into the same category and assign color labels; each Gaussian element stores its corresponding color label.

[0092] Specifically, this application further compresses visual attributes. The core idea is to stop carrying a complete color for each Gaussian color and instead use shorter labels to represent color categories. Then, during the rendering stage, approximate colors are recovered using a color table. Specifically, in the model preprocessing stage, K-Means clustering is performed on all Gaussian colors, grouping similar colors into the same category and saving a representative color for each category. Each Gaussian color only saves its corresponding color category number. During rendering, the red, green, and blue colors of that Gaussian color are obtained by looking up the color table based on this number.

[0093] S291: Divide the opacity into a finite number of opacity levels in the range of 0 to 1, and store the opacity levels using opacity labels.

[0094] Specifically, this application uses shorter quantization labels to represent opacity. Experimental observations show that most opacities in the model being processed are concentrated in the higher range, especially a large number of Gaussian opacities approaching 1. Therefore, this application no longer stores complete floating-point values ​​for opacity, but instead divides the range of 0 to 1 into a finite number of levels, storing the opacity levels using a small bit width. The rendering stage then recovers an approximate opacity based on these levels.

[0095] S292: Merge the color label and the opacity label into a compact visual field. The compact visual field uses a 16-bit fixed-length encoding; the high 12 bits of the fixed-length encoding are used to store the color label, and the low 4 bits are used to store the opacity label; the color label is configured with 4096 color categories, and the opacity label is configured with 16 levels.

[0096] Specifically, the compact visual field occupies 16 bits, with the high 12 bits representing the color cluster label and the low 4 bits representing the opacity level. The color cluster label is used to look up the color table, and the opacity level is used to recover the transparency value. In this way, color and opacity, which originally required multiple numerical components to be stored, are compressed into a single compact visual field.

[0097] It's worth noting that color clustering and opacity quantization are not isolated data compression operations, but rather serve the subsequent "portable transfer." This is because only when visual attributes are sufficiently compact can the visual information required during the rendering stage be packaged together with geometric information such as screen coordinates and curvature parameters, and then transferred along with the sorted records. Otherwise, if the complete color and opacity are still carried, the load on the sorting stage will increase significantly, and the benefits of "portable transfer" will be negated.

[0098] In addition to visual attributes, this application can also quantize and store model geometric information. For example, Gaussian position, scale, and other geometric information can be saved in a compact format, read during the preprocessing stage, and then unpacked for use. This quantization primarily aims to reduce the data scale during model reading and inter-stage transfer, without altering the basic workflow of 3D Gaussian forward rendering. Compared to geometric quantization itself, this application emphasizes the coordination between visual field compression and subsequent "portable transfer."

[0099] After the above processing, the data that actually needs to be passed to the rendering stage along with the sorting results is compressed into: Gaussian screen coordinates, curvature parameters, and a compact visual field. The compact visual field is responsible for restoring color and opacity, while the screen coordinates and curvature parameters are responsible for calculating pixel contribution. Thus, the information required for the rendering stage is converged into a smaller data packet, providing a foundation for the next step of "carry-through transmission."

[0100] Furthermore, this application adopts a numerical organization strategy of "fixed-point at stage boundaries and floating-point within stages". That is, the original numerical values ​​are quantized in fixed-point form, but fixed-point calculation is not used throughout the entire process. Instead, fixed-point numbers or compact coding are used when storing across stages, sorting key-value pairs, and transferring compact loads. Within each stage, single-precision floating-point numbers are still mainly used to complete the calculations.

[0101] Specifically, the preprocessing stage reads input data in fixed-point or compact encoding form, internally converts it to single-precision floating-point numbers for coordinate transformation, screen projection, and scale calculation, and reverts to fixed-point or compact encoding when outputting quantization depth, coverage, and compact load. The sorting stage primarily processes tile numbers, quantization depth, and compact load, using fixed-point form throughout. The rendering stage reads the sorted compact load, restores it to a numerical form suitable for calculation, and uses single-precision floating-point numbers to calculate pixel contribution and color accumulation. Since the rendering stage output only contains the RGB parameters of the image, the overall data scale is controllable, and the final output can be directly in single-precision floating-point format.

[0102] By employing the above strategy, this application retains the bandwidth benefits of fixed-point numbers in cross-stage transmission, sorting, and load compression, while avoiding the introduction of complex fixed-point arithmetic control overhead within stages, thus balancing data compactness, implementation complexity, and computational stability.

[0103] S300: Generate Gaussian records according to the image patch range covered by each Gaussian primitive in the three-dimensional Gaussian model; the Gaussian records are sorted according to the image patch number and depth value order covered by each Gaussian primitive; the sorting stage is completed together with the compact load field.

[0104] It is worth noting that the Gaussian records are sorted together with their corresponding compact payload fields during the sorting process, i.e., "carry-through". Specifically, the Gaussian records are sorted according to the image patch number and depth value covered by each Gaussian primitive.

[0105] For example, in the current 3D Gaussian forward rendering process, the sorting stage typically mainly passes Gaussian numbers. After sorting, although the rendering stage knows which Gaussians should be processed in which order within a certain tile, it still needs to retrieve data such as screen coordinates, cone parameters, color, and opacity from the original array or intermediate result array based on the Gaussian numbers. This approach can be understood as "passing numbers during sorting and looking them up during rendering".

[0106] The problem with the above lookup method is that the sorted Gaussian order is usually different from the storage order of the original Gaussian array. When accessing Gaussian properties according to the sorting result during the rendering phase, it is easy to cause discontinuous reads. Furthermore, a Gaussian array may cover multiple tiles, and the same Gaussian property may be read repeatedly during the rendering of multiple tiles. For scenarios that require frequent forward rendering, this kind of random memory access and repeated reads will limit the overall efficiency.

[0107] This application proposes a "portable transfer" scheme after simplifying the algorithm and compressing visual attributes. The core idea is that since the data actually needed in the rendering stage has been compressed into a smaller load, it is not necessary to carry only Gaussian numbers in the sorting stage. Instead, the compact load required in the rendering stage can be sorted and transferred together with the tile-Gaussian records.

[0108] Specifically, in the preprocessing stage, after completing coordinate transformation, visibility determination, screen projection, and coverage calculation, a compact payload is generated. This compact payload contains Gaussian screen coordinates, curvature parameters, and compact visual fields. Subsequently, the cardinality sorting stage expands a Gaussian into multiple tile-Gaussian records based on the tile range covered by the Gaussian. The sorting of each record is still based on tile number and depth order, but the content carried by this record is no longer simply the Gaussian number; instead, it contains the compact payload generated in the preprocessing stage, such as... Figure 5 As shown.

[0109] After sorting, the Gaussian loads within the same tile will form contiguous intervals in the sorted array. The system then generates a tile offset table to mark the start and end positions of each tile. During the rendering phase, when processing a tile, it only needs to find the corresponding contiguous interval based on the tile offset table and read the compact loads within it sequentially, such as... Figure 6 As shown. After reading, the rendering stage can directly unpack the data to obtain the screen coordinates, curvature parameters, color labels, and opacity labels, and complete pixel blending.

[0110] S400: During the rendering phase, for each pixel within the current image block, the first target Gaussian pixel in the current image block is read according to the Gaussian record.

[0111] The step of reading the first target Gaussian cell in the current image block according to the Gaussian record includes the following sub-steps: S410: Read the compact load field corresponding to the first target Gaussian primitive in the current image block in the order of the Gaussian record.

[0112] It is worth noting that this application will currently use "sorted index, rendering back lookup", such as Figure 7 As shown, change it to "sorting carries load, rendering consumes in order", such as Figure 8 As shown. The key to this change is that the sorting stage carries the compact data that the rendering stage actually needs, rather than numbers used for indirect access. This reduces back-looking up the original Gaussian attribute table and intermediate variable table during the rendering stage, reducing random memory access and repeated reads.

[0113] It is important to emphasize that "portable transfer" is not simply about putting more data into the sorting results. Without the aforementioned algorithmic simplification and visual field compression, directly carrying the complete cone parameters, complete color, and opacity would lead to an excessive sorting load, thus reducing efficiency. The progressive relationship in this application is as follows: first, geometric intermediate quantities are reduced through projection morphology simplification; then, the amount of attribute data is reduced through visual compression; and finally, a compact load suitable for sorting is formed. It is precisely because the load is compact enough that "portable transfer" has practical significance.

[0114] The method further includes the following steps: S420: Cache the sorted Gaussian records into a cache.

[0115] It's worth noting that in the parallel implementation, the sorted compact workload can be read into the on-chip cache or shared memory in batches during the rendering phase, allowing multiple pixel threads within the same tile to reuse the same batch of Gaussian workload. This way, the same batch of workload can serve the early screening and blending calculations of multiple pixels, reducing redundant reads. This batch reuse, combined with two-level pixel-Gaussian early screening, reduces subsequent accesses and use of the Gaussian workload in the shared memory of the GPU streaming multiprocessor by pixel threads, and reduces the redundant overhead caused by exponential decay, opacity contribution calculations, and color accumulation, thereby simultaneously alleviating memory access and computational pressure.

[0116] S500: Based on the current pixel, construct an axis-aligned bounding box according to the effective influence radius of the first target Gaussian pixel, perform coarse screening on the first target Gaussian pixel, and determine the second target Gaussian pixel.

[0117] For example, during the rendering phase, a tile typically contains multiple Gaussians. A single pixel thread needs to traverse these Gaussians in depth order and calculate their color contribution to the current pixel. Currently, even if a Gaussian is far from the current pixel and ultimately contributes almost nothing, the decay exponent and transparency still need to be calculated before deciding whether to skip it based on the contribution magnitude. This leads to a lot of unnecessary calculations, especially since the exponent calculation itself is costly.

[0118] This application utilizes the aforementioned curvature parameter for pixel-Gaussian early screening. The curvature parameter is essentially related to the reciprocal of the radius of the Gaussian in screen space, so the effective influence range of the Gaussian on the image plane can be quickly estimated based on the curvature during the rendering stage.

[0119] like Figure 11 and Figure 12 The diagram shown is a schematic of pixel-Gaussian early screening based on curvature.

[0120] The step of constructing an axis-aligned bounding box based on the effective influence radius of the first target Gaussian pixel based on the current pixel, performing coarse screening on the first target Gaussian pixel, and determining the second target Gaussian pixel includes the following sub-steps: S510: Determine the effective influence radius of the first target Gaussian element based on the screen coordinates and curvature parameters corresponding to the first target Gaussian element.

[0121] S520: Based on the effective influence radius, construct an axis-aligned bounding box.

[0122] S530: If the current pixel is outside the axis-aligned bounding box, skip the corresponding first high-order pixel.

[0123] S540: After the first target Gaussian element has been traversed, the second target Gaussian element is determined.

[0124] Specifically, this application divides pixel-Gaussian coverage determination into two levels: coarse screening and fine screening. First, during the rendering stage, an axis-aligned bounding box of the Gaussian in screen space is constructed based on the Gaussian's screen coordinates and the effective influence radius estimated by the curvature parameter. For pixels within the current tile, if the pixel's position is clearly outside this bounding box, it can be considered that the pixel is unlikely to be effectively affected by the Gaussian, and the rendering stage can skip the Gaussian before subsequent exponential decay calculations. This bounding box determination mainly involves coordinate comparison, has a low computational cost, and can serve as a preliminary screening for pixel-level coverage relationships.

[0125] It is worth noting that the coarse screening excludes high-order pixels that do not intersect with the current pixel, in order to reduce the pressure on subsequent shared memory accesses.

[0126] like Figure 9 and Figure 10 The diagram shown is a schematic of the fine screening process.

[0127] S600: Based on the current pixel, determine a preset pixel region; the preset pixel region is a circular region with the current pixel as the center and a preset length as the radius.

[0128] S700: Based on the preset pixel area, perform fine screening on the second target high-resolution pixel to determine the third target high-resolution pixel.

[0129] The step of performing fine screening on the second target Gaussian pixels based on the preset pixel region to determine the third target Gaussian pixels includes the following sub-steps: S710: Calculate the screen space distance between the current pixel center and the two-dimensional circular projection center corresponding to the second target Gaussian element.

[0130] S720: If the screen space distance is greater than the sum of the preset length and the effective influence radius of the second target Gaussian element, then skip the corresponding second Gaussian element.

[0131] S730: After the second target Gaussian element has been traversed, the third target Gaussian element is determined.

[0132] Specifically, since the axis-aligned bounding box is only an approximation of the effective Gaussian influence range, it may still contain pixels outside the actual Gaussian influence range. To reduce this redundancy, this application treats each pixel as a circular region with a radius of 0.5 during pixel rendering and calculates the screen space distance d between the pixel center and the Gaussian projection center. If the effective Gaussian influence radius is denoted as R, when d > R + 0.5, it can be determined that the pixel circle does not intersect with the effective Gaussian influence range. At this point, the rendering stage no longer calculates the exponential decay, opacity contribution, and color accumulation corresponding to that Gaussian; only Gaussian pixels that pass this fine-tuning judgment will continue to enter subsequent blending calculations.

[0133] It's worth noting that this application changes the original processing order from "calculating the contribution first, then determining its validity" to "determining whether it might be valid first, then deciding whether to calculate the contribution." This early screening is particularly suitable for situations where there are many Gaussians within a tile. This is because while a Gaussian may cover a tile, it doesn't necessarily mean it makes a valid contribution to all pixels within that tile. Through a two-stage early screening mechanism combining "bounding box coarse screening with pixel circle fine screening," obviously irrelevant pixel-Gaussian combinations can be quickly eliminated first, and then redundant coverage caused by bounding boxes can be further eliminated, thus avoiding many pixels performing repeated exponential calculations and mixing judgments on invalid Gaussians.

[0134] For Gaussians that pass the early screening, the rendering stage continues to perform normal attenuation calculations, transparency calculations, and color accumulation. If the current opacity contribution of a Gaussian is below the threshold, its color accumulation can still be skipped; if the transmittance of the current pixel is already low enough, it means that the pixel is basically occluded by the Gaussians in front, and the subsequent Gaussian traversal of that pixel can be terminated early.

[0135] This early screening mechanism works in conjunction with the "carry-through" scheme. Since the curvature parameter is already included in the sorted compact load, pixel-Gaussian early screening can be performed immediately after the load is read during the rendering stage, without the need for additional lookup of radius, cone parameters, or scale information. In other words, the preceding algorithm simplification makes the curvature parameter possible, the "carry-through" method delivers the curvature parameter to the rendering stage, and the early screening during the rendering stage further utilizes the curvature parameter to reduce unnecessary computation.

[0136] S800: Read the compact load field corresponding to the third target Gaussian primitive, and unpack it to obtain the corresponding screen coordinates, target curvature parameters and compact visual field; after all image blocks have been traversed, complete pixel color accumulation and transmittance update to generate the rendered image.

[0137] This application proposes an optimization scheme for the 3D Gaussian splash forward rendering process, organizing the forward rendering workflow into three stages: preprocessing, cardinality sorting, and rendering. This scheme reduces the complexity of projection calculation and Gaussian attenuation calculation through "3D Gaussian isotropic simplification," "radial focal length consistency approximation," and "projection morphology isotropic approximation." It converges the originally complex projection morphology parameters into compact curvature parameters and further introduces curvature parameter compensation based on major axis matching, making the simplified circular projection closer to the original 2D elliptical projection in terms of effective influence range. This reduces the amount of data transferred across stages while minimizing the accuracy loss caused by projection approximation. Furthermore, by clustering and quantizing color and opacity for compression, the data size of visual attributes is further reduced.

[0138] Building upon this, this application proposes a "portable transfer" mechanism: the preprocessing stage generates a compact load required for the rendering stage; the sorting stage sorts this load along with the corresponding records of tiles and Gaussians; and the rendering stage directly reads the sorted compact load sequentially and completes pixel blending, instead of primarily relying on Gaussian numbers to look up the original attributes. Thus, this application can reduce redundant calculations, lower random memory access and intermediate data transfer overhead, and improve the efficiency of 3D Gaussian forward rendering while maintaining relatively stable rendering quality.

[0139] Meanwhile, during the rendering stage, this application introduces a curvature-based pixel-Gaussian early screening mechanism. Utilizing the screen space scale parameters carried in the compact load, the effective influence range of the Gaussian on the pixel plane is quickly recovered or determined. For Gaussians that do not intersect with the current pixel or are outside their contribution range, the rendering stage can skip them directly before calculating exponential decay, thereby reducing invalid exponential calculations and transparency accumulation calculations, further reducing the computational burden of the pixel blending stage.

[0140] This application provides a three-dimensional Gaussian splash forward rendering method, which has the following beneficial effects: 1. Algorithm Simplification and Projection Shape Approximation Method: This application combines the simplification of a three-dimensional Gaussian isotropic ellipsoid, the radial focal length consistency approximation, and the isotropic approximation of the projection shape. The key is that it not only approximates the three-dimensional Gaussian from an anisotropic ellipsoid to an isotropic Gaussian, but also further approximates its two-dimensional elliptical shape after projection onto the image plane as a circle. Through this method, the two-dimensional covariance and conic parameters that originally needed to be calculated and transmitted are simplified to a small amount of data such as screen coordinates and curvature parameters, thereby reducing the computational load in the preprocessing stage and reducing the intermediate variables that need to be transmitted in subsequent stages.

[0141] 2. Compact Geometric Representation Based on Curvature Parameter: This application recursively transforms the Gaussian screen space scale into a curvature parameter, and uses this parameter to calculate the normalized distance between pixels and Gaussians during the rendering stage. This design eliminates the need to read the complete cone parameters during the rendering stage, and also eliminates the need to re-derive the relationship between the Gaussian radius, focal length, and depth.

[0142] 3. Curvature Parameter Compensation Method Based on Major Axis Matching: This application approximates a two-dimensional elliptical projection as a circular projection and then compensates for the curvature parameter using the major axis matching concept. By appropriately reducing the curvature parameter through compensation coefficients, the effective influence range of the compensated circular Gaussian is made closer to the major axis expansion of the original two-dimensional elliptical projection. In the specific implementation, the compensation coefficient corresponding to each Gaussian is calculated based on its normalized projection position at the current viewpoint. This compensates for the accuracy loss caused by the simplification of the projection shape without restoring the complete two-dimensional covariance and conic parameters.

[0143] 4. Compact Visual Field Compression Method for Color and Opacity: This application employs compact encoding of Gaussian color and opacity, merging color clustering labels and opacity quantization labels into a compact visual field. During the rendering stage, color is recovered by looking up a table using the color label, and approximate opacity is recovered using the opacity label. This method reduces the data width of color and opacity transmitted across stages, enabling visual attributes to form a compact load together with geometric intermediates.

[0144] 5. "Carry-through" data flow of compact payload: In the preprocessing stage, this application packages the screen coordinates, curvature parameters, and compact visual fields required for the rendering stage into a compact payload; in the sorting stage, this compact payload is sorted along with the records as a sorting payload of the tile-Gaussian records; in the rendering stage, the sorted compact payload is directly read sequentially and color accumulation and transmittance updates are completed. This method differs from the method of "sorting Gaussian numbers and then looking up attributes in the rendering stage," and can reduce random memory accesses and repeated readings caused by looking up intermediate variables and visual attributes based on Gaussian numbers after sorting.

[0145] 6. Curvature-based Pixel-Gaussian Early Screening Mechanism: This application utilizes curvature parameters to quickly estimate the effective influence range of Gaussians in screen space. During the rendering stage, all Gaussians are traversed first, and the pixel-Gaussian coverage relationship is coarsely screened using axis-aligned bounding boxes, excluding some Gaussians that do not intersect with the current pixel to reduce subsequent shared memory access pressure. Before calculating exponential decay, the pixel is treated as a circle with a radius of 0.5, and fine screening is performed based on the distance between the pixel center and the Gaussian projection center. If the pixel circle does not intersect with the effective influence range of the Gaussian, the Gaussian is skipped before calculating exponential decay, opacity contribution, and color accumulation. This mechanism can reduce invalid calculations generated when pixels within a tile traverse a large number of Gaussians, and is particularly beneficial for reducing the computational overhead of the exponential function.

[0146] The above detailed embodiments further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A three-dimensional Gaussian splash forward rendering method, characterized in that, include: Obtain the 3D Gaussian model to be rendered; Through projection shape transformation processing, the screen space projection shape of the three-dimensional Gaussian model is simplified from two-dimensional elliptical projection to two-dimensional circular projection, and a compact load field containing screen coordinates, target curvature parameters and compact visual fields is generated. The projection shape transformation process includes: three-dimensional Gaussian isotropic simplification, radial focal length consistency approximation, and projection shape isotropic approximation; the screen space projection shape after the three-dimensional Gaussian isotropic simplification and radial focal length consistency approximation is a two-dimensional elliptical projection, until after the isotropic approximation, the screen space projection shape is simplified from a two-dimensional elliptical projection to a two-dimensional circular projection; the compact visual field includes: color labels and opacity labels. Gaussian records are generated according to the image patch range covered by each Gaussian primitive in the three-dimensional Gaussian model; the Gaussian records are sorted according to the image patch number and depth value order covered by each Gaussian primitive; the sorting stage is completed together with the compact load field; During the rendering phase, for each pixel within the current image block, the first target Gaussian pixel in the current image block is read according to the Gaussian record; Based on the current pixel, an axis-aligned bounding box is constructed according to the effective influence radius of the first target Gaussian element. The first target Gaussian element is then coarsely screened to determine the second target Gaussian element. Based on the current pixel, a preset pixel region is determined; the preset pixel region is a circular region with the current pixel as the center and a preset length as the radius. Based on the preset pixel region, the second target high-resolution pixel is finely screened to determine the third target high-resolution pixel; Read the compact load field corresponding to the third target Gaussian primitive, and obtain the corresponding screen coordinates, target curvature parameters, and compact visual field; after all image blocks have been traversed, complete pixel color accumulation and transmittance update to generate the rendered image.

2. The three-dimensional Gaussian splashing forward rendering method according to claim 1, characterized in that, In the three-dimensional Gaussian model, the Gaussian elements are three-dimensional ellipsoids with different directional scales and rotational attitudes; the spatial shape of the Gaussian elements is determined by the three-dimensional covariance. The step of determining screen coordinates and simplifying the screen space projection form of the 3D Gaussian model from a 2D elliptical projection to a 2D circular projection through projection shape transformation includes: The three-dimensional Gaussian model is simplified isotropically, and the three-dimensional covariance is approximated from a direction-dependent ellipsoidal expression to a uniform-scale spherical expression; the three-dimensional covariance is: ； In the formula, The radius of the Gaussian element is represented as approximately the size of a sphere. It is the identity matrix; By using the radial focal length consistency approximation, the lateral focal length of the camera is made equal to the longitudinal focal length. Based on the horizontal and vertical focal lengths, calculate the normalized projection factor; Based on the normalized projection factor, the screen coordinates are determined and the isotropic approximation of the projection shape is used to degenerate the two-dimensional covariance of the screen space into a diagonal matrix, thus simplifying the screen space projection shape of the three-dimensional Gaussian model from a two-dimensional elliptical projection to a two-dimensional circular projection.

3. The three-dimensional Gaussian splashing forward rendering method according to claim 1, characterized in that, The steps for generating the target curvature parameters include: Based on the screen space projection under the retained off-axis perspective stretching term, determine the major axis matching compensation coefficients corresponding to each Gaussian element. Obtain the original curvature parameter; the original curvature parameter is the reciprocal of the screen space scale radius; Based on the major axis matching compensation coefficient, the original curvature parameters are corrected to generate target curvature parameters; the target curvature parameters are: ； In the formula, For the first i The long axis matching compensation coefficient of each Gaussian element; These are the original curvature parameters.

4. The three-dimensional Gaussian splashing forward rendering method according to claim 3, characterized in that, The step of determining the major axis matching compensation coefficients corresponding to each Gaussian element based on the screen space projection under the retained off-axis perspective stretching term includes: Based on the major and minor semi-axis of the two-dimensional elliptical projection, a major axis matching compensation coefficient is calculated to match the major semi-axis; the major axis matching compensation coefficient makes the radius of the two-dimensional circular projection match the major semi-axis of the two-dimensional elliptical projection. Based on the major axis matching compensation coefficient, and according to the normalized projection position of each Gaussian element in the current viewpoint, the major axis matching compensation coefficient corresponding to each Gaussian element is determined: The major axis matching compensation coefficient is: ； In the formula, and It is represented as the normalized coordinates of the Gaussian metacenter from the current perspective.

5. The three-dimensional Gaussian splashing forward rendering method according to claim 1, characterized in that, The steps for generating the compact visual field include: K-Means clustering is performed on the color components of all Gaussian elements in the three-dimensional Gaussian model to group similar colors into the same category and assign color labels; each Gaussian element stores its corresponding color label. The opacity is divided into a finite number of opacity levels in the range of 0 to 1, and the opacity levels are stored using opacity labels; The color label and opacity label are merged into a compact visual field.

6. The three-dimensional Gaussian splashing forward rendering method according to claim 5, characterized in that, The compact visual field uses a 16-bit fixed-length encoding; the high 12 bits of the fixed-length encoding are used to store color labels, and the low 4 bits are used to store opacity labels; the color labels are configured with 4096 color categories, and the opacity labels are configured with 16 levels.

7. The three-dimensional Gaussian splashing forward rendering method according to claim 1, characterized in that, The step of reading the first target Gaussian cell in the current image block according to the Gaussian record includes: The compact load field corresponding to the first target Gaussian element in the current image block is read sequentially according to the sorting in the Gaussian record.

8. A three-dimensional Gaussian splashing forward rendering method according to claim 7, characterized in that, The method further includes: The sorted Gaussian records are cached in a cache.

9. A three-dimensional Gaussian splashing forward rendering method according to claim 1, characterized in that, The step of constructing an axis-aligned bounding box based on the effective influence radius of the first target Gaussian pixel based on the current pixel, performing coarse screening on the first target Gaussian pixel, and determining the second target Gaussian pixel includes: The effective influence radius of the first target Gaussian element is determined based on the screen coordinates and target curvature parameters corresponding to the first target Gaussian element. Based on the effective radius of influence, construct an axis-aligned bounding box; If the current pixel is outside the axis-aligned bounding box, then skip the corresponding first target high-order pixel; After the first target Gaussian element has been traversed, the second target Gaussian element is determined.

10. A three-dimensional Gaussian splashing forward rendering method according to claim 9, characterized in that, The step of performing fine screening on the second target Gaussian pixels based on the preset pixel region to determine the third target Gaussian pixels includes: Calculate the screen space distance between the current pixel center and the two-dimensional circular projection center corresponding to the second target Gaussian element; If the screen space distance is greater than the sum of the preset length and the effective influence radius of the second target Gaussian element, then skip the corresponding second target Gaussian element; After the second target Gaussian element has been traversed, the third target Gaussian element is determined.

Images