A picture rendering method and device, computer equipment and storage medium

Abstract

The application provides a picture rendering method and device, computer equipment and a storage medium. Based on the spatial geometric parameters and appearance parameters of a Gaussian point, the equivalent maximum radius of each Gaussian point in a screen space is determined, and accurate bounding box information is generated accordingly. Compared with the approximate method of a traditional axis-aligned bounding box, the application can more accurately represent the actual coverage range of the Gaussian point on the screen, improve the accuracy and picture detail fidelity of the rasterization process, reduce the number of irrelevant tiles divided into the bounding box, and effectively improve the rasterization efficiency.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a screen rendering method, apparatus, computer device, and storage medium. Background Technology

[0002] With the development of applications such as mobile robots, virtual reality, and extended reality, achieving efficient and realistic 3D rendering on edge devices has become a crucial requirement. 3D Gaussian Splatting (3DGS) is an explicit scene representation technique that models a 3D scene as a large number of Gaussian points. Each Gaussian point contains spatial geometric parameters and appearance parameters, such as position, covariance, color coefficients, and opacity information. Gaussian points can compactly represent the geometric and optical properties of complex scenes, facilitating high-quality real-time rendering.

[0003] Existing 3DGS rendering methods typically employ a tile-based rendering (TBR) pipeline, mainly including projection and bounding box calculation, tile allocation and index generation, and rasterization. However, traditional methods often use axis-aligned bounding boxes to approximate the projected shape, which cannot accurately reflect the actual coverage of the rotated elliptical Gaussian point in screen space. This results in a large number of irrelevant tiles being included in the bounding box, thus affecting rasterization efficiency. Summary of the Invention

[0004] In view of this, this application provides a screen rendering method, apparatus, computer equipment, and storage medium to solve the problem of low rasterization efficiency in the prior art.

[0005] Specifically, this application is implemented through the following technical solution: Firstly, this application provides a method for rendering a screen, including: Obtain Gaussian point data of the 3D scene to be rendered; the Gaussian point data includes the spatial geometric parameters and appearance parameters of multiple Gaussian points; For any Gaussian point, based on the spatial geometric parameters and the appearance parameters, determine the equivalent maximum radius of the Gaussian point projected onto the screen space; Based on the spatial geometric parameters, the appearance parameters, and the equivalent maximum radius, the bounding box information of the Gaussian point in the screen space is determined; Based on the bounding box information of each Gaussian point, multiple rasterization index information are determined; the rasterization index information is used to indicate the Gaussian point corresponding to the tile falling on the bounding box in the screen space at at least one depth; Based on the rasterization index information and the Gaussian point data, the 3D scene to be rendered is rasterized. Based on the rasterization processing results, the 3D scene to be rendered is rendered.

[0006] Optionally, the spatial geometric parameters include two-dimensional covariance information; the appearance parameters include opacity. Determining the equivalent maximum radius of the Gaussian point projection in screen space based on the spatial geometric parameters and the appearance parameters includes: The maximum radius of the Gaussian point is determined based on the maximum eigenvalue in the two-dimensional covariance information of the Gaussian point. The first adaptive radius of the Gaussian point is determined based on the logarithm of the ratio between the opacity of the Gaussian point and the preset opacity threshold, and the maximum eigenvalue of the two-dimensional covariance information. The minimum value between the maximum radius and the first adaptive radius is determined to be the equivalent maximum radius.

[0007] Optionally, determining the bounding box information of the Gaussian point in screen space based on the spatial geometric parameters, the appearance parameters, and the equivalent maximum radius includes: Based on the logarithm of the ratio between the opacity of the Gaussian point and the preset opacity threshold, and the maximum value of the two-dimensional covariance information in the first coordinate axis direction, a second adaptive radius in the first coordinate axis direction is determined. Based on the logarithm of the ratio between the opacity of the Gaussian point and the preset opacity threshold, and the maximum value of the two-dimensional covariance information in the direction of the second coordinate axis, the third adaptive radius in the direction of the second coordinate axis is determined. The minimum value between the second adaptive radius and the equivalent maximum radius is determined to be the first adaptive coverage radius in the first coordinate axis direction of the bounding box information; The minimum value between the third adaptive radius and the equivalent maximum radius is determined to be the second adaptive coverage radius in the second coordinate axis direction of the bounding box information.

[0008] Optionally, the step of rasterizing the 3D scene to be rendered based on the rasterization index information and the Gaussian point data includes: Based on the tile number and depth information indicated by the rasterized index information, the rasterized index information is sorted; Based on the sorting order, the Gaussian point data corresponding to the tiles is read, and the rendering information of the tiles is calculated.

[0009] Optionally, reading the Gaussian point data corresponding to the map tile includes: For any given tile, read the Gaussian point data corresponding to that tile from the internal cache; In the event that a read from the internal cache fails, the Gaussian point data corresponding to the tile is read from main memory.

[0010] Optionally, the method further includes: The Gaussian point data read from the main memory is written into the internal cache; If there is no free storage space in the internal cache, the Gaussian point data that meets the preset discard conditions in the internal cache will be discarded, and the Gaussian point data read from the main memory will be rewritten into the internal cache.

[0011] Optionally, the rendering information of the computational tile includes: The first type of data in the rendering information is calculated using a unit with a first computational precision; The second type of data in the rendering information is calculated using a unit with a second calculation precision; The first calculation precision is higher than the second calculation precision; the first type includes relative coordinates and conic section parameters; the second type includes opacity composition and RGB color accumulation.

[0012] Optionally, when calculating the rendering information of a tile, the index calculation result is determined by looking up a table.

[0013] Secondly, this application provides a screen rendering apparatus, comprising: The acquisition module is used to acquire Gaussian point data of the 3D scene to be rendered; the Gaussian point data includes the spatial geometric parameters and appearance parameters of multiple Gaussian points. The first determining module is used to determine, for any Gaussian point, the equivalent maximum radius of the Gaussian point projected onto the screen space based on the spatial geometric parameters and the appearance parameters. The second determining module is used to determine the bounding box information of the Gaussian point in the screen space based on the spatial geometric parameters, the appearance parameters, and the equivalent maximum radius. The third determining module is used to determine multiple rasterization index information based on the bounding box information of each Gaussian point; the rasterization index information is used to indicate the Gaussian point corresponding to the tile falling on the bounding box in the screen space at at least one depth; A rasterization module is used to perform rasterization processing on the 3D scene to be rendered based on the rasterization index information and the Gaussian point data. The rendering module is used to render the 3D scene to be rendered based on the rasterization processing results.

[0014] Optionally, the spatial geometric parameters include two-dimensional covariance information; the appearance parameters include opacity. The first determining module is used for: The maximum radius of the Gaussian point is determined based on the maximum eigenvalue in the two-dimensional covariance information of the Gaussian point. The first adaptive radius of the Gaussian point is determined based on the logarithm of the ratio between the opacity of the Gaussian point and the preset opacity threshold, and the maximum eigenvalue of the two-dimensional covariance information. The minimum value between the maximum radius and the first adaptive radius is determined to be the equivalent maximum radius.

[0015] Optionally, the second determining module is used to: Based on the logarithm of the ratio between the opacity of the Gaussian point and the preset opacity threshold, and the maximum value of the two-dimensional covariance information in the first coordinate axis direction, a second adaptive radius in the first coordinate axis direction is determined. Based on the logarithm of the ratio between the opacity of the Gaussian point and the preset opacity threshold, and the maximum value of the two-dimensional covariance information in the direction of the second coordinate axis, the third adaptive radius in the direction of the second coordinate axis is determined. The minimum value between the second adaptive radius and the equivalent maximum radius is determined to be the first adaptive coverage radius in the first coordinate axis direction of the bounding box information; The minimum value between the third adaptive radius and the equivalent maximum radius is determined to be the second adaptive coverage radius in the second coordinate axis direction of the bounding box information.

[0016] Optionally, the rasterization module is used for: Based on the tile number and depth information indicated by the rasterized index information, the rasterized index information is sorted; Based on the sorting order, the Gaussian point data corresponding to the tiles is read, and the rendering information of the tiles is calculated.

[0017] Optionally, the rasterization module is used for: For any given tile, read the Gaussian point data corresponding to that tile from the internal cache; In the event that a read from the internal cache fails, the Gaussian point data corresponding to the tile is read from main memory.

[0018] Optionally, the device further includes a cache management module for: The Gaussian point data read from the main memory is written into the internal cache; If there is no free storage space in the internal cache, the Gaussian point data that meets the preset discard conditions in the internal cache will be discarded, and the Gaussian point data read from the main memory will be rewritten into the internal cache.

[0019] Optionally, the rasterization module is used for: The first type of data in the rendering information is calculated using a unit with a first computational precision; The second type of data in the rendering information is calculated using a unit with a second calculation precision; The first calculation precision is higher than the second calculation precision; the first type includes relative coordinates and conic section parameters; the second type includes opacity composition and RGB color accumulation.

[0020] Optionally, when calculating the rendering information of a tile, the index calculation result is determined by looking up a table.

[0021] Thirdly, embodiments of this application also provide a computer device, which includes a processor and a memory. The memory stores machine-readable instructions executable by the processor. The processor is used to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, they perform the steps of the first aspect above, or any possible implementation of the first aspect.

[0022] Fourthly, optional embodiments of this application also provide a computer-readable storage medium storing a computer program that, when run, performs the steps of the first aspect or any possible implementation of the first aspect.

[0023] The rendering method, apparatus, computer device, and storage medium provided in this application determine the equivalent maximum radius of each Gaussian point in screen space based on the spatial geometric and appearance parameters of the Gaussian point, and generate accurate bounding box information accordingly. Compared with the traditional approximation method of axis-aligned bounding boxes, this application can more accurately represent the actual coverage area of ​​the Gaussian point on the screen, improve the accuracy of the rasterization process and the fidelity of image details, reduce the number of irrelevant tiles being assigned to bounding boxes, and effectively improve rasterization efficiency. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating a screen rendering method according to an exemplary embodiment of this application; Figure 2 This is a flowchart illustrating the determination of bounding box information as shown in an exemplary embodiment of this application; Figure 3 This is a schematic diagram of a hardware accelerator shown in an exemplary embodiment of this application; Figure 4 This is a schematic diagram of the hardware pipeline structure of the preprocessing engine shown in an exemplary embodiment of this application; Figure 5a This is a schematic diagram of the write path circuit of a four-way cache module shown in an exemplary embodiment of this application; Figure 5b This is a schematic diagram of the read path circuit of a four-way cache module shown in an exemplary embodiment of this application; Figure 6 This is a schematic diagram of hierarchical prefetching key-value storage shown in an exemplary embodiment of this application; Figure 7 This is a schematic diagram of the data path and processing flow of a mixed-precision rendering engine shown in an exemplary embodiment of this application; Figure 8 This is a schematic diagram of a screen rendering apparatus shown in an exemplary embodiment of this application; Figure 9 This is a schematic diagram of a computer device illustrated in an exemplary embodiment of this application. Detailed Implementation

[0025] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0026] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0027] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0028] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0029] Research has revealed that existing 3DGS rendering methods typically employ a tile-based rendering (TBR) pipeline, primarily involving projection and bounding box calculation, tile allocation and index generation, and rasterization. However, traditional methods often approximate the projected shape using axis-aligned bounding boxes, which fails to accurately reflect the actual coverage of the rotated elliptical Gaussian point in screen space. This results in a large number of irrelevant tiles being included in the bounding box, thus impacting rasterization efficiency.

[0030] In view of this, embodiments of this application provide a screen rendering method, apparatus, computer device, and storage medium. Based on the spatial geometric parameters and appearance parameters of Gaussian points, the equivalent maximum radius of each Gaussian point in screen space is determined, and accurate bounding box information is generated accordingly. Compared with the traditional approximation method of axis-aligned bounding boxes, this application can more accurately represent the actual coverage area of ​​Gaussian points on the screen, improve the accuracy of the rasterization process and the fidelity of image details, reduce the number of irrelevant tiles being assigned to bounding boxes, and effectively improve rasterization efficiency.

[0031] To facilitate understanding of this embodiment, the application scenario of the screen rendering method disclosed in this application embodiment will first be introduced. The execution subject of the screen rendering method provided in this application embodiment can be a computer device. In some possible implementations, the screen rendering method can be implemented by the processor calling computer-readable instructions stored in memory.

[0032] See Figure 1 The diagram shown is a flowchart illustrating a screen rendering method according to an exemplary embodiment of this application. The method includes steps S101 to S106, wherein: S101. Obtain Gaussian point data of the 3D scene to be rendered; the Gaussian point data includes the spatial geometric parameters and appearance parameters of multiple Gaussian points.

[0033] The Gaussian point data of the 3D scene to be rendered can include spatial geometric parameters and appearance parameters of multiple Gaussian points. The spatial geometric parameters represent the position and shape characteristics of the Gaussian points in 3D space, while the appearance parameters can include color information, opacity, and other data used to describe the appearance characteristics of the Gaussian points. The aforementioned Gaussian point data can refer to the original Gaussian point parameters. Subsequent operations such as projection transformation and conversion from spherical harmonics to RGB on the Gaussian point data can obtain the correspondence between the Gaussian points and tiles in screen space.

[0034] S102. For any Gaussian point, based on the spatial geometric parameters and the appearance parameters, determine the equivalent maximum radius of the Gaussian point projected onto the screen space.

[0035] The equivalent maximum radius is used to determine the influence range of the Gaussian point in the screen space, thus providing a basis for subsequent rasterization assignment and bounding box calculation.

[0036] In some possible implementations, the spatial geometric parameters include two-dimensional covariance information; the appearance parameters include opacity; determining the equivalent maximum radius of the Gaussian point projection in screen space based on the spatial geometric parameters and the appearance parameters includes: The maximum radius of the Gaussian point is determined based on the maximum eigenvalue in the two-dimensional covariance information of the Gaussian point. The first adaptive radius of the Gaussian point is determined based on the logarithm of the ratio between the opacity of the Gaussian point and the preset opacity threshold, and the maximum eigenvalue of the two-dimensional covariance information. The minimum value between the maximum radius and the first adaptive radius is determined to be the equivalent maximum radius.

[0037] Among them, the two-dimensional covariance information is used to describe the projection shape and principal axis scale of each Gaussian point in the screen space, and the opacity is used to reflect the visual influence intensity of the Gaussian point.

[0038] Specifically, the maximum geometric radius of the Gaussian point can be determined based on the largest eigenvalue of its two-dimensional covariance matrix. This radius reflects the maximum extent of the Gaussian point's projection onto the screen space, ensuring that the bounding box covers the geometrically influential area of ​​the Gaussian point. Further, a first adaptive radius is calculated using a logarithmic function based on the ratio between the Gaussian point's opacity and a preset opacity threshold. Then, the maximum radius is compared with the first adaptive radius, and the minimum of the two is taken as the final equivalent maximum radius.

[0039] Wherein, the first adaptive radius It can be represented as: ; in, It is the largest eigenvalue of the two-dimensional covariance matrix of the Gaussian point (characterizing its principal axis scale). The opacity of the Gaussian point. This represents the opacity threshold.

[0040] Equivalent maximum radius for: ; in, The maximum radius.

[0041] The equivalent maximum radius defines the effective circular influence region of a Gaussian point in screen space. If this circular region has no intersection with the current view frustum or its radius is extremely small, the Gaussian point is directly discarded.

[0042] The equivalent maximum radius calculated by this method not only ensures the geometric coverage of Gaussian points, but also adaptively considers the impact of their opacity on visual contribution, avoiding redundant calculations for transparent or tiny Gaussian points, thereby improving rasterization and rendering efficiency.

[0043] S103. Based on the spatial geometric parameters, the appearance parameters, and the equivalent maximum radius, determine the bounding box information of the Gaussian point in the screen space.

[0044] In this step, for the determined equivalent maximum radius of the Gaussian point, its bounding box information in screen space is further calculated to guide tile allocation and rasterization processing. This bounding box has adaptive coverage radii in the X and Y axes of the screen coordinate system to ensure that the effective area of ​​the Gaussian point is covered while avoiding redundant calculations.

[0045] The bounding box describes the pixel area that the Gaussian point may affect on the screen and provides the basis for the generation of rasterization indexes. The calculation of the bounding box takes into account the projection range in the X and Y directions of the screen coordinate system. By adaptively calculating the coverage radius in each direction and limiting it to the equivalent maximum radius, a compact axis-aligned bounding box is generated.

[0046] Specifically, a second adaptive radius in the first coordinate axis direction can be determined based on the logarithm of the ratio between the opacity of the Gaussian point and a preset opacity threshold, and the maximum value of the two-dimensional covariance information in the first coordinate axis direction; and a third adaptive radius in the second coordinate axis direction can be determined based on the logarithm of the ratio between the opacity of the Gaussian point and a preset opacity threshold, and the maximum value of the two-dimensional covariance information in the second coordinate axis direction; finally, the minimum value between the second adaptive radius and the equivalent maximum radius can be determined as the first adaptive coverage radius in the first coordinate axis direction of the bounding box information; and the minimum value between the third adaptive radius and the equivalent maximum radius can be determined as the second adaptive coverage radius in the second coordinate axis direction of the bounding box information.

[0047] The first and second coordinate axes can refer to the coordinate axes in the screen coordinate system, such as the X-axis and Y-axis.

[0048] Second adaptive radius and the third adaptive radius for: ; in, and These represent the maximum values ​​of the two-dimensional covariance information (i.e., the two-dimensional covariance matrix) in the first coordinate axis direction and the second coordinate axis direction, respectively (i.e., the extreme values ​​of the Gaussian projection of the covariance matrix in screen space along the coordinate axis direction, which are the one-dimensional variance scale or equivalent principal scale of the corresponding axis).

[0049] Thus, by combining the first adaptive coverage radius, the second adaptive coverage radius, and the projection center coordinates of the Gaussian point in screen space, a rectangular bounding box can be obtained. This bounding box information is used in the subsequent rasterization stage to quickly determine the Gaussian point falling on each tile. By generating compact bounding boxes based on adaptive coverage radii along the coordinate axes, the allocation of invalid tiles is effectively reduced, improving rasterization and rendering efficiency while ensuring image quality and visual effects.

[0050] See Figure 2 The diagram shown is a flowchart illustrating the determination of bounding box information in an exemplary embodiment of this application. In this diagram, the maximum radius is first determined through the step of (1) forming the Original Bounding. , Let the coordinates of the Gaussian point be , and the bounding box representation is as follows: Then, through (2) First-stage Culling, the opacity threshold is set, for example, to Opacity=0.45, to determine... The bounding box is represented as Then, through (3) the second-stage culling, based on... Calculate the second adaptive radius ,based on Calculate the third adaptive radius The final bounding box information is obtained, and the half-length of the bounding box is... Half width is Here, Culling Range represents the clipping region, Bounding Box represents the bounding box, 3D Gaussian represents the Gaussian projection region, Hit Tile Range represents the hit tiles, and Miss Tile Range represents the missed tiles.

[0051] S104. Based on the bounding box information of each Gaussian point, determine multiple rasterization index information; the rasterization index information is used to indicate the Gaussian point corresponding to the tile falling on the bounding box in the screen space at at least one depth.

[0052] In this step, only rasterization index information for tiles with corresponding Gaussian points can be generated. For tiles not associated with Gaussian points, no rasterization index information is generated, so that the subsequent rasterization process skips these irrelevant tiles.

[0053] The rasterization index information mentioned above can provide an efficient data access path for rasterization processing, thereby reducing unnecessary computation.

[0054] S105. Based on the rasterization index information and the Gaussian point data, perform rasterization processing on the 3D scene to be rendered.

[0055] In this step, to ensure the order and efficiency of rasterization calculation, the rasterization index information can be sorted according to the tile number and depth information. Then, based on the sorting order, the Gaussian point data corresponding to the tile is read, and the rendering information of the tile is calculated.

[0056] In some embodiments, reading the Gaussian point data corresponding to the tile includes: For any given tile, read the Gaussian point data corresponding to the tile from the internal cache; if reading from the internal cache fails, read the Gaussian point data corresponding to the tile from main memory.

[0057] In this step, the Gaussian point data of the required tile can be read first from the internal on-chip cache. The internal cache adopts a set-associative cache structure, with each cache line storing several Gaussian point attributes (such as coordinates, color, covariance, opacity, etc.). Tags and indexes are generated using the Gaussian point IDs (GSIDs) in the rasterization index information, and the internal cache performs parallel matching to quickly locate the target Gaussian point data. If a cache hit occurs, the data is directly sent to the rasterization engine's processing unit without accessing main memory, thus significantly reducing access latency.

[0058] The internal cache can consist of multiple static random-access memories (SRAMs).

[0059] When an internal cache miss occurs, the rasterization engine reads the Gaussian point data corresponding to the tile from external main memory (DRAM). The main memory can be Dynamic Random Access Memory (DRAM).

[0060] Main memory access can employ batch burst reads to fully utilize memory bandwidth and reduce pipeline stalls caused by random accesses. Read data can be temporarily stored in a DRAM return buffer (FIFO) and then written to the internal cache for use by the rasterization engine.

[0061] Through the aforementioned data reading mechanism, this application achieves efficient and low-latency Gaussian point data access, significantly reduces random access overhead, maintains pipeline continuity, and supports high frame rate rendering.

[0062] In one embodiment of this application, to improve the access efficiency of Gaussian point data during the rasterization stage, a dynamic write and replace mechanism is implemented in the on-chip cache. This mechanism is used to process data read from main memory (DRAM) and to effectively manage cache space when it is limited.

[0063] When the internal cache misses and the rasterization engine retrieves Gaussian point data from the main memory, the Gaussian point data read from the main memory can be written into the internal cache. If there is no free storage space in the internal cache, the Gaussian point data in the internal cache that meets the preset discard conditions will be discarded, and the Gaussian point data read from the main memory will be rewritten into the internal cache.

[0064] The preset discard conditions can be set based on one or more of the following information: access time, access frequency, tile priority, and opacity / contribution value.

[0065] S106. Based on the rasterization processing results, render the three-dimensional scene to be rendered.

[0066] In one embodiment of this application, to improve rendering performance and reduce hardware power consumption, mixed-precision computation is used to process the rendering information of tiles during the rasterization stage. This step can distribute the geometric and color attributes of Gaussian points to computation units of different precisions according to their error sensitivity.

[0067] Specifically, a unit with a first calculation precision can be used to calculate the first type of data in the rendering information; and a unit with a second calculation precision can be used to calculate the second type of data in the rendering information.

[0068] The first calculation precision is higher than the second calculation precision; the first type includes relative coordinates and conic section parameters; the second type includes opacity composition and RGB color accumulation.

[0069] For example, a high-precision computing unit can be used to perform calculations on error-sensitive data, using a higher bit width (e.g., 16-bit fixed-point number FXP16) to ensure calculation accuracy, while a low-precision computing unit is used to calculate data with high error tolerance, using a lower bit width (e.g., 8-bit fixed-point number FXP8) to reduce hardware resource overhead and power consumption.

[0070] In some implementations, to further reduce power consumption and logic complexity, this application employs a lookup table (LUT) method for Gaussian point exponent calculation when calculating tile rendering information. For example, for the exponent term S of the Gaussian response in rasterization, if S is less than the lower limit, 0 is directly output; if S is greater than 0, 1 is directly output. For valid S values ​​within the interval, linear interpolation is performed using the lookup table to obtain approximate exponent values.

[0071] Specifically, the LUT can store pre-calculated exponent values; the hardware logic quickly indexes the LUT based on the input exponent and performs necessary linear interpolation. This avoids the need for a high-power floating-point exponentiation unit, improving hardware efficiency.

[0072] The rendering method provided in this application determines the equivalent maximum radius of each Gaussian point in screen space based on its spatial geometric and appearance parameters, and generates accurate bounding box information accordingly. Compared with the traditional approximation method of axis-aligned bounding boxes, this application can more accurately represent the actual coverage area of ​​Gaussian points on the screen, improve the accuracy of the rasterization process and the fidelity of image details, reduce the number of irrelevant tiles being assigned to bounding boxes, and effectively improve rasterization efficiency.

[0073] This application also provides a hardware accelerator for implementing the above-described image rendering method. See also Figure 3The diagram shown is a schematic of a hardware accelerator according to an embodiment of this application. The hardware accelerator includes a preprocessing engine, a 4-way cache module, a radix sorting unit, a top controller, and a mixed-precision rendering engine. Each module communicates with the external DDR4 synchronous dynamic random access memory (SDRAM) via an interconnect network. The preprocessing engine includes a Points Projection module, a 3D Cov to 2D Cov module, a Spherical Harmonics (SH) to RGB encoder (SH2RGB SH Encoder), an Output First-In-First-Out (OUTPUT FIFO) queue, and an Adaptive Tile-aligned Extent (ATAE) Core (which includes an Adaptive AABB Bounding Unit, a Scheduler, and a Kernel Value Generation Unit (KVGU)). The four-way cache module includes multiple SRAMs, an SRAM Interface, a Valid Array, a Least Recently Used (LRU) Array, a Write Logic (WR Logic), and a Query Logic. The mixed-precision rendering engine includes an early termination controller, a 16×16 mixed-precision rendering unit (MPRU) array, a geometry shader (GS) bipolar random access memory (GS Parameter BRAM), a coordinate offset module, a DDR data FIFO queue, and output memory.The radix sorting unit includes a 4-bit-level distributor, four storage bucket element counters, and sorting SRAM. The top-level controller includes a data restructuring unit, an instruction decoder, a DDR memory controller, and configuration, status, and constants modules.

[0074] Figure 4 This diagram illustrates the hardware pipeline structure of the preprocessing engine provided in this application. In the diagram, COV represents the covariance in the Gaussian point data, 3D Covariance Solver represents the 3D covariance solver, Σ represents the covariance matrix, J represents the coefficient matrix, Original Radius Comp represents the original radius calculation unit, Fp16 Compute represents the half-precision floating-point (Fp16) calculation unit, Det represents the determinant, Div represents division, Sqrt represents the square root, Add represents addition, Sub represents subtraction, and Mul represents multiplication. The values ​​represent the equivalent maximum radius, Original Radius, Adaptive RadiusComp, Opacity, Fp16 Exp-8, Early Culling, Fully Pipelined, Log, Fp16 Exp-1, and Cmp. This indicates taking the minimum value. , This indicates taking the minimum value. .

[0075] Figure 5a A schematic diagram of the write path circuit of the four-way cache module provided in this application. Figure 5b A schematic diagram of the read path circuit for the four-way cache module provided in this application. This indicates the write ID, offset indicates the offset, index indicates the index, query indicates the query, valid check indicates the validity check, valid way indicates the valid way, way indicates the cache way, P-LRU Reg Array indicates the least recently used register array, LRU indicates the least recently used, and 4-Way DP SRAM Port B(RD) indicates a 4-way dual-port static random access memory (DP SRAM) port B (read). This represents the input data, Update represents the update, Valid Reg Array represents the valid bit register array, QueryGaussian ID represents the Gaussian query ID, Tag matching represents the tag matching, Tag Way represents the tag way, Way[offset] represents the cache way corresponding to the offset, Miss represents the cache miss, and P-LRU Reg Array represents the least recently used register array.

[0076] Figure 6 This diagram illustrates the hierarchical pre-fetch key-value storage provided in this application. Hierarchy Pre-fetch based KV storage represents hierarchical pre-fetch key-value (KV) storage; Sorted Render Queue (4096 GSIDDepth) represents the sorted render queue (4096 Gaussian fragment IDs (GSIDs) at a depth of 4096); 4-Way Gaussian Rendering Cache represents a 4-way Gaussian render cache; GDPS Scheduler represents the Gaussian data pre-fetch scheduler; OffChip DDR represents off-chip DDR memory; DDR Data FIFO represents the DDR data first-in-first-out queue; On Chip Rasterization BRAM represents on-chip rasterization block bipolar random access memory; Pre-fetch Hit represents a pre-fetch hit; uv (coordinates)... (Opacity), RGB, and conic (conic curve) are Gaussian point data obtained from main memory.

[0077] Figure 7 This diagram illustrates the data path and processing flow of the mixed-precision rendering engine provided in this application. In it, Relative coordinates<1s,13i,2d>covariance<1s,3i,12d>calculation represents the calculation of relative coordinates<1s,13i,2d> and covariance<1s,3i,12d> (the values ​​in parentheses represent precision, periodicity, and dimension), GS Pos represents a Gaussian fragment, Pixel represents a pixel, and offset represents the offset. and This represents the difference in direction between the x and y coordinate axes. , , This section represents the quadratic curve parameters along each coordinate axis. FXP16 Computing indicates 16-bit fixed-point computation. Power represents the power value. EXP represents exponential operation. Input 8-bit Power indicates an 8-bit input power value. LM-LUT represents a logarithmic-exponential lookup table. 128 Lines indicates 128 rows. Results indicates the results. Alpha represents opacity. 8b-opac represents 8-bit opacity. Clamp indicates clamping. Early Termin indicates early termination. The value represents the test threshold, blending represents blending, ACC represents the accumulator, Map Output Buffer represents the map output buffer, 16-bit-Precision path represents a 16-bit precision path, and 8-bit-Precision path represents an 8-bit precision path.

[0078] Corresponding to the aforementioned embodiments of the image rendering method, this application also provides embodiments of the image rendering apparatus.

[0079] Figure 8 This is a schematic diagram illustrating a screen rendering apparatus as an exemplary embodiment of this application. The apparatus includes: The acquisition module 810 is used to acquire Gaussian point data of the 3D scene to be rendered; the Gaussian point data includes spatial geometric parameters and appearance parameters of multiple Gaussian points. The first determining module 820 is used to determine, for any Gaussian point, the equivalent maximum radius of the Gaussian point projected onto the screen space based on the spatial geometric parameters and the appearance parameters. The second determining module 830 is used to determine the bounding box information of the Gaussian point in the screen space based on the spatial geometric parameters, the appearance parameters, and the equivalent maximum radius. The third determining module 840 is used to determine multiple rasterization index information based on the bounding box information of each Gaussian point; the rasterization index information is used to indicate the Gaussian point corresponding to the tile falling on the bounding box in the screen space at at least one depth; The rasterization module 850 is used to perform rasterization processing on the three-dimensional scene to be rendered based on the rasterization index information and the Gaussian point data. The rendering module 860 is used to render the three-dimensional scene to be rendered based on the rasterization processing results.

[0080] Optionally, the spatial geometric parameters include two-dimensional covariance information; the appearance parameters include opacity. The first determining module 820 is used for: The maximum radius of the Gaussian point is determined based on the maximum eigenvalue in the two-dimensional covariance information of the Gaussian point. The first adaptive radius of the Gaussian point is determined based on the logarithm of the ratio between the opacity of the Gaussian point and the preset opacity threshold, and the maximum eigenvalue of the two-dimensional covariance information. The minimum value between the maximum radius and the first adaptive radius is determined to be the equivalent maximum radius.

[0081] Optionally, the second determining module 830 is used for: Based on the logarithm of the ratio between the opacity of the Gaussian point and the preset opacity threshold, and the maximum value of the two-dimensional covariance information in the first coordinate axis direction, a second adaptive radius in the first coordinate axis direction is determined. Based on the logarithm of the ratio between the opacity of the Gaussian point and the preset opacity threshold, and the maximum value of the two-dimensional covariance information in the direction of the second coordinate axis, the third adaptive radius in the direction of the second coordinate axis is determined. The minimum value between the second adaptive radius and the equivalent maximum radius is determined to be the first adaptive coverage radius in the first coordinate axis direction of the bounding box information; The minimum value between the third adaptive radius and the equivalent maximum radius is determined to be the second adaptive coverage radius in the second coordinate axis direction of the bounding box information.

[0082] Optionally, the rasterization module 850 is used for: Based on the tile number and depth information indicated by the rasterized index information, the rasterized index information is sorted; Based on the sorting order, the Gaussian point data corresponding to the tiles is read, and the rendering information of the tiles is calculated.

[0083] Optionally, the rasterization module 850 is used for: For any given tile, read the Gaussian point data corresponding to that tile from the internal cache; In the event that a read from the internal cache fails, the Gaussian point data corresponding to the tile is read from main memory.

[0084] Optionally, the device further includes a cache management module for: The Gaussian point data read from the main memory is written into the internal cache; If there is no free storage space in the internal cache, the Gaussian point data that meets the preset discard conditions in the internal cache will be discarded, and the Gaussian point data read from the main memory will be rewritten into the internal cache.

[0085] Optionally, the rasterization module 850 is used for: The first type of data in the rendering information is calculated using a unit with a first computational precision; The second type of data in the rendering information is calculated using a unit with a second calculation precision; The first calculation precision is higher than the second calculation precision; the first type includes relative coordinates and conic section parameters; the second type includes opacity composition and RGB color accumulation.

[0086] Optionally, when calculating the rendering information of a tile, the index calculation result is determined by looking up a table.

[0087] The rendering apparatus provided in this application determines the equivalent maximum radius of each Gaussian point in screen space based on its spatial geometric and appearance parameters, and generates accurate bounding box information accordingly. Compared with the traditional approximation method of axis-aligned bounding boxes, this application can more accurately represent the actual coverage area of ​​Gaussian points on the screen, improve the accuracy of the rasterization process and the fidelity of image details, reduce the number of irrelevant tiles being assigned to bounding boxes, and effectively improve rasterization efficiency.

[0088] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0089] This application also provides a computer device, such as... Figure 9 The diagram shown is a schematic representation of a computer device structure according to an exemplary embodiment of this application. The computer device includes: A processor 91 and a memory 92; the memory 92 stores machine-readable instructions executable by the processor 91, and the processor 91 executes the machine-readable instructions stored in the memory 92. When the machine-readable instructions are executed by the processor 91, the processor 91 performs the following steps: Obtain Gaussian point data of the 3D scene to be rendered; the Gaussian point data includes the spatial geometric parameters and appearance parameters of multiple Gaussian points; For any Gaussian point, based on the spatial geometric parameters and the appearance parameters, determine the equivalent maximum radius of the Gaussian point projected onto the screen space; Based on the spatial geometric parameters, the appearance parameters, and the equivalent maximum radius, the bounding box information of the Gaussian point in the screen space is determined; Based on the bounding box information of each Gaussian point, multiple rasterization index information are determined; the rasterization index information is used to indicate the Gaussian point corresponding to the tile falling on the bounding box in the screen space at at least one depth; Based on the rasterization index information and the Gaussian point data, the 3D scene to be rendered is rasterized. Based on the rasterization processing results, the 3D scene to be rendered is rendered.

[0090] Optionally, the spatial geometric parameters include two-dimensional covariance information; the appearance parameters include opacity. Determining the equivalent maximum radius of the Gaussian point projection in screen space based on the spatial geometric parameters and the appearance parameters includes: The maximum radius of the Gaussian point is determined based on the maximum eigenvalue in the two-dimensional covariance information of the Gaussian point. The first adaptive radius of the Gaussian point is determined based on the logarithm of the ratio between the opacity of the Gaussian point and the preset opacity threshold, and the maximum eigenvalue of the two-dimensional covariance information. The minimum value between the maximum radius and the first adaptive radius is determined to be the equivalent maximum radius.

[0091] Optionally, determining the bounding box information of the Gaussian point in screen space based on the spatial geometric parameters, the appearance parameters, and the equivalent maximum radius includes: Based on the logarithm of the ratio between the opacity of the Gaussian point and the preset opacity threshold, and the maximum value of the two-dimensional covariance information in the first coordinate axis direction, a second adaptive radius in the first coordinate axis direction is determined. Based on the logarithm of the ratio between the opacity of the Gaussian point and the preset opacity threshold, and the maximum value of the two-dimensional covariance information in the direction of the second coordinate axis, the third adaptive radius in the direction of the second coordinate axis is determined. The minimum value between the second adaptive radius and the equivalent maximum radius is determined to be the first adaptive coverage radius in the first coordinate axis direction of the bounding box information; The minimum value between the third adaptive radius and the equivalent maximum radius is determined to be the second adaptive coverage radius in the second coordinate axis direction of the bounding box information.

[0092] Optionally, the step of rasterizing the 3D scene to be rendered based on the rasterization index information and the Gaussian point data includes: Based on the tile number and depth information indicated by the rasterized index information, the rasterized index information is sorted; Based on the sorting order, the Gaussian point data corresponding to the tiles is read, and the rendering information of the tiles is calculated.

[0093] Optionally, reading the Gaussian point data corresponding to the map tile includes: For any given tile, read the Gaussian point data corresponding to that tile from the internal cache; In the event that a read from the internal cache fails, the Gaussian point data corresponding to the tile is read from main memory.

[0094] Optionally, processor 91 also performs: The Gaussian point data read from the main memory is written into the internal cache; If there is no free storage space in the internal cache, the Gaussian point data that meets the preset discard conditions in the internal cache will be discarded, and the Gaussian point data read from the main memory will be rewritten into the internal cache.

[0095] Optionally, the rendering information of the computational tile includes: The first type of data in the rendering information is calculated using a unit with a first computational precision; The second type of data in the rendering information is calculated using a unit with a second calculation precision; The first calculation precision is higher than the second calculation precision; the first type includes relative coordinates and conic section parameters; the second type includes opacity composition and RGB color accumulation.

[0096] Optionally, when calculating the rendering information of a tile, the index calculation result is determined by looking up a table.

[0097] The aforementioned memory 92 includes a main memory 921 and an external memory 922; the main memory 921, also known as internal memory, is used to temporarily store the computational data in the processor 91, as well as the data exchanged with external memory 922 such as a hard disk. The processor 91 exchanges data with the external memory 922 through the main memory 921.

[0098] The specific execution process of the above instructions can be referred to the steps of the screen rendering method described in the embodiments of this application, and will not be repeated here.

[0099] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0100] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the screen rendering method described in the above method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0101] This application also provides a computer program product, including a computer program / instruction, which, when executed by a computer program / instruction processor, implements the screen rendering method provided in the various embodiments of this application.

[0102] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0103] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0104] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0105] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0106] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0107] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0108] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for rendering images, characterized in that, The method includes: Obtain Gaussian point data of the 3D scene to be rendered; the Gaussian point data includes the spatial geometric parameters and appearance parameters of multiple Gaussian points; For any Gaussian point, based on the spatial geometric parameters and the appearance parameters, determine the equivalent maximum radius of the Gaussian point projected onto the screen space; Based on the spatial geometric parameters, the appearance parameters, and the equivalent maximum radius, the bounding box information of the Gaussian point in the screen space is determined; Based on the bounding box information of each Gaussian point, multiple rasterization index information are determined; the rasterization index information is used to indicate the Gaussian point corresponding to the tile falling on the bounding box in the screen space at at least one depth; Based on the rasterization index information and the Gaussian point data, the 3D scene to be rendered is rasterized. Based on the rasterization processing results, the 3D scene to be rendered is rendered.

2. The method according to claim 1, characterized in that, The spatial geometric parameters include two-dimensional covariance information; the appearance parameters include opacity. Determining the equivalent maximum radius of the Gaussian point projection in screen space based on the spatial geometric parameters and the appearance parameters includes: The maximum radius of the Gaussian point is determined based on the maximum eigenvalue in the two-dimensional covariance information of the Gaussian point. The first adaptive radius of the Gaussian point is determined based on the logarithm of the ratio between the opacity of the Gaussian point and the preset opacity threshold, and the maximum eigenvalue of the two-dimensional covariance information. The minimum value between the maximum radius and the first adaptive radius is determined to be the equivalent maximum radius.

3. The method according to claim 2, characterized in that, The process of determining the bounding box information of the Gaussian point in screen space based on the spatial geometric parameters, the appearance parameters, and the equivalent maximum radius includes: Based on the logarithm of the ratio between the opacity of the Gaussian point and the preset opacity threshold, and the maximum value of the two-dimensional covariance information in the first coordinate axis direction, a second adaptive radius in the first coordinate axis direction is determined. Based on the logarithm of the ratio between the opacity of the Gaussian point and the preset opacity threshold, and the maximum value of the two-dimensional covariance information in the direction of the second coordinate axis, the third adaptive radius in the direction of the second coordinate axis is determined. The minimum value between the second adaptive radius and the equivalent maximum radius is determined to be the first adaptive coverage radius in the first coordinate axis direction of the bounding box information; The minimum value between the third adaptive radius and the equivalent maximum radius is determined to be the second adaptive coverage radius in the second coordinate axis direction of the bounding box information.

4. The method according to claim 1, characterized in that, The rasterization process for the 3D scene to be rendered, based on the rasterization index information and the Gaussian point data, includes: Based on the tile number and depth information indicated by the rasterized index information, the rasterized index information is sorted; Based on the sorting order, the Gaussian point data corresponding to the tiles is read, and the rendering information of the tiles is calculated.

5. The method according to claim 4, characterized in that, The process of reading the Gaussian point data corresponding to the map tile includes: For any given tile, read the Gaussian point data corresponding to that tile from the internal cache; In the event that a read from the internal cache fails, the Gaussian point data corresponding to the tile is read from main memory.

6. The method according to claim 5, characterized in that, The method further includes: The Gaussian point data read from the main memory is written into the internal cache; If there is no free storage space in the internal cache, the Gaussian point data that meets the preset discard conditions in the internal cache will be discarded, and the Gaussian point data read from the main memory will be rewritten into the internal cache.

7. The method according to claim 4, characterized in that, The rendering information of the computational graph includes: The first type of data in the rendering information is calculated using a unit with a first computational precision; The second type of data in the rendering information is calculated using a unit with a second calculation precision; The first calculation precision is higher than the second calculation precision; the first type includes relative coordinates and conic section parameters; the second type includes opacity composition and RGB color accumulation.

8. The method according to claim 7, characterized in that, When calculating the rendering information of a tile, the index calculation result is determined by looking up a table.

9. A screen rendering device, characterized in that, The device includes: The acquisition module is used to acquire Gaussian point data of the 3D scene to be rendered; the Gaussian point data includes the spatial geometric parameters and appearance parameters of multiple Gaussian points. The first determining module is used to determine, for any Gaussian point, the equivalent maximum radius of the Gaussian point projected onto the screen space based on the spatial geometric parameters and the appearance parameters. The second determining module is used to determine the bounding box information of the Gaussian point in the screen space based on the spatial geometric parameters, the appearance parameters, and the equivalent maximum radius. The third determining module is used to determine multiple rasterization index information based on the bounding box information of each Gaussian point; the rasterization index information is used to indicate the Gaussian point corresponding to the tile falling on the bounding box in the screen space at at least one depth; A rasterization module is used to perform rasterization processing on the 3D scene to be rendered based on the rasterization index information and the Gaussian point data. The rendering module is used to render the 3D scene to be rendered based on the rasterization processing results.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 8.

Images