Partition rendering mode graphic processing method and system based on position and workload

Abstract

The invention discloses a position and workload-based block rendering mode graphic processing method and system. The system comprises a geometry processing system and a fragment processing system. The geometric processing system is used for carrying out geometric processing on the primitives and partitioning the visible primitives into a plurality of blocks in a screen visual space; the fragment processing system comprises a block processing module and a plurality of fragment processing pipelines, and the block processing module is used for traversing the macro blocks one by one and distributing the blocks in the macro blocks to the plurality of fragment processing pipelines for rendering according to the sequence of rendering workloads of the blocks in the macro blocks from large to small; wherein the macro image blocks are obtained by grouping a plurality of image blocks, and each macro image block comprises a plurality of adjacent image blocks. According to the method, the workload in rendering can be balanced, the cache consistency can be improved, the pressure of the memory space can be reduced, and flexible adjustment can be performed according to actual requirements, so that the method is suitable for occasions with high cost requirements on a computer graphic processing system.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and more specifically, relates to a block rendering mode graphics processing method and system based on position and workload. Background Art

[0002] A computer graphics processing system is used to process graphics data (including primitive data and vertex data) for various computer graphics applications (such as computer games) and output a rendered image produced by a rendering pipeline.

[0003] The three-dimensional model in the graphics application is established by dividing the objects in the scene using primitives (such as but not limited to triangles, lines, and points), which are defined by vertices for their positions in the three-dimensional space, as well as lighting effects and shading characteristics. The geometric data of the primitives and vertices in the three-dimensional model are sent to the computer graphics processing system as an input data stream of the geometry processing stage (geometry processing) in the rendering process, and the input primitives are subjected to geometry processing, including converting the primitives to the screen space and deleting the primitives that are not visible in the screen space. After geometry processing, the geometric data of the primitives and vertices are sent to the fragment processing pipeline (fragment processing pipeline) for rendering in the computer graphics processing system. As a result of the rendering process, the computer graphics processing system generates an output image of the 3D model and displays it on a display unit (such as a display screen).

[0004] Existing computer graphics processing systems include a tile-based block rendering mode. In the block rendering mode, the screen is divided into multiple rectangular tiles, and the primitives are classified into different tiles on the screen after geometric processing, and then fragment processing is performed separately in each tile on the screen to generate a rendered image. Summary of the invention

[0005] In response to the above defects or improvement needs of the prior art, the present invention provides a block rendering mode graphics processing method and system based on position and workload, which can balance the workload in rendering, improve cache consistency, and reduce the pressure of memory space, and can be flexibly adjusted according to actual needs to be suitable for occasions with higher cost requirements for computer graphics processing systems.

[0006] To achieve the above-mentioned purpose, according to one aspect of the present invention, a graphics processing system is provided, including a geometry processing system and a fragment processing system; the geometry processing system is used to divide the visible primitives into multiple tiles in the screen visual space after geometric processing; the fragment processing system includes a tile processing module and multiple fragment processing pipelines, and the tile processing module is used to: traverse the macro tiles one by one, and distribute the tiles in the macro tiles to multiple fragment processing pipelines for rendering in the order of the rendering workload of the tiles in the macro tiles from large to small; wherein the macro tiles are obtained by grouping multiple tiles, and each macro tile includes multiple adjacent tiles.

[0007] In some implementations, the block processing module is further configured to: traverse the macroblocks one by one according to the positions of the macroblocks in the screen visual space, or in the order of the total workload of the macroblocks from high to low.

[0008] In some implementations, the block processing module is further configured to estimate the rendering workload of the blocks in the macro block according to the workload of the blocks in the same rendering in the previous frame.

[0009] In some embodiments, the block processing module is also used to: in the same rendering of the previous frame, for each tile in the macro tile, obtain the total number of cycles CycleCount required for rendering in the corresponding fragment processing pipeline; based on the total number of cycles CycleCount required for rendering each tile, calculate the workload weight Weight = CycleCount / CycleUnit of each tile, where CycleUnit is a set cycle unit; and distribute the tiles in the macro tile to multiple fragment processing pipelines for rendering in the order of the calculated workload weight Weight from large to small.

[0010] In some embodiments, the block processing module is also used to: set Lw workload levels for each macro block; calculate the workload level TileLevel of each block in the macro block based on the calculated workload weight Weight; and determine the workload level corresponding to each block in the macro block among the Lw workload levels based on the calculated workload level TileLevel of the block.

[0011] In some implementations, the total workload of a macroblock is the sum of the workload weights of all tiles in the macroblock, or the number of tile entries in the highest workload level Lw of the macroblock.

[0012] In some embodiments, the block processing module is also used to: generate a tile rendering list containing Lw sequentially arranged workload levels for each macro tile; classify each tile in the macro tile into the workload level corresponding to the tile in the Lw workload levels; and distribute the tiles in the macro tile to multiple fragment processing pipelines for rendering in order of workload levels from high to low.

[0013] In some implementations, the tile processing module is further configured to: generate a tile rendering list for each macro tile; and place all tiles at the highest workload level Lw in the macro tile into the tile rendering list of the macro tile.

[0014] In some embodiments, the block processing module is further used to: generate a block rendering list for each macro block; and put the blocks in the macro block that are at the highest workload level Lw and whose number does not exceed a preset value MaxE into the block rendering list of the macro block.

[0015] In some embodiments, the block processing module is also used to: first distribute the tiles in the tile rendering list of the macro tile to multiple fragment processing pipelines for rendering, and then use a fixed pattern inside the macro tile to distribute the tiles one by one to multiple fragment processing pipelines for rendering; when encountering a tile in the tile rendering list of the macro tile using a fixed pattern, directly skip the tile.

[0016] According to another aspect of the present invention, a graphics processing method is provided, comprising: after geometric processing of primitives, dividing visible primitives into multiple tiles in a screen visual space; traversing macro tiles one by one, and distributing tiles in the macro tiles to multiple fragment processing pipelines for rendering in descending order of the rendering workload of the tiles in the macro tiles; wherein the macro tiles are obtained by grouping multiple tiles, and each macro tile includes multiple adjacent tiles.

[0017] In some implementations, the macroblocks are traversed one by one according to the positions of the macroblocks in the screen visual space or in the order of the total workload of the macroblocks from high to low.

[0018] In some embodiments, the method further includes: estimating the rendering workload of the tile in the macro tile according to the workload of the tile in the same rendering in the previous frame.

[0019] In some embodiments, estimating the rendering workload of the tiles in the macro tile based on the workload of the tiles in the same rendering of the previous frame includes: in the same rendering of the previous frame, for each tile in the macro tile, obtaining the total number of cycles CycleCount required for rendering in the corresponding fragment processing pipeline; based on the total number of cycles CycleCount required for rendering each tile, calculating the workload weight Weight = CycleCount / CycleUnit of each tile, where CycleUnit is a set cycle unit; and using the calculated workload weight Weight as the workload weight of the tile in the macro tile.

[0020] In some embodiments, distributing the tiles in the macro block to multiple fragment processing pipelines for rendering in order of the rendering workload of the tiles in the macro block from large to small includes: distributing the tiles in the macro block to multiple fragment processing pipelines for rendering in order of the calculated workload weights Weight from large to small.

[0021] In some embodiments, distributing tiles in a macro tile to multiple fragment processing pipelines for rendering in order of calculated workload weights Weight from large to small includes: setting Lw workload levels for each macro tile; calculating the workload level TileLevel of each tile in the macro tile according to the calculated workload weight Weight; and determining the workload level corresponding to each tile in the macro tile among the Lw workload levels according to the calculated tile workload level TileLevel.

[0022] In some implementations, the total workload of a macroblock is the sum of the workload weights of all tiles in the macroblock, or the number of tile entries in the highest workload level Lw of the macroblock.

[0023] In some embodiments, distributing the tiles in the macro tile to multiple fragment processing pipelines for rendering in order of the calculated workload weight Weight from large to small also includes: for each macro tile, generating a tile rendering list containing Lw sequentially arranged workload levels; classifying each tile in the macro tile to the workload level corresponding to the tile in the Lw workload levels; distributing the tiles in the macro tile to multiple fragment processing pipelines for rendering in order of the workload level from high to low.

[0024] In some implementations, distributing the tiles in the macro tile to the multiple fragment processing pipelines for rendering in descending order of the calculated workload weights Weight further includes: For each macro tile, generate a tile rendering list; All tiles at the highest workload level Lw in the macro tile are placed in the tile rendering list of the macro tile, or tiles at the highest workload level Lw in the macro tile whose number does not exceed the preset value MaxE are placed in the tile rendering list of the macro tile.

[0025] In some embodiments, distributing the tiles in the macro block to multiple fragment processing pipelines for rendering in order of the calculated workload weight Weight from large to small also includes: first distributing the tiles in the tile rendering list of the macro block to multiple fragment processing pipelines for rendering, and then distributing the tiles one by one to the multiple fragment processing pipelines for rendering within the macro block using a fixed pattern; when encountering a tile in the tile rendering list of the macro block using a fixed pattern, directly skipping the tile.

[0026] According to another aspect of the present invention, there is provided an electronic device comprising the above-mentioned graphics processing system; or, the electronic device comprises: a processor; a memory communicatively connected to the processor; the memory stores instructions executable by the processor, the instructions being executed by the processor so that the processor can execute the above-mentioned method.

[0027] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the above method is implemented.

[0028] In general, the above technical solution conceived by the present invention has the following beneficial effects compared with the prior art: considering the position of the tile and the workload of the tile at the same time, rendering is performed in units of macro tiles on the one hand, and on the other hand, within the macro tile, the tiles are allocated according to the workload of the tiles, so that the tiles with large workload are processed first, which helps to balance the workload in rendering, improve the consistency of the cache, and reduce the pressure of the memory space. In addition, the tiles are sorted according to the total workload, while improving the consistency of the cache and reducing the pressure of the memory space, the workload in rendering is further balanced, and a simplified rendering list and a fixed rendering list can be used according to actual needs to save storage space and the read / write bandwidth of the memory, so it can be suitable for occasions with high cost requirements for computer graphics processing systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a typical tile allocation method when distributing tiles to multiple fragment processing pipelines for rendering; Figure 2 is an example of a tile to be rendered on the screen; Figure 3 Yes Figure 2 The amount of work to be done to shade the tiles shown; Figure 4It is a schematic diagram of distributing tiles in units of macro tiles when distributing tiles to multiple fragment processing pipelines for rendering; Figure 5 is a structural schematic diagram of a graphics processing system in a block rendering mode based on position and workload according to an embodiment of the present invention; Figure 6 It is a schematic diagram of the macro tile rendering list and the tile rendering list; Figure 7 It is a flowchart of a block rendering mode graphics processing method based on position and workload according to an embodiment of the present invention; Figure 8 It is a structural block diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. As those skilled in the art can appreciate, the described embodiments can be modified in various ways without departing from the spirit or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0031] In the tile rendering mode, tiles can be distributed to multiple fragment processing pipelines for rendering. Since the rendering of different tiles is performed independently, the allocation of tiles can follow any rules. Figure 1 As shown in FIG. 1 , a typical allocation method is to use a fixed scan line mode, with 2×2 tiles as the unit, linear scan allocation, and within the 2×2 tiles, allocation in an inverted N-type order. This fixed tile allocation mode is easy to implement in hardware, but it is not the optimal solution for the performance of computer graphics processing systems, especially when the rendering workload is unevenly distributed among the tiles on the screen.

[0032] In fact, the amount of work required to render a tile on the screen depends on the settings of the rendering scene. In many cases, the amount of work required to render different tiles varies greatly. Figure 2 Take the tiles to be rendered on the screen as an example. Figure 3 The amount of work required to shade these tiles is given, represented by the execution cycles in the tiles on the screen.

[0033] from Figure 3It can be seen that due to the uneven distribution of rendering workload, the shading execution time of each tile varies greatly. The minimum shading execution time of a tile is 105 cycles, while the maximum shading execution time of a tile is 9075 cycles. In other words, the maximum shading execution time of a tile is about 86 times the minimum shading execution time. If a fixed tile allocation mode is used and the workload of the tile is not controlled, Figure 3 The busiest tile in a rendering may not be processed until near the end, so it is difficult to avoid long processing delays relative to other tiles, thus affecting overall performance.

[0034] To this end, the embodiments of the present invention take the workload into consideration when allocating tiles during rendering, so that tiles with large workloads are processed first. Allocating tiles based on their workload will help avoid long processing delays for tiles with large workloads, thereby improving overall performance.

[0035] However, since tiles with similar workloads are likely to be sparsely located on the screen, when tiles are rendered simultaneously and are not adjacent to each other, the data overlap used for rendering is reduced, thus reducing cache coherence. In addition, for macrotiles containing multiple adjacent tiles, typically the storage pages used for primitive and vertex data in the macrotile can be released after the rendering of the macrotile is complete, but since the positions of tiles with similar workloads on the screen are random, it is difficult to mark the end of a macrotile event, so the storage pages are released only when the rendering of all macrotiles is complete. This may put additional pressure on the memory space of the next geometry processing stage in parallel with the current fragment processing stage.

[0036] like Figure 4 As shown, the screen is divided into macroblocks M0 to M15, and linear scanning is performed in units of macroblocks M0 to M15. Figure 1 When rendering tiles in the order of macro tiles, the positions of tiles processed in the multi-pipeline of the GPU core on the screen are more clear, compared with the linear scan allocation in the default order in the screen width direction as shown in Figure 1. Figure 4 The approach shown helps improve cache coherence, and memory pages allocated to a macrotile can be reallocated after all tiles in the macrotile are rendered. Therefore, memory pages can become available early in the fragment processing stage for use by the concurrent geometry processing stage of the next rendering.

[0037] Therefore, in an embodiment of the present invention, on the one hand, rendering is performed in units of macro blocks, and on the other hand, within the macro blocks, tiles are allocated according to their workload, so that tiles with large workloads are processed first, which helps to balance the workload in rendering, improve cache consistency, and reduce the pressure on memory space.

[0038] like Figure 5 As shown, the position and workload-based block rendering mode graphics processing system of an embodiment of the present invention includes a geometry processing system and a fragment processing system. The geometry processing system is used to construct primitives according to input geometry data (such as vertex data), and after geometric processing of the primitives, the visible primitives are divided into multiple tiles in the screen visual space, and a tile display list is generated for each tile.

[0039] The geometry processing system further includes an input assembly module, a geometry processing pipeline and a tile module. The input assembly module is used to construct primitives according to the input geometry data. The geometry processing pipeline is used to process the constructed primitives, delete the primitives that are not visible in the screen visual space, and only retain the primitives that are visible in the screen visual space. The tile module is used to tile the visible primitives into multiple tiles of the screen visual space and generate a tile display list for each tile.

[0040] In some embodiments, the tile display list includes all graphics elements that at least partially overlap with the tile and need to be rendered in the tile; when a certain graphics element is located in multiple tiles, the graphics element will be included in the tile display list of each tile in the screen visual space where the graphics element is located.

[0041] The geometry processing pipeline further includes a geometry conversion module, a clipping and culling module, and a data receiving module. The geometry conversion module is used to convert the vertex data of the primitives constructed by the input assembly module to the screen visual space. The clipping and culling module is used to delete the primitives that are not visible in the screen visual space and only retain the primitives that are visible in the screen visual space. These primitives are the primitives required for rendering the scene. The data receiving module is used to receive and store the primitive data and the converted vertex data of the primitives that are visible in the screen visual space.

[0042] In some embodiments, the geometry processing system includes a geometry processing pipeline, and the primitives constructed by the input assembly module are sent to the geometry processing pipeline for processing. In some embodiments, the geometry processing system includes multiple geometry processing pipelines, and the primitives constructed by the input assembly module are distributed to multiple downstream geometry processing pipelines for processing respectively by the multiple geometry processing pipelines.

[0043] The graphic element data and the converted vertex data in the data receiving module are written into the first memory of the storage module, and the block display list generated by the block module is written into the second memory of the storage module.

[0044] The fragment processing system is used to render multiple tiles according to the tile display list generated by the geometry processing system, and generate rendering images of the multiple tiles. The fragment processing system includes a tile processing module and multiple fragment processing pipelines. The tile processing module is used to read the tile display list of the tile generated by the geometry processing system from the second memory, and distribute the primitives required for rendering the tiles in the tile display list (i.e., the primitives referenced in the tile display list) to multiple fragment processing pipelines in units of tiles; each fragment processing pipeline is used to render the tile according to the primitive metadata and converted vertex data of the primitive referenced in the tile display list, and generate a rendering image of the tile.

[0045] That is to say, the block processing module distributes the graphics primitives required for rendering the multiple tiles in the block display list of the multiple tiles to the multiple fragment processing pipelines, wherein the graphics primitives required for rendering a tile in the block display list of the tile are sent to a fragment processing pipeline, and the multiple fragment processing pipelines work together to complete the rendering of the multiple tiles.

[0046] In some embodiments, the fragment processing pipeline includes a rasterization module, a hidden surface removal module, and a pixel shading module. In other embodiments, the fragment processing pipeline includes a rasterization module, a hidden surface removal module, a pixel shading module, and a post-processing module.

[0047] The rasterization module is used to read the primitive data and converted vertex data required by the primitives referenced in the tile display list from the first memory, and rasterize the primitives into visible pixels. In some embodiments, for each tile, the primitives are rasterized into visible pixels within the tile.

[0048] The hidden surface removal module is used to perform a depth test on the visible pixels output by the rasterization module. Specifically, the depth value of the visible pixels output by the rasterization module is compared with the depth value of the previous primitive pixel stored in the depth buffer. When the primitive pixel output by the rasterization module is blocked by the previous primitive pixel, it is considered that the depth test has not passed, and the primitive pixel output by the rasterization module is removed. Otherwise, the primitive pixel output by the rasterization module is sent to the pixel shading module. In other words, the pixels behind other primitives at the same pixel position are removed by the hidden surface removal module. In some embodiments, the depth value of the previous primitive pixel is a preset value. In some embodiments, the depth value of the previous primitive pixel is the depth value of the primitive pixel that previously passed the depth test.

[0049] The pixel coloring module is used to color the visible pixels output by the hidden surface removal module to obtain the final color of the primitive pixel. In some embodiments, the color information of the pixels in the tile is stored in an on-chip color buffer in a computer graphics processing system in a tile rendering mode to avoid frequent data exchange with an external memory, which affects processing efficiency.

[0050] The post-processing module is used to perform post-processing operations on the accumulated primitive pixel colors, such as masking, format conversion, downsampling and pixel filtering, to obtain the final rendered image of the block. The final rendered image can be sent to a display unit outside the computer graphics processing system for display.

[0051] In some embodiments, reference Figure 4 , the screen visual space is divided to obtain a plurality of macro blocks, each of which includes a plurality of adjacent blocks. In some embodiments, each macro block includes an equal number of blocks. The block processing module distributes the blocks to a plurality of fragment processing pipelines for rendering in a manner of traversing the macro blocks one by one.

[0052] Specifically, in one embodiment of the present invention, the block processing module traverses the macroblocks one by one according to the positions of the macroblocks in the screen visual space. For example, the first row of macroblocks is distributed horizontally from left to right, and after the rendering of the first row of macroblocks is completed, the second row of macroblocks is distributed vertically downward from left to right until the rendering of all macroblocks in the screen visual space is completed.

[0053] For the tiles in each macro tile, the tile processing module distributes the tiles to multiple fragment processing pipelines for processing according to the order of the rendering workload of the tiles from large to small.

[0054] Furthermore, the workload of a tile needs to be known before rendering so that it can be used for tile allocation in rendering. However, the workload of rendering can only be known after the tile is rendered, so the workload of the tile must be estimated before rendering.

[0055] In some embodiments, a time statistics method is used to estimate the workload of the current rendering based on the workload in the previous rendering. Specifically, the workload of the rendering of the tile can be known through some statistical counters after rendering. Assuming that the rendering content does not change much between adjacent frames, the workload of the rendering of the tile can be used to estimate the workload of the tile in the same rendering of the next frame.

[0056] In some embodiments, during the stage when the fragment processing pipeline renders tiles, a statistical counter is used to count the processing cycles of the tiles to obtain the number of cycles required for the fragment processing pipeline to complete all work of rendering the tiles. Therefore, it is very suitable for estimating the workload of the fragment processing pipeline for rendering tiles.

[0057] In some embodiments, in order to estimate the workload of the fragment processing pipeline for rendering a tile, the accumulation of statistical counters needs to be done on a per-tile basis. Figure 5The block processing module shown stores the initial value of the statistical counter at the beginning of a tile (that is, before the fragment processing pipeline renders the tile). After the fragment processing pipeline starts rendering the tile, the statistical counter starts counting the processing cycles of the tile. When the rendering of this tile is completed, the current value of the statistical counter is subtracted from the initial value to obtain the total number of cycles required for the fragment processing pipeline to render the tile.

[0058] In some embodiments, for each tile, after rendering is completed, the number of cycles required to complete the rendering is stored in a separate memory, so as to estimate the workload of the fragment processing pipeline to render the tile in the same rendering of the next frame.

[0059] When the rendering of the tile is completed, the total number of cycles required to render the tile can be calculated by tracking the cycle count of the tile. In some embodiments, for each tile, a workload weight of the tile is generated based on the total number of cycles required to render the tile. Specifically, the total number of cycles required to render the tile may be thousands, and for the workload weight of the tile, there is no need to use the actual cycle count of the tile. For example, the unit of the total number of cycles required to render the tile can be 4024 cycles to form the workload weight of the tile. In this case, only 8 bits are needed to store the workload weight of the tile, which allows the total number of cycles for each tile to be as high as 1M cycles.

[0060] In some implementations, the workload weight Weight of a tile and the total number of cycles CycleCount required to render the tile satisfy the following relationship: Weight = CycleCount / CycleUnit, Wherein, CycleUnit is a set cycle unit. In some implementations, CycleUnit is 4024 cycles.

[0061] In some embodiments, after the rendering of a tile is completed, an 8-bit workload weight of the tile is calculated. The workload weight can be stored in a memory so as to be used for sorting the tile workload after the rendering of all tiles on the screen is completed.

[0062] In order to balance the workload of the tiles on the screen, the tiles with large workload should be processed first to avoid the increase in the total rendering cycle due to tailing. Therefore, in some embodiments, after the rendering is completed, the tiles are sorted according to the workload weight of the tiles within the macro tile to form a tile rendering list of the macro tile. The sorted tile rendering list can be written to the memory so as to be read out in the same rendering of the next frame.

[0063] In some embodiments, a maximum workload weight and a minimum workload weight are set to calculate the range of workload weights in rendering. In some embodiments, the maximum workload weight in rendering is initialized to 0 at the beginning of rendering, and the minimum workload weight in rendering is initialized to 0xFF at the beginning of rendering. After calculating the workload weight of the tile, the maximum workload weight and the minimum workload weight are updated. At the end of rendering, the maximum workload weight and the minimum workload weight of the rendering can be known.

[0064] Although 8 bits have been used to represent the workload weight of a tile in rendering, in order to allocate tiles according to their workload, in some embodiments, within a macro tile, the workload weight of the tile is divided into levels, and the 8-bit workload weight of the tile is converted into a workload level of the tile. In some embodiments, the workload weight of the tile is divided into Lw levels, and the workload level TileLevel of the tile can be expressed as: TileLevel = Weight / WeightGrade, WeightGrade = (WeightMax – WeightMin) / Lw, Among them, WeightMax is the maximum workload weight of the blocks in the calculated macro block, WeightMin is the minimum workload weight of the blocks in the calculated macro block, and WeightGrade is the weight interval size of each level.

[0065] After the rendering is completed, the workload levels of all tiles in the macro tile are calculated, and according to the workload levels of the tiles, a workload-based tile rendering list of the macro tile is generated.

[0066] In some embodiments, the tile coordinates Tile(X, Y) are stored in a tile rendering list to indicate the tile in the workload level of the rendering list. In some embodiments, the tile rendering list includes Lw workload levels arranged in sequence. After rendering is completed, each tile on the screen is classified into one of the workload levels according to the workload level of each tile. After all tiles are sorted, the number of tile entries in each workload level is stored respectively.

[0067] In some embodiments, each macro block has its own Lw levels of tile rendering list, and each tile within the macro block is classified into the workload level corresponding to the tile in the tile rendering list of the macro block. That is, the tile rendering list formed at this time is a complete rendering list containing all tiles in the macro block.

[0068] According to the tile rendering list of the macro tile, the tiles in the macro tile are processed in the order of workload level from high to low, that is, the tiles in the macro tile are distributed to multiple fragment processing pipelines for processing. Specifically, the tiles in the highest workload level in the tile rendering list, that is, the tiles with level Lw, are processed first; after the rendering of all tiles in the current workload level is completed, the tiles in the next workload level are processed, and the workload levels are processed in descending order until the rendering of tiles in all workload levels is completed. In some embodiments, the workload level with 0 tile entries is skipped.

[0069] In another embodiment of the present invention, a workload-based macroblock rendering list is generated according to the total workload sorting of the macroblocks; and according to the macroblock rendering list, all macroblocks on the screen are rendered one by one in the order of total workload from high to low.

[0070] In some embodiments, the total workload Wmt of a macroblock is the sum of the workload weights of all tiles in the macroblock. In some embodiments, the total workload Wmt of a macroblock is the number of tile entries in the highest workload level Lw of the macroblock, that is, the number of tiles in the highest workload level Lw.

[0071] When some large-volume tiles are not finished rendering in one or two GPU pipelines while rendering in other pipelines is completed, it will cause the total cycle count of the rendering to increase. Therefore, sorting the macrotiles to render the macrotiles with the largest total workload in the macrotile rendering list first can help further avoid the tailing effect near the end of the rendering.

[0072] The rendering of the tiles in each macro tile can be processed in different ways according to actual needs.

[0073] In some implementations, the tile processing module distributes the tiles to multiple fragment processing pipelines for processing according to the order of rendering workload of the tiles from large to small.

[0074] In some implementations, the tile processing module distributes the tiles to multiple fragment processing pipelines for processing in descending order of the workload weights of the tiles.

[0075] In some embodiments, a tile rendering list of a macro tile is generated, and the tile rendering list of the macro tile has Lw workload levels, each workload level includes all tiles in the macro tile at that workload level, that is, the tile rendering list at this time is a complete rendering list. According to the tile rendering list of the macro tile, the tiles in the macro tile are processed in the order of workload levels from high to low.

[0076] like Figure 6As shown, for the (e.g., 16) macroblocks on the screen, the macroblock rendering list is stored in the memory. Specifically, the macroblock information is stored in the order of the total workload of the macroblock from high to low, marked as MT0 to MT15. For the tiles in each macroblock, the tile rendering list is stored in the memory. First, in descending order of the workload level of the tiles, the number of tile entries in each workload level is stored respectively, and then the tile information in each workload level is stored respectively. Each workload level contains all the tiles in the macroblock at that workload level. For example, for macroblock MT0, first, the number of tile entries E3 to E0 in level 4 to level 1 are stored respectively, and then the tile information T of E3 in level 4 is stored respectively. 31 To T 3E3 , E2 tile information in level 3 T 21 To T 2E2 , E1 tile information in level 2 T 11 To T 1E1 , and E0 tile information T in level 1 01 To T 0E0 .

[0077] In some embodiments, the tile information is tile coordinates, that is, the tile coordinates Tile(X, Y) are placed in the tile rendering list to indicate the tile in the workload level of the rendering list.

[0078] In some implementations, within a macroblock, two 8-bit unsigned integers are used as block coordinates, and the block coordinate range can represent 256 32×32 pixel blocks in the X and Y directions. For a 4×4 macroblock on the screen, the maximum rendering size of 32K×32K can be achieved.

[0079] In other embodiments, the tile rendering list of the macro tile has only one workload level, which includes all tiles in the macro tile at the highest workload level Lw, that is, the tile rendering list at this time is a simplified rendering list.

[0080] In some other embodiments, the tile rendering list of the macro tile has only one workload level, which includes tiles in the macro tile that are at the highest workload level Lw and whose number does not exceed the preset value MaxE, that is, the tile rendering list at this time is a fixed rendering list, and the number of tiles in the tile rendering list may be less than MaxE, but can only be MaxE at most. In some embodiments, the value of MaxE is much smaller than the number of tiles in the macro tile.

[0081] When rendering macro tiles, for the cases of simplified rendering list and fixed rendering list, it is divided into two stages. First, in the first stage, the tiles in the tile rendering list of the macro tile are distributed to multiple fragment processing pipelines for rendering; then in the second stage, according to the position of the tiles on the screen, a fixed pattern is used to distribute the tiles one by one to multiple fragment processing pipelines for rendering. At this time, it is necessary to skip the tiles in the tile rendering list. In some embodiments, in the second stage, a fixed pattern is used to distribute the tiles one by one to multiple fragment processing pipelines for rendering. Figure 4 In the fixed mode shown, tiles are distributed one by one to multiple fragment processing pipelines for rendering.

[0082] The use of simplified rendering lists and fixed rendering lists helps save storage space and memory read / write bandwidth, and is particularly suitable for situations where the cost requirements for computer graphics processing systems are high. Compared with simplified rendering lists, fixed rendering lists can further save storage space and memory read / write bandwidth. For example, when MaxE is set to 64, that is, a fixed rendering list can only accommodate a maximum of 64 tiles. When the rendering size is 2048×2048, there are 256 tiles in a macro tile. Compared with the simplified rendering list, the storage space required for the fixed rendering list is only one-fourth of that for the simplified rendering list.

[0083] The storage space for storing tile workload weight and tile workload level should be allocated by the software driver at the beginning of the tile processing stage of rendering. For example, for each tile, 8 bits of storage space can be allocated for the tile workload weight at the beginning of rendering. After the tile workload level is calculated based on the tile workload weight and the tile is sorted into the tile rendering list, the storage space for the tile workload weight is released at the end of rendering.

[0084] The storage space of the tile rendering list can be allocated from the first frame when rendering starts. The storage space of the tile rendering list can be used for the same rendering in the next frame, so it cannot be released at the end of rendering. The storage space of the tile rendering list can be allocated as persistent memory during the life cycle of the application.

[0085] At the start of rendering, you can Figure 5 The block processing module of the computer graphics processing system shown in FIG. 1 reads out the block rendering list generated in the rendering of the previous frame. Figure 6 As shown, when a macroblock rendering list exists, the macroblock rendering list is read out before the tile rendering list in each macroblock is read out.

[0086] Render the macroblocks on the screen according to the macroblock rendering list obtained in the previous frame rendering. Taking the rendering shown in Figure 3 as an example, the block with heavy workload is located in the center of the screen. Therefore, giving priority to rendering the macroblocks M5, M6, M9 and M10 in the center of the screen will help avoid long execution delays caused by the heavy workload of these macroblocks. Figure 3 For the macroblock M13 where the tile with the maximum execution time of 9075 cycles is located, if the macroblock rendering list is sorted according to the total workload of all tiles in the macroblock, macroblock M13 may be rendered after macroblocks M5, M6, M9 and M10; and if the macroblock rendering list is sorted based on the number of tile entries with the largest workload level in the tile rendering list of the macroblock, macroblock M13 may be rendered before macroblocks M5, M6, M9 and M10.

[0087] For rendering in a macro tile, the tile rendering list of the macro tile and the number of entries in each workload level in the tile rendering list need to be read from the storage space, and then the tiles are distributed to multiple fragment processing pipelines for rendering according to the order of the workload levels in the tile rendering list. Tiles in the highest workload level Lw are processed first, and then tiles in other workload levels are processed in descending order of workload levels.

[0088] When the rendering of all the tiles in the tile rendering list of the macro tile is completed, the macro tile end event will be triggered to release the storage page allocated to the macro tile. According to the macro tile rendering list, the rendering process of the next macro tile will be started. When the rendering of all the macro tiles in the macro tile rendering list is completed, the rendering end event will be triggered to release all the storage pages allocated for rendering.

[0089] like Figure 7 As shown, the block rendering mode graphics processing method based on position and workload in an embodiment of the present invention includes: Step S701: After geometric processing of the primitives, the visible primitives are divided into a plurality of blocks in the screen visual space.

[0090] Step S703: Estimate the rendering workload of the tiles in the macro tile according to the workload of the tiles in the same rendering process in the previous frame.

[0091] In some embodiments, in the same rendering of the previous frame, for each tile in the macro tile, the total number of cycles CycleCount required for rendering in the corresponding fragment processing pipeline is obtained; based on the total number of cycles CycleCount required for rendering each tile, the workload weight Weight = CycleCount / CycleUnit of each tile is calculated, where CycleUnit is a set cycle unit; the calculated workload weight Weight is used as the workload weight of the tile in the macro tile.

[0092] Step S705: traverse the macroblocks one by one, and distribute the tiles in the macroblock to multiple fragment processing pipelines for rendering in the order of the rendering workload of the tiles in the macroblock from large to small; wherein the macroblock is obtained by grouping the multiple tiles, and each macroblock includes multiple adjacent tiles.

[0093] In some implementations, the macroblocks are traversed one by one according to the positions of the macroblocks in the screen visual space or in the order of the total workload of the macroblocks from high to low.

[0094] In some implementations, tiles in a macro tile are distributed to multiple fragment processing pipelines for rendering in descending order of the calculated workload weights Weight.

[0095] In some embodiments, Lw workload levels are set for each macro tile; the workload level TileLevel of each tile in the macro tile is calculated based on the calculated workload weight Weight; and the workload level corresponding to each tile in the macro tile among the Lw workload levels is determined based on the calculated tile workload level TileLevel.

[0096] In some implementations, the total workload of a macroblock is the sum of the workload weights of all tiles in the macroblock, or the number of tile entries in the highest workload level Lw of the macroblock.

[0097] In some embodiments, for each macro tile, a tile rendering list containing Lw sequentially arranged workload levels is generated; each tile in the macro tile is classified into a workload level corresponding to the tile in the Lw workload levels; and the tiles in the macro tile are distributed to the multiple fragment processing pipelines for rendering in order of workload levels from high to low.

[0098] In some implementations, for each macroblock, a tile rendering list is generated; all tiles in the macroblock at the highest workload level Lw are placed in the tile rendering list of the macroblock. At this time, there is only one workload level in the tile rendering list of the macroblock.

[0099] In some implementations, for each macroblock, a tile rendering list is generated; tiles in the macroblock that are at the highest workload level Lw and whose number does not exceed a preset value MaxE are placed in the tile rendering list of the macroblock. At this time, there is only one workload level in the tile rendering list of the macroblock.

[0100] In some embodiments, when there is only one workload level in the tile rendering list of the macro tile, the tiles in the tile rendering list of the macro tile are first distributed to multiple fragment processing pipelines for rendering, and then a fixed pattern is used inside the macro tile to distribute the tiles one by one to multiple fragment processing pipelines for rendering; when a tile in the tile rendering list of the macro tile is encountered using a fixed pattern, the tile is directly skipped.

[0101] More details of the above-mentioned block rendering mode graphics processing method based on position and workload have been recorded in detail in the above-mentioned part of the block rendering mode graphics processing system based on position and workload, and the present invention will not be repeated here.

[0102] The present invention considers both the position of the tile and the workload of the tile. On the one hand, the rendering is performed in units of macro tiles. On the other hand, within the macro tile, the tiles are allocated according to the workload of the tiles, so that the tiles with large workloads are processed first, which helps to balance the workload in rendering, improve cache consistency, and reduce the pressure of memory space. In addition, the tiles are sorted according to the total workload, which further balances the workload in rendering while improving cache consistency and reducing memory space pressure. A simplified rendering list and a fixed rendering list can be used according to actual needs to save storage space and memory read / write bandwidth, so it can be suitable for occasions with high cost requirements for computer graphics processing systems.

[0103] Figure 8 is a structural block diagram of an electronic device according to an embodiment of the present application. The present application embodiment also provides an electronic device, such as Figure 8As shown, the electronic device includes: at least one processor 801, and a memory 803 connected to the at least one processor 801 in communication. Instructions that can be executed by at least one processor 801 are stored in the memory 803. The instructions are executed by at least one processor 801. When the processor 801 executes the instructions, the graphics processing method in the above embodiment is implemented. The number of the memory 803 and the processor 801 can be one or more. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples, and are not intended to limit the implementation of the present application described and / or required herein.

[0104] The electronic device may also include a communication interface 805 for communicating with external devices and performing data exchange transmission. The various devices are interconnected using different buses and may be installed on a common motherboard or in other ways as needed. The processor 801 may process instructions executed in the electronic device, including instructions stored in or on a memory to display graphical information of a graphical user interface (GUI) on an external input / output device (such as a display device coupled to an interface). In other embodiments, if necessary, multiple processors and / or multiple buses may be used together with multiple memories and multiple memories. Similarly, multiple electronic devices may be connected, each device providing some of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 8 The fact that only one line is used in the diagram does not mean that there is only one bus or only one type of bus.

[0105] Optionally, in a specific implementation, if the memory 803, the processor 801 and the communication interface 805 are integrated on a chip, the memory 803, the processor 801 and the communication interface 805 can communicate with each other through an internal interface.

[0106] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor that supports the Advanced RISC Machines (ARM) architecture.

[0107] The embodiment of the present application provides a computer-readable storage medium (such as the memory 803 described above), which stores computer instructions. When the program is executed by a processor, the method provided in the embodiment of the present application is implemented.

[0108] Optionally, the memory 803 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created according to the use of the electronic device of the graphics processing method, etc. In addition, the memory 803 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 803 may optionally include a memory remotely arranged relative to the processor 801, and these remote memories may be connected to the electronic device of the graphics processing method via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0109] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0110] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0111] Any process or method description in the flow chart or otherwise described herein can be understood to represent a module, fragment or portion of a code including one or more (two or more) executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0112] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0113] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0114] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0115] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A graphics processing system, characterized in that: Includes geometry processing system and fragment processing system; The geometry processing system is used to divide the visible graphics primitives into a plurality of graphics blocks in the screen visual space after geometry processing is performed on the graphics primitives; The fragment processing system includes a block processing module and a plurality of fragment processing pipelines, wherein the block processing module is used to: traverse the macroblocks one by one, and distribute the blocks in the macroblocks to the plurality of fragment processing pipelines for rendering in the order of the rendering workload of the blocks in the macroblocks from large to small; The macroblock is obtained by grouping the multiple blocks, and each macroblock includes multiple adjacent blocks.

2. The graphics processing system according to claim 1, wherein: The block processing module is also used to: traverse the macroblocks one by one according to the positions of the macroblocks in the screen visual space, or in the order of the total workload of the macroblocks from high to low.

3. The graphics processing system according to claim 2, characterized in that: The block processing module is also used to estimate the rendering workload of the blocks in the macro block according to the workload of the blocks in the same rendering in the previous frame.

4. The graphics processing system according to claim 3, characterized in that: The block processing module is also used to: in the same rendering of the previous frame, for each tile in the macro tile, obtain the total number of cycles CycleCount required for rendering in the corresponding fragment processing pipeline; according to the total number of cycles CycleCount required for rendering each tile, calculate the workload weight Weight = CycleCount / CycleUnit of each tile, where CycleUnit is a set cycle unit; and distribute the tiles in the macro tile to the multiple fragment processing pipelines for rendering in the order of the calculated workload weight Weight from large to small.

5. The graphics processing system according to claim 4, characterized in that: The block processing module is also used to: set Lw workload levels for each macro block; calculate the workload level TileLevel of each block in the macro block according to the calculated workload weight Weight; determine the workload level corresponding to each block in the macro block among the Lw workload levels according to the calculated workload level TileLevel of the block.

6. The graphics processing system according to claim 5, characterized in that: The total workload of a macroblock is the sum of the workload weights of all tiles in the macroblock, or the number of tile entries in the highest workload level Lw of the macroblock.

7. The graphics processing system according to claim 5, characterized in that: The block processing module is also used to: generate a block rendering list containing Lw sequentially arranged workload levels for each macro block; classify each block in the macro block into the workload level corresponding to the block in the Lw workload levels; and distribute the blocks in the macro block to the multiple fragment processing pipelines for rendering in order of workload levels from high to low.

8. The graphics processing system according to claim 5, characterized in that: The block processing module is further used to: generate a block rendering list for each macro block; and put all blocks in the macro block at the highest workload level Lw into the block rendering list of the macro block.

9. The graphics processing system according to claim 5, characterized in that: The block processing module is also used to: generate a block rendering list for each macro block; and put the blocks in the macro block that are at the highest workload level Lw and whose number does not exceed a preset value MaxE into the block rendering list of the macro block.

10. The graphics processing system according to claim 8 or 9, characterized in that: The block processing module is also used to: first distribute the tiles in the tile rendering list of the macro tile to the multiple fragment processing pipelines for rendering, and then use a fixed pattern inside the macro tile to distribute the tiles one by one to the multiple fragment processing pipelines for rendering; when encountering a tile in the tile rendering list of the macro tile using a fixed pattern, directly skip the tile.

11. A graphics processing method, characterized in that: include: After geometric processing of primitives, visible primitives are divided into multiple tiles in the screen visual space; The macroblocks are traversed one by one, and the tiles in the macroblock are distributed to multiple fragment processing pipelines for rendering in the order of the rendering workload of the tiles in the macroblock from large to small; wherein the macroblock is obtained by grouping the multiple tiles, and each macroblock includes multiple adjacent tiles.

12. The graphics processing method according to claim 11, characterized in that: The macroblocks are traversed one by one according to the positions of the macroblocks in the screen visual space or in the order of the total workload of the macroblocks from high to low.

13. The graphics processing method according to claim 12, characterized in that: Also includes: Estimate the rendering workload of the tiles in the macro tile based on the workload of the tiles in the same rendering in the previous frame.

14. The graphics processing method according to claim 13, characterized in that: According to the workload of the tiles in the same rendering of the previous frame, estimating the rendering workload of the tiles in the macro tile includes: in the same rendering of the previous frame, for each tile in the macro tile, obtaining the total number of cycles CycleCount required for rendering in the corresponding fragment processing pipeline; according to the total number of cycles CycleCount required for rendering each tile, calculating the workload weight Weight = CycleCount / CycleUnit of each tile, where CycleUnit is a set cycle unit; and using the calculated workload weight Weight as the workload weight of the tile in the macro tile.

15. The graphics processing method according to claim 14, characterized in that: Distributing the tiles in the macro block to multiple fragment processing pipelines for rendering in a descending order of the rendering workload of the tiles in the macro block includes: distributing the tiles in the macro block to multiple fragment processing pipelines for rendering in a descending order of the calculated workload weights Weight.

16. The graphics processing method according to claim 15, characterized in that: Distributing the tiles in the macro tile to multiple fragment processing pipelines for rendering in the order of the calculated workload weight Weight from large to small includes: setting Lw workload levels for each macro tile; calculating the workload level TileLevel of each tile in the macro tile according to the calculated workload weight Weight; determining the workload level corresponding to each tile in the macro tile among the Lw workload levels according to the calculated workload level TileLevel of the tile.

17. The graphics processing method according to claim 16, characterized in that: The total workload of a macroblock is the sum of the workload weights of all tiles in the macroblock, or the number of tile entries in the highest workload level Lw of the macroblock.

18. The graphics processing method according to claim 16, characterized in that: Distributing the tiles in the macro tile to multiple fragment processing pipelines for rendering in order of the calculated workload weight Weight from large to small also includes: generating a tile rendering list containing Lw sequentially arranged workload levels for each macro tile; classifying each tile in the macro tile to the workload level corresponding to the tile in the Lw workload levels; distributing the tiles in the macro tile to the multiple fragment processing pipelines for rendering in order of the workload levels from high to low.

19. The graphics processing method according to claim 16, wherein: Distributing the tiles in the macro tile to multiple fragment processing pipelines for rendering in descending order of the calculated workload weights Weight also includes: For each macro tile, generate a tile rendering list; All tiles at the highest workload level Lw in the macro tile are placed in the tile rendering list of the macro tile, or tiles at the highest workload level Lw in the macro tile whose number does not exceed the preset value MaxE are placed in the tile rendering list of the macro tile.

20. The graphics processing method according to claim 19, characterized in that: Distributing the tiles in the macro block to multiple fragment processing pipelines for rendering in order of the calculated workload weights Weight from large to small also includes: first distributing the tiles in the tile rendering list of the macro block to multiple fragment processing pipelines for rendering, and then distributing the tiles one by one to the multiple fragment processing pipelines for rendering within the macro block using a fixed pattern; when encountering a tile in the tile rendering list of the macro block using the fixed pattern, directly skipping the tile.

Images