Rendering system, chip, device, and rendering method used in rendering system

Abstract

The present application provides a rendering system, chip, device, and rendering method applied in the rendering system, and relates to the field of image rendering technology. The above-mentioned rendering system includes: a rendering pipeline manager, an intermediate buffer, M geometry processing pipelines, and N pixel processing pipelines; wherein, the multiple data segments divided from the input data stream of the rendering system are assigned to the M geometry processing pipelines for processing, and the metadata obtained after each data segment is processed is stored in the intermediate buffer; the rendering pipeline manager is used to obtain the number and memory information of the data segment processed by the geometry processing pipeline; the rendering pipeline manager is also used to send rendering instructions to the N pixel processing pipelines respectively, and the rendering instructions include the numbers and memory information of k data segments, where k is an integer greater than or equal to 1. When k is greater than 1, the numbers of the k data segments are continuous. The above-mentioned method improves the efficiency of image rendering.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image rendering technology, and in particular to a rendering system, a chip, a device, and a rendering method applied in the rendering system. Background Art

[0002] GPU (Graphics Processing Unit) is used to render images displayed on electronic devices.

[0003] In related technologies, there's a GPU architecture called tile-based rendering (TBR). In TBR, the electronic device screen is divided into multiple tiles, and rendering is performed tile by tile. Specifically, multiple geometry processing pipelines are used to process the input data streams separately. The results are then divided into multiple pieces of metadata based on the tiles. Furthermore, the pixel processing pipeline reads and processes these multiple pieces of metadata to produce the final rendering result.

[0004] However, in the aforementioned related technologies, the pixel processing pipeline must wait for multiple geometry processing pipelines to complete processing before it can process the multiple pieces of metadata. During this process, the geometry processing pipeline that completes processing first must wait for the others to complete before triggering the subsequent rendering process. In other words, multiple geometry processing pipelines must run synchronously. As a result, image rendering latency is significant and efficiency is low. Summary of the Invention

[0005] The embodiments of the present application provide a rendering system, a chip, a device, and a rendering method used in the rendering system. The technical solutions provided by the embodiments of the present application include the following contents.

[0006] According to one aspect of an embodiment of the present application, a rendering system is provided, comprising: a rendering pipeline manager, an intermediate buffer, M geometry processing pipelines, and N pixel processing pipelines, where M is an integer greater than 1 and N is an integer greater than or equal to 1; wherein a plurality of data segments divided from an input data stream of the rendering system are assigned to the M geometry processing pipelines for processing, and primitive data obtained after processing each of the data segments is stored in the intermediate buffer;

[0007] The rendering pipeline manager is configured to obtain the serial number and memory information of the data segment processed by the geometry processing pipeline, wherein the memory information of the data segment is used to determine the memory page occupied by the graphics metadata corresponding to the data segment in the intermediate buffer;

[0008] The rendering pipeline manager is also used to send rendering instructions to the N pixel processing pipelines respectively, and the rendering instructions include the numbers and memory information of k data segments, where k is an integer greater than or equal to 1. When k is greater than 1, the numbers of the k data segments are continuous.

[0009] According to one aspect of an embodiment of the present application, a GPU chip is provided, wherein the GPU chip includes the above-mentioned rendering system.

[0010] According to one aspect of an embodiment of the present application, an electronic device is provided, wherein the electronic device includes a GPU chip, and the GPU chip includes the above-mentioned rendering system.

[0011] According to one aspect of an embodiment of the present application, a rendering method applied to a rendering system is provided, wherein the rendering system includes: a rendering pipeline manager, an intermediate buffer, M geometry processing pipelines and N pixel processing pipelines, where M is an integer greater than 1 and N is an integer greater than or equal to 1; wherein a plurality of data segments divided from an input data stream of the rendering system are assigned to the M geometry processing pipelines for processing, and the graphic metadata obtained after processing each of the data segments is stored in the intermediate buffer; the method includes the following steps.

[0012] The rendering pipeline manager obtains the serial number and memory information of the data segment processed by the geometry processing pipeline, wherein the memory information of the data segment is used to determine the memory page occupied by the graphics metadata corresponding to the data segment in the intermediate buffer;

[0013] The rendering pipeline manager sends rendering instructions to the N pixel processing pipelines respectively, and the rendering instructions include the numbers and memory information of k data segments, where k is an integer greater than or equal to 1. When k is greater than 1, the numbers of the k data segments are continuous.

[0014] The technical solutions provided by the embodiments of the present application may include the following beneficial effects.

[0015] The rendering system proposed in the present application introduces a rendering pipeline manager to uniformly manage the data segments processed by the geometry processing pipeline. Specifically, the rendering pipeline manager obtains the numbers and memory information of the data segments processed by the geometry processing pipeline, and after there are k consecutively numbered data segments, it sends rendering instructions to the N pixel processing pipelines respectively. That is, as long as there are k consecutively numbered data segments, regardless of the processing progress of the geometry processing pipeline, the rendering pipeline manager will send rendering instructions to instruct rendering. Therefore, each geometry processing pipeline does not need to wait for other geometry processing pipelines to synchronize, and the asynchronous operation of multiple geometry processing pipelines can be achieved through the rendering pipeline manager (each geometry processing pipeline continues to process data segments on its own without the need to maintain synchronization). Therefore, in the embodiment of the present application, the image rendering delay is reduced, thereby improving the image rendering efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 is a schematic diagram of a rendering system provided in a possible implementation of the present application;

[0018] Figure 2 is a schematic diagram of the internal processing flow of a GPU core provided in one possible implementation of the present application;

[0019] Figure 3 is a schematic diagram of a rendering system provided in another possible implementation of the present application;

[0020] Figure 4 is a schematic diagram of a queue provided in a possible implementation of the present application;

[0021] Figure 5 This is a schematic diagram of a possible implementation of the present application in which no merged rows exist in the queue;

[0022] Figure 6 This is a schematic diagram of a possible implementation of the present application in which a queue contains merged rows;

[0023] Figure 7 is a schematic diagram of a queue after a merged row is removed, provided in a possible implementation of the present application;

[0024] Figure 8 is a schematic diagram of a queue provided in another possible implementation of the present application;

[0025] Figure 9 This is a schematic diagram of a possible implementation of the present application after a data segment in a queue is removed;

[0026] Figure 10 is a schematic diagram of a rendering system provided in yet another possible implementation of the present application;

[0027] Figure 11 is a schematic diagram of a rendering system provided in yet another possible implementation of the present application;

[0028] Figure 12 is a schematic diagram of a rendering system provided in another possible implementation of the present application;

[0029] Figure 13 is a flowchart of a rendering method applied to a rendering system provided in a possible implementation of the present application;

[0030] Figure 14 This is a simplified structural block diagram of an electronic device provided in a possible implementation of the present application. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0032] Before introducing the rendering system proposed in this application, the image rendering technology involved in the related art is briefly described below.

[0033] In related technologies, GPU architectures can be broadly categorized into two types: Immediate Mode Rendering (IMR) and TBR, also known as tile-based rendering.

[0034] In some embodiments, TBR mainly includes the following parts.

[0035] Tile rendering. TBR divides the screen of an electronic device into several small rectangular blocks, or tiles, and each tile is rendered independently. The GPU first converts all geometric data into screen space coordinates, and then sorts and processes this data according to the tiles it covers. For each tile, the GPU concatenates the metadata and stores it in external memory. For example, the external memory is video memory.

[0036] Tile Buffer. When processing each tile, the GPU stores all necessary geometry and texture data in a small, high-speed cache (typically on-chip). This approach allows the GPU to reduce accesses to external memory, improving efficiency. For example, the tile buffer is also referred to as the intermediate buffer described below.

[0037] Tile-by-tile rendering. The GPU processes each tile in turn until the entire screen is rendered. After each tile is rendered in its own tile buffer, the result is written to the final frame buffer.

[0038] Exemplarily, the main advantages of TBR include the following aspects.

[0039] On the one hand, memory bandwidth is saved. Since the TBR architecture only needs to access local geometry and texture data within each tile, it can significantly reduce the read and write operations to external memory, thereby reducing memory bandwidth requirements.

[0040] On the other hand, energy saving: reducing the demand for memory bandwidth also means lower power consumption, so the TBR architecture is widely used in mobile devices.

[0041] On the other hand, some algorithms can be used to reduce overdraw to reduce the workload of the fragment shader.

[0042] Exemplarily, the main disadvantages of TBR include the following aspects.

[0043] On the one hand, the process is complex. Tiled rendering involves a complex process, including slicing, storing in the processor, and reloading. This is especially true when dealing with advanced graphics effects (such as global illumination and complex transparency processing), which may require more hardware and software support.

[0044] On the other hand, there is a delay. Since TBR needs to process the screen in blocks, the entire rendering process may have a certain delay, especially in scenes that require frequent updates.

[0045] On the other hand, insufficient memory may cause out of memory (OOM) due to the need to output a large amount of data (element linked lists) to the intermediate buffer (i.e. the external memory mentioned above).

[0046] In summary, for TBR, the entire rendering process needs to be broken down into the following two steps. The entire rendering process includes the geometry processing pipeline and the pixel processing pipeline. The geometry processing pipeline performs a series of processing on the data stream, including vertex shaders, tessellators, geometry shaders, viewport transformations, and other operations. Next, the data results are divided into multiple data streams (primitive lists) based on tiles and stored in external memory. After this work is completed, the pixel processing pipeline reads these data streams and performs rasterization and fragment shaders, depth and stencil testing, and color blending output.

[0047] For multi-core systems, if multiple GPU core units are enabled (each core contains a geometry processing pipeline and a pixel processing pipeline), the following two problems will be encountered.

[0048] First, because the input data stream is serial, after passing through multiple geometry processing pipelines, it needs to be reordered before being processed by the pixel processing pipeline. To address this issue, a synchronization unit is typically added at the end of the geometry processing pipeline. However, this can result in performance losses due to waiting between pipelines.

[0049] Secondly, due to the presence of tessellators and geometry shaders, the number of primitives can swell several times. Therefore, it is impossible to estimate the storage space required to store the data stream output by the geometry processing pipeline, so insufficient storage space may occur.

[0050] Based on the above problems, the present application proposes a rendering system. A rendering pipeline manager is introduced in the rendering system to uniformly manage the data segments processed by the geometry processing pipeline. Specifically, the rendering pipeline manager obtains the numbers and memory information of the data segments processed by the geometry processing pipeline, and after there are k consecutively numbered data segments, it sends rendering instructions to the N pixel processing pipelines respectively. That is, as long as there are k consecutively numbered data segments, regardless of the processing progress of the geometry processing pipeline, the rendering pipeline manager will send rendering instructions to instruct rendering. Therefore, each geometry processing pipeline does not need to wait for other geometry processing pipelines to synchronize. Through the rendering pipeline manager, asynchronous operation of multiple geometry processing pipelines can be achieved (each geometry processing pipeline continues to process data segments on its own without the need to maintain synchronization). Therefore, in the embodiments of the present application, the image rendering delay is reduced, thereby improving the image rendering efficiency. Please refer to the explanation of the following embodiments for details.

[0051] Please refer to Figure 1 , which shows a schematic diagram of a rendering system provided in a possible implementation of the present application.

[0052] In some embodiments, as Figure 1As shown, the rendering system includes: a rendering pipeline manager 120, an intermediate buffer 110, M geometry processing pipelines 100 and N pixel processing pipelines 130, where M is an integer greater than 1 and N is an integer greater than or equal to 1; wherein, multiple data segments divided into the input data stream of the rendering system are assigned to the M geometry processing pipelines 100 for processing, and the graphics metadata obtained after processing each data segment is stored in the intermediate buffer 110.

[0053] Exemplarily, the rendering system includes multiple GPU cores. Exemplarily, the internal processing flow of the GPU core is as follows: Figure 2 As shown. For example, Figure 2 As shown, a GPU core includes a geometry processing pipeline 210 , a tile divider 220 , an intermediate buffer 230 , and a pixel processing pipeline 240 .

[0054] Exemplarily, a geometry processing pipeline is a module for processing input data streams or data segments divided based on the input data streams. Exemplarily, the processing of data segments by the geometry processing pipeline includes the following aspects: vertex shader, tessellator, geometry shader, viewport transformation and other operations. Exemplarily, each GPU core includes one or more geometry processing pipelines. The number of geometry processing pipelines in different GPUs is the same or different. In some embodiments, a GPU core includes M geometry processing pipelines, that is, the number of GPU cores corresponding to the M geometry processing pipelines is 1. In other embodiments, a GPU core includes 1 geometry processing pipeline, that is, the number of GPU cores corresponding to the M geometry processing pipelines is M.

[0055] Exemplarily, the pixel processing pipeline is a module for reprocessing the primitive data obtained after processing by the geometry processing pipeline to obtain the final rendering result. Exemplarily, the processing of primitive data by the pixel processing pipeline includes at least one of the following: performing rasterization and fragment shader, depth and template testing, and mixed color output. Exemplarily, each GPU core includes one or more pixel processing pipelines. The number of pixel processing pipelines in different GPUs is the same or different. Exemplarily, the pixel processing pipelines can also be distributed outside the GPU. Exemplarily, each pixel processing pipeline is used to process and implement the rendering of multiple tiles on the screen. In some embodiments, a GPU core includes N pixel processing pipelines, that is, the number of GPU cores corresponding to the N pixel processing pipelines is 1. In other embodiments, a GPU core includes 1 pixel processing pipeline, that is, the number of GPU cores corresponding to the N pixel processing pipelines is N. In some embodiments, M is equal to N.

[0056] Exemplarily, the intermediate buffer is a storage space for storing metadata obtained after processing each data segment. Exemplarily, the intermediate buffer is distributed within the GPU core, or it can be distributed outside the GPU core. Exemplarily, when the intermediate buffer is distributed within the GPU core, each GPU core includes one or more intermediate buffers. Exemplarily, when the intermediate buffer is distributed outside the GPU core, the intermediate buffer is also referred to as external memory or video memory.

[0057] Exemplarily, the metadata is data related to tile rendering. Exemplarily, the metadata includes the following information. ‌Tile color information‌: stored in the tile buffer, used to record the color data of each tile. ‌Depth information‌: stored in the depth buffer (Depth Buffer), used to record the depth information of each tile. ‌Stencil information‌: stored in the stencil buffer (Stencil Buffer), used to record the stencil information of each tile. Exemplarily, the intermediate buffer includes at least one of the above-mentioned tile buffer, depth buffer and stencil buffer.

[0058] In some embodiments, an input data stream is divided into a plurality of data segments. Exemplarily, the plurality of data segments are assigned to M geometry processing pipelines for processing. Exemplarily, each data segment is assigned to one geometry processing pipeline for processing. Exemplarily, no two data segments are identical, no two data segments are different, or no two data segments overlap.

[0059] The embodiment of the present application does not limit the method of dividing the input data stream, and the input data stream is divided according to a preset division rule to obtain multiple data segments. Exemplarily, the input data stream is the original data input to the rendering system for rendering to obtain an image. Exemplarily, the input data stream includes multiple data related to triangular facets, such as vertex data of triangular facets. Exemplarily, vertex data is data provided for the subsequent geometry processing pipeline, including at least one of vertex attributes such as vertex coordinates, texture coordinates, vertex normals and vertex colors.

[0060] In some embodiments, the rendering pipeline manager is used to obtain the number and memory information of the data segment processed by the geometry processing pipeline, and the memory information of the data segment is used to determine the memory page occupied by the graphics metadata corresponding to the data segment in the intermediate buffer. Figure 1 As shown, the rendering pipeline manager 120 is used to obtain the number and memory information of the data segment processed by the geometry processing pipeline. The memory information of the data segment is used to determine the memory page occupied by the graphics metadata corresponding to the data segment in the intermediate buffer.

[0061] Exemplarily, when the input data stream is segmented, the multiple data segments obtained by segmentation are numbered to obtain a number for each data segment. Exemplarily, the data segments are numbered sequentially starting from 0, and each data segment number is a non-negative number. Exemplarily, the numbers of adjacent data segments are also consecutive.

[0062] Exemplarily, data segments are allocated to each of the M geometry processing pipelines in a certain order. Exemplarily, the method for allocating data segments to the M geometry processing pipelines is as follows. Exemplarily, a third value is set. Starting from the first data segment, a third value of data segments are allocated to the first geometry processing pipeline of the M geometry processing pipelines, a third value of data segments are allocated to the second geometry processing pipeline of the M geometry processing pipelines, a third value of data segments are allocated to the third geometry processing pipeline of the M geometry processing pipelines, and so on, until each of the M geometry processing pipelines is allocated a third value of data segments. If there are still unallocated data segments, the third value of data segments are allocated, starting from the first geometry processing pipeline of the M geometry processing pipelines, until all data segments are allocated. Exemplarily, the third value is 1.

[0063] In some embodiments, an input data stream is simultaneously fed into the geometry processing pipelines of multiple GPU cores. Exemplarily, the entire data stream is segmented using an algorithm and processed separately in different geometry processing pipelines. The algorithm may segment the entire data stream according to a certain rule, and these segments are distributed among the geometry processing pipelines of the multiple GPU cores in a round-robin manner. Exemplarily, each segment of the data stream (a data segment) may be assigned a number. For example, in a four-core GPU system (including GPU core 0 (GPU0), GPU core 1 (GPU1), GPU core 2 (GPU2), and GPU core 3 (GPU3)), data segment 0 (0 being the data segment number) will be distributed to the geometry processing pipeline on GPU0, data segment 1 will be distributed to the geometry processing pipeline on GPU1, data segment 2 will be distributed to the geometry processing pipeline on GPU2, data segment 3 will be distributed to the geometry processing pipeline on GPU3, and data segment 4 will be distributed again to the geometry processing pipeline on GPU0, and so on. For example, the pixel processing pipeline needs to re-sort the primitive data stream processed by the geometry processing pipeline that was previously distributed out of order according to the order of the original input data stream.

[0064] This approach can ensure that the number of data segments allocated to each geometry processing pipeline is relatively consistent without too much difference in number, which is conducive to balancing the number of data segments required by each geometry processing pipeline and realizing parallel processing.

[0065] Exemplarily, each data segment is sequentially numbered according to the order in which it is assigned to the geometry processing pipeline for processing. Exemplarily, each geometry processing pipeline processes the data segments one by one in the order in which the data segments are received. Exemplarily, the geometry processing pipeline processes smaller-numbered data segments first, followed by larger-numbered data segments, such as data segments numbered 0, 4, 8, and 12, in sequence. Exemplarily, the geometry processing pipeline processes the assigned data segments sequentially in the order in which the data segments are assigned.

[0066] In some embodiments, the memory information of a data segment is used to determine the memory pages occupied by the metadata corresponding to the data segment in the intermediate buffer. Exemplarily, the memory information of the data segment includes at least one of the number of memory pages, the storage address, etc. Exemplarily, the geometry processing pipeline continuously requests and uses memory pages while processing the data stream.

[0067] Exemplarily, if the metadata is continuously occupied in the memory buffer, the memory information includes the number of memory pages. Exemplarily, the location where the metadata is stored in the intermediate buffer can be determined based on the number of memory pages. Exemplarily, a memory page corresponds to a small storage space in the memory space, and the specific size of the storage space is not limited in this application. Exemplarily, since it is continuously occupied, the storage address of the previously occupied memory page can be determined based on the number of previously occupied memory pages. Furthermore, the storage address of the memory page occupied by the current metadata can be determined based on the number of memory pages occupied by the current metadata. Exemplarily, whenever a data segment is processed, the geometry processing pipeline will send the data segment number and the number of memory pages applied for processing the data segment to the rendering pipeline manager. The rendering pipeline manager is responsible for collecting data sent separately by the geometry pipeline processors of all GPU cores.

[0068] For example, if the metadata is non-continuously occupied in the memory buffer, also known as out-of-order occupation, the memory information includes a storage address. For example, the location where the metadata is stored in the memory buffer can be determined based on the specific storage address.

[0069] This application determines the storage address by the number of memory pages, which can reduce data transmission and data recording costs (for example, while K bits were originally required to record the specific storage address, this application only needs one bit to record the number of memory pages, where K is an integer greater than 1), thereby reducing the amount of data processing during image rendering. Of course, directly recording the specific storage address can improve the accuracy of storage address determination and avoid subsequent storage address calculation errors caused by errors in the previous recording of the number of memory pages.

[0070] In some embodiments, the rendering pipeline manager is further configured to send rendering instructions to each of the N pixel processing pipelines, wherein the rendering instructions include the numbers and memory information of k data segments, where k is an integer greater than or equal to 1. When k is greater than 1, the numbers of the k data segments are continuous. For example, Figure 1 As shown, the rendering pipeline manager 120 is also used to send rendering instructions to N pixel processing pipelines 130 respectively. The rendering instructions include the numbers and memory information of k data segments, where k is an integer greater than or equal to 1. When k is greater than 1, the numbers of the k data segments are continuous.

[0071] In some embodiments, because the pixel processing pipeline requires order-preserving rendering of the input data stream (color blending and depth testing, etc.), the data in the workflow fed into the pixel processing pipeline must be sequence-preserving. Alternatively, it can be understood that the data segments containing the primitive data fed into the pixel processing pipeline for rendering must be continuous. Therefore, although different geometry processing pipelines process different data segments and different geometry processing pipelines run at different speeds, by assigning a data segment number to each data segment, the pixel processing pipeline can obtain the data segment number and reorder them for processing.

[0072] In some embodiments, since the operation speed of each geometry processing pipeline is different, when the rendering pipeline manager triggers the pixel processing pipeline workflow, the data segments in the current queue may not be continuous. Figure 4 As shown, data segments numbered 0, 1, 2, 3, 4, 5, 7, and 8 have been completed, but the others have not yet been completed. For example, if all of these are packaged and sent to the pixel processing pipeline, an error will occur. This is because the data segments numbered 0, 1, 2, 3, 4, 5, 7, and 8 are not continuous intervals, and there is a "hole" in the middle with data segment numbered 6, which does not comply with the order preservation principle. Therefore, when the pixel processing pipeline workflow is triggered once (that is, when a rendering instruction is sent), consecutive data segment numbers are selected as the pixel processing pipeline workflow (as the numbers of the data segments submitted to the pixel processing pipeline for processing). For example, in the above example, data segments numbered 0, 1, 2, 3, 4, and 5 can be selected for submission to the pixel processing pipeline, while data segments numbered 7 and 8 will not be included in this pixel processing pipeline workflow. For example, in the above example, a consecutive k of the data segments numbered 0, 1, 2, 3, 4, and 5 can be selected for submission to the pixel processing pipeline.

[0073] Exemplarily, the rendering pipeline manager obtains the numbers of the data segments processed by the geometry processing pipeline. If there are k consecutive data segment numbers, the rendering pipeline manager is further configured to send rendering instructions to each of the N pixel processing pipelines. Exemplarily, the k data segments correspond to the workflow of the triggered pixel processing pipeline.

[0074] Exemplarily, the rendering instruction is used to instruct the pixel processing pipeline to perform a rendering process based on the processed primitives corresponding to the k data segments. Exemplarily, the rendering pipeline manager sends the rendering instruction to each of the N pixel processing pipelines.

[0075] Exemplarily, each of the N pixel processing pipelines obtains the numbers and memory information of k data segments based on the rendering instruction. Exemplarily, each pixel processing pipeline obtains, from the corresponding memory page, the metadata corresponding to the k data segments based on the memory information of the k data segments. Furthermore, the rendering process corresponding to the pixel processing pipeline is executed based on the metadata corresponding to the k data segments.

[0076] Exemplarily, this application does not limit the specific value of k. Exemplarily, k is a preset value. Exemplarily, k is the number of GPU cores included in the rendering system, such as 4. Exemplarily, the numbers of the data segments processed by the geometry processing pipeline obtained by the rendering pipeline manager include numbers 0 to 10, and numbers 0 to 3 are the numbers of the data segments submitted to N pixel processing pipelines for processing, then numbers 4 to 7 are used as the k data segments indicated by the rendering instruction sent next time. That is, the number of data segments indicated by each rendering instruction is the same. Of course, the number of data segments indicated by each rendering instruction may also be different.

[0077] In some embodiments, as Figure 3 As shown, the rendering system's input data stream is divided into multiple data segments, which are then assigned to M geometry processing pipelines (including geometry processing pipeline 310) for processing. The resulting primitive data from each data segment is stored in an intermediate buffer (e.g., intermediate buffer 0 320 corresponding to geometry processing pipeline 310). The rendering pipeline manager 330 is used to obtain the numbers and memory information of the data segments processed by the geometry processing pipelines (including geometry processing pipeline 310). This data segment memory information is used to determine the memory pages occupied by the primitive data corresponding to the data segment in the intermediate buffer. The rendering pipeline manager 330 is also used to send rendering instructions to each of the N pixel processing pipelines (including pixel processing pipeline 340). The rendering instructions include the numbers and memory information of k data segments, where k is an integer greater than or equal to 1. When k is greater than 1, the k data segments are consecutively numbered.

[0078] The rendering system proposed in the present application introduces a rendering pipeline manager to uniformly manage the data segments processed by the geometry processing pipeline. Specifically, the rendering pipeline manager obtains the numbers and memory information of the data segments processed by the geometry processing pipeline, and after there are k consecutively numbered data segments, it sends rendering instructions to the N pixel processing pipelines respectively. That is, as long as there are k consecutively numbered data segments, regardless of the processing progress of the geometry processing pipeline, the rendering pipeline manager will send rendering instructions to instruct rendering. Therefore, each geometry processing pipeline does not need to wait for other geometry processing pipelines to synchronize, and the asynchronous operation of multiple geometry processing pipelines can be achieved through the rendering pipeline manager (each geometry processing pipeline continues to process data segments on its own without the need to maintain synchronization). Therefore, in the embodiment of the present application, the image rendering delay is reduced, thereby improving the image rendering efficiency.

[0079] In some embodiments, the rendering pipeline manager is further configured to select one or more data segments numbered between a first value and a second value from the data segments processed by the geometry processing pipeline as the k data segments.

[0080] In some embodiments, the first value is the maximum number of data segments submitted to N pixel processing pipelines plus 1, and the second value is the maximum number of consecutive numbers of k data segments.

[0081] For example, for a data segment that has been submitted to N pixel processing pipelines for processing, the rendering pipeline manager will not submit the number or memory information of the data segment again.

[0082] For example, if numbers 0 to 4 are numbers of data segments submitted to N pixel processing pipelines for processing, then the first value is the maximum number 4 of the data segments submitted to N pixel processing pipelines for processing plus 1, that is, the first value is 5.

[0083] Exemplarily, the numbers of the data segments processed by the geometry processing pipeline obtained by the rendering pipeline manager include numbers 0-10 and numbers 12-13, where numbers 0-4 are the numbers of the data segments submitted to N pixel processing pipelines for processing. The first value is 5, and the second value is 10, where the second value is the maximum number that satisfies the continuous numbering of k data segments. Starting with data segment number 5, numbers 5 to 10 are consecutive data segments that have been processed. Exemplarily, there are 6 consecutive data segments between numbers 5 and 10, so one or more data segments with numbers between the first value 5 and the second value 10 are selected as the k data segments.

[0084] For example, when the pixel processing pipeline workflow is triggered, the largest consecutive data segment number is selected as the pixel processing pipeline workflow. For example, in the above example, data segments numbered 0, 1, 2, 3, 4, and 5 are selected to be submitted to the pixel processing pipeline, while data segments numbered 7 and 8 are not included in this pixel processing pipeline workflow.

[0085] For example, since data segment number 11 has not been processed yet, data segment number 10 and data segment number 12 are not continuous. Therefore, even if data segment number 12 is processed, it is not considered as one of the k data segments.

[0086] The technical solution provided in the embodiment of the present application sends rendering instructions to the rendering pipeline manager using the data segments with the most consecutive numbers (also called the maximum consecutive data segment numbers) from the data segments processed by the geometry processing pipeline as k data segments. There is no need to pause the geometry processing pipeline to wait for other geometry processing pipelines to synchronize, thereby realizing asynchronous operation of the geometry processing pipeline and improving GPU operation performance.

[0087] In addition, it enables rendering of as many consecutive data segments as possible at once, which helps speed up rendering and improve graphics rendering efficiency. In addition, it avoids retransmitting data segments that have already been submitted to the pixel processing pipeline, which helps avoid duplication and reduces data transmission and processing costs.

[0088] In some embodiments, M queues are provided in the rendering pipeline manager, and each queue is used to store the number and memory information of a data segment processed by the geometry processing pipeline.

[0089] Exemplarily, there is a one-to-one correspondence between the M queues and the M geometry processing pipelines, with one queue corresponding to one geometry processing pipeline. For example, in a system with four GPU cores, the rendering pipeline manager has four queues for storing the numbers and memory information of processed data segments sent from the geometry processing pipelines of the four GPU cores (e.g., the numbers of the processed data segments and the memory information of the metadata corresponding to each numbered data segment).

[0090] Exemplarily, each queue includes multiple grids, one grid is used to store the number of a completed data segment and the memory information of the metadata corresponding to the data segment (such as the number of memory pages or page number). Exemplarily, M is 4, and four queues correspond to four GPU cores. Exemplarily, at a certain moment, the queues in the rendering pipeline manager are as follows: Figure 4As shown in the figure, the gray blocks are the information that has been received at the current moment, while the white blocks are the information that has not been received at the current moment. It can be seen that the running speeds of different GPU cores are different, that is, the data segments processed by different geometry processing pipelines are different, so the amount of data received by each queue is also different. Figure 4 As shown, in the first queue (i.e., queue 410 corresponding to GPU core 0), the data segments that have been processed by the geometry processing pipeline include data segments 0, 4, and 8. Figure 4 As shown, in the second queue (that is, the queue corresponding to GPU core 1), the data segments that have been processed by the geometry processing pipeline include data segments 1 and 5. Figure 4 As shown, in the third queue (i.e., the queue corresponding to GPU core 2), the data segments that have been processed by the geometry processing pipeline include data segment 2. Figure 4 As shown, in the fourth queue (that is, the queue corresponding to GPU core 3), the data segments that have been processed by the geometry processing pipeline include data segments 3 and 7.

[0091] The technical solution provided by this embodiment uses M queues to record the data sent by each of the M geometry processing pipelines. This allows the rendering pipeline manager to have a clear understanding of the operation status of the M geometry processing pipelines (including which data segments have been processed and the memory information of the corresponding metadata). This facilitates triggering the execution of subsequent rendering instruction sending steps and improves the processing efficiency of the rendering pipeline manager.

[0092] In some embodiments, the i-th queue among the M queues is used to store the numbers and memory information of data segments processed by the i-th geometry processing pipeline among the M geometry processing pipelines, where i is a positive integer less than or equal to M.

[0093] In some embodiments, the rendering pipeline manager is further configured to add the serial number and memory information of the first data segment to the end of the i-th queue after the i-th geometry processing pipeline completes processing the first data segment.

[0094] For example, for the i-th queue, the numbers and memory information of the completed data segments are added to the i-th queue in the order in which the data segments are completed. For example, the order in which the numbers and memory information of the processed data segments in the i-th queue correspond one-to-one to the order in which the data segments are completed.

[0095] In some embodiments, the rendering pipeline manager is further configured to remove the number and memory information of the second data segment from the i-th queue if the second data segment stored in the i-th queue meets the conditions for being submitted to the pixel processing pipeline.

[0096] Exemplarily, the condition for being submitted to the pixel processing pipeline is a preset submission condition. Exemplarily, the condition for being submitted to the pixel processing pipeline is a condition for submitting a data segment. Exemplarily, when the second data segment stored in the i-th queue is one of the k data segments, the second data segment is considered to meet the condition for being submitted to the pixel processing pipeline. Exemplarily, when the second data segment stored in the i-th queue is a data segment numbered between a first value and a second value, the second data segment is considered to meet the condition for being submitted to the pixel processing pipeline.

[0097] Exemplarily, when the number and memory information of the second data segment stored in the i-th queue are moved out of the i-th queue, the number and memory information of the next data segment after the second data segment stored in the i-th queue occupy the storage position corresponding to the number and memory information of the second data segment in the original i-th queue.

[0098] According to the technical solution provided by the embodiment of the present application, when the second data segment will be or has been submitted to the pixel processing pipeline for processing, the rendering instruction already carries the number and memory information of the second data segment. Therefore, the number and memory information of the second data segment are removed from the i-th queue. Under the premise of not affecting the image rendering, the number and memory information of the submitted data segment are avoided as much as possible from occupying too much memory space, which is conducive to saving storage space.

[0099] In some embodiments, the rendering pipeline manager is further used to remove the numbers and memory information of the M data segments included in the merged row from the M queues when there is a merged row in the M queues; wherein the merged row includes the number and memory information of a data segment with the smallest number stored in each queue in the M queues, and the numbers of the M data segments included in the merged row are continuous.

[0100] Exemplarily, for the number and memory information of a data segment with the smallest number stored in each of the M queues, if the numbers of a data segment with the smallest number stored in each of the M queues are continuous, then the data segment with the smallest number stored in each of the M queues is used as the data segment in the merged row.

[0101] Exemplarily, M is 4. Exemplarily, queue 1 includes numbers 0 and 4, queue 2 includes number 1, queue 3 includes number 2, and queue 4 includes number 3. Then, the smallest data segment in queue 1 is number 0, the smallest data segment in queue 2 is number 1, the smallest data segment in queue 3 is number 2, and the smallest data segment in queue 4 is number 3. Since numbers 0, 1, 2, and 3 are consecutive, numbers 0, 1, 2, and 3 are used as the numbers in the merged row.

[0102] For example, Figure 4 As shown, the data segments in the first row of the four queues are all completed and are numbered consecutively. Then all elements in the first row are merged, and each element corresponds to the number of a data segment and the memory information of the data segment (including the number of pages).

[0103] Exemplarily, a cumulative value is set for each queue, with an initial value of 0. Exemplarily, the cumulative value of each queue is used to indicate the data segments in that queue that have been merged or the data segments that have been submitted to the pixel processing pipeline (including the number of pages). Exemplarily, when all positions in the merged row have an element, the first element of each queue can be popped out (i.e., removed from the queue), and the number of pages is added to the corresponding cumulative value, and so on. In some embodiments, the rendering pipeline manager also records the number of the data segments in this row and merges them into the maximum consecutive data segment number.

[0104] For example, Figure 5 As shown, at time t0, there is no processed data segment in the queue 510 where the GPU core 2 is located, so there is no merge row (the first row lacks the processed data segment in the queue 510), and removal is not triggered.

[0105] For example, Figure 6 As shown, at time t2, if there are processed data segments in queue 610 where GPU core 2 is located (the first row contains the processed data segments in queue 610), a merge row is present, triggering removal. For example, the cumulative values ​​of each queue at this time are: GPU0: page number A, GPU1: page number B, GPU2: page number C, GPU3: page number D, and the maximum contiguous data segment number is [0:3]. Page number A indicates the number of memory pages for the metadata corresponding to data segment 0, and so on. A-D are positive integers.

[0106] For example, based on Figure 6 After removing the merged rows from the M queues, the result is as follows Figure 7 As shown. For example, Figure 7 As shown, after data segment 0 and page number A are removed from queue 710 where GPU core 0 is located, data segment 4 and page number E in the second grid are moved up one grid. The same applies to other queues.

[0107] For example, as the GPU continues to run, that is, the geometry processing pipeline continues to run, the queue results are as follows: Figure 8As shown. Assuming that at this time, the rendering pipeline manager triggers a pixel processing pipeline workflow, that is, sends a rendering instruction, it will first find the maximum continuous data segment number. Since the continuous number has been found once in the previous round of merged rows, that is, the number of the merged row is [0:3], two data segments that can be spliced ​​with the results found in the previous round can be found in the queue at this time, thus becoming the new maximum continuous data segment. These two data segments, that is, the data segment 4 recorded in the first element 810 in the queue where GPU core 0 is located and the data segment 5 recorded in the first element 820 in the queue where GPU core 1 is located, the result after merging is [0:5]. Then, at this time, the rendering pipeline manager accumulates the page numbers of data segments 4 and 5 into their respective cumulative values, and then removes them from their respective queues, obtaining the following Figure 9 The four queues shown. Data segments 4 and 5 are not included in the four queues. In some embodiments, the cumulative values ​​of each queue at this time are GPU0: page number A+E, GPU1: page number B+F, GPU2: page number C, GPU3: page number D, and the maximum continuous data segment number is [0:5]. Exemplarily, the rendering pipeline manager will initiate a pixel processing pipeline workflow (i.e., send a rendering instruction) and also send the content of this rendering data segment to each pixel processing pipeline, that is, send the maximum continuous data segment number [0:5] to each pixel processing pipeline. The elements here are the grids mentioned above.

[0108] In some embodiments, the rendering pipeline manager is further used to record the pending information corresponding to each of the M queues; wherein, for the i-th queue among the M queues, the pending information corresponding to the i-th queue is used to determine the data segments that have been moved out of the i-th queue and have not been submitted to the pixel processing pipeline.

[0109] Exemplarily, the information to be processed includes the cumulative value and the consecutive data segment numbers. Exemplarily, the cumulative value for each queue indicates the sum of the number of pages corresponding to the data segments removed from the queue. Exemplarily, the consecutive data segment numbers indicate the data segments that have been removed from the i-th queue and have not been submitted to the pixel processing pipeline.

[0110] The technical solution provided in the embodiments of this application takes into account the fixed length of each queue. Therefore, when the merge conditions are met, the numbers and memory information of the M data segments included in the merged row are removed from the M queues. This prevents the queues from being overloaded with data and unable to accommodate new data, thus achieving the proper allocation of memory resources. Furthermore, pending information is used to indicate data segments that have been removed from the i-th queue and not submitted to the pixel processing pipeline, prompting the rendering pipeline manager to submit these data segments, thus avoiding missed submissions and ensuring accurate image rendering.

[0111] In some embodiments, the rendering pipeline manager is further configured to execute the step of sending rendering instructions to each of the N pixel processing pipelines when the memory occupancy rate corresponding to any one of the geometry processing pipelines is greater than or equal to a set threshold.

[0112] Illustratively, the memory usage rate corresponding to the geometry processing pipeline is the ratio of the number of memory pages occupied in the intermediate buffer by the metadata corresponding to the data segments processed by the geometry processing pipeline and not submitted to the pixel processing pipeline, to the total number of memory pages included in the intermediate buffer. Illustratively, a certain amount of memory space is pre-allocated for each GPU core where the geometry processing pipeline resides. Illustratively, this allocated memory space serves as the intermediate buffer.

[0113] Exemplarily, different geometry processing pipelines have different corresponding memory occupancy rates at different processing stages. Exemplarily, when a memory occupancy rate corresponding to a geometry processing pipeline is greater than or equal to a set threshold, the step of sending rendering instructions to each of the N pixel processing pipelines is performed.

[0114] For example, the intermediate buffer contains S memory pages, where S is a positive integer and the threshold is set to a%. If the number of memory pages occupied by any geometry processing pipeline reaches S×a%, the rendering pipeline manager needs to initiate a pixel processing pipeline workflow, that is, send a rendering instruction.

[0115] For example, during the pixel processing pipeline processing, even if the memory occupancy rate corresponding to any geometry processing pipeline is again greater than or equal to the set threshold, the rendering pipeline manager must wait until the pixel processing pipeline completes processing of the rendering instruction sent last time before sending the rendering instruction again.

[0116] In some embodiments, when the memory occupancy rate corresponding to any geometry processing pipeline is greater than or equal to a set threshold, the geometry processing pipeline does not stop working. That is, the sending of rendering instructions and the triggering of the rendering process of the pixel processing pipeline are also asynchronous with the geometry processing pipeline, thereby achieving partial decoupling of the dependencies between the two rendering pipelines (including the geometry processing pipeline and the pixel processing pipeline).

[0117] It is taken into consideration that if multiple cores wait for each other at a fixed position, a deadlock problem may occur due to insufficient memory space. Specifically, the deadlock problem refers to the situation where the geometry processing pipeline of a GPU core runs out of space before running to the synchronization point (that is, the memory occupancy rate reaches 100%, but it still does not run to the fixed position), and other cores will never wait for the signal that the core has run to the synchronization point. The technical solution provided in the embodiment of the present application sends a rendering instruction when the memory occupancy rate corresponding to any geometry processing pipeline is greater than or equal to a set threshold, so as to avoid the occurrence of a deadlock problem. The set threshold proposed in the present application ensures that the rendering instruction is triggered before the memory is exhausted, thereby avoiding the deadlock problem caused by insufficient memory.

[0118] In some embodiments, the j-th pixel processing pipeline among N pixel processing pipelines is used to obtain the graphics metadata corresponding to k data segments from the intermediate buffer according to the rendering instruction, and render the graphics metadata corresponding to the k data segments. After the rendering of the graphics metadata corresponding to the k data segments is completed, a completion indication message is sent to the rendering pipeline manager. The completion indication message is used to indicate that the j-th pixel processing pipeline has completed the rendering of the graphics metadata corresponding to the k data segments, where j is a positive integer less than or equal to N.

[0119] Exemplarily, the completion indication information is used to indicate that the j-th pixel processing pipeline has completed rendering of the graphics metadata corresponding to k data segments. Exemplarily, each pixel processing pipeline sends completion indication information to the rendering pipeline manager. When all N pixel processing pipelines send completion indication information, that is, when the rendering pipeline manager receives completion indication information respectively sent from the N pixel processing pipelines, it is considered that all pixel processing pipelines have completed rendering of the graphics metadata for the k data segments.

[0120] In some embodiments, the rendering pipeline manager is further used to release the memory pages occupied by the graphics metadata corresponding to the k data segments in the intermediate buffer after the N pixel processing pipelines have completed rendering of the graphics metadata corresponding to the k data segments.

[0121] For example, after the pixel processing pipeline completes rendering of k data segments, it will send a rendering completion signal (i.e., completion indication information) to the rendering pipeline manager. The rendering pipeline manager will release the memory pages occupied by the metadata corresponding to the k data segments in the intermediate buffer.

[0122] For example, as shown in the above embodiment, after the pixel processing pipeline completes rendering of data segments 0-5, it will respond with a rendering completion signal (i.e., completion indication information) to the rendering pipeline manager. The rendering pipeline manager will then release the memory space of each GPU core. The number of pages released is the cumulative value of the queues corresponding to each core previously recorded, that is, the cumulative number of pages of the data segment removed.

[0123] The technical solution provided by the embodiments of this application reclaims the freed memory pages occupied by the metadata corresponding to the rendered data segments in the intermediate buffer and allocates them to the metadata corresponding to the subsequent data segments to be processed. This not only avoids the occurrence of Out-of-Memory (OOM) issues but also enables dynamic and reasonable memory allocation and recycling.

[0124] In some embodiments, the rendering system further includes: M tile dividers.

[0125] In some embodiments, the i-th geometry processing pipeline among M geometry processing pipelines is used to process the data segment assigned to the i-th geometry processing pipeline among multiple data segments divided into the input data stream to obtain a processed data segment, where i is a positive integer less than or equal to M.

[0126] Exemplarily, one geometry processing pipeline corresponds to one tiler. Exemplarily, each GPU core includes one or more tilers.

[0127] In some embodiments, the i-th tile divider among the M tile dividers is used to tile the processed data segment, obtain metadata corresponding to the data segment, and store the metadata corresponding to the data segment in an intermediate buffer.

[0128] Exemplarily, the tile divider is used to divide the screen of the electronic device into a plurality of tiles. Exemplarily, each tile is a rectangular area. Exemplarily, each tile has the same area. Exemplarily, each tile may have different areas.

[0129] Exemplarily, the i-th tile divider among the M tile dividers is configured to divide the processed data segment according to tiles to obtain metadata corresponding to the data segment.

[0130] Exemplarily, the tile divider further stores the metadata corresponding to the data segment in an intermediate buffer.

[0131] The technical solution provided by the embodiment of the present application uses a tile divider to divide the data segments processed by the geometry processing pipeline to obtain the corresponding metadata of the data segments. This is conducive to achieving tile division and thus realizing the subsequent rendering process based on the pixel processing pipeline.

[0132] In some embodiments, the i-th geometry processing pipeline is further configured to suspend processing new data segments when the number of memory pages occupied in the intermediate buffer by the graphics metadata corresponding to the data segments that have been processed by the i-th geometry processing pipeline and not submitted to the pixel processing pipeline reaches the maximum usage number allocated to the i-th geometry processing pipeline.

[0133] For example, if the number of memory pages occupied by the metadata corresponding to the data segment that has been processed by the i-th geometry processing pipeline and has not been submitted to the pixel processing pipeline in the intermediate buffer reaches the maximum usage number allocated to the i-th geometry processing pipeline, the processing of new data segments is suspended.

[0134] For example, after reaching a set occupancy threshold, the pixel rendering workflow is triggered. At this point, the geometry processing pipeline of each core can continue to operate, reporting the segment number and the number of pages it occupies to the rendering pipeline manager each time a segment is processed. For example, only when all pre-allocated pages within a core have been used up and no pages have been released will the geometry processing pipeline stop and wait for page releases.

[0135] For example, if the number of memory pages occupied by the metadata corresponding to the data segment that has been processed by the i-th geometry processing pipeline and has not been submitted to the pixel processing pipeline in the intermediate buffer is the maximum usage number allocated to the i-th geometry processing pipeline, that is, the memory space allocated to the i-th geometry processing pipeline is full, the processing of new data segments is suspended.

[0136] This application considers that there is no extra memory space to store the metadata corresponding to the data segments to be processed later. Therefore, the processing of new data segments is suspended to avoid the situation where there is no memory space to store them. This helps to ensure the smooth operation of the image rendering process and reduce additional processing overhead.

[0137] In some embodiments, the rendering system includes multiple GPU cores (or core units), and each GPU core includes at least one geometry processing pipeline and at least one pixel processing pipeline.

[0138] Exemplarily, each GPU core includes a geometry processing pipeline and a pixel processing pipeline. Exemplarily, each GPU core also includes a tile divider and an intermediate buffer.

[0139] Exemplarily, each GPU core includes multiple geometry processing pipelines and multiple pixel processing pipelines. Exemplarily, each GPU core also includes multiple tile dividers and multiple intermediate buffers.

[0140] Exemplarily, the geometry processing pipeline in each GPU core is used to process the data segments assigned to the GPU core. Exemplarily, the geometry processing pipeline in each GPU core not only needs to process the metadata processed by itself, but also needs to process the metadata processed by other GPU cores to achieve tile rendering.

[0141] The technical solution provided in the embodiment of the present application configures at least one geometry processing pipeline and at least one pixel processing pipeline for each GPU core, which is conducive to parallel processing of at least one geometry processing pipeline and at least one pixel processing pipeline through multiple GPU cores, thereby accelerating the efficiency of image rendering.

[0142] In some embodiments, as Figure 10 As shown, it shows a schematic diagram of a rendering system provided in another possible implementation of the present application. Figure 10 As shown, each GPU core 1010 includes a geometry processing pipeline, a tile splitter, an intermediate buffer, and a pixel processing pipeline. Exemplarily, a rendering pipeline manager 1020 in the rendering system is located outside the GPU core 1010.

[0143] In some embodiments, as Figure 11 As shown, it shows a schematic diagram of a rendering system provided in another possible implementation of the present application. Figure 11 As shown, each GPU core 1110 includes a geometry processing pipeline, a tile divider, an intermediate buffer, a pixel processing pipeline, and a rendering pipeline manager. Exemplarily, the rendering pipeline manager is located inside the GPU core 1110. Exemplarily, during each rendering process, when there are multiple GPU cores and rendering pipeline managers located in different GPU cores, only one of the rendering pipeline managers is called. Exemplarily, no matter how many GPU cores there are, only the rendering pipeline manager in the GPU core 1110 is used to trigger the workflow of the pixel processing pipeline. Of course, even if only the rendering pipeline manager in one GPU core is used to trigger the workflow of the pixel processing pipeline, it is also necessary to trigger the workflows of the pixel processing pipelines in different GPU cores separately to achieve the purpose of rendering.

[0144] In some embodiments, as Figure 12 As shown, it shows a schematic diagram of a rendering system provided in another possible implementation of the present application. Figure 12As shown, each GPU core 1210 includes a geometry processing pipeline, a tile splitter, and a pixel processing pipeline. Exemplarily, a rendering pipeline manager 1230 in the rendering system is located outside the GPU core 1210. Exemplarily, an intermediate buffer 1220 in the rendering system is located outside the GPU core 1210. In this case, different GPU cores can share different memory spaces in the same intermediate buffer 1220.

[0145] For example, when the rendering pipeline manager or the intermediate buffer is located outside the GPU core, the number of the rendering pipeline manager or the intermediate buffer may be one or more.

[0146] The technical solutions provided by the embodiments of the present application will be described in detail below using a method embodiment. For matters not described in the method embodiment, reference can be made to the above embodiments and will not be repeated here.

[0147] Please refer to Figure 13 , which shows a flowchart of a rendering method applied to a rendering system provided in a possible implementation of the present application. The execution subject of each step of the method may be Figure 1 The rendering pipeline manager in the rendering system shown in FIG. The method may include at least one of the following steps ( 1310 - 1320 ).

[0148] In some embodiments, the rendering system includes: a rendering pipeline manager, an intermediate buffer, M geometry processing pipelines and N pixel processing pipelines, where M is an integer greater than 1 and N is an integer greater than or equal to 1; wherein, multiple data segments divided from the input data stream of the rendering system are assigned to the M geometry processing pipelines for processing, and the graphics metadata obtained after processing each data segment is stored in the intermediate buffer.

[0149] In step 1310 , the rendering pipeline manager obtains the serial number and memory information of the data segment processed by the geometry processing pipeline. The memory information of the data segment is used to determine the memory page occupied by the primitive data corresponding to the data segment in the intermediate buffer.

[0150] In step 1320, the rendering pipeline manager sends rendering instructions to each of the N pixel processing pipelines. The rendering instructions include the numbers and memory information of k data segments, where k is an integer greater than or equal to 1. When k is greater than 1, the numbers of the k data segments are continuous.

[0151] In some embodiments, the rendering pipeline manager selects one or more data segments numbered between a first value and a second value from the data segments processed by the geometry processing pipeline as k data segments; wherein the first value is the maximum number of the data segments submitted to N pixel processing pipelines for processing plus 1, and the second value is the maximum number that satisfies the continuous numbering of the k data segments.

[0152] In some embodiments, M queues are provided in the rendering pipeline manager, and each queue is used to store the number and memory information of a data segment processed by the geometry processing pipeline.

[0153] In some embodiments, the i-th queue among the M queues is used to store the numbers and memory information of data segments processed by the i-th geometry processing pipeline among the M geometry processing pipelines, where i is a positive integer less than or equal to M.

[0154] After the i-th geometry processing pipeline completes processing the first data segment, the rendering pipeline manager adds the number and memory information of the first data segment to the end of the i-th queue.

[0155] When the second data segment stored in the i-th queue meets the condition of being submitted to the pixel processing pipeline, the rendering pipeline manager removes the number and memory information of the second data segment from the i-th queue.

[0156] In some embodiments, when there is a merged row in M ​​queues, the rendering pipeline manager removes the numbers and memory information of the M data segments included in the merged row from the M queues; wherein the merged row includes the number and memory information of a data segment with the smallest number stored in each queue in the M queues, and the numbers of the M data segments included in the merged row are continuous.

[0157] In some embodiments, the rendering pipeline manager records the pending information corresponding to M queues respectively; wherein, for the i-th queue among the M queues, the pending information corresponding to the i-th queue is used to determine the data segments that have been moved out of the i-th queue and have not been submitted to the pixel processing pipeline.

[0158] In some embodiments, when the memory occupancy rate corresponding to any one of the geometry processing pipelines is greater than or equal to a set threshold, the rendering pipeline manager executes the step of sending rendering instructions to each of the N pixel processing pipelines.

[0159] The memory occupancy rate corresponding to the geometry processing pipeline is the ratio of the number of memory pages occupied by the graphics metadata corresponding to the data segments that have been processed by the geometry processing pipeline and have not been submitted to the pixel processing pipeline in the intermediate buffer to the total number of memory pages included in the intermediate buffer.

[0160] In some embodiments, the j-th pixel processing pipeline among N pixel processing pipelines obtains the graphics metadata corresponding to k data segments from the intermediate buffer according to the rendering instruction, and renders the graphics metadata corresponding to the k data segments. After the rendering of the graphics metadata corresponding to the k data segments is completed, a completion indication message is sent to the rendering pipeline manager. The completion indication message is used to indicate that the j-th pixel processing pipeline has completed the rendering of the graphics metadata corresponding to the k data segments, where j is a positive integer less than or equal to N.

[0161] In some embodiments, after the N pixel processing pipelines have completed rendering of the graphics metadata respectively corresponding to the k data segments, the rendering pipeline manager releases the memory pages occupied by the graphics metadata respectively corresponding to the k data segments in the intermediate buffer.

[0162] In some embodiments, the rendering system further includes: M tile dividers.

[0163] In some embodiments, the i-th geometry processing pipeline among M geometry processing pipelines processes the data segment assigned to the i-th geometry processing pipeline among multiple data segments divided into the input data stream to obtain a processed data segment, where i is a positive integer less than or equal to M.

[0164] In some embodiments, the i-th tile divider among the M tile dividers performs tile division on the processed data segment to obtain metadata corresponding to the data segment, and stores the metadata corresponding to the data segment in an intermediate buffer.

[0165] In some embodiments, the i-th geometry processing pipeline suspends processing new data segments when the number of memory pages occupied in the intermediate buffer by the graphics metadata corresponding to the data segments that have been processed by the i-th geometry processing pipeline and not submitted to the pixel processing pipeline reaches the maximum usage number allocated to the i-th geometry processing pipeline.

[0166] In some embodiments, the rendering system includes a plurality of GPU cores, each GPU core including at least one geometry processing pipeline and at least one pixel processing pipeline.

[0167] The rendering system proposed in the present application introduces a rendering pipeline manager to uniformly manage the data segments processed by the geometry processing pipeline. Specifically, the rendering pipeline manager obtains the numbers and memory information of the data segments processed by the geometry processing pipeline, and after there are k consecutively numbered data segments, it sends rendering instructions to the N pixel processing pipelines respectively. That is, as long as there are k consecutively numbered data segments, regardless of the processing progress of the geometry processing pipeline, the rendering pipeline manager will send rendering instructions to instruct rendering. Therefore, each geometry processing pipeline does not need to wait for other geometry processing pipelines to synchronize, and the asynchronous operation of multiple geometry processing pipelines can be achieved through the rendering pipeline manager (each geometry processing pipeline continues to process data segments on its own without the need to maintain synchronization). Therefore, in the embodiment of the present application, the image rendering delay is reduced, thereby improving the image rendering efficiency.

[0168] In some embodiments, a GPU chip is also provided, which includes the above-mentioned rendering system.

[0169] Please refer to Figure 14 , which is a simplified structural block diagram of an electronic device provided in a possible implementation of this application. Figure 14 As shown, the electronic device 1400 includes the aforementioned GPU chip, which includes the aforementioned rendering system. The electronic device 1400 can be used to implement the rendering method provided in the aforementioned embodiment and applied to the rendering system.

[0170] Typically, the electronic device 1400 includes a processor 1401 and a memory 1402 .

[0171] Processor 1401 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 1401 may be implemented in hardware using at least one of the following: a DSP (Digital Signal Processing), an FPGA (Field Programmable Gate Array), or a PLA (Programmable Logic Array). Processor 1401 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), processes data while awake, while the coprocessor is a low-power processor that processes data while in standby mode. In some embodiments, processor 1401 may be integrated with a GPU, which is responsible for rendering and drawing content displayed on the display. In some embodiments, processor 1401 may also include an AI (Artificial Intelligence) processor for handling computational operations related to machine learning.

[0172] Memory 1402 may include one or more computer-readable storage media, which may be non-transitory. Memory 1402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory storage devices. In some embodiments, the non-transitory computer-readable storage media in memory 1402 is used to store a computer program, which is configured to be executed by one or more processors to implement the above-mentioned rendering method applied to the rendering system.

[0173] Those skilled in the art will understand that Figure 14 The structure shown in the figure does not constitute a limitation on the electronic device 1400, and the electronic device 1400 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0174] In some embodiments, the electronic device 1400 can be a server, a server cluster, an artificial intelligence computing cluster, a cloud computing cluster, etc., where the artificial intelligence computing cluster can also be referred to as an intelligent computing cluster or an intelligent computing cluster, which is not limited in this application.

[0175] It should be understood that the term "multiple" as used herein refers to two or more than two. Furthermore, the step numbers described herein merely illustrate a possible order of execution of the steps. In some other embodiments, the steps may be executed out of sequence, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the order shown. This is not a limitation of the present application.

[0176] The above description is merely an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A rendering system, characterized in that: The rendering system includes: a rendering pipeline manager, an intermediate buffer, M geometry processing pipelines, and N pixel processing pipelines, where M is an integer greater than 1 and N is an integer greater than or equal to 1; wherein a plurality of data segments divided from an input data stream of the rendering system are assigned to the M geometry processing pipelines for processing, and metadata obtained from processing each of the data segments is stored in the intermediate buffer; The rendering pipeline manager is configured to obtain the serial number and memory information of the data segment processed by the geometry processing pipeline, wherein the memory information of the data segment is used to determine the memory page occupied by the graphics metadata corresponding to the data segment in the intermediate buffer; The rendering pipeline manager is also used to send rendering instructions to the N pixel processing pipelines respectively when there are k data segments with consecutive numbers among the data segments processed by the geometry processing pipeline. The rendering instructions include the numbers and memory information of the k data segments, where k is an integer greater than or equal to 1.

2. The rendering system according to claim 1, wherein: The rendering pipeline manager is further configured to select, from the data segments processed by the geometry processing pipeline, one or more data segments numbered between a first value and a second value as the k data segments; The first value is the maximum number of data segments submitted to the N pixel processing pipelines for processing plus 1, and the second value is the maximum number that satisfies the consecutive numbers of the k data segments.

3. The rendering system according to claim 1, wherein: The rendering pipeline manager is provided with M queues, each queue being used to store the number and memory information of a data segment processed by the geometry processing pipeline.

4. The rendering system according to claim 3, wherein: The i-th queue among the M queues is used to store the serial number and memory information of the data segment processed by the i-th geometry processing pipeline among the M geometry processing pipelines, where i is a positive integer less than or equal to M; The rendering pipeline manager is further configured to add the serial number and memory information of the first data segment to the end of the i-th queue after the i-th geometry processing pipeline completes processing the first data segment; The rendering pipeline manager is further configured to remove the number and memory information of the second data segment from the i-th queue when the second data segment stored in the i-th queue meets the conditions for being submitted to the pixel processing pipeline.

5. The rendering system according to claim 3, wherein: The rendering pipeline manager is further configured to, if a merged row exists in the M queues, remove the numbers and memory information of the M data segments included in the merged row from the M queues; wherein the merged row includes the number and memory information of a data segment with the smallest number stored in each of the M queues, and the numbers of the M data segments included in the merged row are consecutive; The rendering pipeline manager is further used to record the pending information corresponding to the M queues respectively; wherein, for the i-th queue among the M queues, the pending information corresponding to the i-th queue is used to determine the data segments that have been moved out of the i-th queue and have not been submitted to the pixel processing pipeline.

6. The rendering system according to claim 1, wherein: The rendering pipeline manager is further configured to execute the step of sending rendering instructions to the N pixel processing pipelines respectively when the memory occupancy rate corresponding to any one of the geometry processing pipelines is greater than or equal to a set threshold; Among them, the memory occupancy rate corresponding to the geometry processing pipeline is the ratio of the number of memory pages occupied in the intermediate buffer by the graphic element data corresponding to the data segments that have been processed by the geometry processing pipeline and have not been submitted to the pixel processing pipeline, to the total number of memory pages included in the intermediate buffer.

7. The rendering system according to claim 1, wherein: The j-th pixel processing pipeline among the N pixel processing pipelines is configured to obtain, from the intermediate buffer according to the rendering instruction, graphics metadata corresponding to the k data segments, respectively, and render the graphics metadata corresponding to the k data segments, and after rendering of the graphics metadata corresponding to the k data segments is completed, send completion indication information to the rendering pipeline manager, wherein the completion indication information is used to indicate that the j-th pixel processing pipeline has completed rendering of the graphics metadata corresponding to the k data segments, where j is a positive integer less than or equal to N; The rendering pipeline manager is further configured to release the memory pages occupied by the graphics metadata respectively corresponding to the k data segments in the intermediate buffer after the N pixel processing pipelines have completed rendering of the graphics metadata respectively corresponding to the k data segments.

8. The rendering system according to claim 1, wherein: The rendering system further includes: M tile dividers; The i-th geometry processing pipeline among the M geometry processing pipelines is configured to process a data segment allocated to the i-th geometry processing pipeline among the multiple data segments divided into the input data stream to obtain a processed data segment, where i is a positive integer less than or equal to M; The i-th tile divider among the M tile dividers is used to tile the processed data segment to obtain the graphic element data corresponding to the data segment, and store the graphic element data corresponding to the data segment in the intermediate buffer.

9. The rendering system according to claim 8, characterized in that: The i-th geometry processing pipeline is further configured to suspend processing new data segments when the number of memory pages occupied in the intermediate buffer by the graphics metadata corresponding to the data segments that have been processed by the i-th geometry processing pipeline and not submitted to the pixel processing pipeline reaches the maximum usage number allocated to the i-th geometry processing pipeline.

10. The rendering system according to claim 1, wherein: The rendering system includes a plurality of graphics processing unit (GPU) cores, and each GPU core includes at least one geometry processing pipeline and at least one pixel processing pipeline.

11. A GPU chip, characterized in that: The GPU chip includes the rendering system according to any one of claims 1 to 10.

12. An electronic device, characterized in that: The electronic device includes a GPU chip, and the GPU chip includes the rendering system according to any one of claims 1 to 10.

13. A rendering method used in a rendering system, characterized in that: The rendering system includes: a rendering pipeline manager, an intermediate buffer, M geometry processing pipelines, and N pixel processing pipelines, where M is an integer greater than 1 and N is an integer greater than or equal to 1; wherein a plurality of data segments divided from an input data stream of the rendering system are assigned to the M geometry processing pipelines for processing, and metadata obtained from processing each of the data segments is stored in the intermediate buffer; The method comprises: The rendering pipeline manager obtains the serial number and memory information of the data segment processed by the geometry processing pipeline, wherein the memory information of the data segment is used to determine the memory page occupied by the graphics metadata corresponding to the data segment in the intermediate buffer; When there are k data segments with consecutive numbers among the data segments processed by the geometry processing pipeline, the rendering pipeline manager sends rendering instructions to the N pixel processing pipelines respectively, and the rendering instructions include the numbers and memory information of the k data segments, where k is an integer greater than or equal to 1.

Images