Hybrid rendering method and graphics processing unit

Abstract

Provided in the present application are a hybrid rendering method and a graphics processing unit. The method is applied to a graphics processing unit. The method comprises: acquiring a rendering instruction, wherein the rendering instruction is used for instructing the execution of a first rasterization task and a first ray tracing task on a geometric primitive in a viewport, the geometric primitive serves as a part of an input of the first ray tracing task and an input of the first rasterization task, and an output of the first rasterization task serves as another part of the input of the first ray tracing task; on the basis of M sub-viewports of the viewport, respectively determining M second rasterization tasks, and on the basis of the M sub-viewports, respectively determining M second ray tracing tasks, wherein an output of an i-th second rasterization task serves as a part of an input of an i-th second ray tracing task; and executing the M second rasterization tasks and the M second ray tracing tasks, wherein at least one second rasterization task and at least one second ray tracing task are executed in parallel.

Description

Hybrid rendering methods and graphics processing units

[0001] This application claims priority to Chinese Patent Application No. 202510114908.3, filed on January 23, 2025, entitled "Hybrid Rendering Method and Graphics Processing Unit", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of graphics rendering, and more specifically, to a hybrid rendering method and a graphics processing unit. Background Technology

[0003] The hybrid rendering process first performs rasterization calculations to quickly generate basic geometric information and preliminary visual effects for the scene, and stores these intermediate results. Then, ray tracing calculations are performed, reading these pre-calculated intermediate data to further enhance image quality and simulate realistic lighting, reflections, and shadows.

[0004] However, this phased working mode has obvious limitations. On the one hand, in rasterization calculations, frequent off-chip cache read / write operations between the rasterization engine and shader cores increase system power consumption and performance overhead. On the other hand, in ray tracing calculations, the shading and lighting calculations handled by the ray tracing cores must wait for the rasterization task to complete before they can begin. There are unusable time periods in the calculation tasks of all three hardware units, thus limiting overall rendering efficiency. Especially in application scenarios that pursue high performance and low latency, this sequential dependency becomes a bottleneck for improving real-time rendering performance. Summary of the Invention

[0005] This application provides a hybrid rendering method and a graphics processing unit to improve the efficiency of the hybrid rendering method.

[0006] In a first aspect, a hybrid rendering method is provided, the method being applied to a graphics processing unit, the method comprising: obtaining rendering instructions, the rendering instructions being used to instruct the execution of a first rasterization task and a first ray tracing task on geometric primitives within a viewport, wherein the geometric primitives are used as part of the input to the first ray tracing task and the input to the first rasterization task, and the output of the first rasterization task is used as another part of the input to the first ray tracing task; determining M second rasterization tasks according to M sub-viewports of the viewport, and determining M second ray tracing tasks according to the M sub-viewports, wherein the output of the i-th second rasterization task is used as part of the input to the i-th second ray tracing task, i≤M, M>1, and i and M are positive integers; executing the M second rasterization tasks and the M second ray tracing tasks, wherein at least one second rasterization task and at least one second ray tracing task are executed in parallel.

[0007] In an embodiment of the present application, at least one pair of rasterization subtasks and ray tracing subtasks divided according to sub-viewport can be executed in parallel to perform a hybrid rendering task in parallel, thereby reducing the total duration of the hybrid rendering task and improving the efficiency of the hybrid rendering method.

[0008] In some implementations of the first aspect, the execution period of the (j + 1)-th second rasterization task and the execution period of the j-th second ray tracing task at least partially overlap, where j < M and j is a positive integer.

[0009] In such an implementation, different execution periods can be planned for different subtasks according to the execution duration required by the rasterization subtasks and the execution duration required by the ray tracing subtasks, so as to reduce the idle duration of the hardware units of the graphics processing unit, thereby reducing the total duration of the hybrid rendering task and improving the efficiency of the hybrid rendering method.

[0010] In some implementations of the first aspect, determining the execution period of the second rasterization task and the execution period of the second ray tracing task includes: starting the (j + 1)-th second rasterization task and the j-th second ray tracing task at the completion moment of the execution period of the j-th second rasterization task.

[0011] In such an implementation, after completing a rasterization subtask, the start events of the corresponding ray tracing subtask and the next rasterization subtask can be immediately triggered without maintaining the status information of the completed rasterization subtasks, so as to simplify task scheduling and thereby improve the efficiency of the hybrid rendering method.

[0012] In some implementations of the first aspect, the execution period of the j-th second ray tracing task and the execution period of the (j + 1)-th second ray tracing task at least partially overlap.

[0013] In such an implementation, the corresponding ray tracing subtasks can be executed in parallel after the sequentially executed rasterization subtasks are completed, so as to make full use of the parallel capabilities of the ray tracing pipeline, thereby reducing the total duration of the hybrid rendering task and improving the efficiency of the hybrid rendering method.

[0014] In some implementations of the first aspect, the second rasterization task corresponds to pixel candidate slices obtained from geometric primitives, and the pixel candidate slices are used to determine the color values of the sub-viewport corresponding to the second rasterization task and are stored in the on-chip cache of the graphics processing unit, where the resolution of the sub-viewport is determined according to the capacity of the on-chip cache.

[0015] In such an implementation, the size of the sub-viewport and the amount of data processed by a single rasterization subtask can be reasonably adjusted according to the capacity of the on-chip cache of the GPU, so as to improve the data transmission efficiency inside the rasterization pipeline, thereby improving the efficiency of the hybrid rendering method.

[0016] In a second aspect, a graphics processing unit is provided, including: a graphics processing unit front end for obtaining rendering instructions, where the rendering instructions are used to indicate performing a first rasterization task and a first ray tracing task on geometric primitives within a viewport. The geometric primitives are used as part of the input for the first ray tracing task and the input for the first rasterization task, and the output of the first rasterization task is used as another part of the input for the first ray tracing task; a scheduler for respectively determining M second rasterization tasks according to M sub-viewport of the viewport, and respectively determining M second ray tracing tasks according to the M sub-viewport, where the output of the i-th second rasterization task is used as part of the input for the i-th second ray tracing task, i ≤ M, M > 1, and i and M are positive integers; a rasterization pipeline for performing the M second rasterization tasks, and a ray tracing pipeline for performing the M second ray tracing tasks, where at least one second rasterization task and at least one second ray tracing task are executed in parallel.

[0017] In some implementations of the second aspect, the scheduler includes a hybrid rendering task scheduling unit, and the hybrid rendering task scheduling unit is used to schedule the execution periods of the second rasterization tasks and the execution periods of the second ray tracing tasks, where the execution period of the (j + 1)-th second rasterization task and the execution period of the j-th second ray tracing task at least partially overlap, j < M, and j is a positive integer.

[0018] In some implementations of the second aspect, the hybrid rendering task scheduling unit is further used to: at the completion moment of the execution period of the j-th second rasterization task, start the (j + 1)-th second rasterization task and the j-th second ray tracing task.

[0019] In some implementations of the second aspect, the execution period of the j-th second ray tracing task and the execution period of the (j + 1)-th second ray tracing task at least partially overlap.

[0020] In some implementations of the second aspect, the second rasterization task corresponds to pixel candidate slices obtained from geometric primitives, and the pixel candidate slices are used to determine the color values of the sub-viewport corresponding to the second rasterization task and are stored in an on-chip cache, where the resolution of the sub-viewport is determined according to the capacity of the on-chip cache.

[0021] In a third aspect, a computer device is provided, the computer device includes at least one processor and at least one memory; the at least one memory stores at least one computer program, and the at least one computer program includes instructions, when the instructions are executed by the at least one processor, the method described in any implementation of the first aspect is caused to be executed.

[0022] In some implementations of the third aspect, the at least one processor includes the graphics processing unit described in any implementation of the second aspect.

[0023] Fourthly, a computer-readable storage medium is provided, wherein computer instructions are stored therein, which, when executed on a computer device, cause the method described in any implementation of the first aspect to be performed.

[0024] Fifthly, a computer program product comprising instructions is provided, which, when executed by a computer device, cause the computer device to perform the method as described in any implementation of the first aspect. Attached Figure Description

[0025] Figure 1 is a schematic block diagram of a graphics processing unit (GPU) provided in an embodiment of this application.

[0026] Figure 2 is a schematic block diagram of a GPU provided in an embodiment of this application.

[0027] Figure 3 is a flowchart illustrating a hybrid rendering method provided in an embodiment of this application.

[0028] Figures 4 to 6 are schematic block diagrams of task scheduling for the hybrid rendering method provided in the embodiments of this application.

[0029] Figure 7 is a schematic block diagram of a computer device provided in an embodiment of this application.

[0030] Figure 8 is a schematic block diagram of a computer program product provided in an embodiment of this application. Detailed Implementation

[0031] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0032] A driver acts as a bridge between the operating system and the hardware, managing and controlling the operation of specific hardware devices. For a graphics processing unit (GPU), the driver's primary task is to translate graphics application programming interface (API) calls (such as OpenGL, Vulkan, DirectX, etc.) issued by applications into low-level instructions that the GPU can understand, and to manage GPU resource allocation and task scheduling.

[0033] The GPU frontend is a logical unit of the GPU responsible for receiving configuration information and command buffers from the driver, parsing and distributing them. It acts as an intermediary between the driver and the GPU computing unit, ensuring that rendering tasks are correctly allocated to appropriate hardware resources.

[0034] Rasterization is a 3D graphics rendering method used to project vector objects in a 3D scene onto a 2D screen and quickly determine which pixels are covered by these objects. This process involves converting geometry into pixels in screen space, followed by operations such as shading, texture mapping, and depth testing. Rasterization offers extremely high performance when handling complex 3D scenes, but its ability to simulate real-world lighting effects, such as complex lighting phenomena like reflection, refraction, and shadows, is limited.

[0035] A rasterization pipeline refers to a combination of hardware units and / or logic units corresponding to a series of steps and operations in the rasterization process, used to convert 3D geometric primitives into pixels or fragments in a 2D image to achieve rasterization.

[0036] Ray tracing is a physically based 3D graphics rendering method that simulates the real-world path of light rays as they travel through a scene, resulting in extremely realistic images. This method calculates the color of each pixel by emitting virtual rays from a camera and tracing their interactions with vector objects in the scene. While ray tracing can produce high-quality visual effects, it places extremely high demands on computational resources, especially when real-time performance is required.

[0037] A ray tracing pipeline refers to a combination of hardware and / or logic units corresponding to a series of steps and operations in the ray tracing process. It generates realistic images by simulating the propagation of light in a scene to achieve ray tracing.

[0038] Hybrid rendering typically refers to a graphics rendering method that combines the advantages of rasterization and ray tracing, two major 3D graphics rendering techniques. Specifically, hybrid rendering strategies utilize rasterization to quickly process key aspects of a scene, such as the shape and position of objects, while applying ray tracing to enhance specific effects, such as global illumination, reflections, and accurate shadows. This combination not only improves the quality of the final image but also enables near real-time rendering on modern hardware, making it one of the key technologies for enhancing visual fidelity in current games and graphics applications.

[0039] The following diagram, using a schematic block diagram of a GPU as shown in Figure 1, illustrates common hybrid rendering methods. The units represented by rectangles can be implemented using a combination of hardware and software, while the units or methods represented by rounded rectangles are primarily implemented in software.

[0040] Those skilled in the art will readily recognize that, based on the units and algorithm steps described in conjunction with the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0041] In some embodiments, the GPU front end sends rendering instructions to other hardware units or virtual logic units in the GPU based on instructions from the driver. The draw call of the rendering instruction includes information such as geometric primitives (e.g., triangles, line segments, points) and index buffers. The rendering instruction also defines the target area (viewport) and clipping plane for rendering, so that only visible geometry is rendered.

[0042] In some embodiments, the working process of the rasterization pipeline performing rasterization tasks is described. First, vertex processing is performed by the vertex shader, which processes the vertex data of the input 3D geometric primitives, transforming the vertices from object space to screen space, and outputting the transformed vertex positions and other vertex attributes (such as color, texture coordinates, normals, etc.). Then, primitive assembly is performed by the rasterization unit, combining the vertices into geometric primitives and performing some basic geometric processing to remove geometric primitives that are not within the view frustum.

[0043] Next, the rasterization unit converts the geometric primitives into fragments, also known as pixel candidate pieces. These fragments contain information such as position, normals, depth values, color, and texture coordinates, and determine which pixels are covered by the geometric primitives. One or more fragments are generated for each covered pixel. Because fragments are large in size, they are difficult to store directly in the GPU's on-chip cache. They are typically temporarily stored in the GPU's off-chip cache, which can be either video memory or system memory.

[0044] Secondly, fragment shading is performed using a fragment shader. Detailed shading calculations are performed on each fragment, including lighting calculations, texture sampling, and shadow calculations, to output pixel colors as well as other relevant data such as the fragment's position, normals, material properties, and depth values. The data output by the fragment shader is called the geometry buffer, which can be stored in the geometry buffer (G-buffer) for use in subsequent processes.

[0045] Finally, per-fragment operations are performed through the rasterization unit to determine which fragments should be retained and how to combine the color of the new fragment with the pixel colors already existing in the frame buffer, so that the fragment color calculated based on the new fragment and the pixel colors in the frame buffer is written to the frame buffer.

[0046] In one possible implementation, the rasterization unit is implemented through a hardware module called a rasterization engine. In this embodiment, the rasterization unit and the rasterization engine have similar meanings and can be used interchangeably. Vertex shaders and fragment shaders are typically virtual logical units. Vertex shaders, fragment shaders, and other common shader units are implemented through a hardware module called a shader core, which can also be simply referred to as a shader. Texture sampling in the fragment shader can be implemented through a hardware module called a texture sampling unit. The texture sampling unit can be connected to the shader core and off-chip buffers to efficiently read fragments output by the rasterization unit and perform texture sampling to provide data to the shader core. The lighting calculation unit and shadow calculation unit are typically virtual logical units.

[0047] In some embodiments, the process of the ray tracing pipeline performing ray tracing tasks is described. First, acceleration structure construction is performed. Based on the input 3D geometric primitives, material properties, and light source information, data structures are constructed to accelerate subsequent calculations, such as bounding volume hierarchy (BVH) and k-dimensional spatial segmentation tree (K-Dtree). These structures can help quickly determine whether rays intersect with objects in the scene, thereby significantly improving rendering efficiency. Then, ray generation is performed based on the camera's position, orientation, and field of view (FOV). Initial rays are emitted from the camera or light source to simulate the propagation path of light in the scene. These rays can be primary rays, used to directly observe the scene from the camera's perspective, or secondary rays, used to simulate effects such as reflection, refraction, and shadows.

[0048] In one possible implementation, according to the aforementioned embodiments and Figure 1, the fragment shader of the rasterization pipeline stores geometric cache information such as the fragment's position, normals, material properties, and depth values ​​in a geometric buffer. The ray tracing pipeline can use the geometric buffer to omit some redundant calculations, thereby speeding up the calculation speed of subsequent steps. This approach is called hybrid rendering.

[0049] Secondly, shading and lighting calculations are performed. Based on the intersection point information and light source information, the color of the light at that intersection point is calculated. This includes applying lighting models (such as Phong, Blinn-Phong, PBR, etc.), considering the influence of the light source, and handling effects such as reflection and refraction to output the color at the intersection point and secondary rays such as reflected rays, refracted rays, and shadow rays. Specifically, the geometry buffer in the G-buffer can be used for lighting calculations. This method separates geometry processing and lighting calculations, allowing the final color value of each pixel to be calculated separately based on the geometry and material information in the geometry buffer.

[0050] Next, recursive tracing is performed. This includes sub-processes such as intersection testing, which checks if a ray intersects with an object in the scene and finds the nearest intersection point. This process typically uses acceleration structures to speed up computation, avoiding testing each geometric primitive individually. If a ray intersects an object, the location, depth, normal, and material properties of the intersection point are returned. Further, for the generated secondary rays, intersection testing and shading calculations are repeated until a predetermined recursion depth is reached or certain termination conditions are met. This process can simulate complex effects such as multiple reflections and refractions. Finally, results are merged and post-processed to combine all ray tracing results and generate the final pixel color. For each pixel, there may be contributions from multiple rays (e.g., primary and secondary rays), and these color values ​​need to be combined in some way to obtain the final pixel color. The final pixel color undergoes post-processing such as tone mapping, anti-aliasing, depth of field, and motion blur before being written to the framebuffer.

[0051] In one possible implementation, the pixel colors generated by the rasterization pipeline already exist in the frame buffer. It can be determined how to combine the newly generated color with the pixel colors already existing in the frame buffer to write the final color into the frame buffer, thereby obtaining the pixel color to be displayed.

[0052] In one possible implementation, the ray tracing pipeline is implemented through a hardware module called the ray tracing core. In this embodiment, the ray tracing pipeline and the ray tracing core have similar meanings and can be used interchangeably.

[0053] In summary, a GPU consists of multiple shader cores, multiple ray tracing cores, and multiple rasterization engines. A single core can only execute one task at a time, while multiple cores of the same type can perform concurrent computations of the same type of task.

[0054] The hybrid rendering process first performs rasterization calculations to quickly generate basic geometric information and preliminary visual effects for the scene, and stores these intermediate results. Then, ray tracing calculations are performed, reading these pre-calculated intermediate data to further enhance image quality and simulate realistic lighting, reflections, and shadows.

[0055] However, this phased working mode has obvious limitations. On the one hand, in rasterization calculations, frequent off-chip cache read / write operations between the rasterization engine and shader cores increase system power consumption and performance overhead. On the other hand, in ray tracing calculations, the shading and lighting calculations handled by the ray tracing cores must wait for the rasterization task to complete before they can begin. There are unusable time periods in the calculation tasks of all three hardware units, thus limiting overall rendering efficiency. Especially in application scenarios that pursue high performance and low latency, this sequential dependency becomes a bottleneck for improving real-time rendering performance.

[0056] In view of this, this application provides a hybrid rendering method applied to a graphics processing unit. The method includes: obtaining rendering instructions, which instruct the execution of a first rasterization task and a first ray tracing task on geometric primitives within a viewport, wherein the geometric primitives are used as part of the input to the first ray tracing task and the input to the first rasterization task, and the output of the first rasterization task is used as another part of the input to the first ray tracing task; determining M second rasterization tasks according to M sub-viewports of the viewport, and determining M second ray tracing tasks according to the M sub-viewports, wherein the output of the i-th second rasterization task is used as part of the input to the i-th second ray tracing task, i≤M, M>1, and i and M are positive integers; and executing the M second rasterization tasks and the M second ray tracing tasks, wherein at least one second rasterization task and at least one second ray tracing task are executed in parallel.

[0057] In the embodiments of this application, at least one pair of rasterization subtasks and ray tracing subtasks divided according to the sub-viewport can be executed in parallel to perform hybrid rendering tasks in parallel, thereby reducing the total duration of hybrid rendering tasks and improving the efficiency of hybrid rendering methods.

[0058] In some embodiments, the GPU is configured with hardware units and logic units as shown in Figure 2. The following description, in conjunction with the flowchart of the hybrid rendering method shown in Figure 3, illustrates the specific process of applying this method to the GPU shown in Figure 2.

[0059] S310, Obtain rendering instructions. The rendering instructions are used to instruct the execution of a first rasterization task and a first ray tracing task on the geometric primitives within the viewport. The geometric primitives are used as part of the input to the first ray tracing task and the input to the first rasterization task. The output of the first rasterization task is used as another part of the input to the first ray tracing task.

[0060] In one possible implementation, the GPU scheduler can achieve load balancing. For example, it can dynamically allocate computational tasks to different streaming multiprocessors (including multiple shader cores) and other dedicated hardware units (such as ray tracing cores, rasterization engines, etc.) based on the current workload. The scheduler will allocate thread blocks to these cores, so that tasks (e.g., subtasks of the first rasterization task or subtasks of the first ray tracing task) are evenly distributed across different cores, avoiding resource idleness or overload.

[0061] In one possible implementation, the scheduler manages dependencies between tasks and optimizes overall performance through priority adjustments and load balancing. For example, some tasks may need to wait for other tasks to complete before starting, or multiple tasks may share the same set of resources, and it must be ensured that they do not access the same resources simultaneously.

[0062] In one possible implementation, the GPU front end sends rendering instructions to other hardware units or virtual logic units within the GPU based on instructions from the driver. The draw call of the rendering instruction includes information such as geometric primitives (e.g., triangles, line segments, points) and index buffers. The rendering instruction also defines the viewport and clipping plane, ensuring that only visible geometry is rendered.

[0063] In one possible implementation, rendering instructions are also used to indicate rendering parameters, including blending mode, depth test, stencil test, shader procedure, etc., which determine how each fragment (pixel) is processed. Rendering parameters may also include identifiers or physical locations specifying rendering targets (such as color buffers, depth buffers, stencil buffers).

[0064] It should be understood that, according to Figure 2 and the foregoing embodiments, the output of the first rasterization task is used as another part of the input of the first ray tracing task, indicating that the first rasterization task and the first ray tracing task in this embodiment are used to jointly complete the hybrid rendering task.

[0065] S320, M second rasterization tasks are determined according to the M sub-viewports of the viewport, and M second ray tracing tasks are determined according to the M sub-viewports, wherein the output of the i-th second rasterization task is used as part of the input of the i-th second ray tracing task, i≤M, M>1, i and M are positive integers, i≤M, M>1.

[0066] In some embodiments, the scheduler includes a hybrid rendering task partitioning unit, which is used to partition a second rasterization task and a second ray tracing task.

[0067] In some embodiments, the number of pixels in the viewport is the same as the resolution of the display. For example, if the resolution of the display is 2560×1440, then the number of pixels in the viewport is 2560×1440.

[0068] In the second rasterization task, a complete rasterization process is performed on each sub-viewport (tile), including vertex processing, primitive assembly, conversion of geometric primitives into fragments, and fragment shading. In most sub-processes, each sub-viewport independently completes the conversion and calculation of its corresponding data. For example, in the process of converting geometric primitives into fragments, each sub-viewport is only responsible for processing the geometric primitives associated with it; only triangles intersecting with that sub-viewport are rasterized into fragments. However, steps such as vertex processing and primitive assembly only process global data and do not require separate processing for each sub-viewport.

[0069] In one possible implementation, for geometric primitives spanning multiple sub-viewports, it is necessary to ensure that fragments at the boundaries are handled correctly to avoid gaps or overlaps. This can be achieved by extending the boundaries of each sub-viewport, i.e., each sub-viewport slightly beyond its actual boundary, to ensure continuity between adjacent blocks.

[0070] In one possible implementation, the second rasterization task corresponds to pixel candidate patches obtained from geometric primitives. These pixel candidate patches are used to determine the color value of the sub-viewport corresponding to the second rasterization task and are stored in the on-chip cache of the graphics processing unit. The resolution of the sub-viewport is determined based on the capacity of the on-chip cache. Specifically, typically, the rasterization task needs to generate one or more fragments for each covered pixel to determine the pixel's color value. Since the fragment data is large, it is difficult to store it directly in the GPU's on-chip cache and it generally needs to be temporarily stored in the GPU's off-chip cache. Compared to the complete first rasterization task, the second rasterization task generates fragments with relatively small data size during the process of converting geometric primitives into fragments. Therefore, as shown in Figure 2, these fragments (i.e., pixel candidate patches) can be stored in the on-chip cache.

[0071] In this implementation, the size of the sub-viewport and the amount of data processed by a single rasterization subtask can be reasonably adjusted according to the capacity of the GPU's on-chip cache, so as to improve the data transmission efficiency within the rasterization pipeline and thus improve the efficiency of the hybrid rendering method.

[0072] In the second ray tracing task, a complete ray tracing process is performed on the sub-viewports. In most sub-processes, each sub-viewport independently completes the transformation and calculation of its corresponding data. For example, in the ray generation stage, each sub-viewport independently generates the master rays emitted from the camera. The direction and origin of these rays are determined by the position of the sub-viewport. However, because the acceleration structure is global, all sub-viewports share the same acceleration structure, and all sub-viewports depend on global geometric data and light source information, therefore, some shading and lighting calculations are global.

[0073] In one possible implementation, geometric primitives spanning multiple sub-viewports can be implemented using a method similar to that described in the preceding embodiments, which will not be elaborated here.

[0074] It should be understood that, for ease of explanation, the following embodiments of this application assume that global steps such as vertex processing, primitive assembly, and accelerated structure construction are completed instantaneously. In other words, the impact of global steps on task duration is not considered in the duration analysis.

[0075] S330, execute M second rasterization tasks and M second ray tracing tasks, wherein at least one second rasterization task and at least one second ray tracing task are executed in parallel.

[0076] In some embodiments, according to the schematic block diagram of task scheduling for the hybrid rendering method shown in Figure 4, for a complete hybrid rendering task consisting of a first rasterization task and a first ray tracing task, at time #1 (i.e., the initial time), on the one hand, the ray tracing pipeline performs accelerated structure building and ray generation; the combination of these tasks is referred to as task C, which is completed at time #2. On the other hand, the rasterization pipeline performs the rasterization task (referred to as task A), which is completed at time #3. Typically, the time taken by task C is less than the time taken by task A, i.e., time #2 is earlier than time #3. If time #2 is later than time #3, then time #2 is used as time #3 in subsequent steps.

[0077] At time #3, the ray tracing pipeline performs shading and lighting calculations, recursive tracing, and pixel generation; these tasks are called Task B, and Task B is completed at time #4. Clearly, the start time of Task B cannot be earlier than the completion time of Task A. Ideally, both the start time of Task B and the completion time of Task A would be time #3.

[0078] In typical hybrid rendering methods, the first rasterization task corresponds to Task A, the first ray tracing task corresponds to Task C and Task B, and Task B is started only after Task A is completed. Parallel execution of hybrid rendering tasks means that the rasterization engine and shader core can perform some specific mathematical calculations in Task A in parallel, and the ray tracing core can perform some specific mathematical calculations in Tasks B and C in parallel.

[0079] In some embodiments of this application, the scheduler includes a hybrid rendering task scheduling unit, which schedules the execution periods of the second rasterization task and the second ray tracing task so that the execution periods of the (j+1)th second rasterization task and the jth second ray tracing task at least partially overlap. Specifically, the jth second rasterization task corresponds to task A#j, and the jth second ray tracing task corresponds to task C#j and task B#j, and task B#j is started only after task A#j is completed.

[0080] For ease of explanation, the execution period of the (j+1)th second rasterization task is referred to as the first time period, and the execution period of the jth second ray tracing task is referred to as the second time period.

[0081] In one possible implementation, the ray tracing core executes task C#j+1 immediately after completing task B#j. For ease of understanding and to make the accompanying drawings concise and clear, task B#j in subsequent embodiments of this application may refer to the combination of executing task B#j first, followed immediately by executing task C#j+1. Furthermore, according to the foregoing embodiments, task C#1 completes at time #2, and task A#1 completes at time #3. If time #2 is later than time #3, then time #2 will be used as time #3 in subsequent steps. Therefore, task A#1 in subsequent embodiments of this application may refer to the combination of task C#1 and task A#1, which are started at the same time.

[0082] In one possible implementation, task A #j+1 and task B #j (e.g., task A #2 and task B #1) correspond to the first time period and the second time period. For ease of explanation, subsequent embodiments assume the duration of each first time period (denoted as t). A The duration of each second time period is the same (denoted as t). B )same.

[0083] In one possible implementation, according to the schematic block diagram of task scheduling shown in Figure 5, the duration of the first time period is shorter than the duration of the second time period. After the rasterization pipeline completes task A #2, the obtained geometry buffer can be output to the G-buffer. Then, according to the instruction of the hybrid rendering task scheduling unit, task A #3 is started. Task B #2 finds the required geometry buffer data from the G-buffer and starts before task A #3 completes. That is, the first time period corresponding to task A #3 and the second time period corresponding to task B #2 partially overlap. However, at the moment task B #3 starts, task A #4 has already completed, so the first time period corresponding to task A #4 and the second time period corresponding to task B #3 do not overlap. Therefore, j can be either 1 or 2, but cannot be 3.

[0084] In a more general case, t A t can represent the average duration of each first time period. B This can represent the average duration of each second time period, and the total time for the hybrid rendering task is t. A +Mt B When executing hybrid rendering tasks sequentially, the total task time is Mt. A +Mt B That is, the total time of the task in this embodiment is reduced by (M-1)t. A .

[0085] In this implementation, different execution periods can be planned for different subtasks based on the execution time required for rasterization and ray tracing, thereby reducing the idle time of the graphics processing unit's hardware units, thus reducing the total duration of the hybrid rendering task and improving the efficiency of the hybrid rendering method.

[0086] In some embodiments, the hybrid rendering task scheduling unit is configured to: start the (j+1)th second rasterization task and the jth second ray tracing task at the completion time of the execution period of the jth second rasterization task. That is, the first period and the second period correspond to the same start time.

[0087] In one possible implementation, according to the schematic block diagram of task scheduling shown in Figure 6(a), the duration of the first time period is greater than or equal to the duration of the second time period. After the rasterization pipeline completes task A #1, the obtained geometry buffer can be output to the G-buffer. Then, according to the instruction of the hybrid rendering task scheduling unit, task A #2 is started. At the same time, the ray tracing pipeline starts task B #1 based on the geometry buffer in the G-buffer.

[0088] In this implementation, after completing a rasterization subtask, the start events for the corresponding ray tracing subtask and the next rasterization subtask can be triggered immediately without maintaining the state information of the completed rasterization subtasks, thus simplifying task scheduling and improving the efficiency of the hybrid rendering method.

[0089] In one possible implementation, the execution time of the j-th second ray tracing task and the (j+1)-th second ray tracing task at least partially overlap, meaning that multiple second ray tracing tasks are executed in parallel. According to the schematic block diagram of task scheduling shown in Figure 6(b), the duration of the first time period is shorter than the duration of the second time period. After the rasterization pipeline completes task A #2, the obtained geometry buffer can be output to the G-buffer, and then task A #3 is started according to the instructions of the hybrid rendering task scheduling unit. Meanwhile, although task B #1 has not yet been completed, the ray tracing pipeline includes multiple ray tracing cores, so task B #2 can start immediately to execute in parallel with task B #1.

[0090] In a more general case, t A It can represent the average duration of M first time periods, t B This can represent the average duration of M second time periods. The two implementations of this embodiment can include t with arbitrary size relationships. A and t B The total time for the hybrid rendering task is Mt A +t B The total execution time is reduced by (M-1)t compared to sequential execution. B .

[0091] In this implementation, the corresponding ray tracing subtasks can be executed in parallel after the rasterization subtasks are executed sequentially, so as to make full use of the parallel capabilities of the ray tracing pipeline, thereby reducing the total duration of the hybrid rendering task and improving the efficiency of the hybrid rendering method.

[0092] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0093] This application also provides a computing device 900. As shown in FIG7, the computing device 900 includes: a bus 902, a processor 904, a memory 906, and a communication interface 908. The processor 904, the memory 906, and the communication interface 908 communicate with each other via the bus 902. The computing device 900 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 900.

[0094] Bus 902 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 7, but this does not imply that there is only one bus or one type of bus. Bus 904 can include pathways for transmitting information between various components of computing device 900 (e.g., memory 906, processor 904, communication interface 908).

[0095] The processor 904 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP). This application primarily describes an embodiment where the processor 904 is a GPU, but it is not intended to limit the scope of the application to this type of processor.

[0096] The memory 906 may include volatile memory, such as random access memory (RAM). The processor 904 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0097] The memory 906 stores executable program code, which the processor 904 executes to implement the aforementioned hybrid rendering methods. That is, the memory 906 stores instructions for executing the hybrid rendering methods.

[0098] The communication interface 908 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 900 and other devices or communication networks.

[0099] Another embodiment of this application provides a computer-readable storage medium storing instructions that, when executed on a means for accessing memory, perform the various steps of the method flow shown in the above-described method embodiments. In some embodiments, the disclosed method can be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of art.

[0100] Figure 8 schematically illustrates a conceptual partial view of a computer program product provided in an embodiment of this application, the computer program product including a computer program for executing computer processes on a computing device.

[0101] In one embodiment, the computer program product is provided using a signal bearer medium 1000. The signal bearer medium 1000 may include one or more program instructions that, when run by one or more processors, can execute the hybrid rendering method provided in this application embodiment.

[0102] In some examples, the signal carrying medium 1000 may include a computer-readable medium 1001, such as, but not limited to, a hard disk drive, a compact disc (CD), a digital video disc (DVD), a digital magnetic tape, a memory, a read-only memory (ROM), or a random access memory (RAM), etc.

[0103] In some implementations, the signal carrying medium 1000 may include a computer recordable medium 1002, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, and so on.

[0104] In some implementations, the signal-bearing medium 1000 may include a communication medium 1003, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.). The signal-bearing medium 1000 may be transmitted by a wireless communication medium 1003 (e.g., a wireless communication medium conforming to the IEEE 1502.11 standard or other transmission protocols). One or more program instructions may be, for example, computer-executable instructions or logical implementation instructions.

[0105] In some examples, the computing device may be configured to provide various operations, functions, or actions in response to one or more program instructions in a computer-readable medium 1001, a computer-recordable medium 1002, and / or a communication medium 1003.

[0106] It should be understood that the arrangements described herein are for illustrative purposes only. Therefore, those skilled in the art will understand that other arrangements and other elements (e.g., machines, interfaces, functions, sequences, and functional groups, etc.) can be used instead, and some elements may be omitted depending on the desired outcome. Furthermore, many of the described elements are functional entities that can be implemented as discrete or distributed components, or in any suitable combination and location with other components.

[0107] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, it can be implemented, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When executed on a computer and when the computer execution instructions are executed, all or part of the processes or functions according to the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0108] Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access, or it can contain one or more data storage devices such as servers or data centers that can be integrated with that medium. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0109] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A hybrid rendering method, characterized in that, The method is applied to a graphics processing unit, and the method includes: Obtain a rendering instruction for instructing to perform a first rasterization task and a first ray tracing task on geometric primitives within a viewport, where the geometric primitives serve as a part of the input for the first ray tracing task and the input for the first rasterization task, and the output of the first rasterization task serves as another part of the input for the first ray tracing task; Determine M second rasterization tasks respectively according to M sub - viewports of the viewport, and determine M second ray tracing tasks respectively according to the M sub - viewports, where the output of the i - th second rasterization task serves as a part of the input for the i - th second ray tracing task, i ≤ M, M > 1, and i and M are positive integers; Execute the M second rasterization tasks and the M second ray tracing tasks, where at least one of the second rasterization tasks is executed in parallel with at least one of the second ray tracing tasks.

2. The method according to claim 1, characterized in that, The execution period of the (j + 1)-th second rasterization task and the execution period of the j - th second ray tracing task at least partially overlap, j < M, and j is a positive integer.

3. The method according to claim 2, characterized in that, The determination of the execution periods of the second rasterization task and the second ray tracing task includes: At the completion moment of the execution period of the j - th second rasterization task, start the (j + 1)-th second rasterization task and the j - th second ray tracing task.

4. The method according to claim 2 or 3, characterized in that, The execution period of the j - th second ray tracing task and the execution period of the (j + 1)-th second ray tracing task at least partially overlap.

5. The method according to any one of claims 1 to 4, characterized in that, The second rasterization task corresponds to pixel candidate slices obtained from the geometric primitives, and the pixel candidate slices are used to determine the color value of the sub - viewport corresponding to the second rasterization task and are stored in the on - chip cache of the graphics processing unit, where the resolution of the sub - viewport is determined according to the capacity of the on - chip cache.

6. A graphics processing unit, characterized in that, It includes: A graphics processing unit front - end for obtaining a rendering instruction for instructing to perform a first rasterization task and a first ray tracing task on geometric primitives within a viewport, where the geometric primitives serve as a part of the input for the first ray tracing task and the input for the first rasterization task, and the output of the first rasterization task serves as another part of the input for the first ray tracing task; A scheduler for determining M second rasterization tasks respectively according to M sub - viewports of the viewport, and determining M second ray tracing tasks respectively according to the M sub - viewports, where the output of the i - th second rasterization task serves as a part of the input for the i - th second ray tracing task, i ≤ M, M > 1, and i and M are positive integers; A rasterization pipeline for executing the M second rasterization tasks, and A ray tracing pipeline for executing the M second ray tracing tasks, where at least one of the second rasterization tasks is executed in parallel with at least one of the second ray tracing tasks. ' 7. The graphics processing unit according to claim 6, characterized in that, The scheduler includes a hybrid rendering task scheduling unit, and the hybrid rendering task scheduling unit is configured to schedule the execution period of the second rasterization task and the execution period of the second ray tracing task. Wherein, the execution period of the (j + 1)-th second rasterization task and the execution period of the j-th second ray tracing task at least partially overlap, j < M, and j is a positive integer.

8. The graphics processing unit according to claim 7, characterized in that, The hybrid rendering task scheduling unit is further configured to: At the completion moment of the execution period of the j-th second rasterization task, start the (j + 1)-th second rasterization task and the j-th second ray tracing task.

9. The graphics processing unit according to claim 7 or 8, characterized in that, The execution period of the j-th second ray tracing task and the execution period of the (j + 1)-th second ray tracing task at least partially overlap.

10. The graphics processing unit according to any one of claims 6 to 9, characterized in that, The second rasterization task corresponds to pixel candidate tiles obtained from the geometric primitives. The pixel candidate tiles are used to determine the color values of the sub-viewport corresponding to the second rasterization task and are stored in the on-chip cache. Wherein, the resolution of the sub-viewport is determined according to the capacity of the on-chip cache.

11. A computer device, characterized in that, The computer device includes at least one processor and at least one memory; the at least one memory stores at least one computer program, and the at least one computer program includes instructions. When the instructions are executed by the at least one processor, the method according to any one of claims 1 to 5 is executed.

12. The computer device according to claim 11, characterized in that, The at least one processor includes a graphics processing unit according to any one of claims 6 to 10.

13. A computer-readable storage medium, characterized in that, Computer instructions are stored in the computer-readable storage medium. When the computer instructions run on the computer device, the method according to any one of claims 1 to 5 is executed.

14. A computer program product containing instructions, characterized in that, When the instructions are run on the computer device, the computer device is caused to execute the method according to any one of claims 1 to 5.

Images