Method and apparatus for processing rendering task, graphic processing hardware and storage medium

Abstract

The application provides a processing method and device of a rendering task, graphics processing hardware and a storage medium. The method comprises the following steps: after receiving a rendering task, querying a hardware capability type of underlying hardware; receiving a plurality of calculation packets, and analyzing each calculation packet to obtain a corresponding task type; for each calculation packet, determining an execution unit for executing the calculation packet according to the task type and the hardware capability type; and scheduling the calculation packet to the corresponding execution unit, and processing a core instruction set in the calculation packet by the execution unit. In the application, the rendering task is not bound to a specific graphics unit, but the task is abstracted first, and then the task is scheduled to the best execution unit according to the hardware capability, the cooperation of pure calculation execution and hardware auxiliary execution is realized, the research and development and iteration risks caused by the split of software and hardware paths are avoided, the progressive evolution route of software first and hardware enhancement is supported, and the stability and compatibility of the upper layer architecture are maintained.

Description

Technical Field

[0001] This application relates to the field of computer hardware technology, and more specifically, to a method, apparatus, graphics processing hardware, and storage medium for processing rendering tasks. Background Technology

[0002] Traditional graphics rendering typically relies on fixed or semi-fixed graphics hardware paths, such as dedicated units for rasterization, sampling, texture processing, or ray intersection. If the corresponding hardware is lacking, the upper-level rendering workflow often requires the development of a separate software path.

[0003] In existing technologies, software paths and hardware paths are usually designed and maintained separately, resulting in high development costs, long verification cycles, and ecosystem fragmentation. Furthermore, switching from software implementation to hardware implementation often requires modifying upper-level interfaces, recompiling programs, or refactoring execution models. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, graphics processing hardware, and storage medium for processing rendering tasks, so that software paths and hardware paths can coexist in the same system during graphics rendering, which facilitates subsequent continuous evolution.

[0005] In a first aspect, embodiments of this application provide a method for processing a rendering task, including: It receives multiple computation packets and parses each packet to obtain the corresponding task type; the computation packets are obtained by decomposing the rendering task. For each computation packet, the execution unit for executing the computation packet is determined based on the task type and hardware capability type; the execution unit can be a pure computation execution unit or a hardware-accelerated execution unit. The computation packet is scheduled to the corresponding execution unit, which then processes the core instruction set within the computation packet.

[0006] In this embodiment, the rendering task is not bound to a specific graphics unit. Instead, the task is first abstracted, and then the task is scheduled to the best execution unit based on the hardware capabilities. This method realizes the synergy between pure computational execution and hardware-assisted execution, avoids the R&D and iteration risks caused by the split between software and hardware paths, supports the gradual evolution path of software first and hardware enhancement, and maintains the stability and compatibility of the upper-level architecture.

[0007] In one possible implementation of the first aspect, scheduling the computation packet to the corresponding execution unit includes: If the execution unit is a pure computation execution unit, then the computation packet will be scheduled to the general-purpose computing core array; If the execution unit is a hardware-accelerated execution unit, then the computation packet will be scheduled to the hardware accelerator.

[0008] This application's embodiments automatically distribute computation packets to the corresponding general-purpose computing core array or hardware accelerator based on the execution unit type (pure computation / hardware acceleration). This eliminates the need for writing two separate sets of scheduling logic for different hardware in the upper-layer rendering or computation process. This allows software and hardware paths to share the same scheduling framework, avoiding the high costs and ecosystem fragmentation issues associated with separate design and maintenance in traditional solutions.

[0009] In one possible implementation of the first aspect, the method further includes: Upon receiving a rendering task, the underlying hardware capability type is queried; this hardware capability type includes pure computing architecture type and hardware-accelerated architecture type.

[0010] In one possible implementation of the first aspect, the hardware capability type of the underlying hardware is queried, including: Read the capability flag bit from the hardware capability register; The capability identifier is parsed, and the hardware capability type is obtained based on the parsing result.

[0011] This application embodiment can dynamically and accurately obtain the actual capability type of the underlying hardware (e.g., whether it supports specific acceleration instructions, dedicated units, or computing features) by reading and parsing the capability flag bits in the hardware capability register. This improves the portability and compatibility of software across hardware platforms, reduces development and maintenance costs caused by hardware differences, and fully leverages the potential performance of the hardware, avoiding regression to inefficient general paths or functional deficiencies due to unknown capabilities.

[0012] In one possible implementation of the first aspect, the method further includes: Both the pure computation execution unit and the hardware-accelerated execution unit write their respective execution results to the standard output area in the thread register; the location and format of the standard output area are consistent in both the pure computation execution unit and the hardware-accelerated execution unit.

[0013] In this embodiment, since both the pure computation execution unit and the hardware-accelerated execution unit write their respective execution results into a unified standard output area, the pure computation execution results and the hardware-accelerated execution results maintain the same format, thereby maintaining the stability of the upper-layer interface.

[0014] In one possible implementation of the first aspect, the execution results of the pure computational execution unit and the execution results of the hardware-accelerated execution unit are written back through a unified register interface.

[0015] This application embodiment uses a unified register interface to write back the execution result, so that the interface for reading the result at the upper layer remains consistent regardless of which type of execution unit performs the task at the lower layer, thus improving the stability of the upper layer interface.

[0016] In one possible implementation of the first aspect, the method further includes: Define the core control flow instruction set; the core control flow instruction set includes rasterized execution instructions, wait instructions, and signal-triggered instructions; For cases where the execution unit is a pure computational execution unit, the core control flow instruction set is simulated and executed through a general-purpose computing core array; In cases where the execution unit is a hardware-accelerated execution unit, the core control flow instruction set is executed through a hardware accelerator.

[0017] This application embodiment defines and locks the core control flow instruction set, allowing upper-layer computing packages to run on different architecture types without recompiling.

[0018] In one possible implementation of the first aspect, the core control flow instruction set remains binary compatible in both pure computational execution units and hardware-accelerated execution units.

[0019] This application embodiment maintains binary compatibility in pure computation execution units and hardware-accelerated execution units by defining a core control flow instruction set, so that upper-layer computation packages can run on different architecture types without recompiling.

[0020] In one possible implementation of the first aspect, the method further includes: In the process of executing computational packets, the pure computational execution unit and the hardware-accelerated execution unit realize data prefetching and transportation through loading thread bundles, and realize core computational tasks through computational thread bundles. The loading thread bundle and the computation thread bundle maintain a consistent coordination mode in both the pure computation execution unit and the hardware-accelerated execution unit.

[0021] In this embodiment, the loading thread bundle and the computing thread bundle maintain a consistent collaborative mode in the pure computing execution unit and the hardware-accelerated execution unit, so that the same task package maintains consistent scheduling semantics and portability across different generations of hardware.

[0022] In one possible implementation of the first aspect, the pure computation execution unit and the hardware-accelerated execution unit are controlled by the same scheduler and share a context. The intermediate results generated by the pure computation execution unit do not need to be written back to global memory and are directly sent to the hardware-accelerated execution unit. Different fields, data blocks, and processing stages of the same computational packet are executed in parallel by pure computational execution units and hardware-accelerated execution units, and then merged through a unified synchronization mechanism.

[0023] The embodiments of this application can also be used for deep integration of pure computational execution units and hardware-accelerated execution units, supporting more complex collaborative execution modes.

[0024] Secondly, embodiments of this application provide a processing apparatus for a rendering task, comprising: The hardware capability query module is used to query the hardware capability type of the underlying hardware after receiving a rendering task; the execution capability type includes pure computing architecture type and hardware accelerated architecture type. The parsing module receives multiple computation packets and parses each packet to obtain the corresponding task type; the computation packets are obtained by decomposing the rendering task. The execution unit determination module is used to determine the execution unit for each computation packet based on the task type and hardware capability type; the execution unit can be a pure computation execution unit or a hardware-accelerated execution unit. The scheduling module is used to schedule computation packets to the corresponding execution units, which then process the core instruction set in the computation packets.

[0025] Thirdly, embodiments of this application provide graphics processing hardware, including: a processing unit and a storage unit. The processing unit includes a scheduler, a general-purpose computing core array, and a hardware acceleration execution unit. The storage unit includes global memory and thread registers. The global memory stores computer program instructions, which are read and executed by the processing unit to perform the method of the first aspect.

[0026] Fourthly, embodiments of this application provide a non-transitory computer-readable storage medium, comprising: The non-transitory computer-readable storage medium stores computer instructions that cause the computer to perform the methods in various possible implementations of the first aspect.

[0027] Fifthly, embodiments of this application provide a computer program product, including computer program instructions, which, when read and executed by a processor, perform the methods in various possible implementations of the first aspect.

[0028] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings. Attached Figure Description

[0029] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 A schematic flowchart illustrating a rendering task processing method provided in an embodiment of this application; Figure 2 A software reconstruction logic diagram of a rasterized scene is provided in an embodiment of this application; Figure 3 A software reconstruction logic diagram of a ray tracing scene provided in an embodiment of this application; Figure 4 A schematic flowchart of another rendering task processing method provided in an embodiment of this application; Figure 5 A schematic diagram of a rendering task processing device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the physical structure of the graphics processing hardware provided in an embodiment of this application. Detailed Implementation

[0031] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification and the foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0033] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0034] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0035] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0036] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0037] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0038] In computer graphics, rendering refers to the process of converting a 3D model or scene into a 2D image. The various steps involved in rendering are linked together to form the Graphics Pipeline, which can be divided into four main stages: the application stage, the geometry stage, the rasterization stage, and the pixel processing stage. The application stage, executed on the CPU, mainly includes scene data preparation (e.g., acquiring geometric data, material data, lighting data, camera data, etc.), culling objects completely outside the camera's field of view or completely obscured by other objects, and setting the rendering state. At the end of this stage, the vertex data of all objects to be rendered is packaged and sent to the GPU's memory (VRAM) via the data bus. The geometry stage, executed on the GPU, is mainly responsible for processing vertices and shapes, implemented through vertex shaders, tessellation shaders, and geometry shaders, obtaining the final coordinates and depth values ​​of each vertex on the screen. The rasterization stage is used to convert continuous geometric primitives (such as triangles) into discrete pixel fragments on the screen. In the pixel processing stage, the fragment shader calculates the corresponding color value for each fragment received from the rasterization stage. Then, for each fragment, depth testing, stencil testing, alpha testing, and blending are performed. All fragment colors that pass the tests and blending are ultimately written to the frame buffer. The frame buffer is a region in video memory that stores the color value of each pixel to be displayed on the screen. Once the entire scene has been rendered, the display controller reads data from the frame buffer at a fixed refresh rate and sends it to the monitor, thus obtaining the image displayed on the screen.

[0039] To improve the performance of graphics rendering tasks, engineers are constantly researching and iterating on computing chips. However, during this process, they often face the choice between software and hardware implementations. In traditional solutions, software and hardware implementations often employ independent technical approaches, which leads to technical problems: (1) If hardware development is invested first, it will result in huge sunk costs if there are problems with the architecture design. If software is developed first, it will be difficult to make full use of the hardware performance advantages.

[0040] (2) The split between software and hardware paths leads to the need to maintain multiple code baselines, increasing R&D costs and causing a split in the user ecosystem.

[0041] (3) In traditional solutions, switching from software to hardware often requires recompiling or modifying the upper-layer application, resulting in a discontinuous evolution process and a poor user experience.

[0042] To address the aforementioned technical issues, this application provides a method for software and hardware co-evolution, which achieves intelligent routing through dynamic hardware capability querying and maintains the stability of the upper-layer architecture.

[0043] Figure 1 This application provides a flowchart illustrating a rendering task processing method, which includes: Step 101: Receive multiple computation packets and parse each computation packet to obtain the corresponding task type; the computation packets are obtained after decomposing the rendering task.

[0044] A computational packet is the smallest unit of work that can be independently scheduled, executed, and traced, obtained by decomposing a task to be executed (such as a rendering task) into a packet generation module in the compiler or driver. This is a key abstraction in the hardware-software co-design of this application embodiment. The upper-layer application, graphics interface, runtime system, or compiler front-end first submits the task to be executed. The packet generation module performs intermediate representation conversion and field encapsulation on the task to be executed to generate a standardized computational packet, which is then sent to the scheduler by a unified packet dispatcher.

[0045] It should be noted that, in order to maintain format consistency across different architecture types, a unified computation packet format, a unified intermediate representation format, and unified field semantics can be predefined, enabling different architecture types to receive tasks based on the same packet syntax. The underlying differences are not reflected in changing the packet format, but rather in selecting different execution units for interpretation and execution through hardware capability descriptors and dynamic routing mechanisms, thereby achieving format consistency of computation packets across different architecture types.

[0046] The aforementioned hardware capability descriptor is an identifier used to characterize the type of hardware capability. See step 103 for the dynamic routing mechanism.

[0047] The computation packet contains a task type identifier, which identifies the type of computation task and its dependencies. Therefore, by parsing the computation packet, the corresponding task type can be obtained.

[0048] Step 102: For each computation packet, determine the execution unit that executes the computation packet based on the task type and hardware capability type; the execution unit can be a pure computation execution unit or a hardware-accelerated execution unit.

[0049] After obtaining the task type and underlying hardware capability type of the computation packet, a corresponding execution unit can be matched for that computation packet. Specifically, a routing rule table, capability mapping relationship, or scheduling strategy can be pre-defined. Taking the routing rule table as an example, this routing rule table includes the correspondence between multiple task types, hardware capability types, and execution units. The parsed task type and the queried hardware capability type are matched with each item in the routing rule table to determine the execution unit that will execute the computation packet.

[0050] The execution unit can be a pure computational execution unit or a hardware-accelerated execution unit. A pure computational execution unit includes a general-purpose computing core array, while a hardware-accelerated execution unit can be a dedicated unit that specifically executes the computational packet, such as a rasterization dedicated unit.

[0051] Step 103: Schedule the computation packet to the corresponding execution unit, and the execution unit processes the core instruction set in the computation packet.

[0052] Based on the execution unit determined in the previous step, the scheduler schedules the computation packet to the corresponding execution unit, which then processes the core instruction set within the computation packet. Specifically, if the execution unit is a pure computation execution unit, the computation packet is scheduled to a general-purpose computing core array; if the execution unit is a hardware-accelerated execution unit, the computation packet is scheduled to a hardware accelerator.

[0053] The core instruction set is a predefined set of basic instructions with unified control semantics across organizational architectures. When upper-layer tasks (i.e., rendering tasks) are converted into a sequence of instructions in a unified intermediate representation format during the packet generation stage, the instructions related to task scheduling, stage switching, synchronization waiting, and trigger execution are encoded into corresponding instructions in the core control instruction set.

[0054] The following section uses rasterization and ray tracing as examples to introduce specific implementation methods in scenarios where the execution unit is the computational execution unit: (1) Software refactoring method for rasterization like Figure 2 As shown, in rasterized scenes, the steps that are traditionally completed by dedicated hardware, such as vertex processing, primitive processing, screen segmentation, coverage testing, attribute interpolation, depth comparison, fragment shading, and output write-back, are broken down into schedulable computational stages.

[0055] The scheduler selects execution resources based on the computational characteristics of each computational stage and in conjunction with hardware capability descriptors to schedule these computational stages. For discrete, comparison-based, and control-flow-intensive steps, such as post-vertex discrete processing, primitive processing, cover testing, attribute interpolation, depth comparison, fragment shading, atomic updates, and buffer management, the scheduler can route them to CUDA Cores for execution. For matrix-intensive steps such as batch matrix transformations, skeleton skinning, pose transformations, parameter pre-computation, and some post-processing matrix tasks, they can be routed to Tensor Cores for execution.

[0056] CUDA Core and Tensor Core write the processing results (color, depth, normals, and property buffers) back to the standard output area in a unified format.

[0057] Therefore, even if the chip does not have a dedicated rasterization unit in the traditional sense, the present invention can still complete the same rendering chain in the pure computation execution path and ensure that the output format is consistent with the subsequent hardware enhancement path.

[0058] (2) Software reconstruction method for ray tracing like Figure 3 As shown, in a ray tracing scene, the computation phase can be divided into steps such as: main ray generation, scene traversal, ray intersection, hit shading, secondary ray generation, result accumulation, and unified output. These steps are incorporated into a unified encapsulation and unified scheduling framework.

[0059] Similar to rasterization, the scheduler selects execution resources based on the computational characteristics of each computational stage and in conjunction with hardware capability descriptors to schedule these computational stages. For complex control flow steps such as master ray generation, scene traversal, ray intersection, hit filtering, shading, task queue advancement, and result accumulation, execution can be preferentially routed to CUDA Cores. For intensive tasks such as instance and ray packet transformation, batch preprocessing, denoising, reconstruction, and super-resolution and matrix augmentation, execution can be routed to Tensor Cores.

[0060] CUDA Core and Tensor Core write the processing results (hit information, color, depth, and intermediate attributes) back to the standard output area in a unified format.

[0061] For hardware-accelerated execution units (such as GPU rasterizers), the processing flow includes: the front end first parses the computation packet and loads data such as triangle vertices, indices, and rendering states; then the Setup Engine calculates the side functions, attribute plane equations, and screen space bounding boxes for each triangle, providing geometric parameters for subsequent rasterization; the Traversal Engine then traverses the coverage area based on scan lines or tile / block partitioning algorithms, and performs a coverage test with a basic granularity of 2×2 pixel quads to determine which pixels are covered by triangles; for pixels that pass the coverage test, the Interpolator uses barycentric coordinates for perspective correction interpolation to generate depth values ​​and fragment attributes such as color and texture coordinates; then the Rasterizer generates fragment / quad packets and sends them to the Fragment Shader or pixel processing pipeline to perform texture sampling, lighting, and other calculations; the back end ROP (Raster Operation Processor) unit further completes Early-Z / Depth Test, Stencil Test, Blending, and other operations, and writes the final color and depth results to the Framebuffer or on-chip tile. Buffer; After all triangles in the current batch have been processed, the hardware updates the completion count, fence, or event signal to notify the scheduler or CPU that the current rendering task has been completed.

[0062] In this embodiment, the rendering task is not bound to a specific graphics unit. Instead, the task is first abstracted, and then the task is scheduled to the best execution unit based on the hardware capabilities. This method realizes the synergy between pure computational execution and hardware-assisted execution, avoids the R&D and iteration risks caused by the split between software and hardware paths, supports the gradual evolution path of software first and hardware enhancement, and maintains the stability and compatibility of the upper-level architecture.

[0063] Prior to step 101, the following steps may also be included: Step 100: After receiving the rendering task, query the hardware capability type of the underlying hardware; where the hardware capability type includes pure computing architecture type and hardware acceleration architecture type.

[0064] Rendering tasks can be specifically categorized as rasterization tasks, ray tracing tasks, and computational shader tasks. The underlying hardware refers to the computational execution hardware platform itself that carries the methods of the embodiments of this application, that is, the chip or processor architecture where the task actually runs. Different chips possess different hardware capabilities; for example, some chips only have pure computational capabilities, while others, in addition to pure computational capabilities, also possess hardware acceleration capabilities. Therefore, hardware capability types can include pure computational architecture types and hardware acceleration architecture types.

[0065] Among them, the pure computational architecture type is based on massive parallel throughput and is commonly found in GPGPUs. It excels at handling rendering processes with low correlation between data, and performs related computational tasks through CUDA Cores and Tensor Cores. CUDA Cores can perform tasks such as discretization, branching, comparison, and queue advancement. Tensor Cores can perform tasks such as matrix transformation, batch processing, reconstruction, and denoising.

[0066] Hardware-accelerated architectures refer to architectures that include modules designed for specific tasks. They are commonly found in dedicated units within GPUs and in AI-specific chips. Examples of dedicated units include rasterization units, sampling units, and texture units. These dedicated units are used to dynamically accelerate parts of the computational packets.

[0067] By querying the types of hardware capabilities possessed by the underlying hardware, a basis is provided for scheduling subsequent tasks.

[0068] It should be noted that the step of querying the hardware capability type of the underlying hardware can be performed not only after receiving the rendering task, but also when the system starts up or when hardware hot-plugging is detected.

[0069] Based on the above embodiments, the hardware capability type can be obtained by querying in the following way: Hardware capability type identification relies on parsing the capability flag bits in the hardware capability register to determine whether the underlying hardware contains at least one of the following: a rasterization acceleration unit, a sampling acceleration unit, and a texture processing unit. This process consists of two key steps: first, reading the binary flag bits stored in the register, which are written by the hardware during startup or initialization, reflecting the supported functional modules (such as rasterization, ray tracing, tensor computation, texture processing, etc.); then, decoding these bits using predefined mapping rules to determine which hardware capabilities are activated.

[0070] For example, if one bit is 1, it indicates support for a rasterization acceleration module; if another bit is 1, it indicates the presence of a dedicated ray tracing unit. Based on the combination of these bits, the system dynamically identifies whether the current hardware is graphics rendering-oriented, general-purpose computing-oriented, or a hybrid capability. This mechanism allows the software layer to intelligently schedule tasks based on the actual hardware capabilities. On the A100 / H100, which lacks a dedicated module, graphics rendering tasks can be decomposed into subtasks executable by the general-purpose computing unit; while on the 4090, which has a dedicated module, the hardware acceleration path is directly invoked, achieving optimal performance and energy efficiency.

[0071] This application embodiment can dynamically and accurately obtain the actual capability type of the underlying hardware (e.g., whether it supports specific acceleration instructions, dedicated units, or computing features) by reading and parsing the capability flag bits in the hardware capability register. This improves the portability and compatibility of software across hardware platforms, reduces development and maintenance costs caused by hardware differences, and fully leverages the potential performance of the hardware, avoiding regression to inefficient general paths or functional deficiencies due to unknown capabilities.

[0072] Building upon the above embodiments, after the pure computation execution unit and the hardware-accelerated execution unit process the received computation packets and obtain the corresponding processing results, in order to achieve collaborative scheduling and result aggregation across different execution units, the output results of both pure computation execution units (such as general-purpose CUDA cores) and dedicated hardware-accelerated execution units (such as rasterization, tensor, or texture processing modules) are written to the standard output area in the thread register through a unified register interface. This area maintains complete consistency across all execution units in terms of physical address, data structure, and data format, ensuring that regardless of which unit completes the task, its output can be read and parsed without discrimination by the subsequent scheduler or fusion module.

[0073] For example, the intermediate color values, transparency, and illumination intensity of a rasterization task, and the output vector of a tensor computation task, are all written to a specified register offset in a byte-aligned structure (such as four 32-bit floating-point numbers: R, G, B, and Alpha). This standardized design eliminates data compatibility barriers between heterogeneous units, enabling the system to uniformly manage task flows, merge results, and connect pipelines. This allows for efficient simulation of the entire rendering pipeline's execution even on A100 / H100s without dedicated graphics hardware or on A100 / H100s in general computing mode.

[0074] Based on the above embodiments, this application defines a core control flow instruction set to uniformly schedule various execution units to complete graphics rendering tasks. This core control flow instruction set includes three types of key instructions: rasterization execution instructions, used to trigger core rendering operations such as mesh mapping, interpolation, and shading; waiting instructions, used to synchronize the completion status of multiple parallel tasks; and signal trigger instructions, used to notify downstream modules that data is ready or to start the next stage of processing.

[0075] When the execution unit is a pure computational execution unit (such as a general-purpose CUDA core), these instructions are compiled by the software layer into a sequence of instructions executable by the general-purpose computing core array, simulating the rasterization process in parallel through multi-threading. For example, the pixel calculation task of a triangle can be broken down into multiple threads to perform interpolation, lighting calculation, and color mixing respectively, and the execution order of each stage can be coordinated by waiting and signal instructions to achieve a logically equivalent rendering pipeline.

[0076] When the execution unit is a hardware-accelerated execution unit (such as a dedicated rasterization or ray tracing module), the same core control flow instruction set is directly parsed and executed by the hardware, with high-speed parallel processing completed by the underlying circuitry, eliminating the need for software simulation. This design of homogeneous instructions and heterogeneous execution allows the same scheduling logic to run across different hardware platforms. Functionality is implemented through software simulation on the A100 / H100 without dedicated modules, while on newer chips with acceleration units, it is executed natively by hardware, thus supporting hardware-software co-evolution and ensuring the compatibility and scalability of future architectures.

[0077] In addition, the core control flow instruction set maintains binary compatibility in both pure computational execution units and hardware-accelerated execution units. This means that whether a task is simulated by a general-purpose computing core (such as CUDA threads) or executed by a dedicated hardware module (such as a rasterization engine), the instruction encoding format, opcode structure, and register addressing mode received and parsed are completely consistent.

[0078] This feature allows compilers or schedulers to generate only the same binary instruction stream, enabling seamless operation across different hardware environments. On the A100 / H100, which lacks dedicated graphics hardware, instructions are interpreted and simulated line by line by the general-purpose computing core; while on newer chips equipped with dedicated acceleration modules, the same binary instructions can be directly decoded and executed at high speed by the hardware, without the need for recompilation or conversion.

[0079] This binary compatibility eliminates the adaptation barriers between the software and hardware layers, and is a key foundation for achieving a single codebase that runs on multiple platforms. It not only simplifies the development and deployment process but also ensures semantic consistency of rendering logic across different generations of hardware. Regardless of what acceleration units are added in the future, as long as they conform to this instruction set specification, they can be directly supported by the existing software architecture, greatly improving the flexibility of system evolution and long-term maintainability.

[0080] Based on the above embodiments, both the pure computation execution unit and the hardware-accelerated execution unit achieve efficient parallel scheduling of tasks through a warp mechanism. The warp can be divided into a loading warp and a computation warp. The loading warp is responsible for data prefetching and memory movement, while the computation warp focuses on core computational tasks (such as interpolation, lighting calculation, color mixing, etc.). The two are strictly decoupled and work together in the execution process.

[0081] Whether simulated on a general-purpose computing core or natively running in a dedicated hardware accelerator, the coordination mode of these two thread bundles remains completely consistent. That is, the loading thread bundle reads data such as mesh vertices, texture coordinates, and lighting parameters from global memory or texture cache in advance and moves them to shared memory or registers. The compute thread bundle starts after the data is ready and uses this prefetched data to perform high-density computation, during which synchronization barriers ensure that data dependencies are correct.

[0082] This unified coordination mechanism eliminates the need for the software layer to refactor task scheduling logic for different hardware. For example, on the A100 / H100, multiple CUDA thread bundles are scheduled to simulate the separation of loading and computation. On chips with dedicated rendering modules, the hardware automatically allocates physical thread bundles to perform the same tasks. Thus, regardless of whether the underlying hardware has dedicated graphics units, the rendering pipeline can run with the same concurrency structure, memory access mode, and task partitioning granularity, ensuring the predictability and portability of cross-platform performance.

[0083] Based on the above embodiments, this application embodiment can further integrate pure computational execution units (such as general-purpose CUDA cores) with hardware-accelerated execution units (such as rasterization and tensor processing modules) to form a unified heterogeneous computing pipeline. Its core functions include: (1) Unified scheduling and context sharing; A single scheduler manages instruction dispatch and resource allocation for both types of execution units, ensuring seamless task flow switching between hardware and software. Execution contexts (such as register states, memory address mappings, task dependencies, and thread bundle identifiers) are shared between the two types of units, avoiding redundant initialization overhead.

[0084] (2) Direct transmission of intermediate results; Intermediate data generated by pure computational execution units (such as vertex coordinates and lighting parameters) do not need to be written back to global memory. Instead, they are directly input to hardware-accelerated execution units via high-speed interconnect channels (such as NVLink or on-chip networks). For example, the vertex attributes of a mesh calculated by the CUDA core can be directly passed to the hardware rasterization module, reducing memory bandwidth usage and latency.

[0085] (3) Parallel execution and stage fusion; Different fields (such as color and transparency), data blocks (such as mesh triangles), or processing stages (such as interpolation → blending) within the same computational packet can be executed in parallel by two types of units and merged through a unified synchronization mechanism. For example, while the hardware-accelerated execution unit processes pixel shading, the pure computational execution unit synchronously executes vertex calculations for subsequent layers, ensuring data consistency between stages through a unified synchronization mechanism (such as barrier instructions).

[0086] (4) Unified synchronization mechanism; By sharing semaphores, atomic operations, and barrier instructions, the execution order and data visibility between hardware and software accelerated execution units are ensured. For example, during the blending phase, the hardware accelerated execution unit waits for the pure computational execution unit to complete the transparency sorting before starting pixel compositing, avoiding rendering errors caused by data races.

[0087] On platforms like the A100 / H100 that lack dedicated graphics hardware, key stages are simulated by pure computational execution units, combined with programmable fusion logic, achieving rendering efficiency close to hardware acceleration. On newer chips, hardware-accelerated execution units can dynamically take over high-frequency computational tasks (such as ray tracing), while complex logic is still flexibly handled by the software layer, achieving a balance between performance and flexibility.

[0088] Figure 4 This is a schematic flowchart illustrating another rendering task processing method provided in this application embodiment. This application embodiment first queries the underlying hardware execution capabilities before task scheduling to generate a capability descriptor, used to identify whether the current platform possesses pure computational execution capabilities, rasterization-related capabilities, sampling and texture processing capabilities, intersection-related capabilities, or matrix computation enhancement capabilities. Based on this, the upper-layer rendering tasks (including rasterization tasks and ray tracing tasks) are transcribed into standardized computational packets. These computational packets are not pre-bound to any specific hardware but are a unified intermediate representation, including task type representation, input / output addresses, dependencies, and execution parameters.

[0089] The scheduler dynamically determines whether a computation packet should enter a pure computation execution path or a hardware-enhanced execution path based on the task type and hardware capability type.

[0090] In this model, the pure computation path can be handled by CUDA Cores for tasks such as discretization, branching, comparison, and queue advancement, while Tensor Cores can handle tasks such as matrix transformation, batch processing, and reconstruction enhancement. The hardware enhancement path accelerates steps with dedicated units, such as rasterization, sampling, texturing, and intersection, and can dynamically accelerate some computational packets.

[0091] Regardless of the execution resource used to perform the computation packet, the results (including color, depth, normals, hit information, and intermediate attributes) are written back to the standard output area in a unified format through the unified register interface, thereby maintaining the stability of the upper-layer interface.

[0092] Therefore, this application's embodiments introduce dynamic hardware capability querying before scheduling, without pre-setting a unique rendering execution path. Instead, it first identifies the execution resources available to the underlying platform and then determines the execution method. Traditional rendering tasks are uniformly abstracted into standardized computational packets, allowing the same task to be reused across pure computational architectures, hardware-accelerated architectures, and fusion execution architectures where both are configured collaboratively. The dynamic routing mechanism is based on a joint determination of task type and hardware capability type, rather than simply mapping all tasks to a single type of hardware. Result write-back is achieved through a unified register interface or a unified output buffer, ensuring that the format of pure computational execution results and hardware-accelerated execution results remains consistent. Through control flow locking and thread bundle specialization, the same task packet maintains consistent scheduling semantics and portability across different generations of hardware.

[0093] In short, this application's embodiments use dynamic capability querying as the entry point, standardized computational packets as the unified task expression, dynamic routing as the core decision-making mechanism, unified write-back as the result consistency guarantee, and control flow locking and thread bundle coordination to ensure cross-generational compatibility. Furthermore, this application's embodiments establish a unified architecture oriented towards generational evolution, rather than a replacement of a single algorithm or a single execution kernel. It is this unified architecture that allows pure computational paths and dedicated hardware paths to coexist and evolve continuously within the same system.

[0094] Figure 5 This is a schematic diagram of a rendering task processing device provided in an embodiment of this application. The device can be a module, program segment, or code on an electronic device. It should be understood that this device is similar to the one described above. Figure 1 The method implementation corresponds to this and can be executed. Figure 1 The specific functions of the device involved in the various steps of the method embodiment can be found in the description above; to avoid repetition, detailed descriptions are appropriately omitted here. The device includes: a parsing module 501, an execution unit determination module 502, and a scheduling module 503, wherein: The parsing module 501 is used to receive multiple computation packets and parse each computation packet to obtain the corresponding task type; the computation packet is obtained by decomposing the rendering task. The execution unit determination module 502 is used to determine, for each computation packet, the execution unit to execute the computation packet according to the task type and the hardware capability type; the execution unit is a pure computation execution unit and a hardware-accelerated execution unit; The scheduling module 503 is used to schedule the computation packet to the corresponding execution unit, and the execution unit processes the core instruction set in the computation packet.

[0095] Based on the above embodiments, the scheduling module 503 is specifically used for: If the execution unit is a pure computation execution unit, then the computation packet is scheduled to the general-purpose computing core array; If the execution unit is a hardware-accelerated execution unit, then the computation packet is scheduled to the hardware accelerator.

[0096] Based on the above embodiments, the device further includes a hardware capability query module, used to query the hardware capability type of the underlying hardware after receiving a rendering task; the hardware capability type includes pure computing architecture type and hardware acceleration architecture type.

[0097] Based on the above embodiments, the hardware capability query module is specifically used for: Read the capability flag bit from the hardware capability register; The capability identifier is parsed, and the hardware capability type is obtained based on the parsing result.

[0098] Based on the above embodiments, the device further includes a write-back module, used for: Both the pure computation execution unit and the hardware-accelerated execution unit write their respective execution results into the standard output area of ​​the thread register; the position and format of the standard output area are consistent in both the pure computation execution unit and the hardware-accelerated execution unit.

[0099] Based on the above embodiments, the execution results of the pure computational execution unit and the execution results of the hardware-accelerated execution unit are written back through a unified register interface.

[0100] Based on the above embodiments, the device further includes an instruction locking module, used for: Define a core control flow instruction set; the core control flow instruction set includes rasterized execution instructions, wait instructions, and signal trigger instructions; When the execution unit is a pure computational execution unit, the core control flow instruction set is simulated and executed through the general-purpose computing core array; When the execution unit is a hardware-accelerated execution unit, the core control flow instruction set is executed through the hardware accelerator.

[0101] Based on the above embodiments, the core control flow instruction set remains binary compatible in both the pure computational execution unit and the hardware-accelerated execution unit.

[0102] Based on the above embodiments, the device further includes a thread beam control module, used for: During the execution of computational packets, the pure computational execution unit and the hardware-accelerated execution unit respectively implement data prefetching and transportation by loading thread bundles, and implement core computational tasks by computing thread bundles. The loading thread bundle and the computing thread bundle maintain the same cooperative mode in the pure computing execution unit and the hardware-accelerated execution unit.

[0103] Based on the above embodiments, the pure computation execution unit and the hardware accelerated execution unit are controlled by the same scheduler and share the same context. Furthermore, the intermediate results generated by the pure computation execution unit do not need to be written back to global memory and are directly sent to the hardware accelerated execution unit. Different fields, data blocks, and processing stages of the same computational packet are executed in parallel by the pure computational execution unit and the hardware-accelerated execution unit, and merged through a unified synchronization mechanism.

[0104] Figure 6 This is a schematic diagram of the physical structure of the graphics processing hardware provided in the embodiments of this application, such as... Figure 6 As shown, the graphics processing hardware includes a processing unit 601 and a storage unit 602. The processing unit 601 can be used to read and run computer program instructions to implement the methods described in the above embodiments, while the storage unit 602 can be used to store computer program instructions and data, and may include volatile and / or non-volatile storage media.

[0105] The processing unit 601 may further include a scheduler 6011, a general-purpose computing core array 6012, and a hardware-accelerated execution unit 6013, while the storage unit 602 may further include global memory 6021, shared memory of the thread cluster 6022, shared memory of the thread block 6023, and thread registers 6024.

[0106] Specifically, global memory 6021 is memory shared by all threads in the graphics processing hardware; thread cluster shared memory 6022 is memory shared by threads within a thread cluster, with each thread cluster having its own dedicated shared memory allocation; thread block shared memory 6023 is memory shared by threads within a thread block, with each thread block having its own dedicated shared memory allocation; and thread registers 6024 are storage units exclusively for each thread, with each thread having its own set of registers. A cache can also be set in global memory 6021 or shared memory to improve data access efficiency.

[0107] Global memory 6021 can store computer program instructions, and the computer program formed by these instructions can be executed. Figure 1 The methods for handling rendering tasks (and their various possible implementations) in graphics processing hardware, these programs executed by the graphics processing hardware, can also be called kernel programs. Figure 1 The rendering task can be implemented using a single kernel program.

[0108] Scheduler 6011 can be used to read computer program instructions from global memory 6021, perform instruction decoding (i.e., parse out what the current instruction is), and schedule the decoded instructions to a general-purpose computing core array or hardware-accelerated execution unit.

[0109] This embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute the methods provided in the above-described method embodiments, such as: after receiving a rendering task, querying the hardware capability type of the underlying hardware; the hardware capability type includes a pure computing architecture type and a hardware-accelerated architecture type; receiving multiple computing packets and parsing each computing packet to obtain the corresponding task type; the computing packet is obtained by decomposing the rendering task; for each computing packet, determining the execution unit to execute the computing packet according to the task type and the hardware capability type; the execution unit is a pure computing execution unit and a hardware-accelerated execution unit; scheduling the computing packet to the corresponding execution unit, and having the execution unit process the core instruction set in the computing packet.

[0110] This embodiment provides a non-transitory computer-readable storage medium storing computer instructions that cause the computer to execute the methods provided in the above-described method embodiments. For example, the instructions include: upon receiving a rendering task, querying the hardware capability type of the underlying hardware; the hardware capability type includes a pure computing architecture type and a hardware-accelerated architecture type; receiving multiple computing packets and parsing each computing packet to obtain a corresponding task type; the computing packets are obtained by decomposing the rendering task; for each computing packet, determining an execution unit to execute the computing packet based on the task type and the hardware capability type; the execution unit is a pure computing execution unit or a hardware-accelerated execution unit; scheduling the computing packet to the corresponding execution unit, whereby the execution unit processes the core instruction set in the computing packet.

[0111] This application also provides an electronic device that includes the graphics processing hardware described in the above embodiments. For example, the electronic device may be a server or server cluster, a PC, a mobile phone, a robot, etc.

[0112] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0113] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0114] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0115] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0116] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for processing rendering tasks, characterized in that, include: Receive multiple computation packets and parse each computation packet to obtain the corresponding task type; The computational packet is obtained by decomposing the rendering task; For each computation packet, an execution unit is determined based on the task type and the hardware capability type of the underlying hardware; the execution unit can be a pure computation execution unit or a hardware-accelerated execution unit. The computation packet is scheduled to the corresponding execution unit, which then processes the core instruction set within the computation packet.

2. The method according to claim 1, characterized in that, The step of scheduling the computation packet to the corresponding execution unit includes: If the execution unit is a pure computation execution unit, then the computation packet is scheduled to the general-purpose computing core array; If the execution unit is a hardware-accelerated execution unit, then the computation packet is scheduled to the corresponding hardware accelerator.

3. The method according to claim 1, characterized in that, The method further includes: Upon receiving a rendering task, the underlying hardware capability type is queried; the hardware capability type includes pure computing architecture type and hardware acceleration architecture type.

4. The method according to claim 3, characterized in that, The type of hardware capability of the underlying hardware being queried includes: Read the capability flag bit from the hardware capability register; The capability identifier is parsed, and the hardware capability type is obtained based on the parsing result.

5. The method according to claim 1, characterized in that, The method further includes: Both the pure computation execution unit and the hardware-accelerated execution unit write their respective execution results into the standard output area of ​​the thread register; the position and format of the standard output area are consistent in both the pure computation execution unit and the hardware-accelerated execution unit.

6. The method according to claim 1, characterized in that, The execution results of the pure computational execution unit and the execution results of the hardware-accelerated execution unit are written back through a unified register interface.

7. The method according to claim 2, characterized in that, The method further includes: Define a core control flow instruction set; the core control flow instruction set includes rasterized execution instructions, wait instructions, and signal trigger instructions; When the execution unit is a pure computational execution unit, the core control flow instruction set is simulated and executed through the general-purpose computing core array; When the execution unit is a hardware-accelerated execution unit, the core control flow instruction set is executed through the hardware accelerator.

8. The method according to claim 7, characterized in that, The core control flow instruction set remains binary compatible in both the pure computational execution unit and the hardware-accelerated execution unit.

9. The method according to claim 1, characterized in that, The method further includes: During the execution of computational packets, the pure computational execution unit and the hardware-accelerated execution unit respectively implement data prefetching and transportation by loading thread bundles, and implement core computational tasks by computing thread bundles. The loading thread bundle and the computing thread bundle maintain the same cooperative mode in the pure computing execution unit and the hardware-accelerated execution unit.

10. The method according to any one of claims 1-9, characterized in that, The pure computation execution unit and the hardware-accelerated execution unit are controlled by the same scheduler and share the same context. The intermediate results generated by the pure computation execution unit do not need to be written back to global memory and are directly sent to the hardware-accelerated execution unit. Different fields, data blocks, and processing stages of the same computational packet are executed in parallel by the pure computational execution unit and the hardware-accelerated execution unit, and merged through a unified synchronization mechanism.

11. A processing apparatus for a rendering task, characterized in that, include: The parsing module is used to receive multiple computation packets and parse each computation packet to obtain the corresponding task type; The computational packet is obtained by decomposing the rendering task; An execution unit determination module is used to determine, for each computation packet, the execution unit to execute the computation packet based on the task type and the hardware capability type of the underlying hardware; the execution unit is a pure computation execution unit or a hardware-accelerated execution unit. The scheduling module is used to schedule the computation packet to the corresponding execution unit, and the execution unit processes the core instruction set in the computation packet.

12. A graphics processing hardware, characterized in that, include: The processing unit includes a scheduler, a general-purpose computing core array, and a hardware-accelerated execution unit; the storage unit includes global memory and thread registers. The global memory stores computer program instructions, which are read and executed by the processing unit to perform the method as described in any one of claims 1-10.

13. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions, which, when executed by a computer, cause the computer to perform the method as described in any one of claims 1-10.

14. A computer program product, characterized in that, It includes computer program instructions, which, when read and executed by a processor, perform the method as described in any one of claims 1-10.

Images