Light stack manager and graphic processing method and system
By introducing a ray stack manager into the ray tracing technology, optimizing ray traversal with parallel management units and stack block structures, the problem of insufficient existing ray tracing performance is solved and more efficient ray tracing performance is achieved.
Patent Information
- Application Number
- CN202310649821.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-31
- Publication Date
- 2024-12-03
Smart Images

Figure CN119068099A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of image processing technology, and more specifically, relates to a ray stack manager, a graphics processing method and a system. Background Art
[0002] In the field of computer graphics, a graphics processing unit (GPU) is a dedicated chip that performs functions related to rendering images to a screen. User-created applications can connect to the GPU through a graphics application programming interface (API). Commonly used graphics APIs include OpenGL, Vulkan, and Direct X.
[0003] Currently, some APIs have been extended to support a rendering technique called ray tracing. Ray tracing itself is not a new technology, but only in the past 10 years have consumer-grade GPUs been produced that specifically support ray tracing capabilities. Ray tracing is a graphics technique in which primitives (usually triangles) that describe a scene are intersected with one or more lines or line segments (called rays) to determine which primitives each ray intersects, and usually also find the primitive that is closest to a point on the ray (called the ray origin). These queries for finding primitives that rays "-intersect" are very useful in graphics because rays can be used to model light in a scene, so the results can be used to determine how light effects should be handled in scene rendering.
[0004] To avoid testing every ray against every triangle in the scene, primitives are typically grouped together in layers using an acceleration structure, often referred to as a "tree". The ray will first test for intersection with the root node of the acceleration structure, which represents a volume covering the entire scene. If and only if the ray intersects the volume, it can then move down the tree structure, testing the direct children of the root node. Each of these child nodes represents a volume covering a smaller portion of the scene, so as the ray traverses down the tree structure, it will intersect with smaller and smaller volumes. Eventually, it will reach the leaf level of the tree structure and directly test a small number of primitives. Traversing the entire tree structure in this way will know which primitives the ray intersects with, without having to test each primitive individually.
[0005] As the complexity of graphics processing content continues to increase, it is necessary to further improve the performance of ray tracing. Summary of the invention
[0006] In response to the need for improvement in the prior art, the present invention provides a ray stack manager, a graphics processing method and a system, which have good stack management capabilities and can improve the parallelism of ray traversal, thereby effectively improving the performance of ray tracing.
[0007] To achieve the above-mentioned purpose, according to one aspect of the present invention, a light stack manager is provided, comprising a plurality of management units, each of which comprises a stack storage module and an access logic module, the stack storage module being used to store a stack corresponding to a light ray, the access logic module being used to receive a stack request, and manage the stack in the stack storage module corresponding to it according to the stack request, wherein each light ray corresponds to a stack, and each of the stacks stores the traversal status of the corresponding light ray.
[0008] In some embodiments, each of the stacks includes at least one stack block, each of the stack blocks includes a plurality of entries, and stack blocks in different stacks have the same size.
[0009] In some embodiments, each of the access logic units includes a front-end processing module, which is used to receive the stack request and classify the stack request according to the stack status in the stack storage module; each of the access logic units also includes multiple processing modules, and the multiple processing modules are used to respond to the stack request according to the classification of the front-end processing module.
[0010] In some embodiments, the plurality of processing modules include a first processing module, and when the stack block has more than one entry, in response to a pop stack request, the first processing module is used to control popping the entry at the top of the stack.
[0011] In some embodiments, the plurality of processing modules include a second processing module, and when the stack block has only one entry, in response to a stack pop request, the second processing module is used to control popping the entry at the top of the stack block and control prefetching operations on the stack blocks below the stack block.
[0012] In some embodiments, the plurality of processing modules include a third processing module, and when the stack request is a push request and the current stack block overflows, the third processing module is used to control writing the current stack block back to the memory and updating the current stack block with new data.
[0013] In some implementations, the new data is data in a stack block next to the current stack block.
[0014] In some embodiments, the plurality of processing modules include a fourth processing module, and when the stack request is a push request and the current stack block does not overflow, the fourth processing module is used to control updating of the current stack block.
[0015] In some embodiments, each of the access logic units further includes a stack size storage module, which is used to store the size of the stack, and the front-end processing module determines whether the stack block overflows or underflows based on the stack size information stored in the stack size storage module.
[0016] In some implementations, each of the access logic units further includes a row status storage module, and the row status storage module is used to store a status of a stack block to determine whether the stack block is waiting for row filling.
[0017] In some embodiments, the row status storage module is further used to track a dirty bit of each stack block, and write the data in the stack block back to the memory when the stack block has a dirty bit, wherein the dirty bit is used to indicate whether the data in the stack block is overwritten.
[0018] In some embodiments, each of the access logic units further includes a delay request module, which is used to store the stack request when the stack block corresponding to the stack request waits for row filling; the delay request module is also used to suspend the input interface of the light stack manager when it is full.
[0019] In some embodiments, the plurality of processing modules are further configured to update the stack blocks in the stack storage module through an arbitrator, and to generate a row fill request and / or a write back request to be sent to a memory.
[0020] In some implementations, the stack storage module is a dual-port static random access memory, and each of the stack entries includes a pointer to at least one node that the corresponding ray needs to traverse, and the node is determined according to an acceleration structure of ray tracing.
[0021] In some implementations, the ray stack manager further includes a memory interface multiplexed by a plurality of the management units, and configured to perform data interaction with a memory, wherein the data interaction includes line filling and write back.
[0022] According to another aspect of the present invention, a ray tracing kernel is provided, comprising a ray stack manager as described above, the ray tracing kernel further comprising a ray scheduling module, a ray intersection module and a ray traversal module, the ray scheduling module being used to generate a ray index, and select a ray that enters an intersection test from a plurality of rays, the ray intersection module being used to perform a ray-primitive bounding box and / or ray-polygon intersection test, the ray traversal module being used to send a stack request to the ray stack manager to determine the next node that needs to be traversed by each ray that passes the intersection test, the ray stack manager being used to receive the stack request and manage the stack state of each ray.
[0023] In some embodiments, the ray tracing kernel also includes a ray data storage module, which is used to store information of multiple rays during ray traversal, wherein the information includes at least one of the ray origin, ray direction, ray minimum distance, ray maximum distance, and the ray hitting the nearest primitive, or a combination thereof.
[0024] In some embodiments, the ray tracing kernel also includes a node data acquisition module, which is used to obtain node information of the acceleration structure from the hierarchical memory and to send the ray index and the node information received from the ray scheduling module to the ray intersection module.
[0025] In some embodiments, the ray intersection module includes a conversion unit for converting the ray from a world space coordinate system to a model space coordinate system.
[0026] In some implementations, the ray traversal module is further configured to send a pointer back to the ray scheduling module after determining the next node that each ray that passes the intersection test needs to traverse.
[0027] In some implementations, the ray tracing kernel processes multiple rays in parallel during each cycle.
[0028] According to another aspect of the present invention, there is provided a graphics processing system, comprising the ray tracing kernel as described above, the graphics processing system further comprising a plurality of shader kernels, the plurality of the shader kernels being used to call a ray tracing function, sending initialization information required for ray traversal to the ray tracing kernel, the ray tracing kernel being used to perform ray traversal according to the initialization information and return a result, and the plurality of the shader kernels being further used to perform lighting calculations in rendering according to the returned result.
[0029] According to another aspect of the present invention, there is provided a graphics processing method, comprising:
[0030] Using the ray stack manager in the ray tracing kernel as described above to maintain a stack for each ray to be traversed, so as to store the traversal state information of each ray;
[0031] Determine the next node that each light ray needs to traverse according to the stack request and the traversal state information until the traversal of all light rays is completed and the traversal results are determined;
[0032] The traversal result is returned to the shader kernel for subsequent rendering.
[0033] According to another aspect of the present invention, there is provided an electronic device, comprising the above-mentioned ray stack manager, or the above-mentioned ray tracing kernel, or the above-mentioned graphics processing system; or, the electronic device comprises: a processor; a memory communicatively connected to the processor; the memory stores instructions executable by the processor, and the instructions are executed by the processor so that the processor can execute the above-mentioned method.
[0034] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the above method is implemented.
[0035] In general, the above technical solution conceived by the present invention has the following beneficial effects compared with the prior art: a stack is managed for each ray through multiple parallel processing units, and each stack is divided into multiple stack blocks, the top stack block of each ray is kept in the stack storage module of the processing unit, and multiple rays can be processed in parallel, thereby effectively managing multiple stacks, improving the efficiency of ray traversal processing, and optimizing the performance of ray tracing. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 is a schematic diagram of a group of triangles in a two-dimensional coordinate system according to an embodiment of the present invention;
[0037] Figure 2 The embodiment of the present invention is Figure 1 Topological view in the same acceleration structure;
[0038] Figure 3 is a schematic diagram of the structure of a light stack manager according to an embodiment of the present invention;
[0039] Figure 4 is a schematic diagram of a stack block according to an embodiment of the present invention;
[0040] Figure 5 is a schematic diagram of the structure of a ray tracing kernel according to an embodiment of the present invention;
[0041] Figure 6 is a schematic diagram of the structure of a graphics processing system according to an embodiment of the present invention;
[0042] Figure 7 is a flowchart of a graphics processing method according to an embodiment of the present invention;
[0043] Figure 8 It is a structural block diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0044] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. As those skilled in the art can appreciate, the described embodiments can be modified in various ways without departing from the spirit or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.
[0045] Both Vulkan and Direct X have been extended to support ray tracing, and they both use similar frameworks to do so. Ray tracing functionality is roughly divided into two parts - building the ray tracing acceleration structure and traversing the acceleration structure. The following architecture only involves the latter traversal of the acceleration structure, and assumes that the acceleration structure can be created on a general-purpose processor or another undisclosed specialized processor.
[0046] In the prior art, acceleration structures are used to reduce the number of ray-primitive intersection tests that need to be performed to determine which primitives a ray intersects with. A scene can be defined by a set of primitives, and for ease of description, it is assumed that these primitives are triangles. However, it will be understood that the present invention is not limited to this. A triangle is given by three points in a three-dimensional coordinate system. A ray is a ray segment in a three-dimensional coordinate system, given by a point (the ray origin O), a vector (the ray direction D) and two scalars (the minimum distance and the maximum distance). A point along such a ray is given by P=O+Dt, where t must be between the minimum distance and the maximum distance.
[0047] Figure 1 The figure shows a schematic diagram of a group of triangles in a two-dimensional coordinate system. Figure 1 As shown, multiple triangles are grouped hierarchically and strictly bounded by axis-aligned bounding boxes (AABBs). Figure 1 In the figure, 11 triangles are divided into 4 groups and defined by 4 primitive bounding boxes. Specifically, primitive bounding box 1 includes triangles A, B, C and D, primitive bounding box 2 includes triangles E and F, primitive bounding box 3 includes triangles G and H, and primitive bounding box 4 includes triangles I, J and K. Figure 2 Shown with Figure 1Topological view in the same acceleration structure. In the embodiment of the present invention, the axis-aligned primitive bounding box is represented by a square (also called a node) in the tree structure, and the arrow from one figure to another represents the parent-child relationship. A node may have multiple child nodes (up to 4 child nodes in this example), and the primitive bounding boxes of all child nodes should be contained in the primitive bounding box of the parent node. Specifically, node 1 corresponds to Figure 1 The primitive bounding box 1 shown in the figure has child nodes 5 to 8 corresponding to triangles A, B, C and D respectively; node 2 corresponds to Figure 1 The primitive bounding box 2 shown in the figure has child nodes 9 to 10 corresponding to triangles E and F respectively; node 3 corresponds to Figure 1 The primitive bounding box 3 shown in the figure has child nodes 11 to 12 corresponding to triangles G and H respectively; node 4 corresponds to Figure 1 The primitive bounding box 4 shown has child nodes 13 to 15 corresponding to triangles I, J and K respectively.
[0048] The ray tracing architecture can use stack traversal Figure 1 The acceleration structure shown in Figure 1 is used to implement a depth-first search. First, the ray is tested to see if it intersects the root node 0. If the ray is determined to intersect the root node 0, references to its children 1, 2, 3, and 4 are pushed onto the stack. Note that children 1, 2, 3, and 4 can be stored contiguously in memory, so only one reference to the beginning of the group is actually required. Next, the top entry in the stack is popped to obtain the primitive bounding box for the next intersection test. Children 1, 2, 3, and 4 are obtained and tested for intersection. Primitive bounding boxes that intersect the ray push their children onto the stack, while primitive bounding boxes that do not intersect the ray will not change the stack. If a node that references a triangle is determined to intersect, the reference added to the stack will be a reference to that triangle. When this item is popped from the stack, a ray-triangle intersection test will occur. If the ray does intersect the triangle, this can be recorded. Typically, the closest triangle intersection (i.e., the minimum t in the ray equation) needs to be determined, so a record of the closest triangle can be saved and updated when the triangle intersects. This process continues until the stack is empty and the tree structure has been completely traversed.
[0049] It is understandable that the order in which the sub-references of the hit node are pushed to the stack may affect the overall performance. If it is known that the nearest triangle is being searched, then when any intersecting triangle is found, all paths in the tree structure that are farther than the triangle can be skipped, which is called distance culling. If the nearest triangle happens to be found first, then many branches in the tree structure may be skipped. In contrast, if the nearest triangle is not found before searching a large number of tree structures, fewer triangles will be skipped (at least due to distance). Therefore, it is best to search the tree structure in an order that can prioritize finding the nearest triangle. In an embodiment of the present invention, the sub-references can be pushed into the stack in reverse order of distance so that they are popped from the stack in order of increasing distance.
[0050] Although the use of acceleration structures can reduce the complexity and time of each ray traversal of each primitive during ray tracing, ray tracing systems often need to introduce special states and data to manage the traversal of the acceleration structure. Such states and data required for the management of the acceleration structure traversal may have a large demand for computation and memory bandwidth.
[0051] One way to implement the traversal of the acceleration structure is to use a stack to store the status and data of nodes in the acceleration structure that have not been traversed yet, so as to be used for subsequent traversal of the acceleration structure.
[0052] Introducing a dedicated hardware manager to optimize the management of the traversal of the acceleration structures will further improve the performance of ray tracing in computer graphics processing systems.
[0053] In order to further improve the performance of ray tracing, an embodiment of the present invention provides a ray stack manager for managing ray stacks and improving ray traversal efficiency, thereby effectively improving ray tracing performance.
[0054] like Figure 3 As shown, the ray stack manager of the embodiment of the present invention includes multiple management units, each of which includes a stack storage module and an access logic module, the stack storage module is used to store the stack corresponding to the ray, and the access logic module is used to receive a stack request and manage the stack in the stack storage module corresponding to it according to the stack request, wherein each ray corresponds to a stack, and each stack stores the traversal status of its corresponding ray. In the embodiment of the present invention, the ray stack manager is used to store the traversal status of each ray to complete the quadtree search.
[0055] In some embodiments, the ray stack manager also includes a memory interface multiplexed by a plurality of the management units for data interaction with the memory, wherein the data interaction includes line filling and write back. In an embodiment of the present invention, the ray stack manager includes four management units, the internal structures of the four management units may be the same, and four rays may be processed in parallel in one cycle. The four management units multiplex multiple memory interfaces through an arbitrator for data interaction with an external memory. Specifically, it may include an interface for sending commands / data to a memory (a memory located outside the ray stack manager), an interface for receiving commands / data from the memory, line filling (Linefill) and / or write back (Writeback), and a line filling response and / or write back response sent from the memory. It is understandable that the ray stack manager of an embodiment of the present invention may set different numbers of management units and memory interfaces according to actual applications, and the present invention is not limited to this.
[0056] In some embodiments, the stack storage module is a dual-port static random access memory, and each of the stack entries includes a pointer to at least one node that the corresponding ray needs to traverse, and the node is determined according to the acceleration structure of ray tracing. It is understandable that the present invention does not limit which acceleration structure is used. In an embodiment of the present invention, the stack storage module caches the stack block at the top of each stack.
[0057] In some embodiments, each stack includes at least one stack block, each stack block includes multiple entries, and the stack blocks in different stacks have the same size. Specifically, each ray stack is divided into multiple stack blocks of the same size, wherein each stack block has 4 consecutive stack entries, for example, each entry has a 4B address size, and each stack block is 16B. In an embodiment of the present invention, the stack block of each ray being traversed is kept at the top of its corresponding stack, and the stack block of each ray in the stack storage module is kept in an available state, thereby effectively avoiding delays or pauses caused by cache misses.
[0058] Figure 4 FIG. 4 is a schematic diagram of a stack block of an embodiment of the present invention. In the embodiment of the present invention, taking 4 rays as an example, 4 stacks are constructed respectively, and each stack is divided into at least one stack block, and each stack block has 4 entries. Specifically, the size of each stack is determined by Figure 4 The arrow in indicates that the cached stack blocks are represented by the blocks with slashes. For Ray 2, its stack size is an integer multiple of the stack block size. In some embodiments, if the current stack block is empty, the next stack block of the current stack block is cached in the stack storage module, so that there is always available data for the pop request to avoid delay or pause.
[0059] In some embodiments, each of the access logic units includes a front-end processing module for receiving the stack request and classifying the stack request according to the stack status in the stack storage module; each of the access logic units also includes multiple processing modules, and the multiple processing modules are used to respond to the stack request according to the classification of the front-end processing module. In an embodiment of the present invention, the front-end processing module receives stack requests from other modules, including pop requests and push requests, and classifies the stack requests. In an embodiment of the present invention, the front-end processing module can process one stack request per cycle. Specifically, a stack request can be classified into the following four situations:
[0060] Category 1: Pop and no row filling, that is, pop data from the stack and do not need row filling.
[0061] Category 2: Pop and line fill, that is, pop data from the stack and need line filling.
[0062] Category 3: writing and evicting data, that is, writing data to the stack and writing the original data in the stack back to the memory.
[0063] Category 4: Write and do not evict data, that is, write data to the stack, and do not need to write the original data in the stack back to the memory.
[0064] In some implementations, each access logic unit includes four processing modules for correspondingly processing the above four types of stack requests. In an embodiment of the present invention, the four processing modules are also used to update the stack blocks in the stack storage module through the arbitrator, and to generate a row fill request and / or a write back request sent to the memory.
[0065] In the embodiment of the present invention, when the stack request is a pop request and the currently cached stack block has more than one entry, the first processing module performs processing to pop the entry at the top of the requested stack block from the stack.
[0066] In the embodiment of the present invention, when the stack request is a pop-up request, and the stack block currently cached has only one entry, the second processing module processes the pop-up request, generates a line fill request and sends it to the memory, and performs a pre-fetch operation on the stack block below the current stack block. This avoids the delay caused by waiting to obtain data from the memory. It is understandable that when the stack is empty, no pre-fetch operation is performed.
[0067] In the embodiment of the present invention, when the stack request is a push request and the stack block of the current cache overflows, the third processing module processes it, generates a write-back request and sends it to the memory, writes the data of the current stack block back to the memory, and updates the data. Specifically, the data in the current stack block is replaced with the data in the next higher stack block.
[0068] In the embodiment of the present invention, when the stack request is a push request and the stack block currently cached has not overflowed, the fourth processing module performs processing to write the data into the current stack block for updating.
[0069] In some embodiments, each access logic unit further includes a stack size storage module, the stack size storage module is used to store the size of the stack, and the front-end processing module determines whether the stack block overflows or underflows according to the stack size information stored in the stack size storage module. In an embodiment of the present invention, the stack size storage module stores the size of each stack, and the front-end processing module can classify the stack request in combination with the stack size information. Specifically, the original size information of the stack and the new size information adjusted after the push / pop operation are compared to determine whether the current stack block will overflow or underflow.
[0070] In some embodiments, very large memory delays can cause stalls. If a push or pop operation is performed on the memory, the ray will continue to transfer back to the ray scheduling module for the next test. Before the stack entry is updated, it may be selected for scheduling, node acquisition, intersection, and traversal. In response to the above problems, in some embodiments, each access logic unit also includes a row status storage module, which is used to store the status of the stack block to determine whether the stack block is waiting for row filling. In an embodiment of the present invention, for example, if the status shows that the current stack block is waiting for row filling from the memory, the pop stack request needs to wait until the row filling is completed before processing.
[0071] In some implementations, the line state storage module is further used to track dirty bits of each stack block, and write the stack block back to the memory when the stack block has a dirty bit, wherein the dirty bit is used to indicate that the data of the stack block is overwritten. In an embodiment of the present invention, when the acquisition and refresh of the stack block are not modified (i.e., not dirty), there is no need to write the stack block back to the memory, thereby improving data processing efficiency.
[0072] In some embodiments, each access logic unit further includes a delay request module, the delay request module being used to store the stack request when the stack block corresponding to the stack request is waiting for line filling; the delay request module is also used to suspend the input interface of the light stack manager when it is full. In an embodiment of the present invention, if the requested stack block is not yet ready, in order to avoid pausing the processing flow, the stack request is placed in the delay request module until the requested stack block is ready. And when the delay request module is full, that is, when the stored stack requests to be delayed reach a preset number, the input interface is suspended to avoid stack processing errors.
[0073] In some embodiments, the plurality of processing modules are further used to update the stack blocks in the stack storage module through an arbitrator, and to generate a row fill request and / or a write back request sent to the memory. In an embodiment of the present invention, the stack requests are classified and processed by the corresponding first to fourth processing modules, or the stack requests are placed in a delay request module. The first to fourth processing modules update the stack blocks in the stack storage module through the arbitrator, and generate a row fill request or a data evict request when necessary and send it to the memory. In some embodiments, the processing module is further used to return the pop-up stack result.
[0074] In some embodiments, the rowfill (read) response from the memory eventually returns and enters the rowfill module (e.g., a FIFO). The rowfill must update the stack storage module and the row status storage module and send a signal that any stack requests pending on this stack block can be processed. For eviction (write) responses, only the row status storage module needs to be updated.
[0075] The ray stack manager of the embodiment of the present invention manages a stack for each ray through multiple parallel processing units, and each stack is divided into multiple stack blocks. The top stack block of each ray is kept in the stack storage module of the processing unit, and multiple rays can be processed in parallel, thereby effectively managing multiple stacks, improving the efficiency of ray traversal processing, and optimizing the performance of ray tracing.
[0076] Figure 5The structure diagram of the ray tracing kernel of the embodiment of the present invention is shown. In some embodiments, the ray tracing kernel includes the ray stack manager as described above, and also includes a ray scheduling module, a ray intersection module and a ray traversal module, the ray scheduling module is used to generate a ray index, select a ray that enters the intersection test from multiple rays, the ray intersection module is used to perform an intersection test between the ray and the primitive bounding box and / or the ray and the polygon (such as a triangle), the ray traversal module is used to send a stack request to the ray stack manager according to the result of the ray intersection module to determine the next node that each ray that passes the intersection test needs to traverse, and the ray stack manager is used to receive the stack request and manage the stack state of each ray. The ray tracing kernel of the embodiment of the present invention has the ray stack manager as described above, optimizes the management of the traversal states of multiple rays, and realizes parallel processing of multiple rays, thereby improving the efficiency of the overall ray tracing.
[0077] In an embodiment of the present invention, a ray scheduling module of a ray tracing core (RT Core) receives data of multiple rays, wherein each ray waiting to be scheduled has an address associated with one or more nodes to be tested in an acceleration structure. For a new ray, its address defaults to the root node of the acceleration structure.
[0078] In some embodiments, the ray tracing kernel further includes a ray data storage module, which is used to store information of multiple rays during ray traversal, wherein the information includes at least one of the ray origin, ray direction, ray minimum distance, ray maximum distance, and the ray hit nearest primitive, or a combination thereof. Specifically, the ray tracing kernel initializes an on-chip ray data memory upon receiving a new ray. During the ray traversal, the ray data storage module saves the ray data locally. The data stored here includes, for example, the ray origin, ray direction, ray minimum distance, ray maximum distance, and information about the nearest intersection primitive (hit primitive). It is understandable that the ray data may also include more information according to actual needs, and the present invention is not limited to this.
[0079] In some embodiments, the ray tracing kernel further includes a node data acquisition module, which is used to obtain node information of the acceleration structure from a memory hierarchy, and is also used to send the ray index and the node information received from the ray scheduling module to the ray intersection module. Specifically, the ray scheduling module advances the selected ray to the node data acquisition stage. In this stage, the node data acquisition module is responsible for requesting and receiving the acceleration structure node data from the memory hierarchy. Once the node data is received, the node data acquisition module can send the node data and the ray index to the ray intersection module, and the ray can be advanced to the ray intersection test stage.
[0080] In some embodiments, the ray intersection module includes a hardware unit for performing ray-primitive bounding box and ray-polygon (e.g., triangle) intersections during the ray intersection test phase. In some embodiments, the ray intersection module also includes a conversion unit for converting the ray from the world space coordinate system to the model space coordinate system, which is necessary to support geometric transformations in the acceleration structure. In an embodiment of the present invention, the ray intersection module sends the intersection test and transformation calculation results to the ray traversal module.
[0081] In some embodiments, the ray traversal module is used to send a pointer back to the ray scheduling module after determining the next node that each ray that passes the intersection test needs to traverse. Specifically, the ray traversal module determines the next node to be tested for each ray, and the ray stack manager manages a test address stack for each ray to manage the traversal status of the ray. Once the ray traversal module determines which nodes need to be tested next, it forwards the pointer back to the ray scheduling module. In the ray scheduling module, the ray waits to be scheduled again.
[0082] In an embodiment of the present invention, the nodes are divided into four groups, thereby obtaining a quadtree structure. Up to four primitive bounding boxes will be stored consecutively in the memory and taken and tested together. If the ray has just tested the primitive bounding box in the acceleration structure, there are three possible results:
[0083] Type 1: There is no hit primitive bounding box. At this time, the next node to be tested is popped out of the stack.
[0084] Type 2: Hitting a primitive bounding box. In this case, the next node to be tested is designated as a child node of the intersection node.
[0085] Type 3: hit more than one primitive bounding box. In this case, the next node to be tested is pointed to as the child of the nearest intersection node. This is determined by the hit distance returned by the primitive bounding box intersection tester. The remaining child node pointers are pushed onto the stack from farthest to nearest.
[0086] If the ray is transformed and has only one child, this child is considered a hit and is forwarded to the ray scheduling module. If the ray performs a ray-triangle intersection, the address of the next node is popped from the stack.
[0087] In some implementations, a ray stack manager is responsible for managing these stacks and reducing the memory bandwidth generated by accessing them.
[0088] In some embodiments, in the ray tracing kernel, multiple rays can be in each processing flow at the same time, that is, at the same time, there are rays, such as the ray scheduling stage, the ray intersection node and the ray traversal stage, to cover the delay caused by node acquisition. In an embodiment of the present invention, the processing of rays can also be parallelized, so each stage can process N rays in parallel in each cycle.
[0089] In some implementations, the pause or delay of the data pipeline is avoided by optimizing the access to the memory. For example, in the case of the above-mentioned category 3, when the original data needs to be evicted when the data is pushed into the stack, it can be put into the write-back memory queue and continue to receive other stack requests before the write-back is completed, because there is no data waiting to be returned at this time.
[0090] In some embodiments, for a pop stack request, a line fill is performed when the current stack block becomes empty to ensure that the complete stack block below is available when it is accessed again. If line fill is not performed in advance, a pop stack request may cause a long delay waiting for immediate line fill. In an embodiment of the present invention, taking a quadtree as an example, at most three entries can be pushed in a cycle (the fourth entry is the most recent hit, which can bypass the ray stack manager and be forwarded directly to the ray scheduler), but there is only one pop stack in a cycle, so it is more reasonable to prefetch the lower stack block.
[0091] In some embodiments, the data arrangement of the stack blocks in the memory is optimized. A standard data arrangement is: there are R rays, each ray has a stack, each stack has N stack blocks, and each stack block has M entries. Then in a C-like language, the corresponding data format is: data [R] [N] [M].
[0092] However, when considering access patterns, it is more likely that multiple rays access the same stack layer at the same time rather than the same ray accessing the next layer of the stack. This is because there is a long delay for a ray to pass through the entire ray tracing kernel, and many other rays will pass through before looping back to the first ray. Because these stack blocks may be obtained through cache lines of a hierarchical storage structure, these cache lines may be larger than a stack block. Therefore, it is not necessary for a cache line to contain 0 to 3 stack blocks (0 to 15 stack entries) for a ray. In an embodiment of the present invention, these stack blocks are arranged horizontally for all rays, and their data format is: data [N] [R] [M]. This allows higher utilization of data in higher-level caches and improves data processing efficiency.
[0093] Figure 6 What is shown is a schematic diagram of the structure of a graphics processing system according to an embodiment of the present invention, including multiple ray tracing cores and multiple shader cores (Shader Core) as described above, wherein the multiple shader cores are used to call the ray tracing function, and send the initialization information required for the ray traversal to the ray tracing core, and the ray tracing core is used to perform ray traversal according to the initialization information and return the result, and the multiple shader cores are also used to perform lighting calculations according to the returned result.
[0094] In some embodiments, the GPU is composed of multiple shader cores, which are responsible for executing shader programs defined by the application. Multiple ray tracing cores are responsible for performing ray tracing functions. In the embodiment of the present invention, the ray tracing core and the shader core are set in a 1:1 relationship. It is understandable that the ray tracing core and the shader core can also be mapped to any ratio using a crossbar or other method, and the present invention is not limited to this.
[0095] In some existing APIs, rays are defined by the application which is executed by the shader kernel, and a TraceRay() function (or similar function) may be called to request the hardware ray tracing kernel to perform a ray traversal according to a specified acceleration structure. Figure 6In the architecture shown, the shader kernel executes a general application, which can call the TraceRay() function. At this time, the data used to initialize the traversal (such as light data, acceleration structure data and parameters, etc.) is sent from the shader kernel to the ray tracing kernel, and then all traversals are processed by the ray tracing kernel. In an embodiment of the present invention, the TraceRay() function is a blocking call, so the shader kernel waits for the ray tracing kernel to complete the traversal and return the result (the shader kernel will execute other threads). Specifically, the returned result will indicate whether the ray intersects with a primitive, and which of the intersecting primitives is the closest and how far away it is. The shader kernel will use this information for lighting calculations. Due to the use of the above-mentioned ray tracing manager, it has a ray stack manager as described above, which optimizes the management of the ray traversal state, thereby improving the efficiency of the overall ray tracing.
[0096] like Figure 7 As shown, corresponding to the above-mentioned graphics processing system, an embodiment of the present invention further provides a graphics processing method, the method comprising:
[0097] Step 701: Use the ray stack manager in the ray tracing kernel as described above to maintain a stack for each ray to be traversed to store the traversal state information of each ray. In some embodiments, a stack is created for each ray to be traversed, each stack is divided into a plurality of stack blocks, and each stack block has a plurality of entries. In an embodiment of the present invention, multiple rays can be processed in parallel in the same cycle, and multiple rays can also be processed in different stages, and the stack block at the top of each ray is cached.
[0098] Step 703: Determine the next node that each ray needs to traverse according to the stack request and the traversal status information, until the traversal of all rays is completed and the traversal results are determined. In an embodiment of the present invention, the stack of each ray is managed according to the stack status and the stack request, and each stack request is responded to, such as popping entries, pushing entries, row filling, updating stack block data, etc. Specifically, the stack blocks are arranged horizontally according to the rays, so the memory addresses of N stack blocks are arranged continuously for all rays, but the stack blocks are arranged vertically for the stack, so the memory addresses corresponding to the stack blocks in a stack are not arranged continuously.
[0099] In some implementations, when a cached stack block becomes empty, a pre-fetch operation is performed on the next stack block.
[0100] In some implementations, the state of stack blocks is tracked and unmodified stack blocks are not written back to memory.
[0101] In some implementations, stack requests that are still waiting for a line fill are deferred to avoid data processing flow stalls.
[0102] Step 705: Return the traversal result to the shader kernel for subsequent rendering. In some implementations, the tiles are rendered according to the generated tile display list to obtain a rendered image.
[0103] The more specific implementation of each step of the graphics processing method of the present invention can refer to the description of the graphics processing system of the present invention, and has similar beneficial effects, which will not be repeated here.
[0104] Figure 8 is a structural block diagram of an electronic device according to an embodiment of the present application. The present application embodiment also provides an electronic device, such as Figure 8 As shown, the electronic device includes: at least one processor 801, and a memory 803 connected to the at least one processor 801 in communication. Instructions that can be executed by at least one processor 801 are stored in the memory 803. The instructions are executed by at least one processor 801. When the processor 801 executes the instructions, the graphics processing method in the above embodiment is implemented. The number of the memory 803 and the processor 801 can be one or more. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples, and are not intended to limit the implementation of the present application described and / or required herein.
[0105] The electronic device may also include a communication interface 805 for communicating with external devices and performing data exchange transmission. The various devices are interconnected using different buses and may be installed on a common motherboard or in other ways as needed. The processor 801 may process instructions executed in the electronic device, including instructions stored in or on a memory to display graphical information of a graphical user interface (GUI) on an external input / output device (such as a display device coupled to an interface). In other embodiments, if necessary, multiple processors and / or multiple buses may be used together with multiple memories and multiple memories. Similarly, multiple electronic devices may be connected, each device providing some of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0106] Optionally, in a specific implementation, if the memory 803, the processor 801 and the communication interface 805 are integrated on a chip, the memory 803, the processor 801 and the communication interface 805 can communicate with each other through an internal interface.
[0107] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.
[0108] An embodiment of the present application provides a computer-readable storage medium (such as the memory 803 described above), which stores computer instructions. When the program is executed by a processor, the method provided in the embodiment of the present application is implemented.
[0109] Optionally, the memory 803 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created according to the use of the electronic device of the graphics processing method, etc. In addition, the memory 803 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 803 may optionally include a memory remotely arranged relative to the processor 801, and these remote memories may be connected to the electronic device of the graphics processing method via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0110] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.
[0111] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0112] Any process or method description in the flow chart or otherwise described herein can be understood to represent a module, fragment or portion of a code including one or more (two or more) executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.
[0113] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.
[0114] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.
[0115] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.
[0116] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A ray stack manager, characterized in that: It comprises a plurality of management units, each of which comprises a stack storage module and an access logic module, the stack storage module is used to store a stack corresponding to a light ray, the access logic module is used to receive a stack request, and manage the stack in the stack storage module corresponding to it according to the stack request, wherein each light ray corresponds to a stack, and each stack stores the traversal status of its corresponding light ray.
2. The ray stack manager of claim 1, wherein: Each of the stacks includes at least one stack block, each of the stack blocks includes a plurality of entries, wherein stack blocks in different stacks have the same size.
3. The ray stack manager of claim 2, wherein: Each of the access logic units includes a front-end processing module, which is used to receive the stack request and classify the stack request according to the stack status in the stack storage module; each of the access logic units also includes multiple processing modules, which are used to respond to the stack request according to the classification of the front-end processing module.
4. The ray stack manager of claim 3, wherein: The plurality of processing modules include a first processing module, and when the stack block has more than one entry, in response to a pop stack request, the first processing module is used to control popping the entry at the top of the stack.
5. The ray stack manager of claim 3, wherein: The plurality of processing modules include a second processing module, which is used to control popping the entry at the top of the stack block and controlling pre-fetching operations on stack blocks below the stack block in response to a stack pop request when the stack block has only one entry.
6. The ray stack manager of claim 3, wherein: The plurality of processing modules include a third processing module, and when the stack request is a push request and the current stack block overflows, the third processing module is used to control writing the current stack block data back to the memory and updating the current stack block with new data.
7. The ray stack manager of claim 6, wherein: The new data is the data in the next stack block of the current stack block.
8. The ray stack manager of claim 3, wherein: The plurality of processing modules include a fourth processing module, and when the stack request is a push request and the current stack block does not overflow, the fourth processing module is used to control updating of the current stack block.
9. The ray stack manager of claim 3, wherein: Each of the access logic units further includes a stack size storage module, which is used to store the size of the stack. The front-end processing module determines whether the stack block overflows or underflows according to the stack size information stored in the stack size storage module.
10. The ray stack manager of claim 3, wherein: Each of the access logic units further includes a row status storage module, and the row status storage module is used to store the status of the stack block to determine whether the stack block is waiting for row filling.
11. The ray stack manager of claim 10, wherein: The row status storage module is further used to track the dirty bit of each stack block, and write the stack block data back to the memory when the stack block has a dirty bit, wherein the dirty bit is used to indicate whether the data of the stack block is overwritten.
12. The ray stack manager of claim 3, wherein: Each of the access logic units further comprises a delay request module, which is used to store the stack request when the stack block corresponding to the stack request waits for line filling; the delay request module is also used to suspend the input interface of the light stack manager when it is full.
13. The ray stack manager of claim 3, wherein: The plurality of processing modules are further configured to update the stack blocks in the stack storage module through an arbitrator, and to generate a row fill request and / or a write back request to be sent to a memory.
14. The ray stack manager according to any one of claims 1 to 13, characterized in that: The stack storage module is a dual-port static random access memory, and each of the stack entries includes a pointer to at least one node that the corresponding ray needs to traverse, and the node is determined according to an acceleration structure of ray tracing.
15. The ray stack manager according to any one of claims 1 to 13, characterized in that: The ray stack manager also includes a memory interface multiplexed by a plurality of the management units and used for data interaction with a memory, wherein the data interaction includes line filling and write-back.
16. A ray tracing kernel, characterized in that A ray stack manager comprising any one of claims 1 to 15, wherein the ray tracing kernel further comprises a ray scheduling module, a ray intersection module and a ray traversal module, wherein the ray scheduling module is used to generate a ray index and select a ray to enter an intersection test from a plurality of rays, wherein the ray intersection module is used to perform a ray-primitive bounding box and / or ray-polygon intersection test, wherein the ray traversal module is used to send a stack request to the ray stack manager to determine the next node to be traversed by each ray that passes the intersection test, and wherein the ray stack manager is used to receive the stack request and manage the stack state of each ray.
17. The ray tracing kernel of claim 16, wherein: The ray tracing kernel also includes a ray data storage module, which is used to store information of multiple rays during ray traversal, wherein the information includes at least one of the ray origin, ray direction, ray minimum distance, ray maximum distance, and the ray hitting the nearest primitive, or a combination thereof.
18. The ray tracing kernel of claim 16, wherein: The ray tracing kernel also includes a node data acquisition module, which is used to obtain node information of an acceleration structure from a hierarchical memory and to send the ray index and the node information received from the ray scheduling module to the ray intersection module.
19. A ray tracing kernel as claimed in any one of claims 16 to 18, characterized in that The light intersection module includes a conversion unit, and the conversion unit is used to convert the light from the world space coordinate system to the model space coordinate system.
20. A ray tracing kernel as claimed in any one of claims 16 to 18, characterized in that The ray traversal module is further configured to send a pointer back to the ray scheduling module after determining the next node that each ray that passes the intersection test needs to traverse.
21. A ray tracing kernel as claimed in any one of claims 16 to 18, characterized in that The ray tracing kernel processes multiple rays in parallel during each cycle.
22. A graphics processing system, characterized in that: The graphics processing system comprises a ray tracing kernel as described in any one of claims 16 to 21, wherein the plurality of shader kernels are used to call a ray tracing function, and send initialization information required for ray traversal to the ray tracing kernel, and the ray tracing kernel is used to perform ray traversal according to the initialization information and return a result, and the plurality of shader kernels are also used to perform lighting calculations in rendering according to the returned result.
23. A graphics processing method, characterized in that: include: A ray stack manager according to any one of claims 1 to 15 or a ray tracing kernel according to any one of claims 16 to 21 is used to maintain a stack for each ray to be traversed, so as to store traversal status information of each ray; Determine the next node that each light ray needs to traverse according to the stack request and the traversal state information until the traversal of all light rays is completed and the traversal results are determined; The traversal result is returned to the shader kernel for subsequent rendering.
24. An electronic device, characterized in that: A ray stack manager comprising any one of claims 1 to 15, or a ray tracing kernel according to any one of claims 16 to 21, or a graphics processing system according to claim 21; Alternatively, the electronic device comprises: processor; a memory communicatively coupled to the processor; The memory stores instructions executable by the processor, and the instructions are executed by the processor to enable the processor to perform the method of claim 23.
25. A computer-readable storage medium storing computer instructions, wherein the computer instructions implement the method according to claim 23 when executed by a processor.
