Light ray tracing method and device

Abstract

The invention discloses a ray tracing method and device, relates to the technical field of terminals, and can improve the execution efficiency of ray tracing. The method comprises the steps that a first shader used for ray tracing is executed through at least one first thread set, the first shader comprises first compiling codes, and the first compiling codes comprise conditions for reordering threads corresponding to rays involved in ray tracing; determining information of an object hit by the light; grouping threads in the at least one first thread set according to a reordering condition and information of an object hit by the light to form at least one second thread set; and executing a second shader through at least one second thread set, wherein the second shader is a shader corresponding to the object hit by the light. Visibly, the reordering condition of the thread can be controlled through the first compiled code, so that the compatibility of the API to the ray tracing application is ensured while the hardware utilization rate is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a ray tracing method and device. Background Art

[0002] Taking the game interface as an example, the outer surface of a 3D model in the game interface, such as a character, can be divided into triangles, and the terminal can store the information of the triangle vertices of the 3D model. Subsequently, in the traditional rendering pipeline, the offset (x, y, z) of the triangle corresponding to the 3D model is multiplied by the offset matrix according to the positional relationship between the screen and the 3D model in the 3D world, and the 3D model can be mapped to the screen to form a 2D game interface displayed on the screen.

[0003] Unlike traditional rendering technology, ray tracing (also known as ray tracing, light trace tracing or ray tracing, etc.) tracks the path of light in pixels in an image plane (such as a screen) and simulates the effect of light meeting a virtual object to generate an image. In ray tracing technology, the light is traced in the opposite direction of the light reaching the viewpoint (virtual camera), passing through each pixel on the screen, and finding the point P1 on the surface of the object that intersects with the line of sight (the opposite direction of the light). In ray tracing technology, light is projected from a light source, and when the light passes through the surface of an object, various transformations that conform to the laws of physical optics are applied to the light. Finally, the light enters the virtual camera screen (that is, the virtual camera imaging plane) and an image is generated. Ray tracing technology simulates the physical laws of light propagation in the real physical world, so it can generate rendering results that are close to the real world. It is usually used in scenes such as games, AR / VR, and advertising that pursue extremely high image quality.

[0004] Currently, the execution efficiency of ray tracing is low, and an efficient ray tracing method is urgently needed. Summary of the invention

[0005] The present application provides a ray tracing method, device and system for improving the efficiency of ray tracing.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, the technical solution of the present application provides a ray tracing method, which can be applied to electronic devices or components that support the functions of electronic devices (such as chip systems), and the method includes: executing a first shader for ray tracing through at least one first thread set, the first shader containing a first compiled code, and the first compiled code containing a condition for reordering threads corresponding to rays involved in ray tracing; determining information about objects hit by the rays; grouping threads in the at least one first thread set according to the reordering conditions and the information about the objects hit by the rays to form at least one second thread set; executing a second shader through the at least one second thread set, the second shader being a shader corresponding to the object hit by the rays.

[0008] In the present application, since the threads generating light rays are grouped, the threads in the same thread set are concentrated as much as possible to execute the same shader (the second shader) within a processing time, and the execution efficiency is high.

[0009] In addition, in some cases, programmers know how to reorder more efficiently. According to the technical solution of the present application, before executing ray tracing, the programmer defines the reordering conditions by compiling the code (such as Pragma). The compiled code can represent the programmer's reordering intention, so that the compiler compiles these reordering information into the instructions so that the hardware can perform thread reordering according to the reordering conditions, thereby improving the execution efficiency of ray tracing. At the same time, compared to the related art in which the reordering conditions are passed by function calls, in the present application, the reordering conditions are passed by compiling the code, and there is no need to modify the standard built-in function traceRayEXT() function. Since the standard built-in function traceRayEXT() function is not modified, and the standard function call relationship is not changed, it is compatible with various ray tracing applications (such as game applications).

[0010] In one possible design, the reordering condition indicates that threads consistent with a first feature are divided into the same group; the first feature includes at least one of the following features: the material of the object hit by the generated light, the texture of the object hit by the generated light, the executed branches, and the number of executed loops.

[0011] In this way, threads with the same first characteristic are divided into the same group, so that threads in the same thread set can be concentrated as much as possible within a processing time to execute the same shader (the second shader), thereby achieving high execution efficiency.

[0012] In a possible design, grouping the threads in the at least one first thread set according to the reordering condition and the information of the object hit by the light to form at least one second thread set includes:

[0013] Grouping threads in the at least one first thread set according to an identifier of a second shader corresponding to the object hit by the light to form at least one third thread set;

[0014] According to the reordering condition, the threads in the at least one third thread set are grouped to form at least one second thread set.

[0015] In this way, threads that generate rays that hit the same object can be preferentially grouped into the same thread set, increasing the probability that threads in the thread set execute the same shader. In addition, based on the reordering conditions, further thread set reorganization can be performed, so that threads in the same second thread set can execute the same shader with a high probability, reducing the execution dispersion problem caused by multiple threads in the thread set executing different shaders.

[0016] In one possible design, the method further includes:

[0017] Reading a reordering condition of the first compiled code in the first shader, and using the reordering condition as a source operand of a ray tracing instruction;

[0018] Compiling the first shader into a binary file; the binary file includes the ray tracing instructions;

[0019] The executing the first shader includes: executing the first shader in a compiled binary file format.

[0020] In this way, the reordering conditions in the compiled code can be used as source operands of the ray tracing instructions, so that the electronic device can perform reordering according to the source operands, thereby improving the efficiency of ray tracing.

[0021] In a possible design, grouping the threads in the at least one first thread set according to the reordering condition and the information of the object hit by the light to form at least one second thread set includes:

[0022] The threads in the at least one first thread set are grouped according to the source operand and the information of the object hit by the light to form at least one second thread set.

[0023] In one possible design, the method further includes:

[0024] Allocating a first stack frame to the second thread set, wherein storage addresses in the first stack frame are continuous addresses;

[0025] Storing information of the parent function required to execute the second shader into the continuous address;

[0026] The second shader is executed according to the information of the parent function read from the continuous address, and the information generated by executing the second shader is stored in a second stack frame corresponding to the parent function.

[0027] In this way, by dynamically allocating continuous stack frame space for the second thread set (such as callee warp) and copying the required information in the stack frame space corresponding to the parent function (such as temporary variables, etc.) to the stack frame space, the private storage access within the second thread set can be consistent and continuous when the dynamic non-consistent function call and return are executed on the hardware, keeping the stack frame access of each thread localized and consistent, reducing the probability of storage divergence, and solving the problem of low ray tracing execution efficiency caused by access from discontinuous addresses. Significantly improve the consistency of shader execution in ray tracing and improve hardware execution efficiency.

[0028] In one possible design, the method further includes:

[0029] When the first thread set is suspended, information about the first thread set is written into a stack frame status table; the stack frame status table includes the number of remaining valid threads in the first thread set and a program counter PC address;

[0030] According to the stack frame status table, when the remaining valid threads are zero, the first thread set is resumed to be executed according to the PC address.

[0031] In one possible design, the first compiled code includes any of the following codes: a compilation annotation or a pre-compilation instruction or a pre-processing instruction.

[0032] In a second aspect, a ray tracing device is provided, comprising: a processing unit, configured to execute a first shader for ray tracing through at least one first thread set, wherein the first shader includes a first compiled code, and the first compiled code includes a condition for reordering threads corresponding to rays involved in ray tracing; determining information about an object hit by the ray; grouping threads in the at least one first thread set according to the reordering condition and the information about the object hit by the ray to form at least one second thread set; and executing a second shader through the at least one second thread set, the second shader being a shader corresponding to the object hit by the ray.

[0033] Compared with some scenarios of the related technology, threads in the same thread set execute different shaders, resulting in low execution efficiency. The solution of the embodiment of the present application, through thread reordering, makes the same thread set execute the same shader as much as possible, avoiding execution dispersion caused by the thread set executing different ray tracing shaders, including Shader Divergence, Memory Divergence, Flow-Control Divergence, etc. Therefore, it can improve execution efficiency, high hardware utilization, and optimize overall operating performance.

[0034] In addition, in the solution of the present application, developers can control the conditions for thread reordering through the first compiled code, thereby improving hardware utilization while ensuring API compatibility with ray tracing applications (such as games).

[0035] In one possible design, the reordering condition indicates that threads consistent with a first feature are divided into the same group; the first feature includes at least one of the following features: the material of the object hit by the generated light, the texture of the object hit by the generated light, the executed branches, and the number of executed loops.

[0036] In a possible design, grouping the threads in the at least one first thread set according to the reordering condition and the information of the object hit by the light to form at least one second thread set includes:

[0037] Grouping threads in the at least one first thread set according to an identifier of a second shader corresponding to the object hit by the light to form at least one third thread set;

[0038] According to the reordering condition, the threads in the at least one third thread set are grouped to form at least one second thread set.

[0039] In a possible design, executing the first shader includes:

[0040] A first shader in the form of a compiled binary file is executed; the binary file includes ray tracing instructions; the reordering condition is a source operand of the ray tracing instructions.

[0041] In a possible design, grouping the threads in the at least one first thread set according to the reordering condition and the information of the object hit by the light to form at least one second thread set includes:

[0042] The threads in the at least one first thread set are grouped according to the source operand and the information of the object hit by the light to form at least one second thread set.

[0043] In a possible design, the processing unit is further used for:

[0044] Allocating a first stack frame to the second thread set, wherein storage addresses in the first stack frame are continuous addresses;

[0045] Storing information of the parent function required to execute the second shader into the continuous address;

[0046] The second shader is executed according to the information of the parent function read from the continuous address, and the information generated by executing the second shader is stored in a second stack frame corresponding to the parent function.

[0047] In a possible design, the processing unit is further used for:

[0048] When the first thread set is suspended, information of the first thread set is written into a stack frame status table; the stack frame status table includes the number of remaining valid threads in the first thread set and a program counter PC address;

[0049] According to the stack frame status table, when the remaining valid threads are zero, the first thread set is resumed to be executed according to the PC address.

[0050] In one possible design, the first compiled code includes any of the following codes: a compilation annotation or a pre-compilation instruction or a pre-processing instruction.

[0051] The device also supports various homogeneous and heterogeneous structure expansions such as multi-GPC, multi-GPU, and multi-RTU, expanding the applicable scenarios of the reordering solution.

[0052] In a third aspect, the technical solution of the present application provides a device, comprising: one or more processors and a memory; wherein the processor is coupled to the memory, and one or more computer programs are stored in the memory. When the device is running, the processor executes one or more computer programs stored in the memory so that the device executes the method in any design of any of the above aspects.

[0053] In a fourth aspect, the technical solution of the present application provides a computer-readable storage medium, including computer instructions. When the computer instructions are executed on a device, the device executes any possible design method in any of the above aspects.

[0054] In a fifth aspect, the technical solution of the present application provides a computer program product, which, when executed on a device, enables the device to execute any possible design method in any of the above aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1A-1EAn example diagram of a ray tracing method provided for related art;

[0056] Figure 2 An exemplary diagram of the device structure provided in the embodiments of the present application;

[0057] Figure 3 An example flowchart of a ray tracing method provided in an embodiment of the present application;

[0058] Figure 4 An example diagram of a shader provided for an embodiment of the present application;

[0059] Figure 5 A scene example diagram of the ray tracing method provided in an embodiment of the present application;

[0060] Fig. 6A , Figure 6B A scene example diagram of the ray tracing method provided in an embodiment of the present application;

[0061] Figure 7 An example diagram of an entry for storing light grouping information provided in an embodiment of the present application;

[0062] Fig. 8A , Figure 8B An example diagram of a stack frame provided in an embodiment of the present application;

[0063] Fig. 9 An example flowchart of a ray tracing method provided in an embodiment of the present application;

[0064] Fig. 10A , Fig. 10B An example diagram of the process of copying stack frame information between parent and child functions provided in an embodiment of the present application;

[0065] Figure 11-Figure 17 An example diagram of a ray tracing device provided in an embodiment of the present application;

[0066] Fig.18 A schematic diagram of the structure of a device provided in an embodiment of the present application;

[0067] Fig.19 A schematic diagram of the structure of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The terms "first" and "second" in the specification and drawings of this application are used to distinguish different objects, or to distinguish different treatments of the same object. The words "first" and "second" can distinguish the same or similar items with basically the same functions and effects. For example, the first terminal and the second terminal are only used to distinguish different devices, and their order is not limited. Those skilled in the art can understand that the words "first" and "second" do not limit the quantity and execution order, and the words "first" and "second" do not necessarily limit them.

[0069] "At least one" means one or more, and "plurality" means two or more.

[0070] "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or plural.

[0071] In addition, the terms "including" and "having" and any variations thereof mentioned in the description of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0072] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0073] First, the related technologies involved in the embodiments of the present application are introduced:

[0074] 1. Ray tracing

[0075] like Figure 1A, multiple rays start from the virtual camera, pass through each pixel on the screen, and hit the corresponding object point in three-dimensional space. Then, when the terminal renders the corresponding pixel, the color of the pixel is the color of the point position on the object hit by the ray passing through the pixel. For example, ray ① hits point P1 of object 1, then the screen pixel passed by ray ① displays the color of point P1. Similarly, ray ② hits point P2 of object 1, then the screen pixel passed by ray ② displays the color of point P2. This is just a simple example of ray tracing. In the actual ray tracing process, the light may also be reflected and refracted in three-dimensional space. Therefore, the actual display color process of the rendered pixel may be more complicated.

[0076] 2. Shader

[0077] Shader is a general term for programmable code in GPU. Shader may include Ray Generation shader, Closest-hit shader, Miss shader.

[0078] Among them, Ray Generation shader is the initial shader for ray tracing.

[0079] Closest-hit shader and Miss shader can be used to calculate the bounce of light, which includes but is not limited to reflection, refraction, and scattering.

[0080] Closest-hit shader: During ray intersection, the ray may hit more than one object. In this case, the closest-hit shader can be called to get the nearest opaque object hit by the ray.

[0081] Miss shader: When the light does not hit the object, the Miss shader can be called, such as executing the shader corresponding to the sky by default.

[0082] Any-hit shader: Unlike the Closest-hit shader, which only calculates the frontmost object hit by the light, the Any-hit shader is called and performs a calculation each time it hits a transparent or semi-transparent object.

[0083] Intersection shader: There is a box in the scene, and the objects in the box are not triangles. The hardware can intersect the light with the box. If the light hits the box, it is likely to hit the object in the box.

[0084] There can be many kinds of objects in a scene, some of which are transparent and some are opaque. Different types of shaders can be called for different types of objects. Figure 1B Shows the calls to various shaders.

[0085] Ray tracing involves functions that can call shaders, such as traceRayEXT() and executeCallableEXT(). The difference between the two functions is that traceRayEXT() calls different shaders depending on the objects hit, and executeCallableEXT() calls different shaders depending on the objects hit by the light.

[0086] The shader may contain a traceRayEXT() function, which is also called a ray tracing function. For example, a RayGeneration shader or a Closest-hit shader or a Miss shader may contain a traceRayEXT() function. In the shader, once the developer writes traceRayEXT(), the hardware will know that it needs to generate light rays, intersect the rays, and call the function (the shader corresponding to the hit object) to render the image. After the function call is completed, it returns to the main function. In different APIs, the function used to generate light rays can be different. For example, it can also be a traceRay() function.

[0087] Take Ray Generation shader as an example. Figure 1C In the ray tracing rendering pipeline, when several threads in the execution unit (EU) in the graphics processing unit (GPU) execute the Ray Generation shader, one thread generates a ray each time it executes the traceRayEXT() function. The EU passes the ray-related information generated by each thread to the ray tracing unit (RTU) of the GPU. The RTU performs a ray intersection operation on each ray to obtain ray hit information (also called ray intersection information), such as whether the ray hits an object, whether the hit object is transparent, whether the hit object is composed of triangles or other shapes, the location of the hit object, and the ID of the hit object.

[0088] After that, the GPU calls the shader corresponding to the object hit by each ray of light passing through the screen pixel, and colors the object to form the color corresponding to the corresponding pixel on the screen. Figure 1A, the GPU calls the shader corresponding to object 1, renders object 1, and forms a display image of object 1 on the screen.

[0089] 3. Single instruction multiple data (SIMD)

[0090] The underlying hardware of the GPU single instruction multiple thread (SIMT) computing model can adopt the SIMD architecture, in which the long data to be processed is decomposed into multiple shorter data, and the processing of multiple short data can be completed in parallel at the same time.

[0091] 4. Warp Divergence

[0092] Different from the logic of CPU, where multiple CPUs can execute different programs at the same time, the execution logic of GPU is: GPU performs operations such as instruction fetching, decoding, execution, and write-back in thread sets. This means that in the same clock cycle, all threads in a thread set can only execute the same program, and different thread sets can execute different programs. Figure 1D A set of threads can be called a warp. For example, 32 threads form a warp, and each thread can occupy a lane.

[0093] For example, when different threads in a warp hit different objects, since different objects call different shader codes, the threads in the warp want to execute multiple shader codes in the same clock cycle, but due to the limitations of the SIMD architecture, multiple threads in the warp can only execute a certain shader code in the same clock cycle, and only after executing the shader code can other shader codes be executed serially. Or, in the same clock cycle, different shaders are executed by different warps, such as warp1 executes shader 1, and warp executes shader 2. It can be seen that although the GPU has a lot of hardware resources, multiple threads in a warp can only execute code programs serially, resulting in serious warp divergence problems.

[0094] In extreme cases, different threads execute different shader codes, and several threads in a warp execute completely serially, so the GPU's execution performance is completely reduced to the level of a single thread, and the hardware utilization is very low. Figure 1E , there are threads 1-4 in the warp, and the light generated by threads 1-4 hits the leaves, trunks, stones and grass respectively. Then threads 1-4 need to serially execute the shaders of leaves, trunks, stones, and grass respectively. In other words, multiple threads in the warp completely execute multiple shaders serially, and the execution efficiency is low.

[0095] 5. Memory Divergence

[0096] In addition to the warp divergence caused by calling different shaders as mentioned above, ray tracing also faces other problems that can cause divergence, such as memory divergence, branch / control-flow divergence, etc.

[0097] Memory Divergence refers to the fact that different threads in the same warp executing the same shader access discontinuous storage spaces.

[0098] For example, the discontinuous memory access index values ​​of adjacent threads within a warp result in read operations from non-continuous addresses. Another example is that in a bindless scenario, different textures are accessed in dynamically non-uniform indexing. Take the following code as an example, Tex[index] represents a texture map. In a warp, the first thread accesses the texture map of a stone (such as granite), and the second thread accesses the texture map of a leaf. If the texture maps of the two are stored in discontinuous spaces, the access efficiency will be affected.

[0099]

[0100] In some other scenarios, Memory Divergence may also reduce the efficiency of memory coalescing of the load-store unit (LSU) or texture unit (TU) in the GPU, affecting the computing performance of the GPU.

[0101] Memory access address merging means that if the access addresses of multiple threads are the same or adjacent, the accesses of multiple threads can be merged into one access. For example, the access address of thread 1 is A, the access address of thread 2 is B, and the access address of thread 3 is C. Address A and address C are adjacent, so the accesses of thread 1 and thread 3 can be merged into one access, and the three threads access twice in total. When memory divergence occurs, the access addresses are dispersed, and the possibility of merging accesses is reduced, which affects the computing performance of the GPU.

[0102] 6. Branch / Control-flow Divergence

[0103] Branch / Control-flow Divergence refers to the fact that different threads of the same warp executing the same shader execute different Branch / Control-flow instructions, resulting in the serial execution of the branch part of the code.

[0104] For example, 32 threads in a warp share instructions and execute the following shader code in the same clock cycle. If different threads execute different code branches, the warp will invalidate some threads, execute the code of one branch first, and then invalidate the other threads and execute the code of the other branch. For example, if some threads execute the if branch and some threads execute the else branch, the thread executing the else branch will be invalidated first, execute the if branch code, and then invalidate the thread executing the if branch and execute the else branch code. That is, the code of all branches is executed sequentially, which is inefficient.

[0105]

[0106] For another example, when multiple threads of a warp execute the following code, the number of for calls in different threads may be different, resulting in low execution efficiency.

[0107]

[0108] 7. Light Estimation

[0109] Related technology provides a ray estimation (also called reordering) method for improving the execution efficiency of ray tracing. For the light from the virtual camera to the screen pixel, the intersection of the light and the object in the three-dimensional world is obtained, and the direction of the light is estimated. According to the shader ID corresponding to the object hit by the light, and the direction of the light, different light information is stored in different queues. In this way, in the same queue, the stored light information is the light that hits the same object. Exemplarily, the GPU packs the 15 rays that hit the leaves together and resends them to the EU. The EU knows that all 15 rays hit the leaves, so it can execute the remaining functions in parallel to improve the utilization of the hardware. For example, for 32 threads in a warp, the shaders of the leaves are executed in parallel in the same clock cycle.

[0110] This method follows the general ray tracing API and is compatible with general ray tracing applications. It can improve the efficiency of ray tracing to a certain extent by reordering rays. However, in some scenarios, the GPU may not know the optimal strategy for ray estimation or reordering.

[0111] 8. Split built-in functions

[0112] In other related technologies, another method of reordering rays is proposed, which is to split the original traceRayEXT() built-in function into multiple built-in functions.

[0113] HitObject hit=TraceRayHitObject(ray,payload); Function 1

[0114] ReorderThread(hit,materialFlags,4); Function 2

[0115] InvokeHitObject(hit, payload); Function 3

[0116] Through function 1, the intersection of the light ray and the object in the three-dimensional world is calculated to obtain the hit point.

[0117] Function 2 is used to reorder items based on the hit points and the reordering conditions defined by the application developer, such as the material ID. For example, the material ID is metal or wood.

[0118] Through function 3, call the closest-hit / miss shader and other shaders to perform rendering.

[0119] In this method, the Coherence Hint is used to provide developers with an explicit interface for controlling reordering conditions. However, since the standard built-in functions in the shader are modified, it is not compatible with general game APIs (such as Vulkan and DirectX), is not compatible with ray tracing applications (such as game applications), and has poor scalability.

[0120] 9. Non-uniform function call

[0121] When a function is called (nested), a corresponding stack space can be allocated for the function to store the data required for the function call. When a function is called, the stack space grows dynamically, for example, the stack space pointer increases from S0->S1->S2->…->Sn. When the function returns, the stack space pointer decreases from Sn->Sn-1->…->S0.

[0122] Function calls include consistent function calls and inconsistent function calls.

[0123] Consistent function calls mean that all threads in a warp can only call the same function at the same time. For the GPUSIMT structure, if only consistent function calls are allowed, the stack frame management during the function call process can be implemented in the same way as the CPU.

[0124] The stack frame can be used to store some data required by the function in the function call. For example, the F0 function calls the F1 function, and saves the data of the register on the chip (such as the CPU), for example, temporarily storing the information of F0 in the S0 stack frame. Later, when F1 is executed, the stack frame will continue to grow, and some data will be generated and stored in the stack frame.

[0125] Inconsistent function calls: refers to threads within a warp calling different functions in the same clock cycle. In a ray tracing scenario, when the traceRayEXT() function is called, different threads will call different Closest-hit shaders or Miss shaders or other shaders based on the hit objects. Nested function calls will further lead to WarpDivergence and Recursion Depth Divergence.

[0126] To this end, an embodiment of the present application provides a ray tracing method, which can be applied to devices or corresponding modules (such as chips) that use ray tracing methods to perform rendering, including but not limited to applications in terminals, servers and other devices. Figure 2 An exemplary structure of an electronic device is shown, and the electronic device includes at least one processor 201 and a memory 202. Optionally, the electronic device may further include at least one transceiver 203.

[0127] The processor 201 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present specification.

[0128] The above components may include communication lines for transmitting information between the above components.

[0129] The transceiver 203 is used to communicate with other devices. In this specification, the transceiver can be a module, a circuit, a bus, an interface or other device capable of realizing a communication function, and is used to communicate with other devices. Optionally, the transceiver can be an independently arranged transmitter, which can be used to send information to other devices, and the transceiver can also be an independently arranged receiver, which is used to receive information from other devices. The transceiver can also be a component that integrates the functions of sending and receiving information, and this specification does not limit the specific implementation of the transceiver.

[0130] The memory 202 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor through a communication line. The memory may also be integrated with the processor.

[0131] The memory 202 is used to store computer-executable instructions for implementing the solution of this specification, and the execution is controlled by the processor 201. The processor 201 is used to execute the computer-executable instructions stored in the memory 202, so as to implement the various methods provided in the following embodiments of this specification.

[0132] Optionally, the computer-executable instructions in this specification may also be referred to as application code, instructions, computer program or other names, which are not specifically limited in this specification.

[0133] In a specific implementation, as an embodiment, the processor 201 may include one or more CPUs.

[0134] In a specific implementation, as an embodiment, an electronic device may include multiple processors. Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0135] The electronic device mentioned above can be any device with data processing function, for example, it can be a computer device with data processing function. Figure 2 It should be understood that the electronic device shown in the figure is only an example, and in actual applications, the electronic device may have more Figure 2 More or fewer components may be shown, two or more components may be combined, or there may be a different configuration of components.

[0136] As follows, the technical solution provided in this specification is described in detail.

[0137] Figure 3 FIG. 2 shows an example of a process of a ray tracing method according to an embodiment of the present application. Figure 3 , the method comprising:

[0138] S101. A device executes a first shader for ray tracing through at least one first thread set.

[0139] The first shader includes a first compiled code, and the first compiled code includes a condition for reordering threads corresponding to rays involved in ray tracing.

[0140] Optionally, the first compiled code includes any of the following codes: a compilation annotation or a precompilation instruction or a preprocessing instruction. For example, the following takes the compilation annotation Pragma including the reordering condition as an example for explanation, but this does not constitute a limitation on the specific implementation method of the reordering condition in the compiled code, and is explained uniformly here.

[0141] Pragma is a compiler-related concept, which can assist the compiler in program control. Pragma can be used to instruct the compiler to complete certain specific actions. In conjunction with the embodiments of the present application, developers can define Pragma in the shader, which contains the reordering conditions of rays in ray tracing. The compiler can compile the shader into a binary file and pass the binary file to the EU of the GPU. Among them, the compiled binary file contains ray tracing instructions. When the compiler compiles, the ray reordering conditions (such as the above hint0-hint2) are used as the source operands of the ray tracing instructions. The ray reordering conditions can be used as the source operands of the traceRayEXT() function in the shader.

[0142] Optionally, shaders (such as Ray Generation shader, Closest-Hit shader and Missshader) are stored on external storage media, and developers specify light reordering conditions (such as hint0-hint2 mentioned above) by adding pragma in the shader.

[0143] In this way, the compiler can read the ray reordering conditions in Pragma and send the ray reordering conditions as instructions to the GPU. The GPU can reorder the rays involved in ray tracing according to the ray reordering conditions carried by the instructions.

[0144] For example, Figure 4 , the left side is the Ray Generation shader written by the developer. The Ray Generation shader contains Pragma, such as #pragma hw_rewarp_memory_hint(hint0), which contains the reordering condition hint0.

[0145] The Ray Generation shader also contains #pragma hw_rewarp_branch_hint(hint1), which contains the reordering condition hint1.

[0146] The Ray Generation shader also contains #pragma hw_rewarp_controlflow_hint(hint2), which contains the reordering condition hint2.

[0147] The compiler can read the reordering conditions hint0-hint2 in the pragma, and use hint0-hint2 as the source operand of the traceRay function, passing hint0-hint2 to the EU in the GPU through binary instructions. The EU executes the Ray Generation shader through the first thread set (caller warp) to generate rays. Subsequently, the device can reorder these rays according to the ray reordering conditions hint0-hint2 defined by the developer.

[0148] Exemplarily, the reordering condition hint0 is the material of the hit object. Thus, the device may reorder the light rays according to the reordering condition, and classify the light rays hitting the same material part of the same object into the same group, for example, classifying the light rays hitting the leaves of a tree into the same group.

[0149] Exemplarily, the traceRayEXT function is as follows:

[0150] void traceRayEXT(accelerationStructureEXT topLevel,

[0151] uint rayFlags,

[0152] uint cullMask,

[0153] uint sbtRecordOffset,

[0154] uint sbtRecordStride,

[0155] uint missIndex,

[0156] vec3 origin,

[0157] float Tmin,

[0158] vec3 direction,

[0159] float Tmax,

[0160] int payload);

[0161] Taking into account that in some cases, programmers know how to reorder more efficiently, the technical solution of the embodiment of the present application is that before executing ray tracing, the programmer defines the reordering conditions by compiling the code (such as the above-mentioned Pragma), for example, the reordering conditions can be the above-mentioned hint0\1\2. The compiled code can represent the programmer's reordering intention, so that the compiler compiles these reordering information into the instructions, so that the hardware performs ray reordering according to the reordering conditions, thereby improving the execution efficiency of ray tracing. At the same time, compared to the related art in which the reordering conditions are passed by function calls, in the embodiment of the present application, the reordering conditions are passed by compiling the code, and there is no need to modify the standard built-in function traceRayEXT() function. Since the standard built-in function traceRayEXT() function is not modified, and the standard function call relationship is not changed, it is compatible with various ray tracing applications (such as game applications).

[0162] S102: The device determines information about the object hit by the light.

[0163] As a possible implementation manner, the RTU in the GPU intersects the rays generated by the threads in the first thread set to obtain information about the rays hitting the objects (which may be referred to as hit information).

[0164] S103: The device groups threads in at least one first thread set according to a reordering condition and information about an object hit by the light ray to form at least one second thread set.

[0165] As a possible implementation manner, a thread re-warping station (TRS) may be added to the GPU, and the TRS may be used to execute S103.

[0166] As a possible implementation method, S103 can be implemented as follows: grouping the threads in at least one first thread set according to the identifier of the second shader corresponding to the object hit by the light to form at least one third thread set; grouping the threads in at least one third thread set according to the reordering condition to form at least one second thread set.

[0167] Optionally, the reordering condition indicates that threads consistent with the first feature are divided into the same group; the first feature includes at least one of the following features: the material of the object hit by the generated light, the texture of the object hit by the generated light, the executed branch, and the number of executed loops. In this way, the device can divide threads executing the same branch into the same second thread set, or divide threads with the same number of loops of the loop body into the same second thread set, or divide threads with the same material (or texture) of the object hit by the generated light into the same second thread set, so that the same second thread set executes the same shader as much as possible, thereby improving the execution efficiency of ray tracing.

[0168] Optionally, the reordering conditions may be of multiple types, such as Memory, Branch, and Control-flow hints. Different hints have different weights when doing reordering in hardware.

[0169] First, the rays are grouped according to the objects they hit. Figure 5 , assuming that 9 threads in warp0 generate 9 rays, 4 of which hit leaves, 2 hit tree trunks, and 3 hit stones. Using the reordering method of the embodiment of the present application, the 4 rays hitting leaves, 2 rays hitting tree trunks, and 3 rays hitting stones are divided into different warps. Figure 5 (a), using the embodiment of the present application, within a processing time, the threads in each warp execute the same shader as much as possible. In addition, within a processing time, multiple threads in warp1 execute shaders for leaves, multiple threads in warp2 execute shaders for trunks, and multiple threads in warp3 execute shaders for stones, that is, three warps can execute different shaders in parallel, thereby improving execution efficiency.

[0170] Compared with the method of the embodiment of the present application, Figure 5 (b) shows the processing flow of the related technology. Since the light rays are not reordered, the light rays are disorganized. In the first processing time, two light rays hit the leaves, and multiple threads in warp0 execute part of the shader code of the leaves. In the second processing time, multiple threads in warp0 execute the shader of the trunk. In the third processing time, multiple threads in warp0 execute the shader of the stone. In the fourth processing time, two more light rays hit the leaves, and multiple threads in warp0 execute the rest of the shader code of the leaves. It can be seen that the shader code hitting the same object may be scattered in multiple processing time periods, and the execution efficiency is low. In contrast, in the embodiment of the present application, since the light rays are reordered, the shader code hitting the same object is concentrated in one processing time as much as possible, and the execution efficiency is high.

[0171] Afterwards, the device may reorganize the first thread set caller warp corresponding to the light according to the situation that the light hits the object, and obtain the second thread set callee warp. The caller warp is the warp that executes the parent function, and the callee warp is the warp that executes the child function.

[0172] As a possible implementation, when each thread of N (positive integer) warps calls the traceRayEXT() function, generates light, and reorders the light according to the objects hit by the light and the reordering conditions of the light (such as the above-mentioned hint0, etc.), these threads can be reorganized into M (positive integer) warps. The values ​​of N and M are the same or different. In one example, after executing the traceRayEXT() function in the Ray Generation shader, traceRayEXT() can call the function in the Closest-hit shader or the function in the Miss shader. After the called function is executed, it returns to the Ray Generation shader for execution. In this example, the above-mentioned warp that executes the parent function of traceRayEXT() is called a caller warp. The warp that executes the child function formed by reorganizing the caller warp is called a callee warp. In other examples, the caller warp that executes the parent function and the callee warp that executes the child function can also be other, and the embodiments of the present application are not limited to this.

[0173] Still Fig. 6ADuring the ray tracing process, caller warp0 executes the Ray Generation shader. In caller warp0, the rays generated by threads 1 and 2 hit object 1, and the rays generated by threads 3 and 4 hit object 2. Caller warp1 executes the Ray Generation shader. In caller warp1, the rays generated by threads 5 and 6 hit object 2, and the rays generated by threads 7 and 8 hit object 1.

[0174] As a possible implementation, when a caller warp executes a function call, the device passes the stack frame ID corresponding to the caller warp and the Lane ID corresponding to each thread in the caller warp to all threads in the caller warp. Fig. 6A , the device allocates stack frame S0 for caller warp0, S0 can be used for temporary data required for caller warp0 execution and parameter passing between functions. The device passes the stack frame ID of caller warp0 (S0's ID) and the Lane IDs of threads 1-4 (such as lane1-4) to threads 1-4. The device allocates stack frame S1 for caller warp1, S0 can be used for temporary data required for callerwarp1 execution and parameter passing between functions. The device passes the stack frame ID of caller warp1 (S1's ID) and the Lane IDs corresponding to threads 5-8 (such as lane5-8) to threads 5-8.

[0175] Afterwards, the device reorders the rays according to the shader ID corresponding to the object hit by the rays and the reordering conditions defined by Pragma, and reorganizes the threads corresponding to the rays. Fig. 6A , threads 1, 2, 7, 8 (an example of the first thread set) that generate rays that hit object 1 are reorganized into warp 2. Threads 5, 6, 3, 4 that generate rays that hit object 2 are reorganized into warp 3.

[0176] Afterwards, the device may further group threads in warp2 and warp3 (an example of a third thread set) according to the reordering condition. Figure 6B Assuming that the light rays generated by threads 1 and 2 in warp2 hit the granite stone, and the light rays generated by threads 7 and 8 hit the marble stone, the device can divide threads 1 and 2 into the same warp, such as callee warp4 (an example of the second thread set), and divide threads 7 and 8 into another warp, such as callee warp5.

[0177] It should be noted that, in the above, the example is taken that the device preferentially groups the threads of the first thread set according to the information of the object hit by the light, and then groups the threads again according to the grouping situation and the reordering condition to obtain the second thread set. In other embodiments, the device can also group the threads of the first thread set according to the information of the object hit by the light and the reordering condition to obtain the second thread set.

[0178] As a possible implementation, the device stores the light information in a Figure 7 In the corresponding entries shown, each entry may include several fields.

[0179] Among them, the Vld field segment: can occupy 1-bit, indicating whether the current entry is valid. If the entry is valid, it means that the entry stores light information; otherwise, the entry is invalid, which means that there is no light information in the entry. Exemplarily, initially, each entry is invalid, and the light generated in ray tracing is stored in one of the entries. The subsequently generated light is matched with the valid entry. If the light is of the same type as the light in the valid entry, for example, the direction is similar and it hits the same part of the same object, the light information is written into the valid entry. On the contrary, if the light is not of the same type as the light in the valid entry, the light information is written into the invalid entry, and so on, and each light generated in ray tracing is written into the corresponding entry.

[0180] Pragma Hint field: can be used to indicate ray reordering conditions, such as hint0-hint2 above.

[0181] shader ID: can be used to indicate the identity of a shader, such as the identity of the shader executed by a warp.

[0182] The shader ID and Pragma Hint fields can be used together to determine whether a ray matches the entry. In an example, when the shader ID of the object hit by the ray is the same as the shader indicated by the entry, and the ray meets the hint reordering condition indicated by the entry, the ray matches the entry and the information of the ray is stored in the entry.

[0183] Ray INfo segment: If the characteristics of the ray match an entry, the information of the ray is written into the Ray INfo segment of the entry. For example, if the ray hits a leaf, and the shader indicated by the shader ID segment in entry1 includes the shader corresponding to the leaf, the information of the ray can be written into entry1.

[0184] Config domain: can be used by the GPU to read the information corresponding to the shader indicated by the shader domain from the shader binding table (SBT). The device maintains the shaders corresponding to all objects through SBT. The GPU can know the shader corresponding to the object by querying SBT based on the information of the hit object. The information corresponding to the shader includes but is not limited to any one or more of the following information: program counter (PC), general purpose register (GPR) usage. For example, the entry is used to render leaves, the storage address of the shader for rendering leaves in memory, the program counter corresponding to this shader code, the number of registers for this code, etc. The GPU obtains this information in order to obtain the shader code for the leaves at the corresponding storage address and execute the corresponding leaf rendering process.

[0185] SF ID field: GPU can apply for stack frame (SF) from stack frame manager (SFM) and save SF ID. The stack frame can store all temporary data needed by functions (such as shaders).

[0186] Warp ID field: is the identifier of the warp to which this entry corresponds. Optionally, one warp corresponds to one entry, which stores the relevant information required for multiple threads in the warp to perform ray tracing, such as storing information about 32 rays generated by 32 threads in the warp. Of course, the correspondence between warps and entries is not limited to one-to-one.

[0187] Rdy field: can occupy 1-bit, indicating whether the current entry meets the sending conditions. Optionally, the condition can be that the entry is full, such as 32 rays have been written to the entry. Optionally, the condition can also be that the entry is not full, but the write timeout, such as only 20 rays have been written, but the time threshold has been exceeded. The embodiments of the present application do not limit the conditions involved.

[0188] For each ray, the device determines the entry that matches the ray. For example, if the shader ID corresponding to the object hit by the ray is the same as the shader ID indicated by entry1, and the hint condition indicated by entry1 is the same, the device adds the information of the ray to entry1. In this way, through reordering, the device can add the information of rays with different characteristics (such as ID, etc.) to different entries, and execute subsequent shader calls accordingly.

[0189] In the embodiment of the present application, before a new warp is executed, the device dynamically allocates a stack frame (SF) ID to the warp.

[0190] Each stack frame can support the execution of one warp, and the stack frame is used to store data required for the warp execution, such as temporary data and parameters passed between functions.

[0191] As a possible implementation method, the device can assign a dynamic stack frame ID to each warp, so that in SIMD scenarios, it can meet the usage requirements of different Divergence situations, to prevent different functions from stepping on each other when calling, such as preventing temporary data between two functions from stepping on each other to avoid data errors.

[0192] For example, Fig. 8A , the device allocates stack frame S0 for warp1 and stack frame S1 for warp2.

[0193] As a possible implementation, Figure 8B In the offline phase, the driver layer of the device applies for N (positive integer) stack frame spaces in the process space according to at least one of the following information: the number of graphics processing cores (GPCs) in the GPU, the maximum number of concurrent Ray Generation warps, the maximum iteration depth, and the SIMD width of the warp. Optionally, N = (1 + SIMD_W * max_recur_depth) * warp_num * gpc_num. Among them, SIMD_W represents the SIMD width of the warp, max_recur_depth represents the maximum iteration depth of ray tracing, warp_num represents the maximum number of concurrent Ray Generation warps, and gpc_num represents the number of GPCs in the GPU. Ray Generation warp represents the warp that executes the Ray Generation shader.

[0194] As a possible implementation, the compiler traverses all shader functions and takes the maximum number of stack frames required by all shaders. Subsequently, each time a new warp is created, the device assigns a SF ID to the new warp.

[0195] S104: The device executes a second shader for ray tracing through at least one second thread set, where the second shader is a shader corresponding to an object hit by the ray.

[0196] For example, Figure 6B, callee warp4 is reorganized to execute the closest-hit shader corresponding to the tree trunk (an example of the second shader), and callee warp5 executes the closest-hit shader corresponding to the stone.

[0197] In the embodiment of the present application, since the threads that generate light rays are grouped, the threads in the same thread set are concentrated as much as possible within a processing time to execute the same shader (the second shader), and the execution efficiency is high.

[0198] The present application also provides a ray tracing method, such as Fig. 9 , the method comprising:

[0199] S201. When the caller warp suspends execution, the device writes the caller warp information into the stack frame status table.

[0200] Exemplarily, the caller warp executes a SUSPEND instruction to suspend execution, and the device may write the caller warp information into the stack frame status table.

[0201] Optionally, as shown in Table 1, the stack frame status table may include at least one of the following information: the number of valid threads of the caller warp, the return program counter (PC) address, and the number of warp resources. The valid thread refers to the thread that executes the ray generation function (such as the traceRayEXT() function).

[0202] In a warp, not all threads are valid. For example, the programmer instantiates a total of 300 threads, but the hardware may organize warps with a granularity of 32, so the last warp has only 12 threads, and the number of valid threads of the warp is 12.

[0203] For example, during the execution of a warp, conditional judgments may occur, with some threads executing if branches and some threads executing else branches. Not all threads will generate light. For example, only 10 threads out of 32 threads generate light, and the remaining 22 threads do not generate light. In this case, the number of effective threads is 10.

[0204] Table 1

[0205]

[0206] S202 . When the callee warp finishes executing, the device reduces the number of valid threads in the stack frame status table corresponding to the callee warp by one.

[0207] As a possible implementation, the subtraction operation may be an atomic subtraction.

[0208] Atomic operations refer to operations that cannot be interrupted. For example, if 1,000 threads of a GPU are executed simultaneously, and threads 9 and 17 are both from warp7, then after threads 9 and 17 are executed, the device will operate the stack frame state table of warp7 to reduce the number of valid threads. The result of the execution is that the number of valid threads of warp7 is reduced by 2. For atomic subtraction, the hardware can ensure serial subtraction, for example, thread 9 is reduced by one first, and thread 17 is reduced by one later, so the total number of valid threads can be guaranteed to be reduced by 2.

[0209] Alternatively, the subtraction operation may also be a non-atomic operation, as long as the calculation of the number of effective threads can be guaranteed to be accurate, such as a serial subtraction operation, and the embodiments of the present application do not impose any restrictions on this.

[0210] As a possible implementation method, when the callee warp executes a function, it needs to index the corresponding stack frame according to the stack frame ID and obtain the required input parameters in the stack frame. For example, obtain the information of the first ray from the stack frame that saves the information of the ray.

[0211] For example, Figure 6B For object 1, threads 1 and 2 of reorganized callee warp4 read the required input parameters from stack frame S0, and combined with the input parameters, occupy lane 1 and lane 2 respectively to execute the closest-hit shader. Threads 7 and 8 of reorganized callee warp5 read the required input parameters from stack frame S1, and combined with the input parameters, occupy lane 7 and lane 8 respectively to execute the closest-hit shader. For example, callee warp4 executes the closest-hit shader corresponding to the tree trunk, and callee warp5 executes the closest-hit shader corresponding to the stone.

[0212] It can be seen that after the above-mentioned ray reordering, the device can reorganize the threads that generate rays that hit the same object into corresponding callee warps, and execute the corresponding shader through the reorganized callee warps. Since the rays generated by the threads in the caller warp corresponding to the reorganized callee warp all hit the same object, multiple threads in a callee warp can concentrate on executing the same shader corresponding to the corresponding object within the same period of time, thereby improving the execution efficiency of ray tracing.

[0213] In the embodiment of the present application, in order to improve the memory access efficiency of ray tracing, a method for managing stack frames is proposed. Before the sub-function is executed, the device copies the information stored in the stack frame from the stack frame address corresponding to the caller warp to the stack frame of the callee warp that executes the sub-function. After the sub-function is executed, some new information is added to the stack frame, and the device copies the new information to the stack frame of the caller warp.

[0214] For example, Fig. 10A ,After the rays are reordered, the threads corresponding to the rays can be reorganized accordingly. For example, the three threads in caller warp0 and the three threads in caller warp7 that execute the parent function can be reorganized into callee warp18.

[0215] Before callee warp 18 that executes the sub-function is executed, the device allocates stack frame 27 for callee warp 18. The storage addresses in stack frame 27 are continuous, and the information (such as temporary variables) in stack frame 19 of caller warp0 and stack frame 99 of caller warp7 are copied to the allocated stack frame 27. In this way, when the thread in callee warp18 executes the sub-function, it can access the continuous address space in stack frame 27, improve the memory access efficiency, and then improve the efficiency of ray tracing.

[0216] As a possible implementation method, take the sub-function as Closest-hit shader as an example, Fig. 10B Before the Closest-hit shader is executed, the device calculates the stack frame address corresponding to the parent function according to the compiled load code, and copies the information from the stack frame address corresponding to the parent function (such as traceRayEXT()) to the stack frame of the Closest-hit shader. After the copy is completed, the Closest-hit shader is executed. After the Closest-hit shader is executed, the device copies the information (such as temporary data) generated by executing the Closest-hit shader (an example of the second shader) to the stack frame corresponding to traceRayEXT() (an example of the second stack frame) according to the store code in the compiled code.

[0217] As a possible implementation, when the callee warp is executed, for example, when the RETUREN instruction is executed, the hardware will access the stack frame status table of the caller warp according to the caller SF ID of each thread in the callee warp, and perform an atomic subtraction operation on the number of valid threads in the stack frame status table. When the number of valid threads of the caller warp is reduced to 0, the execution of the caller warp is resumed.

[0218] For example, caller warp7 executes the parent function traceRayEXT(). Initially, the stack frame status table of caller warp7 is as shown in Table 2. The number of valid threads executing traceRayEXT() is 6.

[0219] Table 2

[0220]

[0221]

[0222] Fig. 10A In callee warp18, which executes the closest-hit shader, three threads come from caller warp7. After these three threads finish executing the closest-hit shader, they access the stack frame status table of caller warp7 and atomically reduce the number of valid threads of caller warp7 in Table 2 from 6 to 3.

[0223] Similarly, in warp20 that executes the closest-hit shader, three threads come from callerwarp7. After executing the closest-hit shader, these three threads access the stack frame state table of callerwarp7 and atomically reduce the number of valid threads of callerwarp7 from 3 to 0.

[0224] The effective threads of caller warp7 are reduced to 0, indicating that all callee threads corresponding to caller warp7 have been executed. For example, the calling functions of the six rays in caller warp7 have been executed. In this case, the execution of caller warp7 can be resumed.

[0225] As a possible implementation, the caller warp that is resumed uses the SF ID that was initially allocated to access the stack frame without applying for a new SF ID. Alternatively, the caller warp that is resumed can also use the newly allocated SFID to access the stack frame. This embodiment of the application does not limit this.

[0226] S203: When the number of valid threads of the caller warp is 0, the device resumes executing the caller warp according to the PC address.

[0227] In this way, by allocating a continuous stack frame space (an example of the first stack frame) for the callee warp and copying the information in the stack frame space corresponding to the caller warp (such as temporary variables, etc.) into the stack frame space, it is possible to make the private memory access inside the callee warp consistent and continuous when executing dynamic non-consistent function calls on SIMD / SIMT hardware, thereby reducing the probability of memory divergence and solving the problem of low ray tracing execution efficiency caused by accessing from discontinuous addresses.

[0228] In addition, the above method does not require complicated stack pointer management, so as to avoid device performance degradation caused by stack pointer management.

[0229] In combination with the above method, the present application embodiment also provides a device, which can be used to implement the above method. Fig.11 , the device may include a GPU. The GPU may include the following modules or units:

[0230] Execution unit (EU) 702: can be used to execute compiled binary files. Exemplarily, in an embodiment of the present application, EU can be used to execute the ray generation function (such as traceRay EXT() function) in the compiled shader, and the programmer-defined reordering condition hint is used as the source operand of the ray tracing instruction. When EU executes the TraceRay EXT() function, it generates rays. EU sends a ray intersection instruction to RTU, which carries the information required for ray intersection, and carries the SF ID of the caller warp and the lane ID of the thread in the caller warp. The source operand of the ray intersection instruction is the reordering condition. Fig. 10A , the EU passes the SF ID (19) of caller warp0, the SF ID (99) of caller warp7, the lane ID of the thread in caller warp0, and the lane ID of the thread in caller warp7 to the RTU.

[0231] Ray tranversal unit (RTU) 703: can be used to perform traversal (such as acceleration structure traversal) and intersection on the input rays, and write the ray intersection information into the ray hit buffer. In addition to the information mentioned above, the ray intersection information can also include the SF ID corresponding to the ray. Optionally, the RTU can also pass the Pragma as the source operand to the ray hit buffer.

[0232] RTU can also be used to send ray intersection information to TRS. For example, after ray intersection, each thread generates a ray corresponding to an SF ID each time it executes traceRayEXT(). RTU can send the SF ID corresponding to each ray to TRS, and TRS performs ray reordering.

[0233] RTU can also be used to push the SF ID of each warp involved in ray tracing to EU. Fig. 10A , the RTU pushes the SF IDs corresponding to warp0, warp7, warp18, and warp20 to the EU, and the EU pushes each SF ID to the register (such as GPR) of the EU, for example, to register R0. In this way, the device can calculate the stack frame address corresponding to the warp (caller warp) that executes the parent function in R0 based on the SFID corresponding to the warp. For example, multiply the value of R0 (SF ID) by the size of SF, and then add the base address of SF to get the base address of SF of caller warp. That is, the base address of SF of caller warp can be obtained based on R0.

[0234] In this way, the device can copy information in the stack frame of the caller warp to the stack frame corresponding to the callee warp from the stack frame address corresponding to the callee warp, so as to achieve memory access to the continuous address.

[0235] Ray hit buffer 704: can be used to temporarily store ray intersection information. Optionally, the ray hit buffer is composed of on-chip and off-chip storage. Since the ray hit buffer can buffer ray intersection information, the EU can release GPR and other related resources on the EU in a timely manner after sending a ray intersection instruction to the RTU, so that the released resources can be used for other execution processes in a timely manner, which can improve the execution concurrency of ray tracing and reduce the overall execution delay. For example, new rays are subsequently sent to the EU, or reordered rays are sent to the EU.

[0236] Thread re-warping station (TRS) 705: This embodiment of the application adds a thread re-warping station for performing light reordering. As a possible implementation, TRS can obtain the shader ID corresponding to the object hit by the ray and the above-mentioned source operand from the ray hit buffer, and reorder the rays input into the ray hit buffer 704 according to the shader ID (or shader index) corresponding to the object hit by the ray and the above-mentioned source operand, and store the rays with different characteristics into a file such as Figure 7 In the different warp entries shown in FIG. 1 , warp reorganization is implemented. Figure 6B , threads 1 and 2 that generate rays that hit object 1 are reorganized into callee warp 4. Threads 7 and 8 that generate rays that hit object 1 are reorganized into callee warp 5. Alternatively, TRS may also obtain shader ID and source operands in other ways, which are not limited in the present embodiment.

[0237] Optionally, the TRS may apply to the stack frame manager (SFM) 707 for an SF ID for the reassembled callee warp.

[0238] Optionally, the TRS may apply for the warp ID of the callee warp from the warp manager (WM) 708. Optionally, the TRS may also obtain other required configuration parameter information to implement warp reorganization.

[0239] When the warp reorganization is completed, the hardware indicator bit will be pulled high, and TRS can send the light reordering result to EU, and EU calls the corresponding shader according to the reordering result. Fig. 10A , the Closest-Hit shader is executed by the reorganized callee warp18 and calleewarp20. For example, TRS sends the following information back to EU: caller warp stack frame ID, caller warp lane ID, callee warp stack frame ID, ray ID.

[0240] Warp restore unit (WRU) 706: can be used to detect whether the caller warp corresponding to each thread in the warp that has ended or paused can resume execution, and perform a recovery process;

[0241] Resume the execution of the caller warp. When the number of remaining valid threads of the caller warp is reduced to 0, the WRU executes the recovery process of the caller warp. As a possible implementation method, the WRU reads the return PC address (next_pc), the number of valid threads and other information from the stack frame status table. When the number of remaining valid threads of the caller warp is reduced to 0, the WRU initiates an application to the TRS, and the TRS applies to the WM 708 to assign a new warp ID to the caller warp. The TRS can send the new warp ID of the caller warp to the EU, and the EU resumes the execution of the caller warp.

[0242] SFM 707: can be used to dynamically allocate, recycle, and manage stack frame IDs.

[0243] WM 708: Can be used to dynamically allocate, recycle, and manage warp IDs.

[0244] Optionally, the above device may further include a compiler 701 .

[0245] Fig.12 Another example structure of the device provided by the embodiment of the present application is shown, and the device may be a device or chip including a GPU, or the device is a GPU. The GPU may include one or more GPCs, each of which includes an EU, an LSU, a TU, an RTU, and a TRS. Among them, the EU can be used to execute a shader, one or more LSUs can be used to execute a load-store operation, one or more TUs can be used to execute a texture sampling operation, and one or more RTUs can be used to execute a ray intersection operation. Optionally, multiple GPCs can share an RTU.

[0246] Optionally, the GPU may also include Fig.11 The illustrated ray hit buffer 704, WRU 706, SFM 707, at least one unit or module in WM 708. The embodiment of the present application does not limit the specific structure of the GPU.

[0247] Fig.12 The connection relationship between modules and units in the function can be referred to Fig.11 The relevant description will not be repeated here. Fig.13 Another example structure of the device provided by an embodiment of the present application is shown, in which a TRS can be shared between multiple GPCs, and multiple GPCs can achieve light reordering through the shared TRS.

[0248] like Fig.14In this structure, multiple GPUs can share TRS, and multiple GPUs can implement ray reordering through the shared TRS. As a possible implementation method, multiple GPUs can be connected through on-chip buses, networks, or shared storage. Fig.14 Take bus connection as an example, Fig.15 Taking network connection as an example, this does not constitute a limitation on the connection method between GPUs.

[0249] like Fig.16 Another example structure of the device provided by the embodiment of the present application is shown, and the embodiment of the present application supports a decoupled interconnection architecture of multiple GPCs and multiple RTUs. Fig.16 , the GPU may include a GPC subsystem, an RTU subsystem and a TRS. Among them, the GPC subsystem may include an EU, an LSU and a TU. The RTU subsystem includes at least one RTU. The RTU subsystem and the GPC subsystem may be connected to the TRS via a certain connection method (including but not limited to a bus method), and light reordering is implemented through the TRS.

[0250] As mentioned above, a thread set (such as caller warp) can generate light by executing the ray tracing function (such as traceRay() function) in Shader (such as Ray Generation Shader, Closest-Hit Shader, Miss Shader), and intersect the light with the corresponding object. According to the ray intersection information and the input parameters of the ray tracing function (such as the reordering conditions and Shader index defined in Pragma), the light is reordered and the threads that generate the light are reorganized. After the threads are reorganized, each reorganized thread set (such as callee warp) can call its own Shader (such as Closest-Hit Shader or Miss Shader). Ray tracing is then realized.

[0251] In the embodiment of the present application, reordering the light rays may also be referred to as grouping the light rays.

[0252] In other scenarios, a thread set (such as a caller warp) can execute a callback function, and the callback function can call a shader. Optionally, the callback function can be, but is not limited to, an ExecuteCallable() function. Taking the ExecuteCallable() function as an example, the thread set can be reorganized according to the input parameters of the ExecuteCallable() function (such as the reordering conditions and Shader index defined in Pragma). After reorganizing the thread set, each reorganized thread set (such as a callee warp) can call its own shader (such as a Closest-Hit Shader or a Miss Shader). In this way, it is easy to execute functions and try to make a reorganized thread set execute the same shader within a period of time, thereby improving execution efficiency.

[0253] The process of calling the shader by the ExecuteCallable() function is similar to the process of calling the shader by the traceRay() function mentioned above. The difference is that when executing the ExecuteCallable() function, no light is generated, and there is no need to intersect the light with the corresponding object. Instead, the reordering is performed directly according to the reordering conditions (such as hint0, etc.) and the Shader index specified by the input parameter of the function.

[0254] like Fig.17 The flowchart of ExecuteCallable() function calling shader is shown. EU directly sends ExecuteCallable() function calling instruction to TRS for sorting. TRS returns the information of sorted and reorganized threads to EU, and EU calls the corresponding shader.

[0255] Similarly, for the ExecuteCallable() function, after the EU sends the ExecuteCallable() function call instruction to the TRS, it can send the thread information (such as the shader ID and the ray reordering condition Hint defined by Pragma) into the Ray Buffer and release the EU's resources in time.

[0256] It should be noted that the above-mentioned multiple embodiments can be combined and the combined scheme can be implemented. Optionally, some operations in the process of each method embodiment are optionally combined, and / or the order of some operations is optionally changed. In addition, the execution order between the steps of each process is only exemplary and does not constitute a limitation on the execution order between the steps. There can also be other execution orders between the steps. It is not intended to indicate that the execution order is the only order in which these operations can be performed. Ordinary technicians in this field will think of many ways to reorder the operations of this article. In addition, it should be pointed out that the process details involved in a certain embodiment of this article are also applicable to other embodiments in a similar manner, or different embodiments can be used in combination.

[0257] In addition, some steps in the method embodiment may be equivalently replaced by other possible steps. Alternatively, some steps in the method embodiment may be optional and may be deleted in certain usage scenarios. Alternatively, other possible steps may be added in the method embodiment. Alternatively, the execution subject (such as a functional module) of some steps in the method embodiment may be replaced by other execution subjects.

[0258] Furthermore, the above method embodiments may be implemented separately or in combination.

[0259] Some other embodiments of the present application provide a device, which may be the above-mentioned terminal or server, etc. The device may include: a memory and one or more processors. The memory and the processor are coupled. The memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the device may perform the various functions or steps performed by the corresponding device in the above-mentioned method embodiment. The structure of the device may refer to Figure 2 The equipment (device) shown.

[0260] The core structure of the device can be expressed as Fig.18 In the structure shown, the device includes: a processing module 1301 and a storage module 1303.

[0261] The processing module 1301 may include a GPU. The processing module 1301 may also include at least one of a central processing unit (CPU), an application processor (AP), or a communication processor (CP). The processing module 1301 may perform operations or data processing related to the control and / or communication of at least one of the other elements of the user equipment. Exemplarily, the processing module 1301 may be used to perform the above S101-S104, S201-S203.

[0262] The storage module 1303 may include a volatile memory and / or a non-volatile memory. The storage module is used to store at least one instruction or data related to other modules of the user equipment device.

[0263] Optionally, the device may further include an input module 1302 and a display module 1304

[0264] The input module 1302 is used to obtain the instructions or data input by the user and transmit the obtained instructions or data to other modules of the device. Specifically, the input mode of the input module 1302 may include touch, gesture, approaching the screen, etc., or voice input. For example, the input module may be the screen of the device, obtain the user's input operation and generate an input signal according to the obtained input operation, and transmit the input signal to the processing module 1301.

[0265] The display module 1304 may include, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a micro-electromechanical system (MEMS) display, or an electronic paper display, and is used to display content (e.g., text, images, videos, icons, symbols, etc.) that can be viewed by a user.

[0266] Optionally, a communication module 1305 is also included to support personal devices (via a communication network) to communicate with other personal devices. For example, the communication module can be connected to a network via wireless communication or wired communication to communicate with other personal devices or network servers. Wireless communication can use at least one of cellular communication protocols, such as long-term evolution (LTE), advanced long-term evolution (LTE-A), code division multiple access (CDMA), wideband code division multiple access (WCDMA), universal mobile telecommunications system (UMTS), wireless broadband (WiBro) or global mobile communication system (GSM). Wireless communication may include, for example, short-range communication. Short-range communication may include at least one of wireless fidelity (Wi-Fi), Bluetooth, near field communication (NFC), magnetic stripe transmission (MST) or GNSS.

[0267] It should be noted that each functional module of the device can execute one or more steps in the above method embodiment.

[0268] The present application also provides a chip system, such as Fig.19As shown, the chip system includes at least one processor 1401 and at least one interface circuit 1402. The processor 1401 and the interface circuit 1402 can be interconnected via lines. For example, the interface circuit 1402 can be used to receive signals from other devices (such as the memory of the device). For another example, the interface circuit 1402 can be used to send signals to other devices (such as the processor 1401). Exemplarily, the interface circuit 1402 can read the instructions stored in the memory and send the instructions to the processor 1401. When the instruction is executed by the processor 1401, the device can execute the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0269] An embodiment of the present application also provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned device, the device executes each function or step executed by the mobile phone in the above-mentioned method embodiment.

[0270] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes each function or step executed by the mobile phone in the above method embodiment.

[0271] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0272] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0273] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0274] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0275] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0276] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A ray tracing method, characterized in that: include: Executing a first shader for ray tracing through at least one first thread set, wherein the first shader includes a first compiled code, and the first compiled code includes a condition for reordering threads corresponding to rays involved in the ray tracing; Determine information of an object hit by the ray; According to the reordering condition and the information of the object hit by the light, the threads in the at least one first thread set are grouped to form at least one second thread set; A second shader is executed by the at least one second thread set, where the second shader is a shader corresponding to the object hit by the light.

2. The method according to claim 1, characterized in that The reordering condition indicates that threads consistent with the first feature are divided into the same group; the first feature includes at least one of the following features: the material of the object hit by the generated light, the texture of the object hit by the generated light, the executed branch, and the number of executed loops.

3. The method according to claim 1 or 2, characterized in that: The grouping of threads in the at least one first thread set according to the reordering condition and the information of the object hit by the light to form at least one second thread set includes: Grouping the threads in the at least one first thread set according to an identifier of a second shader corresponding to the object hit by the light to form at least one third thread set; According to the reordering condition, the threads in the at least one third thread set are grouped to form at least one second thread set.

4. The method according to any one of claims 1 to 3, characterized in that: The executing the first shader comprises: A first shader in the form of a compiled binary file is executed; the binary file includes ray tracing instructions; the reordering condition is a source operand of the ray tracing instructions.

5. The method according to claim 4, characterized in that The grouping of threads in the at least one first thread set according to the reordering condition and the information of the object hit by the light to form at least one second thread set includes: The threads in the at least one first thread set are grouped according to the source operand and the information of the object hit by the light to form at least one second thread set.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Allocating a first stack frame to the second thread set, wherein storage addresses in the first stack frame are continuous addresses; Storing information of the parent function required to execute the second shader into the continuous address; The second shader is executed according to the information of the parent function read from the continuous address, and the information generated by executing the second shader is stored in a second stack frame corresponding to the parent function.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: When the first thread set is suspended, information of the first thread set is written into a stack frame status table; the stack frame status table includes the number of remaining valid threads in the first thread set and a program counter PC address; According to the stack frame status table, when the remaining valid threads are zero, the first thread set is resumed to be executed according to the PC address.

8. The method according to any one of claims 1 to 7, characterized in that: The first compiled code includes any of the following codes: a compilation annotation or a pre-compilation instruction or a pre-processing instruction.

9. A ray tracing device, characterized in that: include: A processing unit, configured to execute a first shader for ray tracing through at least one first thread set, wherein the first shader includes a first compiled code, and the first compiled code includes a condition for reordering threads corresponding to rays involved in the ray tracing; Determine information of an object hit by the ray; According to the reordering condition and the information of the object hit by the light, the threads in the at least one first thread set are grouped to form at least one second thread set; A second shader is executed by the at least one second thread set, where the second shader is a shader corresponding to the object hit by the light.

10. The method according to claim 9, characterized in that The reordering condition indicates that threads consistent with the first feature are divided into the same group; the first feature includes at least one of the following features: the material of the object hit by the generated light, the texture of the object hit by the generated light, the executed branch, and the number of executed loops.

11. The method according to claim 9 or 10, characterized in that: The grouping of threads in the at least one first thread set according to the reordering condition and the information of the object hit by the light to form at least one second thread set includes: Grouping the threads in the at least one first thread set according to an identifier of a second shader corresponding to the object hit by the light to form at least one third thread set; According to the reordering condition, the threads in the at least one third thread set are grouped to form at least one second thread set.

12. The method according to any one of claims 9 to 11, characterized in that: The executing the first shader comprises: A first shader in the form of a compiled binary file is executed; the binary file includes ray tracing instructions; the reordering condition is a source operand of the ray tracing instructions.

13. The method according to claim 12, characterized in that The grouping of threads in the at least one first thread set according to the reordering condition and the information of the object hit by the light to form at least one second thread set includes: The threads in the at least one first thread set are grouped according to the source operand and the information of the object hit by the light to form at least one second thread set.

14. The method according to any one of claims 9 to 13, characterized in that: The processing unit is further used for: Allocating a first stack frame to the second thread set, wherein storage addresses in the first stack frame are continuous addresses; Storing information of the parent function required to execute the second shader into the continuous address; The second shader is executed according to the information of the parent function read from the continuous address, and the information generated by executing the second shader is stored in a second stack frame corresponding to the parent function.

15. The method according to any one of claims 9 to 14, characterized in that: The processing unit is further used for: When the first thread set is suspended, information of the first thread set is written into a stack frame status table; the stack frame status table includes the number of remaining valid threads in the first thread set and a program counter PC address; According to the stack frame status table, when the remaining valid threads are zero, the first thread set is resumed to be executed according to the PC address.

16. The method according to any one of claims 9 to 15, characterized in that: The first compiled code includes any of the following codes: a compilation annotation or a pre-compilation instruction or a pre-processing instruction.

17. A chip system, characterized in that: It comprises a processor and an interface circuit; the processor and the interface circuit are interconnected; the interface circuit is used to read instructions stored in a memory and send the instructions to the processor; when the instructions are executed by the processor, the method according to any one of claims 1 to 8 is implemented.

18. A computer-readable storage medium, characterized in that: The method comprises a program or an instruction. When the program or the instruction is executed, the method according to any one of claims 1 to 8 is implemented.

19. An electronic device, characterized in that: The electronic device comprises a processor and a memory; The memory is used to store computer-executable instructions. When the device is running, the processor executes the computer-executable instructions stored in the memory, so that the device executes the method according to any one of claims 1 to 8.

Images