Thread group creation method, graphics processing unit, and electronic device

Abstract

A thread group creation method, a graphics processing unit, and an electronic device are used to improve the execution efficiency of a graphics processing unit. The thread group creation method includes: obtaining masks of multiple pixel points in an image (S301), wherein the mask of each pixel point is used to indicate whether each pixel point is to be processed by the image; and creating a target thread group based on the masks of the multiple pixel points (S302), wherein the target thread group includes at least one thread for performing image processing.

Description

Technical Field

[0001] The present application relates to the field of graphics processing, and in particular to a thread group creation method, a graphics processing unit (GPU), and an electronic device. Background Art

[0002] In the field of graphics processing, the GPU can create at least one thread group, each of which includes at least one thread. Each thread can perform image processing on one or more pixels, such as ray intersection testing and rendering. Currently, in applications where ray tracing is used to achieve reflection effects, only some reflective surfaces require ray intersection testing and rendering. Therefore, only threads in some thread groups perform image processing. However, thread groups corresponding to other non-reflective surfaces still need to reserve resources, participate in instruction scheduling, execute thread group destruction, and release resources. This occupies GPU processing resources and reduces GPU execution efficiency. Summary of the Invention

[0003] Embodiments of the present application provide a thread group creation method, a graphics processing unit, and an electronic device for improving the execution efficiency of a graphics processing unit.

[0004] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0005] In the first aspect, a thread group creation method is provided, comprising: obtaining masks of multiple pixel points in an image, wherein the mask of each pixel point is used to indicate whether each pixel point is to be processed by image; creating a target thread group according to the masks of the multiple pixel points, wherein the target thread group includes at least one thread for image processing. The above-mentioned thread creation method, graphics processing unit, and electronic device provided in the embodiments of the present application indicate whether the pixel point needs to be processed by image (or whether the corresponding thread needs to be created) through the mask of each pixel point, and then determine whether to create the corresponding thread group. Thread groups that do not perform image processing on any thread do not need to be created, thereby saving GPU processing resources and improving GPU execution efficiency.

[0006] In one possible implementation, obtaining the masks of the plurality of pixels in the image includes: receiving the masks of the plurality of pixels from a central processing unit. The central processing unit may send the masks of the plurality of pixels together with the instruction to draw the geometric figure to the graphics processing unit.

[0007] In one possible implementation, obtaining masks of multiple pixel points in an image includes generating masks of the multiple pixel points based on attributes of the multiple pixel points (e.g., geometric parameters, material information, etc.). The geometric parameters may include the position and depth of the pixel points.

[0008] In one possible implementation, the attributes of the plurality of pixels include material information, and the material information includes at least one of a reflection coefficient, roughness, and a material identifier, wherein the material identifier is used to indicate the material of the pixel, such as water, metal, ceramic, glass, etc.

[0009] In one possible implementation, creating a target thread group based on a mask includes: creating a first thread group based on a mask of a first group of pixels among a plurality of pixels; wherein the first thread group includes at least one thread that does not perform image processing; creating a second thread group based on a mask of a second group of pixels among the plurality of pixels; wherein the second thread group includes at least one thread that does not perform image processing; and merging the threads in the first thread group that perform image processing with the threads in the second thread group that perform image processing to obtain the target thread group. This can further conserve processing resources of the graphics processing unit and improve the execution efficiency of the graphics processing unit.

[0010] In one possible implementation, a thread of the target thread group corresponds to a mask of multiple pixels. One thread can correspond to one pixel, that is, one thread can perform image processing on one pixel, or one thread can correspond to multiple pixels, that is, one thread can perform image processing on multiple pixels.

[0011] In one possible implementation, the system further includes managing mask storage space. The mask storage space can be explicitly managed by the application, for example, by allocating or destroying the mask storage space. Alternatively, the mask storage space can be managed by the GPU driver, without the application being aware of the mask storage space. These applications can include those running on the graphics processing unit (GPU) and those running on the central processing unit (CPU). For example, the GPU driver allocates or destroys the mask storage space, and the application running on the GPU directly obtains the mask through the driver.

[0012] In one possible implementation, the mask is stored in a tile cache of the graphics processing unit, or in a system cache, or in a memory. The storage location of the mask can be very flexible.

[0013] In a second aspect, a graphics processing unit is provided, characterized in that it includes a streaming multiprocessor, and the streaming multiprocessor is used to execute the thread group creation method as described in the first aspect and any embodiment thereof.

[0014] According to a third aspect, an electronic device is provided, comprising the graphics processing unit as described in the second aspect.

[0015] In a fourth aspect, a computer-readable storage medium is provided, in which instructions are stored. The instructions are executed on a graphics processing unit, so that the graphics processing unit executes the thread group creation method described in the first aspect and any embodiment thereof.

[0016] In a fifth aspect, a computer program product comprising instructions is provided, wherein the instructions are run on a graphics processing unit, so that the graphics processing unit executes the thread group creation method described in the second aspect and any embodiment thereof.

[0017] Regarding the technical effects of the second to fifth aspects, refer to the technical effects of the first aspect and any embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0019] Figure 2 A schematic diagram of performing thread group compression during a deferred rendering process provided by an embodiment of the present application;

[0020] Figure 3 A flowchart of a thread group creation method provided in an embodiment of the present application;

[0021] Figure 4 A schematic diagram of another delayed rendering process provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] First, some concepts involved in this application are described:

[0023] Ray tracing is used to simulate the propagation of light in the real world. It does this by emitting light, tracing its bounces in the scene, calculating the intersection of light and objects in the scene, and calculating direct and indirect lighting for the intersection points.

[0024] Ray-plane intersection test refers to testing whether a ray intersects with a geometric figure, and determining the position where the ray intersects with the geometric figure. The ray can be a direct ray from a light source, or a reflected ray from another reflective surface, or a refracted ray, etc.

[0025] Rasterization refers to the process of converting geometric shapes into two-dimensional images.

[0026] Rendering refers to calculating the lighting information of pixels.

[0027] Deferred rendering means that the GPU rasterizes the geometry to obtain the geometry cache (G-buffer). The geometry cache includes the attributes of each pixel in the two-dimensional image in the original geometry (such as geometric parameters, material information, etc.). The GPU can output the geometry cache to the memory (such as double data rate (DDR) memory), then read the geometry cache from the memory and render the image. Instead of directly rendering the geometry after rasterizing it and then outputting it to the memory. The geometric parameters can include the position and depth of a pixel, and the material information can include the reflection coefficient, roughness, material identifier, normal, etc., where the material identifier is used to indicate the material of the pixel.

[0028] A shader program is an editable program that replaces the fixed rendering pipeline to achieve graphics rendering.

[0029] Thread groups (warps). The GPU creates threads in units of warps. Each thread group can include at least one thread, for example, 4 threads or 16 threads. Each thread can perform image processing on one or more pixels, such as ray intersection testing and rendering. Each thread can be represented by a one-dimensional identifier or a multi-dimensional identifier. For example, as shown in Table 1, the first row indicates that there are two thread groups, and the identifiers of these two thread groups are 0 and 1 respectively. Each thread group includes 4 threads, for a total of 8 threads, which can be represented by one-dimensional numbers 0-7 respectively. Alternatively, a thread group can include 4 threads, which can be represented by two-dimensional numbers (0,0), (1,0), (0,1), and (1,1).

[0030] Table 1

[0031] Thread group ID 0 0 0 0 1 1 1 1 Thread ID 0 1 2 3 4 5 6 7

[0032] like Figure 1 As shown, an embodiment of the present application provides an electronic device 10 including a GPU, including a GPU 101, a central processing unit (CPU) 102, a system cache 103, and a memory 104 connected via a bus. The electronic device may also include a display screen (not shown). GPU 101 includes a streaming multiprocessor 1011 and a tile cache 1012. The electronic device can be a device that displays images, such as a mobile phone, a computer, or a tablet.

[0033] After receiving instructions to draw geometric figures from CPU 102, stream multiprocessor 1011 in GPU 101 can process the image in memory 104 and display it on the display screen. System cache 103 and tile cache 1012 can be used to cache intermediate image processing results. In addition, in the embodiment of the present application, memory 104, system cache 103, or tile cache 1012 can be used to store masks (described in detail later).

[0034] As mentioned above, the thread group created by the GPU may waste GPU resources because no image processing is performed, such as Figure 2 As shown, during the deferred rendering process, after rasterizing the geometric graphics to obtain an image, the GPU can first perform thread group compression. For the thread groups that do not perform image processing, no resources will be reserved or created. The remaining thread groups will then perform image processing (such as ray intersection testing and rendering), thereby saving GPU processing resources and improving GPU execution efficiency.

[0035] If thread group compression is treated as a separate process, it will still consume additional GPU processing resources. In the embodiment of the present application, the mask of each pixel point is used to indicate whether the pixel point needs to be image processed (or whether the corresponding thread needs to be created), and then determine whether to create the corresponding thread group. Thread groups that do not perform image processing on any thread do not need to be created, thereby saving GPU processing resources and improving GPU execution efficiency. It can be applied to scenes in ray tracing when only local areas need to be ray traced, and can also be applied to scenes in global illumination when only local areas need to be rendered by global illumination.

[0036] The GPU provided in the embodiment of the present application (for example, the stream multiprocessor therein) can perform the following operations: Figure 3 The thread group creation method shown:

[0037] S301: Obtain masks of multiple pixel points in an image.

[0038] Among them, the mask of each pixel point is used to indicate whether each pixel point needs to be image processed (for example, perform ray intersection testing and rendering), or, the mask of each pixel point is used to indicate whether each pixel point needs to create a corresponding thread. Exemplarily, as shown in Table 2, a mask value of 0 for each pixel point indicates that the pixel point does not need to be image processed (or does not need to create a corresponding thread), and a mask value of 1 for each pixel point indicates that the pixel point needs to be image processed (or needs to create a corresponding thread). One thread can correspond to one pixel point, that is, one thread can perform image processing on one pixel point, or one thread can correspond to multiple pixels, that is, one thread can perform image processing on multiple pixels.

[0039] The GPU can manage the storage space for the mask. For example, the mask storage space can be explicitly managed by the application, such as by the application allocating or destroying the mask storage space. Alternatively, the mask storage space can be managed by the GPU driver, and the application is unaware of the mask storage space. The above-mentioned applications can include applications running on the graphics processing unit and applications running on the central processing unit. For example, the GPU driver allocates or destroys the mask storage space, and the application running on the GPU directly obtains the mask through the driver. The mask can be stored in the GPU's tile cache, or in the system cache, or in memory. The storage location of the mask can be very flexible.

[0040] The process of obtaining masks for multiple pixels in an image can occur after rasterizing the geometry to generate a geometry buffer (G-buffer). Assuming a G-buffer size of 1280x720 pixels, if the mask corresponding to each pixel is represented by 8 bits, then as shown in Table 2, the GPU can pre-allocate a 1280x720x8-bit buffer for storing the mask. If the mask corresponding to 8x4 pixels is represented by 32 bits, that is, one bit represents one mask, then as shown in Table 3, the GPU can pre-allocate a 160x180x32-bit buffer for storing the mask.

[0041] Table 2

[0042] 0 1 ...... 718 719 0 00000000 00000001 ...... 00000000 00000000 1 00000001 00000001 ...... 00000001 00000000 ...... ...... ...... ...... ...... ...... 1278 00000000 00000001 ...... 00000001 00000000 1279 00000000 00000000 ...... 00000000 00000000

[0043] Table 3

[0044]

[0045]

[0046] In a possible implementation, the GPU may obtain masks of multiple pixel points from the CPU. For example, when the CPU sends an instruction to draw a geometric figure to the GPU, it may also send masks of multiple pixel points.

[0047] In another possible implementation, the GPU may update the mask of each pixel, for example, by generating the mask of the multiple pixels based on the attributes of the multiple pixels (e.g., geometric parameters, material information, etc.). The geometric parameters may include the position and depth of the pixel, and the material information may include at least one of the reflectance, roughness, material identifier, and normal of the pixel. The material identifier is used to indicate the material of the pixel, such as water, metal, ceramic, or glass.

[0048] Take the application scenario of the reflection effect of each pixel point achieved by GPU in ray tracing as an example:

[0049] For example, if a pixel is located on the light-facing side, that is, light reflection will occur, then the mask corresponding to the pixel indicates that the pixel needs to be processed (or the corresponding thread needs to be created); otherwise, if a pixel is located on the backlight side, that is, no light reflection will occur, then the mask corresponding to the pixel indicates that the pixel does not need to be processed (or the corresponding thread does not need to be created).

[0050] For another example, according to the scene design requirements, if the depth of a pixel point is less than a first threshold, the mask corresponding to the pixel point indicates that the pixel point needs to be processed by image (or the corresponding thread needs to be created); otherwise, the mask corresponding to the pixel point indicates that the pixel point does not need to be processed by image (or the corresponding thread does not need to be created).

[0051] For another example, if the reflection coefficient of a certain pixel is greater than the second threshold, that is, it is more likely to reflect light, then the mask corresponding to the pixel indicates that the pixel needs to be processed (or a corresponding thread needs to be created). Otherwise, the mask corresponding to the pixel indicates that the pixel does not need to be processed (or a corresponding thread does not need to be created). For example, as shown in Table 4, assuming that the second threshold is 0.2, the reflection coefficients of pixel 4 and pixel 5 among the 8 pixels are both greater than 0.2, so the mask of these two pixels can take a value of 1 to indicate that these two pixels need to be processed (or a corresponding thread needs to be created).

[0052] Table 4

[0053] Pixels 0 1 2 3 4 5 6 7 Reflection coefficient 0 0 0 0 0.3 0.4 0 0 Mask 0 0 0 0 1 1 0 0 Thread group ID 0 0 0 0 Thread ID 0 1 2 3

[0054] For another example, if the roughness of a pixel is greater than a third threshold and additional light needs to be emitted, the mask corresponding to the pixel indicates that the pixel needs to be processed by image (or the corresponding thread needs to be created); otherwise, the mask corresponding to the pixel indicates that the pixel does not need to be processed by image (or the corresponding thread does not need to be created).

[0055] For another example, if the material of a certain pixel point is a specular reflector (such as water, glass, ceramic, metal, etc.), that is, it is more likely to reflect light, then the mask corresponding to the pixel point indicates that the pixel point needs to be processed by image (or the corresponding thread needs to be created); otherwise, if the material of a certain pixel point is a non-specular reflector (such as cotton, wool, soil, paper, tree, grass, etc.), that is, it is less likely to reflect light, then the mask corresponding to the pixel point indicates that the pixel point does not need to be processed by image (or the corresponding thread does not need to be created).

[0056] S302: Create a target thread group according to the masks of the multiple pixel points.

[0057] The target thread group includes at least one thread for image processing, which can run a shader program that performs image processing on each pixel based on the geometry cache.

[0058] For example, as shown in Table 4, assuming that each thread group includes four threads, each thread corresponds to a pixel and mask, pixels 0-3 do not require image processing (or do not need to create corresponding threads) because the mask is 0, and pixels 4-5 require image processing (or need to create corresponding threads) because the mask is 1. Therefore, a thread group is created for pixels 4-7 (thread group identifier is 0). This thread group includes threads 0-3, where thread 0 is used to perform image processing on pixel 4 and thread 1 is used to perform image processing on pixel 5. Compared with the prior art that requires creating two thread groups, this application only needs to create one thread group, so it can save GPU processing resources and improve GPU execution efficiency.

[0059] Furthermore, the GPU can combine multiple threads to obtain a target thread group. The GPU can create a first thread group based on a mask of a first group of pixels among the multiple pixels; wherein the first thread group includes at least one thread that does not perform image processing. The GPU can create a second thread group based on a mask of a second group of pixels among the multiple pixels; wherein the second thread group includes at least one thread that does not perform image processing. The GPU can then combine the threads in the first thread group that perform image processing with the threads in the second thread group that perform image processing to obtain the target thread group.

[0060] For example, as shown in Table 5, assuming that the second threshold is 0.2, the reflection coefficients of pixels 2-5 among the 8 pixels are all greater than 0.2, so the mask of these four pixels can take a value of 1 to indicate that these four pixels need to be image processed (or corresponding threads need to be created). The GPU can create a first thread group (thread group identifier 0) based on the mask of the first group of pixels (pixels 0-3), wherein thread 2 is used to perform image processing on pixel 2, and thread 3 is used to perform image processing on pixel 3. The GPU can create a second thread group (thread group identifier 1) based on the mask of the second group of pixels (pixels 4-7), wherein thread 4 is used to perform image processing on pixel 4, and thread 5 is used to perform image processing on pixel 5. The GPU can then merge the first thread group (thread group ID 0) and the second thread group (thread group ID 1) to obtain a target thread group (thread group ID 0). In the target thread group (thread group ID 0), thread 0 is used to process the image of pixel 2, thread 1 is used to process the image of pixel 3, thread 2 is used to process the image of pixel 4, and thread 3 is used to process the image of pixel 5. This can further save GPU processing resources and improve GPU execution efficiency.

[0061] Table 5

[0062]

[0063] for Figure 2 For example, the additional thread group compression step is very complicated. Figure 4 As shown in the figure, after the above improvements, compared with Figure 2 For example, there is no need to add a separate thread group compression step. When generating the mask and creating the target thread group according to the mask, it is a simple logical judgment, so the implementation is simple and the workload is saved.

[0064] The thread creation method, graphics processing unit, and electronic device provided in the embodiments of the present application use a mask for each pixel to indicate whether the pixel requires image processing (or whether a corresponding thread needs to be created), thereby determining whether to create a corresponding thread group. Thread groups where no thread performs image processing need not be created, thereby conserving GPU processing resources and improving GPU execution efficiency.

[0065] The embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which are executed on the GPU so that the GPU executes Figure 3 The thread group creation method shown.

[0066] The present application also provides a computer program product including instructions, which are executed on a GPU so that the GPU executes Figure 3 The thread group creation method shown.

[0067] The processor involved in the embodiments of the present application may be a chip. For example, it may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0068] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0069] Those skilled in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0070] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0071] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0072] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located on a single device or distributed across multiple devices. Some or all of the modules may be selected to achieve the purpose of this embodiment based on actual needs.

[0073] In addition, the functional modules in the various embodiments of the present application may be integrated into one device, or each module may exist physically separately, or two or more modules may be integrated into one device.

[0074] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode to another website, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more media that can be integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0075] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A thread group creation method, characterized in that: include: Obtaining masks of a plurality of pixel points in an image, wherein the mask of each pixel point is used to indicate whether image processing is to be performed on each pixel point; Creating a first thread group according to a mask of a first group of pixel points among the plurality of pixel points; wherein the first thread group includes at least one thread that does not perform image processing; Creating a second thread group according to a mask of a second group of pixel points among the plurality of pixel points; wherein the second thread group includes at least one thread that does not perform image processing; The threads performing image processing in the first thread group and the threads performing image processing in the second thread group are merged to obtain the target thread group.

2. The method according to claim 1, characterized in that The obtaining of masks of a plurality of pixels in the image includes: The masks of the plurality of pixels are received from a central processing unit.

3. The method according to claim 1, characterized in that The obtaining of masks of a plurality of pixels in the image includes: Masks of the plurality of pixel points are generated according to attributes of the plurality of pixel points.

4. The method according to claim 3, characterized in that The attributes of the plurality of pixel points include material information, and the material information includes at least one of a reflection coefficient, roughness, and a material identifier, wherein the material identifier is used to indicate the material of the pixel point.

5. The method according to any one of claims 1 to 4, characterized in that One thread of the target thread group corresponds to a mask of multiple pixel points.

6. The method according to any one of claims 1 to 4, characterized in that Also included is a storage space for managing the mask.

7. The method according to any one of claims 1 to 4, characterized in that The mask is stored in a tile buffer of the graphics processing unit, or in a system buffer, or in a memory.

8. A graphics processing unit, characterized in that: It comprises a stream multiprocessor, and the stream multiprocessor is used to execute the thread group creation method according to any one of claims 1 to 7.

9. An electronic device, characterized in that: The device comprises a graphics processing unit and a display screen as claimed in claim 8, wherein the graphics processing unit is used to process an image and display the image on the display screen.

Images