Image processing apparatus, method, device, storage medium and program products

By encoding the primitive index number as a flag and transmitting it during the graphics rendering process, and then decoding it back into a primitive index number, the problems of data transmission bandwidth and cache space are solved, achieving more efficient image processing.

CN123076900APending Publication Date: 2026-11-10GLENFLY TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202680001981.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-06-15
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

In graphics rendering, primitive index numbers need to be transmitted along with vertex information between various units in the thread bundle construction module, resulting in increased data transmission bandwidth and increased cache space requirements.

Method used

By encoding the primitive index number as a primitive index number flag, transmitting it between thread bundle construction modules, and then having each thread bundle construction module decode it into a primitive index number for rendering, the amount of data between modules is reduced.

Benefits of technology

This reduces data transmission bandwidth during image processing and decreases cache space requirements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123076900A_ABST
    Figure CN123076900A_ABST
Patent Text Reader

Abstract

This application relates to an image processing apparatus, method, device, storage medium, and program product. The image processing apparatus includes an input assembly module, a thread bundle execution module, and multiple thread bundle construction modules. The input assembly module is used to acquire the primitive index numbers of each vertex of the primitive to be processed, encode the primitive index numbers to obtain primitive index number flags for each vertex of the primitive to be processed, and sequentially transmit the primitive index number flags to the multiple thread bundle construction modules. The thread bundle construction module is used to construct threads of a target type; determine the primitive index number corresponding to the thread of the target type based on the primitive index number flags; and send the thread of the target type and its corresponding primitive index number to the thread bundle execution module. The thread bundle execution module is used to use the primitive index number corresponding to the thread of the target type as input data for the thread of the target type. This method can reduce the data transmission bandwidth during image processing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing apparatus, method, device, storage medium, and program product. Background Technology

[0002] In graphics rendering, when processing static geometry, the geometry shader in the Graphics Processing Unit (GPU) computes each vertex or pixel of the static geometry independently, lacking awareness of the primitive index being processed. With the development of image technology, more and more scenarios require primitive IDs.

[0003] In related technologies, the primitive index number is generated by the vertex input assembly module in the GPU. The vertex input assembly module can generate a 32-bit primitive index number for each vertex of the primitive and distribute the primitive index number along with the corresponding vertex information to the thread beam construction module in the GPU. The thread beam construction module can store the received 32-bit primitive index number in an internal cache. After the thread beam is constructed, the 32-bit primitive index number is retrieved from the internal cache and written into a general-purpose register for the thread beam instruction to read.

[0004] However, the current primitive index number needs to be transmitted along with vertex information between units in the thread bundle construction module, which increases the bandwidth of data transmission between units. Summary of the Invention

[0005] Therefore, it is necessary to provide an image processing apparatus, method, device, storage medium, and program product that can reduce the data transmission bandwidth in the image processing process, in order to address the above-mentioned technical problems.

[0006] In a first aspect, this application provides an image processing apparatus, which includes an input assembly module, a thread bundle execution module, and multiple thread bundle construction modules:

[0007] The input assembly module is used to obtain the primitive index number of each vertex of the primitive to be processed, encode the primitive index number to obtain the primitive index number flag of each vertex of the primitive to be processed, and transmit the primitive index number flag to the multiple thread bundle construction modules in sequence.

[0008] The thread bundle construction module is used to construct a thread of the target type; determine the primitive index number corresponding to the thread of the target type according to the primitive index number flag; and send the thread of the target type and the primitive index number corresponding to the thread of the target type to the thread bundle execution module.

[0009] The thread bundle execution module is used to take the primitive index number corresponding to the thread of the target type as the input data of the thread of the target type, and execute the calculation logic of the thread of the target type to render the primitive to be processed in the target type.

[0010] In one embodiment, the input assembly module is specifically configured to: encode the primitive index number of the target vertex as a first value when the target vertex is the first vertex of its constituent primitive and the constituent primitive is the first primitive; encode the primitive index number of the target vertex as a second value when the target vertex is the first vertex of its constituent primitive and the constituent primitive is not the first primitive; and encode the primitive index number of the target vertex as a third value when the target vertex is not the first vertex of its constituent primitive.

[0011] In one embodiment, the thread bundle construction module is specifically configured to: when the primitive index number flag of the target type thread is the third value, determine the cached primitive index number as the primitive index number corresponding to the target type thread; when the primitive index number flag of the target type thread is the second value, update the cached primitive index number by incrementing it by one and determine the updated cached primitive index number as the primitive index number corresponding to the target type thread; when the primitive index number flag of the target type thread is the first value, update the cached primitive index number to zero and determine the updated cached primitive index number as the primitive index number corresponding to the target type thread.

[0012] The initial value of the cached primitive index number is the initial primitive index number of the primitive to be processed.

[0013] In one embodiment, the image processing apparatus further includes a surface subdivision module and a rasterization module, and the thread bundle construction module includes a vertex thread bundle construction module, a shell thread bundle construction module, a region thread bundle construction module, a geometry thread bundle construction module, and a pixel thread bundle construction module.

[0014] In one embodiment, the vertex thread bundle construction module is configured to receive the primitive index number flags and initial primitive index numbers of each vertex of the primitive to be processed sent by the input assembly module; construct vertex threads for each vertex; determine the primitive index number flags of the vertex threads as the primitive index number flags of the corresponding vertices; determine the primitive index number corresponding to the vertex thread based on the initial primitive index number and the primitive index number flags of the vertex threads; send the primitive index number corresponding to the vertex thread to the thread bundle execution module; and send the primitive index number flags of each vertex of the primitive to be processed and the initial primitive index number to the shell thread bundle construction module.

[0015] In one embodiment, the shell thread bundle construction module is used to combine multiple vertices into subdivision blocks and construct corresponding subdivision block threads; determine the primitive index number flag of the first vertex constituting the subdivision block as the primitive index number flag of the subdivision block thread; determine the primitive index number corresponding to the subdivision block thread based on the initial primitive index number and the primitive index number flag of the subdivision block thread; send the primitive index number corresponding to the subdivision block thread to the thread bundle execution module; and send the initial primitive index number and the primitive index number flags of each vertex to the surface subdivision module.

[0016] In one embodiment, when the surface tessellation module is in a closed state, the surface tessellation module is used to send the initial primitive index number and the primitive index number flags of each vertex to the region thread bundle construction module;

[0017] When the surface tessellation module is in the enabled state and a first new vertex is generated, the surface tessellation module is used to send the initial primitive index number and the primitive index number flag of the first new vertex to the region thread bundle construction module.

[0018] When the surface tessellation module is in the enabled state and no first new vertex is generated, the surface tessellation module is used to send the primitive index number cached inside the surface tessellation module to the region thread construction module;

[0019] The first new vertex is a new vertex generated after surface subdivision.

[0020] In one embodiment, the surface subdivision module is specifically used to encode the primitive index number flag of the first new vertex as the primitive index number flag of the subdivision block thread when the first new vertex is the first vertex generated by the corresponding subdivision block; and to encode the primitive index number flag of the first new vertex as a third value when the first new vertex is not the first vertex generated by the corresponding subdivision block.

[0021] In one embodiment, when the surface subdivision module is in the enabled state and a first new vertex is generated, the region thread bundle construction module is used to construct a region thread for each first new vertex; determine the primitive index number flag of the first new vertex corresponding to each region thread as the primitive index number flag of the region thread; determine the primitive index number corresponding to the region thread according to the initial primitive index number and the primitive index number flag of the region thread; send the primitive index number corresponding to the region thread to the thread bundle execution module; and send the initial primitive index number and the primitive index number flag of the first new vertex to the geometry thread bundle construction module.

[0022] In one embodiment, when the surface tessellation module is in the enabled state and no first new vertex is generated, the surface tessellation module is further configured to send the received primitive index number cached internally by the surface tessellation module to the region thread construction module.

[0023] In one embodiment, when the surface tessellation module is in a closed state, the surface tessellation module is further configured to send the initial primitive index number and the received primitive index number flags of each vertex to the region thread bundle construction module.

[0024] In one embodiment, the geometry thread bundle construction module is used to construct a geometry thread corresponding to the primitive composed of received vertices and perform geometric transformations on the geometry thread to obtain a new primitive; determine the primitive index number flag of the first new vertex of the first new primitive generated by each geometry thread as the primitive index number flag of the geometry thread; and send the primitive index number corresponding to the geometry thread to the thread bundle execution module.

[0025] In one embodiment, the geometric thread bundle construction module is further configured to encode the non-first new vertex that is combined to form the first new primitive and other new vertices of other new primitives to obtain the primitive index number flag of the second new vertex;

[0026] The second new vertex is a vertex in the new primitive obtained by the geometric transformation of the geometric thread bundle construction module.

[0027] In one embodiment, when the geometry thread bundle construction module is in a closed state, the geometry thread bundle construction module is used to pass information received from the region thread bundle construction module to the rasterization module;

[0028] When the geometry thread bundle construction module is in the enabled state and the geometry thread bundle construction module generates new primitives, the geometry thread bundle construction module is used to send the initial primitive index number and the primitive index number flag of the second new vertex to the rasterization module.

[0029] When the geometry thread bundle construction module is in the enabled state and the geometry thread bundle construction module does not generate new primitives, the geometry thread bundle construction module is used to send the primitive index number cached inside the geometry thread bundle construction module to the rasterization module.

[0030] In one embodiment, when the surface subdivision module is in the enabled state, the geometry thread bundle construction module is further configured to update the primitive index number cached in the geometry thread bundle construction module according to the received primitive index number cached in the surface subdivision module; and send the updated primitive index number cached in the geometry thread bundle construction module to the rasterization module.

[0031] In one embodiment, the rasterization module is used to combine each vertex of the primitive to be processed, the first new vertex, or the second new vertex into a target primitive; sample the target primitive to obtain pixels of the target primitive; encode the primitive index number of the pixel to obtain a primitive index number flag of the pixel; and send the primitive index number flag of the pixel and the initial primitive index number to the pixel thread bundle construction module.

[0032] The pixel thread bundle construction module is used to construct a pixel thread for each pixel; determine the primitive index number corresponding to the pixel thread based on the primitive index number flag of the pixel and the initial primitive index number; and send the primitive index number corresponding to the pixel thread to the thread bundle execution module.

[0033] In one embodiment, the rasterization module is further configured to, when the pixel is the first pixel generated by interpolation constituting the target primitive, encode the primitive index number flag of the pixel as the primitive index number flag constituting the target primitive; and when the pixel is not the first pixel generated by interpolation constituting the target primitive, encode the primitive index number flag of the pixel as a third value.

[0034] Secondly, this application also provides an image processing method, comprising:

[0035] Obtain the primitive index number of each vertex of the primitive to be processed;

[0036] The primitive index number is encoded to obtain the primitive index number flag of each vertex of the primitive to be processed;

[0037] The primitive index number flag is sequentially transmitted to the plurality of thread bundle construction modules to construct threads of the target type;

[0038] Construct a thread of the target type, and determine the primitive index number corresponding to the thread of the target type based on the primitive index number flag;

[0039] The primitive index number corresponding to the thread of the target type is used as the input data of the thread of the target type, and the calculation logic of the thread of the target type is executed to render the primitive to be processed in the target type.

[0040] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the image processing method of the second aspect described above.

[0041] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method of the second aspect described above.

[0042] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the image processing method of the second aspect described above.

[0043] The aforementioned image processing apparatus, method, device, storage medium, and program product include an image processing apparatus comprising an input assembly module, a thread bundle execution module, and multiple thread bundle construction modules. The input assembly module is used to acquire the primitive index numbers of each vertex of the primitive to be processed, encode the primitive index numbers to obtain primitive index number flags for each vertex of the primitive to be processed, and sequentially transmit the primitive index number flags to the multiple thread bundle construction modules. The thread bundle construction module is used to construct threads of a target type; determine the primitive index number corresponding to the thread of the target type based on the primitive index number flags; and send the thread of the target type and its corresponding primitive index number to the thread bundle execution module. The thread bundle execution module is used to use the primitive index number corresponding to the thread of the target type as input data for the thread of the target type, execute the computational logic of the thread of the target type, and render the primitive to be processed in the target type. Since the primitive index number of the vertex is encoded into a primitive index number flag and transmitted between multiple thread bundle construction modules, and then each thread bundle construction module decodes the primitive index number flag into a primitive index number before rendering the corresponding thread, the amount of data transmitted between the modules is greatly reduced, thus reducing the data transmission bandwidth in the image processing process. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A schematic diagram illustrating the principles of image rendering in related technologies;

[0046] Figure 2 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application;

[0047] Figure 3 A schematic diagram illustrating the encoding principle of an input assembly module provided in this application embodiment;

[0048] Figure 4 This is a schematic diagram illustrating the decoding principle of a thread bundle construction module provided in an embodiment of this application;

[0049] Figure 5 A hardware block diagram of an image processing pipeline provided in an embodiment of this application;

[0050] Figure 6 A schematic diagram illustrating the encoding principle of a surface subdivision module provided in an embodiment of this application;

[0051] Figure 7 A schematic diagram illustrating the encoding principle of a geometric thread bundle construction module provided in an embodiment of this application;

[0052] Figure 8 A schematic diagram illustrating the encoding principle of a rasterization module provided in an embodiment of this application;

[0053] Figure 9 A schematic diagram illustrating the encoding of a primitive index number provided in an embodiment of this application;

[0054] Figure 10 A schematic diagram of a buffer for a thread bundle construction module provided in an embodiment of this application;

[0055] Figure 11 This application provides a schematic diagram of decoding a primitive index number.

[0056] Figure 12 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0057] Figure 13 This is an internal structural diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] The relevant technologies will be explained below.

[0060] In graphics rendering, when processing static geometry, the geometry shader in the Graphics Processing Unit (GPU) computes each vertex or pixel of the static geometry independently, lacking awareness of the primitive index being processed. With the development of image technology, more and more scenarios require primitive IDs.

[0061] Firstly, for the processing of dynamic geometry, in the programmable stage such as shaders, it is necessary to know the index of the primitive being processed so that new geometry can be dynamically modified or generated using methods such as subdivision, culling, and Level of Detail (LOD) adjustment. Therefore, there is a requirement for primitive index numbers.

[0062] Secondly, during the fragment shader phase, different primitives may require different shader logic. For example, some primitives need special specular highlights, while others need transparency effects. However, the current vertex attribute interpolation cannot satisfy different shader logics, thus creating a need for primitive index numbers.

[0063] Furthermore, in graphics rendering, when abnormalities such as broken surfaces or flickering occur, it is necessary to quickly locate the problematic primitives, and the primitive index number provides a unique identification method.

[0064] Finally, advanced rendering techniques such as instancing, ray tracing, and GPU-driven culling typically require precise control over primitive-level behavior, thus necessitating the use of primitive index numbers.

[0065] Based on this, the primitive index number becomes the identity identifier of primitives in the GPU pipeline, used to realize dynamic geometry operations, material differentiation, debugging and advanced rendering logic, and is an important tool for efficiently controlling rendering details.

[0066] In related technologies, the primitive index number is generated by the vertex input assembly module in the GPU. The vertex input assembly module can generate a 32-bit primitive index number for each vertex of the primitive and distribute the primitive index number along with the corresponding vertex information to the thread beam construction module in the GPU. The thread beam construction module can store the received 32-bit primitive index number in an internal cache. After the thread beam is constructed, the 32-bit primitive index number is retrieved from the internal cache and written into a general-purpose register for the thread beam instruction to read.

[0067] Primitives can include four types: points, lines, triangles, and patches. Point, line, and triangle primitives consist of 1, 2, and 3 vertices, respectively. Patch primitives consist of 1 to 32 vertices and are a special type of primitive used in the tessellation module of the GPU. For example, if there are 3 triangle primitives, since each triangle primitive consists of 3 vertices, there are a total of 9 vertices. Accordingly, the primitive index number of the 3 vertices of the first triangle primitive is 0, the primitive index number of the 3 vertices of the second triangle primitive is 1, and the primitive index number of the 3 vertices of the third triangle primitive is 2.

[0068] The thread bundle construction module includes a vertex thread bundle construction unit, a shell thread bundle construction unit, a region thread bundle construction unit, a geometry thread bundle construction unit, and a pixel thread bundle construction unit. The vertex thread bundle construction unit transmits the received primitive index number to the shell thread bundle construction unit, which in turn transmits it to the region thread bundle construction unit. The region thread bundle construction unit then transmits the received primitive index number to the geometry thread bundle construction unit, and so on, until it reaches the pixel thread bundle construction unit.

[0069] Figure 1 A schematic diagram illustrating the principle of image rendering in related technologies, such as... Figure 1 As shown, the image processing device includes an input assembly module, a vertex shader (VS), a tessellation module, a geometry shader (GS), a rasterizer, and a pixel shader (PS). The tessellation module includes a hull shader (HS), a domain shader (DS), and a tessellation hardware unit (TS).

[0070] For example, the input assembly module can collect primitives and assign a 32-bit primitive index number to each primitive. Each vertex of a primitive uses this primitive index number as its vertex primitive index number; therefore, vertices of the same primitive have the same primitive index number. The input assembly module then sequentially sends vertex information, including the primitive index numbers, to the vertex shader.

[0071] For example, the vertex shader receives vertices distributed by the input assembly module, performs coordinate transformations and other operations on the vertices, and then outputs them to the tessellation module. Each thread of the vertex shader corresponds to one vertex, and the vertex shader directly uses the primitive index number input by the input assembly module as the input data for the corresponding thread.

[0072] For example, in the surface tessellation module, the HS performs the pre-calculation of surface tessellation, and the DS performs the post-tessellation processing. HS calculates multiple control points (CPs) and tessellation parameters (PCs) corresponding to each tessellation patch. Subsequently, the tessellation parameters PCs are transmitted from HS to TS, so that TS can calculate the 2D coordinates of the new vertices; the control points CPs are transmitted from HS to DS, so that DS can calculate the final 3D coordinates and attribute information of the new vertices, such as color, based on the control points and the 2D coordinates of the new vertices output by TS. Finally, DS outputs the attribute information of the generated new vertices to GS for subsequent geometric processing.

[0073] For example, each thread of the shell shader corresponds to a subdivision block, each subdivision block consists of multiple vertices, and each patch uses the primitive index number of the first vertex that makes up the patch as its primitive index number, which is also used as the input data for the shell shader thread.

[0074] For example, each thread of the region shader corresponds to a new vertex, which is generated by the tessellation module. Each patch subdivides to generate multiple new vertices. The primitive index numbers of the new vertices generated by the same patch are the same, and they are also used as input data for the region shader thread.

[0075] For example, the geometry shader assembles the received vertices into a primitive. Each geometry shader thread corresponds to one primitive. The primitive index number of the first vertex that makes up the primitive is used as the primitive index number of the primitive and is also used as the input data of the geometry shader thread.

[0076] For example, each primitive generates multiple pixels through the rasterization component, and each thread of the pixel shader corresponds to one of the pixels. Pixels generated from the same primitive have the same primitive index number, which is used as the input data of the pixel shader thread.

[0077] However, the current primitive index number needs to be transmitted along with vertex information between various units in the thread bundle construction module during the image rendering process, which increases the bandwidth of data transmission between units. In addition, each unit needs to save the corresponding primitive index number for each vertex, which also increases the cache space within each unit.

[0078] To address the aforementioned issues, embodiments of this application provide an image processing apparatus, method, device, storage medium, and program product. Since the primitive index number of a vertex is encoded as a primitive index number flag and transmitted between multiple thread bundle construction modules, and then each thread bundle construction module decodes the primitive index number flag into a primitive index number before performing rendering for the corresponding thread, the amount of data transmitted between the modules is greatly reduced, thereby lowering the data transmission bandwidth during image processing.

[0079] It should be understood that the image processing method provided in this application embodiment is applied to an image processing device, which can be any graphics processing unit (GPU) or any device including a GPU. This application embodiment does not limit this.

[0080] Figure 2 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application, as shown below. Figure 2 As shown, the image processing device includes an input assembly module, a thread bundle execution module, and multiple thread bundle construction modules.

[0081] In some embodiments, the input assembly module is used to obtain the primitive index number of each vertex of the primitive to be processed, encode the primitive index number to obtain the primitive index number flag of each vertex of the primitive to be processed, and transmit the primitive index number flag to multiple thread bundle construction modules in sequence. The thread bundle construction module is used to construct a thread of the target type; determine the primitive index number corresponding to the thread of the target type based on the primitive index number flag; and send the thread of the target type and the primitive index number corresponding to the thread of the target type to the thread bundle execution module. The thread bundle execution module is used to use the primitive index number corresponding to the thread of the target type as input data for the thread of the target type, execute the calculation logic of the thread of the target type, and render the primitive to be processed in the target type.

[0082] In some embodiments, the input assembly module may include an encoder, which encodes the primitive index numbers of each vertex of the primitive to be processed obtained by the input assembly module. The input assembly module may encode the primitive index number of the target vertex as a first value if the target vertex is the first vertex of its constituent primitive and the constituent primitive is also the first primitive; encode the primitive index number of the target vertex as a second value if the target vertex is the first vertex of its constituent primitive but the constituent primitive is not the first primitive; and encode the primitive index number of the target vertex as a third value if the target vertex is not the first vertex of its constituent primitive.

[0083] The target vertex is any vertex of the primitive to be processed.

[0084] It should be understood that the primitive index number flag can represent the state of the primitive index number of the target vertex relative to the primitive index number of the previous target vertex through different values, thereby reducing the amount of primitive index number data.

[0085] For example, the primitive index number flag can include three values: a first value, a second value, and a third value. The first value can be 2, indicating a reset; the second value can be 1, indicating an increment; and the third value can be 0, indicating a keep. By using the three values ​​0, 1, and 2, the three states of the primitive index number flag can be represented by only 2 bits. This allows the primitive index number flag to replace the primitive index number for transmission between thread bundle construction modules, thereby reducing data bandwidth.

[0086] For example, Figure 3 A schematic diagram of the encoding principle of an input assembly module provided in an embodiment of this application is shown below. Figure 3 As shown, the encoder of the input assembly module can encode the primitive index number of a target vertex by determining whether each target vertex is the first vertex of its corresponding primitive and whether the corresponding primitive is the first primitive. If the target vertex is the first vertex of its corresponding primitive and the corresponding primitive is the first primitive, the encoded primitive index number flag is set to reset (i.e., 2); if the target vertex is the first vertex of its corresponding primitive but the corresponding primitive is not the first primitive, the encoded primitive index number flag is set to incr (i.e., 1); if the target vertex is not the first vertex of its corresponding primitive, the encoded primitive index number flag is set to keep (i.e., 0).

[0087] In some embodiments, the thread bundle construction module may include a decoder. After the input assembly module encodes the primitive index number flags of each vertex of the primitive to be processed, these primitive index number flags can be sequentially transmitted to multiple thread bundle construction modules. Subsequently, each thread bundle construction module determines the primitive index number flags of each target thread to be constructed based on the primitive index number flags of each vertex of the primitive to be processed. Finally, the decoder in the thread bundle construction module can decode the primitive index number flags of each target thread to obtain the primitive index number of the target thread, and then send it to the thread bundle execution module for corresponding rendering.

[0088] It should be understood that the embodiments of this application do not limit how each thread bundle construction module decodes the primitive index number flag. In some embodiments, the thread bundle construction module is specifically used to determine the cached primitive index number as the primitive index number corresponding to the target type thread when the primitive index number flag of the target type thread is a third value; when the primitive index number flag of the target type thread is a second value, update the cached primitive index number by incrementing it by one and determine the updated cached primitive index number as the primitive index number corresponding to the target type thread; when the primitive index number flag of the target type thread is a first value, update the cached primitive index number to zero and determine the updated cached primitive index number as the primitive index number corresponding to the target type thread.

[0089] The initial value of the cached primitive index number is the initial primitive index number of the primitive to be processed, that is, the primitive index number of the first vertex in the primitive to be processed.

[0090] For example, Figure 4 This is a schematic diagram illustrating the decoding principle of a thread bundle construction module provided in an embodiment of this application, as shown below. Figure 4 As shown, the thread bundle construction module can first cache the initial primitive index number of the primitive to be processed, and then input the primitive index number flag of the target thread into the decoder. Subsequently, the decoder checks the input primitive index number flag to determine if it is a third value (i.e., 0, indicating keep). If the primitive index number flag is 0, the cached primitive index number can be directly used as the primitive index number of the target thread. If the primitive index number flag is not 0, it can continue to check if the primitive index number flag is a third value (i.e., 2, indicating reset). If the primitive index number flag is 2, the cached primitive index number is updated to zero, and the updated cached primitive index number is determined as the primitive index number corresponding to the target type thread. If the primitive index number flag is not 2 (it can only be 1, indicating insert), the cached primitive index number is updated by incrementing it by one, and the updated cached primitive index number is determined as the primitive index number corresponding to the target type thread.

[0091] The following section explains how primitive index number flags are transmitted between multiple thread bundle construction modules and how they are encoded and decoded.

[0092] In some embodiments, the image processing apparatus further includes a tessellation module and a rasterization module, and the thread bundle construction module includes a vertex thread bundle construction module, a shell thread bundle construction module, a region thread bundle construction module, a geometry thread bundle construction module, and a pixel thread bundle construction module. Correspondingly, primitive index number markers are transmitted between the vertex thread bundle construction module, the shell thread bundle construction module, the tessellation module, the region thread bundle construction module, the geometry thread bundle construction module, the rasterization module, and the pixel thread bundle construction module.

[0093] For example, Figure 5 A hardware block diagram of an image processing pipeline provided in an embodiment of this application, such as Figure 5 As shown, the graphics processor has multiple slices, each of which can serve as an image processing device including the aforementioned modules. For example, all modules except the input assembly module include a decoder for primitive index numbers to decode the received primitive index number flags. The input assembly module, tessellation module, geometry bundle construction module, and rasterization module include encoders for primitive index numbers to encode the vertices, newly added vertices, or pixels of the primitives to be processed, obtaining the primitive index number flags to compress the amount of primitive index number data.

[0094] It should be understood that an encoder for primitive index numbers can be added to the output of the input assembly module. This allows the input assembly module to encode the primitive index numbers of each vertex of the primitive to be processed into primitive index number flags, which are then transmitted sequentially to multiple thread bundle construction modules. A decoder for primitive index numbers can be added to the input of each thread bundle construction module. This allows the thread bundle construction module to determine the primitive index number flag corresponding to the target type of thread, and then decode the primitive index number flag to obtain the primitive index number corresponding to the target type of thread. Furthermore, since the tessellation module and the geometry thread bundle construction module generate new vertices, and the rasterization module interpolates primitives into pixels, encoders for primitive index numbers are also added to the outputs of the tessellation module, the geometry thread bundle construction module, and the rasterization module to generate primitive index number flags for the new vertices or pixels.

[0095] In some embodiments, after the input assembly module obtains the primitive index numbers of each vertex of the primitive to be processed, it can first encode the primitive index numbers of each vertex of the primitive to be processed to obtain primitive index number flags of each vertex of the primitive to be processed. Subsequently, the input assembly module can send the primitive index number flags of each vertex of the primitive to be processed and the initial primitive index number to be sent to the slice to the vertex thread bundle construction module.

[0096] The primitive index number is carried in the vertex information, which may also include the vertex ID. The primitive index number is 2 bits, and the vertex ID is 32 bits. The vertex ID is used to calculate the address of the vertex attribute value in global memory.

[0097] In some embodiments, the vertex thread bundle construction module can receive the primitive index number flags and initial primitive index numbers of each vertex of the primitive to be processed sent by the input assembly module. Next, it constructs vertex threads for each vertex and determines the primitive index number flags of the vertex threads as the primitive index number flags of the corresponding vertices. Then, based on the initial primitive index number and the primitive index number flags of the vertex threads, it determines the primitive index number corresponding to the vertex thread. Finally, it sends the primitive index number corresponding to the vertex thread to the thread bundle execution module. Simultaneously, the vertex thread bundle construction module can also send the primitive index number flags and initial primitive index numbers of each vertex of the primitive to be processed to the shell thread bundle construction module.

[0098] It should be understood that since each vertex thread corresponds to a vertex sent by the input assembly module, the primitive index number flag of the vertex thread is naturally the primitive index number flag of the corresponding vertex. The primitive index number flag of the vertex is output by the input assembly module to the vertex thread bundle construction module.

[0099] For example, after receiving the vertex information of each vertex of the primitive to be processed and the initial primitive index number sent to the slice from the input assembly module, the vertex thread bundle construction module can transmit the vertex information of each vertex of the primitive to be processed and the initial primitive index number sent to the slice to the shell thread bundle construction module. Furthermore, the vertex thread bundle construction module can construct vertex threads of each vertex to form a vertex thread bundle, and determine the primitive index number flag of the vertex thread as the primitive index number flag of the corresponding vertex. Subsequently, the vertex thread bundle construction module decodes the primitive index number flag of the vertex thread according to the initial primitive index number using the decoder in the vertex thread bundle construction module, obtaining the primitive index number corresponding to the vertex thread. Finally, the vertex thread bundle construction module sends the primitive index number corresponding to the vertex thread to the thread bundle execution module as input data for the vertex thread to execute the computation logic of the target type thread, in order to render the primitive to be processed according to the target type.

[0100] In some embodiments, the shell thread bundle construction module can combine multiple vertices into subdivision blocks and construct corresponding subdivision block threads, and determine the primitive index number flag of the first vertex constituting the subdivision block as the primitive index number flag of the subdivision block thread. Subsequently, the shell thread bundle construction module can determine the primitive index number corresponding to the subdivision block thread based on the initial primitive index number and the primitive index number flag of the subdivision block thread. Finally, the shell thread bundle construction module can send the primitive index number corresponding to the subdivision block thread to the thread bundle execution module. Simultaneously, the shell thread bundle construction module can also send the initial primitive index number and the primitive index number flags of each vertex to the surface subdivision module.

[0101] Among them, a patch is a special primitive used for subdividing geometric surfaces.

[0102] It should be understood that each subdivision thread corresponds to one subdivision, and the subdivision is composed of vertices sent by the vertex thread bundle constructor. Therefore, the primitive index number of a subdivision is the primitive index number of the first vertex that makes up this subdivision, and also serves as the primitive index number of the thread that makes up this subdivision. The primitive index number of the vertices that make up the subdivision is output by the vertex thread bundle constructor.

[0103] For example, the shell thread bundle construction module can group a certain number of vertices into a patch, and construct a corresponding patch thread for each patch. A preset number of patch blocks (e.g., 32) are then constructed into a thread bundle. Subsequently, the shell thread bundle construction module determines the primitive index number of the first vertex forming the patch as the primitive index number of the patch thread. Then, using a decoder, it decodes the primitive index number of the patch thread based on the initial primitive index number to determine the primitive index number corresponding to the patch thread. Finally, the shell thread bundle construction module can send the primitive index number corresponding to the patch thread to the thread bundle execution module as input data to execute the target type thread's computational logic, thereby rendering the primitives to be processed according to the target type.

[0104] For example, the shell thread bundle construction module can also send the received initial primitive index number and the primitive index number flags of each vertex to the surface subdivision module.

[0105] In some embodiments, when the tessellation module is in a closed state, the tessellation module sends the initial primitive index number and the primitive index number flags of each vertex to the region thread bundle construction module. In some embodiments, when the tessellation module is in a closed state and a first new vertex is generated, the tessellation module sends the initial primitive index number and the primitive index number flags of the first new vertex to the region thread bundle construction module. In some embodiments, when the tessellation module is in a closed state and no first new vertex is generated, the tessellation module sends the primitive index numbers cached internally to the region thread bundle construction module.

[0106] The first new vertex is a new vertex generated after surface subdivision.

[0107] It should be understood that the primitive index number of the first new vertex can be generated by encoding in the encoder of the surface tessellation module. In some embodiments, the surface tessellation module may encode the primitive index number of the first new vertex as the primitive index number of the tessellation block thread if the first new vertex is the first vertex generated by the corresponding tessellation block; otherwise, it may encode the primitive index number of the first new vertex as a third value if the first new vertex is not the first vertex generated by the corresponding tessellation block.

[0108] For example, Figure 6 A schematic diagram illustrating the encoding principle of a surface subdivision module provided in this application embodiment is shown below. Figure 6 As shown, the encoder of the surface tessellation module first determines whether the tessellation block has generated the first new vertex. If the tessellation block has not generated the first new vertex, it outputs the primitive index number of the tessellation block and sends the initial primitive index number and the primitive index number flags of each vertex to the region thread bundle construction module. If the tessellation block has generated the first new vertex, it sequentially inputs the first new vertex generated by the tessellation and determines whether the first new vertex is the first vertex generated by the tessellation. For the first vertex generated by the tessellation, the encoder can encode its primitive index number flag as the primitive index number flag of the tessellation block thread. For other first new vertices generated by the tessellation that are not the first vertex, the encoder can encode their primitive index number flags as a third value (i.e., 0, indicating keep).

[0109] In other implementations, if the subdivision block does not generate the first new vertex, the surface subdivision module can also use a decoder to decode the primitive index number flag of each vertex according to the initial primitive index number to obtain the primitive index number corresponding to each vertex, and then send the primitive index number to the region thread bundle construction module.

[0110] It should be noted that although the surface tessellation module does not execute in a thread and does not use primitive index numbers, it may discard tessellation blocks due to invalid tessellation coefficients. This could cause the primitive index number flag output by the surface tessellation module to produce incorrect results during subsequent decoding. Therefore, a decoder can be added to the surface tessellation module. This decoder uses the same decoding method as the thread bundle construction module, thereby updating the cached primitive index number based on the primitive index number flag of the tessellation block. If a tessellation block is discarded, the cached primitive index number is sent to the region thread bundle construction module.

[0111] The primitive index number flag of the subdivision block can be set to the primitive index number flag of the first vertex that makes up this subdivision block.

[0112] In some embodiments, when the surface tessellation module is enabled and a first new vertex is generated, the region thread bundle construction module is used to construct a region thread for each first new vertex; determine the primitive index number flag of the first new vertex corresponding to each region thread as the primitive index number flag of the region thread; determine the primitive index number corresponding to the region thread according to the initial primitive index number and the primitive index number flag of the region thread; send the primitive index number corresponding to the region thread to the thread bundle execution module; and send the initial primitive index number of the primitive to be processed and the primitive index number flag of the first new vertex to the geometry thread bundle construction module.

[0113] In some embodiments, when the tessellation module is in the enabled state and no first new vertex is generated, the tessellation module is further configured to send the received primitive index number cached internally by the tessellation module to the region thread construction module.

[0114] In some embodiments, when the tessellation module is in a closed state, the tessellation module is further configured to send the initial primitive index number and the received primitive index number flags of each vertex to the region thread bundle construction module.

[0115] It should be understood that each region thread corresponds to the first new vertex sent by a tessellation module, and the primitive index number of the region thread is the primitive index number of this first new vertex. The primitive index number of the vertex is output by the tessellation module.

[0116] For example, the region thread bundle construction module can construct a region thread bundle from a certain number of first new vertices, with each region thread corresponding to one first new vertex, and determine the primitive index number flag of the region thread as the primitive index number flag of the corresponding first new vertex. Subsequently, the region thread bundle construction module decodes the primitive index number flag of the region thread using its decoder to obtain the primitive index number corresponding to the region thread. Finally, the region thread bundle construction module sends the primitive index number corresponding to the region thread to the thread bundle execution module as input data to execute the computation logic of the target type thread for rendering the primitives to be processed according to the target type.

[0117] In some embodiments, the geometry thread bundle construction module is used to construct a geometry thread corresponding to the primitive composed of received vertices and perform geometric transformations on the geometry thread to obtain a new primitive; determine the primitive index number flag of the first new vertex of the first new primitive generated by each geometry thread as the primitive index number flag of the geometry thread; and send the primitive index number corresponding to the geometry thread to the thread bundle execution module.

[0118] It should be understood that each geometry thread corresponds to a primitive, which is composed of vertices received by the geometry thread bundle construction module. Each geometry thread performs geometric transformations to obtain a new primitive. The primitive index number of each new primitive is marked by the primitive index number of the first vertex that makes up the new primitive. The primitive index number of the first new vertex of the first new primitive is also used as the primitive index number of the geometry thread.

[0119] For example, the geometry thread bundle construction module can combine a certain number of vertices into a primitive, and then form a thread bundle from multiple primitives (e.g., 32). Each geometry thread in the thread bundle corresponds to one primitive. After the geometry thread bundle construction module obtains new primitives through geometric transformations of the geometry threads, it determines the primitive index number of the first new vertex of the first new primitive generated by each geometry thread as the primitive index number of the geometry thread. Subsequently, the primitive index number of the geometry thread can be decoded by a decoder to determine the primitive index number corresponding to the geometry thread. Finally, the geometry thread bundle construction module sends the primitive index number corresponding to the geometry thread to the thread bundle execution module as input data for the geometry thread to execute the computation logic of the target type thread, so as to render the primitive to be processed according to the target type.

[0120] In some embodiments, the geometry thread bundle construction module is further configured to encode the second vertex that combines to form the first new primitive and other vertices of other new primitives to obtain a primitive index number flag for the second new vertex. The second new vertex is a vertex in the new primitive obtained by the geometric transformation of the geometry thread bundle construction module.

[0121] It should be understood that the embodiments of this application do not limit how the geometry thread bundle construction module encodes the second new vertex. In some embodiments, when the second new vertex is the first new vertex generated to form the corresponding first new primitive, the encoder in the geometry thread bundle construction module encodes the primitive index number flag of the second new vertex as the primitive index number flag of the geometry thread. When the second new vertex is not the first new vertex generated to form the corresponding first new primitive, or is another new vertex of other new primitives, the encoder in the geometry thread bundle construction module encodes the primitive index number flag of the second new vertex as a third value.

[0122] For example, Figure 7 This is a schematic diagram illustrating the encoding principle of a geometric thread bundle construction module provided in an embodiment of this application, as shown below. Figure 7 As shown, if the second new vertex generated by each geometry thread can always form at least one new primitive, and this second new vertex is the first new vertex of the first new primitive generated by each geometry thread, then the primitive index number flag of the second new vertex can be encoded as the primitive index number flag of the new primitive. For example, if the second new vertex generated by each geometry thread can always form at least one new primitive, and this second new vertex is a non-first new vertex of the first new primitive generated by each geometry thread, or a new vertex of other new primitives, then the primitive index number flag of the second new vertex can be encoded as a third value (i.e., 0, indicating keep).

[0123] For example, if the geometry thread does not generate a second new vertex, or if all the second new vertices generated by the geometry thread cannot form any new primitive, the geometry thread bundle construction module can send the primitive index number cached within the geometry thread bundle construction module to the rasterization module. The primitive index number cached within the geometry thread bundle construction module can be obtained through the decoder of the geometry thread bundle construction module.

[0124] In some embodiments, when the geometry thread bundle construction module is in a closed state, the geometry thread bundle construction module is used to pass the information received from the region thread bundle construction module to the rasterization module.

[0125] When the geometry thread bundle construction module is enabled and generates new primitives, the initial primitive index number and the primitive index number flag of the second new vertex are sent to the rasterization module.

[0126] When the geometry thread bundle construction module is active and has not generated any new primitives, the primitive index numbers cached within the geometry thread bundle construction module are sent to the rasterization module. That is, when a second new vertex is combined into multiple new primitives, only those second new vertices that can be combined into new primitives are sent to the rasterization module; otherwise, the second new vertex is discarded. If all second new vertices of a certain geometry thread are discarded, then the primitive index numbers cached within the geometry thread bundle construction module can be sent to the rasterization module.

[0127] In addition, when the surface subdivision module is enabled, the geometry thread bundle construction module is also used to update the primitive index number cached in the geometry thread bundle construction module according to the primitive index number cached in the surface subdivision module; and send the updated primitive index number cached in the geometry thread bundle construction module to the rasterization module.

[0128] In some embodiments, the rasterization module can combine the vertices of the primitive to be processed, the first new vertex, or the second new vertex into a target primitive. Then, the target primitive is sampled to obtain pixels, and the primitive index number of the pixels is encoded to obtain a primitive index number flag. Finally, the rasterization module sends the pixel primitive index number flag and the initial primitive index number to the pixel thread bundle construction module.

[0129] Correspondingly, the pixel thread bundle construction module can construct a pixel thread for each pixel. It determines the primitive index number corresponding to the pixel thread based on the pixel's primitive index number flag and initial primitive index number. Subsequently, the pixel thread bundle construction module can send the primitive index number corresponding to the pixel thread to the thread bundle execution module.

[0130] For example, the rasterization module combines the received vertices into a target primitive and samples them to generate pixels of the target primitive. Subsequently, the encoding component of the rasterization module generates a primitive index number flag for the pixel. If the target primitive is discarded in the rasterization module, the rasterization module can send the primitive index number flag of the target primitive to the pixel thread bundle construction module. The pixel thread bundle construction module can construct a certain number of pixels into a thread bundle, with each thread corresponding to one pixel. The decoder of the pixel thread bundle construction module generates a primitive index number for each pixel and sends it to the thread bundle execution module as input data for the thread.

[0131] It should be understood that the embodiments of this application do not limit how the rasterization module encodes the primitive index number of a pixel. In some embodiments, when the pixel is the first pixel generated by interpolation constituting the target primitive, the rasterization module can encode the primitive index number of the pixel as the primitive index number of the target primitive; when the pixel is not the first pixel generated by interpolation constituting the target primitive, the rasterization module can encode the primitive index number of the pixel as a third value.

[0132] For example, Figure 8 This application provides a schematic diagram of the encoding principle of a rasterization module, as shown in the embodiment. Figure 8 As shown, if a target primitive is culled in the rasterization module, the rasterization module can send the primitive index number of the target primitive to the pixel thread bundle construction module. This primitive index number is obtained through the decoder of the rasterization module. If the pixel is the first pixel generated by interpolation of each target primitive, the pixel's primitive index number flag can be encoded as the primitive index number flag of the target primitive. The primitive index number flag of the target primitive is the primitive index number flag that constitutes this target primitive and is output by the geometry thread bundle construction module. If the pixel is not the first pixel generated by interpolation of each target primitive, the pixel's primitive index number flag can be encoded as a third value (i.e., 0, indicating keep).

[0133] It's important to note that although the rasterization module doesn't execute with a thread and doesn't use primitive index numbers, it might discard target primitives due to pruning, causing errors in the primitive index number flags output by the rasterization module during subsequent decoding. Therefore, a decoder is added to the rasterization module, using the same decoding method as the thread-based module. The primitive index number flag of the target primitive updates the cached primitive index number; if the target primitive is discarded, the cached primitive index number is sent to subsequent modules. The primitive index number flag of the target primitive is set to the primitive index number flags that make up this target primitive.

[0134] The encoding and decoding process of primitive index numbers will be explained below with specific examples.

[0135] For example, the input assembly module distributes 12 vertices of the primitives to be processed for 4 triangles. An encoder encodes each vertex to generate a primitive index number, which is then sent to the vertex thread bundle construction module. The vertex thread bundle construction module transmits the primitive index number to the pixel thread bundle construction module via the shell thread bundle construction module, surface subdivision module, region thread bundle construction module, geometry thread bundle construction module, and rasterization module. The pixel thread bundle construction module decodes each pixel to generate a primitive index number, which is then sent to the thread bundle execution module. The thread bundle execution module writes the primitive index number to a general-purpose register for use as input data for the thread bundle, i.e., the general-purpose register R0 corresponding to v[0].x. Subsequently, the ieq instruction determines whether it is the second triangle, and the movc instruction sets the color of the second triangle to a constant 0x3F800000 (i.e., floating-point number 1.0f). The final effect is that the color of the second primitive to be processed is white, and the colors of the remaining primitives to be processed are the results of interpolation by the rasterization module.

[0136] It should be noted that R0 is not fixed and can be other general-purpose registers, which can be configured in advance via software.

[0137] Figure 9 This application provides an example of a schematic diagram illustrating the encoding of a primitive index number. Figure 9 As shown, the primitives to be processed in the first two triangles are sent to slice0, and the primitives to be processed in the last two triangles are sent to slice1. Each primitive to be processed in a triangle consists of 3 vertices, for a total of 12 vertices (v0~v11). (See reference...) Figure 9 It can be seen that each vertex has a 32-bit primitive index number and an encoded 2-bit primitive index number flag.

[0138] The 2-bit primitive index flag has three states: 0 indicates keep, 1 indicates increase, and 2 indicates reset. For example, keep means the primitive index is the same as the previous vertex, increase means the primitive index is incremented by 1, and reset means the primitive index is 0. It's important to note that the primitive index flag indicates the relationship between vertices processed within the same slice, and is unrelated to vertices between different slices.

[0139] For example, the primitive index number of the first vertex v0 sent to slice0 is 0, and the primitive index number flag of v0 is in the reset state. Therefore, the input assembly module does not need to send the primitive index number of v0 to slice0; it only needs to send the primitive index number flag of v0 to decode and generate the primitive index number of v0 as 0. However, the primitive index number of the first vertex v6 ​​sent to slice1 is 2. The input assembly module needs to send the complete 32-bit primitive index number to slice1, and this primitive index number is 2. At this time, this primitive index number is the complete 32-bit primitive index number of the first vertex of the primitive to be processed (equivalent to the primitive index number of the vertex preceding v6). By using the complete 32-bit primitive index number of the first vertex of the primitive to be processed as 2 and the primitive index number flag of v6 as keep, the primitive index number of v6 can be decoded and generated as 2.

[0140] Figure 10 A schematic diagram of a buffer for a thread bundle construction module provided in an embodiment of this application is shown below. Figure 10 As shown, each thread bundle construction module needs to add a buffer to store one cached primitive index number and 32 primitive index number flags. The primitive index number flag 0 of thread 0 is decoded based on the cached primitive index number flag to generate its own primitive index number, and the newly generated primitive index number replaces the 32 bits of the cached primitive index number in the buffer; then, the primitive index number flag 1 of thread 1 is decoded based on the replaced cached primitive index number in the buffer to generate its own primitive index number; and so on, the primitive index numbers of all threads can be decoded based on the cached primitive index number and primitive index number flags.

[0141] Figure 11 A schematic diagram of decoding a primitive index number provided in an embodiment of this application is shown below. Figure 11 As shown, the cached primitive index number and primitive index number flag stored in the buffer of the pixel thread bundle construction module are as follows: Figure 11 The table above shows the decoded 32-bit primitive index numbers as follows: Figure 11 The table below shows the primitive index number 3, which is the final value of all primitive index numbers in the buffer that are continuously updated during the decoding process. That is, the primitive index number of the last thread stored in the buffer, and the next thread bundle will continue decoding based on this primitive index number.

[0142] Since the primitive index number flag of the 0th pixel thread is 0 and in a keep state, and the cached primitive index number in the buffer is 2, the primitive index number of the 0th thread directly uses the cached primitive index number 2 in the buffer. The primitive index number flags of the 1st to 6th pixel threads are all in a keep state, indicating that the primitive index number is the same as that of the 0th pixel thread. Therefore, the primitive index numbers of the 1st to 6th pixel threads are all 2. The primitive index number flag of the 7th pixel thread is in an incr state. Therefore, the primitive index number of the 7th pixel thread is the cached primitive index number 2 + 1 = 3, and the cached primitive index number in the buffer is updated to 3. And so on, the primitive index numbers of all pixel threads can be decoded and generated.

[0143] In this embodiment, encoders and / or decoders are added to different modules of the GPU. The encoder is responsible for encoding the 32-bit primitive index number into a 2-bit primitive index number flag, and the decoder is responsible for decoding the 2-bit primitive index number flag into a 32-bit primitive index number. The GPU transmits and stores the primitive index number flag using the 2-bit primitive index number flag. When the primitive index number needs to be used as input data for a thread, the 2-bit primitive index number flag is then decoded into a 32-bit primitive index number. In this way, the width of the primitive index number transmitted by the hardware and the buffer space for storing the primitive index number can be reduced, and the method of using primitive index numbers as input data for all shaders can be perfectly unified into the GPU architecture.

[0144] The image processing apparatus provided in this application includes an input assembly module, a thread bundle execution module, and multiple thread bundle construction modules. The input assembly module acquires the primitive index numbers of each vertex of the primitive to be processed, encodes the primitive index numbers to obtain primitive index number flags for each vertex of the primitive to be processed, and sequentially transmits the primitive index number flags to the multiple thread bundle construction modules. The thread bundle construction module constructs threads of a target type; determines the primitive index number corresponding to the thread of the target type based on the primitive index number flags; and sends the thread of the target type and its corresponding primitive index number to the thread bundle execution module. The thread bundle execution module uses the primitive index number corresponding to the thread of the target type as input data for the thread of the target type, executes the calculation logic of the thread of the target type, and renders the primitive to be processed in the target type. Because the primitive index numbers of the vertices are encoded into primitive index number flags and transmitted between the multiple thread bundle construction modules, and then each thread bundle construction module decodes the primitive index number flags into primitive index numbers before rendering the corresponding thread, the amount of data transmitted between the modules is greatly reduced, thus lowering the data transmission bandwidth during image processing.

[0145] Figure 12 This is a schematic flowchart of an image processing method provided in an embodiment of this application, as shown below. Figure 12 As shown, the image processing method is applied to the above-mentioned image processing apparatus, and the image processing method includes S301-S305:

[0146] S301. Obtain the primitive index number of each vertex of the primitive to be processed.

[0147] S302. Encode the primitive index number to obtain the primitive index number flag of each vertex of the primitive to be processed.

[0148] In some embodiments, the primitive index number is encoded to obtain the primitive index number flag of each vertex of the primitive to be processed, including: if the target vertex is the first vertex of its own primitive and the own primitive is the first primitive, then the primitive index number flag of the target vertex is encoded as a first value; if the target vertex is the first vertex of its own primitive and the own primitive is not the first primitive, then the primitive index number flag of the target vertex is encoded as a second value; if the target vertex is not the first vertex of its own primitive, then the primitive index number flag of the target vertex is encoded as a third value, wherein the target vertex is any vertex of the primitive to be processed.

[0149] S303. The primitive index number flag is transmitted sequentially to multiple thread bundle construction modules.

[0150] S304. Construct a thread of the target type, and determine the primitive index number corresponding to the thread of the target type based on the primitive index number flag.

[0151] In some embodiments, determining the primitive index number corresponding to the thread of the target type based on the primitive index number flag includes: when the primitive index number flag of the thread of the target type is a third value, determining the cached primitive index number as the primitive index number corresponding to the thread of the target type; when the primitive index number flag of the thread of the target type is a second value, updating the cached primitive index number by incrementing it by one and determining the updated cached primitive index number as the primitive index number corresponding to the thread of the target type; when the primitive index number flag of the thread of the target type is a first value, updating the cached primitive index number to zero and determining the updated cached primitive index number as the primitive index number corresponding to the thread of the target type.

[0152] The initial value of the cached primitive index number is the initial primitive index number of the primitive to be processed.

[0153] The image processing device also includes a surface subdivision module and a rasterization module, and the thread bundle construction module includes a vertex thread bundle construction module, a shell thread bundle construction module, a region thread bundle construction module, a geometry thread bundle construction module, and a pixel thread bundle construction module.

[0154] In some embodiments, the method further includes: receiving primitive index number flags and initial primitive index numbers of each vertex of the primitive to be processed sent by the input assembly module; constructing vertex threads for each vertex; determining the primitive index number flags of the vertex threads as the primitive index number flags of the corresponding vertices; determining the primitive index number corresponding to the vertex thread based on the initial primitive index number and the primitive index number flags of the vertex threads; sending the primitive index number corresponding to the vertex thread to the thread bundle execution module; and sending the primitive index number flags and initial primitive index numbers of each vertex of the primitive to be processed to the shell thread bundle construction module.

[0155] In some embodiments, the method further includes: combining multiple vertices into subdivision blocks and constructing corresponding subdivision block threads; determining the primitive index number flag of the first vertex constituting the subdivision block as the primitive index number flag of the subdivision block thread; determining the primitive index number corresponding to the subdivision block thread based on the initial primitive index number and the primitive index number flag of the subdivision block thread; sending the primitive index number corresponding to the subdivision block thread to the thread bundle execution module; and sending the initial primitive index number and the primitive index number flags of each vertex to the surface subdivision module.

[0156] In some embodiments, the method further includes: when the surface tessellation module is in a closed state, sending the initial primitive index number and the primitive index number flags of each vertex to the region thread bundle construction module;

[0157] When the surface tessellation module is enabled and the first new vertex is generated, the initial primitive index number and the primitive index number flag of the first new vertex are sent to the region thread bundle construction module.

[0158] When the tessellation module is enabled and no new vertex has been generated, the primitive index number cached inside the tessellation module is sent to the region thread construction module.

[0159] The first new vertex is a new vertex generated after surface subdivision.

[0160] In some embodiments, the method further includes: if the first new vertex is the first vertex generated by the corresponding subdivision block, encoding the primitive index number flag of the first new vertex as the primitive index number flag of the subdivision block thread; if the first new vertex is not the first vertex generated by the corresponding subdivision block, encoding the primitive index number flag of the first new vertex as a third value.

[0161] In some embodiments, the method further includes: when the surface tessellation module is enabled and a first new vertex is generated, constructing a region thread for each first new vertex; determining the primitive index number flag of the first new vertex corresponding to each region thread as the primitive index number flag of the region thread; determining the primitive index number corresponding to the region thread based on the initial primitive index number and the primitive index number flag of the region thread; sending the primitive index number corresponding to the region thread to the thread bundle execution module; and sending the initial primitive index number and the primitive index number flag of the first new vertex to the geometry thread bundle construction module.

[0162] In some embodiments, the method further includes: when the tessellation module is in the enabled state and no first new vertex is generated, sending the received primitive index number cached inside the tessellation module to the region thread construction module.

[0163] In some embodiments, the method further includes: sending the initial primitive index number and the received primitive index number flags of each vertex to the region thread bundle construction module when the surface tessellation module is in a closed state.

[0164] In some embodiments, the method further includes: constructing a geometric thread corresponding to the primitives composed of the received vertices and performing geometric transformations on the geometric thread to obtain new primitives; determining the primitive index number flag of the first new vertex of the first new primitive generated by each geometric thread as the primitive index number flag of the geometric thread; and sending the primitive index number corresponding to the geometric thread to the thread bundle execution module.

[0165] In some embodiments, the method further includes: encoding the non-first new vertices that are combined into the first new primitive and other new vertices of other new primitives to obtain a primitive index number flag for the second new vertex; wherein the second new vertex is a vertex in the new primitive obtained by the geometric transformation of the geometric thread bundle construction module.

[0166] In some embodiments, the method further includes: when the geometry thread bundle construction module is in a closed state, the geometry thread bundle construction module is configured to pass information received from the region thread bundle construction module to the rasterization module;

[0167] When the geometry thread bundle construction module is in the enabled state and the geometry thread bundle construction module generates new primitives, the initial primitive index number and the primitive index number flag of the second new vertex are sent to the rasterization module.

[0168] When the geometry thread bundle construction module is in the open state and no new primitives are generated by the geometry thread bundle construction module, the primitive index number cached inside the geometry thread bundle construction module is sent to the rasterization module.

[0169] In some embodiments, the method further includes: when the surface tessellation module is in an enabled state, updating the primitive index number cached internally by the geometry thread bundle construction module according to the received primitive index number cached internally by the surface tessellation module; and sending the updated primitive index number cached internally by the geometry thread bundle construction module to the rasterization module. In some embodiments, the method further includes: combining each vertex, the first new vertex, or the second new vertex of the primitive to be processed into a target primitive; sampling the target primitive to obtain pixels of the target primitive; encoding the primitive index number of the pixel to obtain a primitive index number flag of the pixel; constructing a pixel thread for each pixel; determining the primitive index number corresponding to the pixel thread according to the primitive index number flag of the pixel and the initial primitive index number; and sending the primitive index number corresponding to the pixel thread to the thread bundle execution module.

[0170] In some embodiments, the method further includes: when the pixel is the first pixel generated by interpolation constituting the target primitive, encoding the primitive index number flag of the pixel as the primitive index number flag constituting the target primitive; when the pixel is not the first pixel generated by interpolation constituting the target primitive, encoding the primitive index number flag of the pixel as a third value.

[0171] S305. Use the primitive index number corresponding to the thread of the target type as the input data of the thread of the target type, execute the calculation logic of the thread of the target type, and render the primitive to be processed in the target type.

[0172] The image processing method provided in this application embodiment obtains the primitive index number of each vertex of the primitive to be processed; encodes the primitive index number to obtain the primitive index number flag of each vertex of the primitive to be processed; transmits the primitive index number flag to multiple thread bundle construction modules in sequence; constructs a thread of the target type, and determines the primitive index number corresponding to the thread of the target type according to the primitive index number flag; uses the primitive index number corresponding to the thread of the target type as the input data of the thread of the target type, executes the calculation logic of the thread of the target type, and renders the primitive to be processed in the target type. Since the primitive index number of the vertex is encoded into a primitive index number flag and transmitted between multiple thread bundle construction modules, and then each thread bundle construction module decodes the primitive index number flag into a primitive index number before performing rendering of the corresponding thread, the amount of data transmitted between the modules is greatly reduced, and the data transmission bandwidth in the image processing process is reduced.

[0173] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0174] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 13 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements an image processing method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0175] Those skilled in the art will understand that Figure 13 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0176] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the image processing method described above.

[0177] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the image processing method described above.

[0178] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the image processing method described above.

[0179] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0180] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0181] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image processing apparatus, characterized in that, The image processing device includes an input assembly module, a thread beam execution module, and multiple thread beam construction modules; The input assembly module is used to obtain the primitive index number of each vertex of the primitive to be processed, encode the primitive index number, and obtain the primitive index number flag of each vertex of the primitive to be processed. The primitive index number flag is sequentially transmitted to the plurality of thread bundle construction modules; The thread bundle construction module is used to construct a thread of the target type; determine the primitive index number corresponding to the thread of the target type according to the primitive index number flag; and send the thread of the target type and the primitive index number corresponding to the thread of the target type to the thread bundle execution module. The thread bundle execution module is used to take the primitive index number corresponding to the thread of the target type as the input data of the thread of the target type, and execute the calculation logic of the thread of the target type to render the primitive to be processed in the target type.

2. The image processing apparatus according to claim 1, characterized in that, The input assembly module is specifically configured to: encode the element index number of the target vertex as a first value when the target vertex is the first vertex of its element and the element it belongs to is the first element; encode the element index number of the target vertex as a second value when the target vertex is the first vertex of its element and the element it belongs to is not the first element; and encode the element index number of the target vertex as a third value when the target vertex is not the first vertex of its element, wherein the target vertex is any vertex of the element to be processed.

3. The image processing apparatus according to claim 2, characterized in that, The thread bundle construction module is specifically used to: determine the cached primitive index number as the primitive index number corresponding to the thread of the target type when the primitive index number flag of the target type thread is the third value; update the cached primitive index number by incrementing it by one when the primitive index number flag of the target type thread is the second value and determine the updated cached primitive index number as the primitive index number corresponding to the thread of the target type; and update the cached primitive index number to zero when the primitive index number flag of the target type thread is the first value and determine the updated cached primitive index number as the primitive index number corresponding to the thread of the target type. The initial value of the cached primitive index number is the initial primitive index number of the primitive to be processed.

4. The image processing apparatus according to any one of claims 1-3, characterized in that, The image processing device further includes a surface subdivision module and a rasterization module, and the thread bundle construction module includes a vertex thread bundle construction module, a shell thread bundle construction module, a region thread bundle construction module, a geometric thread bundle construction module, and a pixel thread bundle construction module.

5. The image processing apparatus according to claim 4, characterized in that, The vertex thread bundle construction module is used to receive the primitive index number flag and initial primitive index number of each vertex of the primitive to be processed sent by the input assembly module; construct the vertex thread of each vertex; and determine the primitive index number flag of the vertex thread as the primitive index number flag of the corresponding vertex. Based on the initial primitive index number and the primitive index number flag of the vertex thread, determine the primitive index number corresponding to the vertex thread; and send the primitive index number corresponding to the vertex thread to the thread bundle execution module. The primitive index number flags of each vertex of the primitive to be processed and the initial primitive index number are sent to the shell thread bundle construction module.

6. The image processing apparatus according to claim 4, characterized in that, The shell thread bundle construction module is used to combine multiple vertices into subdivision blocks and construct corresponding subdivision block threads; the primitive index number of the first vertex that makes up the subdivision block is determined as the primitive index number of the subdivision block thread. Based on the initial primitive index number and the primitive index number flag of the subdivision block thread, determine the primitive index number corresponding to the subdivision block thread; send the primitive index number corresponding to the subdivision block thread to the thread bundle execution module; send the initial primitive index number and the primitive index number flag of each vertex to the surface subdivision module.

7. The image processing apparatus according to claim 6, characterized in that, When the surface tessellation module is in the off state, the surface tessellation module is used to send the initial primitive index number and the primitive index number flag of each vertex to the region thread bundle construction module; When the surface tessellation module is in the enabled state and a first new vertex is generated, the surface tessellation module is used to send the initial primitive index number and the primitive index number flag of the first new vertex to the region thread bundle construction module. When the surface tessellation module is in the enabled state and no first new vertex is generated, the surface tessellation module is used to send the primitive index number cached inside the surface tessellation module to the region thread construction module; The first new vertex is a new vertex generated after surface subdivision.

8. The image processing apparatus according to claim 7, characterized in that, The surface subdivision module is specifically used to encode the primitive index number flag of the first new vertex as the primitive index number flag of the subdivision block thread when the first new vertex is the first vertex generated by the corresponding subdivision block. If the first new vertex is not the first vertex generated by the corresponding subdivision block, the primitive index number of the first new vertex is encoded as a third value.

9. The image processing apparatus according to claim 8, characterized in that, When the surface subdivision module is in the enabled state and a first new vertex is generated, the region thread bundle construction module is used to construct a region thread for each first new vertex; and the primitive index number flag of the first new vertex corresponding to each region thread is determined as the primitive index number flag of the region thread. Based on the initial primitive index number and the primitive index number flag of the region thread, determine the primitive index number corresponding to the region thread; send the primitive index number corresponding to the region thread to the thread bundle execution module; send the initial primitive index number and the primitive index number flag of the first new vertex to the geometry thread bundle construction module.

10. The image processing apparatus according to claim 8, characterized in that, When the surface tessellation module is in the enabled state and no first new vertex is generated, the surface tessellation module is also used to send the received primitive index number cached inside the surface tessellation module to the region thread construction module.

11. The image processing apparatus according to claim 8, characterized in that, When the surface tessellation module is in the off state, the surface tessellation module is also used to send the initial primitive index number and the received primitive index number flags of each vertex to the region thread bundle construction module.

12. The image processing apparatus according to claim 9, characterized in that, The geometric thread bundle construction module is used to construct the geometric thread corresponding to the primitive composed of the received vertices and perform geometric transformations on the geometric thread to obtain a new primitive; the primitive index number of the first new vertex of the first new primitive generated by each geometric thread is determined as the primitive index number of the geometric thread. The primitive index number corresponding to the geometric thread is sent to the thread bundle execution module.

13. The image processing apparatus according to claim 12, characterized in that, The geometric thread bundle construction module is also used to encode the non-first new vertices that are combined to form the first new primitive and other new vertices of other new primitives to obtain the primitive index number flag of the second new vertex; The second new vertex is a vertex in the new primitive obtained by the geometric transformation of the geometric thread bundle construction module.

14. The image processing apparatus according to claim 13, characterized in that, When the geometry thread bundle construction module is in a closed state, the geometry thread bundle construction module is used to pass the information received from the region thread bundle construction module to the rasterization module; When the geometry thread bundle construction module is in the enabled state and the geometry thread bundle construction module generates new primitives, the geometry thread bundle construction module sends the initial primitive index number and the primitive index number flag of the second new vertex to the rasterization module. When the geometry thread bundle construction module is in the enabled state and the geometry thread bundle construction module does not generate new primitives, the geometry thread bundle construction module sends the primitive index number cached inside the geometry thread bundle construction module to the rasterization module.

15. The image processing apparatus according to claim 14, characterized in that, When the surface subdivision module is enabled, the geometry thread bundle construction module is further configured to update the primitive index number cached in the geometry thread bundle construction module according to the received primitive index number cached in the surface subdivision module; and send the updated primitive index number cached in the geometry thread bundle construction module to the rasterization module.

16. The image processing apparatus according to claim 14, characterized in that, The rasterization module is used to combine each vertex of the primitive to be processed, the first new vertex, or the second new vertex into a target primitive; and to sample the target primitive to obtain the pixels of the target primitive; The primitive index number of the pixel is encoded to obtain the primitive index number flag of the pixel; Send the primitive index number flag of the pixel and the initial primitive index number to the pixel thread bundle construction module; The pixel thread bundle construction module is used to construct a pixel thread for each pixel; Based on the primitive index number flag of the pixel and the initial primitive index number, determine the primitive index number corresponding to the pixel thread; and send the primitive index number corresponding to the pixel thread to the thread bundle execution module.

17. The image processing apparatus according to claim 16, characterized in that, The rasterization module is further configured to, when the pixel is the first pixel generated by interpolation constituting the target primitive, encode the primitive index number flag of the pixel as the primitive index number flag constituting the target primitive; and when the pixel is not the first pixel generated by interpolation constituting the target primitive, encode the primitive index number flag of the pixel as a third value.

18. An image processing method, characterized in that, The method includes: Obtain the primitive index number of each vertex of the primitive to be processed; The primitive index number is encoded to obtain the primitive index number flag of each vertex of the primitive to be processed; The primitive index number flag is sequentially transmitted to multiple thread bundle construction modules; Construct a thread of the target type, and determine the primitive index number corresponding to the thread of the target type based on the primitive index number flag; The primitive index number corresponding to the thread of the target type is used as the input data of the thread of the target type, and the calculation logic of the thread of the target type is executed to render the primitive to be processed in the target type.

19. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method of claim 18.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method of claim 18.

21. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method of claim 18.