Metathread application method, chip system, electronic device, medium and program product

Abstract

The present disclosure provides a meta-thread application method, a chip system, an electronic device, a medium and a program product. The method comprises the following steps: first, receiving a kernel function starting instruction based on a meta-thread instruction scheduler, and generating a meta-thread corresponding to the kernel function starting instruction; then, determining the thread task type of the meta-thread through the meta-thread instruction scheduler, and determining the corresponding target function unit according to the thread task type, wherein the target function unit is one or more of a plurality of function units; then, sending the meta-thread to the target function unit through the meta-thread instruction scheduler, and executing the meta-thread through the target function unit. Since the present disclosure proposes a scheme of increasing meta-threads, and can call the corresponding target function unit according to the thread task type of the meta-thread, it is not necessary to start a large number of ordinary work threads through the execution unit every time, thereby avoiding excessive occupation of the resources of the execution unit, and solving the problem of large resource occupation when the demand is insufficient.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of chip design, and in particular, to a metathread application method, a chip system, an electronic device, a medium and a program product. BACKGROUND

[0002] In related technologies, in a modern Graphics Processing Unit (GPU), a large number of ordinary work threads are usually started to run a kernel function. However, the ordinary work threads will occupy a large amount of resources. Specifically, the ordinary work threads will be allocated private general purpose registers, and an active thread bundle will occupy a "warp slot", which contains resources such as warp-scalar registers, program counters, warp state registers, etc. When the kernel function does not perform "element-wise operations", the ordinary work threads will be over-provisioned. For example, communication kernels and block-processing kernels do not perform element-wise operations. SUMMARY

[0003] Embodiments of the present disclosure provide a metathread application method, a chip system, an electronic device, a medium and a program product, aiming to solve the problem of large resource occupation.

[0004] According to an aspect of the present disclosure, a metathread application method is provided, applied to a chip system, the chip system comprising a metathread instruction scheduler and a plurality of functional units, the plurality of functional units being in communication connection with the metathread instruction scheduler; the metathread application method comprising:

[0005] generating a metathread corresponding to the kernel function start instruction based on the metathread instruction scheduler receiving the kernel function start instruction;

[0006] determining a thread task type of the metathread by the metathread instruction scheduler, and determining a corresponding target functional unit according to the thread task type, wherein the target functional unit is one or more of the plurality of functional units;

[0007] sending the metathread to the target functional unit by the metathread instruction scheduler, and executing the metathread by the target functional unit.

[0008] In some embodiments, the determining the corresponding target functional unit according to the thread task type comprises one of the following:

[0009] In a case where the thread task type is a block operation task, the target functional unit is determined as a block operation unit;

[0010] In a case where the thread task type is a scalar operation task, the target functional unit is determined as a scalar operation unit;

[0011] In a case where the thread task type is a vector operation task, the target functional unit is determined as an execution unit.

[0012] In some embodiments, the determining the target functional unit as a block operation unit comprises one of the following:

[0013] In a case where the block operation task is a communication operation task, the block operation unit is determined as a communication operation unit;

[0014] In a case where the block operation task is a data copy task, the block operation unit is determined as a data copy unit;

[0015] In a case where the block operation task is a matrix operation task, the block operation unit is determined as a tensor calculation unit.

[0016] In some embodiments, the sending the meta-thread to the target functional unit by the meta-thread instruction scheduler comprises one of the following:

[0017] The target functional unit executes the meta-thread, comprising:

[0018] In a case where the target functional unit is an execution unit, the meta-thread carrying thread configuration information of a normal worker thread is sent to the execution unit by the meta-thread instruction scheduler;

[0019] The normal worker thread is managed by the execution unit according to the thread configuration information.

[0020] In some embodiments, the managing the normal worker thread by the execution unit according to the thread configuration information comprises one of the following:

[0021] The normal worker thread is woken up by the execution unit;

[0022] The normal worker thread is terminated by the execution unit.

[0023] In some embodiments, the meta-thread application method comprises one of the following:

[0024] One thread block corresponds to one meta-thread;

[0025] One workgroup corresponds to one of the meta-threads;

[0026] One Trition grid point corresponds to one of the meta-threads.

[0027] In some embodiments, one of the meta-threads corresponds to a set of state registers, a set of scalar registers, and a program counter.

[0028] In some embodiments, the generating the meta-threads corresponding to the kernel launch instruction comprises one of:

[0029] For the same kernel, the meta-thread instruction scheduler generates a plurality of meta-threads, wherein one meta-thread corresponds to one workgroup in the kernel.

[0030] For a plurality of different kernels, the meta-thread instruction scheduler generates a plurality of meta-threads, wherein one meta-thread corresponds to one of the kernels.

[0031] In some embodiments, the meta-thread instruction scheduler is located in a compute core, which comprises one of: a compute unit, a streaming multi-processor.

[0032] In some embodiments, the meta-thread application method further comprises:

[0033] Obtaining the dependency relationship of the meta-thread;

[0034] When the dependency relationship has been resolved, continuing to execute the meta-thread by the target functional unit.

[0035] In some embodiments, the meta-thread application method further comprises:

[0036] In the presence of a first meta-thread and a second meta-thread, obtaining a first dependency relationship of the first meta-thread and a second dependency relationship of the second meta-thread;

[0037] When the first dependency relationship has not been resolved and the second dependency relationship has been resolved, deactivating the first meta-thread, and executing the second meta-thread in the deactivation time slot of the first meta-thread.

[0038] In some embodiments, the type of the dependency relationship comprises one of: a synchronization dependency type, a scalar load delay type, a resource occupation type.

[0039] According to an aspect of the present disclosure, a chip system is provided, comprising a meta-thread instruction scheduler and a plurality of functional units, the plurality of functional units are in communication connection with the meta-thread instruction scheduler; wherein the meta-thread instruction scheduler is configured to receive a kernel function starting instruction, generate a meta-thread corresponding to the kernel function starting instruction, and send the meta-thread to a target functional unit according to a thread task type of the meta-thread; and the target functional unit is configured to execute the meta-thread, wherein the target functional unit is one or more of the plurality of functional units.

[0040] According to an aspect of the present disclosure, an electronic device is provided, comprising a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection communication between the processor and the memory, wherein the program is executed by the processor to realize the meta-thread application method as described above.

[0041] According to an aspect of the present disclosure, a computer readable storage medium is provided, which stores one or more programs executable by one or more processors to realize the meta-thread application method as described above.

[0042] The meta-thread application method, chip system, electronic device, medium and program product provided by the present disclosure, the method comprises: first, based on the meta-thread instruction scheduler receiving a kernel function starting instruction, generating a meta-thread corresponding to the kernel function starting instruction; then, determining the thread task type of the meta-thread through the meta-thread instruction scheduler, and determining the corresponding target functional unit according to the thread task type, wherein the target functional unit is one or more of the plurality of functional units; then, sending the meta-thread to the target functional unit through the meta-thread instruction scheduler, and executing the meta-thread through the target functional unit. Since the present disclosure provides a scheme of increasing meta-threads, and the corresponding target functional unit can be called according to the thread task type of the meta-thread, it is not necessary to start a large number of ordinary work threads through the execution unit every time, thereby avoiding excessive occupation of the resources of the execution unit, and solving the problem of large resource occupation when the demand is insufficient.

[0043] Other features and advantages of the present disclosure will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present disclosure. The objects and other advantages of the present disclosure can be achieved and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0044] The accompanying drawings are intended to provide a further understanding of the technical solutions of the present disclosure, and constitute a part of the specification, and are used together with the embodiments of the present disclosure to explain the technical solutions of the present disclosure, and do not constitute a limitation on the technical solutions of the present disclosure.

[0045] Figure 1 is an architecture diagram of a system applying the method of the application of the meta-thread application according to an embodiment of the present disclosure;

[0046] Figure 2 is a schematic diagram of the relationship between the meta-thread, the common work thread, the thread bundle, the thread block and the thread cluster according to an embodiment of the present disclosure;

[0047] Figure 3 is a main flowchart of the method of the application of the meta-thread according to an embodiment of the present disclosure;

[0048] Figure 4 is a flowchart of the method of the application of the meta-thread according to an embodiment of the present disclosure when the thread task type is a block operation task;

[0049] Figure 5 is a flowchart of the method of the application of the meta-thread according to an embodiment of the present disclosure when the thread task type is a scalar operation task; Figure 4 is a sub-step flowchart of step S402 according to an embodiment of the present disclosure;

[0050] Figure 6 is a sub-step flowchart of step S402 according to another embodiment of the present disclosure; Figure 4 is a sub-step flowchart of step S402 according to another embodiment of the present disclosure;

[0051] Figure 7 is a sub-step flowchart of step S402 according to another embodiment of the present disclosure; Figure 4 is a sub-step flowchart of step S402 according to another embodiment of the present disclosure;

[0052] Figure 8 is a flowchart of the method of the application of the meta-thread according to an embodiment of the present disclosure when the thread task type is a vector operation task;

[0053] Figure 9 is a flowchart of the method of the application of the meta-thread according to an embodiment of the present disclosure when the thread task type is a vector operation task;

[0054] Figure 10 is a sub-step flowchart of step S903 according to an embodiment of the present disclosure; Figure 9 is a sub-step flowchart of step S903 according to an embodiment of the present disclosure;

[0055] Figure 11 is a flowchart of the method of the application of the meta-thread according to an embodiment of the present disclosure according to the dependency relationship;

[0056] Figure 12 is a flowchart of the method of the application of the meta-thread according to an embodiment of the present disclosure during the deactivation of the first meta-thread;

[0057] Figure 13 is a structural schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and do not limit the present disclosure.

[0059] Before the embodiments of the present disclosure are further described in detail, the terms and phrases involved in the embodiments of the present disclosure are explained, and the terms and phrases involved in the embodiments of the present disclosure are applicable to the following explanations:

[0060] Chip: Chip generally refers to an integrated circuit, which is a microelectronic device that integrates circuits on a tiny silicon chip; chips are widely used in various electronic devices, such as computers, mobile phones, cars, etc., for implementing various functions such as computing, storage, communication, etc. The chip is a packaged die that can be used in actual electronic systems. And the chip can include a command processor and a plurality of block processing modules, each of which is in communication connection with the command processor.

[0061] GPU (Graphics Processing Unit) is a microprocessor specially designed for parallel processing of a large number of graphics and general-purpose computing tasks. Its core function is a processor specially designed for high-parallel computing tasks, which is good at processing a large number of simple and repetitive computing operations simultaneously. It was originally designed to accelerate image rendering, but due to its powerful parallel computing capabilities, it has been widely used in scientific computing, artificial intelligence, deep learning, physical simulation, etc.

[0062] Command processor (CP): The command processor is a command processing unit in the chip, which is responsible for processing and executing various instructions or commands to ensure that each module inside the chip can run efficiently and in order; in the embodiments of the present disclosure, the synchronization of the CP layer mainly refers to the synchronization between the internal or different block processing clusters inside the chip, and the synchronization information is in the memory of the command processor, and the synchronization operator is the command processor. Among them, the command processor can include a task data packet cache, a parsing module and a segmentation module, the task data packet cache is used for storing and processing the information sent by the host layer, the parsing module is used for parsing the information stored in the task data packet cache, and the segmentation module is used for dividing the task information parsed by the parsing module to send the related task information to the related block processing cluster for execution processing.

[0063] Compute Unit (CU): In graphics processor architecture, a Compute Unit is the basic unit that performs computational tasks. When a workgroup runs on a graphics processor, it is actually executed on a Compute Unit. A Compute Unit is the basic unit on a graphics processor for performing parallel computing tasks, which contains a certain number of processing elements that can execute multiple threads in parallel. A block processing cluster can include multiple Compute Units. In the embodiments of the present disclosure, the synchronization of the CU layer refers to the synchronization within the same CU, the synchronization between different workgroups or different kernels within the CU, and is implemented through data packets and kernels in the synchronization process. Among them, a Compute Unit can include a workgroup cluster data cache, a synchronization engine, a segmentation module, a task scheduling module, and multiple execution units. The workgroup cluster data cache is used to receive information sent by the command scheduling unit, the synchronization engine is used to parse and synchronize the information in the workgroup cluster data cache to obtain task information, and the segmentation module is used to segment the task information and send the corresponding task information to the execution unit for task execution processing.

[0064] Streaming Multiprocessor (SM): is the core computing unit of GPU, composed of multiple stream processors and related resources, responsible for parallel execution of a large number of threads, and realizes efficient graphics rendering and computing tasks. SM is composed of multiple stream processors (SP). In addition, it also includes shared memory, registers, control units, instruction caches and other resources. These components work together to support meta-thread applications. SM schedules threads in units of warps, each warp usually contains 32 threads. Threads within the same warp execute the same instructions in a single instruction multiple thread (SIMT) manner, but process different data. This mechanism allows SM to efficiently handle a large number of parallel tasks, such as rendering multiple pixels in an image. Its core function is parallel computing, mainly used for tasks such as vertex processing and pixel shading in graphics rendering. At the same time, with the development of technology, SM is also widely used in artificial intelligence training, big data analysis and other non-graphic fields, fully exerting its powerful parallel processing capability. SM is the core module of GPU, and multiple SMs form the computing part of GPU. The number of SMs in different GPU models is different, and the number and performance of SMs directly affect the computing power and graphics processing speed of GPU.

[0065] Execution Unit (EU): is the functional hardware module that actually completes the "computing operation" in the processor core. It is assigned one or more decoded instructions by the scheduler and completes arithmetic, logic, memory access or special operations in a specific clock cycle, and writes the results back to the target location.

[0066] Kernel Function: In GPU programming, a kernel function is a function defined specifically for execution on a GPU, serving as the core bridge connecting CPU and GPU computation. It is responsible for describing the specific task logic of GPU parallel processing. Kernel functions are initiated by the CPU but executed on the GPU, utilizing the GPU's stream multiprocessors and large number of threads to achieve parallel computation. The code of a kernel function is executed simultaneously by hundreds or thousands of threads, each processing different data. For example, when processing an image, a thread may be responsible for calculating the color of a single pixel. The logic for calculating a single pixel is defined in the kernel function, and the GPU automatically schedules a large number of threads to execute this logic in parallel. When a kernel function is launched, it needs to specify the thread organization mode (such as the dimensions of thread blocks and thread grids), and the GPU will allocate threads to different stream multiprocessors based on these parameters.

[0067] Thread: The smallest unit of execution scheduled by the operating system, a control flow within a process, with its own stack, program counter, and register state, but sharing the resources of the same process (such as memory, file handles, global variables, etc.).

[0068] Barrier Object: A barrier object is a synchronization mechanism commonly used in multi-threaded or concurrent programming to control the execution order of multiple threads or processes, ensuring that they are synchronized at a certain point. The main role of a barrier object is to make a group of threads "wait" at a certain point until all threads reach this point, and then allow them to continue execution.

[0069] WorkGroup: A group of threads composed of multiple threads, also known as a thread block (block). The threads in a single workgroup are generally used to perform the same pattern of processing on different data.

[0070] CWG (Cluster WorkGroup): A thread execution set composed of multiple workgroups (WG), used to uniformly schedule and cooperatively manage multiple workgroups at a higher level. A CWG includes several workgroups, each containing multiple thread bundles (Warp), used to parallel process different data elements but execute the same or similar computation logic. Multiple workgroups within the same CWG can share part of the hardware or software management resources or participate in the same communication and synchronization operations, such as reduction (Reduction), broadcast (Broadcast), etc.

[0071] SWG (Super WorkGroup): A higher-level set of parallel execution tasks consisting of multiple Cluster Workgroups (CWGs) used for hierarchical organization and unified scheduling of massively parallel computing tasks. An SWG includes multiple CWGs, and each CWG further contains multiple workgroups. SWGs are used to represent a group of cluster workgroups that have logical associations or communication needs within the same parallel computing task or the same grid, thereby enabling cooperative execution, synchronization control, or communication operations on a larger scale.

[0072] A grid consists of multiple workgroups. Each workgroup is a grid point in the grid, and each workgroup has corresponding grid point coordinates. A grid can be one-dimensional, two-dimensional, or three-dimensional. The shape of the grid is defined by the user during the programming process.

[0073] In current technology, for modern graphics processing units (GPUs), kernel functions are typically run by launching a large number of ordinary worker threads. However, these ordinary worker threads consume significant resources. Specifically, they are allocated dedicated general-purpose registers, and each active thread bundle occupies a "thread bundle slot," which contains resources such as the thread bundle scalar register, program counter, and thread bundle status register. When the kernel function does not perform element-wise operations, ordinary worker threads can become over-provisioned. For example, communication-type kernel functions and block-processing kernel functions do not perform element-wise operations.

[0074] Based on this, this disclosure proposes a meta-thread application method, chip system, electronic device, medium, and program product, aiming to solve the problem of excessive resource consumption.

[0075] System architecture description applied in the embodiments of this disclosure

[0076] In one embodiment, such as Figure 1 As shown, Figure 1 This is a system architecture diagram of the method for executing meta-thread applications according to an embodiment of the present disclosure. The system architecture includes, but is not limited to, a meta-thread instruction scheduler 100 and multiple functional units, all of which are communicatively connected to the meta-thread instruction scheduler 100.

[0077] The meta-thread instruction scheduler 100 is located within the computing core 1000. Specifically, after a meta-thread is created, it executes a meta-thread program. The meta-thread program consists of a sequence of meta-thread instructions. The computing core 1000 contains a meta-thread instruction scheduler 100. The meta-thread instruction scheduler 100 fetches, decodes, and dispatches each instruction in the meta-thread instruction sequence to the corresponding functional unit for execution.

[0078] In addition, the functional unit can be located within the computing core 1000 or outside the computing core 1000. Specifically, the functional unit can be a scalar operation unit 300, an execution unit 400, a tensor calculation unit 230, a data replication unit 220, or a communication operation unit 210, wherein the computing core 1000 also includes one or more scalar operation units 300, execution units 400, tensor calculation units 230, and data replication units 220. The communication operation unit 210 is outside the computing core 1000. The meta-thread instruction scheduler 100 fetches, decodes, and transmits instructions in the meta-thread instruction sequence to the corresponding functional units one by one for execution.

[0079] Specifically, the meta-thread instruction scheduler 100 can play a role of core coordination in a parallel computing system through a meta-thread, which can call one or more related functional units through a meta-thread to enable the functional units to perform operations related to their own functions.

[0080] The meta-thread instruction scheduler 100 can sequentially call multiple functional units in order through a meta-thread, or simultaneously call multiple functional units through a meta-thread, and the disclosure does not specifically limit the order and number of calls to the functional units.

[0081] In an embodiment, the computing core 1000 is a key component responsible for executing parallel computing tasks in a GPU, which includes multiple execution units 400, register files, shared memories, and other resources, and can handle multiple threads simultaneously to achieve efficient parallel computing.

[0082] It can be understood that the computing core 1000 described above can be a computing unit or a stream multiprocessor, and the disclosure does not specifically limit this.

[0083] In an embodiment, the functional unit described above can include a block operation unit, which can be a communication operation unit 210 for performing communication operations, a data replication unit 220 for performing data replication, a tensor calculation unit 230 for performing matrix operations, or other types of block operation units, and the disclosure does not specifically limit this.

[0084] Specifically, in the case of a communication operation unit 210, the meta-thread instruction scheduler 100 can call the communication operation unit 210 through a meta-thread to perform a broadcast operation. It can be understood that the broadcast operation described above refers to an operation of copying and sending data of one computing unit to all other computing units.

[0085] For example, in a GPU, a host can broadcast a convolution kernel parameter to all thread blocks, so that each thread block can use the same convolution kernel for calculation.

[0086] Furthermore, when the functional unit is the communication operation unit 210, the meta-thread instruction scheduler 100 can also invoke the communication operation unit 210 through the meta-thread to perform collection operations. It is understood that the aforementioned collection operation refers to the operation of aggregating scattered data from multiple computing units into a single target computing unit in a certain order.

[0087] For example, in a GPU, each thread block computes a sub-block of the image, and the collection operation aggregates the pixel data of all sub-blocks into global memory to reconstruct the complete image.

[0088] Furthermore, when the functional unit is the communication operation unit 210, the meta-thread instruction scheduler 100 can also call the communication operation unit 210 through the meta-thread to perform a reduction operation. It is understood that the reduction operation mentioned above refers to the operation of merging scattered data from multiple computing units into one or a few results through some operation. The operation type can be summation, product, finding the maximum value, or finding the minimum value, etc.

[0089] For example, in a GPU, multiple threads within a thread block each calculate a value, and the maximum value of these multiple values ​​is obtained by reduction and stored in the shared memory within the block.

[0090] Furthermore, when the functional unit is the data copying unit 220, the meta-thread instruction scheduler 100 can invoke the data copying unit 220 through meta-threads to execute data copying operations. It is understood that, regarding the aforementioned data copying operations, in the GPU, the data copying unit 220 is responsible for efficiently transferring data between different storage areas, reducing the burden on the computing core 1000. Its data copying operations mainly revolve around the GPU's internal storage hierarchy and interactions with external devices.

[0091] The types of data copying operations include, but are not limited to, the following: copying within global memory, copying between global memory and shared memory, copying between global memory and registers, copying between constant memory and global memory, data copying between GPU and CPU, data copying of texture memory, and data copying between multiple GPUs.

[0092] It is understood that the aforementioned data copying unit 220 can be either GDMA (GPU Direct Memory Access) or a DMA engine (Direct Memory Access Engine), and this disclosure does not specifically limit it in this regard.

[0093] In addition, in the case that the functional unit is the tensor calculation unit 230, the meta-thread instruction scheduler 100 can invoke the tensor calculation unit 230 through the meta-thread to perform a matrix operation. For example, the meta-thread instruction scheduler 100 can invoke the tensor calculation unit 230 through the meta-thread to perform a basic matrix multiplication, that is, to calculate the product of two matrices; or the meta-thread instruction scheduler 100 can invoke the tensor calculation unit 230 through the meta-thread to perform a matrix multiplication and addition, that is, to add an accumulation step on the basis of the matrix multiplication; or the meta-thread instruction scheduler 100 can invoke the tensor calculation unit 230 through the meta-thread to perform a mixed precision matrix operation, supporting matrix operations of multiple data precisions to balance the performance and accuracy requirements; or the meta-thread instruction scheduler 100 can invoke the tensor calculation unit 230 through the meta-thread to perform a batch matrix operation, that is, to perform parallel operations on a set of multiple independent matrices.

[0094] It can be understood that, regarding the above-mentioned tensor calculation unit 230, it can be a Tensor Core or a Matrix Core, and the present disclosure does not make specific limitations thereon.

[0095] In an embodiment, regarding the above-mentioned functional unit, it can further include a scalar operation unit 300, wherein the scalar operation unit 300 is configured to process the arithmetic and logical operations of scalar data.

[0096] Specifically, the scalar operation unit 300 is a core component in the meta-thread instruction scheduler 100 responsible for performing the arithmetic and logical operations of scalar data, and the meta-thread instruction scheduler 100 can invoke the scalar operation unit 300 through the meta-thread to perform the arithmetic and logical operations of scalar data.

[0097] For example, the meta-thread instruction scheduler 100 can invoke the scalar operation unit 300 through the meta-thread to perform arithmetic operations on numerical values, such as basic addition, subtraction, multiplication, and division, increment, decrement, negation, absolute value, and shift operation, etc.; or the meta-thread instruction scheduler 100 can invoke the scalar operation unit 300 through the meta-thread to perform logical operations, such as bitwise logical operations and comparison operations; or the meta-thread instruction scheduler 100 can invoke the scalar operation unit 300 through the meta-thread to perform other auxiliary operations, such as modulus operation, address calculation, and flag setting.

[0098] In an embodiment, regarding the above-mentioned functional unit, it can further include an execution unit 400, wherein the execution unit 400 is configured to process vector operations.

[0099] Specifically, the execution unit 400 is a hardware component specially optimized for vector operations, and the core function is to perform parallel processing on vector data, and a single operation can process multiple elements at the same time.

[0100] Exemplarily, the meta-thread instruction scheduler 100 can call the execution unit 400 through a meta-thread to perform a vector basic arithmetic operation, such as vector addition, vector subtraction, vector multiplication, and vector division; or the meta-thread instruction scheduler 100 can call the execution unit 400 through a meta-thread to perform a vector scalar operation, such as vector plus scalar and vector multiply scalar; or the meta-thread instruction scheduler 100 can call the execution unit 400 through a meta-thread to perform a vector special operation, such as vector dot product, vector maximum / minimum extraction, vector sum / average, and vector comparison; or the meta-thread instruction scheduler 100 can call the execution unit 400 through a meta-thread to perform a vector data reorganization operation, such as vector shuffling and vector type conversion.

[0101] In an embodiment, as shown in FIG. 1, the core coordination role of the meta-thread in the parallel computing system is shown: Figure 1

[0102] Communication operation management: the computing core 1000 interacts with the communication operation modules on the left side, such as reduction and broadcast. For example, cluster work group (CWG), super work group (SWG), and grid.

[0103] Interaction with scalar operation unit: the meta-thread instruction scheduler 100 calls the scalar operation unit 300 (Scalar ALU) through a meta-thread, so as to process the arithmetic logic operation of scalar data.

[0104] Execution unit 400 call: the meta-thread instruction scheduler 100 calls the execution unit 400 through an asynchronous request and response mechanism through a meta-thread, so as to schedule the execution of the computing task.

[0105] Matrix operation and data replication: the meta-thread instruction scheduler 100 coordinates the matrix multiplication and addition operation and the data replication operation through a meta-thread, so as to guarantee the transmission of data in the computing flow and the execution of complex operation.

[0106] Therefore, as can be seen from the above, the meta-thread is the scheduling center of various computing and communication operations, so that different tasks in parallel computing, such as scalar operation, matrix operation, data communication, and execution unit 400 call, can be efficiently coordinated. Figure 1

[0107] In addition, in an embodiment, the communication kernel function, the block processing computing kernel function, and the element processing computing kernel function can be run in parallel, and no significant resource conflict will be caused.

[0108] ​​In addition, in an embodiment, when the demand is insufficient, the over-allocation of resource-intensive common work threads can be avoided. The element-wise allocated resources such as vector general registers are saved. The metathread is an efficient block-wise processing method, framework and paradigm suitable for future parallel computing systems.

[0109] In an embodiment, as shown in Figure 2 , Figure 2 is a schematic diagram of the relationship of the metathread, the common work thread, the thread bundle, the thread block and the thread cluster of an embodiment of the present disclosure; compared with the prior art, the schematic diagram of the present disclosure adds a metathread for management, wherein, as Figure 2 can be seen, a thread cluster includes a plurality of thread blocks, a thread block includes a plurality of thread bundles and a metathread, and a thread bundle includes a plurality of common work threads.

[0110] Overall implementation of the metathread application method of the embodiment of the present disclosure

[0111] The embodiment of the present disclosure proposes a metathread application method applied to a chip system, the chip system including a metathread instruction scheduler and a plurality of functional units, and the plurality of functional units are in communication connection with the metathread instruction scheduler; referring to Figure 3 , Figure 3 is a main flowchart of the metathread application method of an embodiment of the present disclosure; the metathread application method includes but is not limited to step S301, step S302 and step S303.

[0112] Step S301, based on the metathread instruction scheduler receiving a kernel function start instruction, generating a metathread corresponding to the kernel function start instruction;

[0113] Step S302, determining the thread task type of the metathread through the metathread instruction scheduler, and determining the corresponding target functional unit according to the thread task type, wherein the target functional unit is one or more of the plurality of functional units;

[0114] Step S303, sending the metathread to the target functional unit through the metathread instruction scheduler, and executing the metathread through the target functional unit.

[0115] Specifically, the present disclosure proposes a scheme of adding a metathread, and the corresponding target functional unit can be called according to the thread task type of the metathread, without starting a large number of common work threads through the execution unit each time, thereby avoiding the over-occupation of the resources of the execution unit, and solving the problem of large resource occupation when the demand is insufficient.

[0116] In one embodiment, step S301 specifically involves the following steps: When the CPU-side code executes the kernel function startup instruction, the request is first passed to the GPU driver running on the CPU. The driver parses the kernel function's configuration parameters and converts them into a command format understandable by the GPU hardware. Then, the driver sends the formatted kernel function startup instruction to the command processor in the GPU hardware via the bus. The command processor, as one of the GPU's control centers, is responsible for receiving all instructions from the CPU and placing them into the GPU's command queue for sequential execution. Next, after processing the kernel function startup instruction, the command processor allocates thread blocks to the GPU's meta-thread instruction scheduler based on the thread configuration. The meta-thread instruction scheduler is responsible for actually creating threads, scheduling thread bundles, and executing the kernel function code. Finally, the meta-thread instruction scheduler generates a meta-thread corresponding to the kernel function startup instruction.

[0117] In one embodiment, step S302 specifically involves: the meta-thread instruction scheduler also obtaining the thread task type of the meta-thread, for example, obtaining whether the thread task type of the meta-thread is a communication task type or a computation task type; then, the meta-thread instruction scheduler determines different target functional units based on different thread task types. For example, if the thread task type is a communication task type, the target functional unit is the functional unit responsible for communication; if the thread task type is a computation task type, the target functional unit is the functional unit responsible for computation.

[0118] The target functional unit can be one of multiple functional units, or multiple functional units; this disclosure does not specifically limit this.

[0119] In one embodiment, step S303 specifically involves the meta-thread instruction scheduler sending the meta-thread to the target functional unit. For example, if the thread task type is a communication task type, the meta-thread instruction scheduler sends the meta-thread to the functional unit responsible for communication; if the thread task type is a computation task type, the meta-thread instruction scheduler sends the meta-thread to the functional unit responsible for computation.

[0120] In addition, such as Figure 4 As shown, Figure 4 This is a flowchart of a meta-thread application method according to an embodiment of the present disclosure when the thread task type is a block operation task, including but not limited to steps S401, S402 and S403.

[0121] Step S401: Receive the kernel function start instruction based on the meta-thread instruction scheduler and generate a meta-thread corresponding to the kernel function start instruction;

[0122] Step S402: Determine the thread task type of the meta-thread as a block operation task through the meta-thread instruction scheduler, and determine the target functional unit as a block operation unit based on the block operation task.

[0123] Step S403: Send the meta-thread to the block operation unit through the meta-thread instruction scheduler, and execute the meta-thread through the block operation unit.

[0124] In one embodiment, after the command processor receives the kernel function start instruction, it forwards it to the GPU's meta-thread instruction scheduler. Then, after receiving the kernel function start instruction and related configuration information, the meta-thread instruction scheduler generates a meta-thread corresponding to the kernel function start instruction. Next, if the thread task type of the meta-thread is a block operation task, the meta-thread instruction scheduler sends the meta-thread to the block operation unit. Finally, the block operation unit executes the operation task corresponding to the meta-thread. After the block operation unit completes its operation, it sends a task completion instruction back to the meta-thread instruction scheduler.

[0125] In addition, such as Figure 5 As shown, Figure 5 yes Figure 4 A flowchart of a sub-step of one embodiment of step S402; step S402 may include, but is not limited to, steps S501 and S502.

[0126] Step S501: Determine the thread task type of the meta-thread as a block operation task through the meta-thread instruction scheduler;

[0127] Step S502: When the block operation task is a communication operation task, determine the block operation unit as a communication operation unit.

[0128] In one embodiment, if the thread task type of the meta-thread is a communication operation task, the meta-thread instruction scheduler will send the meta-thread to the communication operation unit, and then the communication operation unit will execute the communication task corresponding to the meta-thread. After the communication operation unit completes its operation, it will send a task completion instruction back to the meta-thread instruction scheduler.

[0129] Furthermore, when the functional unit is a communication operation unit, the meta-thread instruction scheduler can invoke the communication operation unit through meta-threads to perform broadcast operations. It is understood that the aforementioned broadcast operation refers to the operation of copying data from one computing unit and sending it to all other computing units.

[0130] In addition, the meta-thread instruction scheduler can also invoke communication operation units through meta-threads to perform collection operations. It can be understood that the aforementioned collection operation refers to the operation of aggregating scattered data from multiple computing units into a single target computing unit in a certain order.

[0131] In addition, the meta-thread instruction scheduler can also invoke the communication operation unit through the meta-thread to perform a reduction operation. It can be understood that the reduction operation described above refers to an operation of combining scattered data in multiple computing units into one or a small amount of results through certain operations, and the operation type can be summation, multiplication, maximum value taking, or minimum value taking, etc.

[0132] In addition, as shown in Figure 6 , Figure 6 is Figure 4 a sub-step flowchart of another embodiment of step S402. Regarding the above-mentioned step S402, it can include but is not limited to step S601 and step S602.

[0133] Step S601, determining, by the meta-thread instruction scheduler, that the thread task type of the meta-thread is a block operation task.

[0134] Step S602, when the block operation task is a data replication task, determining that the block operation unit is a data replication unit.

[0135] In an embodiment, if the thread task type of the meta-thread is a data replication task, the meta-thread instruction scheduler will send the meta-thread to the data replication unit, and finally the data replication task corresponding to the meta-thread will be executed by the data replication unit. After the operation of the data replication unit is completed, a task completion instruction will be fed back to the meta-thread instruction scheduler.

[0136] In addition, in the case of a data replication unit as the functional unit, the meta-thread instruction scheduler can invoke the data replication unit through the meta-thread to perform a data replication operation. It can be understood that the data replication operation described above is that, in the GPU, the data replication unit is responsible for efficiently transferring data between different storage areas, reducing the burden of the meta-thread instruction scheduler, and the data replication operation performed by the data replication unit mainly revolves around the internal storage hierarchy of the GPU and the interaction with external devices.

[0137] The type of the data replication operation includes but is not limited to the following: global memory internal replication, replication between global memory and shared memory, replication between global memory and register, replication between constant memory and global memory, data replication between GPU and CPU, data replication of texture memory, and data replication between multiple GPUs.

[0138] It can be understood that the data replication unit described above can be a GDMA or a DMA engine, and the present disclosure does not make specific limitations thereto.

[0139] In addition, as shown in Figure 7 , Figure 7 is Figure 4A flowchart of a sub-step of another embodiment of step S402; regarding the above step S402, it may include, but is not limited to, steps S701 and S702.

[0140] Step S701: Determine the thread task type of the meta-thread as a block operation task through the meta-thread instruction scheduler;

[0141] Step S702: When the block operation task is a matrix operation task, determine the block operation unit as a tensor calculation unit.

[0142] In one embodiment, if the thread task type of the meta-thread is a matrix operation task, the meta-thread instruction scheduler will send the meta-thread to the tensor computation unit, and then the tensor computation unit will execute the matrix operation task corresponding to the meta-thread. After the tensor computation unit completes the operation, it will send a task completion instruction back to the meta-thread instruction scheduler.

[0143] When the functional unit is a tensor computation unit, the meta-thread instruction scheduler can invoke the tensor computation unit through a meta-thread to perform matrix operations. For example, the meta-thread instruction scheduler can invoke the tensor computation unit through a meta-thread to perform basic matrix multiplication, i.e., calculate the product of two matrices; or, the meta-thread instruction scheduler can invoke the tensor computation unit through a meta-thread to perform matrix multiplication and addition, i.e., add an accumulation step to matrix multiplication; or, the meta-thread instruction scheduler can invoke the tensor computation unit through a meta-thread to perform mixed-precision matrix operations, supporting matrix operations of various data precisions to balance performance and accuracy requirements; or, the meta-thread instruction scheduler can invoke the tensor computation unit through a meta-thread to perform batch matrix operations, i.e., perform parallel operations on a set of multiple independent matrices.

[0144] It is understood that the tensor computation unit mentioned above can be either a Tensor Core or a Matrix Core, and this disclosure does not make any specific limitation on it.

[0145] In addition, such as Figure 8 As shown, Figure 8 This is a flowchart of a meta-thread application method in the case of a scalar operation task according to an embodiment of the present disclosure, including but not limited to steps S801, S802 and S803.

[0146] Step S801: Receive the kernel function start instruction based on the meta-thread instruction scheduler and generate a meta-thread corresponding to the kernel function start instruction;

[0147] Step S802: Determine the thread task type of the meta-thread as a scalar operation task through the meta-thread instruction scheduler, and determine the target functional unit as a scalar operation unit based on the scalar operation task.

[0148] Step S803: Send the meta-thread to the scalar arithmetic unit through the meta-thread instruction scheduler, and execute the meta-thread through the scalar arithmetic unit.

[0149] In one embodiment, after receiving the kernel function start instruction, the command processor forwards it to the GPU's meta-thread instruction scheduler. Then, after receiving the kernel function start instruction and related configuration information, the meta-thread instruction scheduler generates a meta-thread corresponding to the kernel function start instruction. Next, if the thread task type of the meta-thread is a scalar operation task, the meta-thread instruction scheduler sends the meta-thread to the scalar operation unit. Finally, the scalar operation unit executes the operation task corresponding to the meta-thread. After the scalar operation unit completes its operation, it sends a task completion instruction back to the meta-thread instruction scheduler.

[0150] Specifically, the scalar arithmetic unit is the core component in the meta-thread instruction scheduler responsible for performing arithmetic and logical operations on scalar data. The meta-thread instruction scheduler can use meta-threads to call the scalar arithmetic unit to perform arithmetic and logical operations on scalar data.

[0151] For example, the meta-thread instruction scheduler can use meta-threads to call scalar arithmetic units to perform arithmetic operations on numerical values, such as basic addition, subtraction, multiplication, division, increment, decrement, inversion, absolute value, shift operations, etc.; or, the meta-thread instruction scheduler can use meta-threads to call scalar arithmetic units to perform logical operations, such as bitwise logical operations and comparison operations; or, the meta-thread instruction scheduler can use meta-threads to call scalar arithmetic units to perform other auxiliary operations, such as modulo operations, address calculations, and flag setting.

[0152] In addition, such as Figure 9 As shown, Figure 9 This is a flowchart of a meta-thread application method in the case of a vector operation task according to an embodiment of the present disclosure, including but not limited to steps S901, S902 and S903.

[0153] Step S901: Receive the kernel function start instruction based on the meta-thread instruction scheduler and generate a meta-thread corresponding to the kernel function start instruction;

[0154] Step S902: Determine the thread task type of the meta-thread as a vector operation task through the meta-thread instruction scheduler, and determine the target functional unit as the execution unit based on the vector operation task;

[0155] Step S903: Send the meta-thread to the execution unit through the meta-thread instruction scheduler, and execute the meta-thread through the execution unit.

[0156] In an embodiment, after the command processor receives the kernel launch instruction, the kernel launch instruction is forwarded to the meta-thread instruction scheduler of the GPU. Then, after the meta-thread instruction scheduler receives the kernel launch instruction and the related configuration information, the meta-thread instruction scheduler generates a meta-thread corresponding to the kernel launch instruction. Next, if the thread task type of the meta-thread is a vector operation task, the meta-thread instruction scheduler sends the meta-thread to the execution unit. Finally, the execution unit executes the operation task corresponding to the meta-thread, and feeds back a task completion instruction to the meta-thread instruction scheduler after the operation of the execution unit is completed.

[0157] Specifically, the execution unit is a hardware component specially optimized for vector operations, and the core function of the execution unit is to perform parallel processing on vector data, and a single operation can process multiple elements at the same time.

[0158] For example, the meta-thread instruction scheduler can use a meta-thread to call the execution unit to perform vector basic arithmetic operations, such as vector addition, vector subtraction, vector multiplication, and vector division. Alternatively, the meta-thread instruction scheduler can use a meta-thread to call the execution unit to perform vector scalar operations, such as vector plus scalar and vector multiply scalar. Alternatively, the meta-thread instruction scheduler can use a meta-thread to call the execution unit to perform vector special operations, such as vector dot product, vector maximum / minimum extraction, vector sum / average, and vector comparison. Alternatively, the meta-thread instruction scheduler can use a meta-thread to call the execution unit to perform vector data reorganization operations, such as vector shuffling and vector type conversion.

[0159] In addition, as shown in Figure 10 , Figure 10 is Figure 9 a sub-step flowchart of one embodiment of step S903; regarding the above-mentioned step S903, it can include but is not limited to step S1001 and step S1002.

[0160] Step S1001, sending a meta-thread carrying thread configuration information of a normal worker thread to the execution unit by the meta-thread instruction scheduler;

[0161] Step S1002, managing the normal worker thread according to the thread configuration information by the execution unit.

[0162] Specifically, the metathread instruction scheduler first generates metathreads according to task requirements such as parallel logic of kernel functions or computing tasks. The metathread itself is not a work thread that performs specific computing, but a management carrier. The core content of the metathread is thread configuration information of an ordinary work thread. Then, the metathread instruction scheduler sends the metathread encapsulating the thread configuration information to an execution unit. That is, the metathread instruction scheduler explicitly indicates the number of thread starts, the organization mode of threads, and the operation type to be performed by the threads to the execution unit through the metathread. Next, after the execution unit receives the metathread, it first parses the thread configuration information in the metathread to explicitly indicate the organization rules and resource requirements of the ordinary work threads, for example, allocating hardware resources according to the configuration information, generating a unique identifier for each ordinary work thread according to an ID rule to ensure that the thread can correctly access corresponding data, loading an instruction sequence to be executed by the ordinary thread into an instruction cache, and preparing for scheduling and execution. Finally, based on the parsed configuration, the execution unit starts and manages the ordinary work threads.

[0163] In an embodiment, the thread configuration information generally includes but is not limited to the following information: the total number of ordinary work threads, grouping modes such as the number of work groups and the number of threads included in each group; a global ID / local ID mapping rule of the threads; resource allocation information required by the threads such as the size of registers and shared memory; and a task instruction entry address, that is, an instruction sequence to be executed by the ordinary thread.

[0164] In an embodiment, the step S1002 of managing the ordinary work threads by the execution unit according to the thread configuration information can specifically include but is not limited to the following two implementation cases.

[0165] The first implementation case is to wake up the ordinary work thread by the execution unit. Specifically, when the execution unit wakes up the ordinary work thread, it first monitors the wake-up trigger condition, for example, shared memory data that the thread is waiting for is ready, a synchronization barrier meets the requirement that all threads arrive, or an initial task scheduling needs to start an unexecuted thread. Then, the target thread is located according to the thread ID and work group ID in the trigger signal, and it is checked whether it is currently in a dormant / suspended state and whether the required hardware resources such as registers and instruction caches are still reserved. If the thread has released temporary resources by sleeping before, the execution unit recovers the key information from the thread context cache, including the program counter position for continuous execution, the temporary computing data in the registers, and the thread ID mapping and data access permission configuration. Finally, the ordinary work thread with the recovered context is added to the corresponding execution queue, waiting for allocation to the metathread instruction scheduler to execute subsequent instructions and complete the entire wake-up process.

[0166] The second implementation case is that the execution unit terminates the normal work thread. Specifically, when the execution unit terminates the normal work thread, the termination trigger condition is first monitored, such as the thread completing task execution to an exit instruction, an error trigger interrupt occurring, or an upper task canceling requiring forced resource recovery. Then, the target thread is located according to the thread ID in the termination signal or the error source, and the current instruction execution of the target thread is immediately stopped to freeze the program counter. If the thread is in a synchronization waiting state, the placeholder of the thread in the queue is synchronously removed. Subsequently, the hardware resources occupied by the thread are released, including the exclusive register space, the temporary data in the shared memory, and the ID mapping and access permission configuration are unregistered. Finally, the thread state is marked as terminated, the result storage address is recorded if the termination is normal, and an error report is sent to the meta-thread instruction scheduler if the termination is abnormal, and the entire termination process is completed.

[0167] In an embodiment, the present disclosure further includes one of the following: one thread block corresponds to one meta-thread, one work group corresponds to one meta-thread, and one Trition grid point corresponds to one meta-thread. Specifically, the present disclosure can be responsible for managing the working state of the normal work thread in one thread block, one work group, or one Trition grid point through one meta-thread.

[0168] In an embodiment, the present disclosure further includes one of the following: one meta-thread corresponds to a group of state registers, a group of scalar registers, and a program counter.

[0169] In an embodiment, regarding the generation of the meta-thread corresponding to the kernel function start instruction in the above step S401, the following two cases can be included:

[0170] The first case is that for the same kernel function, multiple meta-threads are generated by the meta-thread instruction scheduler, wherein one meta-thread corresponds to one work group in the kernel function. Specifically, the meta-threads of multiple work groups of the same kernel function can be run in parallel in the same computing unit.

[0171] The second case is that for multiple different kernel functions, multiple meta-threads are generated by the meta-thread instruction scheduler, wherein one meta-thread corresponds to one kernel function. Specifically, the meta-threads of different kernel functions can be run in parallel in the same computing unit.

[0172] In an embodiment, the meta-thread can be implemented by hardware or firmware.

[0173] In one embodiment, the resources related to a meta-thread include the following: 1) each meta-thread has its own minimum set of scalar general-purpose registers. Additional scalar general-purpose registers can be dynamically allocated from a pool. 2) each meta-thread has its own minimum set of barrier objects. Additional barrier objects can be dynamically allocated from a pool. 3) each meta-thread has its own program counter. Each meta-thread has its own state register.

[0174] In addition, as shown in Figure 11 , Figure 11 the flowchart of executing a meta-thread according to the case of dependency relationship for one embodiment of the present disclosure can include but is not limited to step S1101 and step S1102.

[0175] Step S1101, obtaining the dependency relationship of the meta-thread;

[0176] Step S1102, when the dependency relationship is resolved, continue to execute the meta-thread by the target functional unit.

[0177] Specifically, when processing a meta-thread, first, the specific dependency relationship of the meta-thread needs to be obtained, such as synchronization dependency, scalar load delay, and resource complete occupation; then, these dependency types are checked and verified one by one: for synchronization dependency, it is confirmed that all associated meta-threads have reached the barrier point and meet the synchronization condition; for scalar load delay, it is checked whether the loading operation of the scalar data has been completed and the data is available; for resource complete occupation, it is checked whether the corresponding hardware resource has been in a usable state; when all dependency relationships are confirmed to be resolved, according to the thread task type of the meta-thread, it is scheduled to the corresponding target functional unit, and the specific operation carried by the meta-thread is continued to be executed by the target functional unit, to ensure that the task is orderly promoted after the dependency condition is met.

[0178] In one embodiment, in the case of the dependency relationship being resolved, the communication operation carried by the meta-thread is continued to be executed by the communication operation unit, or the data replication operation carried by the meta-thread is continued to be executed by the data replication unit, or the matrix operation carried by the meta-thread is continued to be executed by the tensor calculation unit, or the scalar operation carried by the meta-thread is continued to be executed by the scalar operation unit, or the vector operation carried by the meta-thread is continued to be executed by the execution unit.

[0179] It should be noted that the types of the above-mentioned dependency relationship can include the following:

[0180] The first type is the synchronous dependency type. For example, when the same workgroup performs image processing convolution calculations, all meta-threads need to complete the preliminary calculation of local pixels first, and then rely on the barrier synchronization to wait for all meta-threads in the group to reach the barrier point, so as to avoid the loss of intermediate results due to some threads not completing. Another example is in deep learning inter-layer calculations, the parent meta-thread that assigns the task needs to rely on parent-child thread synchronization, and wait for all child meta-threads that perform convolution operations to complete the calculation and return the results before the subsequent inter-layer feature fusion can be started.

[0181] The second type is the scalar loading delay type. For example, in deep learning inference, it is necessary to load globally constant parameters such as the size and stride of the convolution kernel. In this case, the metathread will become dependent due to the scalar loading delay and will have to wait for the parameters to be loaded from the scalar cache or memory into the register and to complete the verification, so as to avoid subsequent matrix multiplication instructions from being executed incorrectly due to the parameters not being ready. Another example is in graphics rendering, it is necessary to load the offset constant of texture sampling. This scalar data needs to be read from memory through the scalar path. The metathread will be in a waiting state until the loading is completed to ensure the accuracy of the sampling position calculation.

[0182] The third type: resource occupancy type; for example, the first-in-first-out queue is full, or hardware resources such as the tensor processing engine, copying engine, and vector engine are occupied by other tasks.

[0183] In addition, such as Figure 12 As shown, Figure 12 The flowchart for executing a second meta-thread during the first meta-thread's shutdown period, as an embodiment of this disclosure, may include, but is not limited to, steps S1201 and S1202.

[0184] Step S1201: If a first meta-thread and a second meta-thread exist, obtain the first dependency relationship of the first meta-thread and the second dependency relationship of the second meta-thread.

[0185] Step S1202: When the first dependency is not resolved and the second dependency is resolved, the first meta-thread is stopped, and the second meta-thread is executed within the shutdown time slot of the first meta-thread.

[0186] Specifically, when the first meta-thread and the second meta-thread exist in the system at the same time, the dependency relationship of the two is first acquired respectively, wherein the first dependency relationship of the first meta-thread can include an unsatisfied barrier synchronization, an unfinished scalar load or a resource such as a tensor processing engine being occupied, and the second dependency relationship of the second meta-thread has confirmed that all conditions have been solved, for example, a required synchronization point has been reached, a scalar data load is completed, and a target functional unit resource is idle; at this time, since the first meta-thread cannot be promoted temporarily due to the unresolved dependency, the system will disable it, that is, suspend its instruction scheduling, and reserve the context information to avoid state loss, and at the same time, the second meta-thread is scheduled to the corresponding functional unit for execution by using the time gap released by the first meta-thread, that is, the idle window of the functional unit, register and other resources originally allocated to the first meta-thread. Through this resource reuse mechanism, the utilization rate of hardware resources is improved, and the efficiency loss caused by dependency waiting is reduced without affecting the subsequent resumption of execution of the first meta-thread.

[0187] It should be noted that a plurality of independent meta-threads can run in parallel in the same meta-thread instruction scheduler.

[0188] In addition, it can be understood that in addition to the first meta-thread and the second meta-thread mentioned above, there can be more meta-threads, such as a third meta-thread and the like.

[0189] Based on the meta-thread application method of the above embodiment, the following proposes various embodiments of an electronic device, a chip system, a computer readable storage medium and a computer program product of the present disclosure.

[0190] The embodiment of the present disclosure further provides an electronic device 500, comprising:

[0191] at least one processor, and

[0192] a memory in communication connection with the at least one processor; wherein

[0193] The memory stores instructions, and the instructions are executed by the at least one processor to cause the at least one processor to implement the method of any one of the above embodiments of the present disclosure when the instructions are executed.

[0194] The following will be combined Figure 13 The hardware structure of the electronic device is described in detail. The electronic device 500 comprises a processor 510, a memory 520, an input / output interface 530, a communication interface 540 and a bus 550.

[0195] The processor 510 can be implemented by a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present disclosure.

[0196] The memory 520 can be implemented by a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 520 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present disclosure are implemented by software or firmware, the related program codes are stored in the memory 520 and are called and executed by the processor 510 to implement the meta-thread application method of the embodiments of the present disclosure.

[0197] The input / output interface 530 is configured to implement information input and output.

[0198] The communication interface 540 is configured to implement the communication interaction between the device and other devices. The communication can be implemented by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0199] The bus 550 is configured to transmit information between the components (for example, the processor 510, the memory 520, the input / output interface 530, and the communication interface 540) of the device.

[0200] The processor 510, the memory 520, the input / output interface 530, and the communication interface 540 are connected to each other through the bus 550 to realize the communication connection between the components in the device.

[0201] It is worth noting that since the electronic device 500 of the present disclosure can execute the meta-thread application method of any of the above embodiments, the specific implementation and technical effects of the electronic device 500 of the present disclosure can refer to the specific implementation and technical effects of the meta-thread application method of any of the above embodiments.

[0202] Further, an embodiment of the present disclosure provides a chip system, comprising a meta-thread instruction scheduler and a plurality of functional units, wherein the plurality of functional units are in communication connection with the meta-thread instruction scheduler; the meta-thread instruction scheduler is configured to receive a kernel function starting instruction, generate a meta-thread corresponding to the kernel function starting instruction, and send the meta-thread to a target functional unit according to a thread task type of the meta-thread; and the target functional unit is configured to execute the meta-thread, wherein the target functional unit is one or more of the plurality of functional units.

[0203] It is worth noting that, since the chip system of the present disclosure comprises the above-mentioned meta-thread instruction scheduler, and the meta-thread instruction scheduler and the functional units can execute the meta-thread application method of any of the above-mentioned embodiments, the specific implementation and technical effects of the chip system of the present disclosure can refer to the specific implementation and technical effects of the meta-thread application method of any of the above-mentioned embodiments.

[0204] Further, an embodiment of the present disclosure provides a computer readable storage medium storing computer executable instructions for executing the above-mentioned meta-thread application method. For example, the above-mentioned method steps in the above-mentioned Figures 3 to 12 are executed.

[0205] It is worth noting that, since the computer readable storage medium of the present disclosure can execute the meta-thread application method of any of the above-mentioned embodiments, the specific implementation and technical effects of the computer readable storage medium of the present disclosure can refer to the specific implementation and technical effects of the meta-thread application method of any of the above-mentioned embodiments.

[0206] Further, an embodiment of the present disclosure provides a computer program product comprising computer programs or computer instructions stored in a computer readable storage medium, wherein a processor of a computer device reads the computer programs or computer instructions from the computer readable storage medium, and the processor executes the computer programs or computer instructions to make the computer device execute the above-mentioned meta-thread application method. For example, the above-mentioned method steps in the above-mentioned Figures 3 to 12 are executed.

[0207] It is worth noting that, since the computer program product of the present disclosure can execute the meta-thread application method of any of the above-mentioned embodiments, the specific implementation and technical effects of the computer program product of the present disclosure can refer to the specific implementation and technical effects of the meta-thread application method of any of the above-mentioned embodiments.

[0208] Those of ordinary skill in the art will appreciate that all or certain steps, systems of the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Certain physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on computer readable media, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those of ordinary skill in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, it should be appreciated by those skilled in the art that computer storage media generally includes computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

[0209] The terms "first", "second", "third", "fourth" etc. (if any) in the description and the drawings of the present disclosure are used for distinguishing between similar objects, not necessarily described in a particular order or sequence. It should be understood that the data thus described can be interchanged, where appropriate, so that the embodiments of the present disclosure described herein could be carried out in other than the order described or illustrated herein. Furthermore, the terms "comprise" and "include" and their conjugates, as used herein, are intended to be open-ended, i.e., to mean including, but not limited to, for example, the stated elements or steps.

[0210] It should be understood that in the disclosure, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can mean: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0211] It should be understood that in the description of the embodiments of the disclosure, the meaning of multiple (or multiple) is two or more, greater than, less than, more than, etc. Understand that the number is not included, above, below, etc. Understand that the number is included.

[0212] In several embodiments provided by the disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division, and actual implementation can have another division manner. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other form.

[0213] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment.

[0214] In addition, the functional units in each embodiment of the disclosure can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware, or in the form of software functional unit.

[0215] It should also be understood that the various embodiments provided by the disclosure can be combined in any way to achieve different technical effects.

[0216] The above is a specific explanation of the embodiments of the present disclosure, but the present disclosure is not limited to the above-described embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present disclosure, and these equivalent modifications or substitutions are included in the scope defined by the claims of the present disclosure.

Claims

1. A method for applying meta-threads, characterized in that, The chip system is applied to a chip system, which includes a meta-thread instruction scheduler and multiple functional units, all of which are communicatively connected to the meta-thread instruction scheduler. The meta-thread application method includes: Based on the received kernel function start instruction by the meta-thread instruction scheduler, a meta-thread corresponding to the kernel function start instruction is generated. The meta-thread instruction scheduler determines the thread task type of the meta-thread, and determines the corresponding target functional unit based on the thread task type, wherein the target functional unit is one or more of the multiple functional units; The meta-thread is sent to the target functional unit through the meta-thread instruction scheduler, and the meta-thread is executed by the target functional unit. The step of determining the corresponding target functional unit based on the thread task type includes one of the following: When the thread task type is a block operation task, the target functional unit is determined to be a block operation unit; When the thread task type is a scalar operation task, the target functional unit is determined to be a scalar operation unit; When the thread task type is a vector operation task, the target functional unit is determined to be an execution unit.

2. The meta-thread application method according to claim 1, characterized in that, The determination that the target functional unit is a block operation unit includes one of the following: When the block operation task is a communication operation task, the block operation unit is determined to be a communication operation unit; When the block operation task is a data copying task, the block operation unit is determined to be a data copying unit; When the block operation task is a matrix operation task, the block operation unit is determined to be a tensor computation unit.

3. The meta-thread application method according to claim 1, characterized in that, The step of sending the meta-thread to the target functional unit through the meta-thread instruction scheduler, and executing the meta-thread through the target functional unit, includes: When the target functional unit is an execution unit, the meta-thread carrying the thread configuration information of the ordinary worker thread is sent to the execution unit through the meta-thread instruction scheduler; The execution unit manages the ordinary worker threads according to the thread configuration information.

4. The meta-thread application method according to claim 3, characterized in that, The management of the ordinary worker thread by the execution unit according to the thread configuration information includes one of the following: The execution unit wakes up the normal working thread; The ordinary working thread is terminated by the execution unit.

5. The meta-thread application method according to claim 1, characterized in that, Including one of the following: One thread block corresponds to one meta-thread; One workgroup corresponds to one meta-thread; One Trig grid point corresponds to one meta-thread.

6. The meta-thread application method according to claim 1, characterized in that, Each of the aforementioned metathreads corresponds to a set of status registers, a set of scalar registers, and a program counter.

7. The meta-thread application method according to claim 1, characterized in that, The generation of the meta-thread corresponding to the kernel function startup instruction includes one of the following: For the same kernel function, multiple meta-threads are generated by the meta-thread instruction scheduler, wherein one meta-thread corresponds to a workgroup in the kernel function; For multiple different kernel functions, multiple meta-threads are generated by the meta-thread instruction scheduler, wherein one meta-thread corresponds to one kernel function.

8. The meta-thread application method according to claim 1, characterized in that, The meta-thread instruction scheduler is located in the computing core, which includes one of the following: computing unit, stream multiprocessor.

9. The meta-thread application method according to claim 1, characterized in that, Also includes: Obtain the dependencies of the meta-thread; Once the dependency has been resolved, the meta-thread continues to be executed through the target functional unit.

10. The meta-thread application method according to claim 1, characterized in that, Also includes: In the presence of a first meta-thread and a second meta-thread, obtain the first dependency relationship of the first meta-thread and the second dependency relationship of the second meta-thread; When the first dependency is not resolved and the second dependency is resolved, the first meta-thread is stopped, and the second meta-thread is executed within the shutdown time slot of the first meta-thread.

11. The meta-thread application method according to claim 9 or 10, characterized in that, The dependency types include one of the following: synchronous dependency type, scalar load lazy type, and resource consumption type.

12. A chip system, characterized in that, The system includes a meta-thread instruction scheduler and multiple functional units, all of which are communicatively connected to the meta-thread instruction scheduler. The meta-thread instruction scheduler receives kernel function startup instructions, generates a meta-thread corresponding to the kernel function startup instructions, and sends the meta-thread to a target functional unit according to the thread task type of the meta-thread. The target functional unit executes the meta-thread, and the target functional unit is one or more of the multiple functional units. This includes one of the following: When the thread task type is a block operation task, the target functional unit is a block operation unit; When the thread task type is a scalar operation task, the target functional unit is a scalar operation unit; When the thread task type is a vector operation task, the target functional unit is an execution unit.

13. An electronic device, characterized in that, The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. The program is executed by the processor to implement the meta-thread application method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the meta-thread application method as described in any one of claims 1 to 11.

15. A computer program product, comprising a computer program or computer instructions, characterized in that, The computer program or the computer instructions are stored in a computer-readable storage medium, and the processor of the computer device reads the computer program or the computer instructions from the computer-readable storage medium. The processor executes the computer program or the computer instructions, causing the computer device to perform the meta-thread application method as described in any one of claims 1 to 11.

Images