Application method, chip system, electronic device and medium of metathread family

Abstract

The present disclosure provides an application method of a meta-thread family, a chip system, an electronic device and a medium, comprising: a command processor receiving a start instruction corresponding to a kernel function, generating a thread block configuration instruction according to the start instruction, and sending the thread block configuration instruction to a meta-thread instruction scheduler; the meta-thread instruction scheduler configures a thread block according to the thread block configuration instruction; for each thread block, the meta-thread instruction scheduler generates a meta-thread family corresponding to the thread block, the meta-thread family comprising a plurality of member meta-threads, and one member meta-thread corresponding to one member function in the kernel function; the meta-thread instruction scheduler runs a plurality of member functions in parallel through a plurality of member meta-threads. In one thread block, the present disclosure sets a plurality of member meta-threads corresponding one by one to a plurality of member functions in the kernel function, thereby synchronously running a plurality of member functions, avoiding the dependency relationship caused by program order, supporting single-stage parallelism, and improving the overall running performance of the chip system.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of chip design, and in particular, to an application method of a meta-thread family, a chip system, an electronic device and a medium. BACKGROUND

[0002] In the related art, the running mode of a modern graphics processing unit (GPU) usually includes the following two modes. The first mode is thread bundle specialization, a thread block or a workgroup includes multiple special thread bundles (warps), some of which are single-thread thread bundles. For a matrix multiplication accumulation thread bundle, a single-thread thread bundle occupies thread bundle resources, such as vector general-purpose registers. The second mode is that a thread block or a workgroup includes a meta-thread, which must asynchronously process multiple sub-blocks to achieve parallelism, but the parallelism is limited by the dependency relationship generated by the program order of a single thread, and single-stage parallelism cannot be achieved, and the performance is not as good as that of thread bundle specialization. SUMMARY

[0003] Embodiments of the present disclosure provide an application method of a meta-thread family, a chip system, an electronic device and a medium, which aims to avoid the dependency relationship generated by the program order, support single-stage parallelism, and improve overall running performance.

[0004] According to an aspect of the present disclosure, an application method of a meta-thread family is provided, which is applied to a chip system including a command processor and a meta-thread instruction scheduler, and the command processor is in communication connection with the meta-thread instruction scheduler. The application method of the meta-thread family includes the following steps.

[0005] The command processor receives a start instruction corresponding to a kernel function, generates a thread block configuration instruction according to the start instruction, and sends the thread block configuration instruction to the meta-thread instruction scheduler.

[0006] The meta-thread instruction scheduler receives the thread block configuration instruction and configures a thread block according to the thread block configuration instruction.

[0007] For each thread block, the meta-thread instruction scheduler generates a meta-thread family corresponding to the thread block, wherein the meta-thread family includes multiple member meta-threads, and one member meta-thread corresponds to one member function in the kernel function.

[0008] The meta-thread instruction scheduler runs multiple member functions in parallel through multiple member meta-threads.

[0009] In some embodiments, after the meta-thread instruction scheduler generates the meta-thread family corresponding to the thread block, the application method of the meta-thread family further comprises:

[0010] For the meta-thread family, one of the member meta-threads is taken as a master meta-thread, and the rest of the member meta-threads are taken as slave meta-threads, wherein the master meta-thread is used to manage one or more slave meta-threads.

[0011] In some embodiments, the slave meta-thread comprises one of:

[0012] a load store meta-thread corresponding to a load function in the kernel function;

[0013] an MMA meta-thread corresponding to an MMA function in the kernel function;

[0014] a vector meta-thread corresponding to a vector calculation function in the kernel function;

[0015] a communication meta-thread corresponding to a communication operation function in the kernel function.

[0016] In some embodiments, the member function is identified through one of the following steps:

[0017] determining hint information in the kernel function, and identifying a plurality of member functions from the kernel function according to the hint information;

[0018] compiling the kernel function through a compiler, and identifying a plurality of member functions from the kernel function.

[0019] In some embodiments, the meta-thread instruction scheduler generates the meta-thread family corresponding to the thread block, comprising one of:

[0020] the meta-thread instruction scheduler synchronously starts the master meta-thread and a plurality of slave meta-threads;

[0021] after the meta-thread instruction scheduler starts the master meta-thread, the master meta-thread is used to sequentially start a plurality of slave meta-threads.

[0022] In some embodiments, the meta-thread instruction scheduler runs a plurality of member functions in parallel through a plurality of member meta-threads, comprising:

[0023] the meta-thread instruction scheduler determines the current state of a barrier object;

[0024] when the barrier object meets a preset synchronization condition, the meta-thread instruction scheduler synchronously executes a plurality of member meta-threads to realize parallel running of a plurality of member functions.

[0025] In some embodiments, after the meta-thread instruction scheduler generates the meta-thread family corresponding to the thread block, the application method of the meta-thread family further includes one of the following:

[0026] equally distributing a same number of scalar general-purpose registers to the plurality of member meta-threads;

[0027] distributing a respective number of scalar general-purpose registers to the plurality of member meta-threads according to a preset distribution principle, wherein all the scalar general-purpose registers are non-overlapping and located in a same scalar general-purpose register pool.

[0028] In some embodiments, the distributing a respective number of scalar general-purpose registers to the plurality of member meta-threads according to a preset distribution principle includes:

[0029] obtaining a historical usage parameter of each of the member meta-threads, wherein the historical usage parameter includes a historical usage frequency or a historical usage times;

[0030] distributing different numbers of scalar general-purpose registers to different member meta-threads according to the historical usage parameter, wherein the historical usage parameter and the number are in a positive correlation.

[0031] In some embodiments, after the meta-thread instruction scheduler generates the meta-thread family corresponding to the thread block, the application method of the meta-thread family further includes one of the following:

[0032] The meta-thread instruction scheduler schedules the plurality of member meta-threads into a same scalar unit, so that the plurality of member meta-threads run in a preset order in sequence.

[0033] The meta-thread instruction scheduler schedules the plurality of member meta-threads into a plurality of different scalar units, wherein one scalar unit corresponds to one or more member meta-threads, and the plurality of member meta-threads located in a same scalar unit run in a preset order in sequence.

[0034] In some embodiments, the chip system further includes a plurality of functional units, and the plurality of functional units are in communication connection with the meta-thread instruction scheduler; the meta-thread instruction scheduler parallelly runs a plurality of member functions through the plurality of member meta-threads, including:

[0035] The meta-thread instruction scheduler sends the plurality of member meta-threads to a plurality of different functional units respectively, and executes the plurality of member meta-threads through the plurality of different functional units respectively, wherein the functional units include a communication operation unit, a data replication unit, a tensor calculation unit, a scalar operation unit, or an execution unit.

[0036] According to an aspect of the present disclosure, a chip system is provided, comprising:

[0037] a command processor configured to receive a start instruction corresponding to a kernel function, generate a thread block configuration instruction according to the start instruction, and send the thread block configuration instruction to a meta-thread instruction scheduler;

[0038] a meta-thread instruction scheduler communicatively connected to the command processor, wherein the meta-thread instruction scheduler is configured to receive the thread block configuration instruction and configure a thread block according to the thread block configuration instruction; for each thread block, the meta-thread instruction scheduler is further configured to generate a meta-thread family corresponding to the thread block, wherein the meta-thread family comprises a plurality of member meta-threads, and each member meta-thread corresponds to a member function in the kernel function; and the meta-thread instruction scheduler is further configured to run the plurality of member functions in parallel through the plurality of member meta-threads.

[0039] According to an aspect of the present disclosure, an electronic device is provided, comprising a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection communication between the processor and the memory, wherein the program is executable on the processor to realize the application method of the meta-thread family.

[0040] According to an aspect of the present disclosure, a computer readable storage medium is provided, which stores one or more programs executable by one or more processors to realize the application method of the meta-thread family.

[0041] According to an aspect of the present disclosure, a computer program product is provided, comprising a computer program or computer instructions, characterized in that the computer program or the computer instructions are stored in a computer readable storage medium, a processor of a computer device reads the computer program or the computer instructions from the computer readable storage medium, and the processor executes the computer program or the computer instructions, so that the computer device realizes the application method of the meta-thread family.

[0042] The application method of the meta-thread family, the chip system, the electronic device and the medium provided by the present disclosure, the method comprises the following steps: first, the command processor receives a start instruction corresponding to a kernel function, generates a thread block configuration instruction according to the start instruction, and sends the thread block configuration instruction to a meta-thread instruction scheduler; then, the meta-thread instruction scheduler receives the thread block configuration instruction, and configures a thread block according to the thread block configuration instruction; next, for each thread block, the meta-thread instruction scheduler generates a meta-thread family corresponding to the thread block, wherein the meta-thread family comprises a plurality of member meta-threads, and one member meta-thread corresponds to one member function in the kernel function; finally, the meta-thread instruction scheduler runs a plurality of member functions in parallel through a plurality of member meta-threads. In one thread block corresponding to one kernel function, the present disclosure simultaneously sets a plurality of member meta-threads corresponding one by one to a plurality of member functions in the kernel function, so that the meta-thread instruction scheduler can synchronously run a plurality of corresponding member functions when synchronously executing a plurality of member meta-threads, thereby greatly avoiding the dependency relationship caused by program order, and thus the present disclosure realizes support for single-stage parallelism and improves the overall running performance of the chip system.

[0043] Other features and advantages of the present disclosure will be set forth in the following description, and in part will become apparent from the description, or will be learned by practice of the present disclosure. The objects and other advantages of the present disclosure can be realized and obtained by the structure particularly pointed out in the specification, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0044] The accompanying drawings are used to provide a further understanding of the technical solutions of the present disclosure, and constitute a part of the specification, and are used to explain the technical solutions of the present disclosure together with the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions of the present disclosure.

[0045] Figure 1 It is a system architecture diagram of the application method of the meta-thread family according to the embodiments of the present disclosure;

[0046] Figure 2 It is a relationship diagram of a thread block, a thread bundle, a normal work thread, a meta-thread family, a member meta-thread, a main meta-thread and a secondary meta-thread according to one embodiment of the present disclosure;

[0047] Figure 3 It is a main flowchart of the application method of the meta-thread family according to one embodiment of the present disclosure;

[0048] Figure 4 It is a step flowchart of identifying a member function according to one embodiment of the present disclosure;

[0049] Figure 5 It is a step flowchart of identifying a member function according to another embodiment of the present disclosure;

[0050] Figure 6 is a schematic diagram of the meta-thread family synchronization start of one embodiment of the present disclosure;

[0051] Figure 7 is a schematic diagram of the primary meta-thread sequentially starting multiple secondary meta-threads of one embodiment of the present disclosure;

[0052] Figure 8 is a step flow chart of synchronously executing multiple member meta-threads after meeting a preset synchronization condition of one embodiment of the present disclosure;

[0053] Figure 9 is a schematic diagram of the average allocation of scalar general-purpose registers of one embodiment of the present disclosure;

[0054] Figure 10 is a schematic diagram of the on-demand allocation of scalar general-purpose registers of one embodiment of the present disclosure;

[0055] Figure 11 is a step flow chart of allocating scalar general-purpose registers according to the historical use parameters of member meta-threads of one embodiment of the present disclosure;

[0056] Figure 12 is a schematic diagram of scheduling all member meta-threads to the same scalar unit of one embodiment of the present disclosure;

[0057] Figure 13 is a schematic diagram of scheduling all member meta-threads to multiple different scalar units respectively of one embodiment of the present disclosure;

[0058] Figure 14 is a structural schematic diagram of an electronic device according to one embodiment of the present disclosure. DETAILED DESCRIPTION

[0059] In order to make the objectives, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and do not limit the present disclosure.

[0060] Before the present disclosure is further described in detail, the terms and phrases involved in the present disclosure are explained, and the terms and phrases involved in the present disclosure are applicable to the following explanations:

[0061] Chip: Chip generally refers to an integrated circuit, which is a kind of microelectronic device that integrates circuits on a tiny silicon chip. Chips are widely used in various electronic devices such as computers, mobile phones, and cars, and are used to implement various functions such as computing, storage, and communication. Chips are packaged after the die and can be used in actual electronic systems. And the chip can include a command processor and a plurality of block processing modules, each of which is in communication with the command processor.

[0062] GPU (Graphics Processing Unit) is a kind of microprocessor specially designed for parallel processing of a large number of graphics and general-purpose computing tasks. Its core function is a kind of processor specially designed for high parallel computing tasks, which is good at processing a large number of simple and repetitive computing operations at the same time. It was originally designed to accelerate image rendering, but due to its powerful parallel computing capability, it has been widely used in scientific computing, artificial intelligence, deep learning, physical simulation and other fields.

[0063] Command Processor (CP): Command processor is a command processing unit in the chip, which is responsible for processing and executing various instructions or commands to ensure that each module in the chip can run efficiently and orderly. In the embodiments of the present disclosure, the synchronization of the CP layer mainly refers to the synchronization between the internal or different block processing clusters in the chip. The synchronization information is in the memory of the command processor, and the synchronization operator is the command processor. The command processor can include a task data packet cache, an analysis module and a segmentation module. The task data packet cache is used to store and process information sent by the host layer, the analysis module is used to analyze the information stored in the task data packet cache, and the segmentation module is used to divide the task information obtained by the analysis module to send relevant task information to the relevant block processing cluster for execution processing.

[0064] Compute Unit (CU): In graphics processor architecture, a Compute Unit is the basic unit that performs computational tasks. When a workgroup runs on a graphics processor, it is actually executed on a Compute Unit. A Compute Unit is the basic unit on a graphics processor for performing parallel computing tasks, which contains a certain number of processing elements that can execute multiple threads in parallel. A block processing cluster can include multiple Compute Units. In the embodiments of the present disclosure, the synchronization of the CU layer refers to the synchronization within the same CU, the synchronization between different workgroups or different kernels within the CU, and is implemented through data packets and kernels in the synchronization process. Among them, a Compute Unit can include a workgroup cluster data cache, a synchronization engine, a segmentation module, a task scheduling module, and multiple execution units. The workgroup cluster data cache is used to receive information sent by the command scheduling unit, the synchronization engine is used to parse and synchronize the information in the workgroup cluster data cache to obtain task information, and the segmentation module is used to segment the task information and send the corresponding task information to the execution unit for task execution processing.

[0065] Streaming Multiprocessor (SM): is the core computing unit of GPU, composed of multiple stream processors and related resources, responsible for parallel execution of a large number of threads, and implements efficient graphics rendering and computing tasks. SM is composed of multiple stream processors (SP). In addition, it also includes shared memory, registers, control units, instruction caches and other resources. These components work together to support meta-thread applications. SM schedules threads in units of warps, each warp usually contains 32 threads. Threads within the same warp execute the same instructions in a single instruction multiple thread (SIMT) manner, but process different data. This mechanism allows SM to efficiently handle a large number of parallel tasks, such as rendering multiple pixels in an image. Its core function is parallel computing, mainly used for tasks such as vertex processing and pixel shading in graphics rendering. At the same time, with the development of technology, SM is also widely used in artificial intelligence training, big data analysis and other non-graphic fields, fully exerting its powerful parallel processing capability. SM is the core module of GPU, and multiple SMs form the computing part of GPU. The number of SMs in different GPU models is different, and the number and performance of SMs directly affect the computing power and graphics processing speed of GPU.

[0066] Execution Unit (EU): is the functional hardware module that actually completes the "computing operation" in the processor core. It is assigned one or more decoded instructions by the scheduler and completes arithmetic, logic, memory access or special operations in a specific clock cycle, and writes the results back to the target location.

[0067] Kernel Function: In GPU programming, a kernel function is a function defined specifically for execution on a GPU, serving as the core bridge connecting CPU and GPU computation. It is responsible for describing the specific task logic of GPU parallel processing. Kernel functions are initiated by the CPU but executed on the GPU, utilizing the GPU's stream processors and numerous threads to achieve parallel computation. The code of a kernel function is executed simultaneously by hundreds or thousands of threads, each handling different data. For example, when processing an image, a thread may be responsible for calculating the color of a single pixel. The logic for calculating a single pixel is defined in the kernel function, and the GPU automatically schedules a large number of threads to execute this logic in parallel. When a kernel function is launched, it needs to specify the thread organization (such as the dimensions of thread blocks and thread grids), and the GPU will allocate threads to different stream processors based on these parameters.

[0068] Thread: The smallest unit of execution scheduled by the operating system, a control flow within a process, with its own stack, program counter, and register state, but sharing the resources of the same process (such as memory, file handles, global variables, etc.).

[0069] Barrier Object: A barrier object is a synchronization mechanism commonly used in multi-threaded or concurrent programming to control the execution order of multiple threads or processes, ensuring that they execute synchronously at a certain point. The main role of a barrier object is to make a group of threads "wait" at a certain point until all threads reach this point, and then allow them to continue execution.

[0070] WorkGroup: A group of threads composed of multiple threads, also known as a thread block (block). The threads in a single workgroup are generally used to perform the same pattern of processing on different data.

[0071] CWG (Cluster WorkGroup): A thread execution set composed of multiple workgroups (WG), used to uniformly schedule and cooperatively manage multiple workgroups at a higher level. A CWG includes several workgroups, each containing multiple thread bundles (Warp), used to parallel process different data elements but execute the same or similar computation logic. Multiple workgroups within the same CWG can share part of the hardware or software management resources or participate in the same communication and synchronization operations, such as reduction (Reduction) and broadcast (Broadcast) operations.

[0072] SWG (Super WorkGroup): a higher-level parallel execution set composed of multiple cluster workgroups (CWGs), used for hierarchical organization and unified scheduling of large-scale parallel computing tasks. A SWG includes multiple CWGs, and each CWG further includes multiple workgroups. The SWG is used to represent a group of cluster workgroups that have logical association or communication requirements in the same parallel computing task or the same grid, so as to realize cooperative execution, synchronous control or communication operation in a larger range.

[0073] Grid: composed of multiple workgroups. Each workgroup serves as a grid point in the Grid, and each workgroup is provided with a corresponding grid point coordinate. The Grid can be one-dimensional, two-dimensional or three-dimensional. It can be understood that the shape of the Grid is set by the user during programming.

[0074] In the prior art, the running mode of a modern graphics processing unit (GPU) usually includes the following two modes. The first mode is thread bundle specialization. A thread block or a workgroup includes multiple special thread bundles (warps), some of which are single-thread thread bundles. For matrix multiplication accumulation thread bundles, a single-thread thread bundle occupies thread bundle resources, such as vector general-purpose registers. The second mode is that a thread block or a workgroup includes a meta-thread. The meta-thread must asynchronously process multiple sub-blocks to achieve parallelism, but the parallelism is limited by the dependency relationship generated by the program order of a single thread, and single-stage parallelism cannot be achieved. Moreover, the performance is not as good as that of the thread bundle specialization.

[0075] Therefore, the present disclosure proposes an application method of a meta-thread family, a chip system, an electronic device and a medium, which aims to avoid the dependency relationship generated by the program order, support single-stage parallelism and improve overall running performance.

[0076] System architecture description of the system architecture to which the embodiments of the present disclosure are applied

[0077] In an embodiment, as shown in Figure 1 Figure 1 is a system architecture diagram of an application method of a meta-thread family according to an embodiment of the present disclosure. The system architecture includes but is not limited to that the chip system includes a command processor 500 and a meta-thread instruction scheduler 100. The command processor 500 is in communication connection with the meta-thread instruction scheduler 100.

[0078] ​The command processor 500 is responsible for receiving and preprocessing external instructions. When it receives the start instruction of the kernel function issued by the CPU, it will first parse the kernel function execution configuration contained in the start instruction, such as the number of thread blocks and resource requirements, and send this information to the meta-thread instruction scheduler 100 through the thread block configuration instruction.

[0079] In addition, the meta-thread instruction scheduler 100 can receive thread block configuration instructions sent by the command processor 500, complete the pre-allocation of hardware resources for the corresponding thread block, and generate a corresponding meta-thread family for each thread block. That is, the kernel function is decomposed into member meta-threads with different functions such as loading, MMA (Matrix Multiply and Accumulate), vector calculation, and communication. Finally, the meta-thread instruction scheduler 100 can trigger the parallel execution of multiple member meta-threads at the same time, realizing the single-stage parallel operation of the kernel function.

[0080] In one embodiment, such as Figure 1 As shown, the chip system also includes multiple functional units, all of which are communicatively connected to the meta-thread instruction scheduler 100. The functional units include a communication operation unit 210, a data copying unit 220, a tensor calculation unit 230, a scalar operation unit 300, or an execution unit 400.

[0081] The meta-thread instruction scheduler 100 is located within the computing core. Specifically, after a meta-thread family is created, the meta-thread family program is executed. The meta-thread family includes multiple member meta-threads, each of which consists of a meta-thread instruction sequence. The computing core contains a meta-thread instruction scheduler 100. The meta-thread instruction scheduler 100 fetches, decodes, and dispatches the instructions in the meta-thread instruction sequence to the corresponding functional units for execution one by one.

[0082] Furthermore, the functional units can be located either within or outside the computing core. Specifically, the functional units can be scalar operation units 300, execution units 400, tensor calculation units 230, data copying units 220, or communication operation units 210. The computing core also includes one or more scalar operation units 300, execution units 400, tensor calculation units 230, and data copying units 220. The communication operation unit 210 is located outside the computing core. The meta-thread instruction scheduler 100 acquires, decodes, and dispatches instructions from the meta-thread instruction sequence to the corresponding functional units for execution.

[0083] Specifically, the metathread instruction scheduler 100 can play a role of core coordination in a parallel computing system through a metathread family, which can call one or more functional units related through the metathread family to enable the functional units to perform operations related to their own functions.

[0084] In the above, the metathread instruction scheduler 100 can sequentially call multiple functional units through the metathread family one by one, or can simultaneously call multiple functional units through the metathread family, and the disclosure does not make specific limitations on the calling order and the number of the functional units.

[0085] In an embodiment, the computing core is a key component responsible for executing parallel computing tasks in a GPU, which contains multiple execution units 400, register files, shared memories and other resources, and can process multiple threads at the same time to achieve efficient parallel computing.

[0086] It can be understood that the computing core described above can be a computing unit or a stream multiprocessor, and the disclosure does not make specific limitations thereon.

[0087] In an embodiment, the functional unit described above can include a block operation unit, which can be a communication operation unit 210 for performing communication operations, a data replication unit 220 for performing data replication, a tensor computing unit 230 for performing matrix operations, or other types of block operation units, and the disclosure does not make specific limitations thereon.

[0088] Specifically, in the case where the functional unit is the communication operation unit 210, the metathread instruction scheduler 100 can call the communication operation unit 210 through the metathread family to perform a broadcast operation. It can be understood that the broadcast operation described above refers to an operation of copying and sending the data of one computing unit to all other computing units.

[0089] In addition, in the case where the functional unit is the communication operation unit 210, the metathread instruction scheduler 100 can also call the communication operation unit 210 through the metathread family to perform a gather operation. It can be understood that the gather operation described above refers to an operation of collecting the scattered data in multiple computing units to a target computing unit in a certain order.

[0090] Furthermore, in the case where the functional unit is the communication operation unit 210, the metathread instruction scheduler 100 can also call the communication operation unit 210 through the metathread family to perform a reduction operation. It can be understood that the reduction operation described above refers to an operation of combining the scattered data in multiple computing units into one or a small number of results through a certain operation, where the operation type can be summation, multiplication, maximum value taking, or minimum value taking, etc.

[0091] In addition, in the case that the functional unit is the data copy unit 220, the meta-thread instruction scheduler 100 can invoke the data copy unit 220 through the meta-thread family to perform a data copy operation. It can be understood that, regarding the above-mentioned data copy operation, in the GPU, the data copy unit 220 is responsible for efficiently transferring data between different storage areas, reducing the burden of the computing core, and the data copy operation performed by the data copy unit 220 mainly revolves around the internal storage hierarchy of the GPU and the interaction with external devices.

[0092] The types of the data copy operation include but are not limited to the following: internal copy of global memory, copy between global memory and shared memory, copy between global memory and register, copy between constant memory and global memory, data copy between GPU and CPU, data copy of texture memory, and data copy between multiple GPUs.

[0093] In addition, in the case that the functional unit is the tensor computing unit 230, the meta-thread instruction scheduler 100 can invoke the tensor computing unit 230 through the meta-thread family to perform a matrix operation. For example, the meta-thread instruction scheduler 100 can invoke the tensor computing unit 230 through the meta-thread family to perform basic matrix multiplication, i.e., to calculate the product of two matrices; or the meta-thread instruction scheduler 100 can invoke the tensor computing unit 230 through the meta-thread family to perform matrix multiplication and addition, i.e., to add an accumulation step on the basis of matrix multiplication; or the meta-thread instruction scheduler 100 can invoke the tensor computing unit 230 through the meta-thread family to perform mixed-precision matrix operation, supporting matrix operation with multiple data precisions to balance performance and precision requirements; or the meta-thread instruction scheduler 100 can invoke the tensor computing unit 230 through the meta-thread family to perform batch matrix operation, i.e., to perform parallel operation on a set of multiple independent matrices.

[0094] In an embodiment, regarding the above-mentioned functional unit, a scalar operation unit 300 can also be included, wherein the scalar operation unit 300 is configured to process arithmetic and logical operations of scalar data.

[0095] Specifically, the scalar operation unit 300 is a core component in the meta-thread instruction scheduler 100 responsible for performing arithmetic and logical operations of scalar data, and the meta-thread instruction scheduler 100 can invoke the scalar operation unit 300 through the meta-thread family to perform arithmetic and logical operations of scalar data.

[0096] In an embodiment, regarding the above-mentioned functional unit, an execution unit 400 can also be included, wherein the execution unit 400 is configured to process vector operations.

[0097] Specifically, the execution unit 400 is a hardware component specifically optimized for vector operations. Its core function is to perform parallel processing of vector data, and it can process multiple elements simultaneously in a single operation.

[0098] In one embodiment, such as Figure 1 As shown, the meta-thread family plays a core coordinating role in parallel computing systems:

[0099] Communication operation management: The computing core interacts with communication operation modules on the left, such as Reduction and Broadcast. Examples include Cluster Working Groups (CWG), Super Working Groups (SWG), and Grid.

[0100] Interaction with the Scalar ALU: The meta-thread instruction scheduler 100 calls the Scalar ALU 300 through the meta-thread family to process arithmetic and logical operations on scalar data.

[0101] Execution unit 400 calls: The meta-thread instruction scheduler 100 calls execution unit 400 through the meta-thread family to asynchronously request and respond to schedule the execution of computing tasks.

[0102] Matrix operations and data copying: The meta-thread instruction scheduler 100 coordinates matrix multiplication and addition operations and data copying operations through the meta-thread family, ensuring the transmission of data and the execution of complex operations in the computation process.

[0103] Therefore, by Figure 1 As can be seen, the meta-thread instruction scheduler 100 is the scheduling center for various computing and communication operations, enabling different tasks in parallel computing, such as scalar operations, matrix operations, data communication, and execution unit 400 invocation, to coordinate efficiently.

[0104] In another embodiment, the communication kernel function, the block processing kernel function, and the element processing kernel function can run in parallel without causing significant resource conflicts.

[0105] Furthermore, in one embodiment, over-allocation of resource-intensive ordinary worker threads can be avoided when demand is insufficient. Elements-based allocation of resources such as vector general-purpose registers is saved. The meta-thread family is an efficient block-based processing method, framework, and paradigm suitable for future parallel computing systems.

[0106] In one embodiment, such as Figure 2 As shown, Figure 2 This is a schematic diagram illustrating the relationship between thread blocks, thread bundles, ordinary worker threads, meta-thread families, member meta-threads, main meta-threads, and secondary meta-threads in one embodiment of this disclosure. This disclosure adds a meta-thread family for managing a thread block, wherein...Figure 2 It can be known that one thread block includes multiple thread bundles and one meta-thread family, one thread bundle includes multiple normal working threads, and one meta-thread family includes multiple member meta-threads which are respectively one main meta-thread and multiple secondary meta-threads.

[0107] Overall implementation of the application method of the meta-thread family

[0108] The embodiment of the present disclosure provides an application method of a meta-thread family, which is applied to a chip system. The chip system includes a command processor and a meta-thread instruction scheduler, and the command processor is in communication connection with the meta-thread instruction scheduler. Referring to Figure 3 , Figure 3 is the main flowchart of the application method of the meta-thread family according to one embodiment of the present disclosure. The application method of the meta-thread family includes but is not limited to steps S301, S302, S303 and S304.

[0109] In step S301, the command processor receives a start instruction corresponding to a kernel function, generates a thread block configuration instruction according to the start instruction, and sends the thread block configuration instruction to the meta-thread instruction scheduler.

[0110] In step S302, the meta-thread instruction scheduler receives the thread block configuration instruction and configures a thread block according to the thread block configuration instruction.

[0111] In step S303, for each thread block, the meta-thread instruction scheduler generates a meta-thread family corresponding to the thread block, wherein the meta-thread family includes multiple member meta-threads, and one member meta-thread corresponds to one member function in the kernel function.

[0112] In step S304, the meta-thread instruction scheduler runs multiple member functions in parallel through multiple member meta-threads.

[0113] In an embodiment, the meta-thread instruction scheduler is the core of meta-thread family management and parallel execution. After receiving the thread block configuration instruction, the meta-thread instruction scheduler first completes the hardware resource pre-allocation and logical configuration of the thread block according to the parameters in the instruction, to build a basic execution unit for the subsequent generation of the meta-thread family. Then, for each configured thread block, the meta-thread instruction scheduler disassembles the kernel function into multiple functionally independent member functions, and matches each member function with a corresponding member meta-thread, thereby constructing a meta-thread family with clear functional division. Finally, the meta-thread instruction scheduler coordinates and schedules multiple member meta-threads to carry out corresponding member function tasks and execute in parallel, breaking the limitation of traditional single meta-thread serial execution, realizing parallel processing of multiple tasks in the kernel function, and guaranteeing the execution efficiency and data consistency of each link through the functional specialization design of the member meta-thread, and finally completing the efficient parallel operation of the kernel function.

[0114] It should be noted that in a thread block corresponding to a kernel function, the disclosure simultaneously sets a plurality of member meta-threads corresponding to a plurality of member functions in the kernel function, so that the meta-thread instruction scheduler can synchronously run the plurality of member functions corresponding to the plurality of member meta-threads when synchronously executing the plurality of member meta-threads, thereby greatly avoiding the dependency relationship caused by program order, and thus the disclosure supports single-stage parallelism and improves the overall performance of the chip system.

[0115] In an embodiment, for a meta-thread family, the meta-thread instruction scheduler selects one member meta-thread as a master meta-thread, and simultaneously sets the remaining member meta-threads as secondary meta-threads, wherein the master meta-thread is used to manage one or more secondary meta-threads.

[0116] Specifically, the meta-thread instruction scheduler selects one member meta-thread from all member meta-threads as a master meta-thread, and sets the remaining member meta-threads as secondary meta-threads, wherein the master meta-thread undertakes the global management function of the meta-thread family and is the control center of the entire meta-thread family, and its core functions include the start timing control of secondary meta-threads, the execution state monitoring, the resource allocation coordination, and the synchronization barrier triggering, etc.; and each secondary meta-thread is a full-time function executor corresponding to a specific member function in the kernel function, such as a load function, an MMA function, a vector calculation function, and a communication function, and only needs to focus on the efficient completion of the function task it belongs to. This hierarchical architecture not only avoids the disorder of multi-meta-thread parallel execution through the unified management of the master meta-thread, but also guarantees the execution efficiency of each sub-task of the kernel function through the functional specialization of the secondary meta-threads, thereby realizing the organic unification of ordered control and efficient parallelism of the meta-thread family.

[0117] In an embodiment, the types of the above-mentioned secondary meta-threads include but are not limited to the following types:

[0118] The first type: the secondary meta-thread is a load storage meta-thread corresponding to a load function in the kernel function; wherein the load storage meta-thread is directly bound to the load function in the kernel function and is responsible for data read / write operations, such as loading Q / K / V calculation data from global memory to shared memory or writing intermediate or final calculation results back to a specified storage area.

[0119] The second type: the secondary meta-thread is an MMA meta-thread corresponding to an MMA function in the kernel function; wherein the MMA meta-thread corresponds to the matrix multiplication function in the kernel function and can complete high-density computing tasks such as matrix multiplication.

[0120] The third type: the sub-element thread is a vector element thread, and the vector element thread corresponds to a vector calculation function in the kernel function; wherein the vector element thread matches the vector calculation function of the kernel function, and mainly undertakes vector type operations such as Softmax activation, data normalization, feature preprocessing, and guarantees efficient execution of non-matrix type numerical calculation.

[0121] The fourth type: the sub-element thread is a communication element thread, and the communication element thread corresponds to a communication operation function in the kernel function. Wherein the communication element thread is associated with the communication operation function of the kernel function, and is responsible for realizing communication tasks such as data synchronization and atomic operation within the thread block or across the thread block, and providing data consistency guarantee for multi-element thread parallel execution through low-delay data interaction.

[0122] It should be noted that the identification of the above-mentioned member function can include but is not limited to the two identification methods shown in Figure 4 or Figure 5 , and the specific identification methods are as follows:

[0123] As shown in Figure 4 , the step flow chart of identifying the member function is shown in Figure 4 ; the identification process of the above-mentioned member function includes but is not limited to step S401 and step S402.

[0124] Step S401, determining the prompt information in the kernel function;

[0125] Step S402, identifying a plurality of member functions from the kernel function according to the prompt information.

[0126] In an embodiment, in order to realize accurate mapping of the kernel function to the member element thread, the member function that can be independently parallel is first identified from the kernel function; in the specific execution, the system will scan and determine the preset prompt information in the kernel function code through the preset syntax analysis tool, for example, code annotation mark, function declaration identifier, which provides clear functional identity mark for different functional modules in the kernel function; then, the system will disassemble the kernel function into a plurality of independent member functions according to the extracted prompt information, for example, when the "load" type prompt information is identified, the corresponding code segment is marked as a load function, or when the "mma" type prompt information is identified, the MMA function is marked.

[0127] As shown in Figure 5 , the step flow chart of identifying the member function is shown in Figure 5 ; the identification process of the above-mentioned member function includes but is not limited to step S501 and step S502.

[0128] Step S501, compiling the kernel function through the compiler;

[0129] Step S502: Identify multiple member functions from the kernel function.

[0130] In one embodiment, firstly, the compiler performs preprocessing and deep syntax and semantic analysis on the complete code of the kernel function. During this process, the compiler scans key information such as the instruction sequence, function call logic, hardware instruction binding characteristics, and data flow of the kernel function. For example, it identifies code segments that call global memory loading instructions, matrix multiplication and addition instruction blocks that bind tensor computation units, arithmetic instruction sets that perform vector operations, and communication instruction modules that trigger thread synchronization. Subsequently, based on the results of the previous compilation analysis, the compiler automatically decomposes the kernel function into multiple independent member functions according to preset functional classification rules. For example, code segments containing memory read and write instructions are classified as loading functions, instruction blocks that call MMA hardware units are classified as MMA functions, vector operation instruction sets are identified as vector computation functions, and thread synchronization and data interaction instruction modules are defined as communication operation functions.

[0131] Additionally, it should be noted that the generation of the meta-thread family corresponding to the thread block by the meta-thread instruction scheduler in step S303 above can be, but is not limited to, the following two creation or startup methods:

[0132] The first startup method: The meta-thread instruction scheduler synchronously starts the main meta-thread and multiple sub-threads.

[0133] Specifically, such as Figure 6 As shown, Figure 6 This is a schematic diagram of the synchronous startup of a meta-thread family according to an embodiment of this disclosure. Its core is the parallel startup control of the main meta-thread and secondary threads (secondary thread one, secondary thread two, and secondary thread three) by the meta-thread instruction scheduler: Under this architecture, all member meta-threads of the meta-thread family, such as the main meta-thread and multiple secondary threads, are uniformly triggered by the meta-thread instruction scheduler to realize the execution logic of simultaneous startup.

[0134] In this way, the main thread and the secondary thread do not need to wait for each other's start signal and can directly enter their respective task processes. This maximizes the initial parallelism of the meta-thread family and avoids the serial waiting overhead during the startup phase. It is suitable for scenarios where there is no strong data dependency between members and multiple tasks need to be started quickly in parallel.

[0135] This can be started using the following procedure:

[0136] “leader”

[0137] Barrier object initialization

[0138] “member 1” # TCE

[0139] copy0 setup

[0140] copy0

[0141] “member 2” # TPE

[0142] mma0 setup

[0143] mma0

[0144] “member 3” # VPE

[0145] calleu0 setup

[0146] calleu0

[0147] Among them, TCE, or Tensor Copy Engine, is the execution unit of the loading and storing meta thread; TPE, or Tensor Processing Engine, is the execution unit of the MMA meta thread; VPE, or Vector Processing Engine, is the execution unit of the vector meta thread; leader is the main meta thread; member 1 is the first meta thread; member 2 is the second meta thread; and member 3 is the third meta thread.

[0148] The second startup method: After the meta-thread instruction scheduler starts the main meta-thread, multiple sub-threads are started sequentially through the main meta-thread.

[0149] Specifically, such as Figure 7 As shown, Figure 7 This is a schematic diagram illustrating how a main thread sequentially starts multiple secondary threads in one embodiment of this disclosure. The core logic is that the primary thread instruction scheduler first starts the primary thread individually. After the primary thread completes initialization, resource preparation, and other preliminary work, it then triggers the startup of the secondary threads (secondary thread one, secondary thread two, and secondary thread three) in a preset order. Simultaneously, all secondary threads must wait for a "LaunchSync" signal after startup before continuing to execute subsequent tasks.

[0150] In response, the main thread, as the control center of the startup process, can flexibly adjust the startup sequence of the secondary threads according to the task dependencies. The "synchronous startup" mechanism ensures that all started secondary threads are synchronized at a unified node, avoiding data errors caused by inconsistent execution rhythms. It is suitable for complex parallel tasks with strong data dependencies among members and that need to be promoted in stages.

[0151] This can be started using the following procedure:

[0152] “leader”

[0153] launch(member1)

[0154] launch(member2)

[0155] launch(member3)

[0156] “member 1” # TCE

[0157] launch_sync

[0158] “member 2” # TPE

[0159] launch_sync

[0160] “member 3” # VPE

[0161] launch_sync

[0162] Among them, TCE, or Tensor Copy Engine, is the execution unit of the loading and storing meta thread; TPE, or Tensor Processing Engine, is the execution unit of the MMA meta thread; VPE, or Vector Processing Engine, is the execution unit of the vector meta thread; leader is the main meta thread; member 1 is the first meta thread; member 2 is the second meta thread; and member 3 is the third meta thread.

[0163] In addition, such as Figure 8 As shown, Figure 8 This is a flowchart illustrating the steps of synchronously executing multiple member threads after a preset synchronization condition is met, according to an embodiment of this disclosure. Step S304 may include, but is not limited to, steps S801 and S802.

[0164] Step S801: The meta-thread instruction scheduler determines the current state of the barrier object;

[0165] Step S802: When the barrier object meets the preset synchronization conditions, the meta-thread instruction scheduler synchronously executes multiple member meta-threads to achieve parallel execution of multiple member functions.

[0166] In one embodiment, the meta-thread instruction scheduler continuously monitors the current state of a pre-set barrier object, such as the completion status of data loading of the dimensional thread and the resource readiness status. When the state of the barrier object reaches a preset synchronization condition, such as when all dimensional threads have completed data reading and writing to shared memory, the meta-thread instruction scheduler will trigger multiple member meta-threads to synchronously execute the corresponding member functions.

[0167] Additionally, it should be noted that after the meta-thread instruction scheduler generates the meta-thread family corresponding to the thread block, this disclosure also needs to allocate certain scalar general-purpose registers to each member meta-thread. The allocation methods for these scalar general-purpose registers include, but are not limited to, the following two methods, as detailed below:

[0168] The first allocation method is to allocate an equal number of scalar general-purpose registers to multiple member threads.

[0169] Specifically, such as Figure 9 As shown, Figure 9 This is a schematic diagram of the average allocation of scalar general-purpose registers according to an embodiment of this disclosure, where N is a positive integer. The first method is average allocation, which allocates an equal number of scalar general-purpose registers (SGPRs) to all member threads. This method has a simple architecture and requires no additional analysis, making it suitable for scenarios where the register requirements of each member thread are similar.

[0170] The second allocation method is to allocate a corresponding number of scalar general-purpose registers to multiple member threads according to the preset allocation principle. All scalar general-purpose registers are non-overlapping and located in the same scalar general-purpose register pool.

[0171] Specifically, such as Figure 10 As shown, Figure 10 This is a schematic diagram of on-demand allocation of scalar general-purpose registers according to an embodiment of this disclosure, where N0, N1, N2, and N3 are all positive integers. The second method is on-demand allocation, which allocates a corresponding number of registers to different member meta-threads based on preset principles (such as the function type of the member meta-thread, historical usage parameters, etc.). For example, the MMA meta-thread is allocated more registers to support high-density computing, while the communication meta-thread is allocated fewer registers to accommodate lightweight operations. In this way, on-demand allocation improves the utilization rate of register resources.

[0172] In addition, such as Figure 11 As shown, Figure 11 This is a flowchart illustrating the steps of allocating a scalar general-purpose register based on the historical usage parameters of a member thread, according to one embodiment of this disclosure. The second allocation method described above may include, but is not limited to, steps S1101 and S1102.

[0173] Step S1101: Obtain the historical usage parameters of each member thread, wherein the historical usage parameters include historical usage frequency or historical usage count;

[0174] Step S1102: Allocate different numbers of scalar general-purpose registers to different member threads according to the historical usage parameters, wherein there is a positive correlation between the historical usage parameters and the number.

[0175] In an embodiment, first, the historical usage parameters corresponding to each member meta-thread are obtained from the historical running database of the system, which mainly covers two types of core indicators: one is the historical usage frequency, that is, the calling frequency of the register resource by the member meta-thread of the same type in the past execution of the same type of kernel function; the second is the historical usage times, that is, the total number of cumulative triggering of register read-write operation, which provides data-based demand evaluation basis for subsequent resource allocation. Subsequently, the system will allocate different number of scalar general registers to different member meta-threads. For example, the MMA meta-thread has much higher historical usage frequency and times of registers than the communication meta-thread because it undertakes high-density matrix multiplication operation, so it will be allocated more scalar general registers, while the communication meta-thread only needs to adapt to light data interaction task, so it is allocated less register resources, and all allocated registers come from the same non-overlapping global register pool, ensuring that the resources are globally controllable and conflict-free.

[0176] In addition, after the meta-thread instruction scheduler generates the meta-thread family corresponding to the thread block, the disclosure also needs to schedule multiple member meta-threads to the scalar unit for running, wherein the scheduling mode of the member meta-threads includes but is not limited to the following two kinds, which are specifically as follows:

[0177] The first scheduling mode: the meta-thread instruction scheduler schedules multiple member meta-threads into the same scalar unit, so that multiple member meta-threads run in a predetermined order.

[0178] Specifically, as shown in Figure 12 , the first scheduling mode is to schedule all member meta-threads to the same scalar unit. Figure 12 is a schematic diagram of scheduling all member meta-threads to the same scalar unit of an embodiment of the disclosure. The meta-thread instruction scheduler schedules all member meta-threads of the meta-thread family to a single scalar unit, and all member meta-threads occupy the scalar unit in a predetermined execution order, for example, the primary meta-thread executes the corresponding task first, releases the unit resource after completion, and then the secondary meta-thread one, the secondary meta-thread two and the secondary meta-thread three execute in order, and only one member runs on the scalar unit at any time.

[0179] For this, the first scheduling mode can realize the ordered execution of the meta-thread family with lower hardware resource occupation: by multiplexing a single scalar unit, the consumption of hardware resources is reduced, and the predetermined order guarantees that the execution logic between members does not conflict, which is suitable for scenarios where the task load of the member meta-thread is light and the scalar unit resource of the chip is scarce, which not only meets the execution demand of the meta-thread family, but also avoids the idle waste of hardware resources.

[0180] The second scheduling mode is that the meta-thread instruction scheduler schedules multiple member meta-threads into multiple different scalar units, wherein one scalar unit corresponds to one or more member meta-threads, and multiple member meta-threads located in the same scalar unit run in a preset order.

[0181] Specifically, as shown in Figure 13 , Figure 13 is a schematic diagram of scheduling all member meta-threads into multiple different scalar units according to an embodiment of the present disclosure. The meta-thread instruction scheduler allocates different member meta-threads of the meta-thread family into multiple independent scalar units (scalar unit zero, scalar unit one, and scalar unit two), for example, the primary meta-thread is scheduled into the scalar unit zero, the secondary meta-thread one is scheduled into the scalar unit one, and the secondary meta-thread two and the secondary meta-thread three are jointly scheduled into the scalar unit two; wherein multiple member meta-threads in a single scalar unit (such as the secondary meta-thread two and the secondary meta-thread three in the scalar unit two) run in a preset order, and in addition, after the dependency relationship is solved, the member meta-threads on different scalar units can be executed in parallel.

[0182] To this end, the second scheduling mode can balance resource utilization and parallel efficiency: through distributed scheduling of multiple scalar units, the member on different scalar units realizes hardware-level parallelism, which improves the overall execution speed of the meta-thread family, and through sequential scheduling within a single scalar unit, resource conflicts of members in the same unit are avoided, which is suitable for scenarios where the member meta-thread task load is heavy and the parallel capability needs to be maximized, can fully exert the hardware computing power of multiple scalar units, and at the same time, the order of the execution logic is considered.

[0183] In an embodiment, the chip system further includes a plurality of functional units, and the plurality of functional units are in communication connection with the meta-thread instruction scheduler; regarding the step S304, the meta-thread instruction scheduler can parallelly run multiple member functions through multiple member meta-threads, which can be that the meta-thread instruction scheduler respectively sends multiple member meta-threads to multiple different functional units, and executes multiple member meta-threads through multiple different functional units, wherein the functional units include a communication operation unit, a data replication unit, a tensor calculation unit, a scalar operation unit, or an execution unit.

[0184] Based on the application method of the meta-thread family according to the above embodiment, the following embodiments of the electronic device, the chip system, the computer readable storage medium, and the computer program product of the present disclosure are proposed.

[0185] The present disclosure further provides an electronic device 600, which includes:

[0186] at least one processor, and

[0187] a memory in communication connection with the at least one processor; wherein,

[0188] The memory stores instructions that are executed by the at least one processor to cause the at least one processor to implement the method of any of the above embodiments of the present disclosure.

[0189] The following will be described in detail in combination with Figure 14 The hardware structure of the electronic device is described in detail. The electronic device 600 includes a processor 610, a memory 620, an input / output interface 630, a communication interface 640, and a bus 650.

[0190] The processor 610 can be implemented in the form of a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present disclosure.

[0191] The memory 620 can be implemented in the form of a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 620 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present disclosure are implemented by software or firmware, the related program codes are stored in the memory 620 and are called and executed by the processor 610 to implement the application method of the thread family of the embodiments of the present disclosure.

[0192] The input / output interface 630 is configured to implement information input and output.

[0193] The communication interface 640 is configured to implement the communication interaction between the device and other devices. The communication can be realized by a wired manner (for example, a USB, a network cable, etc.), or can be realized by a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0194] The bus 650 is configured to transmit information between various components (for example, the processor 610, the memory 620, the input / output interface 630, and the communication interface 640) of the device.

[0195] The processor 610, the memory 620, the input / output interface 630, and the communication interface 640 are connected to each other by the bus 650 to realize the communication connection between the device.

[0196] It is worth noting that since the electronic device 600 of the present disclosure can perform the application method of the metathread family of any of the above embodiments, the specific implementation and technical effects of the electronic device 600 of the present disclosure can refer to the specific implementation and technical effects of the application method of the metathread family of any of the above embodiments.

[0197] In addition, an embodiment of the present application further provides a chip system, which comprises:

[0198] The command processor is configured to receive a start instruction corresponding to the kernel function, generate a thread block configuration instruction according to the start instruction, and send the thread block configuration instruction to the metathread instruction scheduler.

[0199] The metathread instruction scheduler is in communication connection with the command processor, and is configured to receive the thread block configuration instruction and configure a thread block according to the thread block configuration instruction. For each thread block, the metathread instruction scheduler is further configured to generate a metathread family corresponding to the thread block, wherein the metathread family comprises a plurality of member metathreads, and one member metathread corresponds to one member function in the kernel function. The metathread instruction scheduler is further configured to run a plurality of member functions in parallel through the plurality of member metathreads.

[0200] It is worth noting that since the chip system of the present disclosure comprises the above-mentioned command processor and metathread instruction scheduler, and the command processor and metathread instruction scheduler can perform the application method of the metathread family of any of the above embodiments, the specific implementation and technical effects of the chip system of the present disclosure can refer to the specific implementation and technical effects of the application method of the metathread family of any of the above embodiments.

[0201] In addition, an embodiment of the present application further provides a computer readable storage medium, which stores computer executable instructions for executing the above-mentioned application method of the metathread family. Exemplarily, the method steps in the above description are executed. Figures 3 to 12

[0202] It is worth noting that since the computer readable storage medium of the present disclosure can perform the application method of the metathread family of any of the above embodiments, the specific implementation and technical effects of the computer readable storage medium of the present disclosure can refer to the specific implementation and technical effects of the application method of the metathread family of any of the above embodiments.

[0203] ​Further, one embodiment of the present application also provides a computer program product comprising computer program or computer instructions stored in a computer readable storage medium, wherein the processor of the computer device reads the computer program or computer instructions from the computer readable storage medium, and the processor executes the computer program or computer instructions, so that the computer device executes the application method of the meta-thread family as described above. Exemplarily, the method steps in the above description are executed. Figures 3 to 12

[0204] It is worth noting that since the computer program product of the present application can execute the application method of the meta-thread family of any one of the above embodiments, the specific implementation and technical effects of the computer program product of the present application can refer to the specific implementation and technical effects of the application method of the meta-thread family of any one of the above embodiments.

[0205] Those of ordinary skill in the art can understand that all or some of the steps in the above disclosed method and system can be implemented as software, firmware, hardware and their appropriate combinations. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on a computer readable medium, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those of ordinary skill in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as known to those of ordinary skill in the art, communication media generally includes computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and can include any information delivery medium.

[0206] ​The terms "first", "second", "third", "fourth" and the like in the description of the disclosure and the above drawings, if any, are used to distinguish similar objects, and do not necessarily have to be described in a particular order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the disclosure described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "contain" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device containing a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0207] It should be understood that in the present disclosure, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0208] It should be understood that in the description of the embodiments of the present disclosure, the meaning of "multiple" (or "multiple items") is two or more, and greater than, less than, more than, etc. are not included in the number, and above, below, etc. are included in the number.

[0209] In several embodiments provided by the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division, and actual implementation can have another division manner. For example, multiple units or components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0210] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0211] In addition, each functional unit in various embodiments of the present disclosure can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0212] It should also be understood that various embodiments provided by the present disclosure can be combined in any manner to achieve different technical effects.

[0213] The above is a specific description of the embodiments of the present disclosure, but the present disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present disclosure, and these equivalent modifications or replacements are included in the scope defined by the claims of the present disclosure.

Claims

1. A method for applying a meta-thread family, characterized in that, Applied to a chip system, the chip system includes a command processor and a meta-thread instruction scheduler, the command processor being communicatively connected to the meta-thread instruction scheduler; the application method of the meta-thread family includes: The command processor receives a startup instruction corresponding to the kernel function, generates a thread block configuration instruction based on the startup instruction, and sends the thread block configuration instruction to the meta-thread instruction scheduler. The meta-thread instruction scheduler receives the thread block configuration instruction and configures the thread block according to the thread block configuration instruction. For each thread block, the meta-thread instruction scheduler generates a meta-thread family corresponding to the thread block, wherein the meta-thread family includes multiple member meta-threads, and one member meta-thread corresponds to a member function in the kernel function; The meta-thread instruction scheduler runs multiple member functions in parallel through multiple member meta-threads; The method for applying the meta-thread family after the meta-thread instruction scheduler generates the meta-thread family corresponding to the thread block further includes: for the meta-thread family, one of the member meta-threads is used as the main meta-thread, and the remaining member meta-threads are used as secondary meta-threads, wherein the main meta-thread is used to manage one or more of the secondary meta-threads. In addition, the meta-thread instruction scheduler generates a meta-thread family corresponding to the thread block, including one of the following: The meta-thread instruction scheduler synchronously starts the main meta-thread and multiple sub-threads; After the meta-thread instruction scheduler starts the main meta-thread, it sequentially starts multiple sub-threads through the main meta-thread.

2. The application method of the meta-thread family according to claim 1, characterized in that, The dimensional thread includes one of the following: The storage meta-thread is loaded, corresponding to the loading function in the kernel function; The MMA meta-thread corresponds to the MMA function in the kernel function; The vector meta-thread corresponds to the vector computation function in the kernel function; The communication meta-thread corresponds to the communication operation function in the kernel function.

3. The application method of the meta-thread family according to claim 2, characterized in that, The member functions are identified through one of the following steps: Determine the hint information in the kernel function, and identify multiple member functions from the kernel function based on the hint information; The kernel function is compiled by a compiler, and multiple member functions are identified from the kernel function.

4. The application method of the meta-thread family according to claim 1, characterized in that, The meta-thread instruction scheduler uses multiple member meta-threads to run multiple member functions in parallel, including: The meta-thread instruction scheduler determines the current state of the barrier object; When the barrier object meets the preset synchronization conditions, the meta-thread instruction scheduler synchronously executes multiple member meta-threads to achieve parallel execution of multiple member functions.

5. The application method of the meta-thread family according to claim 1, characterized in that, After the meta-thread instruction scheduler generates the meta-thread family corresponding to the thread block, the application method of the meta-thread family further includes one of the following: Allocate an equal number of scalar general-purpose registers to the multiple member threads; According to a preset allocation principle, a corresponding number of scalar general-purpose registers are allocated to each of the multiple member threads, wherein all the scalar general-purpose registers are non-overlapping and located in the same scalar general-purpose register pool.

6. The application method of the meta-thread family according to claim 5, characterized in that, The step of allocating a corresponding number of scalar general-purpose registers to the multiple member threads according to a preset allocation principle includes: Obtain the historical usage parameters of each member thread, wherein the historical usage parameters include historical usage frequency or historical usage count; Different numbers of scalar general-purpose registers are allocated to different member threads based on the historical usage parameters, wherein the historical usage parameters and the number are positively correlated.

7. The application method of the meta-thread family according to claim 1, characterized in that, After the meta-thread instruction scheduler generates the meta-thread family corresponding to the thread block, the application method of the meta-thread family further includes one of the following: The meta-thread instruction scheduler schedules multiple member meta-threads to the same scalar unit so that the multiple member meta-threads run sequentially in a preset order; The meta-thread instruction scheduler schedules multiple member meta-threads to multiple different scalar units, wherein one scalar unit corresponds to one or more member meta-threads, and multiple member meta-threads located in the same scalar unit run sequentially in a preset order.

8. The application method of the meta-thread family according to claim 1, characterized in that, The chip system further includes multiple functional units, all of which are communicatively connected to the meta-thread instruction scheduler; the meta-thread instruction scheduler uses multiple member meta-threads to run multiple member functions in parallel, including: The meta-thread instruction scheduler sends multiple member meta-threads to multiple different functional units, and executes the multiple member meta-threads through the multiple different functional units. The functional units include a communication operation unit, a data copying unit, a tensor calculation unit, a scalar operation unit, or an execution unit.

9. A chip system, characterized in that, include: The command processor is used to receive the startup instruction corresponding to the kernel function, generate a thread block configuration instruction according to the startup instruction, and send the thread block configuration instruction to the meta-thread instruction scheduler. A meta-thread instruction scheduler, communicatively connected to the command processor, is configured to receive thread block configuration instructions and configure thread blocks according to these instructions. For each thread block, the meta-thread instruction scheduler further generates a meta-thread family corresponding to the thread block, wherein the meta-thread family includes multiple member meta-threads, and each member meta-thread corresponds to a member function within the kernel function. The meta-thread instruction scheduler is also configured to run multiple member functions in parallel using multiple member meta-threads. The meta-thread instruction scheduler is further configured to designate one of the member meta-threads as the main meta-thread and the remaining member meta-threads as secondary meta-threads, wherein the main meta-thread is configured to manage one or more of the secondary meta-threads. In addition, the meta-thread instruction scheduler is also used to generate a meta-thread family corresponding to the thread block, including one of the following: The meta-thread instruction scheduler synchronously starts the main meta-thread and multiple sub-threads; After the meta-thread instruction scheduler starts the main meta-thread, it sequentially starts multiple sub-threads through the main meta-thread.

10. An electronic device, characterized in that, The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. The program is executed by the processor to implement the application method of the meta-thread family as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the application method of the meta-thread family as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program or computer instructions, characterized in that, The computer program or the computer instructions are stored in a computer-readable storage medium, the processor of the computer device reads the computer program or the computer instructions from the computer-readable storage medium, and the processor executes the computer program or the computer instructions to cause the computer device to perform the application method of the meta-thread family as described in any one of claims 1 to 8.

Images