Artificial intelligence chip and cooperative thread bundle computing method

Abstract

The application provides an artificial intelligence chip and a cooperative thread bundle calculation method. The cooperative thread bundle calculation method comprises the following steps: initializing a plurality of thread bundle units by a thread bundle resource allocation unit, so that the plurality of thread bundle units have the same initial vector register base address to form a group of cooperative thread bundles; informing a thread bundle scheduling and instruction emission unit to send a plurality of execution instructions to an execution unit by the thread bundle resource allocation unit; running a plurality of thread bundles of the plurality of thread bundle units according to the plurality of execution instructions by the execution unit; reading different scalar parameters from respective scalar registers of the plurality of thread bundle units by the plurality of thread bundles respectively; and accessing the same register space of the vector register according to the same initial vector register base address by the plurality of thread bundles. The artificial intelligence chip and the cooperative thread bundle calculation method can realize the cooperative thread bundle calculation function of efficiently sharing the vector register group across the thread bundles.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of integrated circuits, and in particular to an artificial intelligence (AI) chip and an operation method thereof. BACKGROUND

[0002] Single Instruction Multiple Threads (SIMT) vector instructions are a common graphics processing unit (GPU) programming instruction and have been widely used in large-scale parallel computing tasks such as graphics rendering and artificial intelligence (AI) operations. SIMT instructions perform parallel operations in a basic unit of a thread bundle (Warp). A thread group includes multiple threads. A GPU or AI chip can thereby process multiple data points or computing tasks at the same time.

[0003] However, existing GPUs or AI chips only support a mode in which vector register regions of different thread bundles are independent of each other. If cross-thread bundle data exchange is to be performed, an external memory module needs to be used, which results in a large data transmission delay, low efficiency, and is not conducive to a "consumer-producer" collaboration mode between thread bundles.

[0004] In addition, existing GPUs or AI chips only support a fixed mode in which the sizes of vector register regions of different thread bundles are fixed, and thus cannot meet the needs of some memory instructions and tensor core instruction buffer regions that are large, and thus more slice calculations are required, which results in more related execution instructions, higher computing requirements, and longer execution time.

[0005] In addition, the programs run by existing GPUs or AI chips between different thread bundles are not the same, which is not conducive to better throughput of parallel computing and increases programming difficulty. SUMMARY

[0006] The present application is directed to an artificial intelligence (AI) chip and an operation method thereof to improve execution efficiency.

[0007] According to an embodiment of the present application, an artificial intelligence chip is provided, comprising:

[0008] a plurality of thread bundle units respectively corresponding to a plurality of thread bundles forming a group of collaborative thread bundles and respectively including different scalar registers;

[0009] a thread bundle resource allocation unit coupled to the plurality of thread bundle units;

[0010] an execution unit coupled to the plurality of threadlet units;

[0011] a threadlet scheduling and instruction issue unit coupled to the execution unit and the threadlet resource allocation unit;

[0012] a vector register coupled to the execution unit;

[0013] wherein the threadlet resource allocation unit is configured to initialize the plurality of threadlet units to have a same initial vector register base address, and the threadlet resource allocation unit is further configured to inform the threadlet scheduling and instruction issue unit to send a plurality of execution instructions to the execution unit;

[0014] wherein the execution unit is configured to run the plurality of threadlets according to the plurality of execution instructions, each of the plurality of execution instructions corresponding to a corresponding one of the plurality of threadlets;

[0015] wherein the plurality of threadlets are configured to read different scalar parameters from scalar registers of corresponding ones of the plurality of threadlet units, and the plurality of threadlets are further configured to access a same register space in the vector register by default according to the same initial vector register base address.

[0016] In some examples, optionally, the plurality of threadlets run a same program file.

[0017] In some examples, optionally, the plurality of threadlets include a first threadlet and a second threadlet, and the first threadlet and the second threadlet are configured to execute a same data load program segment in the program file to load data read from different memory spaces to different locations in a register space corresponding to the initial vector register base address with different scalar parameters read from scalar registers of corresponding ones of the plurality of threadlet units.

[0018] In some examples, optionally, further comprising a threadlet synchronization unit coupled to the threadlet scheduling and instruction issue unit; wherein the plurality of threadlets further include a third threadlet, and the threadlet synchronization unit is configured to receive a barrier instruction of the third threadlet to perform a threadlet barrier check on the third threadlet according to the barrier instruction of the third threadlet; wherein the threadlet synchronization unit is further configured to determine that the third threadlet passes the threadlet barrier check in response to at least one of the first threadlet and the second threadlet completing execution, and send an execution instruction of the plurality of execution instructions corresponding to the third threadlet to the execution unit to cause the execution unit to run the third threadlet.

[0019] In some examples, optionally, the plurality of thread bundles further comprises a third thread bundle configured to execute a computation program segment in the program file to perform parallel computation on data loaded by at least one of the first thread bundle and the second thread bundle into a register space corresponding to the initial vector register base address, with scalar parameters read from scalar registers of a corresponding one of the plurality of thread bundle units.

[0020] In some examples, optionally, the plurality of thread bundles further comprises a third thread bundle and a fourth thread bundle, and the third thread bundle and the fourth thread bundle are configured to execute a same computation program segment in the program file to perform parallel computation on data loaded by at least one of the first thread bundle and the second thread bundle into a register space corresponding to the initial vector register base address, with different scalar parameters read from scalar registers of corresponding thread bundle units of the plurality of thread bundle units respectively, and write different computation results to different memory spaces.

[0021] In some examples, optionally, the plurality of thread bundles further comprises a fifth thread bundle and a sixth thread bundle, and the fifth thread bundle and the sixth thread bundle are configured to execute a base address configuration program segment in the program file to issue a base address configuration instruction; the execution unit is further configured to, according to the base address configuration instruction, notify thread bundle units of the plurality of thread bundle units corresponding to the fifth thread bundle and the sixth thread bundle to modify the initial vector register base address, so that the fifth thread bundle and the sixth thread bundle switch from default accessing a register space corresponding to the initial vector register base address to accessing another memory space corresponding to a modified vector register base address.

[0022] In some examples, optionally, the fifth thread bundle and the sixth thread bundle are further configured to execute the data loading program segment in the program file to load data read from different memory spaces into different locations in another register space corresponding to the modified vector register base address, with different scalar parameters read from scalar registers of corresponding thread bundle units of the plurality of thread bundle units respectively.

[0023] In some examples, optionally, the plurality of thread bundles further comprises a seventh thread bundle and an eighth thread bundle, the seventh thread bundle and the eighth thread bundle are configured to execute the base address configuration program segment in the program file to issue another base address configuration instruction; the execution unit, according to the another base address configuration instruction, informs the thread bundle units in the plurality of thread bundle units corresponding to the seventh thread bundle and the eighth thread bundle to modify the initial vector register base address, so that the seventh thread bundle and the eighth thread bundle switch from default accessing a register space corresponding to the initial vector register base address to accessing another storage space corresponding to the modified vector register base address; the seventh thread bundle and the eighth thread bundle are further configured to execute the calculation program segment in the program file to perform parallel calculation according to data loaded by at least one of the fifth thread bundle and the sixth thread bundle to another register space corresponding to the modified vector register base address with different scalar parameters respectively read from the scalar registers of the corresponding thread bundle units in the plurality of thread bundle units, and write different calculation results obtained by calculation to different memory spaces.

[0024] In some examples, optionally, the execution unit comprises a vector core module, a memory module and a tensor core module, and the plurality of execution instructions comprises a vector core instruction, a memory core instruction and a tensor core instruction.

[0025] According to embodiments of the present application, a cooperative thread bundle calculation method is provided, comprising:

[0026] A plurality of thread bundle units are initialized by a thread bundle resource allocation unit, so that the plurality of thread bundle units have the same initial vector register base address; wherein the plurality of thread bundle units respectively correspond to a plurality of thread bundles forming a group of cooperative thread bundles and respectively comprise different scalar registers;

[0027] A thread bundle scheduling and instruction emission unit is informed by the thread bundle resource allocation unit to send a plurality of execution instructions to an execution unit;

[0028] The plurality of thread bundles are run by the execution unit according to the plurality of execution instructions, wherein each of the plurality of execution instructions corresponds to a corresponding one of the plurality of thread bundles;

[0029] Different scalar parameters are read by the plurality of thread bundles from the scalar registers of the corresponding plurality of thread bundle units respectively;

[0030] The same register space in the vector register is accessed by the plurality of thread bundles according to the same initial vector register base address by default.

[0031] In some examples, optionally, the plurality of thread bundles run the same program file.

[0032] In some examples, optionally, the plurality of thread bundles includes a first thread bundle and a second thread bundle, and the cooperative thread bundle computing method further includes: executing, by the first thread bundle and the second thread bundle, a same data loading program segment in the program file; and loading, by the first thread bundle and the second thread bundle, data read from different memory spaces into different locations in a register space corresponding to the initial vector register base address with different scalar parameters respectively read from scalar registers of corresponding thread bundle units in the plurality of thread bundle units.

[0033] In some examples, optionally, the plurality of thread bundles further includes a third thread bundle, and the cooperative thread bundle computing method further includes: receiving, by a thread bundle synchronization unit, a barrier instruction of the third thread bundle to perform thread bundle barrier detection on the third thread bundle according to the barrier instruction of the third thread bundle; and determining, by the thread bundle synchronization unit, that the third thread bundle passes the thread bundle barrier detection in response to at least one of the first thread bundle and the second thread bundle completing execution, and sending a plurality of execution instructions corresponding to the third thread bundle in the plurality of execution instructions to the execution unit to cause the execution unit to run the third thread bundle.

[0034] In some examples, optionally, the plurality of thread bundles further includes a third thread bundle, and the cooperative thread bundle computing method further includes: executing, by the third thread bundle, a computing program segment in the program file to perform computation according to data loaded by at least one of the first thread bundle and the second thread bundle into a register space corresponding to the initial vector register base address with a scalar parameter read from a scalar register of a corresponding one of the plurality of thread bundle units.

[0035] In some examples, optionally, the plurality of thread bundles further includes a third thread bundle and a fourth thread bundle, and the cooperative thread bundle computing method further includes: executing, by the third thread bundle and the fourth thread bundle, a same computing program segment in the program file to perform parallel computation according to data loaded by at least one of the first thread bundle and the second thread bundle into a register space corresponding to the initial vector register base address with different scalar parameters respectively read from scalar registers of corresponding thread bundle units in the plurality of thread bundle units; and writing, by the third thread bundle and the fourth thread bundle, different computation results obtained by the computation into different memory spaces.

[0036] In some examples, optionally, the plurality of thread bundles further comprises a fifth thread bundle and a sixth thread bundle, and the cooperative thread bundle computing method further comprises: executing, by the fifth thread bundle and the sixth thread bundle, a base address configuration program segment in the same program file to issue a base address configuration instruction; and notifying, by the execution unit according to the base address configuration instruction, thread bundle units corresponding to the fifth thread bundle and the sixth thread bundle in the plurality of thread bundle units to modify the initial vector register base address, so that the fifth thread bundle and the sixth thread bundle switch from default access to register space corresponding to the initial vector register base address to access another storage space corresponding to the modified vector register base address.

[0037] In some examples, optionally, the cooperative thread bundle computing method further comprises: executing, by the fifth thread bundle and the sixth thread bundle, the data loading program segment in the program file; and loading, by the fifth thread bundle and the sixth thread bundle, data read from different memory spaces to different positions in the register space corresponding to the modified vector register base address with different scalar parameters respectively read from scalar registers of corresponding thread bundle units in the plurality of thread bundle units.

[0038] In some examples, optionally, the plurality of thread bundles further comprises a seventh thread bundle and an eighth thread bundle, and the cooperative thread bundle computing method further comprises: executing, by the seventh thread bundle and the eighth thread bundle, the base address configuration program segment in the program file to issue another base address configuration instruction; notifying, by the execution unit according to the another base address configuration instruction, thread bundle units corresponding to the seventh thread bundle and the eighth thread bundle in the plurality of thread bundle units to modify the initial vector register base address, so that the seventh thread bundle and the eighth thread bundle switch from default access to register space corresponding to the initial vector register base address to access another storage space corresponding to the modified vector register base address; executing, by the seventh thread bundle and the eighth thread bundle, the computing program segment in the program file to perform parallel computation according to data loaded by at least one of the fifth thread bundle and the sixth thread bundle to another register space corresponding to the modified vector register base address with different scalar parameters respectively read from scalar registers of corresponding thread bundle units in the plurality of thread bundle units; and writing, by the seventh thread bundle and the eighth thread bundle, different computation results obtained by computation to different memory spaces.

[0039] Based on the above, the artificial intelligence chip and the cooperative thread bundle computing method can realize that different thread bundles can access the same vector register region, and efficiently access registers across thread bundles without the help of slower memory (Memory) for data interaction. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 is a circuit block diagram of an artificial intelligence (AI) chip according to an embodiment of the present application;

[0041] Figure 2 is a flow diagram of a cooperative thread bundle computing method according to an embodiment of the present application;

[0042] Figure 3 is a diagram of a program file according to an embodiment of the present application;

[0043] Figure 4 is a programming model diagram of a program file according to an embodiment of the present application;

[0044] Figure 5 is an operation diagram of a plurality of thread bundles according to an embodiment of the present application;

[0045] Figure 6 is a use diagram of a vector register and a memory module according to an embodiment of the present application.

[0046] BRIEF DESCRIPTION OF DRAWINGS

[0047] 100: AI chip;

[0048] 110: thread bundle resource allocation unit;

[0049] 120_1: 1st thread bundle unit;

[0050] 120_M: Mth thread bundle unit;

[0051] 121_1: 1st scalar register;

[0052] 121_M: Mth scalar register;

[0053] 122_1: 1st vector register base address;

[0054] 122_M: Mth vector register base address;

[0055] 130: execution unit;

[0056] 131: vector core module;

[0057] 132: memory module;

[0058] 133: tensor core module;

[0059] 140: thread bundle scheduling and instruction issue unit;

[0060] 150: vector register;

[0061] 160: thread bundle synchronization unit;

[0062] 300: program file;

[0063] 310: base address configuration program section;

[0064] 320: data load program section;

[0065] 330: computation program section;

[0066] 400: corresponding program section;

[0067] 520_1: first thread bundle;

[0068] 520_2: second thread bundle;

[0069] 520_3: third thread bundle;

[0070] 520_4: fourth thread bundle;

[0071] 520_5: fifth thread bundle;

[0072] 520_6: sixth thread bundle;

[0073] 520_7: seventh thread bundle;

[0074] 520_8: eighth thread bundle;

[0075] 651: first vector register region;

[0076] 652: second vector register region;

[0077] 653: third vector register region;

[0078] 654: fourth vector register region;

[0079] 632_1: first processing region;

[0080] 632_2: second processing region. DETAILED DESCRIPTION

[0081] Reference will now be made in detail to the exemplary embodiments of the present application, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used in the different drawings and the description to refer to the same or like parts.

[0082] The term "coupled (or connected)" as used throughout this specification (including the claims) may refer to any direct or indirect means of connection. For example, if the text describes a first device coupled (or connected) to a second device, it should be interpreted as the first device being directly connected to the second device, or the first device being indirectly connected to the second device through other devices or some means of connection. The terms "first," "second," etc., used throughout this specification (including the claims) are used to name components or distinguish different embodiments or scopes, and are not intended to limit the upper or lower limit of the number of components, nor to limit the order of components. Furthermore, wherever possible, components / components / steps using the same reference numerals in the drawings and embodiments represent the same or similar parts. Components / components / steps using the same reference numerals or the same terms in different embodiments may be referred to mutually in the relevant descriptions. It should be understood that the features of the following embodiments can be combined with each other. For example, the features of the second embodiment can be combined with the features of the first embodiment. Those skilled in the art can select appropriate combinations of features according to actual design requirements.

[0083] Figure 1 This is a schematic diagram of a circuit block of an artificial intelligence (AI) chip according to an embodiment of the present invention. Figure 1 The AI ​​chip 100 includes a warp resource allocation unit 110 and multiple warp units (e.g., Figure 1 The system includes thread bundle units 120_1 to Mth thread bundle units 120_M, execution unit 130, thread bundle scheduling and instruction issuing unit 140, vector register 150, and thread bundle synchronization unit 160, where M is a positive integer. Multiple thread bundle units (e.g., ...) Figure 1 The first thread bundle unit 120_1 to the Mth thread bundle unit 120_M respectively correspond to multiple thread bundles forming a group of cooperative thread bundles and each includes different scalar registers (e.g., Figure 1 The first scalar register 121_1 to the Mth scalar register 121_M). Or, multiple thread bundle units (e.g. Figure 1 The first thread bundle unit 120_1 to the Mth thread bundle unit 120_M can also be referred to as multiple thread bundle configuration units corresponding to multiple thread bundles respectively. The execution unit 130 includes a vector kernel module 131, a memory module 132, and a tensor kernel module 133. The thread bundle resource allocation unit 110 is coupled to the thread bundle units (e.g., ...). Figure 1 The first thread bundle unit 120_1 to the Mth thread bundle unit 120_M). Execution unit 130 is coupled to the thread bundle unit (e.g., Figure 1The first thread bundle unit 120_1 to the Mth thread bundle unit 120_M). The thread bundle scheduling and instruction issuing unit 140 is coupled to the execution unit 130 and the thread bundle resource allocation unit 110. The vector register 150 is coupled to the execution unit 130. The thread bundle synchronization unit 160 is coupled to the thread bundle scheduling and instruction issuing unit 140.

[0084] Depending on the design, in some embodiments, the thread bundle resource allocation unit 110 and multiple thread bundle units (e.g.) Figure 1 At least one of the following: the first thread beam unit 120_1 to the Mth thread beam unit 120_M; the execution unit 130; the thread beam scheduling and instruction issuing unit 140; the vector register 150; and the thread beam synchronization unit 160 can be implemented as hardware circuitry. In other embodiments, the thread beam resource allocation unit 110 and multiple thread beam units (e.g., ...) Figure 1 The implementation of at least one of the following can be a combination of hardware, firmware, and software (i.e., program): thread bundle unit 120_1 to thread bundle unit M 120_M, execution unit 130, thread bundle scheduling and instruction issuing unit 140, vector register 150, and thread bundle synchronization unit 160.

[0085] In hardware terms, the aforementioned thread bundle resource allocation unit 110 and multiple thread bundle units (e.g.) Figure 1 At least one of the following—the first thread bundle unit 120_1 to the Mth thread bundle unit 120_M), the execution unit 130, the thread bundle scheduling and instruction issuing unit 140, the vector register 150, and the thread bundle synchronization unit 160—can be implemented on an integrated circuit as logic circuitry or register circuitry. For example, the thread bundle resource allocation unit 110, multiple thread bundle units (e.g., ...) Figure 1The functions of at least one of the following: the first thread bundle unit 120_1 to the Mth thread bundle unit 120_M), the execution unit 130, the thread bundle scheduling and instruction issuing unit 140, the vector register 150, and the thread bundle synchronization unit 160 can be implemented in various logic blocks, modules, and circuits in one or more hardware controllers, microcontrollers, hardware processors, microprocessors, application-specific integrated circuits (ASICs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), central processing units (CPUs), or other processing units. The thread bundle resource allocation unit 110, multiple thread bundle units (e.g., Figure 1 The related functions of at least one of the following: the first thread bundle unit 120_1 to the Mth thread bundle unit 120_M), the execution unit 130, the thread bundle scheduling and instruction issuing unit 140, the vector register 150, and the thread bundle synchronization unit 160 can be implemented as hardware circuits, such as various logic blocks, modules, and circuits in an integrated circuit, using hardware description languages ​​(such as Verilog HDL or VHDL) or other suitable programming languages.

[0086] In software or firmware form, the aforementioned thread bundle resource allocation unit 110 and multiple thread bundle units (e.g.) Figure 1 The functions of at least one of the following: the first thread bundle unit 120_1 to the Mth thread bundle unit 120_M; the execution unit 130; the thread bundle scheduling and instruction issuing unit 140; the vector register 150; and the thread bundle synchronization unit 160 can be implemented as programming codes. For example, the thread bundle resource allocation unit 110 and the multiple thread bundle units (e.g., ...) described above can be implemented using general programming languages ​​(such as C, C++, or assembly language) or other suitable programming languages. Figure 1The programmable code can be recorded / stored in a non-transitory machine-readable storage medium. In some embodiments, the non-transitory machine-readable storage medium includes, for example, semiconductor memory and / or storage devices. An electronic device (e.g., CPU, hardware controller, microcontroller, hardware processor, or microprocessor) can read and execute the programmable code from the non-transitory machine-readable storage medium to implement the above-described thread bundle resource allocation unit 110 and multiple thread bundle units (e.g., ... Figure 1 The functions of at least one of the following: the first thread beam unit 120_1 to the Mth thread beam unit 120_M), the execution unit 130, the thread beam scheduling and instruction issuing unit 140, the vector register 150, and the thread beam synchronization unit 160.

[0087] Figure 2 This is a flowchart illustrating a cooperative thread bundle calculation method according to an embodiment of the present invention. Please refer to... Figure 1 and Figure 2 In step S210, the thread bundle resource allocation unit 110 initializes multiple thread bundle units (e.g., Figure 1 The first thread bundle unit 120_1 to the Mth thread bundle unit 120_M), so that multiple thread bundle units (e.g. Figure 1 The first thread bundle unit 120_1 to the Mth thread bundle unit 120_M) have the same initial vector register base address (e.g. Figure 1 The base addresses of the first vector register (122_1 to the Mth vector register (122_M) are all initial vector register base addresses). The thread bundle resource allocation unit 110 can receive the initialization request of each thread bundle and allocate a logical thread bundle number (warp ID). After initialization is completed, in step S220, the thread bundle resource allocation unit 110 notifies the thread bundle scheduling and instruction issuing unit 140 to send multiple execution instructions to the execution unit. The multiple execution instructions may include vector kernel instructions, memory instructions, and tensor kernel instructions, and are respectively provided to the vector kernel module 131, memory module 132, and tensor kernel module 133 of the execution unit 130.

[0088] In step S230, execution unit 130 executes multiple thread bundle units (e.g., according to multiple execution instructions) based on multiple execution instructions. Figure 1 Multiple thread bundles (from the first thread bundle unit 120_1 to the Mth thread bundle unit 120_M). Each input thread bundle unit of multiple execution instructions (e.g., ...)Figure 1 The first thread bundle unit 120_1 to the Mth thread bundle unit 120_M) corresponds to one of them, so that the execution unit 130 runs the corresponding thread bundle unit (e.g., ...). Figure 1 Multiple thread bundles (from the first thread bundle unit 120_1 to the Mth thread bundle unit 120_M). In other words, each of the multiple execution instructions corresponds to one of the multiple thread bundles. In step S240, the multiple thread bundles are respectively generated from the multiple thread bundle units (e.g., ... Figure 1 The scalar registers (i.e., the scalar registers of the first thread beam unit 120_1 to the Mth thread beam unit 120_M) Figure 1 The first scalar register 121_1 to the Mth scalar register 121_M read different scalar parameters. In step S250, multiple thread bundles read different scalar parameters based on the same initial vector register base address (e.g., ...). Figure 1 All of them are initialized to the base address of the initial vector register (base address of the first vector register 122_1 to the base address of the Mth vector register 122_M). The access of multiple thread bundles in step S250 is a default access method; that is, in step S250, multiple thread bundles by default access the same register space of the vector register 150 based on the same initial vector register base address (e.g., base address of the first vector register 122_1 to the base address of the Mth vector register 122_M). Figure 1 The vector registers 122_1 to 122_M, which are all initialized to the base address of the initial vector register, access the same register space as the vector register 150.

[0089] In response, multiple thread bundle units (e.g.) Figure 1 Each of the first thread bundle unit 120_1 to the Mth thread bundle unit 120_M has a private scalar register that stores scalar data related to the thread bundle. Because scalar registers (e.g. Figure 1 The first scalar register 121_1 to the Mth scalar register 121_M respectively store different scalar parameters, therefore multiple thread bundle units (e.g. Figure 3 In the working mode of working as a cooperative thread bundle, the first thread bundle unit 120_1 to the Mth thread bundle unit 120_M can execute the same instructions but operate on different threads or data.

[0090] Furthermore, at least one of the vector kernel module 131, memory module 132, and tensor kernel module 133 can access the vector register 150 based on the vector register base address of the currently received instruction thread bundle number. If the base addresses of different module thread bundles are the same, it means that these thread bundles can operate on the same register region of the vector register 150 in parallel. In this way, multiple cooperating thread bundles can share the resource space of the vector register 150, allowing a single thread bundle to access a larger register space for the data buffer of memory module 132 or tensor kernel module 133, thereby achieving efficient and low-latency data interaction.

[0091] Figure 1 This is a schematic diagram of a program file according to an embodiment of the present invention. Please refer to... Figure 3 and Figure 1 Execution unit 130 is based on multiple thread bundle units (e.g. Figure 1 Multiple thread bundles (from the first thread bundle unit 120_1 to the Mth thread bundle unit 120_M) can run the same program file. For this, corresponding to multiple thread bundle units (e.g., ...) Figure 3 Multiple thread bundles (from the first thread bundle unit 120_1 to the Mth thread bundle unit 120_M) can respectively execute corresponding program segments in the program file 300, where different program segments correspond to different application requirements, so that different thread bundles can execute instruction sequences to implement different application requirements. Figure 1 For example, program file 300 includes a base address configuration program segment 310, a data loading program segment 320, and a calculation program segment 330. This corresponds to multiple thread bundle units (e.g., Figure 1 A portion of the multiple thread bundles (from the first thread bundle unit 120_1 to the Mth thread bundle unit 120_M) can execute the instruction sequence of the base address configuration program segment 310 to issue a base address configuration instruction to the execution unit 130, so that the execution unit 130 notifies the corresponding thread bundle unit to modify the base address of the initial vector register according to the base address configuration instruction. This corresponds to multiple thread bundle units (e.g., ... Figure 1 Another portion of the thread bundles in the multiple thread bundles (e.g., the first thread bundle unit 120_1 to the Mth thread bundle unit 120_M) can execute the instruction sequence of the data loading program segment 320 to perform a data loading operation. Corresponding to multiple thread bundle units (e.g., ... Figure 4 A portion of the multiple thread bundles (from the first thread bundle unit 120_1 to the Mth thread bundle unit 120_M) can execute the instruction sequence of the computation program segment 330 to perform data computation operations.

[0092] Figure 5 This is a schematic diagram of a programming model of a program file according to an embodiment of the present invention. Figure 1This is a schematic diagram illustrating the operation of multiple thread bundles according to an embodiment of the present invention. Please refer to... Figure 4 , Figure 5 and Figure 4 Taking the execution of 8 thread bundles (i.e., thread bundle 520_1 to thread bundle 520_8) as an example, thread bundles 520_1 to 520_8 are pre-assigned to the same initial vector register base address, forming a group of cooperative thread bundles. Thread bundles 520_1 to 520_8 can execute according to their respective thread bundle numbers (warp IDs) as follows: Figure 1 The corresponding program segment 400 in the programming model of the program file 300 shown. Specifically, the fifth thread bundle 520_5 to the eighth thread bundle 520_8 executes the instruction sequence of the base address configuration program segment to issue a base address configuration instruction to the execution unit 130, so that the execution unit 130 notifies the corresponding thread bundle unit to modify the base address of the initial vector register according to the base address configuration instruction (e.g., notifying...). Figure 5 The fifth thread bundle unit 120_5 to the eighth thread bundle unit 120_8 modifies the base address of the fifth vector register 122_5 to the base address of the eighth vector register 122_8, thereby switching the fifth thread bundle 520_5 to the eighth thread bundle 520_8 from accessing the register space corresponding to the initial vector register base address by default to accessing another storage space corresponding to the modified vector register base address. Figure 1 As shown, the fifth thread bundle 520_5 to the eighth thread bundle 520_8 can simultaneously perform base address configuration operations to modify the base address of the initial vector register. That is, in addition to performing the default access in step S250, at least one of the thread bundles can also temporarily switch to accessing another register space of the vector register based on the modified vector register base address for a certain period of time by performing the base address configuration operation, so as to realize a temporary access mode different from the default access mode.

[0093] The first thread bundle 520_1, the second thread bundle 520_2, the fifth thread bundle 520_5, and the sixth thread bundle 520_6 execute data loading program segments. The third thread bundle 520_3, the fourth thread bundle 520_4, the seventh thread bundle 520_7, and the eighth thread bundle 520_8 execute computation program segments 330. The first thread bundle 520_1 and the second thread bundle 520_2 execute the same data loading program segments in the corresponding program segments 400 of the programming model of the program file 300. The first thread bundle 520_1 and the second thread bundle 520_2 use different scalar parameters to load data from different memory spaces (i.e.,...). Figure 1 The data read from the memory space of memory module 132 is loaded into the register space corresponding to the base address of the initial vector register (i.e., Figure 1different locations in the register space of the vector register 150.

[0094] Then, the thread bundle synchronization unit 160 receives a barrier instruction of the third thread bundle 520_3 to detect whether at least one of the first thread bundle 520_1 and the second thread bundle 520_2 is executed according to the barrier instruction, to achieve the thread bundle barrier detection of the third thread bundle 520_3. The thread bundle scheduling and instruction issuing unit 140 determines that the third thread bundle 520_3 passes the thread bundle barrier detection in response to the aforementioned at least one of the first thread bundle 520_1 and the second thread bundle 520_2 being executed, and sends the execution instructions corresponding to the third thread bundle 520_3 in the plurality of execution instructions to the execution unit 130, so that the execution unit 130 runs the third thread bundle 520_3. In this regard, the third thread bundle 520_3 can use the corresponding scalar parameters to perform calculations according to the data loaded by the aforementioned at least one of the first thread bundle 520_1 and the second thread bundle 520_2 to the register space corresponding to the initial vector register base address.

[0095] And, since the third thread bundle 520_3 and the fourth thread bundle 520_4 perform the same calculation program segment in the program file, parallel calculation is performed. The third thread bundle 520_3 and the fourth thread bundle 520_4 use different scalar parameters to perform parallel calculations according to the data loaded by the first thread bundle 520_1 and the second thread bundle 520_2 to the register space corresponding to the initial vector register base address, and write different calculation results obtained by the calculations to different memory spaces.

[0096] In this regard, the third thread bundle 520_3 and the fourth thread bundle 520_4 can perform parallel vector calculations (such as floating point calculations, fixed point calculations, or logical operations, etc.), and need to simultaneously wait for the loading data of two data sources loaded by the first thread bundle 520_1 and the second thread bundle 520_2 from the memory module 132, so the third thread bundle 520_3 and the fourth thread bundle 520_4 need to use barrier instructions to wait until the first thread bundle 520_1 and the second thread bundle 520_2 are both completed before execution.

[0097] Then, the fifth thread bundle 520_5 and the sixth thread bundle 520_6 perform the data loading program segment in the corresponding program segment 400 in the programming model of the program file 300. The fifth thread bundle 520_5 and the sixth thread bundle 520_6 can use different scalar parameters to load data read from different memory spaces (i.e. Figure 1 the memory space of the memory module 132) to the register space corresponding to the modified vector register base address (i.e. Figure 6The seventh thread bundle 520_7 and the eighth thread bundle 520_8 perform the computing program segments in the corresponding program segments 400 in the program model of the program file 300 to perform parallel computing. Like the operation mode of the third thread bundle 520_3 and the fourth thread bundle 520_4, the seventh thread bundle 520_7 and the eighth thread bundle 520_8 can utilize different scalar parameters to perform parallel computing according to the data loaded by the fifth thread bundle 520_5 and the sixth thread bundle 520_6 (or at least one of them) to the register space corresponding to the modified vector register base address, and write the different computing results obtained by the computing to different memory spaces. The seventh thread bundle 520_7 and the eighth thread bundle 520_8 may, for example, simultaneously wait for the loading data of the two data sources loaded by the fifth thread bundle 520_5 and the sixth thread bundle 520_6 from the memory module 132, so the seventh thread bundle 520_7 and the eighth thread bundle 520_8 also need to wait for the completion of the fifth thread bundle 520_5 and the sixth thread bundle 520_6 by using the barrier instruction.

[0098] Therefore, the first thread bundle 520_1 to the eighth thread bundle 520_8 can implement parallel operations on different thread bundle data, and the same computing instructions can be executed between the first thread bundle 520_1 to the eighth thread bundle 520_8 to obtain different computing results.

[0099] In addition, it is worth noting that, since the different thread bundles are respectively the producers (for example, the first thread bundle 520_1 and the second thread bundle 520_2) and the consumers (for example, the third thread bundle 520_3 and the fourth thread bundle 520_4) of the data in the cooperative thread bundle, synchronization needs to be performed to ensure correctness. The thread bundle synchronization unit 160 can receive the barrier instruction of the consumer thread bundle (for example, the third thread bundle 520_3 and the fourth thread bundle 520_4), wait for the execution of the instruction of one or more producer thread bundles (for example, the first thread bundle 520_1 and the second thread bundle 520_2) to be completed, and then make the consumer thread bundle (for example, the third thread bundle 520_3 and the fourth thread bundle 520_4) pass the thread bundle barrier detection. After passing the thread bundle barrier detection, the consumer thread bundle (for example, the third thread bundle 520_3 and the fourth thread bundle 520_4) can emit the subsequent instructions to the execution unit 130 to perform related operations.

[0100] Further, in conjunction with the reference Figure 6 , Figure 6 is a use schematic diagram of a vector register and a memory module according to an embodiment of the present application. The vector register 150 and the memory module 132 can have, for example, ​The first thread bundle 520_1 to the fourth thread bundle 520_4 can buffer and execute instructions in the first vector register region 651 and the second vector register region 652 of the vector register 150 according to the initial vector register base address, and the fifth thread bundle 520_5 to the eighth thread bundle 520_8 can buffer and execute instructions in the third vector register region 653 and the fourth vector register region 654 of the vector register 150 according to the vector register base address.

[0101] The first thread bundle 520_1 and the second thread bundle 520_2 load data to the first vector register region 651 using the first processing region 632_1 of the memory module 132, and the third thread bundle 520_3 and the fourth thread bundle 520_4 use the first vector register region 651 as input values of a calculation program, and output calculation results of the calculation program to the second vector register region 652. The third thread bundle 520_3 and the fourth thread bundle 520_4 store the calculation results stored in the second vector register region 652 to the first processing region 632_1 of the memory module 132. The fifth thread bundle 520_5 and the sixth thread bundle 520_6 load data to the third vector register region 653 using the second processing region 632_2 of the memory module 132, and the seventh thread bundle 520_7 and the eighth thread bundle 520_8 use the third vector register region 653 as input values of a calculation program, and output calculation results of the calculation program to the fourth vector register region 654. The seventh thread bundle 520_7 and the eighth thread bundle 520_8 store the calculation results stored in the fourth vector register region 654 to the second processing region 632_2 of the memory module 132.

[0102] The first thread bundle 520_1 to the fourth thread bundle 520_4 can be cooperative thread bundles, and share the use of the first vector register region 651 and the second vector register region 652 of the vector register 150 and data interaction. The third thread bundle 520_3 and the fourth thread bundle 520_4 can share the space of the first vector register region 651 to read data loaded by the first thread bundle 520_1 and the second thread bundle 520_2, and can share the space of the second vector register region 652 to store calculation results. The fifth thread bundle 520_5 to the eighth thread bundle 520_8 can be cooperative thread bundles, and share the use of the third vector register region 653 and the fourth vector register region 654 of the vector register 150 and data interaction. The seventh thread bundle 520_7 and the eighth thread bundle 520_8 can share the space of the third vector register region 653 to read data loaded by the fifth thread bundle 520_5 and the sixth thread bundle 520_6, and can share the space of the fourth vector register region 654 to store calculation results.

[0103] In summary, the artificial intelligence chip and the cooperative thread bundle calculation method can realize that different thread bundles can access the overlapped vector register area, efficiently access the register across the thread bundle, and dynamically adjust the accessed vector register area through the base address configuration instruction. In addition, in the data interaction process between the cooperative thread bundles, the data synchronization between different cooperative thread bundles can also be realized by executing the barrier instruction.

[0104] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An artificial intelligence chip, characterized in that, include: Multiple thread bundle units, each corresponding to a set of cooperating thread bundles, and each including different scalar registers; A thread bundle resource allocation unit is coupled to the plurality of thread bundle units; Execution unit, coupled to the plurality of thread bundle units; A thread beam scheduling and instruction issuing unit is coupled to the execution unit and the thread beam resource allocation unit; and Vector register, coupled to the execution unit; The thread bundle resource allocation unit is used to initialize the plurality of thread bundle units so that the plurality of thread bundle units have the same initial vector register base address, and the thread bundle resource allocation unit is also used to notify the thread bundle scheduling and instruction issuing unit to send a plurality of execution instructions to the execution unit; The execution unit is used to run the plurality of thread bundles according to the plurality of execution instructions, each of the plurality of execution instructions corresponding to one of the plurality of thread bundles; The plurality of thread bundles are configured to read different scalar parameters from the scalar registers of the corresponding plurality of thread bundle units, and the plurality of thread bundles are also configured by default to access the same register space in the vector register according to the same initial vector register base address.

2. The artificial intelligence chip according to claim 1, characterized in that, The multiple thread bundles run the same program file.

3. The artificial intelligence chip according to claim 2, characterized in that, The plurality of thread bundles includes a first thread bundle and a second thread bundle, and the first thread bundle and the second thread bundle are configured to execute the same data loading program segment in the program file to load data read from different memory spaces into different locations in the register space corresponding to the base address of the initial vector register using different scalar parameters read from the scalar registers of the corresponding thread bundle units in the plurality of thread bundle units.

4. The artificial intelligence chip according to claim 3, characterized in that, Also includes: A thread bundle synchronization unit is coupled to the thread bundle scheduling and instruction issuing unit; The plurality of thread bundles further includes a third thread bundle, and the thread bundle synchronization unit is used to receive a barrier instruction from the third thread bundle to perform thread bundle barrier detection on the third thread bundle according to the barrier instruction of the third thread bundle. The thread bundle synchronization unit is further configured to, in response to the completion of execution of at least one of the first thread bundle and the second thread bundle, determine that the third thread bundle has passed the thread bundle barrier detection, and send the execution instruction corresponding to the third thread bundle among the plurality of execution instructions to the execution unit, so that the execution unit runs the third thread bundle.

5. The artificial intelligence chip according to claim 3, characterized in that, The plurality of thread bundles further includes a third thread bundle configured to execute a computation segment in the program file to perform computation based on data loaded into a register space corresponding to the base address of the initial vector register using scalar parameters read from a corresponding scalar register of one of the plurality of thread bundle units.

6. The artificial intelligence chip according to claim 3, characterized in that, The plurality of thread bundles further include a third thread bundle and a fourth thread bundle, and the third thread bundle and the fourth thread bundle are configured to execute the same computation program segment in the program file to perform parallel computation based on data loaded from the register space corresponding to the base address of the initial vector register by at least one of the first thread bundle and the second thread bundle, using different scalar parameters read from the scalar registers of the corresponding thread bundle units in the plurality of thread bundle units, and to write the different computation results obtained to different memory spaces.

7. The artificial intelligence chip according to claim 6, characterized in that, The plurality of thread bundles also include a fifth thread bundle and a sixth thread bundle, and the fifth thread bundle and the sixth thread bundle are configured to execute the base address configuration program segment in the program file to issue a base address configuration instruction; The execution unit is further configured to, according to the base address configuration instruction, notify the thread bundle unit corresponding to the fifth thread bundle and the sixth thread bundle among the plurality of thread bundle units to modify the base address of the initial vector register, so that the fifth thread bundle and the sixth thread bundle switch from accessing the register space corresponding to the base address of the initial vector register by default to accessing another storage space corresponding to the modified vector register base address.

8. The artificial intelligence chip according to claim 7, characterized in that, The fifth and sixth thread bundles are also configured to execute the data loading program segment in the program file to load data read from different memory spaces into different locations in another register space corresponding to the modified vector register base address, using different scalar parameters read from the scalar registers of the corresponding thread bundle units in the plurality of thread bundle units.

9. The artificial intelligence chip according to claim 8, characterized in that, The plurality of thread bundles also include a seventh thread bundle and an eighth thread bundle, the seventh thread bundle and the eighth thread bundle being configured to execute the base address configuration program segment in the program file to issue another base address configuration instruction; The execution unit, according to the other base address configuration instruction, notifies the thread bundle unit corresponding to the seventh thread bundle and the eighth thread bundle among the plurality of thread bundle units to modify the base address of the initial vector register, so that the seventh thread bundle and the eighth thread bundle switch from accessing the register space corresponding to the base address of the initial vector register by default to accessing another storage space corresponding to the modified vector register base address; The seventh and eighth thread bundles are also configured to execute the computation program segment in the program file to perform parallel computation based on data loaded into another register space corresponding to the modified vector register base address by at least one of the fifth and sixth thread bundles, using different scalar parameters read from the scalar registers of the corresponding thread bundle units in the plurality of thread bundle units, and to write the different computation results obtained into different memory spaces.

10. The artificial intelligence chip according to claim 1, characterized in that, The execution unit includes a vector kernel module, a memory module, and a tensor kernel module, and the plurality of execution instructions include vector kernel instructions, memory kernel instructions, and tensor kernel instructions.

11. A method for calculating a cooperative thread bundle, characterized in that, include: Multiple thread bundle units are initialized by a thread bundle resource allocation unit so that the multiple thread bundle units have the same initial vector register base address; wherein, the multiple thread bundle units correspond to multiple thread bundles that form a set of cooperative thread bundles and each includes a different scalar register; The thread bundle resource allocation unit notifies the thread bundle scheduling and instruction issuing unit to send multiple execution instructions to the execution unit; The execution unit executes the plurality of thread bundles according to the plurality of execution instructions, wherein each of the plurality of execution instructions corresponds to one of the plurality of thread bundles; The plurality of thread bundles read different scalar parameters from the scalar registers of their respective thread bundle units; and By default, the multiple thread bundles access the same register space in the vector register based on the same initial vector register base address.

12. The method for calculating cooperative thread bundles according to claim 11, characterized in that, The multiple thread bundles run the same program file.

13. The method for calculating cooperative thread bundles according to claim 12, characterized in that, The plurality of thread bundles includes a first thread bundle and a second thread bundle, and the method for calculating the cooperative thread bundle further includes: The same data loading segment in the program file is executed by the first thread bundle and the second thread bundle; and The first thread bundle and the second thread bundle use different scalar parameters read from the scalar registers of the corresponding thread bundle units in the plurality of thread bundle units to load data read from different memory spaces into different locations in the register space corresponding to the base address of the initial vector register.

14. The method for calculating cooperative thread bundles according to claim 13, characterized in that, The plurality of thread bundles also includes a third thread bundle, and the method for calculating the cooperative thread bundle further includes: The thread bundle synchronization unit receives the barrier instruction from the third thread bundle and performs thread bundle barrier detection on the third thread bundle according to the barrier instruction; and In response to the completion of at least one of the first thread bundle and the second thread bundle, the thread bundle synchronization unit determines that the third thread bundle has passed the thread bundle barrier detection and sends multiple execution instructions corresponding to the third thread bundle from the multiple execution instructions to the execution unit, so that the execution unit runs the third thread bundle.

15. The method for calculating cooperative thread bundles according to claim 13, characterized in that, The plurality of thread bundles also includes a third thread bundle, and the method for calculating the cooperative thread bundle further includes: The third thread bundle executes the computation segment in the program file to perform computation based on data loaded into the register space corresponding to the base address of the initial vector register using scalar parameters read from the corresponding scalar register of one of the plurality of thread bundle units.

16. The method for calculating cooperative thread bundles according to claim 13, characterized in that, The plurality of thread bundles further includes a third thread bundle and a fourth thread bundle, and the method for calculating the cooperative thread bundle further includes: The third and fourth thread bundles execute the same computation segment in the program file to perform parallel computation based on data loaded from the register space corresponding to the base address of the initial vector register by at least one of the first and second thread bundles, using different scalar parameters read from the scalar registers of the corresponding thread bundle units in the plurality of thread bundle units; and The third and fourth thread bundles write the different calculation results to different memory spaces.

17. The method for calculating cooperative thread bundles according to claim 16, characterized in that, The plurality of thread bundles further includes a fifth thread bundle and a sixth thread bundle, and the method for calculating the cooperative thread bundle further includes: The fifth thread bundle and the sixth thread bundle execute the base address configuration program segment in the same program file to issue a base address configuration instruction; and The execution unit notifies the thread bundle units corresponding to the fifth and sixth thread bundles among the plurality of thread bundle units to modify the base address of the initial vector register according to the base address configuration instruction, so that the fifth and sixth thread bundles switch from accessing the register space corresponding to the base address of the initial vector register by default to accessing another storage space corresponding to the modified vector register base address.

18. The method for calculating cooperative thread bundles according to claim 17, characterized in that, The cooperative thread bundle calculation method further includes: The data loading segment in the program file is executed by the fifth thread bundle and the sixth thread bundle; and The fifth and sixth thread bundles use different scalar parameters read from the scalar registers of the corresponding thread bundle units in the plurality of thread bundle units to load data read from different memory spaces into different locations in the register space corresponding to the modified vector register base address.

19. The method for calculating cooperative thread bundles according to claim 18, characterized in that, The plurality of thread bundles further includes a seventh thread bundle and an eighth thread bundle, and the method for calculating the cooperative thread bundle further includes: The seventh thread bundle and the eighth thread bundle execute the base address configuration program segment in the program file to issue another base address configuration instruction; The execution unit notifies the thread bundle units corresponding to the seventh and eighth thread bundles among the plurality of thread bundle units to modify the base address of the initial vector register according to the other base address configuration instruction, so that the seventh and eighth thread bundles switch from accessing the register space corresponding to the base address of the initial vector register by default to accessing another storage space corresponding to the modified vector register base address; The computation segment in the program file is executed by the seventh and eighth thread bundles to perform parallel computation based on data loaded into another register space corresponding to the modified vector register base address by at least one of the fifth and sixth thread bundles, using different scalar parameters read from the scalar registers of the corresponding thread bundle units in the plurality of thread bundle units; and The seventh thread bundle and the eighth thread bundle write the different calculation results obtained to different memory spaces.

Images