Artificial intelligence chip and cooperative thread bundle computing method

Abstract

The application provides an artificial intelligence chip and a cooperative thread bundle calculation method. The artificial intelligence chip comprises M thread bundle units and a thread bundle resource allocation unit corresponding to M thread bundles respectively, wherein M is a positive integer. The thread bundle resource allocation unit is coupled with the M thread bundle units. The thread bundle resource allocation unit initializes the M thread bundle units to allocate M thread bundle numbers and M available register space sizes to the M thread bundle units. The M available register space sizes correspond to the M thread bundle numbers respectively. The thread bundle resource allocation unit sets M vector register base addresses of the M thread bundle units. An Nth vector register base address of the M vector register base addresses plus an Nth available register space size of the M available register space sizes is equal to an N+1th vector register base address of the M vector register base addresses. The artificial intelligence chip provided by the application can guarantee the register resource use efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of integrated circuits, and in particular to an artificial intelligence (AI) chip and an operation method thereof. BACKGROUND

[0002] Single Instruction Multiple Threads (SIMT) vector instructions are a common graphics processing unit (GPU) programming instruction and have been widely used in large-scale parallel computing tasks such as graphics rendering and artificial intelligence (AI) operations. SIMT instructions perform parallel operations in a basic unit of a thread bundle (Warp). A thread group includes multiple threads. A GPU or AI chip can thereby process multiple data points or computing tasks at the same time.

[0003] However, existing GPUs or AI chips only support a mode in which vector register spaces of different thread bundles are independent of each other. If cross-thread bundle data exchange is to be performed, an external memory module needs to be used, which results in a large data transmission delay, low efficiency, and is not conducive to a "consumer-producer" collaboration mode between thread bundles.

[0004] In addition, existing GPUs or AI chips only support a fixed mode in which the sizes of vector register spaces of different thread bundles are fixed, and thus cannot meet the needs of some memory instructions and tensor core instruction buffer regions that are large in size, and thus more slice calculations are required, which results in more related execution instructions, higher computing requirements, and longer execution time.

[0005] In addition, the programs run by existing GPUs or AI chips between different thread bundles are not the same, which is not conducive to better throughput of parallel computing and increases the difficulty of programming. SUMMARY

[0006] The present application is directed to an artificial intelligence (AI) chip and an operation method thereof to improve execution efficiency.

[0007] According to an embodiment of the present application, an artificial intelligence chip can include:

[0008] M thread bundle units corresponding to M thread bundles, M being a positive integer;

[0009] a thread bundle resource allocation unit coupled to the M thread bundle units;

[0010] The thread bundle resource allocation unit is used to initialize the M thread bundle units to allocate M thread bundle numbers and M usable register space sizes to the M thread bundle units. The M usable register space sizes correspond to the M thread bundle numbers respectively. The thread bundle resource allocation unit is also used to set the base addresses of the M vector registers of the M thread bundle units.

[0011] Wherein, the Nth vector register base address among the M vector register base addresses plus the Nth usable register space size among the M usable register space sizes equals the (N+1)th vector register base address among the M vector register base addresses, where N is a positive integer between 1 and M-1.

[0012] Optionally, in some examples, the AI ​​chip further includes: an execution unit coupled to the M thread bundle units; a thread bundle scheduling and instruction issuing unit coupled to the execution unit and the thread bundle resource allocation unit; and a vector register coupled to the execution unit; wherein the thread bundle resource allocation unit is used to notify the thread bundle scheduling and instruction issuing unit to send multiple execution instructions to the execution unit, and the execution unit runs the M thread bundles according to the multiple execution instructions, each of the multiple execution instructions corresponding to one of the M thread bundles; wherein the address ranges of the M registers of the M thread bundles are determined according to the base addresses of the M vector registers and the size of the M usable register spaces, and the address ranges of the M registers are continuous.

[0013] In some examples, the execution unit may optionally include multiple functional modules, and the multiple functional modules are configured to access the vector register space in the vector register corresponding to one of the M thread bundles in the M vector register base addresses according to the vector register base address corresponding to one of the M thread bundles in the M vector register base addresses, and the instruction address offset corresponding to one of the M thread bundles.

[0014] In some examples, optionally, the M thread bundle units each include a corresponding indirect addressing register, the execution unit includes multiple functional modules, and the multiple functional modules are used to access the cross-thread bundle vector register space in the vector register according to the vector register base address corresponding to one of the M thread bundles in the M vector register base address, the instruction address offset corresponding to one of the M thread bundles, and the indirect addressing register corresponding to one of the M thread bundles.

[0015] In some examples, optionally, the M thread bundles run the same program file; wherein when one of the M thread bundles runs an indirect addressing register modification instruction of the program file, the one of the M thread bundles modifies a corresponding indirect addressing register.

[0016] In some examples, optionally, the M thread bundle units further respectively comprise different scalar registers, and the M thread bundles comprise a first thread bundle and a second thread bundle; wherein the first thread bundle and the second thread bundle are configured to execute a same data loading program segment in the program file, the first thread bundle and the second thread bundle perform calculation with different scalar parameters, and generate a corresponding loading address with a respective corresponding indirect addressing register to load data respectively read from different memory spaces into different positions in a vector register space of the vector register corresponding to the loading address.

[0017] In some examples, optionally, the M thread bundles further comprise a third thread bundle, a fourth thread bundle and a fifth thread bundle, the loading address belongs to a vector register space of the vector register corresponding to the fifth thread bundle; wherein the third thread bundle and the fourth thread bundle are configured to execute a same calculation program segment in the program file, and generate a corresponding source data address with a respective corresponding indirect addressing register, the source data address belongs to the vector register space of the vector register corresponding to the fifth thread bundle; wherein the third thread bundle and the fourth thread bundle are further configured to read data loaded by at least one of the first thread bundle and the second thread bundle from the vector register space of the vector register corresponding to the fifth thread bundle according to the source data address to perform calculation with a corresponding scalar parameter.

[0018] In some examples, optionally, the artificial intelligence chip further comprises: a thread bundle synchronization unit coupled to the thread bundle scheduling and instruction issuing unit; wherein the thread bundle synchronization unit is configured to receive a barrier instruction of the third thread bundle and the fourth thread bundle to perform a producer barrier detection on the at least one of the first thread bundle and the second thread bundle according to the barrier instruction of the third thread bundle and the fourth thread bundle; wherein the thread bundle synchronization unit is further configured to, in response to the at least one of the first thread bundle and the second thread bundle completing loading, determine that the third thread bundle and the fourth thread bundle pass the producer barrier detection, and send an execution instruction of the plurality of execution instructions corresponding to the third thread bundle and the fourth thread bundle to the execution unit to enable the execution unit to run the third thread bundle and the fourth thread bundle.

[0019] In some examples, optionally, the thread bundle synchronization unit is further configured to receive a barrier instruction of the fifth thread bundle, to perform a consumer barrier check on the third thread bundle and the fourth thread bundle according to the barrier instruction of the fifth thread bundle; and in response to the third thread bundle and the fourth thread bundle being computed completed, determine that the fifth thread bundle passes the consumer barrier check, and end the fifth thread bundle to release the vector register space in the vector registers corresponding to the fifth thread bundle.

[0020] In some examples, optionally, the M thread bundles include a third thread bundle, and the load address belongs to a vector register space in the vector registers corresponding to the third thread bundle; wherein the third thread bundle is configured to execute the program file, and access the data loaded by the first thread bundle and the second thread bundle in the vector register space corresponding to the third thread bundle.

[0021] According to an embodiment of the present application, a cooperative thread bundle computing method can include:

[0022] initializing, by a thread bundle resource allocation unit, M thread bundle units respectively corresponding to M thread bundles, to allocate M thread bundle numbers and M available register space sizes to the M thread bundle units, wherein the M available register space sizes respectively correspond to the M thread bundle numbers;

[0023] setting, by the thread bundle resource allocation unit, M vector register base addresses of the M thread bundle units;

[0024] wherein an Nth vector register base address in the M vector register base addresses plus an Nth available register space size in the M available register space sizes is equal to an (N+1)th vector register base address in the M vector register base addresses, where N is a positive integer between 1 and M-1.

[0025] In some examples, optionally, the cooperative thread bundle computing method further includes: notifying, by the thread bundle resource allocation unit, a thread bundle scheduling and instruction issuing unit to send a plurality of execution instructions to an execution unit; and running, by the execution unit, the M thread bundles according to the plurality of execution instructions, wherein M register address ranges of the M thread bundles are respectively determined according to the M vector register base addresses and the M available register space sizes, and the M register address ranges are continuous.

[0026] In some examples, optionally, the execution unit includes a plurality of functional modules, and the plurality of functional modules respectively access vector register space corresponding to one of the M thread bundles in the vector registers according to a corresponding vector register base address in the M vector register base addresses, and a corresponding instruction address offset of the one of the M thread bundles.

[0027] In some examples, optionally, the M thread bundle units respectively include a corresponding indirect addressing register, the execution unit includes a plurality of functional modules, and the plurality of functional modules respectively access cross-thread bundle vector register space in the vector registers according to a corresponding vector register base address in the M vector register base addresses, a corresponding instruction address offset of the one of the M thread bundles, and the corresponding indirect addressing register of the one of the M thread bundles.

[0028] In some examples, optionally, the M thread bundles execute a same program file; wherein when one of the M thread bundles executes an indirect addressing register modifying instruction of the program file, the one of the M thread bundles modifies a corresponding indirect addressing register.

[0029] In some examples, optionally, the M thread bundle units further respectively include different scalar registers, and the M thread bundles include a first thread bundle and a second thread bundle; wherein the first thread bundle and the second thread bundle execute a same data loading program segment in the program file, the first thread bundle and the second thread bundle perform computation with different scalar parameters, and generate corresponding loading addresses with respective corresponding indirect addressing registers to load data respectively read from different memory spaces to different locations in vector register space in the vector registers corresponding to the loading addresses.

[0030] In some examples, optionally, the M thread bundles further include a third thread bundle, a fourth thread bundle, and a fifth thread bundle, the loading addresses belong to vector register space in the vector registers corresponding to the fifth thread bundle; wherein the third thread bundle and the fourth thread bundle execute a same computation program segment in the program file, and generate corresponding source data addresses with respective corresponding indirect addressing registers, the source data addresses belong to vector register space in the vector registers corresponding to the fifth thread bundle; wherein the third thread bundle and the fourth thread bundle read the data loaded by at least one of the first thread bundle and the second thread bundle from vector register space in the vector registers corresponding to the fifth thread bundle according to the source data addresses to perform computation with corresponding scalar parameters.

[0031] In some examples, optionally, the cooperative thread bundle computing method further comprises: receiving, by the thread bundle synchronization unit, a barrier instruction of the third thread bundle and the fourth thread bundle, to perform producer barrier detection on the at least one of the first thread bundle and the second thread bundle according to the barrier instruction of the third thread bundle and the fourth thread bundle; determining, by the thread bundle synchronization unit, that the third thread bundle and the fourth thread bundle pass the producer barrier detection in response to the at least one of the first thread bundle and the second thread bundle being loaded, and sending execution instructions corresponding to the third thread bundle and the fourth thread bundle in the plurality of execution instructions to the execution unit, to enable the execution unit to run the third thread bundle and the fourth thread bundle.

[0032] In some examples, optionally, the cooperative thread bundle computing method further comprises: receiving, by the thread bundle synchronization unit, a barrier instruction of the fifth thread bundle, to perform consumer barrier detection on the third thread bundle and the fourth thread bundle according to the barrier instruction of the fifth thread bundle; determining, by the thread bundle synchronization unit, that the fifth thread bundle passes the consumer barrier detection in response to the third thread bundle and the fourth thread bundle being computed, and ending the fifth thread bundle to release a vector register space in the vector register corresponding to the fifth thread bundle.

[0033] In some examples, optionally, the M thread bundles include a third thread bundle, and the load address belongs to a vector register space in the vector register corresponding to the third thread bundle; wherein the third thread bundle executes the program file, and accesses data loaded by the first thread bundle and the second thread bundle in the vector register space in the vector register corresponding to the third thread bundle.

[0034] Based on the above, the artificial intelligence chip and the cooperative thread bundle computing method can set the register address ranges of different cooperative thread bundles to be continuous, to ensure the efficiency of register resource use. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 FIG. 1 is a circuit block diagram of an artificial intelligence (AI) chip according to an embodiment of the present application;

[0036] Figure 2 FIG. 2 is a flowchart of a cooperative thread bundle computing method according to an embodiment of the present application;

[0037] Figure 3 FIG. 3 is a schematic diagram of a program file according to an embodiment of the present application;

[0038] Figure 4 This is a schematic diagram of a programming model of a program file according to an embodiment of the present invention;

[0039] Figure 5 This is a schematic diagram illustrating the operation of multiple thread bundles according to an embodiment of the present invention;

[0040] Figure 6 This is a schematic diagram illustrating the use of a vector register and memory module according to an embodiment of the present invention.

[0041] Explanation of icon numbers

[0042] 100: AI chips;

[0043] 110: Thread bundle resource allocation unit;

[0044] 120_1: First thread beam unit;

[0045] 120_M: The Mth thread beam unit;

[0046] 121_1: First scalar register;

[0047] 121_M: The Mth scalar register;

[0048] 122_1: Base address of the first vector register;

[0049] 122_M: Base address of the Mth vector register;

[0050] 123_1: First indirect addressing register;

[0051] 123_M: The Mth indirect addressing register;

[0052] 130: Execution Unit;

[0053] 131: Vector kernel module;

[0054] 132: Memory module;

[0055] 133: Tensor Kernel Module;

[0056] 140: Thread bundle scheduling and instruction issuing unit;

[0057] 150: Vector register;

[0058] 160: Thread bundle synchronization unit;

[0059] 300: Program file;

[0060] 310: Address register generates program segment;

[0061] 320: Base address configuration program segment;

[0062] 330: data loading program segment;

[0063] 340: computation program segment;

[0064] 350: cache release program segment;

[0065] 400: corresponding program segment;

[0066] 520_1: first thread bundle;

[0067] 520_2: second thread bundle;

[0068] 520_3: third thread bundle;

[0069] 520_4: fourth thread bundle;

[0070] 520_5: fifth thread bundle;

[0071] 651: first vector register space;

[0072] 652: second vector register space;

[0073] 653: third vector register space;

[0074] 654: fourth vector register space;

[0075] 655: fifth vector register space;

[0076] 632_1: first processing region;

[0077] 632_2: second processing region. DETAILED DESCRIPTION

[0078] Reference will now be made in detail to exemplary embodiments of the application, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used in the different drawings and the description to refer to the same or like parts.

[0079] The term "coupled" (or connected) used in the detailed description of the specification including the claims can refer to either a direct or indirect coupling or connection. For example, if a first device is coupled (or connected) to a second device, it can be directly connected to the second device or it can be indirectly connected to the second device through one or more other devices or some other coupling or connection means. The terms "first", "second", and the like used in the detailed description of the specification including the claims are used to name components (elements) and do not necessarily have to follow the components' (elements') order either in the description or in the drawing. In addition, components / elements / means with the same reference numerals in the drawings and embodiments represent the same or similar parts wherever possible. Components / elements / means with the same reference numerals or the same terms in different embodiments can be cross-referenced with the relevant description. It should be understood that features of the following embodiments can be combined with each other. For example, features of the second embodiment can be combined with features of the first embodiment. Those skilled in the art can select appropriate combinations of features according to actual design requirements.

[0080] Figure 1 is a circuit block schematic diagram of an artificial intelligence (AI) chip according to an embodiment of the present application. Please refer to Figure 1 , Figure 1 The AI chip 100 includes a warp resource allocation unit 110, M warp units (e.g., 1st warp unit 120_1 to Mth warp unit 120_M), an execution unit 130, a warp scheduling and instruction issue unit 140, a vector register 150, and a warp synchronization unit 160, where M is a positive integer. The M warp units (e.g., 1st warp unit 120_1 to Mth warp unit 120_M) correspond to M warps, respectively, and the M warp units (e.g., 1st warp unit 120_1 to Mth warp unit 120_M) each include a respective scalar register (e.g., 1st scalar register 121_1 to Mth scalar register 121_M), a respective vector register base address (e.g., 1st vector register base address 122_1 to Mth vector register base address 122_M), and a respective indirect addressing register (e.g., 1st indirect addressing register 123_1 to Mth indirect addressing register 123_M). Figure 1 Figure 1 Figure 1 Figure 1 Figure 1 Figure 1 Figure 1 ​​​​​​The first thread bundle unit (120_1 to the Mth thread bundle unit (120_M)) can also be referred to as the M thread bundle configuration units corresponding to the M thread bundles respectively. M is a positive integer greater than 1.

[0081] Execution unit 130 includes a vector kernel module 131, a memory module 132, and a tensor kernel module 133. Thread bundle resource allocation unit 110 is coupled to M thread bundle units (e.g., ...). Figure 1 The first thread beam unit 120_1 to the Mth thread beam unit 120_M). Execution unit 130 is coupled to M thread beam units (e.g., ...). Figure 1 The first thread bundle unit 120_1 to the Mth thread bundle unit 120_M). The thread bundle scheduling and instruction issuing unit 140 is coupled to the execution unit 130 and the thread bundle resource allocation unit 110. The vector register 150 is coupled to the execution unit 130. The thread bundle synchronization unit 160 is coupled to the thread bundle scheduling and instruction issuing unit 140.

[0082] Depending on the design, in some embodiments, the thread bundle resource allocation unit 110 and M thread bundle units (e.g.) Figure 1 At least one of the following components—the first thread bundle unit 120_1 to the Mth thread bundle units 120_M), the execution unit 130, the thread bundle scheduling and instruction issuing unit 140, the vector register 150, and the thread bundle synchronization unit 160—can be implemented as hardware circuitry. In other embodiments, the thread bundle resource allocation unit 110 and the M thread bundle units (e.g., ...) Figure 1 The implementation of at least one of the following can be a combination of hardware, firmware, and software (i.e., program): thread bundle unit 120_1 to thread bundle unit M 120_M, execution unit 130, thread bundle scheduling and instruction issuing unit 140, vector register 150, and thread bundle synchronization unit 160.

[0083] In terms of hardware, the aforementioned thread bundle resource allocation unit 110 and M thread bundle units (e.g.) Figure 1 At least one of the following components—the first thread bundle unit 120_1 to the Mth thread bundle units 120_M), the execution unit 130, the thread bundle scheduling and instruction issuing unit 140, the vector register 150, and the thread bundle synchronization unit 160—can be implemented on an integrated circuit as logic circuitry or register circuitry. For example, the thread bundle resource allocation unit 110, the M thread bundle units (e.g., ...) Figure 1The functions of at least one of the following: the first thread bundle unit 120_1 to the Mth thread bundle units 120_M; the execution unit 130; the thread bundle scheduling and instruction issuing unit 140; the vector register 150; and the thread bundle synchronization unit 160 can be implemented in various logic blocks, modules, and circuits within one or more hardware controllers, microcontrollers, hardware processors, microprocessors, application-specific integrated circuits (ASICs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), central processing units (CPUs), or other processing units. The thread bundle resource allocation unit 110 and the M thread bundle units (e.g., ...) Figure 1 The related functions of at least one of the following: the first thread bundle unit 120_1 to the Mth thread bundle unit 120_M), the execution unit 130, the thread bundle scheduling and instruction issuing unit 140, the vector register 150, and the thread bundle synchronization unit 160 can be implemented as hardware circuits, such as various logic blocks, modules, and circuits in an integrated circuit, using hardware description languages ​​(such as Verilog HDL or VHDL) or other suitable programming languages.

[0084] In software or firmware form, the aforementioned thread bundle resource allocation unit 110 and M thread bundle units (e.g.) Figure 1 The functions of at least one of the following: the first thread bundle unit 120_1 to the Mth thread bundle units 120_M; the execution unit 130; the thread bundle scheduling and instruction issuing unit 140; the vector register 150; and the thread bundle synchronization unit 160 can be implemented as programming codes. For example, the thread bundle resource allocation unit 110 and the M thread bundle units (e.g., Mth thread bundle units 120_M) can be implemented using general programming languages ​​(e.g., C, C++, or assembly language) or other suitable programming languages. Figure 2The programmable code can be recorded / stored in a non-transitory machine-readable storage medium. In some embodiments, the non-transitory machine-readable storage medium includes, for example, semiconductor memory and / or a storage device. An electronic device (e.g., a CPU, hardware controller, microcontroller, hardware processor, or microprocessor) can read and execute the programmable code from the non-transitory machine-readable storage medium to implement the aforementioned thread bundle resource allocation unit 110 and M thread bundle units (e.g., the first thread bundle unit 120_1 to the Mth thread bundle unit 120_M). The execution unit 130, thread bundle scheduling and instruction issuing unit 140, vector register 150, and thread bundle synchronization unit 160. Figure 1 The functions of at least one of the following: the first thread beam unit 120_1 to the Mth thread beam unit 120_M), the execution unit 130, the thread beam scheduling and instruction issuing unit 140, the vector register 150, and the thread beam synchronization unit 160.

[0085] Figure 2 This is a flowchart illustrating a cooperative thread bundle calculation method according to an embodiment of the present invention. Please refer to... Figure 1 and Figure 1 In step S210, the thread bundle resource allocation unit 110 initializes M thread bundle units (e.g., Figure 1 The first thread bundle unit 120_1 to the Mth thread bundle unit 120_M), to allocate M thread bundle numbers and M usable register space sizes to the M thread bundle units (e.g. Figure 1 The first thread bundle unit 120_1 to the Mth thread bundle unit 120_M), wherein the M available register space sizes correspond to different thread bundle numbers (warp IDs). The thread bundle resource allocation unit 110 can receive setting instructions input by software to set the corresponding available register space size for each thread bundle with a different thread bundle number.

[0086] In step S220, the thread bundle resource allocation unit 110 sets up M thread bundle units (e.g., Figure 1 The base addresses of the M vector registers of the first thread bundle unit 120_1 to the Mth thread bundle unit 120_M, wherein the Nth vector register base address of the M vector register base addresses plus the Nth usable register space size of the M usable register space sizes is equal to the (N+1)th vector register base address of the M vector register base addresses, where N is a positive integer between 1 and M-1.

[0087] At step S230, the thread bundle resource allocation unit 110 instructs the thread bundle scheduling and instruction issuing unit 140 to send a plurality of execution instructions to the execution unit 130. Each of the plurality of execution instructions is input to a corresponding one of the M thread bundle units (e.g., the 1stthread bundle unit 120_1 to the Mththread bundle unit 120_M) to cause the execution unit 130 to execute M thread bundles corresponding to the M thread bundle units (e.g., the 1stthread bundle unit 120_1 to the Mththread bundle unit 120_M). In other words, each of the plurality of execution instructions corresponds to a corresponding one of the M thread bundle units (e.g., the 1stthread bundle unit 120_1 to the Mththread bundle unit 120_M) or a corresponding one of the M thread bundles. The plurality of execution instructions can include vector core instructions, memory instructions, and tensor core instructions, and are provided to the vector core module 131, the memory module 132, and the tensor core module 133 of the execution unit 130, respectively. Figure 1 At step S230, the thread bundle resource allocation unit 110 instructs the thread bundle scheduling and instruction issuing unit 140 to send a plurality of execution instructions to the execution unit 130. Each of the plurality of execution instructions is input to a corresponding one of the M thread bundle units (e.g., the 1stthread bundle unit 120_1 to the Mththread bundle unit 120_M) to cause the execution unit 130 to execute M thread bundles corresponding to the M thread bundle units (e.g., the 1stthread bundle unit 120_1 to the Mththread bundle unit 120_M). In other words, each of the plurality of execution instructions corresponds to a corresponding one of the M thread bundle units (e.g., the 1stthread bundle unit 120_1 to the Mththread bundle unit 120_M) or a corresponding one of the M thread bundles. The plurality of execution instructions can include vector core instructions, memory instructions, and tensor core instructions, and are provided to the vector core module 131, the memory module 132, and the tensor core module 133 of the execution unit 130, respectively. Figure 1 At step S230, the thread bundle resource allocation unit 110 instructs the thread bundle scheduling and instruction issuing unit 140 to send a plurality of execution instructions to the execution unit 130. Each of the plurality of execution instructions is input to a corresponding one of the M thread bundle units (e.g., the 1stthread bundle unit 120_1 to the Mththread bundle unit 120_M) to cause the execution unit 130 to execute M thread bundles corresponding to the M thread bundle units (e.g., the 1stthread bundle unit 120_1 to the Mththread bundle unit 120_M). In other words, each of the plurality of execution instructions corresponds to a corresponding one of the M thread bundle units (e.g., the 1stthread bundle unit 120_1 to the Mththread bundle unit 120_M) or a corresponding one of the M thread bundles. The plurality of execution instructions can include vector core instructions, memory instructions, and tensor core instructions, and are provided to the vector core module 131, the memory module 132, and the tensor core module 133 of the execution unit 130, respectively. Figure 1 At step S230, the thread bundle resource allocation unit 110 instructs the thread bundle scheduling and instruction issuing unit 140 to send a plurality of execution instructions to the execution unit 130. Each of the plurality of execution instructions is input to a corresponding one of the M thread bundle units (e.g., the 1stthread bundle unit 120_1 to the Mththread bundle unit 120_M) to cause the execution unit 130 to execute M thread bundles corresponding to the M thread bundle units (e.g., the 1stthread bundle unit 120_1 to the Mththread bundle unit 120_M). In other words, each of the plurality of execution instructions corresponds to a corresponding one of the M thread bundle units (e.g., the 1stthread bundle unit 120_1 to the Mththread bundle unit 120_M) or a corresponding one of the M thread bundles. The plurality of execution instructions can include vector core instructions, memory instructions, and tensor core instructions, and are provided to the vector core module 131, the memory module 132, and the tensor core module 133 of the execution unit 130, respectively.

[0088] At step S240, the execution unit 130 executes the M thread bundles corresponding to the M thread bundle units (e.g., the 1stthread bundle unit 120_1 to the Mththread bundle unit 120_M) according to the plurality of execution instructions, where the M register address ranges of the M thread bundles are determined according to the M vector register base addresses and the M available register space sizes, respectively, and the M register address ranges are consecutive. The M thread bundles can form a group of cooperating thread bundles. Figure 1 At step S240, the execution unit 130 executes the M thread bundles corresponding to the M thread bundle units (e.g., the 1stthread bundle unit 120_1 to the Mththread bundle unit 120_M) according to the plurality of execution instructions, where the M register address ranges of the M thread bundles are determined according to the M vector register base addresses and the M available register space sizes, respectively, and the M register address ranges are consecutive. The M thread bundles can form a group of cooperating thread bundles.

[0089] In this regard, the thread bundle resource allocation unit 110 can sequentially increase the vector register base addresses with the thread bundle numbers from small to large, and the increment is the available register space size of the previous thread bundle. Under this mechanism, the register address ranges of different cooperating thread bundles are consecutive, and thus the register resource usage efficiency can be guaranteed.

[0090] Furthermore, at least one of the vector core module 131, the memory module 132, and the tensor core module 133 can directly access the vector register space corresponding to one of the M thread bundles in the vector registers 150 according to the corresponding vector register base address (e.g., a corresponding one of the 1stvector register base address 122_1 to the Mthvector register base address 122_M) in the M vector register base addresses and the instruction address offset corresponding to the aforementioned one of the M thread bundles. Figure 3 Alternatively, at least one of the vector core module 131, the memory module 132, and the tensor core module 133 can access the vector register space corresponding to one of the M thread bundles in the vector registers 150 according to the corresponding vector register base address (e.g., a corresponding one of the 1stvector register base address 122_1 to the Mthvector register base address 122_M) in the M vector register base addresses and the instruction address offset corresponding to the aforementioned one of the M thread bundles.

[0091] Figure 1 ​The instruction address offset corresponding to one of the base addresses of the first vector register (122_1 to the base address of the Mth vector register (122_M)), and the indirect addressing register corresponding to one of the aforementioned M thread bundles (e.g., the indirect addressing register corresponding to one of the aforementioned M thread bundles). Figure 3 The vector register space across thread bundles in vector register 150 is accessed by using one of the first indirect addressing registers 123_1 to the Mth indirect addressing registers 123_M. Additionally, when one of the aforementioned executable programs in the M thread bundles executes an instruction to modify the indirect addressing register, the aforementioned indirect addressing register of the M thread bundle is modified.

[0092] In other words, the functional modules of execution unit 130 can access the vector register space within the available range of the current thread bundle through direct addressing, or they can access the vector register space within the available range of other thread bundles through indirect addressing. Furthermore, if multiple thread bundles have the same indirect addressing register, these multiple thread bundles can share the vector register space within the available range of another specified thread bundle to facilitate data interaction. In addition, the thread bundle indirect addressing registers of each thread bundle can also be modified through programming in the program file according to different usage requirements to achieve the function of dynamically adjusting vector register resources.

[0093] Figure 1 This is a schematic diagram of a program file according to an embodiment of the present invention. Please refer to... Figure 1 and Figure 3 Execution unit 130 is based on M thread bundle units (e.g., Figure 1 The M thread bundles (from the first thread bundle unit 120_1 to the Mth thread bundle unit 120_M) can run the same program file. For this, the M thread bundle units (e.g., Figure 1 The first thread bundle unit 120_1 to the Mth thread bundle unit 120_M can respectively execute corresponding program segments in the program file 300, where different program segments correspond to different application requirements, so that different thread bundles can execute instruction sequences to implement different application requirements. Figure 1 For example, program file 300 includes address register generation program segment 310, base address configuration program segment 320, data loading program segment 330, calculation program segment 340 and cache release program segment 350.

[0094] Corresponding to M thread bundle units (e.g.) Figure 1Any of the M thread warps (e.g., the first thread warp unit 120_1 to the Mth thread warp unit 120_M) can execute an address register generation program segment 310 to cause the corresponding indirect addressing register to be modified so as to be indexed into the cross-thread warp vector register space within a register address range of M consecutive vector register base addresses in the vector registers using the modified indirect addressing register. For example, the address register generation program segment 310 can include an indirect addressing register modification instruction. Corresponding to the M thread warp units (e.g., the first thread warp unit 120_1 to the Mth thread warp unit 120_M), the address register generation program segment 310 can include M address register generation program segments 310_1 to 310_M. Figure 4 A portion of the M thread warps (e.g., the first thread warp unit 120_1 to the Mth thread warp unit 120_M) can execute an instruction sequence of a data load program segment 330 to perform a data load operation. Corresponding to the M thread warp units (e.g., the first thread warp unit 120_1 to the Mth thread warp unit 120_M), the data load program segment 330 can include M data load program segments 330_1 to 330_M. Figure 5 Another portion of the M thread warps (e.g., the first thread warp unit 120_1 to the Mth thread warp unit 120_M) can execute an instruction sequence of a computation program segment 340 to perform a data computation operation. Corresponding to the M thread warp units (e.g., the first thread warp unit 120_1 to the Mth thread warp unit 120_M), the computation program segment 340 can include M computation program segments 340_1 to 340_M. Figure 1 Yet another portion of the M thread warps (e.g., the first thread warp unit 120_1 to the Mth thread warp unit 120_M) can execute an instruction sequence of a cache release program segment 350 to perform a cache release operation.

[0095] Figure 4 is a programming model diagram of a program file according to an embodiment of the present application. Figure 5 is an operation diagram of a plurality of thread warps according to an embodiment of the present application. Please refer to Figure 4 , Figure 1 and Figure 1 , for example, to perform the 5 thread warps (i.e., the first thread warp 520_1, the second thread warp 520_2, the third thread warp 520_3, the fourth thread warp 520_4, and the fifth thread warp 520_5), at least one of the first thread warp 520_1, the second thread warp 520_2, the third thread warp 520_3, the fourth thread warp 520_4, and the fifth thread warp 520_5 can first execute the address register generation program segment 310 in the same program file to cause the corresponding indirect addressing register to be modified so as to be indexed into the cross-thread warp vector register space within a register address range of 5 consecutive vector register base addresses.

[0096] The first thread warp 520_1 and the second thread warp 520_2 can execute the address register generation program segment 310 according to the respective thread warp number (warp ID) as Figure 1corresponding program segment 400 in the programming model of the program file 300. In detail, the first thread bundle 520_1, the second thread bundle 520_2 can execute the same data loading program segment 330 in the corresponding program segment 400 of the program file 300 and modify the respective indirect addressing registers. The first thread bundle 520_1, the second thread bundle 520_2 can first modify the respective indirect addressing registers to index to the vector register space corresponding to the fifth thread bundle 520_5 in the vector registers 150 (i.e. the vector register space across thread bundles), then the first thread bundle 520_1, the second thread bundle 520_2 operate different scalar parameters for computation and generate corresponding loading addresses using the respective indirect addressing registers to read data from different memory spaces (i.e. the memory space of the memory module 132 of Figure 6 corresponding to the loading addresses to different locations in the vector register space corresponding to the fifth thread bundle 520_5 in the vector registers 150 (i.e. the vector register space across thread bundles). The loading addresses can belong to the vector register space corresponding to the fifth thread bundle 520_5 in the vector registers 150. Figure 6

[0097] The third thread bundle 520_3, the fourth thread bundle 520_4 execute the same computation program segment 340 in the corresponding program segment 400 of the program file 300 for parallel computation and modify the respective indirect addressing registers. The third thread bundle 520_3, the fourth thread bundle 520_4 can first modify the respective indirect addressing registers to index to the vector register space corresponding to the fifth thread bundle 520_5 in the vector registers 150 (i.e. the vector register space across thread bundles) and generate corresponding source data addresses using the respective modified indirect addressing registers. The source data addresses can belong to the vector register space corresponding to the fifth thread bundle 520_5 in the vector registers 150. The third thread bundle 520_3, the fourth thread bundle 520_4 read data loaded by at least one of the first thread bundle 520_1, the second thread bundle 520_2 from the vector register space corresponding to the fifth thread bundle 520_5 in the vector registers 150 according to the source data addresses to operate corresponding scalar parameters for computation and write different computation results to different memory spaces.

[0098] ​Notably, due to the relationship of different thread bundles as producers (e.g., the first thread bundle 520_1, the second thread bundle 520_2) and consumers (e.g., the third thread bundle 520_3, the fourth thread bundle 520_4) of data in a cooperative thread bundle, synchronization is needed to ensure correctness. The thread bundle synchronization unit 160 can receive a barrier instruction in a consumer thread bundle (e.g., the third thread bundle 520_3, the fourth thread bundle 520_4), and wait for the instruction loading of one or more producer thread bundles (e.g., the first thread bundle 520_1, the second thread bundle 520_2) to be completed before passing the consumer thread bundle (e.g., the third thread bundle 520_3, the fourth thread bundle 520_4) through the barrier check. After passing the barrier check, the consumer thread bundle (e.g., the third thread bundle 520_3, the fourth thread bundle 520_4) can then issue subsequent instructions to the execution unit 130 for execution of related operations.

[0099] In this regard, the third thread bundle 520_3 and the fourth thread bundle 520_4 can perform parallel vector computation (e.g., floating point computation, fixed point computation, or logical operation, etc.), and for example, need to wait for the first thread bundle 520_1 and the second thread bundle 520_2 to load data from the memory module 132 to two data sources in the vector register space corresponding to the fifth thread bundle 520_5 in the vector register 150 at the same time, and thus the third thread bundle 520_3 and the fourth thread bundle 520_4 need to wait for the first thread bundle 520_1 and the second thread bundle 520_2 to complete before execution. The thread bundle synchronization unit 160 can receive barrier instructions of the third thread bundle 520_3 and the fourth thread bundle 520_4, and perform producer barrier checks (i.e., check whether at least one of the first thread bundle 520_1 and the second thread bundle 520_2 is loaded) on the first thread bundle 520_1 and the second thread bundle 520_2 according to the barrier instructions of the third thread bundle 520_3 and the fourth thread bundle 520_4. In response to at least one of the first thread bundle 520_1 and the second thread bundle 520_2 being loaded, the thread bundle synchronization unit 160 determines that the third thread bundle 520_3 and the fourth thread bundle 520_4 pass the producer barrier check, and sends execution instructions corresponding to the third thread bundle 520_3 and the fourth thread bundle 520_4 in the plurality of execution instructions to the execution unit 130, so that the execution unit 130 executes the third thread bundle 520_3 and the fourth thread bundle 520_4. In this regard, the third thread bundle 520_3 and the fourth thread bundle 520_4 can use the same data source in the same vector register space and different scalar parameters, which facilitates different thread bundles to perform the same computation instructions in parallel to obtain different computation results.

[0100] The fifth thread bundle 520_5, as the last thread bundle, can allocate (e.g., dynamically allocate) itself to all the register space between the base address of the vector register 150 and the maximum address of the vector register 150 for access by the first thread bundle 520_1 to the fourth thread bundle 520_4. In addition, the register space size of the vector register 150 in the present embodiment is not limited to between the base address of the vector register 150 and the maximum address of the vector register 150.

[0101] The fifth thread bundle 520_5 can execute the cache release program segment 350 in the program file 300. For this, the fifth thread bundle 520_5 needs to wait until the third thread bundle 520_3 and the fourth thread bundle 520_4 complete the use of the data loaded by the first thread bundle 520_1 and the second thread bundle 520_2, and then exit and end the life cycle, so the fifth thread bundle 520_5 needs to wait until the third thread bundle 520_3 and the fourth thread bundle 520_4 are both completed by using a barrier instruction. For this, the thread bundle synchronization unit 160 receives the barrier instruction of the fifth thread bundle 520_5 to detect the consumer barrier (i.e., whether the third thread bundle 520_3 and the fourth thread bundle 520_4 are completed) of the third thread bundle 520_3 and the fourth thread bundle 520_4 according to the barrier instruction of the fifth thread bundle 520_5. The thread bundle synchronization unit 160 determines that the fifth thread bundle 520_5 passes the consumer barrier detection in response to the completion of the calculation of the third thread bundle 520_3 and the fourth thread bundle 520_4, and ends the fifth thread bundle 520_5 to release the vector register space corresponding to the fifth thread bundle 520_5 in the vector register 150.

[0102] Therefore, the first thread bundle 520_1 to the fifth thread bundle 520_5 can effectively utilize the entire register space of the vector register 150, and the first thread bundle 520_1 to the fourth thread bundle 520_4 can implement parallel operations on different thread bundle data to effectively perform cooperative work between the thread bundles through the sharing of the vector register space corresponding to the fifth thread bundle 520_5 in the vector register 150, and can achieve an effective data interaction effect.

[0103] In addition, in an implementation scenario without the fourth thread bundle 520_4 and the fifth thread bundle 520_5, the first thread bundle 520_1 and the second thread bundle 520_2 can also modify the respective indirect addressing registers, and generate corresponding load addresses by using the respective modified indirect addressing registers. The load addresses can belong to the vector register space corresponding to the third thread bundle 520_3 in the vector register 150. The first thread bundle 520_1 and the second thread bundle 520_2 use different scalar parameters to respectively load data from different memory spaces (i.e., the memory space corresponding to the first thread bundle 520_1 and the memory space corresponding to the second thread bundle 520_2) to the vector register space corresponding to the third thread bundle 520_3 in the vector register 150. Figure 6The data is read from the memory space of memory module 132 and loaded into different locations in the vector register space of vector register 150 corresponding to the load address, i.e., the vector register space corresponding to the third thread bundle 520_3. In this way, during the vector calculation process, the third thread bundle 520_3 can also directly read the data required for vector calculation loaded by the first thread bundle 520_1 and the second thread bundle 520_2 from its own vector register space.

[0104] Furthermore, please refer to the following: Figure 1 , ​ This is a schematic diagram illustrating the use of a vector register and memory module according to an embodiment of the present invention. The vector register 150 and memory module 132 may have, for example: ​ The space allocation is shown. For this, the first thread bundle 520_1 to the fifth thread bundle 520_5 can be based on different M vector register base addresses (e.g., ​ The base address of the first vector register (122_1) to the base address of the Mth vector register (122_M) sequentially occupy the space of the first vector register (651) to the space of the fifth vector register (655) in the vector register 150 for buffering and executing instructions.

[0105] Furthermore, the data loaded by the first thread bundle 520_1 and the second thread bundle 520_2 can be loaded from the second processing area 632_2 of the memory module 132 into the fifth vector register space 655 in the vector register 150 corresponding to the fifth thread bundle 520_5, so that the third thread bundle 520_3 and the fourth thread bundle 520_4 can read the data across bundles. The third thread bundle 520_3 and the fourth thread bundle 520_4 will then store the calculation results stored in the third vector register space 653 and the fourth vector register space 654 into the first processing area 632_1 of the memory module 132. In this regard, the first thread bundle 520_1 to the fifth thread bundle 520_5 can act as cooperative thread bundles, and data interaction is performed using the fifth vector register space 655 in the shared vector register 150 corresponding to the fifth thread bundle 520_5.

[0106] To sum up, the artificial intelligence chip and the cooperative thread bundle calculation method can allocate different sizes of available register space size to each cooperative thread bundle according to software requirements, which is beneficial to configure the data cache area of the memory module or the tensor core module, and can realize the effect of efficient and low-delay data interaction. Moreover, the register address range of different cooperative thread bundles can be continuous to ensure the efficiency of register resource use. The artificial intelligence chip and the cooperative thread bundle calculation method also support each cooperative thread bundle to indirectly access the vector register space of other thread bundles across thread bundles, and can execute the assembly code of the same program file at the same time to realize the effect of convenient programming and strong parallelism. In addition, each thread bundle can also execute different program segments of the same program file to cooperate as producers and consumers.

[0107] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An artificial intelligence chip, characterized in that, include: There are M thread bundle units, each corresponding to a thread bundle, where M is a positive integer; and The execution unit is coupled to the M thread bundle units; and Vector register, coupled to the execution unit; A thread bundle resource allocation unit is coupled to the M thread bundle units; The address ranges of the M registers of the M thread bundles are determined based on the base addresses of the M vector registers of the M thread bundle units and the sizes of the M usable register spaces. The thread bundle resource allocation unit is used to initialize the M thread bundle units to allocate M thread bundle numbers and the M usable register spaces to the M thread bundle units. The M usable register spaces correspond to the M thread bundle numbers respectively. The thread bundle resource allocation unit is also used to set the base addresses of the M vector registers. Wherein, the range of the M register addresses is continuous, and the Nth vector register base address among the M vector register base addresses plus the Nth usable register space size among the M usable register space sizes is equal to the (N+1)th vector register base address among the M vector register base addresses, where N is a positive integer between 1 and M-1; The M thread bundle units each include a corresponding indirect addressing register, and the execution unit includes multiple functional modules. The multiple functional modules are used to access the cross-thread bundle vector register space in the vector register according to the vector register base address corresponding to one of the M thread bundles in the M vector register base address, the instruction address offset corresponding to one of the M thread bundles, and the indirect addressing register corresponding to one of the M thread bundles. The cross-thread bundle vector register space is the vector register space within the available range of other thread bundles besides the current thread bundle of the functional module of the execution unit, which is accessed through indirect addressing.

2. The artificial intelligence chip according to claim 1, characterized in that, Also includes: A thread beam scheduling and instruction issuing unit is coupled to the execution unit and the thread beam resource allocation unit; The thread bundle resource allocation unit is used to notify the thread bundle scheduling and instruction issuing unit to send multiple execution instructions to the execution unit, and the execution unit runs the M thread bundles according to the multiple execution instructions, each of the multiple execution instructions corresponding to one of the M thread bundles.

3. The artificial intelligence chip according to claim 1, characterized in that, The plurality of functional modules are further configured to access the vector register space corresponding to one of the M thread bundles in the vector register based on the vector register base address corresponding to one of the M thread bundles in the M vector register base addresses and the instruction address offset corresponding to one of the M thread bundles.

4. The artificial intelligence chip according to claim 1, characterized in that, The M thread bundles run the same program file; Specifically, when one of the M thread bundles executes the indirect addressing register modification instruction of the program file, the indirect addressing register corresponding to that M thread bundle is modified.

5. The artificial intelligence chip according to claim 4, characterized in that, The M thread bundle units also include different scalar registers, and the M thread bundles include a first thread bundle and a second thread bundle; The first thread bundle and the second thread bundle are configured to execute the same data loading program segment in the program file. The first thread bundle and the second thread bundle use different scalar parameters for calculation and use their respective indirect addressing registers to generate corresponding load addresses, so as to load data read from different memory spaces into different positions in the vector register space corresponding to the load address.

6. The artificial intelligence chip according to claim 5, characterized in that, The M thread bundles also include a third thread bundle, a fourth thread bundle, and a fifth thread bundle, and the load address belongs to the vector register space in the vector register corresponding to the fifth thread bundle; The third thread bundle and the fourth thread bundle are configured to execute the same computation program segment in the program file and generate the corresponding source data address using their respective indirect addressing registers. The source data address belongs to the vector register space in the vector register corresponding to the fifth thread bundle. The third and fourth thread bundles are further configured to read data loaded by at least one of the first and second thread bundles from the vector register space corresponding to the fifth thread bundle in the vector register according to the source data address, so as to perform calculations using the corresponding scalar parameters.

7. The artificial intelligence chip according to claim 6, characterized in that, Also includes: The thread bundle synchronization unit is coupled to the thread bundle scheduling and instruction issuing unit; The thread bundle synchronization unit is used to receive barrier instructions from the third thread bundle and the fourth thread bundle, and to perform producer barrier detection on at least one of the first thread bundle and the second thread bundle according to the barrier instructions from the third thread bundle and the fourth thread bundle. The thread bundle synchronization unit is further configured to, in response to the completion of loading of at least one of the first thread bundle and the second thread bundle, determine that the third thread bundle and the fourth thread bundle have passed the producer barrier detection, and send the execution instruction corresponding to the third thread bundle and the fourth thread bundle among a plurality of execution instructions to the execution unit, so that the execution unit runs the third thread bundle and the fourth thread bundle.

8. The artificial intelligence chip according to claim 7, characterized in that, The thread bundle synchronization unit is also used to receive the barrier instruction of the fifth thread bundle, so as to perform consumer barrier detection on the third thread bundle and the fourth thread bundle according to the barrier instruction of the fifth thread bundle; The thread bundle synchronization unit is further configured to, in response to the completion of the calculation of the third thread bundle and the fourth thread bundle, determine that the fifth thread bundle has passed the consumer barrier detection and terminate the fifth thread bundle, so as to release the vector register space in the vector register corresponding to the fifth thread bundle.

9. The artificial intelligence chip according to claim 5, characterized in that, The M thread bundles include a third thread bundle, and the load address belongs to the vector register space in the vector register corresponding to the third thread bundle; The third thread bundle is configured to execute the program file and access data loaded by the first thread bundle and the second thread bundle respectively in the vector register space corresponding to the third thread bundle in the vector register.

10. A method for calculating a cooperative thread bundle, characterized in that, include: The thread bundle resource allocation unit initializes M thread bundle units corresponding to M thread bundles respectively, and allocates M thread bundle numbers and M usable register space sizes to the M thread bundle units, wherein the M usable register space sizes correspond to the M thread bundle numbers respectively, and the M thread bundle units each include a corresponding indirect addressing register. The base addresses of the M vector registers of the M thread bundle units are set by the thread bundle resource allocation unit; as well as The execution unit runs the M thread bundles according to multiple execution instructions. The range of M register addresses for the M thread bundles is determined by the base addresses of the M vector registers and the sizes of the M usable register spaces for the M thread bundle units. The range of the M register addresses is continuous, and the Nth vector register base address among the M vector register base addresses plus the Nth usable register space size among the M usable register spaces equals the (N+1)th vector register base address among the M vector register base addresses, where N is a positive integer between 1 and M-1. The execution unit includes multiple functional modules, and each of the multiple functional modules accesses the cross-thread-bundle vector register space in the vector register according to the vector register base address corresponding to one of the M thread bundles in the M vector register base addresses, the instruction address offset corresponding to one of the M thread bundles, and the indirect addressing register corresponding to one of the M thread bundles. The cross-thread-bundle vector register space is the vector register space within the available range of other thread bundles besides the current thread bundle of the functional module of the execution unit, which is accessed through indirect addressing.

11. The method for calculating cooperative thread bundles according to claim 10, characterized in that, Also includes: The thread bundle resource allocation unit notifies the thread bundle scheduling and instruction issuing unit to send the multiple execution instructions to the execution unit.

12. The method for calculating cooperative thread bundles according to claim 10, characterized in that, The plurality of functional modules also access the vector register space corresponding to one of the M thread bundles in the vector register according to the vector register base address corresponding to one of the M thread bundles in the M vector register base address and the instruction address offset corresponding to one of the M thread bundles.

13. The method for calculating cooperative thread bundles according to claim 10, characterized in that, The M thread bundles run the same program file; Specifically, when one of the M thread bundles executes the indirect addressing register modification instruction of the program file, the one of the M thread bundles modifies the corresponding indirect addressing register.

14. The method for calculating cooperative thread bundles according to claim 13, characterized in that, The M thread bundle units also include different scalar registers, and the M thread bundles include a first thread bundle and a second thread bundle; The first thread bundle and the second thread bundle execute the same data loading program segment in the program file. The first thread bundle and the second thread bundle use different scalar parameters for calculation and use their respective indirect addressing registers to generate corresponding loading addresses, so as to load data read from different memory spaces into different positions in the vector register space corresponding to the loading address.

15. The method for calculating cooperative thread bundles according to claim 14, characterized in that, The M thread bundles also include a third thread bundle, a fourth thread bundle, and a fifth thread bundle, and the load address belongs to the vector register space in the vector register corresponding to the fifth thread bundle; The third thread bundle and the fourth thread bundle execute the same calculation program segment in the program file, and generate the corresponding source data address using their respective indirect addressing registers. The source data address belongs to the vector register space in the vector register corresponding to the fifth thread bundle. Specifically, the third thread bundle and the fourth thread bundle read the data loaded by at least one of the first thread bundle and the second thread bundle from the vector register space corresponding to the fifth thread bundle in the vector register according to the source data address, so as to perform calculations using the corresponding scalar parameters.

16. The method for calculating cooperative thread bundles according to claim 15, characterized in that, Also includes: The thread bundle synchronization unit receives the barrier instructions of the third thread bundle and the fourth thread bundle, and performs producer barrier detection on at least one of the first thread bundle and the second thread bundle according to the barrier instructions of the third thread bundle and the fourth thread bundle; In response to the completion of loading of at least one of the first thread bundle and the second thread bundle, the thread bundle synchronization unit determines that the third thread bundle and the fourth thread bundle have passed the producer barrier detection, and sends the execution instruction corresponding to the third thread bundle and the fourth thread bundle among the plurality of execution instructions to the execution unit, so that the execution unit runs the third thread bundle and the fourth thread bundle.

17. The method for calculating cooperative thread bundles according to claim 16, characterized in that, Also includes: The thread bundle synchronization unit receives the barrier instruction from the fifth thread bundle and performs consumer barrier detection on the third and fourth thread bundles according to the barrier instruction of the fifth thread bundle. In response to the completion of the calculations of the third and fourth thread bundles, the thread bundle synchronization unit determines that the fifth thread bundle has passed the consumer barrier detection and terminates the fifth thread bundle to release the vector register space corresponding to the fifth thread bundle in the vector register.

18. The method for calculating cooperative thread bundles according to claim 14, characterized in that, The M thread bundles include a third thread bundle, and the load address belongs to the vector register space in the vector register corresponding to the third thread bundle; The third thread bundle executes the program file and accesses data loaded by the first thread bundle and the second thread bundle respectively in the vector register space corresponding to the third thread bundle.

Images