Artificial intelligence chip, method for vector and scalar execution pipeline parallelism, computing device, medium and program product

Abstract

The present application relates to an artificial intelligence chip, a method for vector and scalar execution pipeline parallelism, a computing device, a medium and a program product. The artificial intelligence chip comprises an execution unit configured with a vector execution pipeline and a scalar execution pipeline, the scalar execution pipeline at least comprising: a scalar instruction decoding unit configured to at least acquire an operand type, address information and scalar operation control information of a scalar instruction; a scalar instruction operand acquisition unit configured to acquire an operand source of the scalar instruction; and a scalar instruction operation unit configured to perform scalar calculation based on at least the operand type, the operand source and the scalar operation control information of the scalar instruction, and write the calculation result to a scalar register group comprised in the execution unit. The present application can significantly improve the utilization rate of hardware resources and the actual computing capacity of the execution unit of the artificial intelligence chip.

Description

Technical Field

[0001] The embodiments of the present invention generally relate to the field of artificial intelligence, and more specifically to an artificial intelligence chip, a method for pipelined parallelism of vector and scalar execution, a computing device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] In traditional AI chips, such as, but not limited to, graphics processing units (GPUs), when executing single instruction multiple threads (SIMT) instructions, the instruction fetching unit of the execution unit retrieves multiple instructions from the instruction memory and issues each of the fetched vector or scalar instructions to the instruction decoding unit for decoding. The execution unit then retrieves vector or scalar data based on the decoded operation information and broadcasts the retrieved vector or scalar data along with its corresponding operation information to each vector operation thread to execute the corresponding vector operation. After the vector operation is completed, the calculated vector or scalar result is stored in the corresponding register. Firstly, the execution unit contains only a single execution pipeline shared by all types of instructions (e.g., vector and scalar instructions). Therefore, instructions from the same thread bundle or different thread bundles must be executed sequentially. Furthermore, scalar instructions can hinder the execution efficiency of vector instructions with higher performance requirements, leading to a gap between the actual computing power and the expected computing power of the AI ​​chip. Furthermore, the use of vector operation units to execute scalar instructions reduces the efficiency of the execution unit's hardware resources.

[0003] In summary, the shortcomings of traditional AI chips are that they significantly reduce the utilization rate of the hardware resources of the execution unit, thus limiting the actual computing power of the AI ​​chip. Summary of the Invention

[0004] This invention provides an artificial intelligence chip, a method for pipelined parallel execution of vectors and scalars, a computing device, a computer-readable storage medium, and a computer program product, which can significantly improve the utilization rate of hardware resources of the execution unit of the artificial intelligence chip and its actual computing power.

[0005] According to a first aspect of the present invention, an artificial intelligence chip is provided, comprising: an execution unit, the execution unit being configured with a vector execution pipeline and a scalar execution pipeline, the scalar execution pipeline comprising at least: a scalar instruction decoding unit configured to acquire at least the operand type, address information, and scalar operation control information of a scalar instruction; a scalar instruction operand acquisition unit configured to acquire the operand source of the scalar instruction; and a scalar instruction arithmetic unit configured to perform scalar computation based at least on the operand type, operand source, and scalar operation control information of the scalar instruction, and to write the computation result to a scalar register group included in the execution unit.

[0006] In some embodiments, the operands of scalar instructions are obtained from the scalar register group included in the execution unit; the vector execution pipeline includes at least: a vector instruction decoding unit, a vector instruction operand acquisition unit, and a vector instruction arithmetic unit.

[0007] In some embodiments, the instruction scheduling unit is configured to distinguish the instruction type of the received instructions, the instruction type including: vector instructions, scalar instructions, and merge instructions.

[0008] In some embodiments, the instruction scheduling unit is further configured to: issue vector instructions to a vector instruction decoding unit included in a vector execution pipeline, issue scalar instructions to the scalar instruction decoding unit, and issue merge instructions to a merge instruction decoding unit included in the execution unit.

[0009] In some embodiments, the merge instruction decoding unit is configured to: perform instruction decomposition and decoding on the received merge instruction to generate scalar part information and vector part information; and provide the scalar part information to the scalar instruction operand acquisition unit, and provide the vector part information to the vector instruction operand acquisition unit included in the vector execution pipeline.

[0010] In some embodiments, the merge instruction decoding unit is further configured to: parse a pass enable signal for data transfer between the scalar execution pipeline and the vector execution pipeline based on the received merge instruction, so as to enable the scalar instruction arithmetic unit to at least partially use the calculation result of the vector instruction arithmetic unit as the operand source of the scalar instruction arithmetic unit.

[0011] In some embodiments, the merge instruction decoding unit is further configured to specify whether the transmission enable signal is "valid" via an instruction.

[0012] In some embodiments, the merge instruction decoding unit is further configured to: in response to determining that the pass enable signal has not been specified as "valid" by an instruction, and that the scalar register address associated with the operand associated with the scalar part information is the same as the scalar register address associated with the computation result associated with the vector part information, to set the pass enable signal of the operand to "valid", and to pass the pass enable signal of the operand set to "valid" to the scalar instruction arithmetic unit.

[0013] In some embodiments, the vector-to-scalar conversion unit is configured to convert the computation result of the vector instruction operation unit into scalar data.

[0014] In some embodiments, the scalar instruction arithmetic unit is further configured to: in response to determining that a pass enable signal for data transfer between the scalar execution pipeline and the vector execution pipeline is set to "valid", the scalar instruction arithmetic unit obtains scalar data via a vector-to-scalar conversion unit as an operand source; in response to determining that a pass enable signal for data transfer between the scalar execution pipeline and the vector execution pipeline is not set to "valid", the scalar instruction arithmetic unit obtains the operand source of the scalar instruction via a scalar instruction operand acquisition unit.

[0015] In some embodiments, the instruction scheduling unit includes: an instruction classification unit for each thread bundle, configured to independently classify instructions for the corresponding thread bundle; and a polling arbitrator, the polling arbitrator including a vector polling arbitrator, a scalar instruction polling arbitrator, and a merged instruction polling arbitrator, the polling arbitrator being configured to select vector instructions, scalar instructions, or merged instructions of the thread bundle based on a polling mechanism so as to be correspondingly issued to the vector instruction decoding unit, the scalar instruction decoding unit, or the merged instruction decoding unit.

[0016] In some embodiments, the merge instruction polling arbiter is configured to: in response to determining that a merge instruction has been detected, prevent the vector polling arbiter and the scalar instruction polling arbiter from selecting and issuing vector instructions and scalar instructions of the thread bundle.

[0017] According to a second aspect of the invention, a method for parallelizing vector and scalar execution pipelines is also provided, the method operating on the aforementioned artificial intelligence chip, the method comprising: distinguishing the instruction type of a received instruction, the instruction type including at least: vector instructions and scalar instructions; and issuing vector instructions to a vector execution pipeline included in the artificial intelligence chip, and issuing scalar instructions to a scalar execution pipeline included in the artificial intelligence chip.

[0018] In some embodiments, the instruction type further includes: a merge instruction, and the method further includes: performing instruction decomposition and decoding on the received merge instruction to generate scalar part information and vector part information; and providing the scalar part information to the scalar instruction operand acquisition unit included in the scalar execution pipeline, and providing the vector part information to the vector instruction operand acquisition unit included in the vector execution pipeline.

[0019] In some embodiments, the method further includes specifying whether the pass enable signal is "valid" via an instruction, and in response to determining that the pass enable signal has not been specified as "valid" via an instruction, and that the scalar register address associated with the operand associated with the scalar partial information is the same as the scalar register address associated with the computation result associated with the vector partial information, the method sets the pass enable signal of the operand to "valid", and passes the pass enable signal of the operand set to "valid" to the scalar instruction arithmetic unit.

[0020] In some embodiments, in response to determining that the pass enable signal for data transfer between the scalar execution pipeline and the vector execution pipeline is set to "valid", the scalar instruction arithmetic unit obtains scalar data via the vector-to-scalar conversion unit as an operand source; and in response to determining that the operand pass enable signal is not set to "valid", the scalar instruction arithmetic unit obtains scalar data via the scalar instruction operand acquisition unit as an operand source.

[0021] According to a third aspect of the present invention, a computing device is also provided. The computing device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the computing device to perform the method of the first aspect of the present invention.

[0022] According to a fourth aspect of the present invention, a computer-readable storage medium is also provided. The computer-readable storage medium stores a computer program that, when executed by a machine, performs the method of the first aspect of the present invention.

[0023] According to a fifth aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a machine, performs the method of the first aspect of the present invention.

[0024] This invention configures a scalar execution pipeline in addition to a vector execution pipeline in the execution unit. The scalar execution pipeline further includes a scalar instruction decoding unit, a scalar instruction operand fetching unit, and a scalar instruction arithmetic unit for executing scalar instructions and storing the corresponding calculation results in a scalar register set. Therefore, this invention can execute vector and scalar calculations in parallel through relatively independent vector and scalar execution pipelines. This allows scalar instructions to be calculated in the scalar execution pipeline instead of running in the vector execution pipeline, ensuring that the vector execution pipeline contains only vector instructions. This guarantees the multi-threaded parallelism of the vector arithmetic unit, improves the utilization of the vector arithmetic unit, and further releases the actual computing power of the artificial intelligence chip. Therefore, this invention can significantly improve the utilization rate of hardware resources and the actual computing power of the execution unit of an artificial intelligence chip.

[0025] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0026] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements.

[0027] Figure 1 The diagram illustrates the structure of an artificial intelligence chip according to some embodiments of the present invention.

[0028] Figure 2 A schematic diagram of an artificial intelligence chip according to some embodiments of the present invention is shown.

[0029] Figure 3 A schematic diagram of an instruction acquisition unit, an instruction scheduling unit, and an instruction decoding unit according to some embodiments of the present invention is shown.

[0030] Figure 4 A schematic diagram of an instruction scheduling unit according to some embodiments of the present invention is shown.

[0031] Figure 5 A schematic diagram of a computing apparatus for implementing a method for pipelined parallelism of vector and scalar execution is shown according to some embodiments of the present invention.

[0032] Figure 6 A flowchart of a method for pipelined parallelism of vector and scalar execution is shown according to some embodiments of the present invention.

[0033] Figure 7A flowchart is shown below illustrating a method for a scalar instruction arithmetic unit to obtain scalar data via vector-to-scalar unit as an operand source, according to some embodiments of the present invention.

[0034] In the various figures, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0035] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0036] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects.

[0037] As mentioned earlier, the shortcomings of traditional artificial intelligence chips are that they significantly reduce the utilization rate of the hardware resources of the execution unit of the artificial intelligence chip, thus limiting the actual computing power of the artificial intelligence chip.

[0038] Furthermore, traditional instruction execution pipelines involve an instruction fetch unit reading multiple instructions from instruction memory and issuing them sequentially to an instruction decode unit for decoding various instruction types. After obtaining operand type and address information from the decode unit, the corresponding operands are retrieved from either a vector register set or a scalar register set based on this information. Traditional pipelines also involve obtaining operation control information from the decode unit for various instructions, enabling the vector arithmetic unit to perform corresponding operations on the retrieved operands. Specifically, for scalar operands, they must be converted to vector operands before being processed by the vector arithmetic unit. After obtaining the vector operation result, it is then converted back to a scalar operation result before being stored in a scalar register. Therefore, the execution unit contains only a single pipeline shared by all instruction types (e.g., vector instructions and scalar instructions), which weakens the multi-threaded parallelism of the vector arithmetic unit and significantly reduces its utilization.

[0039] In summary, the shortcomings of the traditional instruction execution pipeline method are that it significantly reduces the utilization rate of the hardware resources of the execution unit of the artificial intelligence chip, thus limiting the actual computing power of the artificial intelligence chip.

[0040] To at least partially address one or more of the aforementioned problems and other potential issues, an exemplary embodiment of the present invention proposes an artificial intelligence chip. In this AI chip, in addition to a vector execution pipeline, the execution unit is also equipped with a scalar execution pipeline. The scalar execution pipeline further includes a scalar instruction decoding unit, a scalar instruction operand acquisition unit, and a scalar instruction arithmetic unit for executing scalar instructions and storing the corresponding calculation results in a scalar register set. Thus, the present invention can execute vector and scalar calculations in parallel through relatively independent vector and scalar execution pipelines, enabling scalar instructions to be calculated using the scalar execution pipeline. This ensures the parallelism of multi-threading in the vector arithmetic unit, improves the utilization rate of the vector arithmetic unit, and further releases the actual computing power of the AI ​​chip. Therefore, the present invention can significantly improve the utilization rate of the execution unit hardware resources and the actual computing power of the AI ​​chip.

[0041] The following combination Figure 1 , Figure 2 and Figure 3 This describes an artificial intelligence chip 100 according to some embodiments of the present invention. For example... Figure 1 As shown, the artificial intelligence chip 100 includes at least an execution unit 200. The execution unit 200 includes, for example, at least a vector execution pipeline 102 and a scalar execution pipeline 104. The scalar execution pipeline 104 further includes, for example, at least a scalar instruction decoding unit 204, a scalar instruction operand fetching unit 206, and a scalar instruction arithmetic unit 208. It should be understood that the artificial intelligence chip 100 may also include additional actions not shown and / or the structures shown may be omitted; the scope of the invention is not limited in this respect.

[0042] Regarding the scalar instruction decoding unit 204, it is configured, for example, to acquire at least the operand type and address information of the scalar instruction. In some embodiments, the scalar instruction decoding unit 204 is further configured to acquire scalar operation control information of the scalar instruction.

[0043] Regarding the scalar instruction operand acquisition unit 206, it is configured, for example, to acquire the operand source of a scalar instruction. In some embodiments, after receiving the operand type and address information decoded by the scalar instruction decoding unit 204, the scalar instruction operand acquisition unit 206 uses the scalar register group 210 included in the execution unit 200 as the operand source and acquires the corresponding operand from the scalar register group 210.

[0044] Regarding the scalar instruction arithmetic unit 208, it is configured, for example, to perform scalar calculations based at least on the operand type, operand source, and scalar operation control information of the scalar instruction, and to write the calculation results to the scalar register group included in the execution unit.

[0045] In the above scheme, the present invention can execute vector computation and scalar computation in parallel through relatively independent vector execution pipelines and scalar execution pipelines. This allows scalar instructions to be computed through the scalar execution pipeline, ensuring the parallel capability of multi-threading in the vector operation unit, improving the utilization rate of the vector operation unit, and further releasing the actual computing power of the artificial intelligence chip. Therefore, the present invention can significantly improve the utilization rate of hardware resources and the actual computing power of the execution unit of the artificial intelligence chip.

[0046] The following will combine Figure 2 The artificial intelligence chip 100 of some embodiments of the present invention is further described. For example... Figure 2 As shown, the artificial intelligence chip 100 includes at least an execution unit 200. The vector execution pipeline 102 includes at least a vector instruction decoding unit 212, a vector instruction operand acquisition unit 214, and a vector instruction arithmetic unit 216.

[0047] Regarding the vector instruction decoding unit 212, it is configured, for example, to decompose and decode the vector instruction issued by the instruction scheduling unit 220 to obtain the vector operand type and vector operand address information of the vector instruction. In some embodiments, the vector instruction decoding unit 212 is further configured to obtain vector operation control information of the vector instruction.

[0048] Regarding the vector instruction operand acquisition unit 214, it is configured, for example, to read vector type operands from the vector register group 218 inside the execution unit 200, where the operand source is the vector register group 218. In some embodiments, the vector instruction operand acquisition unit 214 is also configured, for example, to convert the scalar type operands read from the scalar register group 210 via the scalar-to-vector conversion unit 226 inside the execution unit 200 to obtain the corresponding vector type operands, where the operand source is the scalar register group 210.

[0049] Regarding the vector instruction arithmetic unit 216, it is configured, for example, to perform vector calculations based at least on operands and operation control information of the vector instruction, and write the corresponding calculation results to the vector register set 218. In some embodiments, the calculation results of the vector instruction are converted into corresponding scalar calculation results via the vector conversion scalar unit 224, and the scalar calculation results are written to the scalar register set 210. In some embodiments, the vector instruction arithmetic unit 216 includes, for example, one or more thread arithmetic units, such as... Figure 2 The shown are thread 1 operation unit, thread 2 operation unit, thread 3 operation unit, ..., thread N operation unit.

[0050] In some embodiments, such as Figure 2 As shown, the artificial intelligence chip 100 further includes: an instruction scheduling unit 220, a merged instruction decoding unit 222, and a vector-to-scalar conversion unit 224. In some embodiments, the artificial intelligence chip 100 further includes: an instruction memory 230 and an instruction fetching unit 228.

[0051] Regarding the instruction scheduling unit 220, it is configured, for example, to distinguish the instruction type of the received instructions, including: vector instructions, scalar instructions, and merge instructions.

[0052] The following will combine Figure 3 The instruction scheduling unit 220 of the present invention will be described in detail below. For example... Figure 3 As shown, the instruction scheduling unit 220 further includes, for example, an instruction classifier. In some embodiments, the instruction classifier is configured to: in response to receiving a vector instruction, scalar instruction, or merge instruction from the instruction acquisition unit 228, distinguish the instruction based on the operation type of the instruction, and distribute the current single-threaded instruction to the vector instruction decoding unit 212, the merge instruction decoding unit 222, or the scalar instruction decoding unit 204 of the execution unit 200. It should be understood that the instruction scheduling unit may also include additional actions not shown and / or the structures shown may be omitted, and the scope of the invention is not limited in this respect.

[0053] In some embodiments, the instruction scheduling unit 220 is further configured, for example, to: issue vector instructions to the vector instruction decoding unit 212 included in the vector execution pipeline, issue scalar instructions to the scalar instruction decoding unit 204, and issue merge instructions to the merge instruction decoding unit 222 included in the execution unit. It should be understood that after distinguishing one or more received instructions (e.g., vector instructions, scalar instructions, merge instructions) according to their instruction type, the instruction scheduling unit 220 issues instructions of the same type to the corresponding instruction decoding unit (e.g., vector instruction decoding unit 212, scalar instruction decoding unit 204, merge instruction decoding unit 222).

[0054] Regarding the merge instruction decoding unit 222, it is configured, for example, to: perform instruction decomposition and decoding on the received merge instruction in order to generate scalar part information and vector part information; and to provide the scalar part information to the scalar instruction operand acquisition unit 206, and to provide the vector part information to the vector instruction operand acquisition unit 214 included in the vector execution pipeline.

[0055] The scalar information includes, for example, indications of operand types, address information, and scalar operation control information related to scalar operations, so that the scalar instruction operand acquisition unit 206 can acquire operands related to scalar operations based on the scalar information; the vector information includes, for example, indications of operand types, address information, and vector operation control information related to vector operations, so that the vector instruction operand acquisition unit 214 can acquire operands related to vector operations based on the vector information.

[0056] In some embodiments, the merge instruction decoding unit 222 is configured, for example, to: parse a pass enable signal for data transfer between the scalar execution pipeline and the vector execution pipeline based on the received merge instruction, so as to enable the scalar instruction arithmetic unit 208 to at least partially use the calculation result of the vector instruction arithmetic unit 216 as the operand source of the scalar instruction arithmetic unit 208.

[0057] It should be understood that in some embodiments, at least some of the scalar operands in the scalar instruction arithmetic unit 208 have data correlation with at least some of the calculation results of the vector instruction arithmetic unit 216. In some embodiments, after generating the calculation result, the vector instruction arithmetic unit 216, based on the transmission enable signal, transmits the calculation result to the scalar instruction arithmetic unit 208 via the vector conversion scalar unit 224, such as... Figure 2 As indicated by mark 232.

[0058] In some embodiments, the merge instruction decoding unit 222 is further configured to: in response to determining that the scalar register address associated with the operand associated with the scalar partial information is the same as the scalar register address associated with the calculation result associated with the vector partial information, cause the scalar instruction arithmetic unit 208 to at least partially use the calculation result of vector instruction arithmetic unit 216 as the operand source for scalar instruction arithmetic unit 208. It should be understood that the process of determining whether the scalar register address associated with the operand associated with the scalar partial information is the same as the scalar register address associated with the calculation result associated with the vector partial information in this embodiment can be implemented by existing hardware devices, and the present invention does not limit the structure of the hardware device itself.

[0059] In some embodiments, the merge instruction decoding unit 222 is further configured to: specify whether the pass enable signal is "valid" via an instruction. Further, the merge instruction decoding unit 222 is also configured to: in response to determining that the pass enable signal has not been specified as "valid" via an instruction, and that the scalar register address associated with the operand associated with the scalar portion information is the same as the scalar register address associated with the computation result associated with the vector portion information, set the pass enable signal of the operand to "valid", and pass the pass enable signal of the operand set to "valid" to the scalar instruction arithmetic unit 208.

[0060] The method for the vector-to-scalar conversion unit 224 to convert vector calculation results into corresponding scalar results includes, for example, extracting the thread value of the smallest valid thread from the vector calculation results and using the thread value of the smallest valid thread as the corresponding scalar calculation result after conversion; or selecting a portion of thread values ​​from the results of each thread in the vector results and concatenating them to generate a scalar calculation result. Here, a valid thread is indicated, for example, by a 1-bit barrier signal corresponding to each thread within the same instruction issue cycle of the SIMT instruction architecture. When the barrier signal of a thread is set to 1, it indicates that the thread is "valid"; if the signal is 0, it is determined to be an invalid thread, for example, the thread is suspended and does not participate in the calculation and synchronization of this cycle.

[0061] Regarding the vector-to-scalar conversion unit 224, it is configured, for example, to convert the computation result of the vector instruction operation unit into scalar data (i.e., scalar computation result). Figure 2 As shown, the scalar calculation result output by the vector-to-scalar conversion unit 224 can be provided to the scalar register group 210. The scalar calculation result output by the vector-to-scalar conversion unit 224 can also be provided to the scalar instruction arithmetic unit 208. For example, in some embodiments, the pass enable signal of the operand set to "valid" is passed to the scalar register group 210, and the scalar register group 210 can directly obtain the scalar calculation result output by the vector-to-scalar conversion unit 224, such as... Figure 2 As indicated by mark 232.

[0062] The following will combine Figure 4 The instruction scheduling unit of an embodiment of the present invention is described. It should be understood that the instruction scheduling unit may also include additional actions not shown and / or the structures shown may be omitted, and the scope of the present invention is not limited in this respect.

[0063] In some embodiments, the instruction scheduling unit includes: an instruction classification unit for each thread bundle, configured to independently classify instructions for the corresponding thread bundle; and a polling arbitrator, the polling arbitrator including a vector polling arbitrator, a scalar instruction polling arbitrator, and a merged instruction polling arbitrator, the polling arbitrator being configured to select vector instructions, scalar instructions, or merged instructions of the thread bundle based on a polling mechanism so as to be correspondingly issued to the vector instruction decoding unit, the scalar instruction decoding unit, or the merged instruction decoding unit.

[0064] like Figure 4 As shown, the instruction scheduling unit 220 includes, for example, an instruction classification unit for each thread bundle and a polling arbiter. The instruction classification unit for each thread bundle includes, for example, a thread bundle 1 instruction classifier 2202, a thread bundle 2 instruction classifier 2204, and a thread bundle 3 instruction classifier 2206. The polling arbiter includes, for example, a vector instruction polling arbiter 2208, a merged instruction polling arbiter 2210, and a scalar instruction polling arbiter 2212.

[0065] In some embodiments, each thread bundle instruction classifier independently classifies the instructions received by the current thread and distributes instructions of various operation types to the corresponding polling arbitrators of the operation types. The polling arbitrators of the corresponding operation types then issue instructions to the corresponding decoding units based on a polling mechanism. It should be understood that this invention does not limit the number of thread bundles; the thread bundle 1 instruction classifier, thread bundle 2 instruction classifier, and thread bundle 3 instruction classifier mentioned here are for illustrative purposes only and not for limitation.

[0066] Regarding the method by which the polling arbitrator issues instructions to the decoding unit of the corresponding operation type based on the polling mechanism, it includes, for example, the polling arbitrator (e.g., vector instruction polling arbitrator 2208, merge instruction polling arbitrator 2210, scalar instruction polling arbitrator 2212) collects the instructions of the corresponding operation type distributed by each thread bundle instruction classifier. In the initial state, based on the thread bundle number in ascending order, the instructions of one of the thread bundles are selected and issued to the subsequent decoding unit of the corresponding operation type (e.g., vector instruction decoding unit 212, merge instruction decoding unit 222, scalar instruction decoding unit 204). Subsequently, the priority of the currently selected thread bundle is reduced to the lowest, and the thread bundle with the highest priority is set as the highest priority in the next round of arbitration. The thread bundle with the highest priority indicates, for example, the thread with the largest number, and the numbering space is contiguous, that is, the largest number and the smallest number are adjacent.

[0067] In some embodiments, the merge instruction polling arbiter is configured, for example, to prevent the vector polling arbiter and the scalar instruction polling arbiter from selecting and issuing vector instructions and scalar instructions of the thread bundle in response to determining that a merge instruction has been detected.

[0068] In the above scheme, precise scheduling of the vector execution pipeline and the scalar execution pipeline is achieved through the instruction classification unit and the polling arbitrator. At any given time, the instruction scheduling unit can simultaneously schedule one vector instruction and one scalar instruction from different thread bundles, and issue the vector instruction or scalar instruction to the corresponding vector instruction decoding unit or scalar instruction decoding unit. Alternatively, it can simultaneously schedule one merge instruction from a certain thread bundle and issue the merge instruction to the corresponding merge instruction decoding unit. Furthermore, the above scheme uses the merge instruction polling arbitrator to prevent pipeline occupancy caused by the instruction issuance from the vector polling arbitrator and the scalar polling arbitrator after receiving the merge instruction. This avoids pipeline competition and conflicts with vector or scalar instructions from other threads during the execution of the merge instruction, thus improving the stability of dual-pipeline parallelism in the AI ​​chip.

[0069] The following combination Figure 5 This describes a computing device 500 that implements a method for pipelined parallelism of vector and scalar execution. For example... Figure 5 As shown, the computing device 500 may have one or more processing units, including dedicated processing units such as graphics processing units (GPUs), field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), or general-purpose computing on graphics processing units (GPGPUs), as well as general-purpose processing units such as CPUs. The computing device 500 also includes at least: an instruction type differentiation module 502 and a vector instruction and scalar instruction dispatch module 504.

[0070] Regarding the instruction type differentiation module 502, it is used to differentiate the instruction type of the received instruction, and the instruction type includes at least: vector instruction and scalar instruction.

[0071] Regarding the vector instruction and scalar instruction issuing module 504, it is used to issue vector instructions to the vector execution pipeline included in the artificial intelligence chip, and to issue scalar instructions to the scalar execution pipeline included in the artificial intelligence chip.

[0072] The following will combine Figure 2 and Figure 6 A method 600 for pipelined parallelism of vector and scalar execution is described according to embodiments of the present invention. It should be understood that method 600 can, for example, be implemented in... Figure 5The described computing device 500 is executed. Method 600 can also be performed, for example, at... Figure 1 The described artificial intelligence chip 100 is executed. Method 600 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0073] At step 602, the computing device 500 distinguishes the instruction type of the received instruction, the instruction type including at least: vector instructions and scalar instructions.

[0074] In some embodiments, the instruction type further includes: a merge instruction. A merge instruction is an instruction that simultaneously occupies both the vector execution pipeline and the scalar execution pipeline. Table 1 below illustrates an instruction for performing vector data transfer first, followed by scalar addition, i.e., merge instruction 1 "VMOV v2s, a; &ADD d, v2s, b;". Table 2 below illustrates an instruction for extracting the valid thread portion from each thread in the vector operation result and concatenating the valid threads to generate the corresponding scalar calculation result, i.e., merge instruction 2 "VCMP.EQ v2s, a, 0; &SEL d, PC2, V2S, PC1;", where the semantics of SEL d, PC2, V2S, PC1 include, for example: d = if( V2S != 0)? PC2 : PC1, that is, if V2S is all 0 (all False), then PC2 is returned to d; otherwise, PC1 is returned to d.

[0075] In some embodiments, to avoid pipeline competition and conflicts with vector or scalar instructions of other threads during the execution of merge instructions, Figure 4 The merge instruction polling arbitrator 2210 shown, in response to determining that a merge instruction has been detected, prevents the vector polling arbitrator and the scalar instruction polling arbitrator 2212 from selecting the vector instruction and scalar instruction of the thread bundle.

[0076] In some embodiments, the instruction memory 230, in response to the instruction fetching request of the instruction fetching unit 228, sends all types of arithmetic instructions to the instruction scheduling unit 220. All types of arithmetic instructions include, for example, vector instructions, scalar instructions, and merge instructions.

[0077] At step 604, the computing device 500 issues vector instructions to the vector execution pipeline included in the artificial intelligence chip, and issues scalar instructions to the scalar execution pipeline included in the artificial intelligence chip.

[0078] Regarding the method by which the computing device 500 issues vector instructions to the vector execution pipeline included in the artificial intelligence chip, such as... Figure 2As shown, it includes, for example, the following: the instruction scheduling unit 220 sends a vector instruction to the vector instruction decoding unit 212, so that the vector instruction decoding unit 212 can decompose and decode the vector instruction, thereby generating the vector operand type, vector operand address information, and vector operation control information of the vector instruction. The vector instruction operand acquisition unit 214 reads data from the vector register group 218 or the scalar register group 210 as the operand source, based at least on the vector operand type and vector operand address information, and sends the acquired operand source and the vector operation control information obtained by the vector instruction decoding unit to the vector instruction arithmetic unit 216 to execute the corresponding vector operation.

[0079] Regarding the method by which the computing device 500 issues scalar instructions to the scalar execution pipeline included in the artificial intelligence chip, such as... Figure 2 As shown, it includes, for example, the following: Instruction scheduling unit 220 sends a scalar instruction to scalar instruction decoding unit 204, so that scalar instruction decoding unit 204 can decompose and decode the scalar instruction to generate scalar instruction operand type, scalar operand address information, and scalar operation control information. Scalar instruction operand acquisition unit 206 reads data from scalar register group 210 or vector conversion scalar unit 224 based on scalar operand type and scalar operand address information as operand source, and sends the acquired operand source and the scalar operation control information obtained by scalar instruction decoding unit to scalar instruction arithmetic unit 208 to execute the corresponding scalar operation.

[0080] In some embodiments, method 600 further includes: computing device 500 performing instruction decomposition and decoding on the received merge instruction to generate scalar part information and vector part information; and the instruction type of the scalar part information further includes: providing the merge instruction to a scalar instruction operand acquisition unit included in the scalar execution pipeline, and providing the vector part information to a vector instruction operand acquisition unit included in the vector execution pipeline.

[0081] Through the above scheme, the instruction scheduling unit, based on the type of the operation instruction, distributes the operation type to the corresponding vector execution pipeline or scalar execution pipeline. This allows the execution unit to execute vector and scalar instructions independently through independent pipelines, thereby scheduling vector and scalar instructions to the execution unit in parallel. This relieves the workload of the vector instruction pipeline in processing scalar instructions and increases the throughput of instruction execution. Therefore, this invention can significantly improve the utilization rate of hardware resources and the actual computing power of the execution unit in artificial intelligence chips.

[0082] The following will combine Figure 2 and Figure 7This invention describes a method 700 for a scalar instruction arithmetic unit to obtain scalar data via vector-to-scalar unit as an operand source. It should be understood that method 700 can, for example, be implemented in... Figure 5 The described computing device 500 is executed. Method 700 can also be performed, for example, at... Figure 1 The described artificial intelligence chip 100 is executed. Method 700 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0083] At step 702, the computing device 500 performs instruction decomposition and decoding on the received merging instruction in order to generate scalar part information and vector part information.

[0084] Regarding the vector portion of the information, in some embodiments, it includes at least: vector operand type, vector operand address information, and vector operation control information. Regarding the scalar portion of the information, it includes at least: scalar operand type, scalar operand address information, and scalar operation control information.

[0085] At step 704, the computing device 500 provides scalar portion information to the scalar instruction operand acquisition unit included in the scalar execution pipeline, and provides vector portion information to the vector instruction operand acquisition unit included in the vector execution pipeline.

[0086] like Figure 2 As shown, the merging instruction decoding unit 222 of the computing device 500 provides scalar part information to the scalar instruction operand acquisition unit 206 included in the scalar execution pipeline, and provides vector part information to the vector instruction operand acquisition unit 214 included in the vector execution pipeline.

[0087] At step 706, the computing device 500 specifies by instruction whether the transmission enable signal is "valid".

[0088] In some embodiments, the method of the computing device 500 for disassembling and decoding the received merge instruction further includes, for example, disassembling and decoding the received merge instruction to generate a specified instruction, which is used to directly specify the transmission enable signal without address judgment. The specified instruction directly specifies the operand transmission enable signal as "valid" or "invalid", and transmits the transmission enable signal to the scalar instruction arithmetic unit 208 so that the scalar instruction arithmetic unit 208 selects the corresponding operand source.

[0089] At step 708, in response to determining that the pass enable signal has not been specified as "valid" by the instruction, and that the scalar register address associated with the operand associated with the scalar part information is the same as the scalar register address associated with the calculation result associated with the vector part information, the computing device 500 sets the operand pass enable signal to "valid" and passes the operand pass enable signal set to "valid" to the scalar instruction arithmetic unit.

[0090] like Figure 2 As shown, in response to determining that the pass enable signal has not been specified as "valid" by the instruction, and that the scalar register address associated with the operand of the scalar part information is the same as the scalar register address associated with the calculation result of the vector part information, the scalar instruction arithmetic unit 208 of computing device 500 sets the operand pass enable signal to "valid", and passes the operand pass enable signal set to "valid" to scalar instruction arithmetic unit 208.

[0091] In some embodiments, in response to determining that the pass enable signal is specified as "valid" by an instruction, the scalar instruction arithmetic unit 208 of the computing device 500 sets the operand pass enable signal to "valid" and passes the operand pass enable signal set to "valid" to the scalar instruction arithmetic unit 208.

[0092] At step 710, in response to determining that a transfer enable signal for data transfer between the scalar execution pipeline and the vector execution pipeline is set to "valid", the scalar instruction arithmetic unit obtains scalar data via the vector-to-scalar conversion unit as an operand source.

[0093] like Figure 2 As shown, the scalar instruction operand acquisition unit 206 of the computing device 500 transmits the operand pass enable signal set to "valid" to the scalar instruction arithmetic unit 208, so that the scalar instruction arithmetic unit 208 selects the scalar calculation result generated by the vector transformation scalar unit 224 as the operand source based on the operand pass enable signal set to "valid", as shown by label 232.

[0094] For example, the vector instruction arithmetic unit 216 performs vector operations based on the vector portion information in the merge instruction, and inputs the vector operation result into the vector-to-scalar conversion unit 224 to convert the vector operation result into the corresponding scalar calculation result. This converted scalar calculation result then serves as the operand source for the next instruction, allowing the scalar instruction arithmetic unit 208 to perform the corresponding scalar operation based on the vector portion information of the merge instruction. It should be understood that there is a correlation between the operation result of the vector portion information in the merge instruction and the operand of the scalar portion information.

[0095] At step 712, in response to the fact that the operand transfer enable signal is not set to "valid", the scalar instruction arithmetic unit obtains scalar data via the scalar instruction operand acquisition unit as the operand source.

[0096] like Figure 2 As shown, in response to the fact that the scalar instruction operand acquisition unit 206 of the computing device 500 is not set to "valid", the scalar instruction arithmetic unit 208 obtains scalar data via the scalar instruction operand acquisition unit 206 as the operand source, as shown by reference numeral 234.

[0097] The method for setting the pass enable signal to "valid" includes, for example, setting the pass enable signal to "on" and writing the flag bit associated with the pass enable signal to "1". In this case, the scalar instruction arithmetic unit 208 obtains scalar data via vector conversion scalar unit 224 as the operand source (as shown in flag 232). The method for setting the pass enable signal to "valid" includes, for example, setting the pass enable signal to "zero" and writing the flag bit associated with the pass enable signal to "0". In this case, the scalar instruction arithmetic unit 208 obtains the operand source of the scalar instruction via scalar instruction operand acquisition unit 206 (as shown in flag 234).

[0098] It should be understood that when the scalar register address associated with the operand of the scalar part information is the same as the scalar register address associated with the calculation result of the vector part information, there is a correlation between the vector operation performed by the previous vector part information and the scalar operation performed by the subsequent scalar part information. The output method of the calculation result of the previous vector operation is controlled by setting whether the pass enable signal is "valid". It should also be understood that this invention, by introducing a merge instruction that simultaneously occupies both the vector execution pipeline and the scalar execution pipeline, and by configuring the pass enable signal, achieves data transfer between the vector execution pipeline and the scalar execution pipeline. That is, the scalar execution pipeline can selectively obtain the calculation result of the vector execution pipeline and use this calculation result as the operand source for the scalar execution pipeline, avoiding multiple accesses to the register set. Therefore, this invention significantly shortens the data access chain between instructions with data correlation.

[0099] Table 1 below schematically illustrates the implementation code for selecting the smallest valid thread from the vector execution pipeline results as the operand source for the scalar execution pipeline.

[0100] As shown in the pseudocode in Table 1, the vector computation results (a1, a2, … an) are assigned to vector a, where coordinates a1 to an represent the computation results corresponding to threads 1 to n, and each thread can be independently configured as a valid or invalid thread. The scalar initial variable is assigned to b. The coordinates of vector a are traversed in ascending order (from 1 to n), and it is determined whether the thread corresponding to the current coordinate is a valid thread. If it is a valid thread, the computation result corresponding to the current valid thread is copied to "v2s" (e.g., vector to scalar), and then the current traversal loop is exited. Here, "v2s" represents a special marker passed from the vector execution pipeline to the scalar execution pipeline. Subsequently, the obtained thread value representing the smallest valid thread number, "v2s", is added to the scalar variable "b" to generate the corresponding scalar result after transformation.

[0101]

[0102] Table 1

[0103] The calculation process illustrated by the pseudocode in Table 1 can also be understood: the thread value of the smallest valid thread in vector a is automatically assigned to "v2s" via a vector data transfer instruction (e.g., VMOV instruction). The scalar arithmetic unit performs a scalar addition instruction (e.g., ADD instruction) on the value of "v2s" and the scalar variable "b" to obtain the corresponding scalar calculation result.

[0104] Table 2 below illustrates the instructions used to extract the valid thread portion from each thread in the vector operation results and concatenate the valid threads to generate the corresponding scalar computation result. As shown in the pseudocode in Table 2, the vector computation results a1, a2, … an are assigned to vector a, where coordinates a1 to an represent the computation results corresponding to threads 1 to n, and each thread can be independently configured as a valid or invalid thread. For each thread value in vector a, a judgment is made: if the thread value corresponding to any coordinate in vector a is 0, the address of the next instruction is PC1; otherwise, the address of the next instruction is PC2. After the judgment, the address of the next instruction, PC1 or PC2, is assigned to the variable d, and d is assigned to the variable "next_PC" to be passed to the scalar operation unit, so that the scalar operation unit can directly obtain d as its operand for scalar operations through "next_PC".

[0105] The pseudocode calculation process shown in Table 2 can also be understood as follows: The value of each thread is compared with 0 by a vector comparison instruction (e.g., VCMP.EQ instruction). If they are the same, "True" = 1 is returned; if they are different, "False" = 0 is returned. The results of "True" or "False" of each 1 bit in the valid thread are concatenated to generate a scalar result and assigned to the marker "v2s". "v2s" contains the results of "True" or "False" of each of the 32 threads. "v2s" represents a special marker passed from the vector execution pipeline to the scalar execution pipeline. Upon receiving "v2s", the scalar execution pipeline uses a scalar selection instruction (e.g., the SEL instruction) to determine whether all thread values ​​in "v2s" are False. If so, it returns PC2 to variable d; otherwise, it returns address PC1 to variable d. After the current merge instruction (e.g., including VCMP.EQ and SEL instructions) is executed, it generates the corresponding scalar result d, which is then used by subsequent scalar execution pipelines to execute scalar jump instructions (e.g., JUMP instructions) based on variable d and obtain the jump target, i.e., next_PC.

[0106]

[0107] Table 2

[0108] It should be understood that this invention greatly shortens the execution latency of a single thread by introducing a merge instruction format, and parallelism between vector execution pipelines and scalar execution pipelines can be achieved with a single merge instruction.

[0109] In the above scheme, in response to the scalar register address associated with the operand containing the scalar partial information being the same as the scalar register address associated with the calculation result containing the vector partial information, the operand pass enable signal is set to "valid" and passed to the scalar instruction arithmetic unit. In response to the operand pass enable signal being set to "valid," the scalar instruction arithmetic unit obtains scalar data via the vector-to-scalar conversion unit as the operand source. Based on this merge instruction, the present invention enables the scalar instruction arithmetic unit to partially or fully use the calculation result of the vector instruction arithmetic unit as the operand source for its own scalar data. Therefore, for instructions within the same thread with data correlation, the present invention can directly convert the vector operation result of the vector arithmetic unit into a scalar, which can then be directly used as the operand source for scalar operations, avoiding multiple accesses to the register set. Therefore, the present invention can significantly shorten the execution latency of a single-threaded execution loop and improve its execution throughput.

[0110] The various processes and handling described above, such as methods 600 and 700, can be executed at a computing device. This computing device may include, for example, at least one processor (at least one graphics processor and at least one central processing unit); and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor. In some embodiments, methods 600 and 700 may be implemented as a computer software program or program product tangibly contained in a machine-readable medium. In some embodiments, part or all of the computer program may be loaded and / or installed on the computing device via read-only memory (ROM) and / or a communication unit. When the computer program is loaded into random-access memory (RAM) and executed by the GPU and CPU, one or more actions of methods 600 and 700 described above can be performed.

[0111] This invention can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention. The computer-readable storage medium may be a tangible device capable of holding and storing instructions used by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.

[0112] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network, to an external computer or external storage device. Various aspects of the invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0113] These computer-readable program instructions can be provided to the central processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the central processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0115] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0116] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors.

Claims

1. An artificial intelligence chip, characterized in that, include: Execution unit, the execution unit being configured with a vector execution pipeline and a scalar execution pipeline, the scalar execution pipeline including at least: The scalar instruction decoding unit is configured to acquire at least the operand type, address information, and scalar operation control information of the scalar instruction. The scalar instruction operand acquisition unit is configured to acquire the operand source for scalar instructions; The scalar instruction arithmetic unit is configured to perform scalar computations based at least on the operand type, operand source, and scalar operation control information of the scalar instruction, and to write the computation results to the scalar register set included in the execution unit; and The merge instruction decoding unit is configured to, in response to determining that the pass enable signal for data transfer between the scalar execution pipeline and the vector execution pipeline has not been specified as "valid" by instruction, and that the scalar register address associated with the operand associated with the scalar part information is the same as the scalar register address associated with the computation result associated with the vector part information, set the pass enable signal of the operand to "valid", and pass the pass enable signal of the operand set to "valid" to the scalar instruction arithmetic unit.

2. The artificial intelligence chip according to claim 1, characterized in that, The operands of scalar instructions are obtained from the scalar register group included in the execution unit; the vector execution pipeline includes at least: a vector instruction decoding unit, a vector instruction operand acquisition unit, and a vector instruction arithmetic unit.

3. The artificial intelligence chip according to claim 1, characterized in that, Also includes: The instruction scheduling unit is configured to distinguish the instruction type of the received instructions, including vector instructions, scalar instructions, and merge instructions.

4. The artificial intelligence chip according to claim 3, characterized in that, The instruction scheduling unit is further configured to: issue vector instructions to the vector instruction decoding unit included in the vector execution pipeline, issue scalar instructions to the scalar instruction decoding unit, and issue merge instructions to the merge instruction decoding unit included in the execution unit.

5. The artificial intelligence chip according to claim 3, characterized in that, Also includes: The merge instruction decoding unit is configured to: decompose and decode the received merge instruction in order to generate scalar part information and vector part information; The scalar portion information is provided to the scalar instruction operand acquisition unit, and the vector portion information is provided to the vector instruction operand acquisition unit included in the vector execution pipeline.

6. The artificial intelligence chip according to claim 5, characterized in that, The merge instruction decoding unit is further configured to: parse and transmit an enable signal based on the received merge instruction, so as to enable the scalar instruction arithmetic unit to at least partially use the calculation result of the vector instruction arithmetic unit as the operand source of the scalar instruction arithmetic unit.

7. The artificial intelligence chip according to claim 5, characterized in that, The merge instruction decoding unit is also configured to specify whether the transmission enable signal is "valid" via an instruction.

8. The artificial intelligence chip according to claim 2, characterized in that, Also includes: The vector-to-scalar conversion unit is configured to convert the computation results of the vector instruction operation unit into scalar data.

9. The artificial intelligence chip according to claim 6, characterized in that, The scalar instruction arithmetic unit is further configured to: in response to determining that the pass enable signal for data transfer between the scalar execution pipeline and the vector execution pipeline is set to "valid", the scalar instruction arithmetic unit obtains scalar data via the vector-to-scalar conversion unit as the operand source; in response to determining that the pass enable signal for data transfer between the scalar execution pipeline and the vector execution pipeline is not set to "valid", the scalar instruction arithmetic unit obtains the operand source of the scalar instruction via the scalar instruction operand acquisition unit.

10. The artificial intelligence chip according to claim 3, characterized in that, The instruction scheduling unit includes: The instruction classification unit for each thread bundle is configured to classify instructions independently for the corresponding thread bundle; and The polling arbitrator includes a vector polling arbitrator, a scalar instruction polling arbitrator, and a merge instruction polling arbitrator. The polling arbitrator is configured to select vector instructions, scalar instructions, or merge instructions of a thread bundle based on a polling mechanism so as to issue them to the vector instruction decoding unit, scalar instruction decoding unit, or merge instruction decoding unit respectively.

11. The artificial intelligence chip according to claim 10, characterized in that, The merge instruction polling arbiter is configured to prevent the vector polling arbiter and the scalar instruction polling arbiter from selecting and firing vector instructions and scalar instructions of the thread bundle in response to determining that a merge instruction has been detected.

12. A method for pipelined parallelism of vector and scalar execution, characterized in that, The method operates on an artificial intelligence chip according to any one of claims 1 to 11, the method comprising: Distinguish the instruction type of the received instructions, wherein the instruction type includes at least: vector instructions, scalar instructions; and The vector execution pipeline that issues vector instructions to the AI ​​chip, and the scalar execution pipeline that issues scalar instructions to the AI ​​chip.

13. The method according to claim 12, characterized in that, The instruction type further includes: a merge instruction, and the method further includes: The received merging instruction is decomposed and decoded to generate scalar and vector part information; and The scalar portion information is provided to the scalar instruction operand fetching unit included in the scalar execution pipeline, and the vector portion information is provided to the vector instruction operand fetching unit included in the vector execution pipeline.

14. The method according to claim 13, characterized in that, The method further includes specifying whether the enable signal is "valid" via an instruction: In response to determining that the pass enable signal has not been specified as "valid" by the instruction, and that the scalar register address associated with the operand associated with the scalar part information is the same as the scalar register address associated with the calculation result associated with the vector part information, the pass enable signal of the operand is set to "valid", and the pass enable signal of the operand set to "valid" is passed to the scalar instruction arithmetic unit.

15. The method according to claim 13, characterized in that, Also includes: In response to the pass enable signal for determining data transfer between the scalar execution pipeline and the vector execution pipeline being set to "valid", the scalar instruction arithmetic unit obtains scalar data via the vector-to-scalar conversion unit as the operand source; as well as In response to the fact that the operand pass enable signal is not set to "valid", the scalar instruction arithmetic unit obtains scalar data via the scalar instruction operand acquisition unit as the operand source.

16. A computing device, characterized in that, include: At least one processor; as well as A memory that is communicatively connected to the at least one processor; in The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 12-15.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a machine, performs the method according to any one of claims 12-15.

18. A computer program product, characterized in that, Includes a computer program, which, when executed by a machine, performs the method according to any one of claims 12-15.

Images