Tensor operation accelerator and method and storage medium

Abstract

The invention relates to the technical field of computers, in particular to a tensor operation accelerator, a tensor operation method and a storage medium. When a data stream processing module receives a to-be-processed data stream sent by a data direct management unit, the to-be-processed data stream is sent to a tensor operation module after being subjected to data mapping operation; after periodically receiving a to-be-executed instruction, the scheduling module determines a scheduling instruction for the tensor operation module, and sends the scheduling instruction for instructing the tensor operation module to perform tensor operation and the to-be-executed instruction to the tensor operation module, so that the tensor operation module performs tensor operation according to the scheduling instruction and the to-be-executed instruction; according to the method, the tensor operation is performed on the received to-be-processed data stream to obtain a tensor operation result, the tensor operation result is sent to the data stream processing module, and finally, the data stream processing module performs activation processing on the tensor operation result to obtain a target tensor operation result, so that the problem of resource limitation is solved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a tensor operation accelerator, method and storage medium. Background Art

[0002] With the rapid development of artificial intelligence technology, deep learning has been widely used in many fields in recent years, such as image recognition, natural language processing, and intelligent recommendation. At present, the implementation of deep learning models requires high-performance servers and cloud computing resources to perform a large number of tensor operations. The use of the above traditional centralized computing methods has problems such as data transmission delays and bandwidth limitations.

[0003] Based on this, in order to overcome the above problems, edge computing came into being in the existing technology, that is, by performing artificial intelligence reasoning and analysis on local devices close to the data source, the delay and bandwidth limitation of data transmission are reduced. Specifically, dedicated integrated circuits and graphics processors are configured on local devices to perform tensor operations, so as to realize artificial intelligence reasoning and analysis.

[0004] However, with existing technologies, due to the lack of flexibility of dedicated integrated circuits and the high price and energy consumption of graphics processors, there is a problem of resource limitation when using edge computing for tensor operations. Summary of the invention

[0005] Based on this, it is necessary to provide a tensor operation accelerator, method and storage medium for the above technical problems. When the data flow processing module receives the data flow to be processed sent by the data direct management unit, the data flow processing module performs data mapping operation on the data flow to be processed and sends it to the tensor operation module. After periodically receiving the instructions to be executed, the scheduling module determines the scheduling instructions for the tensor operation module, and sends the scheduling instructions instructing the tensor operation module to perform tensor operation operations and the instructions to be executed to the tensor operation module, so that the tensor operation module performs tensor operations on the received data flow to be processed according to the scheduling instructions and the instructions to be executed, obtains the tensor operation results, and sends them to the data flow processing module. Finally, the data flow processing module activates the tensor operation results to obtain the target tensor operation results. In this way, when realizing artificial intelligence reasoning and analysis, there is no need to perform tensor operations through dedicated integrated circuits and graphics processors, thereby solving the problem of limited resources.

[0006] In a first aspect, an embodiment of the present invention provides a tensor operation accelerator, comprising: a data flow processing module, a tensor operation module, and a scheduling module for instructions to be executed, which are connected in sequence;

[0007] The data stream processing module is used to perform a data mapping operation on the data stream to be processed when receiving the data stream to be processed sent by the data direct management unit, and send the data stream to be processed after the operation to the tensor operation module;

[0008] The scheduling module is used to determine the scheduling instruction for the tensor operation module after periodically receiving the to-be-executed instruction, and send the scheduling instruction and the to-be-executed instruction to the tensor operation module, wherein the scheduling instruction is used to instruct the tensor operation module to perform a tensor operation;

[0009] The tensor operation module is used to perform tensor operation on the received data stream to be processed when receiving the scheduling instruction and the instruction to be executed, obtain the tensor operation result, and send it to the data stream processing module;

[0010] The data flow processing module is also used to activate the tensor operation result after receiving the tensor operation result sent by the tensor operation module to obtain the target tensor operation result.

[0011] In one embodiment, the data stream processing module includes: a data stream input unit;

[0012] The data stream input unit is used to receive the data stream to be processed sent by the data direct management unit;

[0013] For the data stream to be processed, a linear interpolation conversion is performed on the data stream to be processed through a preset mapping function to obtain the data stream to be processed after the data mapping operation is performed.

[0014] In one embodiment, the data stream processing module further includes: a data stream output unit;

[0015] The data stream output unit is used to receive the tensor operation result sent by the tensor operation module;

[0016] For the tensor operation result, determining the activation function corresponding to the currently running artificial intelligence model;

[0017] The tensor operation result is activated by the activation function to obtain a target tensor operation result.

[0018] In one embodiment, the tensor operation module includes: a plurality of tensor operation submodules;

[0019] The tensor operation module is specifically used to determine at least one target tensor operation submodule from multiple tensor operation submodules according to the scheduling instruction. When there are multiple target tensor operation submodules, the multiple target tensor operation submodules are run in parallel to perform tensor operations according to the instructions to be executed.

[0020] In one embodiment, the target tensor operator module includes: an instruction decoding unit, a data storage unit, a computing unit and a scheduling unit, wherein the scheduling unit is connected to the instruction decoding unit, the data storage unit and the computing unit respectively, and the computing unit is connected to the data storage unit;

[0021] The instruction decoding unit is used to decode the instruction to be executed to obtain at least two instruction fields and send them to the scheduling unit;

[0022] The data storage unit is used to store the received data stream to be processed;

[0023] The scheduling unit is used to configure the computing unit according to at least two instruction fields, and schedule the computing unit to obtain the to-be-processed data stream from the data storage unit;

[0024] The computing unit is used to perform tensor operations on the data stream to be processed to obtain tensor operation results.

[0025] In one embodiment, the target tensor operator module further includes: a parameter storage unit, the instruction decoding unit being connected to the parameter storage unit;

[0026] The instruction decoding unit is further used to send at least two instruction fields to the parameter storage unit after obtaining them;

[0027] The parameter storage unit is used to store the at least two received instruction fields.

[0028] In one embodiment, the computing unit includes: a linear computing subunit and a scalar computing subunit; the parameter storage unit includes: a linear parameter storage subunit corresponding to the linear computing subunit, and a scalar parameter storage subunit corresponding to the scalar computing subunit.

[0029] In one embodiment, the instructions to be executed are multiple instruction fields of preset bit length, and the multiple instruction fields at least include: a first instruction field for indicating linear calculation, a second instruction field for indicating scalar calculation, and a control instruction field;

[0030] The first instruction field at least includes: an operation code for linear calculation, an indication operation code for a calculation result obtained by linear calculation, and an indication operation code for the to-be-processed data stream input by linear calculation;

[0031] The second instruction field at least includes: an operation code for scalar calculation, an indication operation code for a scalar result obtained by the scalar calculation, and an indication operation code for the data stream to be processed input by the scalar calculation;

[0032] The control instruction field indication includes: an indication operation code for the target tensor operation result.

[0033] In a second aspect, an embodiment of the present invention provides a tensor operation acceleration method, including:

[0034] When receiving the data stream to be processed sent by the data direct management unit, performing a data mapping operation on the data stream to be processed, and sending the operated data stream to be processed to the tensor operation module;

[0035] After periodically receiving the to-be-executed instruction, determining a scheduling instruction for the tensor operation module, and sending the scheduling instruction and the to-be-executed instruction to the tensor operation module, wherein the scheduling instruction is used to instruct the tensor operation module to perform a tensor operation;

[0036] When receiving the scheduling instruction and the to-be-executed instruction, performing tensor operation on the received data stream to be processed, obtaining the tensor operation result, and sending it to the data stream processing module;

[0037] After receiving the tensor operation result sent by the tensor operation module, the tensor operation result is activated to obtain the target tensor operation result.

[0038] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of a tensor operation acceleration method provided by any embodiment of the present invention are implemented.

[0039] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0040] In the disclosed embodiment, when the data flow processing module receives the data flow to be processed sent by the data direct management unit, the data flow processing module performs data mapping operation on the data flow to be processed and then sends it to the tensor operation module. After periodically receiving the instructions to be executed, the scheduling module determines the scheduling instructions for the tensor operation module, and sends the scheduling instructions instructing the tensor operation module to perform tensor operation operations and the instructions to be executed to the tensor operation module, so that the tensor operation module performs tensor operations on the received data flow to be processed according to the scheduling instructions and the instructions to be executed, obtains tensor operation results, and sends them to the data flow processing module. Finally, the data flow processing module activates the tensor operation results to obtain the target tensor operation results. In this way, when realizing artificial intelligence reasoning and analysis, there is no need to perform tensor operations through dedicated integrated circuits and graphics processors, thereby solving the problem of limited resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a schematic diagram of the structure of a tensor operation accelerator in one embodiment;

[0042] Figure 2 A schematic diagram of the structure of a tensor operation accelerator in another embodiment;

[0043] Figure 3 A schematic diagram of the structure of a tensor operation accelerator in yet another embodiment;

[0044] Figure 4 is a structural schematic diagram of a tensor operation accelerator in yet another embodiment;

[0045] Figure 5 The figure is a flow chart of a tensor operation acceleration method in one embodiment. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0047] In the existing technology, in order to overcome the problems of data transmission delay and bandwidth limitation when deep learning is implemented through traditional centralized computing methods, edge computing is used to configure dedicated integrated circuits and graphics processors on local devices close to the data source to perform tensor operations, so as to perform artificial intelligence reasoning and analysis, thereby reducing data transmission delays and bandwidth limitations.

[0048] However, with existing technologies, due to the lack of flexibility of dedicated integrated circuits and the high price and energy consumption of graphics processors, there is a problem of resource limitation when using edge computing for tensor operations.

[0049] Based on the above problems, the present disclosure provides a tensor operation accelerator. When the data flow processing module receives the data flow to be processed sent by the data direct management unit, the data flow processing module performs data mapping operation on the data flow to be processed and then sends it to the tensor operation module. After periodically receiving the instructions to be executed, the scheduling module determines the scheduling instructions for the tensor operation module, and sends the scheduling instructions instructing the tensor operation module to perform tensor operation operations and the instructions to be executed to the tensor operation module, so that the tensor operation module performs tensor operations on the received data flow to be processed according to the scheduling instructions and the instructions to be executed, obtains the tensor operation results, and sends them to the data flow processing module. Finally, the data flow processing module activates the tensor operation results to obtain the target tensor operation results. In this way, when realizing artificial intelligence reasoning and analysis, there is no need to perform tensor operations through dedicated integrated circuits and graphics processors, thereby solving the problem of limited resources.

[0050] Figure 1 FIG. 1 is a schematic diagram of a structure of a tensor operation accelerator in an embodiment. Figure 1 As shown, the tensor operation accelerator 100 includes: a data flow processing module 10, a tensor operation module 20 and a scheduling module 30 for instructions to be executed, which are connected in sequence;

[0051] The data stream processing module 10 is used to perform data mapping operations on the data stream to be processed when receiving the data stream to be processed sent by the data direct management unit, and send the data stream to be processed after the operation to the tensor operation module 20. The scheduling module 30 is used to determine the scheduling instructions for the tensor operation module 20 after periodically receiving the instructions to be executed, and send the scheduling instructions and the instructions to be executed to the tensor operation module 20. The tensor operation module 20 is used to perform tensor operations on the received data stream to be processed when receiving the scheduling instructions and the instructions to be executed, obtain the tensor operation results, and send them to the data stream processing module 10. The data stream processing module 10 is also used to activate the tensor operation results after receiving the tensor operation results sent by the tensor operation module 20 to obtain the target tensor operation results.

[0052] The data stream processing module 10 is a stream processor, which processes data such as a data stream to be processed, so as to ensure that data can be continuously transmitted during the data transmission process, thereby reducing data transmission delay.

[0053] Direct data management means that data can be directly obtained from local devices, such as smart phones, security cameras, etc., but is not limited to these. The present disclosure does not specifically limit this, and technicians in this field can set it according to actual conditions.

[0054] The above-mentioned data stream to be processed refers to the original data input when performing artificial intelligence reasoning and analysis. For example, when a neural network model is applied on an edge device such as a router, the above-mentioned data stream to be processed is the original data to be detected input into the neural network model, but is not limited to this. The present disclosure does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0055] The above-mentioned data mapping operation refers to a linear conversion operation performed on the data stream to be processed received from the data direct management unit, so that the data stream to be processed can be adjusted to better adapt to subsequent calculation operations. Continuing from the above-mentioned embodiment, for the data stream to be processed, that is, the original data to be detected input into the neural network model, a data mapping operation is performed on the original data to be detected, that is, a linear conversion is performed to change the data distribution range of the original data to be detected, so that the original data to be detected can be adjusted to adapt to subsequent calculation operations.

[0056] The above-mentioned instructions to be executed refer to the instructions issued to indicate various calculation operations when running artificial intelligence, such as when applying a neural network model. The instructions to be executed can be implemented through the Very Long Instruction Word (VLIW) technology. Specifically, the very long instruction word processor uses multiple independent functional components to package multiple instructions into a very long task instruction, and can periodically send multiple task instructions to be executed at the same time, and perform multiple task operations at the same time, thereby improving computing throughput and energy efficiency.

[0057] Optionally, based on the above embodiments, in some disclosed embodiments, the instruction to be executed is a plurality of instruction fields of preset bit length, and the plurality of instruction fields include at least: a first instruction field for indicating linear calculation, a second instruction field for indicating scalar calculation, and a control instruction field.

[0058] Among them, the above-mentioned preset bit length is the bit length set by the pointer for the instruction to be executed. For example, the preset bit length can be 128 bits, but it is not limited to this. The present disclosure does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0059] The first instruction field includes at least: an operation code for linear calculation, an indication operation code for a calculation result obtained by the linear calculation, and an indication operation code for a data stream to be processed input by the linear calculation.

[0060] Among them, linear calculation refers to the linear arithmetic operation performed on the input value during the tensor operation, such as multiplication, addition, subtraction, multiply-accumulate, multiply-accumulate-subtract, assignment, accumulator shift, and 12-bit integer shift. The opcode for linear calculation refers to the opcode that indicates the execution of linear calculation. Exemplarily, the opcode is set to 11111 in the instruction field of the to-be-executed instruction such as the 5-bit very long instruction word, where 11111 indicates the execution of a multiplication operation on the input value, but is not limited to this, and the present disclosure does not specifically limit it, and those skilled in the art can set it according to actual conditions.

[0061] The indication operation code for the calculation result obtained by the linear calculation refers to an operation code indicating an operation to be performed on the linear calculation result. Exemplarily, the operation code is set to 1 or 0 in the instruction field of the to-be-executed instruction such as the very long instruction word, indicating whether to save the calculation result to the accumulator register file, but is not limited thereto. The present disclosure does not specifically limit this, and those skilled in the art can set it according to actual conditions.

[0062] The indication operation code for the data stream to be processed inputted for linear calculation refers to the operation code indicating the operation to be performed on the data stream to be processed when the data stream to be processed is received. Exemplarily, the operation code is set to 1 or 0 in the instruction field of 1 bit of the instruction to be executed, such as the very long instruction word, indicating whether the data stream to be processed is a scalar or a vector. Optionally, the operation code is set to 111111000000 in the instruction field of 12 bits of the instruction to be executed, such as the very long instruction word, indicating the storage location of the data stream to be processed in the memory. Optionally, the operation code can also be set to 1101 in the instruction field of 4 bits of the instruction to be executed, such as the very long instruction word, indicating the memory access mode of the data stream to be processed in the memory, but it is not limited to this. The present disclosure does not specifically limit it, and those skilled in the art can set it according to actual conditions.

[0063] It should be noted that the memory access mode can be accessed through pointers, shared memory, private memory, pointers and indexes, etc., but is not limited to this. The present disclosure does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0064] The second instruction field includes at least: an operation code for scalar calculation, an indication operation code for a scalar result obtained by the scalar calculation, and an indication operation code for a data stream to be processed input by the scalar calculation.

[0065] Among them, scalar calculation refers to arithmetic applied to loop counts, array indexes, memory reference pointers and address calculations. The opcode for scalar calculation refers to an opcode that indicates the execution of scalar calculations. Exemplarily, the opcode is set to 11001 in the instruction field of the to-be-executed instruction such as a very long instruction word with 5 bits, wherein 11001 indicates an arithmetic operation of an array index on the input value, and the indication opcode for the scalar result obtained by scalar calculation refers to an opcode that indicates an operation on the scalar calculation result. The indication opcode for the to-be-processed data stream input for linear calculation refers to an opcode that indicates an operation on the to-be-processed data stream when the to-be-processed data stream is received, but is not limited to this, and the present disclosure does not specifically limit it, and those skilled in the art can set it according to actual conditions.

[0066] The control instruction field indication includes: an indication operation code for the target tensor operation result. Exemplarily, the operation code is set to 00111 in the instruction field of the to-be-executed instruction such as the 5-bit super-long instruction word, indicating that a branch operation such as a multiplication operation is performed on the target tensor operation result. Alternatively, the operation code is set to 11111100000 in the instruction field of the to-be-executed instruction such as the 11-bit super-long instruction word, indicating the code address when jumping to the target tensor operation result, but it is not limited to this, and the present disclosure does not specifically limit it, and those skilled in the art can set it according to actual conditions.

[0067] It should be noted that the first instruction field, the second instruction field and the control instruction field are defined at different bit positions of the to-be-executed instruction of the preset bit length.

[0068] The above-mentioned scheduling instructions refer to instructions used to instruct the tensor operation module 20 how to perform tensor operation operations.

[0069] Optionally, based on the above embodiments, in some disclosed embodiments, such as Figure 2 As shown, the tensor operation module 20 includes: multiple tensor operation sub-modules 201, and the tensor operation module 20 is specifically used to determine at least one target tensor operation sub-module among multiple tensor operation sub-modules according to the scheduling instruction. When there are multiple target tensor operation sub-modules, the multiple target tensor operation sub-modules are run in parallel to perform tensor operations according to the instructions to be executed.

[0070] Among them, the tensor operation submodule refers to the core Pcore in the processor, and tensor operations are performed through Pcore. Pcore is a multi-threaded processor that can perform multi-threaded operations simultaneously to improve computing efficiency.

[0071] Specifically, when the tensor operation module 20 obtains a scheduling instruction, the target tensor operation sub-module that needs to be run currently is determined among multiple tensor operation sub-modules according to the scheduling instruction. If there are multiple target tensor operation sub-modules, multiple target tensor operation sub-modules are scheduled to run at the same time, and multiple target tensor operation sub-modules perform tensor operations according to the instructions to be executed.

[0072] The above-mentioned target tensor operation result refers to the final output result of the currently running artificial intelligence operation. Exemplarily, continuing the above-mentioned embodiment, such as the detection result obtained by applying a neural network model for detection, but not limited to this, the present disclosure does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0073] Specifically, the data flow processing module 10 is connected to the tensor operation module 20. After the data flow processing module 10 performs data mapping operation on the data flow to be processed received from the data direct management unit, the data flow to be processed after the operation is sent to the tensor operation module 20. The tensor operation module 20 is connected to the scheduling module 30. The scheduling module 30 periodically obtains the instructions to be executed, and determines the scheduling instructions for the tensor operation module 20, and sends the scheduling instructions and the instructions to be executed to the tensor operation module 20, so that when the tensor operation module 20 receives the scheduling instructions and the instructions to be executed, it performs tensor operation on the data flow to be processed to obtain the tensor operation result, and sends it to the data flow processing module 10. Finally, after receiving the tensor operation result, the data flow processing module 10 activates the tensor operation result to obtain the target tensor operation result.

[0074] In this way, the tensor operation accelerator provided by the present embodiment, when receiving the data stream to be processed sent by the data direct management unit, performs data mapping operation on the data stream to be processed and sends it to the tensor operation module. After periodically receiving the instructions to be executed, the scheduling module determines the scheduling instructions for the tensor operation module, and sends the scheduling instructions instructing the tensor operation module to perform tensor operation operations and the instructions to be executed to the tensor operation module, so that the tensor operation module performs tensor operations on the received data stream to be processed according to the scheduling instructions and the instructions to be executed, obtains the tensor operation results, and sends them to the data stream processing module. Finally, the data stream processing module activates the tensor operation results to obtain the target tensor operation results. In this way, when realizing artificial intelligence reasoning and analysis, there is no need to perform tensor operations through dedicated integrated circuits and graphics processors, thereby solving the problem of limited resources.

[0075] Figure 3This is a structural diagram of a tensor operation accelerator in another embodiment. Based on the above embodiment, in one embodiment of the present disclosure, the data stream processing module 10 includes: a data stream input unit 101, and the data stream input unit 101 is used to receive the data stream to be processed sent by the data direct management unit. For the data stream to be processed, a linear interpolation conversion is performed on the data stream to be processed through a preset mapping function to obtain the data stream to be processed after the data mapping operation.

[0076] The preset mapping function refers to a function used to perform linear interpolation conversion on the data stream to be processed, and the preset mapping function can be defined according to the following expression:

[0077] y=lookup_quotient[[11:5]]+x[4:0]*lookup_remainedr[[4:0]]

[0078] Wherein, lookup_quotient[[11:5]] represents the scaling factor of the data stream input unit, x[4:0] represents bits 0 to 4 of the data stream to be processed, and lookup_remainedr[[4:0]] represents the offset value for offsetting the data stream to be processed. The present disclosure does not specifically limit the preset mapping function, and those skilled in the art can set it according to actual conditions.

[0079] Specifically, the data stream input unit 101 performs linear interpolation conversion on the data stream to be processed received from the data direct management unit through a preset mapping function, so as to obtain the data stream to be processed after the data mapping operation is performed.

[0080] Exemplarily, following the above embodiment, when the neural network model is applied on an edge device such as a router, the original data to be detected input into the neural network model is linearly interpolated and converted through a preset mapping function, so that the original data to be detected is mapped from the original data range such as -600 to 600 to -255 to 255, so as to be more suitable for subsequent processing, reduce the amount of calculation, and improve calculation efficiency, but it is not limited to this, and the present disclosure is not specifically limited, and technicians in this field can set it according to actual conditions.

[0081] Thus, in this embodiment, the data stream input unit 101 performs linear interpolation conversion on the data stream to be processed through a preset mapping function, thereby realizing data mapping operation, thereby reducing the amount of calculation and improving calculation efficiency.

[0082] Based on the above embodiments, in one embodiment of the present disclosure, continue to refer to Figure 3The data stream processing module 10 also includes: a data stream output unit 102, which is used to receive the tensor operation result sent by the tensor operation module, determine the activation function corresponding to the currently running artificial intelligence model for the tensor operation result, and activate the tensor operation result through the activation function to obtain the target tensor operation result.

[0083] Among them, the activation function is determined according to the artificial intelligence model currently running on the local device such as the edge device. Exemplarily, following the above embodiment, when the neural network model is applied on an edge device such as a router, the activation function of the neural network model is the Sigmoid function, and the Sigmoid function activates the tensor operation result, but is not limited to this. The present disclosure does not specifically limit it, and those skilled in the art can set it according to actual conditions.

[0084] Specifically, the data stream output unit 102 activates the received tensor operation result through the activation function corresponding to the currently running artificial intelligence model to obtain the target tensor operation result.

[0085] Figure 4 This is a structural diagram of a tensor operation accelerator in another embodiment. On the basis of the above embodiment, in one embodiment of the present disclosure, the target tensor operation submodule 400 includes: an instruction decoding unit 401, a data storage unit 402, a computing unit 403 and a scheduling unit 404. The scheduling unit 404 is connected to the instruction decoding unit 401, the data storage unit 402 and the computing unit 403 respectively, and the computing unit 403 is connected to the data storage unit 402. The instruction decoding unit 401 is used to decode the instruction to be executed, obtain at least two instruction fields, and send them to the scheduling unit 404. The data storage unit 402 is used to store the received data stream to be processed. The scheduling unit 404 is used to configure the computing unit 403 according to the at least two instruction fields, and schedule the computing unit 403 to obtain the data stream to be processed from the data storage unit 402. The computing unit 403 is used to perform tensor operations on the data stream to be processed to obtain tensor operation results.

[0086] Among them, the instruction field refers to the operating parameters for executing tensor operations. Exemplarily, continuing the above embodiment, the instruction field can be, for example, an instruction field set in the instruction to be executed, such as a 5-bit instruction field of a very long instruction word, indicating the operating parameter 11111 for performing a multiplication operation on the input value, but not limited to this. The present disclosure does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0087] Specifically, the instruction decoding unit 401 decodes the instruction to be executed to obtain at least two instruction fields, and sends the at least two instruction fields to the scheduling unit 404. The scheduling unit 404 configures the computing unit 403 according to the received at least two instruction fields. After the computing unit 403 is configured, the scheduling computing unit 403 obtains the corresponding data stream to be processed from the data storage unit 402 storing the data stream to be processed, and performs tensor operations to obtain tensor operation results.

[0088] Optionally, based on the above embodiments, in some embodiments of the present disclosure, continue to refer to Figure 4 The target tensor operation submodule further includes: a parameter storage unit 405, and the instruction decoding unit 401 is connected to the parameter storage unit 405. The parameter storage unit 405 is used to store at least two received instruction fields.

[0089] Specifically, when the instruction decoding unit 401 decodes the instruction to be executed and obtains at least two instruction fields, it sends them to the parameter storage unit 405. The parameter storage unit 405 receives and stores the at least two instruction fields.

[0090] Optionally, based on the above embodiments, in some embodiments of the present disclosure, the computing unit 403 includes: a linear computing subunit and a scalar computing subunit; the parameter storage unit includes: a linear parameter storage subunit corresponding to the linear computing subunit, and a scalar parameter storage subunit corresponding to the scalar computing subunit.

[0091] Among them, the linear calculation subunit refers to the one used for linear calculation in the tensor calculation process, and the scalar calculation subunit is used for scalar calculation in the tensor calculation process. The linear parameter storage subunit refers to the one used for storing multiple instruction fields for configuring the linear calculation subunit, and the scalar parameter storage subunit refers to the one used for storing multiple instruction fields for configuring the linear scalar calculation subunit.

[0092] In one embodiment, Figure 5 As shown, a tensor operation acceleration method is provided, which is applied to the tensor operation accelerator described in each of the above embodiments, including:

[0093] S501, when receiving the data stream to be processed sent by the data direct management unit, performing a data mapping operation on the data stream to be processed, and sending the operated data stream to be processed to the tensor operation module.

[0094] S502, after periodically receiving the instructions to be executed, determine the scheduling instructions for the tensor operation module, and send the scheduling instructions and the instructions to be executed to the tensor operation module, wherein the scheduling instructions are used to instruct the tensor operation module to perform tensor operation operations.

[0095] S503, when a scheduling instruction and an instruction to be executed are received, tensor operations are performed on the received data stream to be processed, tensor operation results are obtained, and sent to the data stream processing module.

[0096] S504, after receiving the tensor operation result sent by the tensor operation module, activation processing is performed on the tensor operation result to obtain a target tensor operation result.

[0097] Specifically, the data flow processing module 10 is connected to the tensor operation module 20. After the data flow processing module 10 performs data mapping operation on the data flow to be processed received from the data direct management unit, the data flow to be processed after the operation is sent to the tensor operation module 20. The tensor operation module 20 is connected to the scheduling module 30. The scheduling module 30 periodically obtains the instructions to be executed, and determines the scheduling instructions for the tensor operation module 20, and sends the scheduling instructions and the instructions to be executed to the tensor operation module 20, so that when the tensor operation module 20 receives the scheduling instructions and the instructions to be executed, it performs tensor operation on the data flow to be processed to obtain the tensor operation result, and sends it to the data flow processing module 10. Finally, after receiving the tensor operation result, the data flow processing module 10 activates the tensor operation result to obtain the target tensor operation result.

[0098] In this way, in this implementation, when the data flow processing module receives the data flow to be processed sent by the data direct management unit, it performs data mapping operation on the data flow to be processed and then sends it to the tensor operation module. After periodically receiving the instructions to be executed, the scheduling module determines the scheduling instructions for the tensor operation module, and sends the scheduling instructions instructing the tensor operation module to perform tensor operation operations and the instructions to be executed to the tensor operation module, so that the tensor operation module performs tensor operations on the received data flow to be processed according to the scheduling instructions and the instructions to be executed, obtains the tensor operation results, and sends them to the data flow processing module. Finally, the data flow processing module activates the tensor operation results to obtain the target tensor operation results. In this way, when realizing artificial intelligence reasoning and analysis, there is no need to perform tensor operations through dedicated integrated circuits and graphics processors, thereby solving the problem of limited resources.

[0099] It should be understood that although Figure 5 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 5At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0100] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: when receiving a data stream to be processed sent by a data direct management unit, performing a data mapping operation on the data stream to be processed, and sending the data stream to be processed after the operation to the tensor operation module; after periodically receiving instructions to be executed, determining a scheduling instruction for the tensor operation module, and sending the scheduling instruction and the instructions to be executed to the tensor operation module, wherein the scheduling instruction is used to instruct the tensor operation module to perform a tensor operation; when receiving the scheduling instruction and the instructions to be executed, performing a tensor operation on the received data stream to be processed, obtaining a tensor operation result, and sending it to the data stream processing module; after receiving the tensor operation result sent by the tensor operation module, performing activation processing on the tensor operation result to obtain a target tensor operation result.

[0101] In this way, in this embodiment, when the data flow processing module receives the data flow to be processed sent by the data direct management unit, it performs data mapping operation on the data flow to be processed and then sends it to the tensor operation module. After periodically receiving the instructions to be executed, the scheduling module determines the scheduling instructions for the tensor operation module, and sends the scheduling instructions instructing the tensor operation module to perform tensor operation operations and the instructions to be executed to the tensor operation module, so that the tensor operation module performs tensor operations on the received data flow to be processed according to the scheduling instructions and the instructions to be executed, obtains the tensor operation results, and sends them to the data flow processing module. Finally, the data flow processing module activates the tensor operation results to obtain the target tensor operation results. In this way, when realizing artificial intelligence reasoning and analysis, there is no need to perform tensor operations through dedicated integrated circuits and graphics processors, thereby solving the problem of limited resources.

[0102] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0103] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A tensor operation accelerator, characterized in that: include: A data flow processing module, a tensor operation module and a scheduling module for instructions to be executed connected in sequence; The data stream processing module is used to perform a data mapping operation on the data stream to be processed when receiving the data stream to be processed sent by the data direct management unit, and send the data stream to be processed after the operation to the tensor operation module; The scheduling module is used to determine the scheduling instruction for the tensor operation module after periodically receiving the to-be-executed instruction, and send the scheduling instruction and the to-be-executed instruction to the tensor operation module, wherein the scheduling instruction is used to instruct the tensor operation module to perform a tensor operation; The tensor operation module is used to perform tensor operation on the received data stream to be processed when receiving the scheduling instruction and the instruction to be executed, obtain the tensor operation result, and send it to the data stream processing module; The data flow processing module is also used to activate the tensor operation result after receiving the tensor operation result sent by the tensor operation module to obtain the target tensor operation result.

2. The accelerator according to claim 1, characterized in that The data stream processing module comprises: a data stream input unit; The data stream input unit is used to receive the data stream to be processed sent by the data direct management unit; For the data stream to be processed, a linear interpolation conversion is performed on the data stream to be processed through a preset mapping function to obtain the data stream to be processed after the data mapping operation is performed.

3. The accelerator according to claim 2, characterized in that The data stream processing module further includes: a data stream output unit; The data stream output unit is used to receive the tensor operation result sent by the tensor operation module; For the tensor operation result, determining the activation function corresponding to the currently running artificial intelligence model; The tensor operation result is activated by the activation function to obtain a target tensor operation result.

4. The accelerator according to claim 1, characterized in that The tensor operation module includes: a plurality of tensor operation submodules; The tensor operation module is specifically used to determine at least one target tensor operation submodule from multiple tensor operation submodules according to the scheduling instruction. When there are multiple target tensor operation submodules, the multiple target tensor operation submodules are run in parallel to perform tensor operations according to the instructions to be executed.

5. The accelerator according to claim 4, characterized in that The target tensor operator module includes: an instruction decoding unit, a data storage unit, a computing unit and a scheduling unit, wherein the scheduling unit is connected to the instruction decoding unit, the data storage unit and the computing unit respectively, and the computing unit is connected to the data storage unit; The instruction decoding unit is used to decode the instruction to be executed to obtain at least two instruction fields and send them to the scheduling unit; The data storage unit is used to store the received data stream to be processed; The scheduling unit is used to configure the computing unit according to at least two instruction fields, and schedule the computing unit to obtain the to-be-processed data stream from the data storage unit; The computing unit is used to perform tensor operations on the data stream to be processed to obtain tensor operation results.

6. The accelerator according to claim 5, characterized in that The target tensor operator module further includes: a parameter storage unit, and the instruction decoding unit is connected to the parameter storage unit; The instruction decoding unit is further used to send at least two instruction fields to the parameter storage unit after obtaining them; The parameter storage unit is used to store the at least two received instruction fields.

7. The accelerator according to claim 6, characterized in that The calculation unit includes: a linear calculation subunit and a scalar calculation subunit; the parameter storage unit includes: a linear parameter storage subunit corresponding to the linear calculation subunit, and a scalar parameter storage subunit corresponding to the scalar calculation subunit.

8. The accelerator according to claim 5, characterized in that The instructions to be executed are a plurality of instruction fields of preset bit length, and the plurality of instruction fields at least include: a first instruction field for indicating linear calculation, a second instruction field for indicating scalar calculation, and a control instruction field; The first instruction field at least includes: an operation code for linear calculation, an indication operation code for a calculation result obtained by linear calculation, and an indication operation code for the to-be-processed data stream input by linear calculation; The second instruction field at least includes: an operation code for scalar calculation, an indication operation code for a scalar result obtained by the scalar calculation, and an indication operation code for the data stream to be processed input by the scalar calculation; The control instruction field indication includes: an indication operation code for the target tensor operation result.

9. A tensor operation acceleration method, characterized in that: include: When receiving the data stream to be processed sent by the data direct management unit, performing a data mapping operation on the data stream to be processed, and sending the operated data stream to be processed to the tensor operation module; After periodically receiving the to-be-executed instruction, determining a scheduling instruction for the tensor operation module, and sending the scheduling instruction and the to-be-executed instruction to the tensor operation module, wherein the scheduling instruction is used to instruct the tensor operation module to perform a tensor operation; When receiving the scheduling instruction and the to-be-executed instruction, performing tensor operation on the received data stream to be processed, obtaining the tensor operation result, and sending it to the data stream processing module; After receiving the tensor operation result sent by the tensor operation module, activation processing is performed on the tensor operation result to obtain the target tensor operation result.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method described in claim 9 are implemented.

Images