Tensor processing device and method and computer readable storage medium

Abstract

The invention discloses a tensor processing device which comprises an instruction sequence module used for generating a first instruction sequence and / or a second instruction sequence according to a tensor operation instruction and preset configuration information, the first instruction sequence comprises a plurality of first operation instructions, and the second instruction sequence comprises a plurality of second operation instructions; the preset configuration information comprises first configuration information and / or second configuration information; the data loading module is used for loading the data to be operated into the corresponding registers in sequence according to the first instruction sequence or the second instruction sequence; the first calculation module is used for reading data from the register in sequence according to the first instruction sequence for in-storage calculation; and the second calculation module is used for sequentially reading the data from the register according to the second instruction sequence to carry out non-in-memory calculation. The invention further provides a tensor processing method, the unified macro instruction is adopted to call different calculation modules to perform in-memory calculation or non-in-memory calculation, the programming complexity is reduced, and the calculation efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of computing, and more specifically, to a tensor processing device, method and computer-readable storage medium. Background Art

[0002] In the prior art, AI chip solutions for detection and computing tasks are mainly divided into two categories. One is general-purpose processing devices, such as CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), GPU (Graphics Processing Unit), etc.; the other is acceleration processors specifically for artificial neural networks, such as Google's TPU (Tensor Processing Units). These chips all belong to the von Neumann architecture, where computing and storage are separated, and the two work together to complete data access and computing. However, since the design of the processor is mainly to improve computing speed, and storage is more focused on capacity improvement and cost optimization, there is a performance mismatch between "storage" and "computing", which leads to problems such as low memory access bandwidth, extended latency, and high power consumption, which are commonly referred to as the "storage wall" and "power consumption wall". The denser the memory access, the more serious the "wall" problem, and the more difficult it is to improve computing power.

[0003] As a new computing architecture, storage-computing integration fully integrates storage and computing, effectively overcomes the bottleneck of the von Neumann architecture, and achieves an order of magnitude improvement in computing energy efficiency. However, in-memory computing is only suitable for general matrix calculations with fixed weights, and does not support general matrix calculations with dynamically changing weights. Therefore, two different computing engines need to be called to perform general matrix calculations with fixed weights and general matrix calculations with dynamically changing weights respectively. However, two different computing engines will have two different sets of hardware interfaces, and the matrix multiplication computing power of the two computing engines is also different, which increases the complexity of the programming interface. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to provide a tensor processing device, method and computer-readable storage medium, which use a unified macro instruction to call different computing modules to perform in-memory or non-memory computing, thereby reducing programming complexity.

[0005] According to a first aspect of the present invention, a tensor processing device is provided, comprising: an instruction sequence module, used to generate a first instruction sequence and / or a second instruction sequence according to tensor operation instructions and preset configuration information, wherein the first instruction sequence includes multiple first operation instructions, the second instruction sequence includes multiple second operation instructions, and the preset configuration information includes first configuration information and / or second configuration information; a data loading module, used to load the data to be operated into the corresponding register in sequence according to the first instruction sequence or the second instruction sequence; a first computing module, used to read data from the register in sequence according to the first instruction sequence for in-memory computing; and a second computing module, used to read data from the register in sequence according to the second instruction sequence for non-in-memory computing.

[0006] Preferably, the first configuration information is used to characterize the data size, shape, starting address and addressing method of each calculation by the first computing module; the second configuration information is used to characterize the data size, shape, starting address and addressing method of each calculation by the second computing module.

[0007] Preferably, the instruction sequence module is also used to generate a selection signal according to the tensor operation instruction, and send the first instruction sequence to the first computing module or send the second instruction sequence to the second computing module according to the selection signal.

[0008] Preferably, the tensor operation instruction includes an operation code and an operation field, wherein the operation code includes an instruction type and an operation code, and the operation field includes a source address and a target address of the operation data.

[0009] Preferably, the instruction sequence module includes: an instruction parsing unit, used to parse the tensor operation instruction to obtain the operation code and the source address and target address of the operation data; an instruction sequence unit, used to automatically calculate the source address and target address of multiple sub-operation data according to preset configuration information and the source address of the operation data, and generate multiple first operation instructions and / or multiple second operation instructions according to the operation code and the source address and target address of the multiple sub-operation data.

[0010] Preferably, the instruction sequence unit is also used to automatically calculate the addresses of multiple first sub-operation data based on the first configuration information and the source address of the operation data to obtain multiple first operation instructions; and / or automatically calculate the addresses of multiple second sub-operation data based on the second configuration information and the source address of the operation data to obtain multiple second operation instructions.

[0011] Preferably, the instruction sequence module further includes: a control unit, configured to generate a selection signal according to a source address of the operation data, and send the first instruction sequence to the first computing module or send the second instruction sequence to the second computing module according to the selection signal.

[0012] Preferably, the tensor operation instruction also includes preset configuration information.

[0013] Preferably, the instruction sequence module further includes: a configuration information unit, configured to obtain the first configuration information or the second configuration information from the instruction parsing unit or the configuration register according to the selection signal.

[0014] Preferably, the source address of the operation data is any one of an on-chip memory, an off-chip memory, and a register.

[0015] Preferably, the register includes an input register, a weight register and an accumulation register, wherein the input register is used to store activation data; the weight register is used to store weight data, and the accumulation register is used to store an intermediate result of multiplication and addition of activation data and weight data.

[0016] According to a second aspect of the present invention, a tensor processing method is provided, comprising: generating a first instruction sequence and / or a second instruction sequence according to tensor operation instructions and preset configuration information, wherein the first instruction sequence includes multiple first operation instructions, the second instruction sequence includes multiple second operation instructions, and the preset configuration information includes first configuration information and / or second configuration information; generating a selection signal according to the tensor operation instruction, and executing the first instruction sequence or the second instruction sequence according to the selection signal; loading the data to be operated into the corresponding register in sequence according to the first instruction sequence or the second instruction sequence; reading data from the register in sequence according to the first instruction sequence for in-memory calculation or reading data from the register in sequence according to the second instruction sequence for non-in-memory calculation.

[0017] Preferably, the first configuration information is used to characterize the data size, shape, starting address and addressing method of each calculation by the first computing module; the second configuration information is used to characterize the data size, shape, starting address and addressing method of each calculation by the second computing module.

[0018] Preferably, the tensor operation instruction includes an operation code and an operation field, wherein the operation code includes an instruction type and an operation code, and the operation field includes a source address and a target address of the operation data.

[0019] Preferably, generating a first instruction sequence and / or a second instruction sequence according to tensor operation instructions and preset configuration information includes: parsing the tensor operation instructions to obtain the opcode and the source address and target address of the operation data; automatically calculating the source address and target address of multiple sub-operation data according to the preset configuration information and the source address of the operation data, and generating multiple first operation instructions and / or multiple second operation instructions according to the opcode and the source address and target address of the multiple sub-operation data.

[0020] Preferably, the tensor operation instruction also includes preset configuration information.

[0021] Preferably, the tensor processing method further includes: acquiring the first configuration information or the second configuration information according to a selection signal.

[0022] According to a third aspect of the present invention, there is provided a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above-mentioned methods is implemented.

[0023] The tensor processing device, method and computer-readable storage medium provided by the present invention use a unified macro instruction to call different computing modules to perform in-memory computing or non-in-memory computing, thereby reducing programming complexity and improving computing efficiency.

[0024] Furthermore, the tensor processing device, i.e., the hardware itself, can automatically calculate the source address and target address of multiple sub-operation data according to preset configuration information and the source address and target address of the operation data, and realize matrix multiplication operations of larger shapes through multiple matrix multiplication operations. In this way, the hardware information can be hidden and only one macro instruction is needed to realize matrix multiplication operations of larger shapes, which greatly reduces the complexity of programming, reduces the overhead of instruction issuance and decoding, and reduces power consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0026] Figure 1 A schematic diagram showing the structure of a tensor processing device provided by an embodiment of the present invention is shown;

[0027] Figure 2 A schematic diagram showing the structure of an instruction sequence module provided by an embodiment of the present invention is shown;

[0028] Figure 3 A schematic diagram showing matrix multiplication provided by an embodiment of the present invention is shown;

[0029] Figure 4 A structural block diagram of a computing system provided by an embodiment of the present invention is shown;

[0030] Figure 5 A structural block diagram of another computing system provided by an embodiment of the present invention is shown;

[0031] Figure 6 A flowchart showing a tensor processing method provided by an embodiment of the present invention is shown;

[0032] Figure 7 A flowchart of step S610 provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0033] Various embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. In each of the accompanying drawings, the same elements are represented by the same or similar reference numerals. For the sake of clarity, the various parts in the accompanying drawings are not drawn to scale.

[0034] The specific implementation of the present invention is further described in detail below in conjunction with the drawings and examples.

[0035] As mentioned above, in-memory computing is only suitable for general matrix computing with fixed weights, and does not support general matrix computing with dynamically changing weights. Therefore, it is necessary to call two different computing engines to perform general matrix computing with fixed weights and general matrix computing with dynamically changing weights respectively. However, two different computing engines will have two different sets of hardware interfaces, and the matrix multiplication computing power of the two computing engines is also different, which increases the complexity of the programming interface.

[0036] In response to the above technical problems, the basic concept of the present application is to use the same macro instruction to call different computing modules to perform in-memory calculations or non-memory calculations, thereby reducing programming complexity.

[0037] Figure 1 FIG. 2 is a schematic diagram showing the structure of a tensor processing device provided by an embodiment of the present invention. Figure 1 As shown, the tensor processing device 100 includes an instruction sequence module 110 , a data loading module 120 , a first calculation module 130 , a second calculation module 140 , an input register 150 , a weight register 160 and an accumulation register 170 .

[0038] The instruction sequence module 110 is used to generate a first instruction sequence and / or a second instruction sequence according to the tensor operation instruction and the preset configuration information. The first instruction sequence includes a plurality of first operation instructions, and the second instruction sequence includes a plurality of second operation instructions.

[0039] In this embodiment, the tensor operation instruction includes at least one of a matrix multiplication operation (MM), a matrix multiply-add operation (MAC) and a convolution operation (Conv).

[0040] The instruction format of the tensor operation instruction includes an opcode and an operation field, wherein the opcode includes an instruction type and an operation code, and the operation field includes at least a source address and a target address of the operation data. Specifically, the instruction type is used to describe the type of operation involved, and the instruction type is a tensor operation. The operation code is used to describe the operation to be performed by the calculation instruction (for example, addition, subtraction, multiplication, division, special functions, etc.), which specifically describes the nature and function of the operation.

[0041] The operation domain includes the source address and the target address of the operation data. The source address and the target address can be a memory address or a register address (i.e., a register number). The memory or register can be an off-chip memory. Of course, in practical applications, it can also be an on-chip memory for storing data. The data can specifically be n-dimensional data (tensor), where n is an integer greater than or equal to 2. For example, when n=2, it is 2-dimensional data, i.e., a matrix. When n=3 or more, it is a multi-dimensional tensor.

[0042] In this embodiment, the opcode may be the part of the instruction or field (usually represented by a code) specified in the computer program to perform the operation, which is an instruction sequence number used to inform the device executing the instruction which specific instruction needs to be executed. The operation domain may be the source of all data required to execute the corresponding instruction, such as the corresponding address, etc. All data required to execute the corresponding instruction include the data to be operated and the corresponding instruction processing method, etc. For an operation instruction, it must include an opcode and an operation domain, wherein the operation domain at least includes the source address and the target address of the operation data. It should be understood that those skilled in the art can set the instruction format of the operation instruction and the opcode and operation domain contained therein as needed, and the present disclosure does not limit this.

[0043] In a possible implementation, the instruction format of the operation instruction may be as shown in the following table:

[0044] Table 1:

[0045]

[0046] Wherein, type, opcode is the operation code of the computing instruction, dst, scr is the operation domain of the computing instruction, and dst is the target address. src is the source address of the data to be operated. When there are multiple data to be operated, src may include multiple data to be operated src0, src1, ..., srcn, and dst may also include multiple target addresses dst0, dst1, ..., dstn, which is not limited in the present disclosure.

[0047] In some specific embodiments, the tensor operation instruction may be as shown in the following table:

[0048] Table 2:

[0049]

[0050] When executing the operation instructions shown in the above table, the tensor processing device 100 reads Num1 tensor data from the address specified in the register Reg1, reads Num2 weight data from the address specified in the register Reg2, and then performs a convolution operation or a multiplication and addition operation, and then stores the calculation result in the address space specified in the register Reg3.

[0051] In a preferred embodiment, the source address of the operation data may be the starting address of the storage space where the operation data is located, and the tensor processing device 100 may obtain instructions and data through a data input and output unit, which may be one or more data I / O interfaces or I / O pins. Further, the tensor processing device 100 may determine the operation data according to the source address of the operation data and obtain the operation data. Of course, in other embodiments, the tensor processing device 100 may also determine the operation data required for the operation according to the operation code of the tensor operation instruction.

[0052] In this embodiment, the preset configuration information includes first configuration information and / or second configuration information. The first configuration information is used to characterize the data size, shape, and related information required for address calculation each time calculated by the first calculation module; the second configuration information is used to characterize the data size, shape, and related information required for address calculation each time calculated by the second calculation module. The data size calculated each time by the first calculation module and the data size calculated each time by the second calculation module can be the same or different, and can be set arbitrarily as needed. Among them, the relevant information required for address calculation includes the starting address and addressing method of the data, etc. The addressing method can be an interleave addressing method, but is not limited to this.

[0053] The instruction sequence module 110 is further configured to generate a selection signal according to the tensor operation instruction, and send the first instruction sequence to the first computing module or send the second instruction sequence to the second computing module according to the selection signal.

[0054] In this embodiment, the instruction sequence module 110 converts the tensor operation instruction into a plurality of first operation instructions according to the first configuration information, or converts the tensor operation instruction into a plurality of second operation instructions according to the second configuration information. The operation data of each operation instruction includes first operation data A and second operation data B, the first operation data A is divided into a plurality of first sub-operation data, the second operation data B is divided into a plurality of second sub-operation data, each sub-operation data is a data block, the address of the first sub-operation data of each row is sequentially moved along the row direction, the address of the second sub-operation data of each column is sequentially moved along the column direction, the first sub-operation data of each row is multiplied by the second sub-operation data of each column to obtain a corresponding sub-matrix block, and finally a matrix C is obtained.

[0055] In this embodiment, the instruction sequence module 110 includes an instruction parsing unit 111, an instruction sequence unit 112, a control unit 113 and a configuration information unit 114. The instruction parsing unit 111 is used to parse the tensor operation instruction to obtain the source address and target address of the operation code and the operation data; the instruction sequence unit 112 is used to automatically calculate the source address and target address of multiple sub-operation data according to the preset configuration information and the source address and target address of the operation data, and generate multiple first operation instructions and / or multiple second operation instructions according to the operation code and the source address and target address of the multiple sub-operation data; the control unit 113 is used to generate a selection signal according to the source address of the operation data, and send the first instruction sequence to the first computing module or send the second instruction sequence to the second computing module according to the selection signal; the configuration information unit 114 is used to obtain the first configuration information or the second configuration information according to the selection signal, or parse the tensor operation instruction to obtain the first configuration information or the second configuration information.

[0056] In this embodiment, the preset configuration information can be set in the tensor operation instruction, that is, the tensor operation instruction also includes the preset configuration information, or can be placed in a separate configuration register. When the tensor operation instruction also includes the preset configuration information, the instruction parsing unit parses the tensor operation instruction to obtain the preset configuration information.

[0057] In this embodiment, the operation data includes activation data and weight data; if the source address of the weight data is a storage-computation integrated unit, the weight data is static data, and the control unit 113 sends the first instruction sequence to the first computing module 130 according to the selection signal, and the configuration information unit 114 obtains the first configuration information from the instruction parsing unit or the configuration register according to the selection signal. If the source address of the weight data is a memory, the weight data is dynamic data, and the control unit 113 sends the second instruction sequence to the second computing module 140 according to the selection signal, and the configuration information unit 114 obtains the second configuration information from the instruction parsing unit or the configuration register according to the selection signal.

[0058] Since the hardware itself can only calculate matrix multiplication operations of a certain size, if you want to implement matrix multiplication operations of larger shapes, you usually need to obtain hardware information, that is, the size and shape of the data calculated by the hardware each time, during compilation, and then split an instruction into multiple operation instructions, which will increase the complexity of programming and increase the overhead of instruction emission and decoding. However, the tensor processing device of the present application, that is, the hardware itself, can automatically calculate the source address and target address of multiple sub-operation data according to the preset configuration information and the source address and target address of the operation data, and implement matrix multiplication operations of larger shapes through multiple matrix multiplication operations. In this way, the hardware information can be hidden and only one macro instruction is needed to implement matrix multiplication operations of larger shapes, which greatly reduces the complexity of programming, reduces the overhead of instruction emission and decoding, and reduces power consumption.

[0059] The data loading module 120 is used to load the data to be operated into the corresponding registers in sequence according to the first instruction sequence or the second instruction sequence.

[0060] In this embodiment, the first operation data is sequentially loaded into the input register 150 according to a plurality of first operation instructions in the first instruction sequence. The first operation data is sequentially loaded into the input register 150 and the second operation data is sequentially loaded into the weight register 160 according to a plurality of second operation instructions in the second instruction sequence.

[0061] The first calculation module 130 is used to read data from the register in sequence according to the first instruction sequence to perform in-memory calculation.

[0062] In this embodiment, the first computing module 122 can perform in-memory computing (CIM). The first computing module 122 is composed of an SRAM, ReRAM or other storage medium storage and computing integrated unit.

[0063] The second calculation module 140 is used to read data from the register in sequence according to the second instruction sequence to perform non-memory calculations.

[0064] In this embodiment, the non-in-memory calculation includes near-memory calculation or other calculation, such as general matrix multiplication (GEMM for short).

[0065] The input register 150 is used to store activation data; the weight register 160 is used to store weight data; and the accumulation register 170 is used to store the intermediate result of multiplication and addition of activation data and weight data.

[0066] See also Figure 3, this embodiment is described by taking the matrix A as 512*256, the matrix B as 256*512, and the minimum calculation size as 8*64 as an example, but it is not limited to this. The matrix A and the matrix B are divided into 8*64 data blocks at the minimum. The minimum data block of the first column in the matrix A is moved by 4 along the column direction, and the minimum data block of the first row in the matrix B is moved by 4 along the row direction to perform matrix multiplication calculation. The intermediate result obtained is placed in the accumulation register 170 for cumulative calculation. Correspondingly, the minimum data block of the next column of the matrix A is moved by 4 along the column direction, and the minimum data block of the next row in the matrix B is moved by 4 along the row direction to perform matrix multiplication calculation. The intermediate result obtained is placed in the accumulation register 170 for cumulative calculation. By analogy, the sub-matrix in the dotted box of the matrix A and the sub-matrix in the dotted box of the matrix B complete the matrix multiplication calculation to obtain the sub-matrix block 0 of the matrix C. By analogy, multiple calculations can be performed to complete the matrix multiplication operation of the matrix A and the matrix B. Figure 3 The division method of data blocks and the calculation order are schematically shown, and the division method of data blocks and the calculation order can be configured according to actual needs, and the present invention does not limit this.

[0067] The tensor processing device provided by the present invention adopts a unified macro instruction to call different computing modules to perform in-memory computing or non-in-memory computing, thereby reducing programming complexity and improving computing efficiency.

[0068] Furthermore, the tensor processing device, i.e., the hardware itself, automatically calculates the source addresses and target addresses of multiple sub-operation data according to preset configuration information and the source addresses and target addresses of the operation data, and implements matrix multiplication operations of larger shapes through multiple matrix multiplication operations. In this way, the hardware information can be hidden and only one macro instruction is needed to implement matrix multiplication operations of larger shapes, which greatly reduces the complexity of programming, reduces the overhead of instruction issuance and decoding, and reduces power consumption.

[0069] Figure 4 The block diagram of the computing system provided by the embodiment of the present invention is shown. The computing system includes a tensor processing device 100 and a general processing device 200. The general processing device 200 is used to send a tensor operation instruction to the tensor processing device 100.

[0070] In this embodiment, the first computing module 130 and the second computing module 140 are connected to the general processing device 200 using a unified interface, and the unified interface can be one of a PCIE interface and a UCIE interface. In this embodiment, the data structures of the activation data and the output data of the first computing module 130 and the second computing module 140 are the same, and the first computing module 130 and the second computing module 140 can be driven by one instruction to perform calculations, and the first computing module 130 and the second computing module 140 can also be switched to perform calculations by instructions.

[0071] In this embodiment, the tensor processing device 100 and the general processing device 200 can be packaged in the same core. Figure 4 The tensor processing device 100 and the general processing device 200 may also be packaged into different cores, integrated into the same chip or deployed on different chips. The positional relationship between the general processing device 110 and the tensor processing device 120 may be set according to actual applications, and is not limited thereto.

[0072] Figure 5 FIG. 2 is a schematic diagram showing the structure of a computing system provided by another embodiment of the present invention. Figure 5 As shown, the computing system includes a task distribution module 510 and a plurality of computing devices 520 .

[0073] This embodiment is described by taking two computing devices 520A and 520B as an example, but is not limited thereto.

[0074] In this embodiment, the task distribution module 510 is used to distribute the computing instructions to multiple computing devices 520. The computing device 520 receives the computing instructions and performs corresponding calculations according to the computing instructions. The computing devices 520A and 520B include a tensor processing device 521 and a general processing device 522. The tensor processing device 521 is the same as that described in the above embodiment and will not be repeated here.

[0075] In other embodiments, the computing devices 520A and 520B only include the general processing device 522, the tensor processing device 521 is located outside the computing devices, and multiple computing devices 520 share the same tensor processing device.

[0076] Figure 6 FIG. 1 is a flowchart of a tensor processing method provided by an embodiment of the present invention. Figure 6 The calculation method is executed by the tensor processing device 100 provided in the above embodiment, and includes the following steps.

[0077] In step S610, a first instruction sequence and / or a second instruction sequence is generated according to tensor operation instructions and preset configuration information, wherein the first instruction sequence includes multiple first operation instructions, the second instruction sequence includes multiple second operation instructions, and the preset configuration information includes first configuration information and / or second configuration information.

[0078] In this embodiment, see Figure 7 , step S610 includes step S611 to step S613.

[0079] In step S611, the tensor operation instruction is parsed to obtain the operation code and the source address and target address of the operation data and preset configuration information.

[0080] The tensor operation instruction includes at least one of a matrix multiplication operation (MM), a matrix multiplication-addition operation (MAC), and a convolution operation (Conv).

[0081] The instruction format of the tensor operation instruction includes an opcode and an operation field, wherein the opcode includes an instruction type and an operation code, and the operation field includes at least a source address and a target address of the operation data. Specifically, the instruction type is used to describe the type of operation involved, and the instruction type is a tensor operation. The operation code is used to describe the operation to be performed by the calculation instruction (for example, addition, subtraction, multiplication, division, special functions, etc.), which specifically describes the nature and function of the operation.

[0082] The operation domain includes the source address and the target address of the operation data. The source address and the target address can be a memory address or a register address (i.e., a register number). The memory or register can be an off-chip memory. Of course, in practical applications, it can also be an on-chip memory for storing data. The data can specifically be n-dimensional data (tensor), where n is an integer greater than or equal to 2. For example, when n=2, it is 2-dimensional data, i.e., a matrix. When n=3 or more, it is a multi-dimensional tensor.

[0083] In this embodiment, the opcode may be the part of the instruction or field (usually represented by a code) specified in the computer program to perform the operation, which is an instruction sequence number used to inform the device executing the instruction which specific instruction needs to be executed. The operation domain may be the source of all data required to execute the corresponding instruction, such as the corresponding address, etc. All data required to execute the corresponding instruction include the data to be operated and the corresponding instruction processing method, etc. For an operation instruction, it must include an opcode and an operation domain, wherein the operation domain at least includes the source address and the target address of the operation data. It should be understood that those skilled in the art can set the instruction format of the operation instruction and the opcode and operation domain contained therein as needed, and the present disclosure does not limit this.

[0084] In a preferred embodiment, the source address of the operation data may be the starting address of the storage space where the operation data is located, and the tensor processing device 100 may obtain instructions and data through a data input and output unit, which may be one or more data I / O interfaces or I / O pins. Further, the tensor processing device 100 may determine the operation data according to the source address of the operation data and obtain the operation data. Of course, in other embodiments, the tensor processing device 100 may also determine the operation data required for the operation according to the operation code of the tensor operation instruction.

[0085] In step S612, preset configuration information is acquired according to the selection signal, wherein the preset configuration signal includes the first configuration information and / or the second configuration information.

[0086] In this embodiment, the operation data includes activation data and weight data; if the source address of the weight data is a storage-computing integrated unit, the weight data is static data, and the first configuration information is obtained and the first instruction sequence is executed according to the selection signal. If the source address of the weight data is a memory, the weight data is dynamic data, and the second configuration information is obtained and the second instruction sequence is executed according to the selection signal.

[0087] In step S613, the source addresses and target addresses of multiple sub-operation data are automatically calculated according to the preset configuration information and the source address and target address of the operation data, and multiple first operation instructions and / or multiple second operation instructions are generated according to the operation code and the source address and target address of the multiple sub-operation data.

[0088] In this embodiment, the preset configuration information includes first configuration information and / or second configuration information. The first configuration information is used to characterize the data size, shape, and related information required for address calculation each time calculated by the first calculation module; the second configuration information is used to characterize the data size, shape, and related information required for address calculation each time calculated by the second calculation module. The data size calculated each time by the first calculation module and the data size calculated each time by the second calculation module can be the same or different, and can be set arbitrarily as needed. Among them, the relevant information required for address calculation includes the starting address and addressing method of the data, etc. The addressing method can be an interleave addressing method, but is not limited to this.

[0089] Specifically, the tensor operation instruction is converted into a plurality of first operation instructions according to the first configuration information, or the tensor operation instruction is converted into a plurality of second operation instructions according to the second configuration information. The operation data of each operation instruction includes first operation data A and second operation data B, the first operation data A is divided into a plurality of first sub-operation data, the second operation data B is divided into a plurality of second sub-operation data, each sub-operation data is a data block, the address of the first sub-operation data of each row is moved in sequence along the row direction, the address of the second sub-operation data of each column is moved in sequence along the column direction, the first sub-operation data of each row is multiplied by the second sub-operation data of each column to obtain a corresponding sub-matrix block, and finally a matrix C is obtained.

[0090] In step S620, a selection signal is generated according to the tensor operation instruction, and the first instruction sequence or the second instruction sequence is executed according to the selection signal. In this embodiment, the preset configuration information can be set in the tensor operation instruction, that is, the tensor operation instruction also includes the preset configuration information, and can also be placed in a separate configuration register. When the tensor operation instruction also includes the preset configuration information, the instruction parsing unit parses the tensor operation instruction to obtain the preset configuration information.

[0091] In step S630, the data to be operated is loaded into corresponding registers in sequence according to the first instruction sequence or the second instruction sequence.

[0092] In this embodiment, the first operation data is sequentially loaded into the input register 150 according to a plurality of first operation instructions in the first instruction sequence. The first operation data is sequentially loaded into the input register 150 and the second operation data is sequentially loaded into the weight register 160 according to a plurality of second operation instructions in the second instruction sequence.

[0093] In step S640, data is sequentially read from registers according to the first instruction sequence to perform in-memory calculations, or data is sequentially read from registers according to the second instruction sequence to perform non-in-memory calculations.

[0094] See also Figure 3 , this embodiment is described by taking the matrix A as 512*256, the matrix B as 256*512, and the minimum calculation size as 8*64 as an example, but it is not limited to this. The matrix A and the matrix B are divided into 8*64 data blocks at the minimum. The minimum data block of the first column in the matrix A is moved by 4 along the column direction, and the minimum data block of the first row in the matrix B is moved by 4 along the row direction to perform matrix multiplication calculation. The intermediate result obtained is placed in the accumulation register 170 for cumulative calculation. Correspondingly, the minimum data block of the next column of the matrix A is moved by 4 along the column direction, and the minimum data block of the next row in the matrix B is moved by 4 along the row direction to perform matrix multiplication calculation. The intermediate result obtained is placed in the accumulation register 170 for cumulative calculation. By analogy, the sub-matrix in the dotted box of the matrix A and the sub-matrix in the dotted box of the matrix B complete the matrix multiplication calculation to obtain the sub-matrix block 0 of the matrix C. By analogy, multiple calculations can be performed to complete the matrix multiplication operation of the matrix A and the matrix B.

[0095] The tensor processing method provided by the present invention adopts a unified macro instruction to call different computing modules to perform in-memory computing or non-in-memory computing, thereby reducing programming complexity and improving computing efficiency.

[0096] Furthermore, the hardware itself automatically calculates the source addresses and target addresses of multiple sub-operation data based on preset configuration information and the source addresses and target addresses of the operation data, and implements matrix multiplication operations of larger shapes through multiple matrix multiplication operations. In this way, the hardware information can be hidden and only one macro instruction is needed to implement matrix multiplication operations of larger shapes, which greatly reduces the complexity of programming, reduces the overhead of instruction issuance and decoding, and reduces power consumption.

[0097] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0098] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0099] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / electronic device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0100] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0101] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0102] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the block division of the modules or units is only a logical function block division. There may be other block division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0103] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0104] According to the embodiments of the present invention as described above, these embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and changes can be made based on the above description. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can make good use of the present invention and the modified use based on the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A tensor processing device, characterized in that: include: An instruction sequence module, configured to generate a first instruction sequence and / or a second instruction sequence according to a tensor operation instruction and preset configuration information, wherein the first instruction sequence includes a plurality of first operation instructions, the second instruction sequence includes a plurality of second operation instructions, and the preset configuration information includes first configuration information and / or second configuration information; A data loading module, used for sequentially loading the data to be operated into the corresponding register according to the first instruction sequence or the second instruction sequence; A first calculation module, used to read data from the register in sequence according to a first instruction sequence to perform in-memory calculation; The second computing module is used to read data from the register in sequence according to the second instruction sequence to perform non-memory computing.

2. The tensor processing device according to claim 1, characterized in that: The first configuration information is used to characterize the data size, shape, starting address and addressing method of each calculation by the first calculation module; the second configuration information is used to characterize the data size, shape, starting address and addressing method of each calculation by the second calculation module.

3. The tensor processing device according to claim 1, characterized in that: The instruction sequence module is also used to generate a selection signal according to the tensor operation instruction, and send the first instruction sequence to the first computing module or send the second instruction sequence to the second computing module according to the selection signal.

4. The tensor processing device according to claim 1 or 3, characterized in that: The tensor operation instruction includes an operation code and an operation field, wherein the operation code includes an instruction type and an operation code, and the operation field includes a source address and a target address of the operation data.

5. The tensor processing device according to claim 4, characterized in that: The instruction sequence module comprises: An instruction parsing unit, used for parsing the tensor operation instruction to obtain an operation code and a source address and a target address of operation data; An instruction sequence unit is used to automatically calculate the source address and target address of multiple sub-operation data according to preset configuration information and the source address and target address of the operation data, and generate multiple first operation instructions and / or multiple second operation instructions according to the operation code and the source address and target address of the multiple sub-operation data.

6. The tensor processing device according to claim 5, characterized in that: The instruction sequence unit is also used to automatically calculate the addresses of multiple first sub-operation data based on the first configuration information and the source address and destination address of the operation data to obtain multiple first operation instructions; and / or automatically calculate the addresses of multiple second sub-operation data based on the second configuration information and the source address of the operation data to obtain multiple second operation instructions.

7. The tensor processing device according to claim 5, characterized in that: The instruction sequence module also includes: The control unit is used to generate a selection signal according to a source address of the operation data, and send the first instruction sequence to the first computing module or send the second instruction sequence to the second computing module according to the selection signal.

8. The tensor processing device according to claim 5, characterized in that: The preset configuration information is set in a tensor operation instruction or a configuration register.

9. The tensor processing device according to claim 8, characterized in that: The instruction sequence module also includes: The configuration information unit is used to obtain the first configuration information or the second configuration information from the instruction parsing unit or the configuration register according to the selection signal.

10. A tensor processing method, characterized in that: include: Generate a first instruction sequence and / or a second instruction sequence according to the tensor operation instruction and preset configuration information, wherein the first instruction sequence includes a plurality of first operation instructions, the second instruction sequence includes a plurality of second operation instructions, and the preset configuration information includes first configuration information and / or second configuration information; generating a selection signal according to the tensor operation instruction, and executing a first instruction sequence or a second instruction sequence according to the selection signal; Loading the data to be operated into corresponding registers in sequence according to the first instruction sequence or the second instruction sequence; Data is sequentially read from the register according to the first instruction sequence to perform in-memory calculations, or data is sequentially read from the register according to the second instruction sequence to perform non-in-memory calculations.

11. The tensor processing method according to claim 10, characterized in that: The first configuration information is used to characterize the data size, shape, starting address and addressing method of each calculation by the first calculation module; the second configuration information is used to characterize the data size, shape, starting address and addressing method of each calculation by the second calculation module.

12. The tensor processing method according to claim 10, characterized in that: The tensor operation instruction includes an operation code and an operation field, wherein the operation code includes an instruction type and an operation code, and the operation field includes a source address and a target address of the operation data.

13. The tensor processing method according to claim 12, characterized in that: Generating the first instruction sequence and / or the second instruction sequence according to the tensor operation instruction and the preset configuration information includes: Parsing the tensor operation instruction to obtain an operation code and a source address and a target address of operation data; generating a selection signal according to a source address of the operation data, and acquiring first configuration information or second configuration information according to the selection signal; The source addresses and target addresses of multiple sub-operation data are automatically calculated according to the source address and target address of the operation data and the first configuration information or the second configuration information, and multiple first operation instructions and / or multiple second operation instructions are generated according to the operation code and the source address and target address of the multiple sub-operation data.

14. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 10 to 13 is implemented.

Images