Tensor memory and tensor processor

Abstract

To provide a tensor memory and a tensor processor which suppress occurrence of processing delay in convolution calculation, and reduce a consumption energy amount.SOLUTION: A tensor processor 105 comprises: a plurality of entries in which each of a plurality of data circulation FIFO memories M logically provided in an array stores data; a dual port memory F1 which receives data reading and writing; and a control unit (F2-F12) which controls reading / writing processing. The control unit performs first control which stores the entry number of one or more continuous entry blocks, and a stream-in-pointer indicating a leading address of the entry block, and performs the read processing while circulating one or more data stored in a specific entry block, without deleting the data for which read processing has been performed, when performing data read processing stored in a specific entry block corresponding to the entry number from a leading entry indicated by the leading address.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to tensor memory and tensor processors.

Background Art

[0002] An accelerator that executes a convolution operation in a convolutional neural network (CNN) (hereinafter also referred to as a tensor processor) repeatedly accesses not only a memory provided inside the accelerator (hereinafter also referred to as an internal memory or an on-chip memory) to acquire input data (e.g., three-dimensional data) in the convolution operation, but also a memory provided outside the accelerator (hereinafter also referred to as an external memory or an off-chip memory) (see Patent Documents 1 to 3).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0004] In this case, if the convolution operation uses multiple filters (multiple 2D filters), the accelerator needs to acquire input data each time it performs a convolution operation with each filter. Therefore, if the number of filters used in the convolution operation is large, the accelerator may frequently access external memory, which can cause processing delays in the convolution operation and increase energy consumption.

[0005] Therefore, the object of the present invention is to provide a tensor memory and a tensor processor that can suppress the occurrence of processing delays in convolution operations and reduce energy consumption. [Means for solving the problem]

[0006] To achieve the above objective, the tensor memory in the present invention has a plurality of data-circulating FIFO memories arranged logically in an array, each of the plurality of data-circulating FIFO memories having a plurality of entries for storing data, a first port for outputting the data to be read, and a second port for receiving input of the data to be written, and a control unit for controlling read and write operations in the dual-port memory, the control unit stores the number of entries in an entry block consisting of one or more consecutive entries, and a stream-in pointer indicating the starting address corresponding to the entry block, and when the read operation is performed on data stored in a specific entry block corresponding to the number of entries, with the entry indicated by the starting address as the first entry, the control unit performs a first control that performs the read operation while circulating one or more of the data stored in the specific entry block without deleting the data that has been read. [Effects of the Invention]

[0007] The tensor memory and tensor processor of the present invention make it possible to suppress processing delays in convolution operations and reduce energy consumption. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 shows an example of the configuration of the information processing device 1 in the first embodiment. [Figure 2] Figure 2 illustrates an example configuration of the tensor processor 105 in the first embodiment. [Figure 3] Figure 3 shows an example of the configuration of the data-circulating FIFO memory M in the first embodiment. [Figure 4] Figure 4 illustrates specific examples of the schematics of the first and second control systems. [Figure 5] Figure 5 illustrates a specific example of the details of the second control. [Figure 6] Figure 6 illustrates a specific example of the details of the first control. [Figure 7] Figure 7 illustrates a specific example of the details of the first control. [Figure 8] Figure 8 illustrates a specific example of storing tensor data in a data-circular FIFO memory M. [Figure 9] Figure 9 illustrates a specific example of storing tensor data in a data-circular FIFO memory M. [Figure 10] Figure 10 illustrates a specific example of a case where data is written to multiple data-circulating FIFO memories M. [Figure 11] Figure 11 illustrates a specific example of a case where data is written to multiple data-circulating FIFO memories M. [Figure 12] Figure 12 illustrates a specific example of a case where data is read from multiple data-circulating FIFO memories M. [Figure 13]FIG. 13 is a diagram for explaining a specific example when data is read from a plurality of data circular FIFO memories M. [Figure 14] FIG. 14 is a diagram for explaining a specific example when data is read from a plurality of data circular FIFO memories M. [Figure 15] FIG. 15 is a diagram for explaining a specific example when data is read from a plurality of data circular FIFO memories M.

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, such embodiment examples do not limit the technical scope of the present invention.

[0010] [Configuration Example of Information Processing Apparatus 1 in the First Embodiment] First, a configuration example of the information processing apparatus 1 in the first embodiment will be described. FIG. 1 is a diagram showing a configuration example of the information processing apparatus 1 in the first embodiment.

[0011] The information processing apparatus 1 is a computer apparatus, and performs a process of inferring output data for input data (hereinafter, also simply referred to as an inference process) by using, for example, a CNN neural network learned with teacher data. Hereinafter, it will be described assuming that the input data is three-dimensional data (hereinafter, also referred to as tensor data).

[0012] The information processing apparatus 1 has, for example, as shown in FIG. 1, a CPU (Central Processing Unit) 101, a memory 102, a communication interface 103, a storage medium 104, and a tensor processor 105 (accelerator). Each part is connected to each other via a bus 106. Note that the information processing apparatus 1 may have, for example, a dedicated circuit that performs the same processing as the CPU 101 instead of the CPU 101.

[0013] The storage medium 104 has, for example, a program storage area (not shown) that stores a program (not shown) for performing inference processing.

[0014] Also, the storage medium 104 has, for example, a storage area (not shown) that stores information used when performing inference processing. Note that the storage medium 104 may be, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0015] The CPU 101 performs inference processing by executing a program (not shown) loaded from the storage medium 104 into the memory 102. The memory 102 is, for example, a DRAM (Dynamic Random Access Memory).

[0016] The tensor processor 105 is, for example, an FPGA (Field Programmable Gate Array) or the like, and executes a convolution operation in inference processing.

[0017] Specifically, the tensor processor 105 has, for example, a data circulation FIFO memory M that stores tensor data, and a tensor arithmetic unit 105b (hereinafter, also simply referred to as the arithmetic unit 105b) that performs a convolution operation using the tensor data stored in the data circulation FIFO memory M or the like.

[0018] The communication interface 103 communicates with the worker terminal 2 via a network NW such as the Internet. Note that the worker terminal 2 may be, for example, a PC (Personal Computer), and may be a terminal for a worker (hereinafter, also simply referred to as a worker) who generates a learning model to input necessary information and the like.

[0019] Note that the CPU 101 may be, for example, mounted inside the tensor processor 105.

[0020] [Configuration example of tensor processor 105] Next, we will describe an example of the configuration of the tensor processor 105 in the first embodiment. Figure 2 is a diagram illustrating an example of the configuration of the tensor processor 105 in the first embodiment. Hereinafter, each of the multiple two-dimensional data that constitute the tensor data will simply be referred to as "data".

[0021] The tensor processor 105 includes, for example, a tensor memory 105a, an arithmetic unit 105b, an instruction memory 105c, a DMAC (Direct Memory Access Controller) 105d, and a control unit 105e. Furthermore, the tensor memory 105a includes, for example, a data-cyclic FIFO memory M.

[0022] The control unit 105e controls the tensor memory 105a (data circular FIFO memory M), the arithmetic unit 105b, and the DMAC 105d, for example, by decoding instructions received from the instruction memory 105c and issuing microinstructions.

[0023] For example, the DMAC105d performs data transfer from memory 102 (external memory) to data-circulating FIFO memory M (internal memory) according to the contents of a microinstruction issued by the control unit 105e.

[0024] Specifically, the DMAC105d, for example, in accordance with the contents of a microinstruction issued by the control unit 105e (hereinafter also called a stream-in instruction), transfers data (hereinafter also called a data block) that starts at the address specified by the stream-in instruction and has the number of data points (data length) specified by the stream-in instruction from memory 102 to the data-circulating FIFO memory M.

[0025] The arithmetic unit 105b reads data from the data-circulating FIFO memory M according to the contents of a microinstruction issued by the control unit 105e, for example. Then, the arithmetic unit 105b performs a predetermined operation on the read data, for example, and writes the operation result to the data-circulating FIFO memory M.

[0026] Specifically, the arithmetic unit 105b reads data required for the operation specified by the tensor operation instruction (hereinafter also called a tensor operation) from the data-circulating FIFO memory M, according to the contents of the multi-step microinstruction (hereinafter also called a tensor operation instruction) issued by the control unit 105e, performs the tensor operation on the read data, and then writes the result of the tensor operation to the data-circulating FIFO memory M. The tensor operation is, for example, a multiply-accumulate operation.

[0027] Furthermore, the DMAC105d performs data transfer from the data-circulating FIFO memory M to the memory 102, for example, according to the contents of the microinstructions issued by the control unit 105e.

[0028] Specifically, the DMAC105d, for example, in accordance with the contents of a microinstruction (hereinafter also called a stream-out instruction) issued by the control unit 105e, causes the data circulating FIFO memory M to transfer a data block of the data length (number of data points) specified by the stream-out instruction to the address specified by the stream-out instruction from the data circulating FIFO memory M to memory 102.

[0029] Furthermore, in the data-circulating FIFO memory M, for example, according to the contents of a microinstruction (hereinafter also called a tensor memory control instruction) issued by the control unit 105e, the data blocks stored in the data-circulating FIFO memory M can be deleted or the data blocks to be processed can be switched.

[0030] In other words, the tensor processor 105 in this embodiment has a data-circulating FIFO memory M that stores data transferred from memory 102, which makes it possible to reduce the number of accesses to memory 102, for example. As a result, the tensor processor 105 can reduce, for example, the time required to access data and the amount of energy required to access data.

[0031] [Example configuration of data-circulating FIFO memory M] Next, we will describe an example configuration of the data-circulating FIFO memory M in the first embodiment. Figure 3 is a diagram showing an example configuration of the data-circulating FIFO memory M in the first embodiment.

[0032] In this embodiment, the data-circulating FIFO memory M functions, for example, as one of one or more FIFO (First-In-First-Out) memories arranged logically in an array. The data-circulating FIFO memory M in this embodiment has, for example, multiple entries and performs a control (hereinafter also referred to as first control or ring buffer mode) that stores data in a ring buffer for at least some of the multiple entries.

[0033] First, we will explain each instruction issued by the control unit 105e to the data-circulating FIFO memory M. Specifically, we will explain the instructions issued when the first control is performed in the data-circulating FIFO memory M.

[0034] The Stream-In instruction (hereinafter also referred to as the STINR instruction) is an instruction that transfers, for example, a data block from the source memory 102 to the destination data-circulating FIFO memory M (the data-circulating FIFO memory M specified by the FIFO number included in the STINR instruction), the data block being transferred starting at the transfer start address (address on memory 102) set in the register indicated by the register number included in the STINR instruction, and consisting of a number of data corresponding to the number of data included in the STINR instruction (hereinafter also referred to as STINR_d).

[0035] Specifically, the control unit 105e decodes, for example, the STINR instruction stored in the instruction memory 105c and sends a STINR microinstruction to the DMAC 105d and the data-circulating FIFO memory M, respectively. The STINR microinstruction is an instruction that includes, for example, STINR_d, as well as information (hereinafter also referred to as STINR_start) that instructs the start of the transfer of a data block from memory 102 to the data-circulating FIFO memory M. In other words, in this case, the control unit 105e instructs the DMAC 105d and the data-circulating FIFO memory M, respectively, to transfer data from memory 102 to the data-circulating FIFO memory M. Then, when the DMAC 105d receives, for example, the instruction from the control unit 105e, it starts reading the data stored in memory 102 according to the information contained in the received STINR microinstruction (for example, STINR_start and STINR_d). On the other hand, when the data-circulating FIFO memory M receives a STINR microinstruction from the control unit 105e, for example, it sets the number of entries that function as a ring buffer (hereinafter simply referred to as d) according to the information contained in the received STINR microinstruction (e.g., STINR_d), and waits until data is transferred from the DMAC 105d. Then, the DMAC 105d, for example, starts reading data stored in memory 102, and then transfers data to the data-circulating FIFO memory M.

[0036] Furthermore, the transfer of data from memory 102 to data-circulating FIFO memory M is performed, for example, at a clock cycle where the valid signal, which indicates whether the circuit on the memory 102 side is outputting valid data, is "1", indicating that the circuit on the memory 102 side is outputting valid data, and the full signal, which indicates whether there are any free entries in the data-circulating FIFO memory M, is "0", indicating that there are free entries in the data-circulating FIFO memory M.

[0037] Furthermore, the FLUSH instruction is an instruction that deletes the data block stored at the beginning of the data-circulating FIFO memory M, which is specified by the FIFO number included in the FLUSH instruction, from among the data stored in the data-circulating FIFO memory M by the first control. Specifically, the FLUSH instruction is an instruction that deletes the data block stored at the beginning of the data-circulating FIFO memory M.

[0038] Furthermore, the SWITCH instruction is an instruction that, for example, if multiple data blocks are stored in the data-circular FIFO memory M, changes the data block to be read to the data block corresponding to the number included in the SWITCH instruction.

[0039] Next, we will describe an example configuration of the data-circulating FIFO memory M. As shown in Figure 3, the data-circulating FIFO memory M includes, for example, a dual-port memory F1, a register F2, an arithmetic unit F3 (hereinafter also referred to as Inc), a buffer F4, a register F5, an arithmetic unit F6, a register F7, an arithmetic unit F8 (hereinafter also referred to as Inc), a register F9, an arithmetic unit F10, an arithmetic unit F11, and an arithmetic unit F12. Hereafter, register F2, arithmetic unit F3, buffer F4, register F5, arithmetic unit F6, register F7, arithmetic unit F8, register F9, arithmetic unit F10, arithmetic unit F11, and arithmetic unit F12 will be collectively referred to simply as the control unit.

[0040] The dual-port memory F1 has, for example, a first port (hereinafter also called WritePort) that accepts input of data to be written, and a second port (hereinafter also called ReadPort) that outputs data to be read. In other words, by having a ReadPort and a WritePort, the dual-port memory F1 can perform data writing and data reading simultaneously (in parallel), thereby achieving faster processing. Specifically, the data to be written is, for example, data transferred from memory 102 (source) to data-circulating FIFO memory M. The data to be read is, for example, data read by the arithmetic unit 105b that performs convolution operations. The dual-port memory F1 may be, for example, SRAM (Static Random Access Memory).

[0041] Buffer F4 stores, for example, the STINR_d contained in the STINR microinstruction issued by the control unit 105e. Therefore, buffer F4 has, for example, one or more areas that store the STINR_d corresponding to one or more data blocks.

[0042] Specifically, the buffer F4 shown in Figure 3 has, for example, an area for storing the STINR_d corresponding to the first data block (the area corresponding to "d") and an area for storing the STINR_d corresponding to the second data block (hereinafter also referred to as STINR_d') (the area corresponding to "d'").

[0043] The storage of STINR_d to buffer F4 is performed, for example, when the wen signal, which indicates whether data can be written or not, is "1", indicating that data can be written. The wen signal is "1", for example, when the valid signal is "1" and the full signal of the data-circular FIFO memory M is "0".

[0044] Register F5 stores, for example, the number of entries (hereinafter also referred to as Entrycount) in which data is stored in the dual-port memory F1.

[0045] Furthermore, if the Entrycount reaches the upper limit of the dual-port memory F1 (the upper limit of the number of data that the dual-port memory F1 can store), the full signal in the data-circulating FIFO memory M is updated to indicate "1".

[0046] The arithmetic unit F6 updates the Entrycount stored in register F5, for example. Specifically, the arithmetic unit F6 updates the Entrycount stored in register F5 when, for example, the wen signal indicates "1".

[0047] Furthermore, when a FLUSH instruction is issued, the arithmetic unit F6 stores in register F5 a new Entrycount, which is obtained by subtracting the value corresponding to the number of data in the first data block stored in buffer F4 from the Entrycount stored in register F5.

[0048] Register F2 stores, for example, a write pointer (hereinafter also called Writeptr) that indicates the address of the next entry in which data will be written.

[0049] The arithmetic unit F3 is, for example, an incrementer, which increments the address indicated by the Writeptr stored in register F2.

[0050] Specifically, for example, when the STINR microinstruction is issued by the control unit 105e and the wen signal is "1", the data transferred from memory 102 via din, which is the data input port, is written to the address specified by Writeptr. Then, the arithmetic unit F3 increments the address indicated by Writeptr, which is stored in register F2, for example.

[0051] Furthermore, in this case, the arithmetic unit F6 increments the value indicated by the Entrycount stored in register F5, for example. That is, since the dual-port memory F1 has both a ReadPort and a WritePort, the data-circulating FIFO memory M can perform data writing and data reading simultaneously (in parallel), and therefore updates the Entrycount when data is written.

[0052] Furthermore, if the value indicated by Entrycount stored in register F5 is "1" or greater, the empty signal, which indicates whether or not data can be read, will indicate "1". Register F7 stores, for example, a read pointer (hereinafter also called Readptr) that indicates the address of the next entry to be read.

[0053] The arithmetic unit F8 is, for example, an incrementer, which increments the address indicated by Readptr stored in register F7.

[0054] Specifically, for example, if the ren signal, which indicates whether data can be read, is "1", indicating that data can be read, the data stored at the address specified by Readptr is output to the arithmetic unit 105b via dout, which is the data output port. Then, the arithmetic unit F8, in this case, for example, increments the address indicated by Readptr stored in register F7.

[0055] Furthermore, in this case, the arithmetic unit F6 decrements the value indicated by Entrycount, which is stored in register F5. That is, in the data-circulating FIFO memory M, since the dual-port memory F1 has both a ReadPort and a WritePort, it is possible to write and read data simultaneously (in parallel), and therefore, when data is read, the Entrycount is also updated.

[0056] Register F9 stores, for example, a stream-in pointer (hereinafter also called STINRprt) that indicates the address of the first entry in an entry block (hereinafter also simply called an entry block) consisting of multiple entries in which the data block to be read is stored (the smallest address in the entry block).

[0057] In the data-circulating FIFO memory M shown in Figure 3, only one register F9 is installed, but the data-circulating FIFO memory M may be equipped with multiple registers F9. This allows the data-circulating FIFO memory M to store, for example, multiple STINRprts simultaneously. In this case, the data-circulating FIFO memory M can also perform processing corresponding to the SWITCH instruction, which requires the simultaneous storage of multiple STINRprts.

[0058] For example, when data is read from the arithmetic unit 105b, the arithmetic unit F10 adds the value indicated by STINR_d stored in buffer F4 to STINRprt stored in register F9.

[0059] For example, arithmetic unit F11 compares Readptr, which has been updated (incremented) by arithmetic unit F8, with STINRprt, which has been calculated by arithmetic unit F10.

[0060] For example, if arithmetic unit F12 determines, based on the comparison performed by arithmetic unit F11, that the Readptr updated by arithmetic unit F8 matches the STINRprt calculated by arithmetic unit F10, it updates the Readptr stored in register F7 with the STINRprt calculated by arithmetic unit F10.

[0061] In the above example, we have described the case where the first control is performed in the data-circulating FIFO memory M. However, the data-circulating FIFO memory M may also perform a control that stores data according to FIFO in all entries constituting the data-circulating FIFO memory M (hereinafter also referred to as the second control or FIFO mode). In this case, the data-circulating FIFO memory M may also perform either the first control or the second control according to settings made in advance by the user.

[0062] [Specific examples of the outlines of the first and second control systems] Next, we will explain specific examples of the general outlines of the first and second controls. Figure 4 is a diagram illustrating specific examples of the general outlines of the first and second controls.

[0063] First, we will explain a specific example of the outline of the second control. Specifically, Figure 4(A) is a diagram illustrating a specific example of the outline of the second control. In the following explanation, we will assume that the number of entries in the data-circulating FIFO memory M (n in the example shown in Figure 4(A)) is "8".

[0064] As shown in Figure 4(A), when the data-circulating FIFO memory M receives a data transfer from memory 102 (the source), it stores the received data in order from the first entry. Furthermore, when the data-circulating FIFO memory M receives a read request from the arithmetic unit 105b, it reads the data stored in the first entry and deletes the data that has been read. Specifically, the example shown in Figure 4(A) illustrates the case where the number of data items stored in the data-circulating FIFO memory M is "3".

[0065] Next, we will explain a specific example of the outline of the first control. Specifically, Figure 4(B) is a diagram illustrating a specific example of the outline of the first control.

[0066] In the data-circulating FIFO memory M, as shown in Figure 4(B), when it receives a data transfer from memory 102 (the source), it stores the received data in order from the first entry. On the other hand, when the data-circulating FIFO memory M receives a read request from the arithmetic unit 105b, it reads the data stored in the first entry and maintains a state where it can be read again without deleting the data that has been read. Specifically, the example shown in Figure 4(B) shows a case where the number of entries that function as a ring buffer (d in the example shown in Figure 4(B)) is "6", and the data stored in each entry that functions as a ring buffer is repeatedly read while being circulated.

[0067] As a result, the data-circulating FIFO memory M in this embodiment can suppress the frequency of access to external memory, such as memory 102. Therefore, the data-circulating FIFO memory M can suppress the occurrence of processing delays in convolution operations, for example, and can also suppress power consumption.

[0068] [Specific examples of the details of the first and second control] Next, we will explain specific examples of the details of the first and second controls. Figures 5 to 7 illustrate specific examples of the details of the first and second controls.

[0069] First, we will explain a specific example of the details of the second control. Figure 5 is a diagram illustrating a specific example of the details of the second control.

[0070] For example, if no data is stored in all entries, in the data-circulating FIFO memory M, both Readptr and Writeptr will indicate "0", which is the first entry, as shown in Figure 5(A).

[0071] Then, for example, if each piece of data is stored in an entry corresponding to each of the numbers from "0" to "4", the Writeptr in the data-circulating FIFO memory M will be updated to "5", as shown in Figure 5(B).

[0072] Next, for example, if the data stored in each entry corresponding to "0" through "4" is read and deleted, the Readptr in the data-circulating FIFO memory M is updated to "5", as shown in Figure 5(C). Also, for example, if each piece of data is stored in each entry corresponding to "5" through "7", and then the data is stored in the entry corresponding to "0" (i.e., the entry following the entry corresponding to "7"), the Writeptr in the data-circulating FIFO memory M is updated to "1", as shown in Figure 5(C).

[0073] Furthermore, for example, if data stored in the entry corresponding to "5" is read and deleted, the data-circulating FIFO memory M updates Readptr to "6" as shown in Figure 5(D). Also, for example, if each piece of data is stored in the respective entry corresponding to "1" through "5", the data-circulating FIFO memory M updates Writeptr to "6" as shown in Figure 5(D).

[0074] Next, we will explain specific examples of the details of the first control. Figures 6 and 7 illustrate specific examples of the details of the first control. Below, we will explain the case where STINRprt stored in register F9 is "1", and STINR_d and STINR_d' stored in buffer F4 are "3" and "5", respectively.

[0075] For example, if data is stored in an entry corresponding to "0" and the data has not yet been read, then in the data-circulating FIFO memory M, Readptr will show "0" and Writeptr will show "1", as shown in Figure 6(A).

[0076] For example, if data stored in the entry corresponding to "0" is read, the data-circulating FIFO memory M updates Readptr to "1", as shown in Figure 6(B). In this case, unlike when the second control is performed, the data-circulating FIFO memory M does not delete the data stored in the entry corresponding to "0". Also, for example, if each piece of data is stored in the respective entries corresponding to "1" and "2", the data-circulating FIFO memory M updates Writeptr to "3", as shown in Figure 6(B).

[0077] Next, for example, if data stored in the entry corresponding to "1" is read, the data-circulating FIFO memory M updates Readptr to "2", as shown in Figure 6(C). In this case as well, unlike when the second control is performed, the data-circulating FIFO memory M does not delete the data stored in the entry corresponding to "1". Also, for example, if each piece of data is stored in the respective entries corresponding to "3" and "4", the data-circulating FIFO memory M updates Writeptr to "5", as shown in Figure 6(C).

[0078] Furthermore, for example, if data stored in the entry corresponding to "2" is read, the data-circulating FIFO memory M updates Readptr to "0" as shown in Figure 6(D). That is, in the example shown in Figure 6, since STINRprt is "0" and STINR_d is "3", each of the entries corresponding to "0" through "2" is an entry that functions as a ring buffer. Therefore, in this case, Readptr is updated to indicate the entry corresponding to "0", which is the next entry after the entry corresponding to "2". Note that, even in this case, unlike when the second control is performed, the data stored in the entry corresponding to "2" is not deleted in the data-circulating FIFO memory M. Also, for example, if data is stored in the entry corresponding to "5", the data-circulating FIFO memory M updates Writeptr to "6" as shown in Figure 6(D).

[0079] Subsequently, for example, if a FLUSH instruction is issued, the data-circulating FIFO memory M deletes the data blocks stored in multiple entries (entries corresponding to "0" through "2"), including the entry corresponding to "0" indicated by STINRprt, as shown in Figure 6(E). At the same time, "3," calculated by adding "3" indicated by STINR_d to "0" indicated by STINRprt, is used as the new STINRprt and new Readptr. Also, for example, if data is stored in the entry corresponding to "6," the data-circulating FIFO memory M will show "7" as Writeptr, as shown in Figure 6(E).

[0080] In other words, in this case, since the new STINRprt is "3" and STINR_d' is "5", each of the entries corresponding to "3" through "7" in the data-circular FIFO memory M functions as a new ring buffer.

[0081] This allows the data-circulating FIFO memory M to switch entries that function, for example, as a ring buffer. Therefore, the data-circulating FIFO memory M can switch data that is repeatedly read by circulating it.

[0082] For example, if a SWITCH instruction is issued after the state shown in Figure 6(D), the data circular FIFO memory M will not delete the data blocks stored in multiple entries (entries corresponding to "0" through "2") that include the entry corresponding to "0" indicated by STINRprt, as shown in Figure 7. Instead, it will calculate "3" by adding "3" indicated by STINR_d to "0" indicated by STINRprt, and use this "3" as the new STINRprt and new Readptr.

[0083] In other words, the data-circulating FIFO memory M may store multiple data blocks. Each data block stored in the data-circulating FIFO memory may become the target of reading again after the target of reading has been moved to another data block.

[0084] Therefore, in such cases, the control unit 105e issues a SWITCH instruction, which prevents the deletion of the data block that was being read, even if the data to be read moves to another data block.

[0085] As a result, in the data-circulating FIFO memory M of this embodiment, it becomes possible to further suppress the frequency of access to external memory such as memory 102.

[0086] [Storing tensor data DT in a data-cyclic FIFO memory M] Next, we will explain a specific example of storing tensor data DT in a data-circulating FIFO memory M. Figures 8 and 9 illustrate a specific example of storing tensor data DT in a data-circulating FIFO memory M. Hereafter, we will assume that the tensor processor 105 has nine data-circulating FIFO memories M implemented, each including a dual-port memory F1, register F2, arithmetic unit F3, buffer F4, register F5, arithmetic unit F6, register F7, arithmetic unit F8, register F9, arithmetic unit F10, arithmetic unit F11, and arithmetic unit F12.

[0087] The control unit 105e, for example, divides the tensor data DT into multiple parts and transfers them to each of the nine data-cyclic FIFO memories M.

[0088] Specifically, as shown in Figure 8(A), the control unit 105e generates 20 divided data DTa (hereinafter also called pillar DTa) from the tensor data DT by, for example, dividing it into 5 horizontally and 4 vertically. In this case, the control unit 105e may, for example, temporarily store the generated pillar DTa in memory 102 (external memory). Furthermore, the generation of pillar DTa by dividing the tensor data DT may be performed by a device other than the control unit 105e (for example, the tensor processor 105).

[0089] Then, as shown in Figure 8(B), the control unit 105e stores, for example, the first pillar DTa (pillar DTa corresponding to "1" in Figure 8) in the first data circulating FIFO memory M1, the second pillar DTa (pillar DTa corresponding to "2" in Figure 8) in the second data circulating FIFO memory M2, the third pillar DTa (pillar DTa corresponding to "3" in Figure 8) in the third data circulating FIFO memory M3, the fourth pillar DTa (pillar DTa corresponding to "4" in Figure 8) is stored again in the data circulating FIFO memory M1, and the fifth pillar DTa (pillar DTa corresponding to "5" in Figure 8) is stored again in the data circulating FIFO memory M2.

[0090] Furthermore, the control unit 105e stores, for example, the 6th pillar DTa (pillar DTa corresponding to "6" in Figure 8) in the 4th data circulating FIFO memory M4, the 7th pillar DTa (pillar DTa corresponding to "7" in Figure 8) in the 5th data circulating FIFO memory M5, the 8th pillar DTa (pillar DTa corresponding to "8" in Figure 8) in the 6th data circulating FIFO memory M6, the 9th pillar DTa (pillar DTa corresponding to "9" in Figure 8) back in the data circulating FIFO memory M4, and the 10th pillar DTa (pillar DTa corresponding to "10" in Figure 8) back in the data circulating FIFO memory M5.

[0091] Furthermore, the control unit 105e stores, for example, the 11th pillar DTa (the pillar DTa corresponding to "11" in Figure 8) in the 7th data circulating FIFO memory M7, the 12th pillar DTa (the pillar DTa corresponding to "12" in Figure 8) in the 8th data circulating FIFO memory M8, the 13th pillar DTa (the pillar DTa corresponding to "13" in Figure 8) in the 9th data circulating FIFO memory M9, the 14th pillar DTa (the pillar DTa corresponding to "14" in Figure 8) is stored again in the data circulating FIFO memory M7, and the 15th pillar DTa (the pillar DTa corresponding to "15" in Figure 8) is stored again in the data circulating FIFO memory M8.

[0092] Furthermore, the control unit 105e stores, for example, the 16th pillar DTa (the pillar DTa corresponding to "16" in Figure 8) in the data-circulating FIFO memory M1, the 17th pillar DTa (the pillar DTa corresponding to "17" in Figure 8) in the data-circulating FIFO memory M2, the 18th pillar DTa (the pillar DTa corresponding to "18" in Figure 8) in the data-circulating FIFO memory M3, the 19th pillar DTa (the pillar DTa corresponding to "19" in Figure 8) is stored again in the data-circulating FIFO memory M1, and the 20th pillar DTa (the pillar DTa corresponding to "20" in Figure 8) is stored again in the data-circulating FIFO memory M2.

[0093] Here, for example, if a convolution operation is performed using the 7th pillar DTa, then a portion of the pillars DTa located around the 7th pillar DTa will also be used in the operation.

[0094] Specifically, in the example shown in Figure 8, the pillars DTa adjacent to the 7th pillar DTa are the 1st pillar DTa, the 2nd pillar DTa, the 3rd pillar DTa, the 6th pillar DTa, the 8th pillar DTa, the 11th pillar DTa, the 12th pillar DTa, and the 13th pillar DTa. Therefore, for example, when a convolution operation is performed using the 7th pillar DTa, the arithmetic unit 105b needs to acquire data block B that includes not only the 7th pillar DTa, but also parts of the 1st pillar DTa, the 2nd pillar DTa, the 3rd pillar DTa, the 6th pillar DTa, the 8th pillar DTa, the 11th pillar DTa, the 12th pillar DTa, and the 13th pillar DTa (hereinafter also referred to as the halo region), as shown in Figure 9.

[0095] In the example shown in Figure 8, three adjacent pillars DTa in the row direction are stored in different data-circulating FIFO memories M, and three adjacent pillars DTa in the column direction are also stored in different data-circulating FIFO memories M.

[0096] Specifically, in the example shown in Figure 8, for example, the first, second, and third pillars DTa are stored in different data-circulating FIFO memories M (data-circulating FIFO memories M1, M2, and M3), and the first, sixth, and eleventh pillars DTa are stored in different data-circulating FIFO memories M (data-circulating FIFO memories M1, M4, and M7).

[0097] Furthermore, in the example shown in Figure 8, for example, the 8th, 9th, and 10th pillars DTa are stored in different data-circulating FIFO memories M (data-circulating FIFO memories M6, M4, and M5), and the 8th, 13th, and 16th pillars DTa are stored in different data-circulating FIFO memories M (data-circulating FIFO memories M6, M9, and M3). As a result, as shown in Figure 9, if the number of pillars DTa that the arithmetic unit 105b reads simultaneously is 9 (3 x 3), the data-circulating FIFO memory M can maintain a state where the 9 pillars DTa that are read simultaneously are stored in different data-circulating FIFO memories. Therefore, the data-circulating FIFO memory M can, for example, reduce the time required for data reading by the arithmetic unit 105b.

[0098] In other words, the control unit 105e divides the 3D tensor data DT into smaller pillars DTa, and performs a multiply-accumulate operation on each divided pillar DTa. Here, access to the pillar DTa that is the target of the multiply-accumulate operation must be performed simultaneously not only on the pillar DTa being targeted, but also on the surrounding data (halo region) that spans multiple adjacent pillars DTa.

[0099] In this respect, in the tensor processor 105 (multiple data circulating FIFO memories M) of this embodiment, multiple adjacent pillars DTa in the row direction are stored in different data circulating FIFO memories M, and multiple adjacent pillars DTa in the column direction are stored in different data circulating FIFO memories M. Therefore, the control unit 105e can simultaneously (in parallel) access the pillar DTa to be accessed and the data stored in the halo area corresponding to the pillar DTa to be accessed.

[0100] [Specific example of writing data to multiple data-circular FIFO memory Ms] Next, we will explain a specific example of when data is written to multiple data-circulating FIFO memories M (hereinafter also simply referred to as the write process). Figures 10 and 11 illustrate a specific example of when data is written to multiple data-circulating FIFO memories M. Below, we will explain the case where there are 9 pillars DTa (3 x 3), as shown in Figure 9.

[0101] In the following explanation, the variable DTx represents the column of each pillar DTa in the tensor data DT, and the variable DTy represents the row of each pillar DTa in the tensor data DT. Furthermore, in the following explanation, the variable Fx represents the column of each data cycle FIFO memory M in multiple data cycle FIFO memories M, and the variable Fy represents the row of each data cycle FIFO memory M in multiple data cycle FIFO memories M.

[0102] As shown in Figure 10, the control unit 105e is initialized by, for example, setting variables DTy and Fy to 1, and variables DTx and Fx to 1 (S11 and S12).

[0103] Then, the control unit 105e, for example, by issuing a STINR microinstruction, stores the pillar DTa in row DTy and column DTx into the data circular FIFO memory M in row Fy and column Fx (S13).

[0104] Next, the control unit 105e determines, for example, whether the variable DTx has reached a value corresponding to the number of columns of pillar DTa (S14).

[0105] As a result, if it is determined that the variable DTx has not reached the value corresponding to the number of columns of pillar DTa (NO in S14), the control unit 105e sets the variable Fx to a value obtained by adding 1 to the remainder of variable Fx by 3, as shown in Figure 11 (S21).

[0106] Then, the control unit 105e sets the variable DTx to DTx+1, for example (S22). After that, the control unit 105e repeats the processing from S13 onwards, for example.

[0107] Furthermore, if it is determined that the variable DTx has reached a value corresponding to the number of columns of pillar DTa (YES in S14), the control unit 105e determines, for example, whether the variable DTy has reached a value corresponding to the number of rows of pillar DTa (S15).

[0108] As a result, if it is determined that the variable DTy has not reached the value corresponding to the number of rows of pillar DTa (NO in S15), the control unit 105e sets the variable Fy to a value obtained by adding 1 to the remainder of variable Fy by 3, as shown in Figure 11 (S23).

[0109] Then, the control unit 105e sets the variable DTy to DTy+1, for example (S24). After that, the control unit 105e repeats the processing from S12 onwards, for example.

[0110] On the other hand, if the control unit 105e determines that the variable DTy has reached a value corresponding to the number of rows in pillar DTa (YES in S15), it terminates the write process, for example.

[0111] [Specific example of reading data from multiple data-circular FIFO memory Ms] Next, we will explain a specific example of a case where data is read from multiple data-circulating FIFO memories M (hereinafter also simply referred to as the read operation). Figures 12 to 15 illustrate a specific example of a case where data is read from multiple data-circulating FIFO memories M. Below, we will explain the case where there are 9 pillars DTa (3 x 3), as shown in Figure 9.

[0112] In the following explanation, the variable DTx represents the column of each pillar DTa in the tensor data DT, and the variable DTy represents the row of each pillar DTa in the tensor data DT. Furthermore, in the following explanation, the variables Fx, Fx1, Fx2, and Fx3 represent the columns of each data-cycle FIFO memory M in multiple data-cycle FIFO memories M, and the variables Fy, Fy1, Fy2, and Fy3 represent the rows of each data-cycle FIFO memory M in multiple data-cycle FIFO memories M. Furthermore, in the following explanation, BD21, BD22, BD23, BD31, BD32, and BD33 represent the storage order of data blocks in each data-cycle FIFO memory M.

[0113] As shown in Figure 12, the control unit 105e is initialized by, for example, setting variables DTy and Fy to 1, and variables DTx and Fx to 1 (S31 and S32).

[0114] Then, the control unit 105e sets the variable Fx to variable Fx1, sets the value obtained by adding 1 to the remainder of variable Fx by 3 for variable Fx2, and further sets the value obtained by adding 1 to the remainder of variable Fx+1 by 3 for variable Fx3 (S33). Also, the control unit 105e sets the variable Fy to variable Fy1, sets the value obtained by adding 1 to the remainder of variable Fy by 3 for variable Fy2, and further sets the value obtained by adding 1 to the remainder of variable Fy+1 by 3 for variable Fy3 (S34).

[0115] Next, the control unit 105e sets, for example, the quotient obtained by dividing variable DTx-1 by 3 for each of variables DB21 and DB31 (S35). Also, the control unit 105e sets, for example, the quotient obtained by dividing variable DTx by 3 for each of variables DB22 and DB32 (S36). Furthermore, the control unit 105e sets, for example, the quotient obtained by dividing variable DTx+1 by 3 for each of variables DB23 and DB33 (S37).

[0116] Subsequently, as shown in Figure 13, the control unit 105e switches the target of reading the three data circular FIFO memory M of row Fy1 to the first data block by issuing a SWITCH instruction (S41).

[0117] Furthermore, the control unit 105e switches the target of reading the data circulating FIFO memory M at row Fy2 and column Fx1 from the beginning to the 21st data block in DB, by issuing a SWITCH command, for example (S42). Furthermore, the control unit 105e switches the target of reading the data circulating FIFO memory M at row Fy2 and column Fx2 from the beginning to the 22nd data block in DB, by issuing a SWITCH command, for example (S43). Furthermore, the control unit 105e switches the target of reading the data circulating FIFO memory M at row Fy2 and column Fx3 from the beginning to the 23rd data block in DB, by issuing a SWITCH command, for example (S44).

[0118] Furthermore, the control unit 105e switches the data block to be read from the beginning of the data circulating FIFO memory M at row Fy3 and column Fx1 by issuing a SWITCH command, for example (S45). Also, the control unit 105e switches the data block to be read from the beginning of the data circulating FIFO memory M at row Fy3 and column Fx2 by issuing a SWITCH command, for example (S46). Also, the control unit 105e switches the data block to be read from the beginning of the data circulating FIFO memory M at row Fy3 and column Fx3 by issuing a SWITCH command, for example (S47).

[0119] Then, the control unit 105e reads the current data block from all data-circulating FIFO memory M in ring buffer mode, for example, and inputs it to the arithmetic unit (S48).

[0120] Subsequently, the control unit 105e determines, for example, whether all processing (e.g., convolution operations using all filters) for the current combination of data blocks has been completed, as shown in Figure 14 (S51).

[0121] If, as a result, it is determined that all processing for the current data block combination has not been completed (NO in S51), the control unit 105e will, for example, repeat the processing from S48 onwards.

[0122] On the other hand, if it is determined that all processing for the current data block combination has been completed (YES in S51), the control unit 105e determines, for example, whether the variable DTx+2 has reached a value corresponding to the number of columns of pillar DTa (S52).

[0123] Furthermore, if the control unit 105e determines that all processing for the current data block combination has been completed, it may, for example, issue a FLUSH instruction to delete the pillar DTa in row DTy and column DTx of the data cycle FIFO memory M for row Fy1 and row Fx1.

[0124] As a result, if it is determined that the variable DTx+2 has not reached the value corresponding to the number of columns of pillar DTa (NO in S52), the control unit 105e sets the variable Fx to a value obtained by adding 1 to the remainder of variable Fx multiplied by 3, as shown in Figure 15 (S61).

[0125] Then, the control unit 105e sets the variable DTx to DTx+1, for example (S62). After that, the control unit 105e repeats the processing from S33 onwards, for example.

[0126] Furthermore, if it is determined that the variable DTx+2 has reached a value corresponding to the number of columns of pillar DTa (YES in S52), the control unit 105e determines, for example, whether the variable DTy+2 has reached a value corresponding to the number of columns of pillar DTa (S53).

[0127] As a result, if it is determined that the variable DTy+2 has not reached the value corresponding to the number of columns of pillar DTa (NO in S53), the control unit 105e sets the variable Fy to a value obtained by adding 1 to the remainder of variable Fy by 3, as shown in Figure 15 (S63).

[0128] Then, the control unit 105e sets the variable DTy to DTy+1, for example (S64). After that, the control unit 105e repeats the processing from S62 onwards, for example.

[0129] On the other hand, if the control unit 105e determines that the variable DTy+2 has reached a value corresponding to the number of columns in pillar DTa (YES in S53), the control unit 105e terminates the reading process, for example.

[0130] Thus, the data-circulating FIFO memory M in this embodiment has a plurality of data-circulating FIFO memories arranged logically in an array. Each of the plurality of data-circulating FIFO memories has a dual-port memory F1 having a plurality of entries for storing data, a first port for outputting data to be read, and a second port for receiving input of data to be written, and a control unit (such as a register F2 or arithmetic unit F3) for controlling the read and write operations in the dual-port memory F1. The control unit stores the number of entries in an entry block consisting of one or more consecutive entries, and a stream-in pointer indicating the starting address corresponding to the entry block. When a read operation is performed on data stored in a specific entry block, starting with the entry indicated by the starting address and corresponding to that number of entries, the control unit performs a first control operation in which it reads one or more data stored in the specific entry block in a circular fashion without deleting the data that has been read.

[0131] In other words, in the data-circulating FIFO memory M of this embodiment, for example, at least some of the entries constituting the data-circulating FIFO memory function as a ring buffer, and data that has been read is not deleted. Then, in the data-circulating FIFO memory M, for example, it becomes possible to repeatedly (multiple times) read the data stored in the entries that function as a ring buffer. In other words, in the data-circulating FIFO memory M, for example, it becomes possible to reuse the data stored in the entries that function as a ring buffer.

[0132] As a result, the data-circulating FIFO memory M in this embodiment can suppress the frequency of access to external memory, such as memory 102. Therefore, the data-circulating FIFO memory M can suppress the occurrence of processing delays in convolution operations, for example, and can also suppress power consumption.

[0133] Furthermore, in the data-circulating FIFO memory M of this embodiment, the number of entries that function as a ring buffer can be appropriately changed even when the size of the input data and the size of the filters in each layer constituting the CNN differ, for example, by changing the number of entries that function as a ring buffer by an instruction issued by the control unit 105e.

[0134] Furthermore, in the tensor processor 105 of this embodiment, for example, multiple adjacent pillars DTa in the row direction are stored in different data-circulating FIFO memories M, and multiple adjacent pillars DTa in the column direction are stored in different data-circulating FIFO memories M. Therefore, in the data-circulating FIFO memory M of this embodiment, it becomes possible to perform not only access to a single pillar DTa to be accessed, but also simultaneous access to the pillar DTa to be accessed and the data stored in the halo area of ​​the pillar DTa to be accessed. [Explanation of symbols]

[0135] 1: Information Processing Device 2: Worker terminal 101:CPU 102: Memory 103: Communication Interface 104:Storage medium 105: Tensor Processor 105a: Tensor memory 105b: Arithmetic Unit 105c: Instruction memory 105d: DMAC 105e: Control Unit 106: Bus M: Data-circulating FIFO memory

Claims

1. It has multiple data-circulating FIFO memories arranged logically in an array, Each of the aforementioned plurality of data circular FIFO memories is, A dual-port memory having multiple entries for storing data, a first port for outputting the data to be read, and a second port for receiving input of the data to be written, The system includes a control unit that controls read and write operations in the dual-port memory, The control unit, The system stores the number of entries in an entry block consisting of one or more consecutive entries, and a stream-in pointer indicating the starting address corresponding to the entry block. When the read operation is performed on the data stored in a specific entry block corresponding to the number of entries, with the entry indicated by the aforementioned starting address as the first entry, a first control is performed to perform the read operation while circulating through one or more of the data stored in the specific entry block without deleting the data that has been read. A tensor memory characterized by the following features.

2. The control unit, The read operation stores a read pointer indicating the address of the next entry to be performed. When the read operation is performed on the plurality of entries, the read operation is performed on the data stored in the entry pointed to by the read pointer among the plurality of entries. The tensor memory according to feature 1.

3. The control unit, The write process stores a write pointer indicating the address of the next entry to be performed. When the write operation is performed on the plurality of entries, the write operation of the data is performed on the entry indicated by the write pointer among the plurality of entries. The tensor memory according to feature 1.

4. Each of the aforementioned plurality of data-circulating FIFO memories stores a plurality of pillars generated by partitioning three-dimensional tensor data such that adjacent pillars in the row direction are stored in different data-circulating FIFO memories, and adjacent pillars in the column direction are stored in different data-circulating FIFO memories. The tensor memory according to feature 1.

5. Multiple data-circulating FIFO memories are logically arranged in an array and store multiple pillars generated by dividing three-dimensional tensor data, The system includes a control unit that controls the writing of the plurality of pillars to the plurality of data-circulating FIFO memories and the reading of the plurality of pillars from the plurality of data-circulating FIFO memories. The control unit writes the multiple pillars to each of the multiple data-circulating FIFO memories such that the multiple pillars adjacent in the row direction are written to different data-circulating FIFO memories, and the multiple pillars adjacent in the column direction are written to different data-circulating FIFO memories. A tensor processor characterized by the following features.

6. The control unit reads in parallel the plurality of pillars stored in the plurality of adjacent data circulation FIFO memories in at least one of the row direction and column direction. The tensor processor according to feature 5.

Images