Vector processor based on storage and computing integrated memory and operation method thereof

Abstract

The present invention relates to a vector processor based on a storage-computation integrated memory, comprising: a multiplication storage-computation integrated module, which is configured to calculate the product of a first input vector X and a first input matrix W, and the product is a first output vector Y'; and an addition module, which is configured to add the first input vector X or the first output vector Y' to a second input vector B. The present invention also relates to an operation method of such a vector processor. Through the present invention, addition and multiplication as well as multiplication-addition operations can be completed within the processor chip, thereby significantly expanding the application scenarios of the vector processor and reducing the hardware and software costs required for performing the above-mentioned calculations. In addition, by integrating the addition operation into the vector memory, the present invention makes the vector memory of the present invention better suitable for vector space operations, for example, it can fully perform the multiplication-addition operations in neural network calculations, thereby greatly improving the operation speed and efficiency.

Description

Technical Field

[0001] The present invention generally relates to the field of integrated circuits, and more particularly to a vector processor based on a memory that stores and computes data in one. In addition, the present invention also relates to a method for operating such a vector processor. Background Art

[0002] In recent years, vector processors have developed rapidly. Vector processors are processors used to process vector and matrix operations. Currently, vector processors have low efficiency, high latency and high hardware cost when processing vector-matrix multiplication and vector addition, which limits their application scenarios. Summary of the invention

[0003] The task of the present invention is to provide a vector processor based on a storage-computation integrated memory and an operating method thereof, through which addition, multiplication and multiplication-addition operations can be completed within the processor chip, thereby significantly expanding the application scenarios of the vector processor and reducing the hardware and software costs required to perform the above-mentioned calculations.

[0004] In a first aspect of the present invention, the task is solved by a vector processor based on a storage-computation-in-one memory, the memory comprising:

[0005] a multiplication storage and calculation module configured to calculate the product of the first input vector X and the first input matrix W, wherein the product is the first output vector Y'; and

[0006] An adding module is configured to add the first input vector X or the first output vector Y' to the second input vector B.

[0007] In the present invention, the term "vector" or "vector" refers to an array with n components or elements, and can also be represented by a 1xn matrix, where n is a natural number. For example, each element or component of a vector can be stored in a storage unit respectively, and a single vector is stored together by multiple storage units. In the present invention, the term "matrix" can be regarded as an array containing nxm elements, where m is a natural number. The multiplication of vectors and matrices is calculated according to the matrix multiplication rule. For example, the product of an n-dimensional vector (i.e., a 1xn matrix) and an nxm matrix is ​​an n-dimensional vector, i.e., an nx1 matrix. In the present invention, unless otherwise specified, vectors or matrices that can be multiplied or added meet the corresponding dimensional requirements.

[0008] In a preferred embodiment of the present invention, the vector processor further comprises a storage controller, wherein the storage controller is configured to perform one or more of the following actions:

[0009] The multiplication storage module is enabled and the addition module is disabled to enter the multiplication mode, wherein in the product mode, the product Y' of the first input vector X and the first input matrix W is calculated:

[0010] Y' = XW;

[0011] The multiplication storage and calculation module is enabled and the addition module is enabled to enter the multiplication-addition mode, wherein in the multiplication-addition mode, the sum Y of the product and the second input vector B is calculated:

[0012] Y = XW + B; and

[0013] Disable the multiplication storage module and enable the addition module to enter the addition mode, wherein in the addition mode, the sum Y" of the first input vector X and the second input vector B is calculated:

[0014] Y" = X + B.

[0015] Through this preferred solution, various operations can be implemented, thereby expanding the application field of the memory.

[0016] In another preferred embodiment of the present invention, it is provided that the addition module and the multiplication storage and calculation module are integrated together in the vector processor and communicate directly with the controller. Through this preferred embodiment, the intermediate communication link between the controller and the calculation module or the necessity of off-chip communication can be omitted, thereby saving hardware and software costs. In some scenarios, additional components, such as a data path module, can also be arranged between the addition module and the multiplication storage and calculation module and the storage controller, and the data path module is configured to switch the data input and output paths according to the operation type to achieve the required operation type. For example, in the multiplication-addition mode, the product of the vector and the matrix and the second input vector are input into the addition module; in the addition mode, the two input vectors are input into the addition module; in the multiplication mode, the vector and the matrix are input into the multiplication storage and calculation module.

[0017] In another preferred embodiment of the present invention, it is provided that the multiplication storage and calculation integrated module comprises:

[0018] an array of processing-in-memory (PIM) blocks in rows and columns, each processing-in-memory block being configurable to be in either a compute mode or a memory mode;

[0019] a controller configured to divide the array of PIM blocks into: a first group of PIM blocks, each of which is configured to be in a memory mode; and a second group of PIM blocks, each of which is configured to be in a computation mode, wherein the first group of PIM blocks is configured to store a first input vector X, and the second group of PIM blocks is configured to store a first input matrix W and compute a partial sum of a third vector based on the first input vector X and the second matrix; and

[0020] An accumulator is configured to output the product based on the partial sum of the third matrix.

[0021] Through the preferred solution, efficient in-memory multiplication calculation can be achieved.

[0022] In an extended solution of the present invention, it is provided that the addition module includes a plurality of adders to perform the addition of the corresponding elements of the first input vector X or the product and the corresponding elements of the second input vector B in parallel. Through this extended solution, parallel addition operations can be implemented, thereby improving the operation speed. In some cases, a single adder can also be provided to perform serial addition operations. The adder is preferably implemented as a CMOS circuit so that it can be integrated in the memory.

[0023] In another embodiment of the present invention, it is provided that the vector processor further comprises:

[0024] A register having a data communication connection with the multiplication storage and calculation module and / or the addition module, and configured as follows:

[0025] In the multiplication mode, the first input vector X and the first input matrix W are stored and transmitted to the multiplication storage and calculation integrated module;

[0026] In the multiplication-addition mode, the first input vector X, the first input matrix W and the second input vector B are stored and transmitted to the multiplication storage-calculation module and the addition module; and

[0027] In the addition mode, the first input vector X and the second input vector B are stored and fed to the addition module.

[0028] In the present invention, by using registers to store input data and / or intermediate calculation results, the need for external storage devices can be eliminated, and the parallel computing capability can also be expanded.

[0029] In a second aspect of the present invention, the aforementioned task is solved by a circuit for neural network calculation, the circuit comprising:

[0030] a plurality of layers, each layer comprising a plurality of neural units; and

[0031] According to the vector processor of the present invention, the vector processor is configured to receive input data X0 from a neural unit of a first layer and calculate output data Y as input data of a neural unit of a second layer adjacent to the first layer.

[0032] Here, the input data X0 of the first layer can be the calculation result of the vector processor in the first layer, such as the sum of the product of a vector and a matrix and another vector. It should be noted that the sum is a column vector, but since both row vectors and column vectors can be stored in each storage unit in the memory, the column vector and the row vector are equivalent in the memory and no transposition operation is required. In a typical scenario, the input data X0 is the calculation result of the previous layer of the neural network. After being input to this layer, it is multiplied with the weight matrix of each node (matrix W or a submatrix of matrix W) and then summed. Therefore, the vector processor can fully perform the above operations on the neural network without the need for off-chip software and hardware.

[0033] In a third aspect of the invention, the aforementioned object is solved by a method for operating a vector processor according to the invention, the method comprising the following steps:

[0034] The storage controller enables the integrated multiplication storage module and disables the addition module to enter the multiplication mode, wherein in the product mode, the integrated multiplication storage module calculates the product Y' of the first input vector X and the first input matrix W:

[0035] Y' = XW;

[0036] The storage controller enables the multiplication storage and calculation module and the addition module to enter the multiplication-addition mode, wherein in the multiplication-addition mode, the multiplication storage and calculation module calculates the product of the first input vector X and the first input matrix W, and the addition module calculates the sum Y of the product and the second input vector B:

[0037] Y = XW + B; and

[0038] The storage controller disables the multiplication storage and calculation module and enables the addition module to enter the addition mode, wherein in the addition mode, the addition module calculates the sum Y" of the first input vector X and the second input vector B:

[0039] Y" = X + B.

[0040] In one embodiment of the present invention, it is provided that the method further comprises the following steps:

[0041] The register stores the first input vector X and the first input matrix W in the multiplication mode and transmits them to the multiplication storage and calculation integrated module;

[0042] The register stores the first input vector X, the first input matrix W and the second input vector B in a multiplication-addition mode and transmits them to the multiplication-storage-computation-in-one module and the addition module; and

[0043] The first input vector X and the second input vector B are stored in the register in the addition mode and supplied to the addition module.

[0044] In a fourth aspect of the present invention, the aforementioned task is achieved by a method for data processing in a neural network, comprising:

[0045] Receive a first input vector X from a first layer of a neural network;

[0046] Input the first input vector X into the multiplication storage module, wherein the weight matrix W is stored in the multiplication storage module;

[0047] The multiplication storage module calculates the product of the first input vector X and the weight matrix W, and the product is the first output vector Y'; and

[0048] Input the first output vector Y' and the bias vector B into the addition module;

[0049] The sum Y of the first output vector Y' and the bias vector B is calculated by the addition module; and

[0050] The sum Y is input to the second layer of the neural network adjacent to the first layer.

[0051] It should be noted here that the weight matrix W may include one or more sub-matrices to store the weighting matrices of one or more neural nodes.

[0052] In one embodiment of the present invention, it is provided that the method further comprises the following steps:

[0053] The first input vector X and the bias vector B are stored in registers.

[0054] In a preferred embodiment of the present invention, it is provided that the weight matrix W includes a plurality of sub-matrices W1 . . . W n , wherein n is a natural number, the method further comprises the following steps:

[0055] The multiplication storage module calculates the product of the first input vector X and the submatrix W1, and the product is the first output component vector Y1;

[0056] The multiplication storage module calculates the product of the first input vector X and the submatrix W2, and the product is the second output component vector Y2;

[0057] Input the first output vector Y1 and the second component vector Y2 into the addition module;

[0058] The addition module calculates the sum Y of the first output vector Y1 and the second component vector Y2 as the bias vector B;

[0059] The multiplication storage module calculates the product of the first input vector X and the submatrix W3, and the product is the third output component vector Y3;

[0060] Input the third output component vector Y3 and the bias vector B into the adding module; and

[0061] The addition module calculates the sum Y of the third output component vector Y3 and the bias vector B and updates the bias vector B.

[0062] Through this preferred embodiment, the input vector can be multiplied by the sub-weight matrix of each neural node and then accumulated to calculate the total, thereby realizing the main calculation of the neural network in the memory and reducing its software and hardware costs.

[0063] The present invention realizes both multiplication and addition operations within the vector processor chip, thereby greatly reducing the software and hardware costs and time costs of off-chip addition operations; in addition, the present invention integrates addition operations into the vector memory, so that the vector memory of the present invention can be better applied to vector space operations, for example, it can fully execute multiplication and addition operations in neural network calculations, thereby greatly improving the operation speed and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The present invention will be further described below in conjunction with specific embodiments with reference to the accompanying drawings.

[0065] Figure 1 A schematic diagram of a vector processor according to the present invention is shown;

[0066] Figure 2 A schematic diagram showing an embodiment of a multiplication storage and calculation integrated module of a vector processor according to the present invention is shown;

[0067] Figure 3 An example of application of the vector processor according to the present invention in neural network calculation is shown;

[0068] Figure 4 An embodiment of a PIM block of a multiplication storage and calculation integrated module of a vector processor according to the present invention is shown; and

[0069] Figure 5A-5B A schematic diagram of a PIM block in a memory mode and a calculation mode of a multiplication memory-calculation integrated module of a vector processor according to the present invention is shown. DETAILED DESCRIPTION

[0070] It should be noted that the components in the figures may be shown exaggeratedly for the sake of illustration and are not necessarily true to scale. In the figures, identical or functionally identical components are provided with the same reference numerals.

[0071] In the present invention, unless otherwise specified, "arranged on...", "arranged above..." and "arranged above..." do not exclude the existence of an intermediate between the two. In addition, "arranged on or above..." merely indicates the relative positional relationship between two components, and in certain cases, such as after reversing the product direction, it can also be converted into "arranged below or below...", and vice versa.

[0072] In the present invention, each embodiment is only intended to illustrate the aspects of the present invention and should not be construed as limiting.

[0073] In the present invention, unless otherwise specified, the quantifiers "a", "an" and "an" do not exclude the presence of a plurality of elements.

[0074] It should also be noted that in the embodiments of the present invention, for the sake of clarity and simplicity, only a portion of the parts or components may be shown, but those of ordinary skill in the art will understand that under the teachings of the present invention, the required parts or components may be added according to the needs of the specific scenario. In addition, unless otherwise specified, the features in different embodiments of the present invention may be combined with each other. For example, a feature in the second embodiment may be used to replace a corresponding or functionally identical or similar feature in the first embodiment, and the resulting embodiment also falls within the disclosure scope or recorded scope of the present application.

[0075] It should also be noted that within the scope of the present invention, the terms "same", "equal", "equal to" and the like do not mean that the values ​​of the two are absolutely equal, but allow a certain reasonable error, that is, the terms also cover "substantially the same", "substantially equal", "substantially equal to". By analogy, in the present invention, the terms "perpendicular to", "parallel to" and the like indicating directions also cover the meanings of "substantially perpendicular to" and "substantially parallel to".

[0076] In addition, the numbering of the steps of the methods of the present invention does not limit the execution order of the method steps. Unless otherwise specified, the method steps can be executed in different orders.

[0077] The present invention is based on the following insight of the inventors: existing vector processors cannot perform addition operations within the vector processor, and must use dedicated hardware outside the memory to perform vector and matrix multiplication operations, which results in the addition operation having to be performed outside the vector processor, which increases the software and hardware costs of data communication and processing. At the same time, the use of hardware circuits dedicated to vector multiplication independent of the memory in the vector processor also increases power consumption and chip area; in order to at least partially solve the above problems, the present invention integrates the addition operation into the vector processor and performs the vector multiplication operation in the storage-computation integrated memory, on the one hand, eliminating the circuit dedicated to vector multiplication operation in the vector processor, and performing the multiplication operation in the memory; on the other hand, the addition operation can be performed in the vector memory without performing off-chip addition operation; furthermore, the vector memory of the present invention can be well applied to vector space operations, for example, it can fully perform multiplication-addition operations in neural network calculations (i.e., a vector is multiplied by a matrix and then added to another vector), thereby greatly improving the operation speed and efficiency.

[0078] The present invention will be further described below in conjunction with specific embodiments with reference to the accompanying drawings.

[0079] Figure 1 FIG. 1 is a schematic diagram of a vector processor 100 according to the present invention.

[0080] like Figure 1 As shown, the vector processor 100 according to the present invention has the following components (the dashed boxes represent optional components):

[0081] The multiplication storage module 101 is configured to calculate the product of the first input vector X and the first input matrix W, and the product is the first output vector Y'. The multiplication storage module 101 can be implemented based on non-volatile memory (RRAM, MRAM, PCM, SRAM, etc.). As an example, the multiplication storage module 101 can include a plurality of PIM (Processing In Memory) blocks, and the PIM blocks are used to store vectors and matrices and multiply them. For further details about the multiplication storage module 101, please refer to Figure 2 .

[0082] An addition module 102, which is configured to add the first input vector X or the first output vector Y' to the second input vector B. The addition module 102 can be, for example, a CMOS adder (faster but occupies a larger area within the chip) or implemented based on an in-memory logic unit (slower but occupies a smaller area within the chip), and the addition module 102 and the multiplication storage and calculation module 101 are integrated together in the vector processor 100 and communicate directly with the controller 105. Here, the term "integrated" means that it is arranged in the form of an integrated circuit within a carrier chip. Here, the term "direct communication" means that the communication is performed within the chip (i.e., the memory 100) without going through an off-chip communication path.

[0083] An optional data path module 104, which is configured to provide a data transmission / communication path between the modules 101, 102, 103, and the memory 105, so as to deliver the corresponding data to the corresponding module. The data path module 104 may include, for example, switchable electrical lines between the corresponding modules, and may optionally include path selection and switching devices such as switches and gates.

[0084] A storage controller 105 configured to perform one or more of the following actions:

[0085] (a) The multiplication storage module 101 is enabled and the addition module 102 is disabled to enter the multiplication mode, wherein in the product mode, the product Y' of the first input vector X and the first input matrix W is calculated:

[0086] Y' = XW;

[0087] (b) enabling the multiplication storage and calculation module 101 and enabling the addition module 102 to enter the multiplication-addition mode, wherein in the multiplication-addition mode, the sum Y of the product and the second input vector B is calculated:

[0088] Y = XW + B; and

[0089] (c) The multiplication storage module 101 is disabled and the addition module 102 is enabled to enter the addition mode, wherein in the addition mode, the sum Y" of the first input vector X and the second input vector B is calculated:

[0090] Y" = X + B.

[0091] An optional register, which has a data communication connection with the multiplication storage and calculation module 101 and / or the addition module 102 to store input data and / or result data and / or intermediate result data, and is configured as follows:

[0092] (a) In multiplication mode, the first input vector X and the first input matrix W are stored and transmitted to the multiplication storage and calculation module (optionally via the data path module 104);

[0093] (b) in the multiplication-addition mode, storing the first input vector X, the first input matrix W, and the second input vector B and transmitting them to the multiplication storage-calculation module and the addition module (optionally via the data path module 104); and

[0094] (c) In the addition mode, the first input vector X and the second input vector B are stored and fed to the addition module (optionally via the data path module 104).

[0095] Figure 2 A schematic diagram of an embodiment of a multiplication storage-computation integrated module 101 of a vector processor according to the present invention is shown.

[0096] like Figure 2 As shown, the multiplication storage and calculation integrated module 101 includes:

[0097] An array of in-memory processing PIM blocks 202 in rows and columns, each in-memory processing block 202 being configurable to be in either a compute mode or a memory mode.

[0098] · A controller 206 configured to divide the PIM block array 202 into: a first group of PIM blocks 202, each of which is configured to be in a memory mode; and a second group of PIM blocks 202, each of which is configured to be in a calculation mode, wherein the first group of PIM blocks 202 is configured to store a first input vector X, and the second group of PIM blocks 202 is configured to store a first input matrix W and calculate a partial sum of a third vector based on the first input vector X and the second matrix. The controller 206 can be integrated with the memory controller 105 or separated from each other. Here, the partial sum refers to a partial calculation result of the multiplication of a vector and a matrix, such as the product of a row vector and a column vector of a matrix. For details about the PIM blocks 202, please refer to Figure 4 .like Figure 4 As shown, each PIM block 202 may include a memory array 402 and a VVM (vector-vector multiplication) engine 404, wherein the VVM engine 404 is configured to be disabled in the memory mode. The VVM engine may include, for example, a bit counter, a shift accumulator, and a plurality of AND gates to perform vector-vector multiplication. For details of the VVM engine, see Figure 5A-5B. In some embodiments, the memory array 402 includes a ReRAM array. It is understood that in other examples, the memory array 402 may include any other suitable memory, including but not limited to phase change random access memory (PRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), to name a few examples. The memory array 402 may store a first vector. The PIM block 202 may also include a control circuit 406, which is configured to enable the VVM engine 404 in a computing mode and control the VVM engine 404 to perform a dot product between the first vector and the second vector to generate a partial sum. The control circuit 406 may also be configured to disable the VVM engine 404 in a memory mode and control the memory array 402 to write or read out the first vector. The PIM block 202 may also include various buffers for intermediate data storage, including: a column buffer 408, which is configured to receive and buffer, for example, a second vector from other PIM devices via a memory bus; and a partial sum buffer 410, which is configured to buffer the partial sum and send the partial sum to another PIM device via a partial sum bus.

[0099] An optional accumulator configured to output the product based on the partial sum of the third matrix. In other cases, the partial sum may also be directly output as the calculation result.

[0100] • A plurality of MUXs 204 configured to redirect data flow between different rows and / or columns of PIM blocks 202.

[0101] A bus 212, which is used for transmission between components such as the PIM block 202, the controller 206, the global function unit 210, and the accumulator located on the bus.

[0102] • A global function unit 210, which is configured to perform any suitable global miscellaneous functions, such as pooling, enabling and encoding schemes, etc.

[0103] Each PIM block 202 may be identical and configured to be in either a memory mode for storing data such as vectors or matrices of two or more dimensions, or a computation mode for storing data and performing vector / matrix computations such as VMM or VVM. As the specific task to be performed changes, such as computations in a convolutional layer or a fully connected (FC) layer in a CNN, each PIM block 202 may be reconfigured between the computation mode and the memory mode based on the computation scheme of the specific task. In some embodiments, even if the layout of the array of PIM blocks 202 is preset, such as in the form of orthogonal rows and columns, the configuration of the MUX 204 may still be flexibly changed according to the specific task to be performed. For example, by enabling and disabling certain MUXs 204 between PIM blocks 202 in different rows, the arrangement of the array of PIM blocks 202 may be configured to adapt to the computation scheme and data flow corresponding to the specific task. According to some embodiments, the enabled MUXs 204 divide the array of PIM blocks 202 into two or more groups, each of which is configured to be in the same computation or memory mode. Furthermore, although the default data flow between PIM blocks 202 is in the same row and / or column, an enabled MUX 204 can further redirect the data flow between different rows and / or columns as required for a particular task.

[0104] The bus 212 may be a main / system bus of the multiplication-computation-in-one module 101, which is used to transfer input data, such as matrices, to the array of PIM blocks 202. A group of PIM blocks 202 in the multiplication-computation-in-one module 101 may be configured to be in memory mode to replace the RAM 102. As a result, according to some embodiments, the data flow is no longer between each PIM block 104 and the centralized RAM 102, but follows a specific path based on the arrangement of the array of PIM blocks 202, such as the layout of the array of PIM blocks 202 and / or the configuration of the MUX 204. The output of the array of PIM blocks 202, such as partial summations (sums), may be sent to the accumulator 208, which may be further configured to generate an output matrix based on the partial sums.

[0105] Figure 5A-5B A schematic diagram of a PIM block in a memory mode and a calculation mode of a multiplication memory-calculation integrated module of a vector processor according to the present invention is shown.

[0106] Figure 5A A detailed block diagram of the PIM block 202 in memory mode is shown according to some embodiments of the present disclosure. Figure 5B FIG. 1 shows a computer in computing mode according to some embodiments of the present disclosure. Figure 4 Detailed block diagram of PIM block 202 in FIG. VVM engine 404 may include a bit counter 502, a shift accumulator 508, and a plurality of AND gates 506. Figure 5A As shown, control circuit 406 may disable VVM engine 404 and portion and buffer 410 (shown in dashed lines) in memory mode, such that PIM block 202 acts as a memory element for storing the first vector in memory array 402. Figure 5B As shown, control circuit 406 can enable VVM engine 404 and partial sum buffer 410 (shown in solid lines) in calculation mode, so that a first vector stored in memory array 402 and a second vector buffered in column buffer 408 can be sent to VVM engine 404 to calculate the dot product of the first and second vectors, which can be buffered as a partial sum in partial sum buffer 410.

[0107] Figure 3 An example of application of the vector processor according to the present invention in neural network calculation is shown.

[0108] like Figure 3 As shown, the neural network 300 includes an input layer 301, a hidden layer 302 and an output layer 303. The neural network 300 can be implemented by software, hardware or a combination thereof. The input layer 301 includes three neural units 301a-301c, the hidden layer 302 includes three neural units 302a-302d, and the output layer 303 includes two neural units 303a-303b. It should be noted that the number of layers and the number of neural units are only schematic, and in other embodiments, there may be different numbers of layers and neural units. In the neural network 300, each neural unit performs weighted summation (multiplication and addition) on multiple output data from multiple neural units in the previous layer, that is, each output data is multiplied by a certain coefficient and then summed. The above-mentioned multiplication and addition operation can be performed by the vector processor 100 according to the present invention arranged between two adjacent layers (or at each neural unit), thereby realizing the main calculation of the neural network in the memory, reducing its software and hardware costs. Here, a vector processor 100 is exemplarily arranged between the hidden layer 302 and the output layer 303, so that the corresponding vector of each neural unit 302a-302b of the hidden layer 302 is multiplied by the corresponding weighting matrix at each neural unit of the output layer 303 and then summed to obtain an output vector, which is then transmitted to the neural units 303a-303b of the output layer 303.

[0109] The present invention has at least the following beneficial effects:

[0110] The circuit is relatively simple, and the design area is not significantly increased compared to the traditional one.

[0111] Since the two different computing modules are independent of each other, pipeline buffers of different stages can be added in between to improve the overall throughput of the (single-core) chip.

[0112] The entire neural network acceleration task can be completed in the memory according to the present invention, without the need for off-chip computing resources to complete the non-matrix multiplication part of the neural network, thereby greatly improving the computing speed and efficiency of the neural network.

[0113] Vectors can be used as basic operation units to make software programming more convenient.

[0114] When designing specialized chips for neural networks and machine learning, vector addition often takes up a large amount of computation. Directly utilizing the vector addition integrated on the same chip can greatly improve the spatial continuity of execution.

[0115] Use vectors as the unit of operation to reduce possible loops in the software: use built-in functions to process data instead of using constructs such as loops.

[0116] Although some embodiments of the present invention have been described in this application document, it will be appreciated by those skilled in the art that these embodiments are merely shown as examples. Those skilled in the art may conceive of numerous variations, alternatives, and improvements under the teachings of the present invention without departing from the scope of the present invention. The appended claims are intended to define the scope of the present invention and thus cover methods and structures within the scope of these claims themselves and their equivalents.

Claims

1. A vector processor based on a storage-computation integrated memory, comprising: A multiplication storage module is configured to calculate the product of the first input vector X and the first input matrix W, wherein the product is a first output vector Y', wherein the multiplication storage module includes: an array of in-memory processing PIM blocks in rows and columns, each in-memory processing PIM block being configured to be in a computation mode or a memory mode; a controller configured to divide the PIM block array into: a first group of PIM blocks, each of which is configured to be in a memory mode; and a second group of PIM blocks, each of which is configured to be in a computation mode, wherein the first group of PIM blocks is configured to store a first input vector X, and the second group of PIM blocks is configured to store a first input matrix W and compute a partial sum of a third vector based on the first input vector X and the second matrix, wherein each PIM block includes a memory array and a vector-vector multiplication engine, the vector-vector multiplication engine including a bit counter, a shift accumulator, and a plurality of AND gates; an accumulator configured to output the product based on the partial sum of the third matrix; and a plurality of MUXs configured to redirect data flows between different rows and / or columns of PIM blocks; an adding module configured to add the first input vector X or the first output vector Y' to the second input vector B; A storage controller configured to perform one or more of the following actions: The multiplication storage module is enabled and the addition module is disabled to enter the multiplication mode, wherein in the product mode, the product Y' of the first input vector X and the first input matrix W is calculated: Y'=XW; The multiplication storage and calculation module is enabled and the addition module is enabled to enter the multiplication-addition mode, wherein in the multiplication-addition mode, the sum Y of the product and the second input vector B is calculated: Y = XW + B; and The multiplication storage module is disabled and the addition module is enabled to enter the addition mode, wherein in the addition mode, the sum Y'' of the first input vector X and the second input vector B is calculated: Y''=X+B. 2 . The vector processor according to claim 1 , wherein the addition module and the multiplication storage-calculation module are integrated together in the vector processor and communicate directly with the controller. 3 . The vector processor according to claim 1 , wherein the addition module comprises a plurality of adders to perform addition of corresponding elements of the first input vector X or the product and corresponding elements of the second input vector B in parallel.

4. The vector processor according to claim 1, further comprising: A register having a data communication connection with the multiplication storage and calculation module and / or the addition module, and configured as follows: In the multiplication mode, the first input vector X and the first input matrix W are stored and transmitted to the multiplication storage and calculation integrated module; In the multiplication-addition mode, the first input vector X, the first input matrix W and the second input vector B are stored and transmitted to the multiplication storage-calculation integrated module and the addition module; as well as In the addition mode, the first input vector X and the second input vector B are stored and fed to the addition module.

5. A circuit for neural network computing, comprising: Multiple layers, each layer includes multiple neural units; as well as The vector processor according to any one of claims 1 to 4, the vector processor being configured to receive input data X0 from a neural unit of a first layer and calculate output data Y as input data of a neural unit of a second layer adjacent to the first layer.

6. A vector processor operation method, used to operate the vector processor based on the storage-computation integrated memory according to any one of claims 1 to 4, the method comprising the following steps: The storage controller enables the integrated multiplication storage module and disables the addition module to enter the multiplication mode, wherein in the product mode, the integrated multiplication storage module calculates the product Y' of the first input vector X and the first input matrix W: Y'=XW; The storage controller enables the multiplication storage and calculation module and the addition module to enter the multiplication-addition mode, wherein in the multiplication-addition mode, the multiplication storage and calculation module calculates the product of the first input vector X and the first input matrix W, and the addition module calculates the sum Y of the product and the second input vector B: Y = XW + B; and The storage controller disables the multiplication storage and calculation module and enables the addition module to enter the addition mode, wherein in the addition mode, the addition module calculates the sum Y'' of the first input vector X and the second input vector B: Y''=X+B.

7. The method according to claim 6, further comprising the steps of: The register stores the first input vector X and the first input matrix W in the multiplication mode and transmits them to the multiplication storage and calculation integrated module; The register stores the first input vector X, the first input matrix W and the second input vector B in a multiplication-addition mode and transmits them to the multiplication-storage-computation-in-one module and the addition module; and The first input vector X and the second input vector B are stored in the register in the addition mode and supplied to the addition module.

8. A method for data processing in a neural network, comprising: Receive a first input vector X from a first layer of a neural network; Input the first input vector X into the multiplication storage module, wherein the weight matrix W is stored in the multiplication storage module; The multiplication storage module calculates the product of the first input vector X and the weight matrix W, and the product is the first output vector Y'; as well as Input the first output vector Y' and the bias vector B into the addition module; The sum Y of the first output vector Y' and the bias vector B is calculated by the addition module; as well as Inputting the sum Y into a second layer of the neural network adjacent to the first layer; The weight matrix W includes a plurality of sub-matrices W1…W n , wherein n is a natural number, the method further comprises the following steps: The multiplication storage module calculates the product of the first input vector X and the submatrix W1, and the product is the first output component vector Y1; The multiplication storage module calculates the product of the first input vector X and the submatrix W2, and the product is the second output component vector Y2; Input the first output vector Y1 and the second component vector Y2 into the addition module; The addition module calculates the sum Y of the first output vector Y1 and the second component vector Y2 as the bias vector B; The multiplication storage module calculates the product of the first input vector X and the submatrix W3, and the product is the third output component vector Y3; Input the third output component vector Y3 and the bias vector B into the addition module; as well as The addition module calculates the sum Y of the third output component vector Y3 and the bias vector B and updates the bias vector B; A vector processor is arranged between the hidden layer and the output layer, so that the corresponding vector of each neural unit in the hidden layer is multiplied by the corresponding weight matrix at each neural unit in the output layer and then summed to obtain the output vector, which is then transmitted to the neural unit of the output layer 303.

9. The method according to claim 8, further comprising the steps of: The first input vector X and the bias vector B are stored in registers.

Images