An in-memory computing architecture and acceleration method for deploying deep learning networks

Abstract

The present invention relates to an in-memory computing architecture and method for deploying a deep learning network, and belongs to the field of in-memory computing of a DNN accelerator. The in-memory computing architecture includes an input memory, an inner product operation unit, an in-memory computing unit, and a control module; the control module is used to control the in-memory computing architecture to complete the computing task based on the instruction information of the front module; the input memory is used to store the weights and activation information of the DNN network involved in the operation; the inner product operation unit is used to combine the corresponding weights and activation information to obtain a partial sum; the in-memory computing unit is used to perform accumulation operations on the partial sums to obtain the final calculation result. The in-memory computing unit is a combination of the output memory and the in-memory accumulation. After the inner product operation unit generates the corresponding partial sum, the accumulation operation is completed by the in-memory computing unit to realize the reuse of the accelerator output, thereby solving the memory wall problem existing at the output end of the DNN accelerator.

Description

Technical Field

[0001] The present invention belongs to the field of in-memory computing, and in particular relates to an in-memory computing architecture and acceleration method for deploying deep learning networks. Background Art

[0002] In recent years, due to the rapid development of deep learning networks (DNNs), traditional processors such as CPUs and GPUs are unable to meet their computing needs, which has promoted the development of AI accelerators. DNNs perform multiplication and addition operations on multiple activation and weight values ​​of input to generate corresponding outputs. AI accelerators use less overhead than traditional computing architectures to achieve the same scale of operations.

[0003] One of the most important features of DNN is its full reusability, which is reflected in the three elements involved in DNN calculations: activation, weight, and output. Weights are reused when the same network is used to process different models; activations are reused when the same feature map needs to be calculated simultaneously through multiple networks, or when an activation is calculated with multiple different kernels to obtain the corresponding values ​​of different channels in the next layer; and due to limited hardware resources, the output of the same location must be calculated multiple times before it can be obtained, in which case the output needs to be reused.

[0004] The special data reuse of DNN operations creates a memory wall in traditional computing architectures. Therefore, various accelerators attempt to reuse data in hardware to save read and write operations at all levels of storage, thereby improving the speed and energy efficiency of DNN computing.

[0005] In the existing technology, in-memory computing is designed to solve the memory wall problem by placing the computing unit directly inside the storage module to solve the bottleneck caused by access. This implementation generally reuses the weight or activation of the input, and increases the computing speed and reduces power consumption by reducing the number of refreshes on one side of the input. Due to the different sizes of DNN models, for the sake of universality, it is usually necessary to weigh the size of the memory that needs to be saved: when the in-memory computing unit that utilizes input reuse is too small or the storage level is too high, frequent reading and writing will be introduced, resulting in additional overhead, making the in-memory computing meaningless; when the in-memory computing unit that utilizes input reuse is designed to be too large, it will lead to insufficient deployment of the model, and as long as data reuse is implemented, the tiling problem will be involved, which will result in an additional access operation when the calculation results need to be accumulated, that is, the accumulation of the partial sum obtained by the inner product operation must first be stored in the memory, and then the inner product calculation unit is used to perform the "data acquisition → calculation → writing" process. This process involves at least two addressing operations and is almost completely non-parallel. In the addressing process, not only power consumption is wasted again, but also cycles are wasted. This operation creates a real bottleneck. Summary of the Invention

[0006] In view of the above analysis, the present invention aims to provide an in-memory computing architecture and acceleration method for deploying deep learning networks. An in-memory computing unit is added after the inner product operation unit. The in-memory computing unit is a combination of the output memory and the in-memory accumulation. The accumulation operation is placed in the storage cell. After the inner product operation unit generates the corresponding partial sum, the accumulation operation is completed through the in-memory computing unit to realize the reuse of the accelerator output, thereby achieving an increase in computing speed and a reduction in computing power consumption, and solving the memory wall problem existing at the output end of the DNN accelerator.

[0007] In one aspect, the present invention provides an in-memory computing architecture for deploying a deep learning network, specifically comprising an input memory, an inner product operation unit, an in-memory computing unit, and a control module;

[0008] The control module is used to obtain instruction information and obtain read address information, control information and control instruction group corresponding to the weight and activation data based on the instruction information;

[0009] The input memory is used to store the weights and activation data of the DNN network based on the write address information, and output the corresponding weights and activation data of the DNN network to the inner product operation unit based on the read address information input by the control module;

[0010] The inner product operation unit is used to combine the corresponding weights and activation data based on the control information and the weights and activation data of the DNN network to obtain corresponding partial sums;

[0011] The in-memory computing unit is used to concurrently perform accumulation operations based on the control instruction group and the partial sum to obtain a final operation result.

[0012] Furthermore, the cumulative operation is performed concurrently based on the control instruction group and the partial sum to obtain the final operation result, including:

[0013] The in-memory computing unit obtains concurrent computing tasks based on the partial sum and the control instruction group;

[0014] And for each computing task, complete the accumulation operation and activation processing of its partial sum to obtain the output result corresponding to the task;

[0015] The output results of the in-memory computing architecture are obtained based on the output results of all tasks.

[0016] Furthermore, the in-memory computing unit includes multiple blocks, each of which is used to complete one computing task, and the number of the multiple blocks is determined by the maximum number of concurrent tasks.

[0017] Furthermore, the Block includes a plurality of Cells, each Cell including a storage accumulation part and an output part; the storage accumulation part is used to store input data and perform accumulation operations, and the output part is used to activate the result of the accumulation operation to obtain the output result of the Cell;

[0018] The storage accumulation part includes a state decoder, a pre-storage memory, a result memory, an adder, and a first data selector; wherein,

[0019] The state decoder is used to determine the operation type of the pre-storage memory and the result memory based on the input data, and to obtain the first input data of the adder based on the input data;

[0020] The pre-storage memory is used to output the content in the pre-storage memory to the adder as second input data of the adder based on the operation type;

[0021] The adder is used to perform an addition operation on the first input data and the second input data;

[0022] The first data selector is used for selecting, based on a reset flag, to set the result memory to 0 or to store the output of the adder into the result memory;

[0023] The result memory is used to output the content in the result memory to the pre-storage memory or the output part based on the operation type.

[0024] Furthermore, the output of the in-memory computing architecture includes the output of the in-memory computing unit, the output of the in-memory computing unit is obtained by splicing the output data of all blocks participating in the calculation, and the output data of the block is obtained by splicing the output data of all cells participating in the calculation in the block.

[0025] Furthermore, the output portion of the in-memory computing unit includes an activation unit, which includes multiple activation functions, a custom polynomial, and a second data selector, wherein:

[0026] Multiple activation functions and custom polynomials in the activation unit are used to perform corresponding activation operations on the input data respectively;

[0027] The second selector is used to select one of the activation operation results as the output result of the Cell based on activation selection.

[0028] Furthermore, the input of the in-memory computing unit is based on the control instruction group and part thereof and is obtained through the BUS bus, including read and write status, Block instruction group, reset flag and custom activation configuration; the Block instruction group includes multiple Block instructions, and the Block instructions include activation selection, input data group, and data address group; wherein,

[0029] The reset flag is used to indicate the reset state of the calculation;

[0030] The custom activation configuration is used to customize the polynomial to determine the coefficients;

[0031] The activation selection is input into the activation unit;

[0032] The read and write status, input data group and data address group are input to the status decoder.

[0033] Furthermore, the Block instruction also includes a Block address, and the Block address is used to determine a valid Block involved in the calculation.

[0034] On the other hand, the present invention also provides an in-memory computing acceleration method for deploying a deep learning network, which specifically includes the following steps:

[0035] Step S1: The control module obtains address information, control information, and a control instruction group corresponding to weights and activation data based on the instruction information; the input memory stores the weights and activation data of the DNN network based on the write address information;

[0036] Step S2: The input memory outputs the corresponding DNN network weights and activation data to the inner product operation unit based on the read address information;

[0037] Step S3: The inner product operation unit combines the corresponding weights and activation data based on the control information and the weights and activation data of the DNN network to obtain corresponding partial sums;

[0038] Step S4: The in-memory computing unit concurrently performs accumulation operations based on the control instruction group and the partial sum to obtain the output result of the in-memory computing architecture.

[0039] Furthermore, the in-memory computing unit concurrently performs accumulation operations based on the control instruction group and the partial sum, and calculates output results of the in-memory computing architecture including:

[0040] Step S41: Determine the Block involved in the calculation based on the Block instruction group of the input data;

[0041] Step S42: Determine the Cell involved in the calculation based on the input data group and data address group in the Block instruction;

[0042] Step S43: Determine the operation type of each cell in the in-memory computing unit based on the read / write status;

[0043] Step S44: Based on the operation type, the Cell performs corresponding accumulation operation on the input data;

[0044] Step S45: Based on the activation selection, the Cell performs activation operation on the accumulated operation result to obtain the output result of the Cell.

[0045] Step S46: Based on the output results of each Cell, obtain the results of the in-memory calculation.

[0046] The present invention can achieve at least one of the following beneficial effects:

[0047] On the one hand, after the inner product operation unit in the in-memory computing architecture generates the corresponding partial sum, the controller combines the partial sum as the input data group and the data address group and inputs it into the in-memory computing unit, realizing the input memory for addressing calculation and completing the accumulation operation directly through one addressing. The traditional in-memory architecture's "data fetch → calculation → write data" is directly saved into the "write data" process, and the data fetch and calculation are completed by the in-memory computing unit, thereby saving computing time and computing power.

[0048] On the other hand, the in-memory computing architecture realizes the reuse of DNN accelerator output by placing partial sum accumulation operations into the Cell of the in-memory computing unit, eliminating the need for the process of fetching, accumulating, and then storing. This increases the computing speed and reduces computing power consumption, solving the memory wall problem at the output end of the DNN accelerator.

[0049] Other features and advantages of the present invention will be described in the following description, and some advantages may become apparent from the description or be understood through practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like parts throughout the drawings.

[0051] Figure 1 The in-memory computing architecture used by the present invention to deploy deep learning networks;

[0052] Figure 2 Schematic diagram of the structure of the in-memory computing unit of the present invention;

[0053] Figure 3 Schematic diagram of the Cell structure of the in-memory computing unit of the present invention. DETAILED DESCRIPTION

[0054] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0055] Device embodiment

[0056] A specific embodiment of the present invention discloses an in-memory computing architecture for deploying deep learning networks, such as Figure 1 As shown, it includes input memory, inner product operation unit, in-memory computing unit and control module. The in-memory computing architecture is based on Figure 1 The computing task is completed by the computing instructions issued by the front-end module shown in the figure, where the front-end module is a processor unit that inputs data and instructions to the in-memory computing architecture.

[0057] The control module is used to obtain instruction information and obtain address information, control information and control instruction group corresponding to weights and activation data based on the instruction information.

[0058] The input memory is used to store the weights and activation data of the DNN network based on the write address information, and output the corresponding weights and activation data of the DNN network to the inner product operation unit based on the read address information input by the control module;

[0059] The inner product operation unit is used to combine the corresponding weights and activation data based on the control information and the weights and activation data of the DNN network to obtain corresponding partial sums; specifically, the control module inputs the obtained control information into the inner product operation unit, and the inner product operation unit combines the weights and activation data of the DNN network obtained from the input memory based on the control information, obtains corresponding partial sums, and outputs them to the in-memory computing unit.

[0060] The in-memory computing unit is configured to concurrently perform accumulation operations based on the control instruction group and the partial sum to obtain a final operation result, including:

[0061] The in-memory computing unit obtains concurrent computing tasks based on the partial sum and the control instruction group;

[0062] For each computing task, complete the accumulation operation and activation processing of its partial sum to obtain the output result of the corresponding task;

[0063] The output results of the in-memory computing architecture are obtained based on the output results of all tasks.

[0064] Furthermore, the architecture of the in-memory computing unit is as follows Figure 2 As shown: it includes m parallel blocks; each block includes n parallel cells, and each cell includes a storage accumulation part and an output part; the storage accumulation part is used to store input data and perform accumulation operations, and the output part is used to activate the results of the accumulation operations to obtain the output results of the cell.

[0065] One of the blocks is used to complete one of the computing tasks, and the value of m is determined by the maximum number of concurrent tasks.

[0066] The Cell is used to operate on data based on the instructions of the Block in which it is located. The value of n is determined by the size of the output storage space reserved, and the size of the output storage space reserved is determined by the DNN model.

[0067] Specifically, such as Figure 3 As shown, the storage accumulation part of each Cell includes a state decoder, a pre-storage memory, a result memory, an adder, and a first data selector, and the output part of each Cell includes an activation unit, and the activation unit includes multiple activation functions, a custom polynomial and a second data selector.

[0068] The state decoder is used to receive the read / write state, input data group and data address group. Optionally, the Cell can also receive the block address, which is then concatenated with the data address group as input.

[0069] The read and write states are generated by the read enable signal and the write enable signal, and are used to determine the operation types of the pre-storage memory and the result memory.

[0070] The input data group includes a group of data used to participate in the calculation and the Cell address corresponding to the data.

[0071] The data address group is used to indicate whether the cell participates in this calculation.

[0072] The Block address is used to indicate the address of the Block where the Cell is located.

[0073] The state decoder outputs the corresponding operation type to the pre-storage memory and the result memory respectively based on the read and write states.

[0074] Specifically, the read and write states include a read enable signal (0, 1) and a write enable signal (0, 1); the operation types include sleep, write, stop write, and read. The corresponding relationship between the read and write states and the operation types is shown in Table 1:

[0075]

[0076]

[0077] The state decoder determines whether the cell participates in the current operation based on the cell address in the data address group and outputs the result.

[0078] Specifically, if the cell address doesn't exist in the data address group, all inputs to that cell are ignored and the cell doesn't participate in the calculation. If the cell address exists, the corresponding data participating in the cell calculation is found in the input data group based on the cell address and output to the adder. The block address is used to determine the block participating in the calculation. The state decoder in the block's internal result does not process the block address.

[0079] The adder is used to perform addition operation on the data involved in the Cell calculation and the data in the pre-stored memory, and the result is used as an input of the first data selector.

[0080] The result memory is used to output the content in the result memory to the pre-storage storage or activation unit based on the operation type.

[0081] Specifically, when the operation type is a write operation, the content of the result memory is output to the pre-storage memory; when the operation type is a read operation, the content of the result memory is output to the activation unit; when the operation type is other types, the content of the result memory is not output.

[0082] Specifically, the length of the result memory is determined by the maximum value of the DNN model calculation result supported by the in-memory computing unit, and the length of the result memory needs to meet the upper limit of the cumulative calculation result.

[0083] Preferably, the result memory is SRAM.

[0084] The pre-storage memory is used to output the content in the pre-storage memory to the adder based on the operation type.

[0085] Specifically, when the operation type is a write operation, the content of the pre-storage memory is output to the adder. When the operation type is other types, the content of the pre-storage memory is not output.

[0086] Specifically, the length of the pre-storage memory is determined by the maximum value of the DNN model calculation results supported by the in-memory computing unit, and the length of the pre-storage memory needs to meet the upper limit of the cumulative calculation results.

[0087] Preferably, the pre-storage memory is SRAM.

[0088] The first selector selects two inputs based on a reset flag and outputs the selected inputs to a result memory; one input is the adder output, and the other input is a logic 0. The reset flag indicates a reset state of the calculation; before the calculation begins, the reset flag is true; after the calculation begins, the reset flag is not true.

[0089] Specifically, when the reset flag is true, the output of the first data selector is 0; when the reset flag is not true, the output of the first data selector is the output of the adder.

[0090] The activation unit is used to determine the corresponding activation function or custom polynomial based on the activation selection, perform activation operation on the input data, and obtain the output data of the cell.

[0091] Specifically, multiple activation functions and custom polynomials in the activation unit calculate the input data respectively, and output the results to the second data selector. The second selector selects the corresponding output based on the activation selection to obtain the output data.

[0092] Specifically, the custom polynomial determines coefficients based on the custom activation configuration.

[0093] Next, the input and output data of the in-memory arithmetic unit are explained.

[0094] Specifically, the input data of the in-memory operation unit is composed of the control instruction group output by the control module and the partial sum output by the inner product operation unit through a BUS (bus).

[0095] Specifically, the input of the in-memory computing unit includes a read / write status, a Block instruction group, a reset flag, and a custom activation configuration, and all of the above parts are connected in a splicing manner.

[0096] The read / write status data includes the read / write status of all cells in the in-memory computing unit. The data length is m×n×4 bits. The correspondence between the read / write status and the cell is implicit in the position information of the entire data. When the data is input into the in-memory computing unit, the cell's read / write status data is input into the corresponding cell based on the position relationship. The read / write status is determined by the control instruction group.

[0097] The Block instruction group includes multiple Block instructions connected in a splicing manner, and the number of Block instructions is the number of concurrent tasks to be completed.

[0098] The block instruction includes activation selection, input data group, data address group, and optionally, block address. The activation selection, data address group, and block address are determined by the control instruction group; the input data group is determined by a partial sum.

[0099] When the Block instruction does not include the Block address, the correspondence between the data and the Block is implicit in the location information of the entire data, and the in-memory calculation unit controls the Block's participation in the calculation based on the location information; when the Block instruction includes the Block address, the Block instruction is based on the Block address and the Block correspondence, and the in-memory calculation unit determines the valid Block participating in the calculation based on the Block address.

[0100] The Block instruction is transmitted to all cells in the Block in a broadcast mode, and the input of all cells is the same Block instruction.

[0101] The activation selection is used to select an activation function or a custom polynomial for the activation unit, and is a 3-bit or 2-bit value.

[0102] The input data group and the data address group are in a corresponding relationship. The data address group contains the address of the cell in the block that needs to participate in the calculation, and the input data group contains the input data of the corresponding cell.

[0103] The reset flag includes a reset flag for all blocks in the in-memory computing unit, with a length of mbits. Cells in the same block share a common reset flag. The correspondence between the reset flag and the block is implicit in the location information of the entire data. The reset flag is determined by the control instruction group.

[0104] The custom activation configuration includes custom activation configurations for all blocks of the in-memory computing unit. The same custom activation configuration is used for cells in the same block. The correspondence between the custom activation configuration and the block is implicit in the location information of the entire data. The custom activation configuration is determined by the control instruction group.

[0105] The output result of the in-memory computing architecture is the output of the in-memory computing unit. The output of the in-memory computing unit is obtained by splicing the output data of all blocks participating in the calculation. The output data of the block is obtained by splicing the output data of all cells participating in the calculation in the block.

[0106] This embodiment discloses an in-memory computing architecture, which uses an in-memory computing unit after an inner product operation unit. After the inner product operation unit generates a corresponding partial sum, the controller combines the partial sum as an input data group and a data address group and inputs it into the in-memory computing unit, thereby realizing the input memory and then performing addressing calculation, and completing the accumulation operation directly through one addressing, thereby saving computing time and computing power consumption.

[0107] The in-memory computing architecture places partial sum and accumulation operations in the Cell of the in-memory computing unit, eliminating the need for the process of retrieving and accumulating and then storing. This achieves the reuse of the DNN accelerator output, increases the computing speed and reduces computing power consumption, and solves the memory wall problem at the output end of the DNN accelerator.

[0108] Method Example

[0109] The present invention also discloses an in-memory computing acceleration method using the in-memory computing architecture of the above embodiment, which specifically includes the following steps:

[0110] Step S1: The control module obtains read address information, control information, and a control instruction group corresponding to weights and activation data based on the instruction information; the input memory stores the weights and activation data of the DNN network based on the write address information;

[0111] Step S2: The input memory outputs the corresponding DNN network weights and activation data to the inner product operation unit based on the read address information;

[0112] Step S3: The inner product operation unit combines the corresponding weights and activation data based on the control information and the weights and activation data of the DNN network to obtain corresponding partial sums;

[0113] Step S4, the in-memory computing unit concurrently performs cumulative operations based on the control instruction group and the partial sum to calculate the output result of the in-memory computing architecture; wherein, the control instruction group and the partial sum are combined through the BUS (bus) to obtain the input data of the in-memory computing unit.

[0114] This step specifically includes the following steps:

[0115] Step S41: Determine the Block involved in the calculation based on the Block instruction group of the input data.

[0116] Specifically, when the Block instruction in the Block instruction group does not include the Block address, the Block participating in the calculation is determined based on the position information of the Block instruction in the Block instruction group; when the Block instruction includes the Block address, the Block participating in the calculation is determined based on the Block address.

[0117] Step S42: Determine the Cell involved in the calculation based on the input data group and data address group in the Block instruction.

[0118] Specifically, block instructions within the same block are broadcast to all cells. For a cell, if its address is not found in the data address group, all inputs to that cell are ignored and the cell does not participate in the calculation. If its address is found, the corresponding data participating in the cell calculation is found in the input data group and output to the adder. The state decoder does not process the block address.

[0119] Step S43: Based on the read and write status, determine the operation type of each cell in the in-memory computing unit.

[0120] Specifically, referring to Table 1 of the embodiment, based on the read / write status, the operation type of the Cell can be determined as sleep, write, stop write, or read.

[0121] Step S44: Based on the operation type, the Cell performs corresponding accumulation operation on the input data.

[0122] Specifically, before the calculation starts, the "reset flag" is true, and the output of the first data selector is 0, that is, the content of the result memory is set to 0; when the operation type is write, the Cell inputs the content of the result memory into the pre-storage memory, and outputs the data participating in the Cell calculation obtained by the state decoder to the adder; when the write operation is completed, the operation type changes to stop writing, the content of the pre-storage unit stops changing, and the calculation result of the adder is output to the result memory; repeat the above steps, and the Cell performs accumulation operations on the input data group; when the operation type changes to read, the accumulation ends, and the content of the result memory is the accumulation result, which is output to the activation unit.

[0123] Step S45: Based on the activation selection, the Cell performs an activation operation on the accumulated operation result to obtain the output result of the Cell.

[0124] Specifically, each activation function and custom polynomial in the activation unit calculates the accumulation operation in step 4 respectively, and the activation unit selects one of the calculation results as the output result of the cell based on the activation selection.

[0125] Step S46: Based on the output results of each Cell, obtain the results of the in-memory calculation.

[0126] Specifically, the output data of all cells participating in the calculation in the same block are spliced ​​together to obtain the output result of the block; the output results of all blocks participating in the calculation are spliced ​​together to obtain the result of the in-memory calculation, that is, the output result of the in-memory calculation architecture.

[0127] Compared with the existing technology, this embodiment provides an in-memory computing acceleration method using the above-mentioned in-memory computing architecture, which uses an in-memory computing unit to perform accumulation operations on partial sums, and places the partial sum accumulation operations in the Cell of the in-memory computing unit, avoiding the process of taking out the accumulation and then storing in the traditional method, thereby improving the computing speed and reducing the computing power consumption; and, by combining the input data group and the data address group and inputting them into the in-memory computing unit, the number of addressing times is reduced, thereby saving computing time and computing power consumption, and further improving the computing speed.

[0128] It should be noted that the above embodiments are based on the same inventive concept, and parts not described repeatedly can be used as reference for each other.

[0129] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. An in-memory computing architecture for deploying deep learning networks, characterized in that: It includes an input memory, an inner product operation unit, an in-memory calculation unit and a control module; The control module is used to obtain instruction information and obtain read address information, control information and control instruction group corresponding to the weight and activation data based on the instruction information; The input memory is used to store the weights and activation data of the DNN network based on the write address information, and output the corresponding weights and activation data of the DNN network to the inner product operation unit based on the read address information input by the control module; The inner product operation unit is used to combine the corresponding weights and activation data based on the control information and the weights and activation data of the DNN network to obtain corresponding partial sums; The in-memory computing unit is configured to concurrently perform an accumulation operation based on the control instruction group and the partial sum to obtain a final operation result; The in-memory computing unit includes multiple Blocks, each of which is used to complete a computing task, and the number of the multiple Blocks is determined by the maximum number of concurrent tasks; the Block includes multiple Cells, each of which includes a storage accumulation part and an output part; the storage accumulation part is used to store input data and perform accumulation operations, and the output part is used to activate the results of the accumulation operations to obtain the output results of the Cell; the storage accumulation part includes a state decoder, a pre-storage memory, a result memory, an adder, and a first data selector; wherein the state decoder is used to determine the operation type of the pre-storage memory and the result memory based on the input data, and to obtain the first input data of the adder based on the input data; the pre-storage memory is used to output the content in the pre-storage memory to the adder as the second input data of the adder based on the operation type; the adder is used to perform addition operations on the first input data and the second input data; the first data selector is used to select to set the result memory to 0 or to store the output of the adder in the result memory based on the reset flag; the result memory is used to output the content in the result memory to the pre-storage memory or the output part based on the operation type.

2. The in-memory computing architecture according to claim 1, wherein: The cumulative operation based on the control instruction group and the partial sum is performed concurrently to obtain the final operation result, including: The in-memory computing unit obtains concurrent computing tasks based on the partial sum and the control instruction group; And for each computing task, complete the accumulation operation and activation processing of its partial sum to obtain the output result corresponding to the task; The output results of the in-memory computing architecture are obtained based on the output results of all tasks.

3. The in-memory computing architecture according to claim 2, wherein: The output of the in-memory computing architecture includes the output of the in-memory computing unit, which is obtained by splicing the output data of all blocks participating in the calculation, and the output data of the block is obtained by splicing the output data of all cells participating in the calculation in the block.

4. The in-memory computing architecture according to claim 3, wherein: The output portion of the in-memory computing unit includes an activation unit, and the activation unit includes a plurality of activation functions, a custom polynomial, and a second data selector, wherein, Multiple activation functions and custom polynomials in the activation unit are used to perform corresponding activation operations on the input data respectively; The second data selector is used to select one of the activation operation results as the output result of the Cell based on activation selection.

5. The in-memory computing architecture according to claim 4, wherein: The input of the in-memory computing unit is based on the control instruction group and part and is obtained through the BUS bus, including read and write status, Block instruction group, reset flag and custom activation configuration; the Block instruction group includes multiple Block instructions, and the Block instructions include activation selection, input data group, and data address group; wherein, The reset flag is used to indicate the reset state of the calculation; The custom activation configuration is used to customize the polynomial to determine the coefficients; The activation selection is input into the activation unit; The read and write status, input data group and data address group are input to the status decoder.

6. The in-memory computing architecture according to claim 5, wherein: The Block instruction also includes a Block address, and the Block address is used to determine the valid Block involved in the calculation.

7. An in-memory computing acceleration method using the in-memory computing architecture of claim 6, characterized in that: The steps include: Step S1: The control module obtains address information, control information, and a control instruction group corresponding to weights and activation data based on the instruction information; the input memory stores the weights and activation data of the DNN network based on the write address information; Step S2: The input memory outputs the corresponding DNN network weights and activation data to the inner product operation unit based on the read address information; Step S3: The inner product operation unit combines the corresponding weights and activation data based on the control information and the weights and activation data of the DNN network to obtain corresponding partial sums; Step S4: The in-memory computing unit concurrently performs accumulation operations based on the control instruction group and the partial sum to obtain the output result of the in-memory computing architecture.

8. The in-memory computing acceleration method according to claim 7, characterized in that: The in-memory computing unit performs cumulative operations based on the control instruction group and partial sum concurrently, and the output results of the in-memory computing architecture include: Step S41: Determine the Block involved in the calculation based on the Block instruction group of the input data; Step S42: Determine the Cell involved in the calculation based on the input data group and data address group in the Block instruction; Step S43: Determine the operation type of each cell in the in-memory computing unit based on the read / write status; Step S44: Based on the operation type, the Cell performs corresponding accumulation operation on the input data; Step S45: Based on the activation selection, the Cell performs activation operation on the accumulated operation result to obtain the output result of the Cell. Step S46: Based on the output results of each Cell, obtain the results of the in-memory calculation.

Images