An extension unit

Abstract

The application provides an extension unit, and relates to the technical field of in-memory computing. The extension unit comprises an input-output module, an input FIFO, an output FIFO, an analog memory-computing integrated array, a near-memory circuit and a state machine; wherein: the input-output module is used for realizing the interaction between the extension unit and external signals; the input FIFO is used for buffering input data and converting a CPU clock domain into a computing clock domain in the process of buffering the input data; the output FIFO is used for buffering output data and converting the computing clock domain into the CPU clock domain in the process of buffering the output data; the analog memory-computing integrated array is used for realizing a multiply-accumulate operation; the near-memory circuit is used for processing the multiply-accumulate operation result; and the state machine is used for running the analog memory-computing integrated array. The extension unit provided in the embodiment of the application can improve the utilization rate of computing resources.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of in-memory computing, in particular to an extension unit. BACKGROUND

[0002] Under the background of big data application, a new type of computing architecture centered on data, namely, Computing-in-Memory (CIM), has attracted widespread attention.

[0003] A new neural network hardware design idea is provided through the CIM chip: by embedding certain computing capability in the memory, such as embedding a computing unit in the chip periphery or the storage array, to reduce data migration between the memory and the processor, thereby improving the computing speed, relieving the bandwidth pressure and reducing the power consumption. From the computing paradigm, there are two kinds of digital and analog in the CIM architecture. Among them, the computing unit of the digital CIM scheme is realized by digital adder or multiplier, while the analog CIM realizes the multiply-accumulate operation through the accumulation of current, voltage and charge. At present, due to the more simple and effective design of the multiply-add operation circuit, and the higher tolerance of neural networks to computing accuracy, the analog CIM chip shows better prospects in neural network acceleration.

[0004] The way of realizing multiply-accumulate operation of the analog CIM technology is to realize the accumulation operation through the accumulation of physical quantities such as current, voltage and charge, which is different from the digital CIM scheme. The analog CIM and technology do not need to increase the computing circuit when the computing scale becomes larger, but only need to increase the storage computing unit (modified SRAM or RRAM, MRAM), which makes the analog CIM chip have great advantages in executing large-scale matrix operation. However, the existing neural network computing method cannot fully call the computing capability of the CIM chip. For example: the CNN network has a multi-layer convolution structure, and after the current convolution layer is calculated, the next layer of convolution calculation begins. However, the multiply-accumulate computing power occupied by one layer of convolution only accounts for a small part of the total computing power; on the other hand, the computing bandwidth of the CIM is much larger than that of the CPU, which leads to the time of data scheduling being greater than the time of the chip performing in-memory computing, which leads to the fact that when the analog CIM computing is deployed on a large scale, the computing resource utilization rate is not high, and the high parallelism computing advantage is not obvious. SUMMARY

[0005] In view of the problems in the prior art, the embodiments of the present application provide an extension unit, which can at least partially solve the problems in the prior art.

[0006] In one aspect, the present application provides an extension unit, comprising:

[0007] The input-output module, the input FIFO, the output FIFO, the analog memory-computing integrated array, the near-memory circuit and the state machine; wherein:

[0008] The input-output module is configured to realize interaction between the extension unit and external signals, and is connected with the input FIFO, the output FIFO, the analog memory-computing integrated array and the near-memory circuit, respectively.

[0009] The input FIFO is configured to buffer input data and convert a CPU clock domain into a computing clock domain during the buffering of the input data, and is connected with the analog memory-computing integrated array, the near-memory circuit and the state machine, respectively.

[0010] The output FIFO is configured to buffer output data and convert a computing clock domain into a CPU clock domain during the buffering of the output data, and is connected with the near-memory circuit.

[0011] The analog memory-computing integrated array is configured to realize multiply-accumulate operation, and is connected with the near-memory circuit and the state machine, respectively.

[0012] The near-memory circuit is configured to process multiply-accumulate operation results, and is connected with the state machine.

[0013] The state machine is configured to run the analog memory-computing integrated array.

[0014] The input-output module is specifically configured to receive external address information and corresponding data, and send target data to a corresponding data receiver according to a data type of the address information and the corresponding data.

[0015] The near-memory circuit includes a configuration register, and correspondingly, the input-output module is further specifically configured to:

[0016] receive a computing state signal sent by the configuration register, and determine a computing mode according to the computing state signal;

[0017] determine whether an external data bus high data bit is enabled according to the computing mode, and a data width of feature vector data written to the input FIFO.

[0018] The input FIFO is specifically configured to:

[0019] receive a computing state signal sent by the configuration register, and divide the input FIFO into corresponding sub-modules according to the computing state signal;

[0020] determine a storage location of the feature vector data according to the sub-modules obtained by the division.

[0021] The input FIFO is further specifically configured to:

[0022] The load status signal of the divided sub-module is sent to the state machine, and the data width enable bit corresponding to the calculation mode is determined according to the data width of the load status signal.

[0023] The near-memory circuit further includes a gated register; correspondingly, the state machine is also specifically used for:

[0024] The system receives gating information sent by the gating register, and determines the load status of the sub-module corresponding to the row array of the analog in-memory computing array based on the gating information and the load status signal; the gating information indicates whether the row array of the analog in-memory computing array is enabled.

[0025] If it is determined that the load state of all the sub-modules is not empty, then the state machine is switched to the running state.

[0026] Specifically, the state machine is also used for:

[0027] Send control signals to the analog in-memory computing array to control the analog in-memory computing array to perform multiplication and accumulation operations.

[0028] Specifically, the analog in-memory computing array is also used for:

[0029] The system receives weight data and corresponding address information sent by the input / output module, and writes the weight data to the corresponding storage location according to the storage space mapping relationship of the address information.

[0030] Receive the gating signal sent by the gating register; the gating signal includes a row gating signal and a column gating signal with corresponding bit widths;

[0031] The corresponding subarray is activated based on the row gating signal and the column gating signal;

[0032] The weight data is read from the storage location, and the weight data and the feature vector data are multiplied and accumulated using the subarray.

[0033] The result of the multiplication and accumulation operation is sent to the near-memory circuit.

[0034] The near-memory circuit further includes a vector shift module, a vector adder, and an on-chip net connected sequentially to each other, as well as a line select register separately connected to the on-chip net; wherein:

[0035] The near-memory circuit is also specifically used to split the multiplication-accumulation operation result according to a preset number of bits to obtain the split calculation result vectors;

[0036] The vector shift module is connected to the configuration register and is used to receive the calculation result vectors allocated by the near memory circuit and the shift configuration information of the configuration register, shift the calculation result vectors according to the shift configuration information, and send the first intermediate calculation result obtained after shifting to the vector adder.

[0037] The vector adder is connected to the configuration register and is used to receive the first intermediate calculation result and the first preset bit-width addend data and the second preset bit-width addition / subtraction signal sent by the configuration register;

[0038] Based on the second preset bit width addition / subtraction signal and the first preset bit addend data, the first intermediate calculation result is subjected to a corresponding secondary calculation to obtain the second intermediate calculation result;

[0039] Send the second intermediate calculation result to the on-chip network;

[0040] The on-chip network is used to receive the second intermediate calculation result and the line selection signal sent by the line selection register, and rearrange the second intermediate calculation result according to the line selection signal. The rearranged second intermediate calculation result is used as the multiply-accumulate operation result, and the multiply-accumulate operation result is sent to the output FIFO.

[0041] The expansion unit further includes a controller, which is connected to the state machine and the near memory circuit, respectively.

[0042] The controller is used for:

[0043] The system receives a start signal sent by the state machine and sends enable signals to the vector shift module, the vector adder, and the on-chip net according to the start signal, so as to control the start of the vector shift module, the vector adder, and the on-chip net.

[0044] The expansion unit provided in this embodiment of the invention includes an input / output module, an input FIFO, an output FIFO, an analog-to-in-memory array, a local storage circuit, and a state machine. The input / output module is used to enable the expansion unit to interact with external signals and is connected to the input FIFO, the output FIFO, the analog-to-in-memory array, and the local storage circuit, respectively. The input FIFO buffers input data and converts the CPU clock domain to the computing clock domain during the buffering process; it is connected to the analog-to-in-memory array, the local storage circuit, and the state machine, respectively. The output FIFO buffers output data and converts the computing clock domain to the CPU clock domain during the buffering process; it is connected to the local storage circuit. The analog-to-in-memory array performs multiply-accumulate operations and is connected to the local storage circuit and the state machine, respectively. The local storage circuit processes the results of the multiply-accumulate operations and is connected to the state machine. The state machine runs the analog-to-in-memory array, thereby improving the utilization of computing resources. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0046] Figure 1 This is a schematic diagram of the structure of an extension unit provided in an embodiment of the present invention.

[0047] Figure 2 This is a schematic diagram of the structure of an extended array provided in an embodiment of the present invention.

[0048] Figure 3 This is a schematic diagram of the overall system structure including an extended array provided in an embodiment of the present invention.

[0049] Figure 4 This is a schematic diagram of a process for network inference based on an extended array, provided as an embodiment of the present invention.

[0050] Figure 5 This is a schematic diagram of the process of using an extended array according to an embodiment of the present invention.

[0051] Figure 6 The diagram illustrates the row gating circuit and column gating circuit provided in an embodiment of the present invention.

[0052] Figure 7 This is a schematic diagram illustrating the data processing of the Reshape module according to an embodiment of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments and descriptions of the present invention are used to explain the present invention, but are not intended to limit the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other.

[0054] Figure 1 This is a schematic diagram of the structure of an extension unit provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the extension unit provided in this embodiment of the invention includes:

[0055] Input / output module 1, input FIFO 2, output FIFO 3, analog in-memory array 4, proximity circuit 5, and state machine 6; wherein:

[0056] The input / output module 1 is used to enable the expansion unit to interact with external signals, and is connected to the input FIFO 2, the output FIFO 3, the analog in-memory array 4 and the proximity circuit 5 respectively;

[0057] The input FIFO2 is used to buffer input data and converts the CPU clock domain into the computing clock domain during the buffering process. It is connected to the analog in-memory array 4, the near-memory circuit 5, and the state machine 6, respectively.

[0058] The output FIFO3 is used to buffer output data and converts the calculation clock domain into the CPU clock domain during the buffering process, and is connected to the near memory circuit 5.

[0059] The analog in-memory array 4 is used to implement multiplication and accumulation operations, and is connected to the near-memory circuit 5 and the state machine 6 respectively;

[0060] The near-memory circuit 5 is used to process the results of multiplication and accumulation operations and is connected to the state machine 6;

[0061] The state machine 6 is used to run the analog in-memory computing array.

[0062] The input / output module 1 is specifically used to receive external address information and corresponding data, and to send target data to the corresponding data receiver according to the data type of the address information. Specific details are as follows:

[0063] The data types corresponding to address information include weight data, feature vector data, state control data, and multiply-accumulate operation results.

[0064] When the data type corresponding to the address information is weight data, the input / output module 1 sends the weight data to the analog in-memory array 4.

[0065] When the data type corresponding to the address information is feature vector data, the input / output module 1 sends the address information and the corresponding feature vector data to the input FIFO 2.

[0066] When the data type corresponding to the address information is status control data, the input / output module 1 sends the address information and the corresponding status control data to the gate register, configuration register and line select register in the near memory circuit 5.

[0067] When the data type corresponding to the address information is the result of a multiplication-accumulation operation, input / output module 1 sends the address information to output FIFO3. Then, in the next CPU clock cycle, it receives the buffered multiplication-accumulation result sent by output FIFO3 and sends the result to the external expansion unit.

[0068] The near-memory circuit 5 includes a configuration register; correspondingly, the input / output module 1 is further specifically used for:

[0069] Receive the computation status signal sent by the configuration register, and determine the computation mode based on the computation status signal;

[0070] The calculation mode determines whether the high data bits of the external data bus are enabled, and the data width of the feature vector data written to the input FIFO2. Specific details are as follows:

[0071] Input / output module 1 also receives the computation status signal col_en sent by the configuration register, which is used to switch between CNN computation and matrix computation modes.

[0072] When in CNN computation mode, the high 4 bits of the external data bus are enabled, and the feature vector data written to input FIFO2 is 36 bits wide. When in matrix computation mode, the high 4 bits of the external data bus are disabled, and the feature vector data written to input FIFO2 is 32 bits wide.

[0073] The input FIFO2 is specifically used for:

[0074] Receive the calculation status signal sent by the configuration register, and divide the input FIFO2 into corresponding sub-modules according to the calculation status signal;

[0075] The storage location of the feature vector data is determined based on the sub-modules obtained from the partitioning. Specific details are as follows:

[0076] Input FIFO2 acts as a buffer for data input. During the buffering process, input data in the CPU clock domain is transformed into computable data in the computation clock domain. Since the CPU clock frequency can be much faster than the computation clock frequency, this asynchronous method can fully utilize CPU bandwidth.

[0077] The input FIFO2 receives feature vector data from the input / output module 1 and outputs buffered feature vector data to the analog in-memory array 4. During this process, the data is converted from the CPU clock domain to the computing clock domain.

[0078] Input FIFO2 receives the computation status signal col_en from the configuration register, which is used to switch between CNN computation and matrix computation modes. In CNN computation mode, input FIFO2 is divided into eight 64-bit wide and 4-bit deep sub-modules, and the feature vector data received from input / output module 1 is stored in the first 36 bits of the 64-bit sub-modules. In matrix computation mode, input FIFO2 is divided into sixteen 32-bit wide and 4-bit deep sub-modules, matching the width of the feature vector data received from input / output module 1.

[0079] The input FIFO2 is also specifically used for:

[0080] The load status signal of the divided submodule is sent to the state machine 6, and the data width enable bit corresponding to the calculation mode is determined according to the data width of the load status signal. Specifically, the load status signal can be a full-empty status signal. The input FIFO2 sends the full-empty status signal of the submodule to the state machine 6. This signal has a 16-bit width. When col_en corresponds to CNN mode, the high 8 bits are turned off, and when col_en corresponds to matrix calculation mode, all 16 bits are enabled.

[0081] Input FIFO2 is also used to: receive address information sent by input / output module 1, and determine the sub-module to be written to based on the address information. When col_en corresponds to CNN mode, the address information is shifted one bit to the right.

[0082] Input FIFO2 receives address information sent by the input / output module and determines the submodule to be written to based on the address information. When col_en corresponds to CNN mode, the address information is shifted one bit to the right.

[0083] The near-memory circuit 5 also includes a gated register; correspondingly, the state machine 6 is also specifically used for:

[0084] The system receives gating information sent by the gating register, and determines the load status of the sub-module by dividing the row array corresponding to the row array of the analog in-memory computing array 5 according to the gating information and the load status signal; the gating information indicates whether the row array of the analog in-memory computing array is enabled.

[0085] If it is determined that the load state of all the sub-modules is not empty, then the state machine is switched to the running state. The explanation is as follows:

[0086] State machine 6 receives a 16-bit full-empty status signal from input FIFO2 and a 16-bit row gating information from the gating register of near-memory circuit 5. A single bit of the row gating information being 1 indicates that a row of the analog in-memory array, corresponding to the number of bits in that bit, is enabled. Each row in the analog in-memory array 4 corresponds to a submodule. When all submodules corresponding to enabled rows are not empty, state machine 6 exits the idle state and begins operation.

[0087] The state machine 6 is also specifically used for:

[0088] Control signals are sent to the analog in-memory computing array 4 to control it to perform multiply-accumulate operations. The control signals may specifically include five signals: start calculation, data reset, enable 2b of 4b, pause parallel-to-serial conversion clock, and enable 1b of 2b. These five signals are interleaved and take a total of 12 calculation clock cycles.

[0089] The analog in-memory computing array 4 is also specifically used for:

[0090] The system receives weight data and corresponding address information sent by the input / output module 1, and writes the weight data to the corresponding storage location according to the storage space mapping relationship of the address information.

[0091] Receive the gating signal sent by the gating register; the gating signal includes a row gating signal and a column gating signal with corresponding bit widths;

[0092] The corresponding subarray is activated based on the row gating signal and the column gating signal;

[0093] The weight data is read from the storage location, and the weight data and the feature vector data are multiplied and accumulated using the subarray.

[0094] The result of the multiplication-accumulation operation is sent to the near-memory circuit 5. Specific details are as follows:

[0095] Before the calculation, the analog in-memory array 4 receives weight data and address information from the input / output module 1, and writes the weight data to the corresponding storage location according to the storage space mapping relationship of the address data.

[0096] Before computation, the analog in-memory array 4 receives 16b row gating signals and 16b column gating signals from the gating registers in the near-memory circuit 5. The subarrays of the analog in-memory array 4 are partially enabled under the control of the row gating signals and column gating signals.

[0097] During the computation process, the analog in-memory array 4 receives buffered feature vector data from the input FIFO2.

[0098] During the calculation process, the analog in-memory array 4 receives five signals from the state machine 6 and completes the entire multiply-accumulate operation under its control.

[0099] After the calculation is completed, the simulated in-memory array 4 sends the 64-bit multiply-accumulate result to the near-memory circuit 5.

[0100] The near-memory circuit 5 further includes a vector shift module, a vector adder, and an on-chip net connected sequentially to each other, as well as a line select register separately connected to the on-chip net; wherein:

[0101] The near-memory circuit 5 is also specifically used to split the multiplication-accumulation operation result according to a preset number of bits to obtain the split calculation result vectors;

[0102] The vector shift module is connected to the configuration register and is used to receive the calculation result vectors allocated by the near memory circuit 5 and the shift configuration information of the configuration register, shift the calculation result vectors according to the shift configuration information, and send the first intermediate calculation result obtained after shifting to the vector adder.

[0103] The vector adder is connected to the configuration register and is used to receive the first intermediate calculation result and the first preset bit-width addend data and the second preset bit-width addition / subtraction signal sent by the configuration register;

[0104] Based on the second preset bit width addition / subtraction signal and the first preset bit addend data, the first intermediate calculation result is subjected to a corresponding secondary calculation to obtain the second intermediate calculation result;

[0105] Send the second intermediate calculation result to the on-chip network;

[0106] The on-chip network is used to receive the second intermediate calculation result and the line selection signal sent by the line selection register, rearrange the second intermediate calculation result according to the line selection signal, use the rearranged second intermediate calculation result as the multiply-accumulate operation result, and send the multiply-accumulate operation result to the output FIFO3. Specific details are as follows:

[0107] The near-memory circuit 5 receives the 64b multiplication-accumulation result from the analog in-memory array 4, splits it into 16-element vectors (each calculation result vector) at 4b intervals, and sends them to the vector shifting module.

[0108] The vector shift module receives the calculation result vectors from the near-memory module and receives shift configuration information from the configuration register. Under the control of the shift configuration information, the vector shift module shifts the 16 4-bit calculation result vectors to the right by a certain number of bits and sends the first intermediate calculation result with a width of 64 bits to the vector adder;

[0109] The vector adder receives a 64-bit wide first intermediate calculation result from the vector shift module, and also receives 64-bit addend data from the configuration register, along with 16-bit wide addition / subtraction signals. Each addend data is 4 bits wide, for a total of 16. The vector adder treats the 64-bit wide first intermediate calculation result as 16 4-bit elements. When the addition / subtraction signal corresponds to an addition operation, it adds the result to each of the 16 4-bit addends; when the addition / subtraction signal corresponds to a subtraction operation, the data becomes a subtrahend, the subtraction operation is performed, and the 64-bit wide second intermediate calculation result is sent to the on-chip net.

[0110] The on-chip net receives the 64-bit second intermediate calculation result from the vector adder and receives the line selection signal from the line selection register. The sub-on-chip net rearranges the 16 4-bit second intermediate calculation results according to the line selection signal, uses the rearranged second intermediate calculation results as the multiply-accumulate operation result, and sends the multiply-accumulate operation result to the output FIFO3.

[0111] The expansion unit also includes a controller 7, which is connected to the state machine 6 and the near memory circuit 5 respectively.

[0112] The controller 7 is used for:

[0113] The system receives a start signal sent by the state machine 6, and sends enable signals to the vector shift module, the vector adder, and the on-chip net according to the start signal, so as to control the start of the vector shift module, the vector adder, and the on-chip net.

[0114] Output FIFO3 serves as a buffer for data output. During buffering, the calculation result in the computational clock domain is transformed into a buffered calculation result in the CPU clock domain, thus realizing a complete asynchronous clock paradigm. This allows the expansion unit to have the appearance of a single clock domain and an asynchronous calculation method. Output FIFO3 consists of two FIFO sub-modules with a width of 32 bits and a depth of 4 bits.

[0115] Output FIFO3 receives the multiplication and accumulation results sent by the near-memory circuit 5;

[0116] Output FIFO3 receives the address signal sent by input / output module 1 and sends the corresponding 32-bit width buffered calculation result to input / output module 1.

[0117] From the outside, the expansion unit has the appearance of a typical FIFO, but its asynchronous nature is internal. The external input and output data buses are both in the CPU clock domain, because the input and output FIFOs switch clock domains twice. Internally, the data flow operates automatically under the control of the control circuit and the adjustment of information in the registers. This transforms the original analog in-memory computing circuit, which required 12 steps of processing and had a limited operating frequency, into an expansion unit that can adapt to high CPU clock frequencies and is easy to use.

[0118] It is worth mentioning that all storage resources within the expansion unit have a unique address mapping, and the uniqueness of the address mapping still exists after expansion into an array. This makes it possible to map the expansion array as memory, which makes it possible to design compilers for in-memory computing circuits.

[0119] like Figure 2 As shown, the extended array consists of n×n extended units, all of which share a 32-bit address bus and a 32-bit data bus. The empty / full status signals of all internal extended units are aggregated in the array status register. Taking a 4×4 extended unit array as an example: Figure 2 As shown, the 16 expansion units have 16 input FIFO full state signals and 16 output FIFO empty state signals, totaling 32 signals, which correspond one-to-one with each bit of the 32-bit array status register. In addition to the expansion unit array, the expansion array also includes an adder tree, used to accumulate the calculation results of each expansion unit to realize matrix operations that occupy the storage and computing resources of multiple expansion units.

[0120] like Figure 3 As shown, the overall system architecture including the extended array is used in the same way as ordinary memory. The configuration register settings, input vector writing, weight data writing, and data reading are all implemented through data access via different address mappings. RISC-V provides control and data processing functions. In terms of control, it can control the Direct Memory Access (DMA) module to enable direct writing of external data to the extended array.

[0121] RISC-V fetches instructions from local instruction memory and data from local data cache or external memory. Its data interaction with the extended array, especially when reading computation results, is no different from reading data from general memory. Data writing is divided into two cases: input vector writing and register configuration. During convolution calculations, the former requires a reshape module, while the latter can be written directly. During matrix operations, both can be written directly. Data can be written and read simultaneously to improve data throughput during convolution calculations.

[0122] like Figure 4 As shown, the first step is to configure the extended array, which involves the configuration register of the extended array and the configuration register inside the extended unit that makes up the array, which controls the data flow structure and the working mode of the memory circuit in each extended unit.

[0123] The second step is to write the weights. The specific storage address of the relevant data is related to the control results and network mapping in the first step.

[0124] The third step is network inference. The first stage is input data, corresponding to the input layer of the neural network inference. The second stage involves sequentially calling each extension unit, specifically updating the input of extension unit n-1 and writing the result of unit n-1 into the unit (n≥2), corresponding to the hidden layer of the neural network inference. The third stage is retrieving the inference result, i.e., reading the calculation result from extension unit n. After one inference process is completed, if the input data has been updated, i.e., there is a new frame of image data, then the input data is written to extension unit 1 again; otherwise, the process ends.

[0125] like Figure 5 As shown, the nested pipeline architecture divides the n expansion units into n pipeline levels, each of which can perform computational tasks for different convolutional layers. Within a single level, a four-stage pipeline is further divided into write, in-memory computation, near-memory computation, and read to reduce the overall time consumption of a single multiply-accumulate operation. Write and read can be performed simultaneously to further improve pipeline speed.

[0126] When using extended arrays for convolution, data should be input sequentially: extended units 1 through 16. The extended unit will activate its state machine when the input FIFO is not empty and the output FIFO is not full. After extended unit 1 completes its write operation, it performs normal computation. At this point, data can continue to be written to the input FIFO for subsequent computations, or data can be written to units 2-n. After one round of writing is complete, extended unit 1 has completed several computations. The computation results can then be read and simultaneously written to the next layer's output. The pipelined use of extended arrays is achieved through different input data within the extended units, thereby eliminating the impact of the extended unit's multi-cycle computation time on the convolution computation's data throughput.

[0127] like Figure 6 As shown, row and column gating circuits work together to divide the entire storage computing array into several subarrays. As shown in section b, each subarray has 16 storage computing units, each of which can store 4b of weight data. When the chip is in convolution computing mode, the first 9 of the 16 storage computing units are active, corresponding to a 3×3 convolution kernel; when in matrix computing mode, all 16 storage computing units can be used, but are not required to be used.

[0128] It can be observed that the circuits used in the convolution calculation mode form the circuits of the matrix calculation mode. This is because the bus bandwidth of existing CPU systems is mostly 32 bits. When the data precision is 4 bits, the number of writes required to write an input vector with 16 elements is the same as the number of writes required to write a feature vector with 9 elements.

[0129] As shown in part a, the enable of row and column units is controlled by row and column gating registers, which have a bit width of 32 bits. The 0 / 1 state of each bit corresponds to the enable signal being turned on or off.

[0130] In matrix computing mode, when using a memory computing chip with row and column gating circuits, all subarrays are generally enabled to fully utilize the multiply-accumulate computing power of the memory computing chip.

[0131] In convolutional computing mode, in order to improve computing power utilization, a common approach is to enable multiple units in the diagonal direction. The advantage of doing so is that it can make full use of the computing parallelism of the in-memory array, thereby improving the total data throughput.

[0132] The diagonal direction is enabled, allowing each subarray in the row direction to obtain an independent input feature vector. The column-direction segmentation allows the computation results of each subarray to be obtained at once. The computation results of the subarrays are connected to the column computation circuit through a transmission gate, which is controlled by the row-gated enable signal.

[0133] like Figure 7 As shown, the Reshape module is used to implement data format conversion. Since the data width in RISC-V is 32 bits, and the feature vector corresponding to a 3×3 convolution kernel has 9 elements, the least common multiple of 32 is 288. Therefore, regardless of the computational precision, storing feature vectors using existing CPU architectures will incur additional format processing overhead. CNN convolution has the concept of a convolution window; from a memory perspective, the data required for one convolution is stored at certain address intervals.

[0134] Therefore, a reshape module is designed. During the module's operation, image data is unfolded into a single row and stored in a cache. In this case, the column number of the image corresponds to the address interval between two adjacent data rows. The module retrieves three data points at this interval, simulating the behavior of a convolutional window by using data from different fields within a 32-bit width, such as bits 12 to 23 (a total of 12 bits). These three data points correspond to the data in the three rows covered by the convolutional window. The reshape module then combines the extracted data from different rows into a feature vector used for the next convolution calculation.

[0135] The existing analog-in-memory computing design architecture has the following shortcomings:

[0136] 1. Insufficient data pipeline: Multiplication and accumulation calculations require multiple CPU clock cycles, and pipeline is severely limited when convolution or matrix operations are performed separately on the chip.

[0137] 2. Insufficient chip utilization: A single in-memory computing chip can only perform one convolution calculation at a time, resulting in additional clock overhead.

[0138] 3. Insufficient computing power: Unable to perform operations such as pooling, biasing, and activation.

[0139] 4: Not suitable for convolution calculations; the current calculation result cannot be directly used for the next convolution.

[0140] The above technical problems are solved by the following explanation:

[0141] 1. Peripheral circuitry is arranged around the analog in-memory computing unit, including a state machine, input FIFO, output FIFO, and near-memory circuitry. Based on the asynchronous nature of the FIFO, a unified circuit with near-memory computing functionality, separating read / write (CPU clock domain) and computation (in-memory computing clock domain), is constructed and named the in-memory computing extension sub-unit (or extension unit for short). The extension unit is deployed as a 4×4 array, improving the data pipeline of the analog in-memory computing design at the cost of repeated convolution kernel arrangement; this is named the in-memory computing extension master array (or extension array for short). This two-layer structure addresses problem 1 mentioned above.

[0142] 2. In the extended unit, row gating and column gating are introduced for the storage and computation units of SRAM_CIM. Row gating controls whether each element of the input vector can participate in the operation, thus allowing subsequent computation data to be placed in the row direction; column gating controls the opening and closing of the pulse converter, which is equivalent to allowing custom channels when performing convolution operations. The two gating methods enable the stored data to be densely packed and ultimately mapped to all storage and computation units. This eliminates the need to overwrite the data after computation, and allows up to 8 convolution calculations to be performed at a time, thus solving problem 2.

[0143] 3. Shift and bias (addition) functions are implemented in the near-memory circuit section of the extended unit, and pooling functions are implemented in the extended array. The near-memory circuit is connected to the output section of the in-memory computing circuit in the extended unit, and its parallelism is the same as the number of columns of the in-memory computing circuit (16 in this design). The calculation results of the analog in-memory computing module are sequentially passed through the shift circuit and the bias circuit, and then input into the pooling circuit along with the historical results. Each circuit is a combinational logic circuit. The specific operating states of the three circuits are constrained by the control register, such as the size of the addend in the bias circuit, to solve problem 3.

[0144] 4. Add a reshape module to the fixed-function accelerator architecture of the RISC-V processor. When the extended array performs convolution calculations, it executes in a fixed zigzag order and densely packs the 4-bit precision results into 32-bit memory. When the accumulated convolution calculation results exceed 3 rows, the reshape pipeline is enabled to transform the data into the input required for the next layer of convolution, thus solving problem 4.

[0145] The embodiments of the present invention have the following beneficial effects:

[0146] 1. The nested pipeline architecture increases the computational efficiency of the extended array during CNN inference by up to 128 times.

[0147] 2. The fine-grained internal computing resources of the extended unit increase the parallelism of CNN computation to 8 times when performing convolution calculations, and reduce the weight data update frequency within the extended unit to 1 / 16 of the original.

[0148] 3. The DAC circuit with active quantization capability, in conjunction with the internal fine-grained expansion unit, improves the computational accuracy during CNN inference to 4 times the original.

[0149] 4. The bias, shift, and pooling vector arithmetic units implemented in the near-memory circuit section of the expansion unit enable parallel processing of data, which reduces the number of general-purpose computational operations within the CPU to at most 0.3% of the original.

[0150] 5. A dedicated RISC-V extended instruction set design compresses complex data flow control into 5 extended instructions to fully map the functionality of the extended array, reducing the number of data scheduling operations within the CPU to up to 3% of the original.

[0151] The expansion unit provided in this embodiment of the invention includes an input / output module, an input FIFO, an output FIFO, an analog-to-in-memory array, a local storage circuit, and a state machine. The input / output module is used to enable the expansion unit to interact with external signals and is connected to the input FIFO, the output FIFO, the analog-to-in-memory array, and the local storage circuit, respectively. The input FIFO buffers input data and converts the CPU clock domain to the computing clock domain during the buffering process; it is connected to the analog-to-in-memory array, the local storage circuit, and the state machine, respectively. The output FIFO buffers output data and converts the computing clock domain to the CPU clock domain during the buffering process; it is connected to the local storage circuit. The analog-to-in-memory array performs multiply-accumulate operations and is connected to the local storage circuit and the state machine, respectively. The local storage circuit processes the results of the multiply-accumulate operations and is connected to the state machine. The state machine runs the analog-to-in-memory array, thereby improving the utilization of computing resources.

[0152] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0153] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0154] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0155] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0156] In the description of this specification, the references to terms such as "an embodiment," "a specific embodiment," "some embodiments," "for example," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0157] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An expansion unit, characterized in that, include: Input / output module, input FIFO, output FIFO, analog in-memory array, near-memory circuit and state machine; among which: The input / output module is used to enable the expansion unit to interact with external signals, and is connected to the input FIFO, the output FIFO, the analog in-memory array and the proximity circuit respectively; The input FIFO is used to buffer input data and converts the CPU clock domain into the computing clock domain during the buffering process. It is connected to the analog in-memory array, the near-memory circuit, and the state machine, respectively. The output FIFO is used to buffer output data and converts the calculation clock domain into the CPU clock domain during the buffering process, and is connected to the near memory circuit. The analog in-memory array is used to implement multiply-accumulate operations and is connected to the near-memory circuit and the state machine respectively; The near-memory circuit is used to process the results of multiplication and accumulation operations and is connected to the state machine; The state machine is used to run the analog in-memory computing array.

2. The expansion unit according to claim 1, characterized in that, The input / output module is specifically used to receive external address information and corresponding data, and send target data to the corresponding data receiver according to the data type of the address information and the corresponding data.

3. The expansion unit according to claim 2, characterized in that, The near-memory circuit includes a configuration register; correspondingly, the input / output module is also specifically used for: Receive the computation status signal sent by the configuration register, and determine the computation mode based on the computation status signal; The calculation mode determines whether the high data bits of the external data bus are enabled, and the data width of the feature vector data written to the input FIFO.

4. The expansion unit according to claim 3, characterized in that, The input FIFO is specifically used for: Receive the calculation status signal sent by the configuration register, and divide the input FIFO into corresponding sub-modules according to the calculation status signal; The storage location of the feature vector data is determined based on the sub-modules obtained from the division.

5. The expansion unit according to claim 4, characterized in that, The input FIFO is also specifically used for: The load status signal of the divided sub-module is sent to the state machine, and the data width enable bit corresponding to the calculation mode is determined according to the data width of the load status signal.

6. The expansion unit according to claim 5, characterized in that, The near-memory circuit also includes a gated register; correspondingly, the state machine is also specifically used for: The system receives gating information sent by the gating register, and determines the load status of the sub-module corresponding to the row array of the analog in-memory computing array based on the gating information and the load status signal; the gating information indicates whether the row array of the analog in-memory computing array is enabled. If it is determined that the load state of all the sub-modules is not empty, then the state machine is switched to the running state.

7. The expansion unit according to claim 6, characterized in that, The state machine is also specifically used for: Send control signals to the analog in-memory computing array to control the analog in-memory computing array to perform multiplication and accumulation operations.

8. The expansion unit according to claim 6, characterized in that, The analog in-memory computing array is also specifically used for: The system receives weight data and corresponding address information sent by the input / output module, and writes the weight data to the corresponding storage location according to the storage space mapping relationship of the address information. Receive the gating signal sent by the gating register; the gating signal includes a row gating signal and a column gating signal with corresponding bit widths; The corresponding subarray is activated based on the row gating signal and the column gating signal; The weight data is read from the storage location, and the weight data and the feature vector data are multiplied and accumulated using the subarray. The result of the multiplication and accumulation operation is sent to the near-memory circuit.

9. The expansion unit according to claim 8, characterized in that, The near-memory circuit also includes a vector shift module, a vector adder, and an on-chip net connected sequentially to each other, as well as a line select register separately connected to the on-chip net; wherein: The near-memory circuit is also specifically used to split the multiplication-accumulation operation result according to a preset number of bits to obtain the split calculation result vectors; The vector shift module is connected to the configuration register and is used to receive the calculation result vectors allocated by the near memory circuit and the shift configuration information of the configuration register, shift the calculation result vectors according to the shift configuration information, and send the first intermediate calculation result obtained after shifting to the vector adder. The vector adder is connected to the configuration register and is used to receive the first intermediate calculation result and the first preset bit-width addend data and the second preset bit-width addition / subtraction signal sent by the configuration register; Based on the second preset bit width addition / subtraction signal and the first preset bit addend data, the first intermediate calculation result is subjected to a corresponding secondary calculation to obtain the second intermediate calculation result; Send the second intermediate calculation result to the on-chip network; The on-chip network is used to receive the second intermediate calculation result and the line selection signal sent by the line selection register, and rearrange the second intermediate calculation result according to the line selection signal. The rearranged second intermediate calculation result is used as the multiply-accumulate operation result, and the multiply-accumulate operation result is sent to the output FIFO.

10. The expansion unit according to claim 9, characterized in that, The expansion unit further includes a controller, which is connected to the state machine and the proximity circuit, respectively. The controller is used for: The system receives a start signal sent by the state machine and sends enable signals to the vector shift module, the vector adder, and the on-chip net according to the start signal, so as to control the start of the vector shift module, the vector adder, and the on-chip net.

Images