Storage and calculation integrated neural network acceleration method and accelerator based on Fibonacci coding

Abstract

The invention provides a storage and calculation integrated neural network acceleration method and accelerator based on Fibonacci coding, and the method comprises the steps: a Fibonacci encoder of the storage and calculation integrated neural network accelerator encodes an activation value and a weight value participating in a neural network operation task from a binary system to a Fibonacci code; the Fibonacci activation and the Fibonacci weight are obtained, a DAC of a storage and calculation integrated unit in the storage and calculation integrated neural network accelerator converts the Fibonacci activation into a plurality of analog voltages, and the analog voltages are applied to word lines of each row of an SRAM array in the storage and calculation integrated unit; each unit of the SRAM array multiplies an input analog voltage by a corresponding Fibonacci weight to obtain an intermediate operation result, an adder of each column of the SRAM array redistributes charges to the intermediate operation result to complete an accumulation operation of an inner product, and the accumulation operation result is converted into a Fibonacci coded digital format through an ADC (Analog to Digital Converter) of the storage and calculation integrated unit, so as to complete the accumulation operation of the inner product. And taking the result as an operation result of the neural network operation task.

Description

Technical Field

[0001] The present invention relates to the technical field of charge domain storage and computing integrated neural network accelerator system, and in particular to a Fibonacci-coded storage and computing integrated unit and a Fibonacci-coded sparse computing storage and computing integrated architecture. Background Art

[0002] Currently, the most common compute-in-memory (CIM) neural network accelerators use charge-domain static random access memory (SRAM) arrays for computation, such as Figure 1 As shown in , the structure mainly includes a digital-to-analog converter (DAC), an analog-to-digital converter (ADC), and an SRAM array for calculation. Among them, the SRAM array represents data in voltage amplitude and performs a one-bit inner product. Each row in the SRAM array is connected to a DAC for digital-to-analog conversion, while several columns share an ADC for reading the calculation results. Each cell in the SRAM array integrates a digital AND / XNOR gate or a transmission gate to perform a one-bit multiplication. In the SRAM array, a column of capacitors is connected together through a bit line (BL) to redistribute the charge to complete the accumulation operation of the inner product. In addition, the weights of the neural network are usually stored in SRAM to reuse the weight data. Since storing analog values ​​will cause a loss of neural network accuracy due to non-idealized analog circuits, the CIM accelerator stores the weights in the digital domain and performs calculations in the analog domain. The calculation process of the SRAM array is: multiple binary inputs are converted into multiple analog voltages by the DAC and applied to the word lines (WL) of each row of the SRAM array respectively. Then, each cell of the SRAM array multiplies the input analog voltage by the corresponding 1-bit weight, which can be achieved by the logic gates in the SRAM cell. After that, the capacitors of each column of the SRAM perform charge redistribution to obtain the output voltage. Finally, the output voltage is converted into digital output by the ADC.

[0003] To support multi-bit activation values ​​and weights, the existing CIM neural network accelerator adopts a bit serial-bit parallel (BSBP) scheme. The BSBP scheme refers to a method of splitting the inner product of an activation value vector of length n and a weight vector of length n into multiple one-bit inner products. Assuming that the activation value has p bits, the original activation value vector is split into p one-bit vectors of length n; assuming that the weight has q bits, the original weight vector is split into q one-bit vectors of length n. After splitting, the p one-bit activation value vectors are serially input into the SRAM array, while the q one-bit weight vectors are stored in parallel in the SRAM and inner product operations are performed in parallel with the one-bit activation value vector input each time. After the inner product operation is completed, the results of these one-bit inner products need to be left-shifted and added to obtain the results of the original multi-bit inner products, where the number of bits of the left shift is the sum of the bit position of the one-bit activation value vector and the bit position of the one-bit weight vector in the one-bit inner product.

[0004] The CIM neural network accelerator integrates several of the above SRAM arrays for calculation. The latest CIM neural network accelerator organizes these SRAM arrays into four levels: chip, calculation block, local multiplication and accumulation unit and SRAM array. The chip includes multiple computing blocks, which are connected through a two-dimensional on-chip network. Each computing block is directly connected to an on-chip router, and each router can send the received data to other adjacent routers, so that a computing block can send data to any other computing blocks via one or more routers, and can also perform unicast and multicast communications; the computing block usually includes multiple local multiplication and accumulation units, local caches, pooling units, quantization units and element-by-element calculation units. These units are not many in number and can be connected through a shared bus. The local cache is used to cache a certain number of activation values ​​for data reuse and reduce external storage access. The pooling unit can perform maximum pooling or average pooling operations, the quantization unit performs low-bit operations on the output activation values, and the element-by-element calculation unit performs the sum of the calculation results of multiple local multiplication and accumulation units; the local multiplication and accumulation unit includes multiple SRAM arrays that can execute calculations, and these SRAM arrays are organized according to fixed data paths to complete a vector multiplication matrix operation of a fixed shape; the SRAM array is the charge domain SRAM storage and calculation integrated unit introduced above.

[0005] Existing charge domain CIM neural network accelerators face the problem of high ADC energy consumption. First, the reasoning accuracy of neural networks is very sensitive to the ADC resolution. Experiments have shown that the reasoning accuracy of neural networks drops sharply as the ADC resolution decreases. In CIM neural network accelerators integrated with SRAMs of shape Row×Column, the ADC resolution must be kept at log2(Row) to avoid output overflow (i.e., larger values ​​are truncated to a certain upper bound value). Although signed CIMs can limit the output voltage to a narrow range through saturation calculations to mitigate the overflow problem caused by low-resolution ADCs, they only show advantages on impractical small neural networks or uncommon binary neural networks (BNNs). In contrast, when testing multiple representative DNNs (e.g., ResNet50) on SRAM arrays of shape 288×256, we found that the design with saturation calculations can only reduce the ADC resolution from 8 bits to 6 bits, otherwise, unless expensive neural network retraining is performed, the neural network reasoning accuracy will be significantly reduced (e.g., a 5-bit ADC reduces the DNN accuracy by 4.0% on average). However, high-resolution ADCs limit energy efficiency. Even with the compliant CIM and saturation calculations that reduce the ADC resolution to 6 bits, the ADC still consumes the most energy (64%) than any other component in a charge domain SRAM-based CIM architecture. Previous designs have mitigated the ADC energy at the expense of area overhead and performance loss. For example, using timing offset calibration reduces the ADC energy by 34.3%, but adds 25× area overhead, so to achieve 256TOPS peak throughput using this interleaved SAR ADC in a CIM architecture, the chip area would explode to about 1000mm 2 , which is unacceptable; pipelining and data parallelism are used to solve the energy efficiency problem of ADC, however, this will increase the ADC latency and thus lose performance. In summary, in order to improve the energy efficiency of high-performance charge domain SRAM-CIM neural network accelerators without reducing accuracy, a low-resolution ADC without area overhead and performance loss is urgently needed. Summary of the invention

[0006] The purpose of the present invention is to solve the ADC energy consumption bottleneck problem of the above-mentioned prior art, and propose a Fibonacci-coded storage-computation integrated unit and a Fibonacci-coded sparse computing storage-computation integrated system architecture. The Fibonacci-coded storage-computation integrated unit can reduce the range of calculation results to reduce the ADC resolution, and is combined with sparse computing and integrated into a high-throughput storage-computation integrated neural network accelerator to reduce the accumulated energy consumption and improve energy efficiency.

[0007] In view of the shortcomings of existing technologies, such as Fig.10As shown, the present invention proposes a storage-computation-integrated neural network acceleration method based on Fibonacci coding, which includes:

[0008] The initial step is to build a storage-computation neural network accelerator including a Fibonacci encoder to obtain the neural network computing tasks to be executed;

[0009] Operation steps: the Fibonacci encoder of the storage-computation integrated neural network accelerator converts the activation value and weight value participating in the neural network operation task from binary code to Fibonacci code to obtain Fibonacci activation and Fibonacci weight; the DAC of the storage-computation integrated unit in the storage-computation integrated neural network accelerator converts the Fibonacci activation into multiple analog voltages, and applies them to the word lines of each row of the SRAM array in the storage-computation integrated unit respectively; each unit of the SRAM array multiplies the input analog voltage by the corresponding Fibonacci weight to obtain an intermediate operation result; the adder of each column of the SRAM array redistributes the charge of the intermediate operation result to complete the accumulation operation of the inner product, and converts the accumulation operation result into a Fibonacci coded digital format through the ADC of the storage-computation integrated unit as the operation result of the neural network operation task.

[0010] The storage-computation-in-one neural network acceleration method based on Fibonacci coding, wherein the storage-computation-in-one unit comprises an SRAM absolute value unit, an SRAM symbol unit, and a DAC, an ADC, the Fibonacci encoder and a Fibonacci coded adder;

[0011] The SRAM symbol unit and the SRAM absolute value unit store weight values ​​in Fibonacci encoding format; the Fibonacci encoder converts binary format numbers into Fibonacci encoding format using a hardware lookup table, and the adder adds intermediate calculation results of multiple SRAM columns into a multi-bit inner product result.

[0012] The storage-computation-in-one neural network acceleration method based on Fibonacci coding, wherein

[0013] A plurality of the SRAM absolute value units and the SRAM symbol units form a Fibonacci-coded weight array; the bit line outputs of the absolute value units of multiple columns share the same ADC; the word lines of the SRAM absolute value units and the SRAM symbol units of each row share a DAC;

[0014] The Fibonacci encoder includes multiple lookup tables and a multiplexer. Each lookup table converts an input binary number into a Fibonacci encoding format multiplied by a fixed number. The encoder only activates one of the lookup tables each time it works. The output of the activated lookup table is selected by the MUX as the output of the Fibonacci encoder.

[0015] The storage-computation-in-one neural network acceleration method based on Fibonacci coding, wherein the storage-computation-in-one neural network accelerator comprises: a chip, a computing block, and a storage-computation-in-one unit composed of an accumulator and an SRAM array;

[0016] Each chip includes at least one computing block and at least one Fibonacci-encoded cache, and the computing blocks are connected through an on-chip network; each computing block includes at least one local multiplication and accumulation unit, a computing block local cache, a pooling unit, a compressor and a decompressor; each local multiplication and accumulation unit includes at least one input shift register, a zero value detector, at least one element-by-element operator, at least one accumulator and at least one SRAM array;

[0017] And multiple SRAM arrays share one accumulator, the outputs of the multiple SRAM arrays are first added in the binary domain, and then the addition result is input into the Fibonacci encoder, and the output of the encoder is then input into the adder of the subsequent Fibonacci encoding;

[0018] The zero detection component of the activation value performs a multi-input OR-NOT operation in a column absolute value bit of multiple activation values ​​to calculate the AZERO signal indication, which is used to indicate whether the current column bits are all zero, and broadcasts the AZERO signal indication to the adjacent SRAM array for input sharing; the ADC output zero detection component is equipped in the SRAM array, consisting of a row of OR gates and a row of 1-bit registers, the number of OR gates and registers is the same as the number of SRAM columns, and each bit register indicates whether a column weight is all zero; when the AZERO signal indication represents that all activation value input bits are zero, the ADC conversion process is skipped and the output value of the column is directly determined to be zero.

[0019] like Fig.11 As shown, the present invention also proposes a storage-computation-in-one neural network accelerator based on Fibonacci coding, which includes:

[0020] The initial module builds a storage-computation neural network accelerator including a Fibonacci encoder to obtain the neural network computing tasks to be executed;

[0021] The computing module comprises a Fibonacci encoder of the storage-computation-in-one neural network accelerator which converts the activation values ​​and weight values ​​involved in the neural network computing task from binary codes to Fibonacci codes to obtain Fibonacci activations and Fibonacci weights. The DAC of the storage-computation-in-one unit in the storage-computation-in-one neural network accelerator converts the Fibonacci activations into multiple analog voltages, and applies them to the word lines of each row of the SRAM array in the storage-computation-in-one unit respectively. Each unit of the SRAM array multiplies the input analog voltage by the corresponding Fibonacci weight to obtain an intermediate computing result. The adder of each column of the SRAM array redistributes the charge of the intermediate computing result to complete the accumulation operation of the inner product, and converts the accumulation operation result into a Fibonacci-coded digital format through the ADC of the storage-computation-in-one unit as the computing result of the neural network computing task.

[0022] The storage-computation integrated neural network accelerator based on Fibonacci coding, wherein the storage-computation integrated unit comprises an SRAM absolute value unit, an SRAM symbol unit, and a DAC, an ADC, the Fibonacci encoder and a Fibonacci coded adder;

[0023] The SRAM symbol unit and the SRAM absolute value unit store weight values ​​in Fibonacci encoding format; the Fibonacci encoder converts binary format numbers into Fibonacci encoding format using a hardware lookup table, and the adder adds intermediate calculation results of multiple SRAM columns into a multi-bit inner product result.

[0024] The storage-computation-in-one neural network accelerator based on Fibonacci coding, wherein

[0025] A plurality of the SRAM absolute value units and the SRAM symbol units form a Fibonacci-coded weight array; the bit line outputs of the absolute value units of multiple columns share the same ADC; the word lines of the SRAM absolute value units and the SRAM symbol units of each row share a DAC;

[0026] The Fibonacci encoder includes a plurality of lookup tables and a multiplexer, each of which converts an input binary number into a Fibonacci encoding format multiplied by a fixed number; the encoder activates only one of the lookup tables each time it works; the output of the activated lookup table is selected by the MUX as the output of the Fibonacci encoder;

[0027] The storage-computation integrated neural network accelerator includes: a chip, a computing block, and a storage-computation integrated unit consisting of an accumulator and an SRAM array;

[0028] Each chip includes at least one computing block and at least one Fibonacci-encoded cache, and the computing blocks are connected through an on-chip network; each computing block includes at least one local multiplication and accumulation unit, a computing block local cache, a pooling unit, a compressor and a decompressor; each local multiplication and accumulation unit includes at least one input shift register, a zero value detector, at least one element-by-element operator, at least one accumulator and at least one SRAM array;

[0029] And multiple SRAM arrays share one accumulator, the outputs of the multiple SRAM arrays are first added in the binary domain, and then the addition result is input into the Fibonacci encoder, and the output of the encoder is then input into the adder of the subsequent Fibonacci encoding;

[0030] The zero detection component of the activation value performs a multi-input OR-NOT operation in a column absolute value bit of multiple activation values ​​to calculate the AZERO signal indication, which is used to indicate whether the current column bits are all zero, and broadcasts the AZERO signal indication to the adjacent SRAM array for input sharing; the ADC output zero detection component is equipped in the SRAM array, consisting of a row of OR gates and a row of 1-bit registers, the number of OR gates and registers is the same as the number of SRAM columns, and each bit register indicates whether a column weight is all zero; when the AZERO signal indication represents that all activation value input bits are zero, the ADC conversion process is skipped and the output value of the column is directly determined to be zero.

[0031] The present invention also proposes an electronic device, which includes the above-mentioned storage and computing integrated neural network accelerator, and the electronic device is connected to an information display device, which is used to display the evaluation results according to display parameters and attributes set by the user or through an artificial intelligence model.

[0032] The present invention also proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the storage-computing integrated neural network acceleration methods are implemented.

[0033] The present invention also proposes a computer program product, including a computer program, wherein when the computer program is executed by a processor, the steps of any of the storage-computation-in-one neural network acceleration methods are implemented.

[0034] It can be seen from the above scheme that the advantages of the present invention are:

[0035] The energy efficiency of the present invention is 1.62 times that of the prior art. The energy efficiency improvement comes from Fibonacci coding and sparsity support. First, Fibonacci coding concentrates the range of ADC input and avoids data overflow errors, so that low-resolution ADC can be used. Second, sparsity support skips redundant zero calculations and corrects the defects caused by more "0". In contrast, the prior art uses a high-resolution ADC to avoid data overflow errors in binary coding, resulting in significant energy consumption overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a prior art solution diagram;

[0037] Figure 2 A diagram of a storage-computation-in-one unit for Fibonacci coding;

[0038] Figure 3 It is the structural diagram of DAC;

[0039] Figure 4 This is the structure diagram of the Fibonacci encoder;

[0040] Figure 5 Adding machine and skull diagram for Fibonacci code;

[0041] Figure 6 Diagram of the integrated computing and storage architecture for Fibonacci-encoded sparse computing;

[0042] Figure 7 Skip the structure diagram for zero output of ADC;

[0043] Figure 8 Skip the structure diagram for zero input to addition;

[0044] Fig. 9 is a diagram of data compression method;

[0045] Fig.10 is a flow chart of the method of the present invention;

[0046] Fig.11 It is a module diagram of the device of the present invention;

[0047] Fig.12 This is a schematic diagram of the structure of a first electronic device of the present invention;

[0048] Fig.13 This is a schematic diagram of the application environment structure of the first electronic device of the present invention;

[0049] Fig.14 It is a schematic diagram of the structure of a second electronic device of the present invention.

[0050] Reference numerals:

[0051] A-First electronic device;

[0052] B-A neural network accelerator based on Fibonacci coding and storage and computing;

[0053] C-data acquisition equipment;

[0054] D-information display device;

[0055] 1000 - second electronic device;

[0056] Ⅰ-computational unit;

[0057] II-ROM;

[0058] III-RAM;

[0059] IV-bus;

[0060] V-interface;

[0061] VI - input unit;

[0062] VII-output unit;

[0063] VIII- Storage medium;

[0064] Ⅸ-Communication unit. DETAILED DESCRIPTION

[0065] It should be noted that, in this application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0066] Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0067] The processor described in the present invention is the control center of the electronic device, which can be a processor or a general term for multiple processing elements. For example, it can be one or more central processing units (CPUs), or application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).

[0068] Optionally, the processor can perform various functions of the electronic device by running or executing a software program stored in the memory, and calling data stored in the memory.

[0069] In a specific implementation, as an embodiment, the processor may include one or more CPUs. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions). Electronic devices may include: servers, desktop computers, laptops, smart phones, tablet computers, embedded computers, etc., wherein the embedded computers include vehicles and robots, etc.

[0070] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0071] It should be noted that the structure of the electronic device shown in the drawings of the present invention does not constitute a limitation thereto, and the actual knowledge structure recognition device may include more or fewer components than shown in the drawings, or a combination of certain components, or a different arrangement of components.

[0072] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0073] It should also be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0074] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0075] It should also be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0076] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0077] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0078] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0079] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0080] When conducting research on the charge domain storage and computing integrated neural network accelerator, the present invention discovered that the high energy consumption of the ADC in the prior art is caused by the excessively high resolution of the ADC. The existing solutions focus on optimizing the design and implementation of the ADC itself at a given resolution, while ignoring the reduction of the resolution of the ADC through overall design at the system level, thereby reducing the energy consumption of the ADC. The reason for the high resolution of the ADC is that the density of "1" in the existing binary encoding method is relatively high, which makes the output voltage of each column (i.e., the input voltage of the ADC) vary widely, and thus a high-resolution ADC is required to complete the analog-to-digital conversion operation. How to find an innovative encoding method that reduces the density of "1" in each number becomes the technical difficulty of the present invention.

[0081] The present invention has found through research on digital coding that this defect can be solved by Fibonacci coding. Fibonacci coding is a coding method different from binary. Under Fibonacci coding, there are no two consecutive "1", so that the density of "1" is reduced. A lower density of "1" is conducive to reducing the input voltage range of the ADC, thereby reducing the resolution of the ADC and reducing the power consumption of the ADC.

[0082] The present invention utilizes Fibonacci coding and proposes a Fibonacci coding storage and calculation integrated unit. The Fibonacci coding storage and calculation integrated unit changes the activation value and weight involved in the operation from binary coding to Fibonacci coding. In addition, the output of the ADC is also converted from a binary format to a Fibonacci coding format by the encoder, and is accumulated by a subsequent Fibonacci coding adder.

[0083] The present invention combines the Fibonacci-coded storage-computation integrated unit with sparse support, and invents a Fibonacci-coded sparse computing storage-computation integrated architecture. Since Fibonacci coding increases the density of "0", it is more suitable to use sparse technology to further reduce energy consumption. Specifically, this architecture further reduces the ADC overhead by skipping all-0 activation value columns or all-0 weight columns, and reduces the overhead of the Fibonacci-coded adder by skipping zeros, and reduces the bandwidth of storage and communication by compression at the bit slice level.

[0084] In order to make the above features and effects of the present invention more clearly and understandably described, embodiments are given below and described in detail with reference to the accompanying drawings. This specification discloses one or more embodiments that include the features of the present invention. The disclosed embodiments are only for illustration. The scope of protection of the present invention is not limited to the disclosed embodiments, and the present invention is defined by the attached claims.

[0085] Figure 2The architecture diagram of the Fibonacci-coded storage and computing integrated unit provided in the first embodiment of the present invention is shown. The Fibonacci-coded storage and computing integrated unit can calculate the activation value and weight of the Fibonacci code to improve energy efficiency. The structure of the Fibonacci-coded storage and computing integrated unit is as follows:

[0086] A SRAM storage and computing integrated unit comprises a plurality of SRAM absolute value units, a plurality of SRAM symbol units, a plurality of digital-to-analog converters (DACs), at least one analog-to-digital converter (ADC), at least one Fibonacci encoder and at least one Fibonacci coded adder; each column of the plurality of SRAM symbol units and the plurality of columns of the SRAM absolute value units stores a plurality of weights in the format of Fibonacci coding represented by the original code; the key difference between Fibonacci coding and binary coding is that any position of the absolute value of a number does not contain two consecutive "1"s; the Fibonacci encoder uses a hardware lookup table to convert a binary format number read out by the ADC into a Fibonacci coded format multiplied by a fixed number, and the Fibonacci coded adder adds the results of a plurality of one-bit inner products into a result corresponding to a multi-bit inner product.

[0087] Further references Figure 2 As shown in, in this embodiment, multiple SRAM absolute value units and SRAM symbol units form a Fibonacci-coded weight array. For example, each weight contains a 10-bit absolute value and a 1-bit weight, so each column of symbol units will be accompanied by 10 columns of absolute value units; the bit line outputs of multiple columns of absolute value units can share the same ADC; the word lines of each row of SRAM absolute value units and SRAM symbol units share a DAC.

[0088] In this embodiment, the structure of DAC is as follows: Figure 3 As shown. The input of the DAC is 1 bit, which represents 1 bit in the absolute value of the activation value, and the output of the DAC is two voltages Vx and Vdd-Vx. If the input bit of the DAC is '0', then Vx is Vdd / 2, and Vdd-Vx is also Vdd / 2; if the input bit of the DAC is '1', then Vx is Vdd, and Vdd-Vx is GND.

[0089] In this embodiment, the structure of the Fibonacci encoder is as follows: Figure 4 As shown, it includes multiple lookup tables and a multiple selector (MUX). Each lookup table converts the input binary number into a Fibonacci encoding format multiplied by a fixed number; the encoder only activates one of the lookup tables each time it works; the output of the activated lookup table is selected by the MUX as the output of the Fibonacci encoder.

[0090] In this embodiment, Figure 5As shown, the Fibonacci coded adder is designed based on the algorithm of signed addition of the original Fibonacci coded data (i.e., the sign and the absolute value) in . The algorithm mainly uses a bit-by-bit addition and subtraction and four sliding windows to calculate the addition result. Based on the algorithm, the bit-by-bit adder is composed of a number of 1-bit adders and subtractors, each adder / subtractor has a 2-bit input and a 2-bit output; followed by four stages, each corresponding to the four stages of the sliding window algorithm. Since the Fibonacci coded data is in the original code format, the Fibonacci coded adder needs to first determine whether it is addition or subtraction based on the sign and size of the input. Then, if subtraction is selected, stage 0 executes the first sliding window, otherwise, stage 0 is skipped and the input is directly forwarded to stage 1. Each stage is composed of multiple window adders (WA), each of which converts a number of input bits into outputs of the same number of bits according to certain rules. In each stage, except for the first window adder, each window adder takes one bit of the output of the previous stage and several bits of the output of the previous window adder of the current stage as input, and produces one bit of output and some intermediate bits as part of the input of the next window adder. It should be noted that according to the literature, in stage 1, the output of the last window adder requires a cleanup operation. All transformation rules of each window adder follow the algorithm in.

[0091] In this embodiment, the Fibonacci-coded storage-computation integrated unit is calculated in the manner of activation value bit serial-weight bit parallel. A calculation of the Fibonacci-coded storage-computation integrated unit includes four steps: activation value input, SRAM unit calculation, analog domain charge redistribution, and ADC analog-to-digital conversion. Among them, each bit of the absolute value of a column of activation values ​​is calculated serially with all bits of the weight, and each time a column of activation values ​​is input, one bit of each activation value and the sign bit of each activation value, wherein the absolute value bit is converted into two voltages by the DAC, and injected into the SRAM array together with the sign bit for charge domain analog calculation; the SRAM unit calculation includes the sign unit calculating the sign of each multiplication result, and selecting the appropriate voltage (when the multiplication result is positive, select Vx; when the multiplication result is negative, select Vdd-Vx, when the multiplication result is 0, select Vdd / 2); the analog domain charge redistribution includes performing voltage averaging operation on each column through the physical law of charge redistribution on the selected voltage; finally, the ADC converts the voltage calculated for each column into a digital domain representation.

[0092] In addition, this embodiment can also be applied to a storage and computing integrated unit in the current domain.

[0093] In summary, the Fibonacci-coded storage and computing integrated unit provided in this embodiment can calculate the activation value and weight of the Fibonacci code to improve the energy efficiency of the neural network inference accelerator.

[0094] Further, based on the above-mentioned Fibonacci-coded storage-computing integrated unit, this embodiment 2 further provides a Fibonacci-coded sparse computing storage-computing integrated system architecture, such as Figure 6 As shown in . Overall, the architecture includes four levels: chip, computing block, local multiplication and accumulation unit and SRAM array. Each chip contains at least one computing block and at least one Fibonacci-encoded cache, and the computing blocks are connected through an on-chip network. Each computing block contains at least one local multiplication and accumulation unit, a computing block local cache, a pooling unit, a compressor and a decompressor. Each local multiplication and accumulation unit has at least one input shift register, a zero value detector, at least one element-by-element operator, at least one accumulator and at least one SRAM array, wherein the combination of the accumulator (including at least one Fibonacci encoder and at least one Fibonacci-encoded adder) and the SRAM array is the aforementioned Fibonacci-encoded storage and calculation integrated unit.

[0095] It should be noted that in order to reduce the overhead of the accumulator, in this embodiment, every two SRAM arrays share one accumulator. The specific method is: the outputs of the two SRAM arrays are first added in the binary domain, and then the addition result is input into the Fibonacci encoder, and the output of the encoder is then input into the subsequent Fibonacci encoded adder.

[0096] In order to skip the ADC conversion process of zero output, this embodiment implements two components responsible for zero detection of activation value and zero detection of ADC output, such as Figure 7 As shown. The zero detection component of the activation value performs a multi-input NOR operation in a column absolute value bit of multiple activation values ​​to calculate AZERO, which indicates whether the bits in this column are all zero, and broadcasts AZERO to the adjacent SRAM array for input sharing. The ADC output zero detection component is equipped in the SRAM array, consisting of a row of OR gates and a row of 1-bit registers. The number of OR gates and registers is the same as the number of columns of SRAM, and each bit of the register indicates whether a column of weights is all zero. When all activation value input bits are zero (that is, AZERO is 1) or a column of weight bits is zero (WZERO is 1), the output value of the column can be determined to be zero in advance, thereby avoiding an ADC conversion process. Since the weights are fixed, all WZERO can be calculated in advance before the overall execution starts. Each column of SRAM has a WZERO value. If the column of SRAM is all 0, WZERO is 1, otherwise WZERO is 0.

[0097] Fibonacci Coded Adder with Zero-Input Skip The Fibonacci coded adder is equipped with zero-input skip logic, such as Figure 8As shown. To support zero input skipping, the Fibonacci coded adder takes inputs from two registers whose values ​​are updated only when both inputs are non-zero. When there is a zero input, the values ​​in both registers remain unchanged, so the input to the Fibonacci coded adder remains unchanged. Since the Fibonacci coded adder is a combinational circuit, it does not consume dynamic energy. According to the addition rule, when one input is zero, the addition result is equal to the other input. Therefore, a 3-to-1 MUX circuit is used to select the addition result from the two inputs and the output of the Fibonacci coded adder.

[0098] To compensate for the additional storage capacity and interconnect bandwidth requirements due to the larger bit width of the Fibonacci encoded data, we utilize a bit-slicing based compression method to reduce the external cache capacity and interconnect bandwidth requirements. Fig. 9 As shown, this compression method includes two stages, namely, a bit slice length reduction stage (for reducing the length of each slice) and a mixed compression stage (for reducing the length of multiple slices).

[0099] Bit slice length reduction stage: Since there are no consecutive '1's in Fibonacci encoding, taking a 4-bit slice as an example, the slice can only be one of eight cases: "0000", "0001", "0010", "0100", "1000", "1010", "0101", and "1001". We can define a conversion rule to represent a 4-bit slice as a 3-bit index, such as "000" code "0000" and "111" code "1001". In this way, we compress the length of each slice from 4 bits to 3 bits during data transmission and storage, and decompress it during calculation.

[0100] Hybrid Compression Stage: We classify a set of slices into high sparsity, medium sparsity, and low sparsity based on the compression ratio. The hybrid compression method utilizes three different strategies, namely run-length encoding compression, bitmap compression, and no compression, which correspond to high sparsity, medium sparsity, and low sparsity levels. For low sparsity slices, we do not compress them to avoid overhead. For high sparsity slices, we choose run-length encoding compression to maximize the compression ratio. For medium sparsity slices, we use bitmap compression to balance the overhead and compression ratio.

[0101] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment. In order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiment.

[0102] like Fig.11As shown, the present invention also proposes a storage-computation-in-one neural network accelerator based on Fibonacci coding, which includes:

[0103] The initial module builds a storage-computation neural network accelerator including a Fibonacci encoder to obtain the neural network computing tasks to be executed;

[0104] The computing module comprises a Fibonacci encoder of the storage-computation-in-one neural network accelerator which converts the activation values ​​and weight values ​​involved in the neural network computing task from binary codes to Fibonacci codes to obtain Fibonacci activations and Fibonacci weights. The DAC of the storage-computation-in-one unit in the storage-computation-in-one neural network accelerator converts the Fibonacci activations into multiple analog voltages, and applies them to the word lines of each row of the SRAM array in the storage-computation-in-one unit respectively. Each unit of the SRAM array multiplies the input analog voltage by the corresponding Fibonacci weight to obtain an intermediate computing result. The adder of each column of the SRAM array redistributes the charge of the intermediate computing result to complete the accumulation operation of the inner product, and converts the accumulation operation result into a Fibonacci-coded digital format through the ADC of the storage-computation-in-one unit as the computing result of the neural network computing task.

[0105] The storage-computation integrated neural network accelerator based on Fibonacci coding, wherein the storage-computation integrated unit comprises an SRAM absolute value unit, an SRAM symbol unit, and a DAC, an ADC, the Fibonacci encoder and a Fibonacci coded adder;

[0106] The SRAM symbol unit and the SRAM absolute value unit store weight values ​​in Fibonacci encoding format; the Fibonacci encoder converts binary format numbers into Fibonacci encoding format using a hardware lookup table, and the adder adds intermediate calculation results of multiple SRAM columns into a multi-bit inner product result.

[0107] The storage-computation-in-one neural network accelerator based on Fibonacci coding, wherein

[0108] A plurality of the SRAM absolute value units and the SRAM symbol units form a Fibonacci-coded weight array; the bit line outputs of the absolute value units of multiple columns share the same ADC; the word lines of the SRAM absolute value units and the SRAM symbol units of each row share a DAC;

[0109] The Fibonacci encoder includes a plurality of lookup tables and a multiplexer, each of which converts an input binary number into a Fibonacci encoding format multiplied by a fixed number; the encoder activates only one of the lookup tables each time it works; the output of the activated lookup table is selected by the MUX as the output of the Fibonacci encoder;

[0110] The storage-computation integrated neural network accelerator includes: a chip, a computing block, and a storage-computation integrated unit consisting of an accumulator and an SRAM array;

[0111] Each chip includes at least one computing block and at least one Fibonacci-encoded cache, and the computing blocks are connected through an on-chip network; each computing block includes at least one local multiplication and accumulation unit, a computing block local cache, a pooling unit, a compressor and a decompressor; each local multiplication and accumulation unit includes at least one input shift register, a zero value detector, at least one element-by-element operator, at least one accumulator and at least one SRAM array;

[0112] And multiple SRAM arrays share one accumulator, the outputs of the multiple SRAM arrays are first added in the binary domain, and then the addition result is input into the Fibonacci encoder, and the output of the encoder is then input into the adder of the subsequent Fibonacci encoding;

[0113] The zero detection component of the activation value performs a multi-input OR-NOT operation in a column absolute value bit of multiple activation values ​​to calculate the AZERO signal indication, which is used to indicate whether the current column bits are all zero, and broadcasts the AZERO signal indication to the adjacent SRAM array for input sharing; the ADC output zero detection component is equipped in the SRAM array, consisting of a row of OR gates and a row of 1-bit registers, the number of OR gates and registers is the same as the number of SRAM columns, and each bit register indicates whether a column weight is all zero; when the AZERO signal indication represents that all activation value input bits are zero, the ADC conversion process is skipped and the output value of the column is directly determined to be zero.

[0114] like Fig.12 As shown, the present invention further proposes a first electronic device A in another embodiment, which includes the aforementioned storage and computing integrated neural network accelerator.

[0115] like Fig.13 As shown, the first electronic device A can also be connected to the data acquisition device C and the information display device D through a wired or wireless information transmission scheme. The data acquisition device C is used to collect neural network operation tasks, such as face recognition tasks, image classification tasks, weather forecasting tasks, etc. based on neural network models, and the information display device D is used to display the task reasoning results, that is, the operation results, obtained by the analysis of the present invention.

[0116] The information display device D can sort and process the data output by the first electronic device A based on the information display mechanism to improve the readability of the data output by the first electronic device A. The information display mechanism can be manually preset, for example, the data output by the first electronic device A is visually displayed, which can be based on the display parameters and / or attributes set by the user. The display parameters can be, for example, the display data range, and the display attributes can be, for example, the display font, color, whether to scroll and play, etc. The user is presented with the key information specified by the user, such as the image classification results and weather forecast results in the calculation results, so that the user can understand this information more timely without having to access the secondary page or scroll the page, saving the user's operation. Or the information display mechanism can be an artificial intelligence AI display model, which can learn the user's key information based on the user's previous usage habits, such as viewing time, number of clicks, number of edits, etc., and then automatically present rich and necessary key information to the user.

[0117] The present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a readable storage medium. When the computer program is executed by a processor, the computer can execute the storage-computing integrated neural network acceleration method provided by the above methods.

[0118] The present invention also proposes a storage medium VIII in another embodiment for storing a computer program for executing the storage-computation-in-one neural network acceleration method. It should be understood that the storage medium in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DRRAM).

[0119] Fig.14 A schematic block diagram of a second electronic device 1000 that can be used to implement an embodiment of the present invention is shown. The second electronic device 1000 electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The second electronic device 1000 can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present invention described and / or required herein. The second electronic device 1000 may be the same or different from the first electronic device A.

[0120] The second electronic device 1000 includes a computing unit I, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory II (ROM) or a computer program loaded from a storage medium VIII into a random access memory (RAM) III. In RAM III, various programs and data required for the operation of the device 1000 can also be stored. The computing unit I, ROM II, and RAM III are connected to each other via a bus IV. An input / output (I / O) interface V is also connected to the bus IV.

[0121] A plurality of components in the second electronic device 1000 are connected to the I / O interface V, including: an input unit VI, such as a keyboard, a mouse, etc.; an output unit VII, such as various types of displays, speakers, etc.; a storage medium VIII, such as a disk, an optical disk, etc.; and a communication unit IX, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit IX allows the second electronic device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0122] The computing unit I may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of computing unit I include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit I performs the various methods and processes described above, such as method steps S1-S2. For example, in some embodiments, the method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage medium VIII. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1000 via ROM II and / or a communication unit IX. When the computer program is loaded into RAM III and executed by the computing unit I, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit I may be configured to execute the method in any other appropriate manner (e.g., by means of firmware).

[0123] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. A storage-computation-integrated neural network acceleration method based on Fibonacci coding, characterized in that: include: The initial step is to build a storage-computation neural network accelerator including a Fibonacci encoder to obtain the neural network computing tasks to be executed; Operation steps: the Fibonacci encoder of the storage-computation integrated neural network accelerator converts the activation value and weight value participating in the neural network operation task from binary code to Fibonacci code to obtain Fibonacci activation and Fibonacci weight; the DAC of the storage-computation integrated unit in the storage-computation integrated neural network accelerator converts the Fibonacci activation into multiple analog voltages, and applies them to the word lines of each row of the SRAM array in the storage-computation integrated unit respectively; each unit of the SRAM array multiplies the input analog voltage by the corresponding Fibonacci weight to obtain an intermediate operation result; the adder of each column of the SRAM array redistributes the charge of the intermediate operation result to complete the accumulation operation of the inner product, and converts the accumulation operation result into a Fibonacci coded digital format through the ADC of the storage-computation integrated unit as the operation result of the neural network operation task.

2. The storage-computation-in-one neural network acceleration method based on Fibonacci coding as claimed in claim 1, characterized in that: The storage-computation-in-one unit comprises an SRAM absolute value unit, an SRAM symbol unit, a DAC, an ADC, the Fibonacci encoder and a Fibonacci-encoded adder; The SRAM symbol unit and the SRAM absolute value unit store weight values ​​in Fibonacci encoding format; the Fibonacci encoder converts binary format numbers into Fibonacci encoding format using a hardware lookup table, and the adder adds intermediate calculation results of multiple SRAM columns into a multi-bit inner product result.

3. The storage-computation-in-one neural network acceleration method based on Fibonacci coding as claimed in claim 2, characterized in that: A plurality of the SRAM absolute value units and the SRAM symbol units form a Fibonacci-coded weight array; the bit line outputs of the absolute value units of multiple columns share the same ADC; the word lines of the SRAM absolute value units and the SRAM symbol units of each row share a DAC; The Fibonacci encoder includes a plurality of lookup tables and a multiple selector, each of which converts an input binary number into a Fibonacci encoding format multiplied by a fixed number; The encoder only activates one of the lookup tables each time it works; the output of the activated lookup table is selected by the MUX as the output of the Fibonacci encoder.

4. The storage-computation-in-one neural network acceleration method based on Fibonacci coding as claimed in claim 1 or 2, characterized in that: The storage-computation integrated neural network accelerator includes: a chip, a computing block, and a storage-computation integrated unit consisting of an accumulator and an SRAM array; Each chip includes at least one computing block and at least one Fibonacci-encoded cache, and the computing blocks are connected through an on-chip network; each computing block includes at least one local multiplication and accumulation unit, a computing block local cache, a pooling unit, a compressor and a decompressor; each local multiplication and accumulation unit includes at least one input shift register, a zero value detector, at least one element-by-element operator, at least one accumulator and at least one SRAM array; And multiple SRAM arrays share one accumulator, the outputs of the multiple SRAM arrays are first added in the binary domain, and then the addition result is input into the Fibonacci encoder, and the output of the encoder is then input into the adder of the subsequent Fibonacci encoding; The zero detection component of the activation value performs a multi-input OR-NOT operation in a column absolute value bit of multiple activation values ​​to calculate the AZERO signal indication, which is used to indicate whether the current column bits are all zero, and broadcasts the AZERO signal indication to the adjacent SRAM array for input sharing; the ADC output zero detection component is equipped in the SRAM array, consisting of a row of OR gates and a row of 1-bit registers, the number of OR gates and registers is the same as the number of SRAM columns, and each bit register indicates whether a column weight is all zero; when the AZERO signal indication represents that all activation value input bits are zero, the ADC conversion process is skipped and the output value of the column is directly determined to be zero.

5. A storage and computing integrated neural network accelerator based on Fibonacci coding, characterized in that: include: The initial module builds a storage-computation neural network accelerator including a Fibonacci encoder to obtain the neural network computing tasks to be executed; The computing module comprises a Fibonacci encoder of the storage-computation-in-one neural network accelerator which converts the activation values ​​and weight values ​​involved in the neural network computing task from binary codes to Fibonacci codes to obtain Fibonacci activations and Fibonacci weights. The DAC of the storage-computation-in-one unit in the storage-computation-in-one neural network accelerator converts the Fibonacci activations into multiple analog voltages, and applies them to the word lines of each row of the SRAM array in the storage-computation-in-one unit respectively. Each unit of the SRAM array multiplies the input analog voltage by the corresponding Fibonacci weight to obtain an intermediate computing result. The adder of each column of the SRAM array redistributes the charge of the intermediate computing result to complete the accumulation operation of the inner product, and converts the accumulation operation result into a Fibonacci-coded digital format through the ADC of the storage-computation-in-one unit as the computing result of the neural network computing task.

6. The Fibonacci-encoded integrated storage and computing neural network accelerator according to claim 5, characterized in that: The storage-computation-in-one unit comprises an SRAM absolute value unit, an SRAM symbol unit, a DAC, an ADC, the Fibonacci encoder and a Fibonacci-encoded adder; The SRAM symbol unit and the SRAM absolute value unit store weight values ​​in Fibonacci encoding format; the Fibonacci encoder converts binary format numbers into Fibonacci encoding format using a hardware lookup table, and the adder adds intermediate calculation results of multiple SRAM columns into a multi-bit inner product result.

7. The Fibonacci-encoded integrated storage and computing neural network accelerator according to claim 6, characterized in that: A plurality of the SRAM absolute value units and the SRAM symbol units form a Fibonacci-coded weight array; the bit line outputs of the absolute value units of multiple columns share the same ADC; the word lines of the SRAM absolute value units and the SRAM symbol units of each row share a DAC; The Fibonacci encoder includes a plurality of lookup tables and a multiple selector, each of which converts an input binary number into a Fibonacci encoding format multiplied by a fixed number; The encoder only activates one of the lookup tables at a time; The output of the activated lookup table is selected by the MUX as the output of the Fibonacci encoder; The storage-computation integrated neural network accelerator includes: a chip, a computing block, and a storage-computation integrated unit consisting of an accumulator and an SRAM array; Each chip includes at least one computing block and at least one Fibonacci-encoded cache, and the computing blocks are connected through an on-chip network; each computing block includes at least one local multiplication and accumulation unit, a computing block local cache, a pooling unit, a compressor and a decompressor; each local multiplication and accumulation unit includes at least one input shift register, a zero value detector, at least one element-by-element operator, at least one accumulator and at least one SRAM array; And multiple SRAM arrays share one accumulator, the outputs of the multiple SRAM arrays are first added in the binary domain, and then the addition result is input into the Fibonacci encoder, and the output of the encoder is then input into the adder of the subsequent Fibonacci encoding; The zero detection component of the activation value performs a multi-input OR-NOT operation in a column absolute value bit of multiple activation values ​​to calculate the AZERO signal indication, which is used to indicate whether the current column bits are all zero, and broadcasts the AZERO signal indication to the adjacent SRAM array for input sharing; the ADC output zero detection component is equipped in the SRAM array, consisting of a row of OR gates and a row of 1-bit registers, the number of OR gates and registers is the same as the number of SRAM columns, and each bit register indicates whether a column weight is all zero; when the AZERO signal indication represents that all activation value input bits are zero, the ADC conversion process is skipped and the output value of the column is directly determined to be zero.

8. An electronic device, characterized in that: It includes a storage and computing integrated neural network accelerator as described in claims 5-7, and the electronic device is connected to an information display device, and the information display device is used to display the evaluation result according to display parameters and attributes set by the user or through an artificial intelligence model.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the storage-computation-in-one neural network acceleration method described in any one of claims 1-4.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the storage-computation-in-one neural network acceleration method described in any one of claims 1-4 are implemented.

Images