Static random access memory in-memory computing system and method

Abstract

The embodiment of the invention discloses a static random access memory in-memory computing system and method, and the system comprises a feature memory which is used for storing an input signal of a neural network model; the SRAM storage array is used for storing each weight of the neural network model and performing logical operation on an input signal and each weight to obtain a plurality of logical operation results, each weight is divided into a plurality of binary blocks after CSD coding, complementary mode blocks are reserved after processing of eliminating all-zero mode blocks, and each binary block is divided into a plurality of binary blocks; after information is extracted from each complementary mode block, an index, a symbol and a value are obtained, and each weight is stored in a complementary mode block mode; the metadata storage is used for storing indexes and symbols; the adder tree is used for performing decoding and additive operation on the plurality of logical operation results according to the indexes and the symbols; the shift accumulation unit is used for performing shift accumulation on the addition result; and the post-processing unit is used for post-processing the shift accumulation result to obtain an output signal of the neural network model.

Description

Technical Field

[0001] The present invention relates to the field of in-memory computing technology, and in particular to an in-memory computing system and method for a static random access memory. Background Art

[0002] With the rapid development of deep learning technology and the growing application demand, the pressure on computing resources has also increased. In-memory computing can significantly improve the parallelism and energy efficiency of the system by implementing computing operations in memory, avoiding the frequent transfer of data between computing units and storage units. In-memory computing based on static random access memory (SRAM) has the advantages of fast read and write speed, low write energy consumption, and compatibility with advanced processes.

[0003] At present, an analog SRAM in-memory computing architecture applied to neural network models can divide the storage cell array into sub-storage cell blocks, and through training, the weights stored in each sub-storage cell block are made all zero or non-all zero, thereby achieving block-wise sparseness of the weights of the neural network model to reduce the storage space and computing resources required for the model.

[0004] In the above analog SRAM in-memory computing architecture, the calculation results are converted into digital signals through the analog to digital converter (ADC) for output. Due to the limitation of ADC resolution, there will be a low degree of computational parallelism, resulting in reduced computational efficiency. Moreover, there are also a large number of irregularly distributed zero bits in non-all-zero values, which makes the utilization rate of the in-memory computing array low. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a static random access memory in-memory computing system and method to solve the problems in the prior art.

[0006] To solve the above technical problems, the embodiments of the present invention are implemented as follows: In a first aspect, an embodiment of the present invention provides an in-memory computing system of static random access memory, comprising: A feature memory for storing input signals of the neural network model; A static random access memory SRAM storage array is used to store various weights of the neural network model, and to perform logic operations on the input signal and the various weights to obtain multiple logic operation results, wherein the various weights of the neural network model are divided into multiple binary blocks after being encoded by a regular signed number CSD, and each binary block consists of two bits. After the multiple binary blocks are processed by eliminating the all-zero pattern blocks, complementary pattern blocks are retained, and each complementary pattern block includes two complementary bits. After information is extracted from each complementary pattern block, an index, a sign and a value are obtained. The various weights of the neural network model are stored in the form of complementary pattern blocks; A metadata memory for storing indices and symbols of complementary pattern blocks of respective weights of a neural network model; An adder tree, used for decoding the plurality of logic operation results according to the index and the symbol stored in the metadata memory, and then performing addition operation to obtain the addition result; A shift accumulation unit, used for performing shift accumulation on the addition result to obtain a shift accumulation result; A post-processing unit is used to post-process the shift accumulation result to obtain an output signal of the neural network model.

[0007] In a second aspect, an embodiment of the present invention provides a static random access memory in-memory computing method, comprising: Each weight of the neural network model is CSD-encoded and divided into multiple binary blocks, each of which consists of two bits; Eliminating all-zero pattern blocks in the plurality of binary blocks and retaining complementary pattern blocks, each complementary pattern block including two complementary bits; Extract information for each complementary pattern block to obtain index, sign and value; The weights of the neural network model are stored in the SRAM storage array in the form of complementary pattern blocks, and the index and symbol of the complementary pattern blocks of the weights of the neural network model are stored in the metadata memory; Performing logic operations on the input signal and each weight of the neural network model to obtain multiple logic operation results; Decoding the plurality of logic operation results according to the index and the sign, and then performing an addition operation to obtain an addition result; Performing shift accumulation on the addition result to obtain a shift accumulation result; The shift accumulation result is post-processed to obtain an output signal of the neural network model.

[0008] It can be seen from the technical solution provided by the above embodiment of the present invention that the embodiment of the present invention stores the input signal of the neural network model through the feature memory, stores the various weights of the neural network model through the SRAM storage array, performs logical operations on the input signal and the various weights to obtain multiple logical operation results, wherein the various weights of the neural network model are divided into multiple binary blocks after CSD encoding, each binary block consists of two bits, and the multiple binary blocks are processed to eliminate the all-zero pattern blocks to retain the complementary pattern blocks, each complementary pattern block includes two complementary bits, and each complementary pattern block is extracted after information is extracted to obtain an index, a symbol and a value, and the various weights of the neural network model are stored in the form of complementary pattern blocks, and the metadata memory stores the various weights of the neural network model. The adder tree decodes multiple logic operation results according to the index and sign stored in the metadata memory, and then performs addition operation to obtain the addition result. The shift-accumulation unit performs shift-accumulation on the addition result to obtain the shift-accumulation result. The post-processing unit performs post-processing on the shift-accumulation result to obtain the output signal of the neural network model. Since the digital SRAM in-memory computing architecture does not require ADC for analog-to-digital conversion and is not limited by ADC resolution, the parallelism and efficiency of the calculation are improved. Moreover, the weights based on CSD encoding are stored in the form of binary blocks, which realizes bit-level sparsity of the weights of the neural network model, greatly reduces the storage space and computing resources required for the model, and improves the utilization rate of the SRAM storage array. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0010] Figure 1 A schematic diagram of the structure of a static random access memory in-memory computing system provided by an embodiment of the present invention; Figure 2 A schematic diagram of a SRAM storage unit provided by an embodiment of the present invention; Figure 3 A schematic diagram of a binary multiplication calculation unit provided by an embodiment of the present invention; Figure 4 A schematic diagram of an adder tree provided by an embodiment of the present invention; Figure 5 A schematic diagram of a flow chart of a static random access memory in-memory computing method provided by an embodiment of the present invention; Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0011] The embodiments of the present invention provide a static random access memory in-memory computing system and method.

[0012] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0013] The static random access memory in-memory computing system and method provided by the embodiment of the present invention can be applied to an electronic device, which can be a terminal device or a server, or a microcontroller, etc. The server can be an independent server, or a server cluster composed of multiple servers, which is not specifically limited. The terminal device can be a device such as a personal computer, or a mobile terminal device such as a mobile phone or a tablet computer, which is not specifically limited. The microcontroller can be applied to electronic doorbells, smart door locks, smart lights, smart air conditioners, smart meters or sensors, etc., which are not specifically limited. The application scenarios of the above-mentioned system and method include scenarios in which a neural network model is deployed on an electronic device. The neural network model involved can be CNN (Convolutional Neural Networks) and the like.

[0014] like Figure 1 As shown, an embodiment of the present invention provides a static random access memory in-memory computing system, which may include: The feature memory 11 is used to store the input signal of the neural network model.

[0015] The input signal of the neural network model may be specifically an input feature map. The feature memory 11 may also store output feature maps and intermediate results of partial sums.

[0016] The SRAM storage array 12 is used to store the various weights of the neural network model, and to perform logical operations on the input signal and the various weights to obtain multiple logical operation results.

[0017] The SRAM storage array involved in the embodiment of the present invention refers to a digital in-memory computing array, which will not be specifically stated below.

[0018] Among them, each weight of the neural network model is divided into multiple binary blocks after canonical signed digit (CSD) encoding, each binary block consists of two bits, and the multiple binary blocks are processed to eliminate the all-zero pattern blocks to retain the complementary pattern blocks, each complementary pattern block includes two complementary bits, and each complementary pattern block is extracted after information is extracted to obtain an index, a sign and a value, and the various weights of the neural network model are stored in the form of complementary pattern blocks.

[0019] The above symbols are used to identify positive numbers (i.e. non- ) and negative numbers (i.e. ). For example, the symbol 0 can be used to represent a positive number, and the symbol 1 can be used to represent a negative number, and there is no specific limitation. The above index refers to the label of each weight in the multiple weights.

[0020] The metadata memory 13 is used to store the index and symbol of the complementary pattern blocks of each weight of the neural network model.

[0021] In the embodiment of the present invention, the binary block includes an all-zero pattern block and a complementary pattern block. The all-zero pattern block is 00, and the complementary pattern block may include 01, 10, 0 and The complementary pattern block can store a pair of complementary weight bits, and the two complementary weight bits represent different information respectively.

[0022] For example, the weight , where the number of non-zero bits is 2, and there are two complementary pattern blocks. The first complementary pattern block The sign, value, and index in are 1, 1, and 01 respectively. The second pattern block The sign, value and index in are 0, 0 and 00 respectively. Among them, the value of the weight can be taken as the high bit in the complementary mode block.

[0023] This method of using CSD encoding can ensure that there are no two consecutive non-zero bits adjacent to each weight.

[0024] In the embodiment of the present invention, the above logic operation may be specifically a multiplication operation, such as implemented by a binary multiplication calculation unit.

[0025] The adder tree 14 is used to decode the above multiple logic operation results according to the index and symbol stored in the metadata memory, and then perform addition operation to obtain the addition result.

[0026] In an embodiment of the present invention, the adder tree 14 refers to an adder tree based on CSD coding. Before addition, the above-mentioned multiple logical operation results need to be decoded according to the index and the sign, and the logical operation results of the weights before CSD coding need to be restored to perform the addition calculation to obtain the correct result.

[0027] The shift accumulation unit 15 is used to perform shift accumulation on the addition result to obtain a shift accumulation result.

[0028] The post-processing unit 16 is used to perform post-processing on the shift accumulation result to obtain the output signal of the neural network model.

[0029] In the embodiment of the present invention, the SRAM storage array 13 may include: The SRAM storage unit includes a first inverter INV1, a second inverter INV2 and at least two transistors, wherein INV1 and INV2 are coupled to each other and are used to respectively store two complementary bits in the complementary mode blocks of each weight of the neural network model.

[0030] The binary block includes an all-zero mode block and a complementary mode block, and the complementary mode block includes a zero bit and a non-zero bit.

[0031] The binary multiplication calculation unit includes two logic circuits, which are used to perform multiplication operations on two complementary bits in the complementary mode blocks of each weight and the input signal respectively, and obtain two products as logic operation results.

[0032] In one implementation, the state Q of the first inverter INV1 can store the high bit in the complementary mode block of each weight, and the state Q of the second inverter INV2 can store the high bit in the complementary mode block of each weight. The low bits in the complementary mode block of each weight may be stored.

[0033] The signal stored in INV1 and the signal stored in INV2 are respectively input into the binary multiplication calculation unit through the bit lines BL and BLB of the SRAM storage unit to perform multiplication operation with the input signal.

[0034] In the embodiment of the present invention, the digital SRAM storage unit may include two or more transistors, wherein the number of transistors may be 6, 8, 10 or 12, etc., without specific limitation.

[0035] Figure 2 Schematic diagram of an SRAM storage unit provided by an embodiment of the present invention. Figure 2, the SRAM storage unit includes 6 transistors, M1~M6, that is, a 6T structure. Among them, the first inverter INV1 is formed by the NMOS or PMOS transistor pair M1 and M2, and the second inverter INV2 is formed by the NMOS or PMOS transistor pair M3 and M4. The first inverter INV1 and the second inverter INV2 are coupled to each other and can store weights in the form of a complementary mode block, that is, store a pair of complementary bits. Among them, the state Q of the first inverter INV1 can store the high bit in the complementary mode block of the weight, and the state Q of the second inverter INV2 can store the high bit in the complementary mode block of the weight. The low bits in the complementary mode block of the weights may be stored.

[0036] In the embodiment of the present invention, the SRAM storage array may include: a plurality of SRAM storage cells arranged in rows or columns, and a binary multiplication calculation unit.

[0037] For example, the SRAM storage array includes one SRAM storage unit and a corresponding binary multiplication calculation unit; or the SRAM storage array includes a plurality of SRAM storage units and corresponding binary multiplication calculation units.

[0038] In one implementation, the two logic circuits in the binary multiplication calculation unit may be coupled to the BL and BLB bit lines of the SRAM storage unit, respectively.

[0039] In the embodiment of the present invention, the two logic circuits in the binary multiplication calculation unit can both be implemented by AND gate circuits, that is, the binary multiplication calculation unit includes two AND gate circuits to perform logic operations.

[0040] Figure 3 A schematic diagram of a binary multiplication calculation unit provided by an embodiment of the present invention. Figure 3 The SRAM storage unit stores two complementary bits in the complementary mode block of each weight, which can be output to the binary multiplication calculation unit through the bit lines BL and BLB for calculation with the input signal. The two logic circuits in the binary multiplication calculation unit are AND gate AND1 and AND gate AND2.

[0041] The AND gate AND1 is used to store the bit on the BL bit line, i.e., the state Q of the first inverter INV1. Multiply it with the input signal I to get the product .

[0042] And, AND gate AND2 is used to adjust the bit on the BLB bit line, that is, the state of the second inverter INV2. The bits stored Multiply it with the input signal I to get the product .

[0043] Through the above structure, an SRAM storage unit can store a pair of bit-complementary weight signals, and the bit-complementary weight signals are multiplied with the input signal in the binary multiplication calculation unit via the BL or BLB bit line.

[0044] in, Figure 3 The left figure in the figure shows that one SRAM storage unit corresponds to a pair of AND gate circuits, and the right figure shows that multiple SRAM storage units correspond to a pair of AND gate circuits. In the scenario in the right figure, only one SRAM storage unit is valid in each clock cycle, that is, the data in the valid SRAM storage unit participates in the logic operation.

[0045] Figure 4 A schematic diagram of an adder tree provided by an embodiment of the present invention. Figure 4 , taking the adder tree including 8 CSD adding units as an example for explanation. The adder tree specifically includes CSD adding unit 0 to CSD adding unit 7, a total of 8 CSD adding units. Among them, each CSD adding unit first multiplies the two bits in the complementary mode block of each weight with the input signal according to the index and the sign to obtain two products, and decodes and restores the two bits in the weight before CSD encoding to the product of the input signal, and then adds the two products obtained after the weights are restored to obtain the addition result.

[0046] For example, the first weight after CSD encoding is , storing the complementary pattern block 01 of the first weight in the SRAM storage array, and storing the symbol information of the first weight in the metadata memory , and index information .

[0047] The second weight after CSD encoding is , storing the complementary pattern block of the second weight in the SRAM memory array , storing the symbol information of the second weight in the metadata memory , and index information .

[0048] The two input signals are the first input signal =1, and the second input signal The product of the first input signal and the first weight is , the product of the second input signal and the second weight is .

[0049] Before accumulation, the adder tree is first decoded according to the above-mentioned sign information and index information, and the two products of each weight are decoded and restored to the products of the two bits in the weight before CSD encoding and the input signal respectively, and then the two products obtained after the two weights are restored are added to obtain the addition result.

[0050] That is, the result of the adder tree accumulation is not , but the results are accumulated after decoding and recovery: .

[0051] In an embodiment of the present invention, after the neural network model is trained with quantization perception, all weights in each convolution kernel have the same number of non-zero bits after CSD encoding.

[0052] The above-mentioned system provided by the embodiment of the present invention stores the input signal of the neural network model through the feature memory, and the SRAM storage array stores each weight and performs logical operations on the input signal and each weight to obtain multiple logical operation results, wherein each weight is divided into multiple binary blocks after CSD encoding and the complementary pattern block is retained after the processing of eliminating the all-zero pattern block, each complementary pattern block includes two complementary bits, and each complementary pattern block is extracted to obtain an index, a symbol and a value, each weight of the neural network model is stored in the form of a complementary pattern block, the metadata memory stores the index and symbol of the complementary pattern block of each weight, and the adder tree is stored according to the metadata The index and sign stored in the digital SRAM memory are used to decode multiple logic operation results, and then addition operation is performed to obtain the addition result. The shift accumulation unit shifts and accumulates the addition result to obtain the shift accumulation result. The post-processing unit post-processes the shift accumulation result to obtain the output signal of the neural network model. Since the digital SRAM in-memory computing architecture does not require ADC for analog-to-digital conversion and is not limited by ADC resolution, the parallelism and efficiency of the calculation are improved. Moreover, the weights based on CSD encoding are stored in the form of binary blocks, which realizes the bit-level sparsity of the neural network model weights, greatly reduces the storage space and computing resources required for the model, and improves the utilization rate of the SRAM storage array.

[0053] In addition, the complementary pattern block in the binary block can realize data representation with a length of two bits, which effectively improves the utilization rate of the SRAM storage array and the computing efficiency of the system. When the SRAM storage array performs calculations, it only calculates the values ​​stored in the complementary pattern block, saving computing resources. After quantization-aware training, all weights in each convolution kernel have the same number of non-zero bits after CSD encoding, ensuring the regularity of the matrix after compression of the neural network model.

[0054] The above is a static random access memory in-memory computing system provided by an embodiment of the present invention. Based on the same idea, an embodiment of the present invention also provides a static random access memory in-memory computing method, such as Figure 5 As shown, the method may specifically include the following steps: In S502, each weight of the neural network model is CSD-encoded and divided into a plurality of binary blocks, each binary block consisting of two bits.

[0055] The multiple binary blocks obtained by the above division include all-zero pattern blocks and complementary pattern blocks.

[0056] In S504, all-zero pattern blocks in the plurality of binary blocks are eliminated and complementary pattern blocks are retained, and information is extracted from each complementary pattern block to obtain an index, a sign and a value.

[0057] Each complementary pattern block includes two complementary bits, that is, each complementary pattern block includes a zero bit and a non-zero bit. Specifically, the complementary pattern block may include: 01, 10, 0 and .

[0058] In S506, the various weights of the neural network model are stored in the SRAM storage array in the form of complementary pattern blocks, and the indexes and symbols of the complementary pattern blocks of the various weights of the neural network model are stored in the metadata memory.

[0059] In S508, logical operations are performed on the input signal and each weight of the neural network model to obtain multiple logical operation results.

[0060] In the embodiment of the present invention, the SRAM storage array may include an SRAM storage unit and a binary multiplication calculation unit, and the above step S508 may specifically include: Using two logic circuits in the binary multiplication calculation unit, two complementary bits in the complementary pattern blocks in each weight of the neural network model stored in the SRAM storage unit are multiplied with the input signal respectively to obtain two products as logic operation results.

[0061] In S510, the above-mentioned multiple logic operation results are decoded according to the index and the sign, and then an addition operation is performed to obtain an addition result.

[0062] In the embodiment of the present invention, the above step S510 may specifically include: The above multiple logic operation results are decoded and restored according to the index and the sign to the product of two bits in the weight before CSD encoding and the input signal respectively, and the two products obtained after the weight recovery are added to obtain the addition result.

[0063] In S512, shift accumulation is performed on the addition result to obtain a shift accumulation result.

[0064] In S514, the shift accumulation result is post-processed to obtain an output signal of the neural network model.

[0065] In the embodiment of the present invention, the neural network model is obtained by iterative training using the following steps in advance: Quantize each weight of the neural network model and then perform CSD encoding; Count the number of non-zero bits in each weight of each convolution kernel; For each convolution kernel, the threshold is determined according to the distribution of the number of non-zero bits in each weight. For weights whose number of non-zero bits is not equal to the threshold, the value of the weight is modified so that the number of non-zero bits is equal to the threshold.

[0066] In one implementation manner, the above-mentioned determining the threshold value according to the distribution of the number of non-zero bits in each weight may include: The number of items that appear the most frequently and do not exceed the specified upper limit is selected from the counted number as the threshold.

[0067] For example, a convolution kernel includes N weights, and the number of non-zero bits in each weight includes 0, 1, 2, 3 and 4. The number 2 appears the most times. If the upper limit is specified as 3, the number 2 can be determined as the threshold. Among the N weights, the weights in which the number of non-zero bits is not equal to 2 are all modified, such as by changing bit 1 to 0 to increase the number of zero bits, so that the number of non-zero bits in the modified weight is equal to the threshold 2. Through the above process, it can be ensured that all weights in each convolution kernel have the same number of non-zero bits after CSD encoding and are equal to the threshold.

[0068] The above method provided by the embodiment of the present invention performs CSD encoding on each weight of the neural network model and divides it into multiple binary blocks, eliminates the all-zero mode blocks in the multiple binary blocks and retains the complementary mode blocks, extracts information from each complementary mode block to obtain an index, a sign and a value, performs logical operations on the input signal of the neural network model and each weight to obtain multiple logical operation results, decodes the multiple logical operation results according to the index and the sign, and then performs addition operations to obtain addition results, performs shift accumulation on the addition results to obtain shift accumulation results, and post-processes the shift accumulation results to obtain the output signal of the neural network model. Since the digital SRAM in-memory computing architecture does not require ADC for analog-to-digital conversion and is not limited by ADC resolution, the parallelism and computing efficiency of the calculation are improved. Moreover, the weights based on CSD encoding are stored in the form of binary blocks, which realizes bit-level sparsity of the weights of the neural network model, greatly reduces the storage space and computing resources required for the model, and improves the utilization rate of the SRAM storage array.

[0069] Figure 6 The hardware structure diagram of an electronic device for implementing various embodiments of the present invention is shown in FIG. The electronic device 600 includes but is not limited to: a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609, a processor 610, and a power supply 611. Those skilled in the art will appreciate that Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In the embodiments of the present invention, the electronic device includes but is not limited to a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted terminal, a wearable device, and a pedometer.

[0070] Among them, the processor 610 is used to perform CSD encoding on each weight of the neural network model and divide it into multiple binary blocks, each binary block consists of two bits; eliminate the all-zero pattern blocks in the multiple binary blocks and retain the complementary pattern blocks, each complementary pattern block includes two complementary bits; extract information from each complementary pattern block to obtain an index, a sign and a value; store the various weights of the neural network model in the form of complementary pattern blocks in the SRAM storage array, and store the index and sign of the complementary pattern blocks of the various weights of the neural network model in the metadata memory; perform logical operations on the input signal and each weight of the neural network model respectively to obtain multiple logical operation results; decode the multiple logical operation results according to the index and sign, and then perform addition operations to obtain addition results; perform shift accumulation on the addition results to obtain shift accumulation results; post-process the shift accumulation results to obtain the output signal of the neural network model.

[0071] An embodiment of the present invention provides an electronic device, which performs CSD encoding on each weight of a neural network model and divides the weights into multiple binary blocks, eliminates all-zero mode blocks in the multiple binary blocks and retains complementary mode blocks, extracts information from each complementary mode block to obtain an index, a sign and a value, performs logical operations on an input signal of the neural network model and each weight to obtain multiple logical operation results, decodes the multiple logical operation results according to the index and the sign, then performs addition operations to obtain addition results, performs shift accumulation on the addition results to obtain shift accumulation results, and performs post-processing on the shift accumulation results to obtain an output signal of the neural network model. Since the digital SRAM in-memory computing architecture does not require ADC to perform analog-to-digital conversion and is not limited by ADC resolution, the parallelism and computing efficiency of the computing are improved. Moreover, the weights based on CSD encoding are stored in the form of binary blocks, which realizes bit-level sparsity of the weights of the neural network model, greatly reduces the storage space and computing resources required for the model, and improves the utilization rate of the SRAM storage array.

[0072] It should be understood that in the embodiment of the present invention, the radio frequency unit 601 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving downlink data from the base station, it is sent to the processor 610 for processing; in addition, uplink data is sent to the base station. Generally, the radio frequency unit 601 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 601 can also communicate with the network and other electronic devices through a wireless communication system.

[0073] The electronic device provides users with wireless broadband Internet access through the network module 602, such as helping users to send and receive emails, browse web pages, and access streaming media.

[0074] The audio output unit 603 can convert the audio data received by the RF unit 601 or the network module 602 or stored in the memory 609 into an audio signal and output it as sound. Moreover, the audio output unit 603 can also provide audio output related to a specific function performed by the electronic device 600 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 603 includes a speaker, a buzzer, a receiver, etc.

[0075] The input unit 604 is used to receive audio or video signals. The input unit 604 may include a graphics processor (GPU) 6041 and a microphone 6042. The graphics processor 6041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 606. The image frame processed by the graphics processor 6041 can be stored in the memory 609 (or other storage medium) or sent via the radio frequency unit 601 or the network module 602. The microphone 6042 can receive sound and can process such sound into audio data. The processed audio data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 601 in the case of a telephone call mode.

[0076] The electronic device 600 also includes at least one sensor 605, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 6061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 6061 and / or the backlight when the electronic device 600 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used to identify the posture of the electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; the sensor 605 can also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be repeated here.

[0077] The display unit 606 is used to display information input by the user or information provided to the user. The display unit 606 may include a display panel 6061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0078] The user input unit 607 can be used to receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device. Specifically, the user input unit 607 includes a touch panel 6071 and other input devices 6072. The touch panel 6071, also known as a touch screen, can collect user touch operations on or near it (such as operations performed by users using fingers, styluses, or any other suitable objects or accessories on or near the touch panel 6071). The touch panel 6071 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact point coordinates, and then sends it to the processor 610, receives the command sent by the processor 610 and executes it. In addition, the touch panel 6071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 6071, the user input unit 607 may also include other input devices 6072. Specifically, other input devices 6072 may include but are not limited to a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.

[0079] Furthermore, the touch panel 6071 may be overlaid on the display panel 6061. When the touch panel 6071 detects a touch operation on or near it, it is transmitted to the processor 610 to determine the type of touch event. Then, the processor 610 provides corresponding visual output on the display panel 6061 according to the type of touch event. Figure 6 In the figure, the touch panel 6071 and the display panel 6061 are used as two independent components to realize the input and output functions of the electronic device. However, in some embodiments, the touch panel 6071 and the display panel 6061 can be integrated to realize the input and output functions of the electronic device, which is not limited here.

[0080] The interface unit 608 is an interface for connecting an external device to the electronic device 600. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 608 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the electronic device 600 or may be used to transmit data between the electronic device 600 and an external device.

[0081] The memory 609 can be used to store software programs and various data. The memory 609 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 609 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0082] The processor 610 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 609 and calling data stored in the memory 609, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. The processor 610 may include one or more processing units; preferably, the processor 610 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 610.

[0083] The electronic device 600 may also include a power supply 611 (such as a battery) for supplying power to various components. Preferably, the power supply 611 may be logically connected to the processor 610 via a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0084] Preferably, an embodiment of the present invention further provides an electronic device, comprising a processor 610, a memory 609, and a computer program stored in the memory 609 and executable on the processor 610. When the computer program is executed by the processor 610, the various processes of the above-mentioned method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0085] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0086] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0087] The above-mentioned computer-readable storage medium provided by the embodiment of the present invention performs CSD encoding on each weight of the neural network model and divides it into multiple binary blocks, eliminates the all-zero mode blocks in the multiple binary blocks and retains the complementary mode blocks, extracts information from each complementary mode block to obtain an index, a sign and a value, performs logical operations on the input signal and each weight of the neural network model to obtain multiple logical operation results, decodes the multiple logical operation results according to the index and the sign, and then performs addition operations to obtain addition results, performs shift accumulation on the addition results to obtain shift accumulation results, and post-processes the shift accumulation results to obtain the output signal of the neural network model. Since the digital SRAM in-memory computing architecture does not require ADC for analog-to-digital conversion and is not limited by ADC resolution, the parallelism and computing efficiency of the calculation are improved, and the weights based on CSD encoding are stored in the form of binary blocks, which realizes bit-level sparsity of the weights of the neural network model, greatly reduces the storage space and computing resources required for the model, and improves the utilization rate of the SRAM storage array.

[0088] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0089] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0090] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0092] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0093] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0094] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0095] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0096] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A static random access memory in-memory computing system, characterized in that: include: A feature memory for storing input signals of the neural network model; A static random access memory SRAM storage array is used to store various weights of the neural network model, and to perform logic operations on the input signal and the various weights to obtain multiple logic operation results, wherein the various weights of the neural network model are divided into multiple binary blocks after being encoded by a regular signed number CSD, and each binary block consists of two bits. After the multiple binary blocks are processed by eliminating the all-zero pattern blocks, complementary pattern blocks are retained, and each complementary pattern block includes two complementary bits. After information is extracted from each complementary pattern block, an index, a sign and a value are obtained. The various weights of the neural network model are stored in the form of complementary pattern blocks; A metadata memory for storing indices and symbols of complementary pattern blocks of respective weights of a neural network model; An adder tree, used for decoding the plurality of logic operation results according to the index and the symbol stored in the metadata memory, and then performing addition operation to obtain the addition result; A shift accumulation unit, used for performing shift accumulation on the addition result to obtain a shift accumulation result; A post-processing unit is used to post-process the shift accumulation result to obtain an output signal of the neural network model.

2. The system according to claim 1, characterized in that The SRAM storage array comprises: An SRAM storage unit, comprising a first inverter INV1 and a second inverter INV2 and at least two transistors, wherein the INV1 and INV2 are coupled to each other and are used to respectively store two complementary bits in a complementary mode block of each weight of a neural network model; The binary multiplication calculation unit includes two logic circuits, which are used to perform multiplication operations on the two complementary bits in the complementary mode blocks of each weight and the input signal respectively, and obtain two products as logic operation results.

3. The system according to claim 2, characterized in that The status of INV1 Q The high bits of the complementary mode blocks of the respective weights are stored, and the state of the INV2 storing the low bits in the complementary mode blocks of the respective weights; The signal stored in the INV1 and the signal stored in the INV2 are respectively input into the binary multiplication calculation unit through the bit lines BL and BLB of the SRAM storage unit to perform multiplication operation with the input signal.

4. The system according to claim 1, characterized in that The complementary pattern blocks include 01, 10, 0 and .

5. The system according to claim 1, characterized in that After the neural network model is trained with quantization perception, all weights in each convolution kernel have the same number of non-zero bits after CSD encoding.

6. A static random access memory in-memory computing method, characterized in that: Applied to the static random access memory in-memory computing system according to any one of claims 1 to 5, the method comprising: Each weight of the neural network model is CSD-encoded and divided into multiple binary blocks, each of which consists of two bits; Eliminating all-zero pattern blocks in the plurality of binary blocks and retaining complementary pattern blocks, each complementary pattern block including two complementary bits; Extract information for each complementary pattern block to obtain index, sign and value; The weights of the neural network model are stored in the SRAM storage array in the form of complementary pattern blocks, and the index and symbol of the complementary pattern blocks of the weights of the neural network model are stored in the metadata memory; Performing logic operations on the input signal and each weight of the neural network model to obtain multiple logic operation results; Decoding the plurality of logic operation results according to the index and the sign, and then performing an addition operation to obtain an addition result; Performing shift accumulation on the addition result to obtain a shift accumulation result; The shift accumulation result is post-processed to obtain an output signal of the neural network model.

7. The method according to claim 6, characterized in that The SRAM storage array includes an SRAM storage unit and a binary multiplication calculation unit. The input signal and each weight of the neural network model are respectively subjected to logical operations to obtain multiple logical operation results, including: Using the two logic circuits in the binary multiplication calculation unit, two complementary bits in the complementary pattern block of each weight of the neural network model stored in the SRAM storage unit are multiplied with the input signal respectively to obtain two products as logic operation results.

8. The method according to claim 7, characterized in that The decoding of the plurality of logic operation results according to the index and the sign, and then performing an addition operation to obtain an addition result, comprises: Decoding the multiple logic operation results according to the index and the sign to restore the products of two bits in the weight before CSD encoding and the input signal respectively; The two products obtained after the weights are restored are added to obtain the addition result.

9. The method according to claim 6, characterized in that The neural network model is obtained by iterative training using the following steps in advance: Quantize each weight of the neural network model and then perform CSD encoding; Count the number of non-zero bits in each weight of each convolution kernel; For each convolution kernel, a threshold is determined according to the distribution of the number of non-zero bits in each weight. For weights whose number of non-zero bits is not equal to the threshold, the value of the weight is modified so that the number of non-zero bits therein is equal to the threshold.

10. The method according to claim 9, characterized in that The step of determining the threshold value according to the distribution of the number of non-zero bits in each weight comprises: The number of items that appear the most frequently and do not exceed the specified upper limit is selected from the counted number as the threshold.

Images