Neural network computing core based on nonvolatile memory
By introducing a double-bit line dual reading unit and sparse detection circuit into the neural network computing core, the multiplication and accumulation operation is optimized, and the problem of high computing energy consumption in traditional architectures is solved, and efficient neural network computing is achieved.
Patent Information
- Application Number
- CN202510524471.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-24
- Publication Date
- 2025-08-08
AI Technical Summary
In the prior art, neural network computing cores based on arithmetic logic units need to frequently read and write back data from memory, resulting in a large amount of time and energy consumed by the calculation process, and the parallelism of traditional architectures is limited, which cannot effectively solve the von Neumann bottleneck problem.
A neural network computing core based on nonvolatile memory is adopted, a double-bit line dual reading unit and a sparse detection circuit are introduced. By reading 2-bit data at one time and dynamically detecting sparseness, a reconstructible sparse strategy is designed, multiplication and accumulation operations are optimized, and calculation steps and energy consumption are reduced.
Significantly reduce energy consumption, improve computing speed and efficiency, especially on mobile devices and edge computing platforms to provide energy-efficient computing power.
Smart Images
Figure CN120449958A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence neural networks, and in particular to a neural network computing core based on non-volatile memory. Background Art
[0002] With the breakthroughs in artificial intelligence in recent years, the world is steadily moving towards an era of intelligentization. This has brought with it unprecedented parameter size and model complexity in neural network models. Currently, the mainstream processor design is represented by the central processing unit (CPU). As a general-purpose processor, the CPU can adapt to a wide range of application scenarios, but it often consumes a large amount of resources when processing specific programs or applications. Furthermore, the CPU utilizes a von Neumann architecture that separates the CPU from the memory. With frequent data exchange, the energy consumed by data transmission significantly exceeds the energy consumed by the computation itself, a phenomenon known as the "von Neumann bottleneck." This bottleneck has become a significant obstacle to building efficient AI systems. To effectively address the bottleneck, researchers are integrating computation into the memory unit, creating a "computation-storage architecture." Many new memory technologies, such as MRAM, are currently attracting widespread attention. These technologies preserve data during system power outages, eliminating the need to reload data from the processor upon system restart. With the continuous development of integrated storage and computing technology, researchers are beginning to focus on higher-level systems, using this technology to build efficient neural network accelerators. This approach leverages the high energy efficiency and high throughput of integrated storage and computing solutions to significantly improve system-level performance. Currently, research on integrated storage and computing solutions focuses primarily on matrix multiplication and accumulation (MAC) operations in neural network operations, while other operations must be offloaded to external processors for calculation, which undoubtedly increases computational latency and power consumption.
[0003] The neural network computing core based on the arithmetic logic unit (ALU) chip first preloads the weights into the cache or registers to speed up repeated access. The input is then loaded into the memory near the computing unit in blocks of rows. The ALU is called by instructions to perform matrix element multiplication operations. The neural network computing core based on the arithmetic logic unit chip and the storage unit result in the need to frequently read data from the memory to the ALU and then write the results back during the calculation process. This data movement consumes a lot of time and energy. Traditional ALU architectures typically rely on multi-level caches and pipeline designs to improve parallelism, but their parallelism is limited by the number of physical ALU cores and bus bandwidth. Summary of the Invention
[0004] In view of the above-mentioned problems existing in the prior art, the purpose of the present invention is to provide a neural network computing core based on non-volatile memory to solve the problems raised in the above-mentioned background technology.
[0005] To achieve the above object, the present invention provides the following technical solutions:
[0006] A neural network computing core based on non-volatile memory includes a computing core, wherein a time control circuit, a storage array structure, a data register, a weight selection circuit, a sparse circuit, and a data accumulation circuit are provided inside the computing core; the storage array structure and the data accumulation circuit are communicatively connected; the storage array structure is provided with eight groups of array channels; the array channels include storage and computing units and multiplication and accumulation circuits connected to the storage and computing units; the multiplication and accumulation circuits are communicatively connected to the weight selection circuit; the sparse circuits are communicatively connected to the data registers; the data registers are communicatively connected to the multiplication and accumulation circuits, and the multiplication and accumulation circuits are connected to the time control circuit.
[0007] As a further solution of the present invention: the storage and calculation units provided in the array channel are distributed in a matrix of 64 rows and 8 columns.
[0008] As a further solution of the present invention: the storage and calculation unit is provided with a bit line BL1, a bit line BL2 and a source line SL; the multiplication and accumulation circuit is connected to multiple columns of storage and calculation units on the same group of array channels through the bit line BL1, the bit line BL2, the source line SL.
[0009] As a further solution of the present invention: the storage and calculation unit is a storage and calculation unit read by a dual MRAM.
[0010] As a further solution of the present invention: the storage and calculation unit is connected to different reading circuits through the bit line BL1 and the bit line BL2 respectively.
[0011] As a further solution of the present invention: the multiplication and accumulation circuit includes a calculation unit and a data storage module.
[0012] As a further solution of the present invention: the calculation unit in the multiplication and accumulation circuit is composed of a full adder chain; the data storage module is composed of multiple groups of registers.
[0013] As a further solution of the present invention: the full adder chain of the calculation unit in the multiplication and accumulation circuit is connected to the clock end of the time control circuit.
[0014] As a further solution of the present invention: the data storage module and the data register in the multiplication and accumulation circuit are communicatively connected.
[0015] As a further solution of the present invention: a data selector is provided in the sparse circuit; a preprocessing module and a data reading module for controlling input data are also provided in the sparse circuit.
[0016] Compared with the prior art, the present invention has the following beneficial effects:
[0017] The present invention introduces a dual-bit-line dual-read unit into the non-volatile memory array, and this structure can read out 2 bits of data at a time. This structure greatly improves the speed compared to the traditional reading method. The present invention designs a reconfigurable approximate sparse detection circuit, which can dynamically detect the size of data and weight data and dynamically select the number of sparse bits, thereby improving the power consumption during calculation. The number of calculation cycles can be further saved during multiple rounds of calculation to achieve the purpose of improving the calculation speed. The present invention converts the read weight data into the enable end of the register and the adder through the logic unit, and completes multiple rounds of multiplication and accumulation operations and outputs the final result through the design of the adder and the register. Through a reasonable sparsification strategy, energy consumption can be significantly reduced while ensuring calculation accuracy. The innovative design of the present invention not only improves the energy efficiency of the system, but also provides higher computing efficiency for mobile devices and edge computing platforms in energy-limited environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 A system framework diagram of a neural network computing core based on non-volatile memory disclosed in an embodiment.
[0019] Figure 2 A schematic diagram of the storage array structure in a neural network computing core based on non-volatile memory disclosed in an embodiment.
[0020] Figure 3 A schematic diagram of a multiplication-accumulation circuit in a neural network computing core based on non-volatile memory disclosed in an embodiment.
[0021] Figure 4 A schematic diagram of a sparse circuit in a neural network computing core based on non-volatile memory disclosed in an embodiment.
[0022] Figure 5 A computational flow chart of a neural network computing core based on non-volatile memory disclosed in an embodiment. DETAILED DESCRIPTION
[0023] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort shall fall within the scope of protection of the present invention.
[0024] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "provided with," "connected," and "connected" should be understood in a broad sense; for example, they may refer to fixed connections, detachable connections, or integral connections; they may refer to mechanical connections or electrical connections; they may refer to direct connections or indirect connections via an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention in specific contexts.
[0025] The present invention provides a neural network computing core based on non-volatile random access memory. The introduction of a sparse detection circuit improves the computing speed and energy efficiency. The design of the dual read circuit can calculate two 8-bit multiplication operations at a time, greatly improving the computing speed. The computing core has a total of 8 channels for parallel calculation, which has extremely high computing efficiency compared with traditional methods.
[0026] See also Figure 1-5 , a neural network computing core based on non-volatile memory, comprising a computing core, wherein the computing core is provided with a time control circuit, a storage array structure, a data register, a weight selection circuit, a sparse circuit and a data accumulation circuit;
[0027] The storage array structure is communicatively connected to the data accumulation circuit;
[0028] The storage array structure has eight array channels; each array channel consists of 64 rows and 8 columns of storage and calculation units and a multiplication and accumulation circuit connected to the storage and calculation units;
[0029] Each column of storage and calculation units is provided with a bit line BL1, a bit line BL2 and a source line SL; the multiplication and accumulation circuit is connected to the storage and calculation units of multiple columns via the bit line BL1, the bit line BL2, the source line SL;
[0030] The multiplication and accumulation circuit is communicatively connected to the weight selection circuit;
[0031] The multiplication and accumulation circuit includes a calculation unit and a data storage module; the data storage module is communicatively connected with the register;
[0032] The memory array consists of eight channels, each with 64 rows and eight columns. Therefore, each channel can store 64 8-bit data. The present invention divides the memory array into four groups, each performing 16 multiplication-accumulation operations. To further simplify the discharge control logic, a weight selection module is designed. When a certain group of weighted data is selected, i.e., when data is read from a certain set of 16 memory cells, RWLs are turned on sequentially, controlling the discharge of bit lines BL1 and BL2. This ensures that the read data is output in the correct order.
[0033] The storage and computing unit is a dual-MRAM read storage and computing unit;
[0034] The storage and calculation unit is connected to different read circuits through bit lines BL1 and BL2 respectively;
[0035] In order to further improve the reading and computing speed of the recursive storage and computing architecture, the present invention adopts a dual MRAM reading storage and computing unit, such as Figure 2 As shown. There are two bit lines BL1, BL2 and source line SL on each column of an array, which are mainly used to read data. 1 bit of data is stored in it in the form of MTJ resistance state, high resistance state represents data "1" and low resistance state represents the number "0". Each storage cell stores 1 bit of weight data, and 8 columns constitute an 8-bit weight data. When performing a multiplication operation, BL1 and BL2 will first be precharged to a high potential. Secondly, the two read word line transistors RWL of the corresponding row will be fully turned on, and the bit line will be discharged at this time. Due to the different states of MRAM, the difference between high resistance state and low resistance state will result in different bit line discharge speeds, so different data can be read. The different weight data read out are input into the multiplication operation module for the next step of calculation. Because of the existence of the dual read circuit, the data of two cells can be read at one time.
[0036] The computing unit is composed of a chain of full adders; the data storage module is composed of multiple groups of registers;
[0037] In order to achieve efficient multiplication and accumulation operations, the present invention designs a multiplication and accumulation module based on the above-mentioned dual-unit read module to complete the complete operation. Its circuit structure is as follows: Figure 3 As shown. This module is mainly composed of two parts, one is a calculation unit composed of a full adder chain, and the other is a data storage module composed of registers. First, the input data and weight data are pre-processed and input into the multiplication and accumulation module. For the calculation units on the same row, multiple "11" data may be read out. Therefore, when two 8-bit input data are input into the calculation module, multiple repeated calculations may be performed. In order to save power consumption of this calculation, the input data is first pre-added to obtain the sum of the two input data and the two input data are input into the multiplication and accumulation module together. Figure 3 shown.
[0038] The full adder chain of the calculation unit is connected to the clock terminal of the time control circuit;
[0039] At the start of calculation, a reset signal resets the registers to a stable state for the duration of the calculation. The multiplication-accumulation module's input, S0S1, is connected to the read port of the pipeline read module. The 2-bit data read is first multiplied with the input data, Input1[0:7], Input0[0:7], and the sum of these two data points. This operation is performed through a multiplexer. When the pipeline read module outputs "11," the multiplexer outputs the signal Input_0+1[0:7] to the full adder chain. When the data in S1S0 is "10," the multiplexer outputs Input_1[0:7] to the full adder chain. Similarly, when the data in S1S0 is "01," the multiplexer outputs Input_0[0:7]. At the same time, in order to reduce power consumption, S1S0 will pass through an OR logic gate as the enable end of the full adder chain. Therefore, when the data of S1S0 is "00", the calculation of the full adder chain will be turned off, and the multiplexer will also output all 0 data to reduce power consumption.
[0040] As described above, in the recursive multiplication-accumulation mode, the calculation must be added to the result of the previous cycle to obtain the final calculation result of the current cycle. This calculation result will also be used for the calculation of the next cycle. Therefore, to store the calculation results, the present invention stores the calculation results in a register. The present invention inverts the enable terminal of the full adder chain and connects it to the clock terminal of the register. When the calculation begins, the output of the register will be connected to the input terminal of the full adder chain as part of the input. When the calculation is completed, the new round of calculation results needs to be stored in the register. The calculation begins when the enable terminal is at a high level. When the calculation is complete, the enable terminal changes from "1" to "0". The clock terminal after inversion happens to be the rising clock edge, so the calculated data can be stored in the register.
[0041] Given the widespread data sparsity in AI models, optimizing multiplication-accumulation operations to reduce computational latency and energy consumption is key to achieving efficient computing. To address this, this paper proposes a mechanism based on input and weight sparsity detection and its corresponding circuit design. This mechanism aims to skip some full addition operations in a storage-and-computation-integrated solution, thereby improving computational efficiency.
[0042] When two numbers are multiplied, the size of their product is determined by the sizes of both numbers. If the highest bits of both numbers are zero, their product will be very small and will not significantly affect the final accumulation result. Based on this observation, the present invention proposes a reconfigurable approximate computation scheme. This scheme determines sparsity by monitoring the highest bits of the inputs and weights, thereby skipping unnecessary computation steps and reducing the computational burden. Specifically, the present invention introduces an approximate bit width n to monitor data sparsity. The data participating in the same accumulation cycle includes Input0, Input1, Weight0, and Weight1. By monitoring the highest n bits of this data, if any of these bits contain a "1," the sparsity condition is not met, and the sparsity is marked as 1. Otherwise, the sparsity condition is met, and some computation steps can be skipped. For example, if the highest n bits of Input0 and Weight0 are both 0, the sparsity condition is partially met. In this case, the addition of Input0 and Input1 is skipped, and only the Weight1 portion is output. The accumulation unit then accumulates only the product of Input1 and Weight1. Similarly, similar computational optimizations are performed when the highest n bits of Input1 and Weight1 are all 0. If the highest n bits of all four data are all 0, then complete sparsity is satisfied, and not only is the addition of Input0 and Input1 skipped, but the entire MAC operation is also skipped.
[0043] Figure 4 The working principle of the sparsity detection circuit proposed in the present invention is demonstrated. The sparsity circuit is connected to the data register for communication;
[0044] A data selector is provided in the sparse circuit; a pre-processing module and a data reading module for controlling input data are also provided in the sparse circuit;
[0045] In this circuit, the first k bits of the input data and weights are monitored simultaneously. If any of the first k bits of the input or weight are "1," the sparsity detection circuit's output is set to "1," indicating that the data does not meet certain sparsity conditions. Conversely, when the first k bits of the input or weight are all "0," the circuit's output is set to "0," indicating that the data meets certain sparsity conditions and that some calculations can be skipped. These sparsity flags are stored in registers for subsequent operations. In neural network algorithms, input data and weights are often reused across multiple rounds of multiplication-accumulation operations. Therefore, once the sparsity conditions are determined and stored in the first round, the related read and calculation operations can be skipped in subsequent rounds, significantly saving time and computing resources. To adapt to different computing scenarios, the sparsity detection circuit is equipped with a data selector that dynamically selects the value of k based on a control signal, Sel. As k increases, the detected data sparsity gradually decreases. The output signal of the sparsity detection circuit also controls the input data preprocessing and data reading module.
[0046] In order to achieve ultra-fast multiplication and accumulation operations with high precision, the present invention proposes a recursive multiplication and accumulation storage and calculation integrated solution whose calculation speed is independent of data precision and only related to the number of accumulations, such as Figure 5 shown. Figure 5 Taking the multiplication and accumulation operation of 8 8-bit input data and weight data as an example, it only takes 8 cycles to complete the calculation. During the calculation process, a complete 8-bit data will be input in each cycle. For example, in cycle 0, the input data Input0[7:0] is dot-producted with each bit of the first weight data W0[0:15] to obtain the partial result MAC 0-0 to MAC 0-7 In cycle 1, the same calculation process is performed to obtain the partial result MAC 1-0 to MAC 1-7 At this time, the MAC obtained in cycle 1 1-0 to MAC 1-7 Combine them with the MAC obtained in cycle 0 0-0 to MAC 0-7 The result will be used as the input of the next cycle and the MAC obtained in cycle 2 2-0 to MAC 2-7 Add. Similarly, after eight cycles, all input data has been calculated, yielding partial MAC results LMAC0 through LMAC7. Finally, the corresponding shift-and-add operations are performed based on their weights to complete the entire multiplication-accumulation operation. Clearly, eliminating weighted operations from the input effectively improves the computational efficiency of the integrated storage and computing architecture. Furthermore, the number of cycles executed is dependent solely on the number of accumulations, not the precision of the input data. This significantly reduces the execution time of high-precision multiplication-accumulation operations.
[0047] The present invention solves the problem of computing speed in high-precision computing by inputting all input data at once, performing dot products with weights respectively, and temporarily storing the calculation results for the next cycle of calculation. The present invention designs a dual-unit based reading module and introduces it into the storage array, which can read 2 bits of data at a time, greatly improving the reading speed. On this basis, in order to realize matrix multiplication and accumulation operations, the present invention designs a multiplication accumulator that can complete multi-bit multiplication and temporarily store the calculation results. In order to further improve energy efficiency, the present invention designs an approximate computing design and introduces a sparse structure. Through a reasonable sparsification strategy, energy consumption can be significantly reduced while ensuring calculation accuracy. This innovative design not only improves the energy efficiency of the system, but also provides higher computing efficiency for mobile devices and edge computing platforms in energy-limited environments.
[0048] Using SARM as the storage medium, matrix multiplication operations are performed through methods such as bit line discharge. Weight data is stored in the SARM unit, and the input data is input into the SRAM word line in the form of voltage or current. The final result is calculated by characterizing the voltage or current value of the bit line. Storage and computing integrated architectures often rely on analog computing or mixed signal processing to improve energy efficiency. The purely digital storage characteristics of SRAM require additional circuit support such as DAC / ADC in analog computing, which increases the design complexity. The resistance characteristics of MRAM are naturally suitable for analog multiplication and addition operations, and can achieve more efficient in-memory calculations. SRAM requires continuous power supply to maintain data, and even when idle, it will generate static power consumption due to transistor leakage current. MRAM is a non-volatile memory that retains data after power failure and consumes almost no power when in standby mode.
[0049] The present invention introduces a dual-bit-line dual-read unit into the non-volatile memory array. This structure can read out 2 bits of data at a time. This structure greatly improves the speed compared to the traditional reading method. The present invention designs a reconfigurable approximate sparse detection circuit, which can dynamically detect the size of data and weight data and dynamically select the number of sparse bits, thereby improving the power consumption during calculation. The number of calculation cycles can be further saved during multiple rounds of calculation to achieve the purpose of improving the calculation speed. The present invention converts the read weight data into the enable end of the register and the adder through the logic unit, and completes multiple rounds of multiplication and accumulation operations and outputs the final result through the design of the adder and the register.
[0050] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-limiting. The scope of the invention is defined by the appended claims, not the foregoing description, and it is intended that all variations within the meaning and range of equivalents of the claims be encompassed within the present invention, and any reference signs in the claims should not be construed as limiting the claim to which they relate.
[0051] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.
Claims
1. A neural network computing core based on non-volatile memory, characterized in that: The computing core includes a time control circuit, a storage array structure, a data register, a weight selection circuit, a sparse circuit, and a data accumulation circuit. The storage array structure is in communication with the data accumulation circuit. The storage array structure is provided with eight array channels. The array channels include storage and calculation units and multiplication and accumulation circuits connected to the storage and calculation units. The multiplication and accumulation circuits are in communication with the weight selection circuit. The sparse circuit is connected to the data register for communication; the data register is connected to the multiplication and accumulation circuit for communication; and the multiplication and accumulation circuit is connected to the time control circuit.
2. The neural network computing core based on non-volatile memory according to claim 1, characterized in that: The storage and calculation units provided in the array channel are distributed in a matrix of 64 rows and 8 columns.
3. The neural network computing core based on non-volatile memory according to claim 2, characterized in that: The storage and calculation unit is provided with a bit line BL1, a bit line BL2 and a source line SL; the multiplication and accumulation circuit is connected to multiple columns of storage and calculation units on the same group of array channels through the bit line BL1, the bit line BL2, the source line SL.
4. The neural network computing core based on non-volatile memory according to claim 3, characterized in that: The storage and calculation unit is a storage and calculation unit read by a dual MRAM.
5. The neural network computing core based on non-volatile memory according to claim 4, characterized in that: The storage and calculation unit is connected to different reading circuits via the bit line BL1 and the bit line BL2 respectively.
6. The neural network computing core based on non-volatile memory according to claim 1, characterized in that: The multiplication and accumulation circuit includes a calculation unit and a data storage module.
7. The neural network computing core based on non-volatile memory according to claim 6, characterized in that: The calculation unit in the multiplication and accumulation circuit is composed of a full adder chain; the data storage module is composed of multiple groups of registers.
8. The neural network computing core based on non-volatile memory according to claim 7, characterized in that: The full adder chain of the calculation unit in the multiplication-accumulation circuit is connected to the clock terminal of the time control circuit.
9. The neural network computing core based on non-volatile memory according to claim 8, characterized in that: The data storage module and the data register in the multiplication and accumulation circuit are communicatively connected.
10. The neural network computing core based on non-volatile memory according to claim 1, characterized in that: The sparse circuit is provided with a data selector; the sparse circuit is also provided with a pre-processing module for controlling input data and a data reading module.