In-memory computing accelerated system and method for linear attention
By using a dual-wave pulsed storage pipeline control module, a ferroelectric weight storage module, and a dynamic state storage module, the recursive dependency of Delta rules is decoupled, enabling parallel state updates and computations. This solves the problems of frequent data movement and recursive dependency in Delta linear attention, improving the efficiency of real-time reasoning in long contexts and the utilization of hardware resources.
Patent Information
- Application Number
- CN202610930038.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-25
- Publication Date
- 2026-11-10
AI Technical Summary
Existing technologies cannot effectively solve the problems of frequent data movement and recursive dependencies in Delta linear attention, which leads to frequent pauses in the computation process, fails to meet the requirements of real-time inference with long contexts, and results in low hardware resource utilization, high inference latency, and redundant power consumption.
By employing a dual-wave pulsed storage pipeline control module, a ferroelectric weight storage module, and a dynamic state storage module, and decoupling the recursive dependency of Delta rules, state updates and calculations can be performed in parallel, eliminating data transfer delays and idle waiting between modules.
It achieves parallel processing of state updates and computation, reduces data movement, lowers power consumption, improves inference throughput, meets the needs of real-time inference with long contexts, and improves hardware resource utilization.
Smart Images

Figure CN123070204A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to an in-memory computing accelerated computing system and method for linear attention. Background Technology
[0002] With the profound development of artificial intelligence technology, Large Language Models (LLMs) have achieved breakthroughs in natural language processing, symbolic reasoning, and code development. However, the computational and storage bottlenecks in long-context reasoning scenarios are becoming increasingly prominent. Traditional self-attention mechanisms based on the nonlinear activation function Softmax have a quadratic computational complexity of O(L). 2 In other words, the computational cost increases quadratically with the sequence length L. When the sequence length L increases significantly, it not only generates massive computational overhead but also puts enormous pressure on the key-value (KV) cache memory, failing to meet the real-time inference requirements of long contexts. Against this backdrop, the linear attention mechanism emerged. Its core is to simplify the self-attention computation process and abandon the Softmax function, achieving O(Ld) computational complexity. 2 The linear computational complexity (d is the hidden dimension of the token) significantly reduces the computational and storage overhead of long sequence inference, becoming a core technical direction for long context optimization in LLM. However, the original linear attention suffers from representation degradation due to limited memory space, making it difficult to balance inference speed and accuracy. To address this, the differential Delta rule was introduced into the linear attention mechanism, forming DeltaNet, a linear attention based on the Delta rule. It recursively updates the internal state matrix S, simulating the learning and forgetting process of memory, effectively solving the problem of representational degradation in the original linear attention and further achieving a balance between inference speed and accuracy.
[0003] Computing-in-Memory (CIM) architecture, as a key technology to break through the storage wall bottleneck of the traditional von Neumann architecture, has the core advantage of deep integration of computing and storage. It enables data to be processed directly at the storage location, greatly reducing the transmission of data between storage and computing units, reducing data migration overhead, and significantly improving computing energy efficiency. It has become an ideal hardware architecture for linear attention-based large-scale parallel computing.
[0004] Currently, several acceleration architectures based on the CIM architecture and supporting linear attention acceleration exist. One type is based on a Field-Programmable Gate Array (FPGA) platform, employing speculative decoding technology to accelerate Mamba model inference, optimizing data flow to reduce computational latency, and adapting to long-sequence inference requirements. Its core advantage is latency optimization for long-sequence inference. Another type uses a 28nm all-digital circuit design, adapting to the computational requirements of the Mamba model through a reconfigurable architecture, specifically optimizing element operations and nonlinear functions. Its core advantage is strong reconfigurability, capable of adapting to the computational needs of different neural network layers. A third type is a ReRAM-based in-memory computing macrocell, using a 22nm process, designing a non-volatile in-memory computing macrocell based on multi-level ReRAM, supporting acceleration of multiple models such as Mamba, Transformer, and CNN, and possessing advantages of high storage density and high energy efficiency. None of the existing technical solutions have optimized for the core data flow requirements of Delta linear attention, thus failing to achieve efficient hardware acceleration.
[0005] However, the introduction of a dynamic state matrix S by linear attention based on the Delta rule, together with the static weight matrix, constitutes two types of core data. This leads to a significant increase in the amount of data that needs to be transmitted during long context inference, further exacerbating data migration losses. It cannot support the low-power, high-throughput long context inference requirements of linear attention, and also brings new challenges to the hardware acceleration of in-memory computing (CIM) architecture. It fails to fully leverage the advantages of CIM's integration of computing and storage to reduce data migration. Moreover, most existing in-memory computing architectures use a single storage medium, failing to achieve flexible adaptation and reasonable division of labor among multiple types of storage units. They cannot simultaneously meet the high-density, low-leakage storage requirements of static weights and the high-frequency read / write, high-endurance, and low-power update requirements of dynamic state matrices, resulting in low storage utilization and power redundancy.
[0006] Furthermore, Delta linear attention suffers from strong recursive data dependencies in its state matrix updates. The current token's state update depends on the previous state matrix, and the next token's computation must wait for the current state update to complete, forming a serial execution chain. Traditional in-memory computing circuits, employing fixed computational granularity and data flow, are only compatible with static weight matrix-vector multiplication (MVM) operations and cannot accommodate Delta linear attention's closed-loop recursive computation process of "reading the state matrix - computation - writing back to update." Traditional systolic arrays, fixed pipeline CIM architectures, and existing similar technologies cannot decouple these recursive dependencies, leading to frequent pauses in the computation process and hardware limitations. High resource idle rate, low inference throughput, and latency that cannot meet the real-time inference requirements of long contexts; at the same time, linear attention based on Delta rules has unique hybrid data characteristics of static weights combined with dynamic state matrices and recursive update rules. Existing in-memory computing technologies are mostly designed for traditional self-attention or conventional neural networks and are mostly optimized for models such as Mamba. They have not fully adapted to the computational data flow of linear attention, and have not made specific optimizations for the recursive update logic of Delta rules. This further exacerbates the problems of low hardware resource utilization, high inference latency, and power redundancy, limiting the efficient deployment of linear attention mechanisms in long context inference of LLM. Summary of the Invention
[0007] This invention provides an in-memory computing acceleration system and method for linear attention, which addresses the shortcomings of existing technologies such as frequent data movement in Delta linear attention and the inability to decouple the recursive dependencies of Delta linear attention, resulting in frequent pauses in the computation process and failing to meet the requirements of real-time reasoning in long contexts. By decoupling the recursive dependencies of Delta rules, state updates and computations can be parallelized, eliminating data movement delays and idle waiting between modules.
[0008] This invention provides an in-memory computing acceleration system for linear attention, comprising a dual-wave pulsating in-memory computing pipeline control module, a ferroelectric weighted in-memory computing module, and a dynamic state in-memory computing module. The dual-wave pulsating in-memory computing pipeline control module sends scheduling commands to the ferroelectric weighted in-memory computing module and the dynamic state in-memory computing module respectively to control the working timing of the ferroelectric weighted in-memory computing module and to control the input of input terms into the ferroelectric weighted in-memory computing module. The ferroelectric weighted in-memory computing module, based on the received scheduling commands, performs in-situ multiplication and accumulation operations on the input terms and the static weights of the previously stored differential Delta linear attention model to obtain the corresponding query vector, key vector, and value vector; the input terms are obtained based on the input dialogue data. The dynamic state in-memory computing module, based on the received scheduling commands, retrieves the state data from the previous time step, updates the state data by combining the key vector and value vector, and obtains the attention calculation result for the corresponding input terms based on the query vector and the updated state data.
[0009] According to the present invention, an in-memory computing acceleration system for linear attention is provided, comprising a dynamic state storage module, including: a state reading unit, used to select a target state storage unit based on a received scheduling instruction and read the state data of the previous time step stored therein; a state storage unit, used to obtain an old state value based on the state data of the previous time step and a key vector; a vector processing unit, used to determine an update difference based on the old state value and the value vector, and to determine a value update vector based on a decay coefficient; the state storage unit is also used to update the state data of the previous time step based on the value update vector and the key vector, and to obtain the attention calculation result of the corresponding input word based on the query vector and the updated state data.
[0010] According to the present invention, a memory-based accelerated computing system for linear attention is provided. The dynamic state memory module adopts a hierarchical architecture and further includes multiple storage update components arranged in columns. Each storage update component includes an update unit and multiple state storage units. The state storage unit includes a first preset number of operation clusters, and each operation cluster includes a second preset number of storage columns. Each storage column includes a third preset number of dual-transistor 2T embedded dynamic random access memory (eDRAM). The target number of 2T eDRAMs in the same row are grouped together to jointly store state data corresponding to the target number of bits. The update unit is used to apply a write voltage through the write word line WWL to write the updated state data into the 2T eDRAM of the corresponding target state storage unit, thereby completing the in-situ update of the state matrix.
[0011] According to the present invention, a dual-wave pulsating in-memory computing pipeline control module for linear attention is further configured to: receive the input operation mode; when the operation mode is determined to be pulsating mode, control the query vector, key vector, and value vector to be sequentially input into each storage update component of the dynamic state in-memory computing module, so that the state in-memory computing unit performs multiplication and accumulation operations according to the data receiving order of the corresponding storage update component to obtain the attention calculation result of the corresponding input word; or, when the operation mode is determined to be reduction mode, control all query vectors, key vectors, and value vectors to be simultaneously sent to all storage update components of the dynamic state in-memory computing module, so that the state in-memory computing units corresponding to each storage update component complete multiplication and accumulation operations in parallel to obtain the attention calculation result of the corresponding input word.
[0012] According to the present invention, a dual-wave pulsating in-memory computing pipeline control module for linear attention-oriented in-memory computing is further configured to: when the operation mode is determined to be pulsating mode, use query vector, key vector, and value vector as input data, and group the input data according to the number of state storage units to obtain each group of data; input the first group of grouped data into the first state storage unit of the first column in the storage update component, so that the corresponding state in-memory computing unit can perform multiplication and accumulation operations based on the first group of grouped data input into the first state storage unit of the first column and the state data stored in the first state storage unit of the first column at the previous moment, to obtain the corresponding intermediate calculation result; input the second group of grouped data into the second state storage unit of the first column in the storage update component, so that the corresponding state in-memory computing unit can perform multiplication and accumulation operations based on the second group of grouped data input into the second state storage unit of the first column and the state data stored in the second state storage unit of the first column, to obtain the corresponding intermediate calculation result; and input the second group of grouped data into the second state storage unit of the first column in the storage update component, so that the corresponding state in-memory computing unit can perform multiplication and accumulation operations based on the second group of grouped data input into the second state storage unit of the first column and the state data stored in the second state storage unit of the first column. The state data from the previous time step stored in the storage unit is multiplied and accumulated to obtain the corresponding intermediate calculation result. This intermediate calculation result is then combined with the intermediate calculation result obtained from the previous state storage unit in the corresponding column and accumulated to obtain the intermediate cache result. The first group of grouped data is input into the first state storage unit of the second column in the storage update component, and the next first group of grouped data is input into the first state storage unit of the first column in the storage update component. This allows the corresponding state storage unit to perform multiplication and accumulation based on the first group of grouped data input from the first state storage unit of the second column and the previous time step state data stored in the first state storage unit of the second column to obtain the corresponding intermediate calculation result. The data is then cyclically controlled to be passed and calculated in the same column and subsequent column storage update components according to the above time sequence until the last state storage unit in each column storage update component outputs the corresponding intermediate cache result, thus obtaining the attention calculation result for the corresponding input word.
[0013] According to the present invention, a memory-based accelerated computing system for linear attention is provided, wherein the dynamic state memory module further includes a Booth encoder, wherein: the Booth encoder is used to encode the query vector, key vector and value vector respectively to generate control signals for the corresponding vectors; the control signals include a negative signal, a multiplication signal and a zero signal; the state memory unit is used to invert the corresponding vector according to the negative signal and participate in the operation, perform a preset shift operation on the corresponding vector according to the multiplication signal and participate in the operation, and / or skip the operation of the corresponding vector according to the zero signal.
[0014] According to the present invention, a storage-based accelerated computing system for linear attention is provided. The ferroelectric weight storage module includes a first weight storage unit, a second weight storage unit, and a third weight storage unit, which are used to perform in-situ multiplication and accumulation operations based on the input tokens and the corresponding stored static weights to obtain the corresponding query vector, key vector, and value vector. Each weight storage unit includes: a word line decoder, used to convert the input address into a word line enable signal based on the received scheduling instruction; and a weight storage unit, used to perform in-situ multiplication and accumulation operations between the weight data stored in the corresponding weight storage unit and the input tokens based on the received scheduling instruction and the word line enable signal to obtain the corresponding query vector, key vector, or value vector.
[0015] According to the present invention, a memory-based accelerated computing system for linear attention is provided. The weighted in-memory computing unit adopts a cross-array structure, including a fourth preset number of cross-array blocks. Each cross-array block includes a fifth preset number of weighted in-memory computing WCIM columns. Each WCIM column includes a source line, a bit line, a bit line capacitor, an enable terminal, an analog-to-digital converter, a shift adder, and a sixth preset number of weighted storage units. The enable terminal is used to pull the source line of each cross-array block to a read voltage according to the word line enable signal, so as to input the input words bit by bit into the cross-array block based on the received scheduling instructions. For each cross-array block, the input word bit data is broadcast to all WCIM columns via the word line to achieve parallel computing. The parallel computing includes: for each WCIM column, determining the threshold value at which the target weighted storage unit exists in the corresponding WCIM column. When the voltage is less than a preset voltage threshold, the target weight storage unit is turned on, providing current to the corresponding bit line to multiply the stored weight data with the corresponding input word bit data to obtain the corresponding product result; when the threshold voltage of the target weight storage unit in the corresponding WCIM column is greater than or equal to the preset voltage threshold, the target weight storage unit is turned off, no current is generated, and the stored weight data does not participate in the multiplication operation with the corresponding input word bit data; the bit line capacitor is used to accumulate the charge of all target weight storage units when the target weight storage unit is turned on to obtain the total charge; the analog-to-digital converter is used to quantize the total charge to obtain a partial sum; the shift adder is used to summarize the partial sum of the outputs of the analog-to-digital converters of all WCIM columns in the cross array block, perform cumulative quantization, and obtain the corresponding query vector, key vector, or value vector.
[0016] According to the present invention, an in-memory computing acceleration system for linear attention is provided, the system further includes a quantization and dequantization module, wherein: the dual-wave pulse in-memory computing pipeline control module is further used to extract floating-point input words from the global buffer and input them into the quantization and dequantization module; the quantization and dequantization module is used to convert the input floating-point input words into integers.
[0017] This invention also provides an in-memory computing acceleration method for linear attention, applied to any of the in-memory computing acceleration systems for linear attention described above, comprising: performing in-situ multiplication and accumulation operations on the input word and the static weights of the previously stored difference Delta linear attention model to obtain the corresponding query vector, key vector, and value vector; the input word is obtained based on the input dialogue data; updating the state data according to the key vector and value vector, combined with the state data of the previous time step, and performing multiplication and accumulation operations on the query vector and the updated state data to obtain the attention calculation result of the corresponding input word.
[0018] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the in-memory computing acceleration method for linear attention as described above.
[0019] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the in-memory computing acceleration method for linear attention as described above.
[0020] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the in-memory computing acceleration method for linear attention as described above.
[0021] The present invention provides an in-memory computing accelerated computing system and method for linear attention. A dual-wave pulsed in-memory computing pipeline control module sends scheduling instructions to the ferroelectric weight in-memory module and the dynamic state in-memory module to control the working sequence and the flow of input words, thereby decoupling the recursive dependency of Delta rules, achieving parallel state updates and computation, and eliminating data transfer delays and idle waiting between modules. The ferroelectric weight in-memory module performs in-situ multiplication and accumulation operations on the input words and the static weights of the previously stored differential Delta linear attention model. This leverages the non-volatility and in-situ computation advantages of ferroelectric storage to avoid the high energy consumption and bandwidth bottlenecks caused by frequent weight reads, achieving high-efficiency and high-density parallel generation of query, key, and value vectors. Furthermore, the dynamic state in-memory module calls the state data from the previous time step and updates it in conjunction with the key and value vectors, then interacts with the query vector to obtain the attention calculation result. This accurately captures the contextual dependencies that dynamically evolve with time steps in the dialogue data, achieving low-latency compression and efficient iteration of state information under the linear attention mechanism.
[0022] Furthermore, through the heterogeneous storage architecture of the ferroelectric weight storage module and the dynamic state storage module, the ferroelectric weight storage module is used for high-density, low-leakage storage of static weights, while the dynamic state storage module is used for high-frequency read / write and recursive updates of the dynamic state matrix. This allows for flexible adaptation to the differentiated needs of static weights and dynamic state matrices, overcoming the shortcomings of mismatch between storage media and data characteristics. It avoids the process of moving large amounts of weight data from memory to the computing unit, achieving in-situ data computation, thereby greatly reducing data movement. Through the overall coordination of dual-wave pulsation pipeline control, the deep coupling of ferroelectric in-situ static weight storage and dynamic state time-series evolution, it ensures that data is correctly transmitted when needed, thereby minimizing the number of data movements and latency between modules, avoiding resource waste and data accumulation caused by waiting. Thus, while strictly ensuring the accuracy of attention calculation, it endows the system with the outstanding ability to break through memory wall limitations and achieve extremely low power consumption and high throughput inference when processing long sequence dialogue data. Attached Figure Description
[0023] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0024] Figure 1 This is one of the structural schematic diagrams of the in-memory computing accelerated computing system for linear attention provided by the present invention; Figure 2 This is the second schematic diagram of the in-memory computing accelerated computing system for linear attention provided by the present invention. Figure 3 This is a logical schematic diagram of the ferroelectric weight storage module provided by the present invention; Figure 4 This is a schematic diagram of the working timing of the ferroelectric weight storage module provided by the present invention; Figure 5 This is a schematic diagram of the state matrix and working timing of the dynamic state storage module provided by the present invention; Figure 6 This is a schematic diagram of pipeline scheduling for the dual-wave pulsed storage pipeline control module provided by the present invention; Figure 7 This is a flowchart illustrating the in-memory computing acceleration method for linear attention provided by the present invention. Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0026] Figure 1 This is a schematic diagram of the in-memory computing acceleration system for linear attention provided by the present invention, as shown below. Figure 1 As shown, the system includes a dual-wave pulsed storage pipeline control module 10, a ferroelectric weight storage module 20, and a dynamic state storage module 30, wherein: The dual-wave pulsed storage pipeline control module 10 is used to send scheduling instructions to the ferroelectric weight storage module and the dynamic state storage module respectively, so as to control the working sequence of the ferroelectric weight storage module and the dynamic state storage module, and to control the input of input words into the ferroelectric weight storage module. The ferroelectric weight storage module 20 is used to perform in-situ multiplication and accumulation operations on the input words and the static weights of the previously stored differential Delta linear attention model based on the received scheduling instructions, so as to obtain the corresponding query vector, key vector and value vector; the input words are obtained based on the input dialogue data; The dynamic state storage module 30, based on the received scheduling instructions, calls up the state data of the previous moment, updates the state data by combining the key vector and value vector, and obtains the attention calculation result of the corresponding input word based on the query vector and the updated state data.
[0027] In this embodiment, reference Figure 2 The ferroelectric weight storage module includes a first weight storage unit, a second weight storage unit, and a third weight storage unit. These units are used to perform in-situ multiplication and accumulation operations based on the input tokens and the corresponding stored static weights to obtain the corresponding query vector, key vector, and value vector. Each weight storage unit includes: a word line decoder, used to convert the input address into a word line enable signal based on the received scheduling instructions; and a weight storage unit, used to perform in-situ multiplication and accumulation operations on the weight data stored in the corresponding weight storage unit and the input tokens based on the received scheduling instructions and the word line enable signals to obtain the corresponding query vector, key vector, or value vector.
[0028] It should be added that the word line decoder is also used to apply a preset programming voltage to the gate of the corresponding weight storage cell according to the weight data to be stored, so as to change the polarization state of the ferroelectric layer in the weight storage cell, thereby adjusting its threshold voltage and realizing the non-volatile storage of 1 bit of weight information. After programming, the weight data can be stably retained even if the power supply is disconnected, reducing static leakage current power consumption. Furthermore, the bit information of the weight with a low threshold voltage is 1, and the bit information of the weight with a high threshold voltage is 0. The threshold voltage is greater than or equal to the preset voltage threshold and is a high threshold voltage, and vice versa.
[0029] In addition, static weights include query projection weight Wq, key projection weight Wk, and value projection weight Wv; the first weight storage unit, the second weight storage unit, and the third weight storage unit are used to store the projection weights Wq, Wk, and Wv in Delta linear attention, respectively.
[0030] Specifically, refer to Figure 3 The weighted storage unit adopts a cross-array structure, including a fourth preset number of cross-array blocks. Each cross-array block includes a fifth preset number of weighted storage WCIM columns. Each WCIM column includes a source line, bit line, bit line capacitor, enable terminal, analog-to-digital converter, shift adder, and a sixth preset number of weighted storage units. Each cross-array block shares a set of address logic, including a word line (WL) decoder and bit line control circuitry, to ensure coordinated control of the read / write and calculation processes. Specifically: the enable terminal (EN) is used to pull the source line (SL) of each cross-array block to the read voltage according to the enable signal of the word line (WL), so that the input token is input bit by bit into the cross-array block based on the received scheduling instructions; for each cross-array block, the input token bit data is broadcast to all WCIM columns through the word line (WL) to achieve parallel computation; the parallel computation includes: for each WCIM column, determining the corresponding WCIM... When the threshold voltage of the target weight storage unit in a column is less than the preset voltage threshold, the target weight storage unit is turned on, providing current to the corresponding bit line (BL) to multiply the stored weight data with the corresponding input word bit data to obtain the corresponding product result; when the threshold voltage of the target weight storage unit in the corresponding WCIM column is greater than or equal to the preset voltage threshold, the target weight storage unit is turned off, no current is generated, and the stored weight data does not participate in the multiplication operation with the corresponding input word bit data; the bit line capacitor (CBL) is used to accumulate the charge of all target weight storage units when the target weight storage unit is turned on to obtain the total charge; the analog-to-digital converter (ADC) is used to quantize the total charge to obtain a partial sum; the shift adder is used to summarize the partial sum of the outputs of the ADCs of all WCIM columns in the cross array block, perform cumulative quantization, and obtain the corresponding query vector, key vector, or value vector.
[0031] It should be added that the fourth, fifth, and sixth preset quantities can be determined according to the actual weight data to be stored. For example, the fourth preset quantity can be 8, 16, or 64. When the fourth preset quantity is 8, the fifth preset quantity can be 128 and the sixth preset quantity can be 128. No further limitation is made here. The eight cross array blocks output eight 128-dimensional parts and data, which are accumulated and quantized to obtain a 1024-dimensional vector. Based on the 1024-dimensional vector output by the three weight storage units, the corresponding K vector, V vector, and Q vector are obtained as inputs to the dynamic state storage module.
[0032] In addition, the weighted storage cell can use a single-transistor 1T ferroelectric field-effect transistor (FeFET) to significantly reduce the circuit area, increase the storage density, and adapt to the large-scale storage requirements of static weights. By relying on the advantages of FeFET devices such as non-volatility, CMOS compatibility, high read / write speed and high density, low leakage current and high-density storage of static weights can be achieved, while completing analog domain multiply-accumulate (MAC) operations and reducing data movement.
[0033] In addition, during the startup phase, the source line (SL) needs to be grounded first, and a high voltage needs to be applied to the enable terminal (EN) to pull the bit line (BL) to the ground potential through the precharge transistor. Before the enable terminal inputs the input words bit by bit into the cross array block, it first converts the input words into word line voltages through the word line decoder, and then inputs them bit by bit into the cross array block.
[0034] Furthermore, each weight storage unit stores one bit of weight data, and N consecutive weight storage units form a group to collaboratively store N bits of weight data, where N is an integer greater than 1, to balance calculation precision and storage efficiency and meet different precision requirements. It should be noted that N can be chosen according to actual needs, such as 4 or 8, etc., without further limitation here.
[0035] It should be noted that the ferroelectric weighted computing module mainly consists of three stages: weight writing, multiplication and accumulation, and idle. It leverages the ferroelectric polarization characteristics of FeFET devices and the conductivity characteristics of transistors to achieve weight storage and in-situ computation. The core computing unit can control the current flowing through it by controlling the on / off state of the transistors, enabling in-situ multiplication of input and stored data. Furthermore, thanks to the non-volatile nature of FeFETs, the power supply to the ferroelectric weighted computing module can be directly cut off during the idle period, retaining the weight data without additional refresh operations. This significantly reduces static power consumption and adapts to the low-power deployment requirements at the edge.
[0036] For example, refer to Figure 4When IN_0=1 and IN_1=0, WL <0> High level, WL <1> When N_0=0 and IN_1=1, WL... <0> Low level, WL <1> When IN_0=1 and IN_1=1, WL... <0> and WL <1> Both are at high level, and the BL voltage rises significantly. A high level corresponds to a threshold voltage greater than or equal to a preset voltage threshold, while a low level corresponds to a threshold voltage less than the preset voltage threshold.
[0037] In an optional embodiment, the dynamic state storage module includes: a state reading unit, configured to select a target state storage unit based on a received scheduling instruction and read the previously stored state data; a state storage unit, configured to obtain an old state value based on the previously stored state data and a key vector; a vector processing unit, configured to determine an update difference based on the old state value and the value vector, and determine a value update vector based on a decay coefficient; the state storage unit is further configured to update the previously stored state data based on the value update vector and the key vector, and obtain the attention calculation result for the corresponding input lexical based on the query vector and the updated state data.
[0038] It should be noted that the dynamic state storage module is mainly used to store and process the dynamic state matrix S in Delta linear attention, adapting to its high-frequency read / write, high endurance, and low-power recursive update requirements. It forms a heterogeneous division of labor with FeFET WCIM to jointly solve the problem of poor compatibility of hybrid data storage.
[0039] It should be added that 2T eDRAM is selected as the storage medium for the dynamic state storage module to overcome the shortcomings of traditional SRAM, such as large area, high power consumption, poor non-volatile storage durability, and unsuitability for high-frequency updates. It meets the requirements of frequent read and write and real-time updates of the dynamic state matrix S. A hierarchical architecture is adopted, equipped with update unit and state storage unit to realize in-situ calculation and real-time update of state data, and adapt to the recursive characteristics of Delta rule.
[0040] Furthermore, the 2T eDRAM includes a gate transistor (MG) and a storage transistor (MS). The MG is responsible for writing state data, and the MS is responsible for reading state data. The parasitic capacitance of the MS gate serves as the data storage carrier. Thus, only two transistors are needed to realize data reading and writing, and the reading and writing process only requires a single bit line to operate. Compared with traditional SRAM, simulation results show that its energy consumption can be reduced by more than 30 times, significantly reducing the power consumption of dynamic updates. At the same time, the frequent update operations inherent in Delta rules can serve as the implicit refresh mechanism of eDRAM, effectively mitigating its volatility defect, realizing refresh-free operation in the active reasoning process, ensuring the continuity of token processing, and the slight leakage characteristics of eDRAM can achieve moderate forgetting of the state matrix, adapting to the learning and forgetting mechanism of Delta linear attention.
[0041] In addition, during reading, the dynamic state storage module selects the target state storage unit through word line RWL according to the scheduling instructions of the dual-wave pulse storage pipeline control module, turns on MS, and transmits the state data stored in the parasitic capacitance to the state storage unit through word line RBL to complete the reading of the state matrix.
[0042] Furthermore, the read state data from the previous moment is multiplied and accumulated in the state storage unit along with the query vector, key vector, and value vector output by the ferroelectric weight storage module. Specifically, the old state value is represented as: in, express The old state value at time 1; Indicates the previous one Status data at any given moment; express The key vector at time step.
[0043] Value update vector, represented as: in, express Update the vector with the value at each time step; This indicates that the difference has been updated; express The value vector at any given time; This represents the attenuation coefficient.
[0044] Furthermore, the system includes a nonlinear function module, used to input the value vector and the old state value into a nonlinear sigmoid function before the vector processing unit determines the value update vector, and to determine the decay coefficient using a piecewise second-order approximation method. The piecewise second-order approximation method involves using the value vector and the old state value as input values, dividing the domain of the input values into multiple intervals; within each interval, a preset quadratic polynomial, such as ax², is used. 2 The curve of the Sigmoid function is approximated by bx + c, and the attenuation coefficient is obtained, thereby greatly reducing the hardware complexity while ensuring a certain level of accuracy.
[0045] The updated status data is represented as follows: in, This represents the updated state data, i.e. Status data at any given moment; This indicates transpose.
[0046] The attention calculation result is expressed as: in, express The result of the mental effort calculation at any given moment; express The query vector at time step.
[0047] Specifically, refer to Figure 5 The dynamic state storage module adopts a hierarchical architecture and also includes multiple storage update components arranged in columns. Each storage update component includes an update unit and multiple state storage units. The state storage unit includes a first preset number of operation clusters, and each operation cluster includes a second preset number of storage columns. Each storage column includes a third preset number of dual-transistor 2T embedded dynamic random access memory (eDRAM). The target number of 2T eDRAMs in the same row are grouped together to jointly store state data corresponding to the target number of bits. The update unit is used to apply a write voltage through the write word line WWL to write the updated state data into the 2T eDRAM of the corresponding target state storage unit, thus completing the in-situ update of the state matrix.
[0048] It should be added that the input dimension of the Dynamic State Inventory (SCIM) should be consistent with the output dimension of the Ferroelectric Weight Inventory (WCIM). That is, the column length of the SCIM corresponds to the column length of the WCIM, and the horizontal length of the SCIM should be consistent with its column length. Figure 5 There are 16 lanes in the middle, and each lane contains 8 eight-bit numbers in one row.
[0049] In addition, the number of storage update components, update units, state storage units, and state computing units, as well as the first preset number, second preset number, third preset number, and target number, can be set according to actual design requirements. For example, when the fourth preset number is 8, the fifth preset number is 128, and the sixth preset number is 128, there can be 16 storage update components, 1 update unit, 16 state storage units, 16 state computing units, 16 first preset number, 8 second preset number, 8 third preset number, and 8 target number. No further limitations are made here.
[0050] In other words, the dynamic state storage module includes 16 columns of storage update components. Each column of storage update components includes an update unit and 16 state storage units. Each state storage unit contains 16 operation clusters, and each operation cluster includes 8 storage columns. Each local column includes 8 2T eDRAMs. Eight adjacent eDRAM units in the same row form a group and jointly store an 8-bit state value to adapt to the precision requirements of the state matrix.
[0051] Furthermore, each arithmetic cluster is equipped with a state storage unit, which can use a local addition tree to perform multiplication and accumulation operations on 8 input data channels. A single arithmetic cluster can simultaneously support parallel computation of 8 output channels. All storage columns within an arithmetic cluster share the global write bit line (GWBL) and global read bit line (GRBL), while also supporting independent read and write operations for individual units, thereby achieving synchronous computation and updates within the block and improving operational efficiency. Word lines (WWL, RWL) are shared between storage columns, further optimizing circuit area.
[0052] In addition, each arithmetic cluster is equipped with a partial buffer for storing the results of multiply-accumulate operations.
[0053] In an optional embodiment, the in-situ update of the state matrix of the update unit and the corresponding in-situ multiplication and accumulation operation are performed in the ferroelectric weight storage module controlled by the dual-wave pulsating storage pipeline control module. Parallel overlap can be achieved through pulsating pipeline scheduling, decoupling recursive dependencies and avoiding computational pauses.
[0054] In an optional embodiment, to improve computational efficiency and utilize data sparsity, the dynamic state storage module further includes a Booth encoder, wherein: the Booth encoder is used to encode the query vector, key vector, and value vector respectively to generate control signals for the corresponding vectors; the control signals include a negative signal, a doubling signal, and a zero signal; the state storage unit is used to invert the corresponding vector according to the negative signal and participate in the calculation, perform a preset shift operation on the corresponding vector according to the doubling signal and participate in the calculation, and / or skip the calculation of the corresponding vector according to the zero signal, thereby improving the sparsity of the input and reducing invalid computation.
[0055] In an optional embodiment, the dual-wave pulsating storage pipeline control module is further configured to: receive the input operation mode; when the operation mode is determined to be pulsating mode, control the query vector, key vector, and value vector to be input sequentially into each storage update component of the dynamic state storage module, so that the state storage unit performs multiplication and accumulation operations according to the data receiving order of the corresponding storage update component to obtain the attention calculation result of the corresponding input word; or, when the operation mode is determined to be reduction mode, control all query vectors, key vectors, and value vectors to be simultaneously sent to all storage update components of the dynamic state storage module, so that the state storage units corresponding to each storage update component complete multiplication and accumulation operations in parallel to obtain the attention calculation result of the corresponding input word.
[0056] Furthermore, the dual-wave pulsating storage pipeline control module is also used to: when the operation mode is determined to be pulsating mode, take the query vector, key vector, and value vector as input data, and group the input data according to the number of state storage units to obtain each group of data; input the first group of grouped data into the first state storage unit of the first column in the storage update component, so that the corresponding state storage unit can perform multiplication and accumulation operations based on the first group of grouped data input into the first state storage unit of the first column and the state data of the previous moment stored in the first state storage unit of the first column to obtain the corresponding intermediate calculation result; input the second group of grouped data into the second state storage unit of the first column in the storage update component, so that the corresponding state storage unit can perform multiplication and accumulation operations based on the second group of grouped data input into the second state storage unit of the first column and the state data of the previous moment stored in the second state storage unit of the first column. The state data is multiplied and accumulated to obtain the corresponding intermediate calculation result. This intermediate calculation result is then combined with the intermediate calculation result obtained from the previous state storage unit in the corresponding column and accumulated to obtain the intermediate cache result. The first group of grouped data is input into the first state storage unit of the second column in the storage update component, and the next first group of grouped data is input into the first state storage unit of the first column in the storage update component. This allows the corresponding state storage unit to perform multiplication and accumulation based on the first group of grouped data input from the first state storage unit of the second column and the state data stored in the first state storage unit of the second column from the previous time step, to obtain the corresponding intermediate calculation result. The data is then cyclically controlled to be passed and calculated in the same column and subsequent column storage update components according to the above time sequence until the last state storage unit in each column storage update component outputs the corresponding intermediate cache result, thus obtaining the attention calculation result for the corresponding input word.
[0057] For example, refer to Figure 6In pulse mode, the first group of data (IN_0~IN_7) is input into the first state storage unit (0,0) of the first column. After one pulse cycle, the multiplication-accumulation operation is completed, and the first group of data (IN_0~IN_7) is transferred to the first state storage unit (0,1) of the second column. At the same time, the second group of data (IN_8~IN_15) is input into the second state storage unit (1,0) of the first column, and the next first group of data is input into (0,0). (0,1), (0,0), and (1,0) respectively complete the corresponding multiplication-accumulation operation based on the corresponding input. (1,0) also adds the intermediate calculation result obtained by itself to the intermediate calculation result obtained by (0,0) based on the previous group (IN_0~IN_7) to obtain the intermediate cache result; in this way, after 16 pulse cycles, the last state storage unit (15,0) of the first column outputs the first group of intermediate cache results, and the subsequent results are output in sequence. Thus, relying on the dual-wave pulse storage pipeline control module, the whole process scheduling is realized, giving full play to the heterogeneous collaborative advantages of the ferroelectric weight storage module and the dynamic state storage module, realizing the overlapping execution of multi-word parallel computing and state update, greatly improving the inference throughput and reducing latency.
[0058] It should be added that the results of each operation cluster are temporarily stored in a partial buffer, and then transferred through the data stream between blocks to complete the cumulative operation.
[0059] In addition, the dual-wave pulsed storage pipeline control module adopts a dual-wave parallel scheduling logic of computation wave and update wave. It controls the computation wave to perform the current word state reading and intermediate calculation, and controls the update wave to perform the previous word state update and the next word preprocessing. This allows the state update and subsequent calculation to be executed in parallel and overlapped, decoupling the recursive dependency of Delta rules. Through the controller, it achieves time-series coordination, realizes parallel state update and calculation, eliminates calculation pauses, improves hardware utilization, improves inference efficiency, and adapts to the real-time inference requirements of long context.
[0060] In an optional embodiment, the system further includes a quantization and dequantization module, wherein: the dual-wave pulsed storage pipeline control module is further configured to extract floating-point input tokens from the global buffer and input them into the quantization and dequantization module; the quantization and dequantization module is configured to convert the input floating-point input tokens into integers.
[0061] In summary, the dual-wave pulsed storage pipeline control module sends scheduling instructions to the ferroelectric weight storage module and the dynamic state storage module to control the working sequence and the flow of input words, thereby decoupling the recursive dependency of Delta rules, achieving parallel state updates and computation, and eliminating data transfer delays and idle waiting between modules. The ferroelectric weight storage module performs in-situ multiplication and accumulation operations on the input words and the static weights of the previously stored differential Delta linear attention model, leveraging the non-volatility and in-situ computation advantages of ferroelectric storage to avoid the high energy consumption and bandwidth bottlenecks caused by frequent weight reads, achieving high-efficiency and high-density parallel generation of query, key, and value vectors. Furthermore, the dynamic state storage module calls the state data from the previous time step and updates it in combination with the key and value vectors, then interacts with the query vector to obtain the attention calculation result, thereby accurately capturing the contextual dependencies that dynamically evolve with time steps in the dialogue data, achieving low-latency compression and efficient iteration of state information under the linear attention mechanism.
[0062] Furthermore, through the heterogeneous storage architecture of the ferroelectric weight storage module and the dynamic state storage module, the ferroelectric weight storage module is used for high-density, low-leakage storage of static weights, while the dynamic state storage module is used for high-frequency read / write and recursive updates of the dynamic state matrix. This allows for flexible adaptation to the differentiated needs of static weights and dynamic state matrices, overcoming the shortcomings of mismatch between storage media and data characteristics. It avoids the process of moving large amounts of weight data from memory to the computing unit, achieving in-situ data computation, thereby greatly reducing data movement. Through the overall coordination of dual-wave pulsation pipeline control, the deep coupling of ferroelectric in-situ static weight storage and dynamic state time-series evolution, it ensures that data is correctly transmitted when needed, thereby minimizing the number of data movements and latency between modules, avoiding resource waste and data accumulation caused by waiting. Thus, while strictly ensuring the accuracy of attention calculation, it endows the system with the outstanding ability to break through memory wall limitations and achieve extremely low power consumption and high throughput inference when processing long sequence dialogue data.
[0063] The in-memory computing acceleration device for linear attention provided by the present invention will be described below. The in-memory computing acceleration device for linear attention described below can be referred to in correspondence with the in-memory computing acceleration method for linear attention described above.
[0064] Figure 7 A flowchart illustrating a linear attention-oriented in-memory computing acceleration method is shown, applicable to any of the linear attention-oriented in-memory computing acceleration systems described above. The method includes: S71, perform in-situ multiplication and accumulation operations on the input words and the static weights of the previously stored difference Delta linear attention model to obtain the corresponding query vector, key vector and value vector; the input words are obtained based on the input dialogue data; S72 updates the state data based on the key vector and value vector, combined with the state data from the previous time step, and performs multiplication and accumulation operations based on the query vector and the updated state data to obtain the attention calculation result for the corresponding input word.
[0065] It should be noted that the specific principles of the embodiments of the present invention are the same as those of the system embodiments described above. For details, please refer to the system embodiments above. More detailed explanations will not be repeated here.
[0066] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840. The processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute an in-memory computation acceleration method for linear attention. This method includes: performing in-situ multiplication and accumulation operations on the input lexical units and the static weights of the previously stored difference Delta linear attention model to obtain the corresponding query vector, key vector, and value vector; the input lexical units are obtained based on the input dialogue data; updating the state data based on the key vector and value vector, combined with the state data from the previous time step, and performing multiplication and accumulation operations on the query vector and the updated state data to obtain the attention calculation result for the corresponding input lexical unit.
[0067] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0068] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the in-memory computing acceleration method for linear attention provided by the above methods. The method includes: performing in-situ multiplication and accumulation operations on the input lexical and the static weights of the previously stored difference Delta linear attention model to obtain the corresponding query vector, key vector, and value vector; the input lexical is obtained based on the input dialogue data; the state data is updated according to the key vector and value vector, combined with the state data of the previous time step, and multiplication and accumulation operations are performed according to the query vector and the updated state data to obtain the attention calculation result of the corresponding input lexical.
[0069] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the in-memory computing acceleration method for linear attention provided by the above methods. The method includes: performing in-situ multiplication and accumulation operations on input words and the static weights of a previously stored difference Delta linear attention model to obtain corresponding query vectors, key vectors, and value vectors; the input words are obtained based on the input dialogue data; updating the state data according to the key vectors and value vectors, combined with the state data of the previous time step, and performing multiplication and accumulation operations on the query vectors and the updated state data to obtain the attention calculation result for the corresponding input words.
[0070] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0071] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A memory-based accelerated computing system for linear attention, characterized in that, It includes a dual-wave pulsed storage pipeline control module, a ferroelectric weight storage module, and a dynamic state storage module, among which: The dual-wave pulsed storage pipeline control module is used to send scheduling instructions to the ferroelectric weight storage module and the dynamic state storage module respectively, so as to control the working timing of the ferroelectric weight storage module and the dynamic state storage module, and to control the input of input words into the ferroelectric weight storage module. The ferroelectric weight storage module is used to perform in-situ multiplication and accumulation operations on the input lexical units and the static weights of the previously stored differential Delta linear attention model based on the received scheduling instructions, so as to obtain the corresponding query vector, key vector and value vector; the input lexical units are obtained based on the input dialogue data. The dynamic state storage module, based on the received scheduling instruction, calls up the state data from the previous moment, updates the state data by combining the key vector and the value vector, and obtains the attention calculation result of the corresponding input word based on the query vector and the updated state data.
2. The in-memory computing acceleration system for linear attention as described in claim 1, characterized in that, The dynamic state storage module includes: The status reading unit is used to select a target status storage unit based on the received scheduling instructions and read the status data of the previous moment stored in the corresponding unit. The state storage unit is used to obtain the old state value based on the state data of the previous time step and the key vector; A vector processing unit is used to determine the update difference based on the old state value and the value vector, and to determine the value update vector in combination with the decay coefficient; The state storage unit is further configured to update the state data of the previous time step based on the value update vector and the key vector, and obtain the attention calculation result of the corresponding input word based on the query vector and the updated state data.
3. The in-memory computing acceleration system for linear attention as described in claim 2, characterized in that, The dynamic state storage module adopts a hierarchical architecture and further includes multiple storage update components arranged in columns. Each storage update component includes an update unit and multiple state storage units, wherein: The state storage unit includes a first preset number of operation clusters, and each operation cluster includes a second preset number of storage columns, and each storage column includes a third preset number of dual-transistor 2T embedded dynamic random access memory (eDRAM). The target number of 2T eDRAMs in the same row form a group for jointly storing state data with a bit length corresponding to the target number. The update unit is used to apply a write voltage through the write word line WWL to write the updated state data into the 2T eDRAM of the corresponding target state storage unit, thereby completing the in-situ update of the state matrix.
4. The in-memory computing acceleration system for linear attention as described in claim 3, characterized in that, The dual-wave pulsed storage pipeline control module is also used for: Upon receiving the input operation mode, if the operation mode is determined to be a pulse mode, the query vector, key vector, and value vector are controlled to be input sequentially into each storage update component of the dynamic state storage module, so that the state storage unit performs multiplication and accumulation operations according to the data receiving order of the corresponding storage update components to obtain the attention calculation result of the corresponding input word; or, When the operation mode is determined to be the reduction mode, all query vectors, key vectors and value vectors are simultaneously sent to all storage update components of the dynamic state storage module, so that the state storage units corresponding to each storage update component can perform multiplication and accumulation operations in parallel to obtain the attention calculation results of the corresponding input lexical.
5. The in-memory computing acceleration system for linear attention as described in claim 4, characterized in that, The dual-wave pulsed storage pipeline control module is also used for: When the operation mode is determined to be pulse mode, the query vector, key vector and value vector are used as input data, and the input data is grouped according to the number of state storage units to obtain each group of data; The first group of grouped data is input into the first state storage unit of the first column in the storage update component, so that the corresponding state storage unit can perform multiplication and accumulation operations based on the first group of grouped data input into the first state storage unit of the first column and the state data of the previous moment stored in the first state storage unit of the first column to obtain the corresponding intermediate calculation result. The second group of grouped data is input into the second state storage unit of the first column in the storage update component, so that the corresponding state storage unit can perform multiplication and accumulation operations based on the second group of grouped data input into the second state storage unit of the first column and the state data stored in the second state storage unit of the first column at the previous moment to obtain the corresponding intermediate calculation result. The intermediate calculation result obtained from the previous state storage unit of the corresponding column is then accumulated to obtain the intermediate cache result. The first group of grouped data is input into the first state storage unit of the second column in the storage update component, and the next first group of grouped data is input into the first state storage unit of the first column in the storage update component, so that the corresponding state storage unit can perform multiplication and accumulation operations based on the first group of grouped data input into the first state storage unit of the second column and the state data stored in the first state storage unit of the second column at the previous moment to obtain the corresponding intermediate calculation result. The data is passed and calculated in the same and subsequent columns of the storage update component according to the above time sequence control until the last state storage unit in each column storage update component outputs the corresponding intermediate cache result, thus obtaining the attention calculation result of the corresponding input word.
6. The in-memory computing acceleration system for linear attention as described in claim 2, characterized in that, The dynamic state storage module also includes a Booth encoder, wherein: The Booth encoder is used to encode the query vector, key vector, and value vector respectively, and generate control signals for the corresponding vectors; the control signals include negative signals, multiplication signals, and zero signals; The state storage unit is used to invert the corresponding vector according to the negative signal and then participate in the calculation, perform a preset shift operation on the corresponding vector according to the multiplication signal and then participate in the calculation, and / or skip the calculation of the corresponding vector according to the zero signal.
7. The in-memory computing acceleration system for linear attention as described in claim 1, characterized in that, The ferroelectric weight storage module includes a first weight storage unit, a second weight storage unit, and a third weight storage unit, which are used to perform in-situ multiplication and accumulation operations based on the input tokens and the corresponding stored static weights to obtain the corresponding query vector, key vector, and value vector. Each weight storage unit includes: A word line decoder is used to convert the input address into a word line enable signal based on the received scheduling instructions. The weight storage unit is used to perform in-situ multiplication and accumulation operations on the weight data stored in the corresponding weight storage unit and the input word byte based on the received scheduling instruction and the enable signal of the word line, so as to obtain the corresponding query vector, key vector or value vector.
8. The in-memory computing acceleration system for linear attention according to claim 7, characterized in that, The weighted storage unit adopts a cross-array structure, including a fourth preset number of cross-array blocks. Each cross-array block includes a fifth preset number of weighted in-memory calculation WCIM columns. Each WCIM column includes a source line, a bit line, a bit line capacitor, an enable terminal, an analog-to-digital converter, a shift adder, and a sixth preset number of weighted storage units, wherein: The enable terminal is used to pull the source line of each of the cross array blocks to the read voltage according to the enable signal of the word line, so as to input the input word into the cross array block bit by bit based on the received scheduling instruction; For each of the cross array blocks, the input lexical bit data is broadcast to all WCIM columns via word lines to achieve parallel computation; the parallel computation includes: For each WCIM column, when it is determined that the threshold voltage of the target weight storage unit in the corresponding WCIM column is less than the preset voltage threshold, the target weight storage unit is turned on to provide current to the corresponding bit line so as to perform a multiplication operation between the corresponding stored weight data and the corresponding input word bit data to obtain the corresponding product result. When it is determined that the threshold voltage of the target weight storage unit in the corresponding WCIM column is greater than or equal to the preset voltage threshold, the target weight storage unit is turned off and no current is generated. The corresponding stored weight data does not participate in the multiplication operation with the corresponding input word bit data. The bit line capacitor is used to accumulate the charge of all target weight storage cells when the target weight storage cell is turned on, so as to obtain the total charge. The analog-to-digital converter is used to quantize the total charge to obtain a partial sum; The shift adder is used to summarize the partial sum of the outputs of all WCIM column analog-to-digital converters in the cross array block, perform cumulative quantization, and obtain the corresponding query vector, key vector, or value vector.
9. The in-memory computing acceleration system for linear attention as described in claim 1, characterized in that, The system also includes quantization and dequantization modules, wherein: The dual-wave pulsed storage pipeline control module is also used to extract floating-point input tokens from the global buffer and input them into the quantization and dequantization module; The quantization and dequantization module is used to convert the input floating-point input tokens into integers.
10. A method for accelerating in-memory computing based on linear attention, applied to the in-memory computing system for accelerating in-memory computing based on linear attention as described in any one of claims 1-9, characterized in that, include: The input tokens are multiplied and accumulated in situ with the static weights of the previously stored difference Delta linear attention model to obtain the corresponding query vector, key vector and value vector. The input lexical units are obtained based on the input dialogue data; Based on the key vector and the value vector, the state data is updated by combining the state data from the previous time step, and a multiplication and accumulation operation is performed on the query vector and the updated state data to obtain the attention calculation result for the corresponding input word.
