A method and system for hardware-accelerated deployment of a gated linear attention model

Abstract

The application discloses a hardware acceleration deployment method and system of a gated linear attention model, and belongs to the field of collaborative design of storage-computing integration and novel algorithm model architecture. The method comprises the following steps: performing query, key, value and gated projection through a storage-computing weight array; reading out a predicted value of a current key from an old state through a state array; calculating a prediction residual in a digital control unit and modulating the prediction residual through a write gate; converting residual vector and key vector formed outer product formula update into state write driven by row-column pulse coincidence; and controlling old state retention proportion or erasing intensity by using a decay gate or an erasing gate. The application matches the state update process of linear attention with the matrix readout and pulse write mechanism of a cross array, can reduce the dynamic attention graph and Softmax special calculation path in standard attention, reduce the memory access and data transfer overhead in long sequence reasoning, and improve the deployment efficiency of the Transformer facing the storage-computing integration platform.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary technical fields of artificial intelligence model inference acceleration, Transformer structure optimization and hardware acceleration (Computing-in-Memory, CIM), specifically involving a hardware acceleration deployment method and system for a gated linear attention model. Background Technology

[0002] With the widespread application of large-scale Transformer models in tasks such as natural language processing, multimodal understanding, and long-context inference, the overhead of memory access, caching, and sequential mixed computation during model deployment has become a major bottleneck for system efficiency. In-memory computing technology, by performing multiply-accumulate operations within or near the storage array, allows model weights or state data to participate in computation while in a resident state, thereby reducing frequent data transfers between memory and computing units. This type of technology has a high degree of compatibility with the matrix-vector multiplication, matrix multiplication, and state update operations that are prevalent in neural network inference, and therefore shows a positive development trend in edge intelligence, low-power inference, dedicated AI accelerators, and high-energy-efficiency model deployment scenarios.

[0003] Standard self-attention typically requires explicitly calculating the correlation between the query matrix and the key matrix, i.e. And further perform Softmax normalization and multiplication of the value matrix. For a length of The sequence, this process will form a sequence that grows in length as the sequence length increases. Dynamic attention graphs, relying on a continuously growing key-value cache in autoregressive inference, result in numerous off-chip memory accesses and intermediate result relocations. Linear attention and state-space attention, by compressing the sequence history into fixed-size states, can avoid explicit attention to some extent. Attention Matrix. Gated Delta-type linear attention further describes state updates as follows: first, the old state is read using the current key vector to obtain a prediction of the current value vector; then, a residual is generated based on the prediction error; finally, the residual and the current key vector are combined to form an outer product incremental write-back of the state. This computational form naturally includes the process of "reading the state, calculating the residual, writing the state, and gated forgetting," which has a potential correspondence with the read-out, write-out, erase, and conductance retention mechanisms of devices. Specifically, in-memory computing platforms, by storing weights or states in the form of conductance in the cross array, use Ohm's law and Kirchhoff's current law to perform matrix-vector multiplication, making them suitable for performing matrix computations with high reuse and low handling. However, the dynamic attention matrix, Softmax exponent normalization, maximum value comparison, and dynamic intermediate result write-back operations in standard self-attention are not static matrix-vector multiplications that in-memory computing cross arrays are best at. Direct mapping can easily introduce compute-write-compute dependencies, ADC / DAC round trips, on-chip cache pressure, and control logic complexity.

[0004] Existing deployment methods for in-memory computing platforms primarily focus on weight mapping, quantization sharding, fault injection, network architecture search, or standard matrix multiplication operator deployment. They pay insufficient attention to the state reconstruction of the Transformer attention mechanism itself, the mapping of dynamic inference states, gated residual outer product writing, and the erase / write scheduling mechanism. Therefore, it is necessary to propose a gated Delta-state linear attention mapping method for in-memory computing platforms. This method transforms the sequence hybrid operators in Transformer long-sequence inference from dynamic attention graphs into persistent, readable, writable, and schedulable state memories, thereby improving deployment efficiency and specialized adaptability on in-memory computing platforms. Summary of the Invention

[0005] The purpose of this invention is to provide a hardware-accelerated deployment method and system for a gated linear attention model, which solves the problems of dynamic attention graph size expansion, Softmax nonlinear complexity, large KV cache memory access overhead, and difficulty in efficiently writing dynamic intermediate results to the cross array on a standard Transformer attention platform.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a hardware-accelerated deployment method for a gated linear attention model, comprising the following steps:

[0007] Perform projection calculation on the input vector of the current token to obtain the current query vector. Current key vector Current value vector Write gate and attenuation gate ;

[0008] The runtime state matrix in gated Delta-state linear attention Mapped to an on-chip state storage plane;

[0009] Based on the current key vector From the old state matrix Read the current value vector from Predicted value ;

[0010] Based on the current value vector and the predicted value Generate prediction residuals; based on the write gate Modulate the predicted residual to obtain the residual vector to be written. ;

[0011] Based on the residual vector to be written With the current key vector Generate outer product state update quantity ;

[0012] The outer product state update amount The row and column pulses converted into a state array are written as control signals; and based on the attenuation gate... Alternatively, an erase gate can be used to control the preservation, decay, or erasure of the old state, resulting in an updated state matrix. ;

[0013] Based on the current query vector Read the updated state matrix Get the attention output of the current token. The final output of the current token is obtained by output projection. .

[0014] Furthermore, the projection calculation performed on the input vector of the current token includes: mapping the query projection, key projection, value projection, gated projection, and output projection parameters in the Transformer layer to the weight plane, and performing the corresponding matrix-vector multiplication through a cross array.

[0015] Furthermore, the state storage plane is implemented by a memory device state array, or by a combination of a memory device state array and an SRAM state buffer.

[0016] Furthermore, the outer product state update quantity The result of the cross product with the residual to be written and the key vector They are proportional and satisfy a linear relationship.

[0017] Further, converting the outer product state update quantity into row and column pulse write control signals for the state array includes: writing the residual vector to be written. Encoded as a row-direction pulse sequence of a storage device state array, the current key vector The column direction pulse sequence is encoded as a state array of in-memory computing devices. When the row direction pulse and the column direction pulse coincide at the intersection, the conductance write is performed on the in-memory computing device cell corresponding to the intersection.

[0018] Furthermore, the number of pulse coincidences or the probability of coincidence at the intersection point is proportional to the product of the residual component to be written and the key vector component, so that the conductance change at the intersection point is proportional to the corresponding element of the outer product state update quantity.

[0019] Furthermore, the basic computational relationship of the gated Delta state linear attention can be expressed as:

[0020] ;

[0021] ;

[0022] ;

[0023] ;

[0024] ;

[0025] in, For the current input, , , and These are the query, key, value, and output projection weights, respectively. To write the gate, For attenuation gate, For the first Step state matrix, The outer product increment of the state matrix to be written.

[0026] Furthermore, the runtime state matrix in the gated Delta-state linear attention... A differential conductance array is used, where each state element is represented by the difference between the positive and negative conductance elements:

[0027] ;

[0028] in, and The first and second lines in the positive and negative arrays, respectively. Line number The conductivity value of the column, is the scaling factor. Differential representation can support positive and negative values ​​in the state matrix, and positive and negative updates can be mapped to SET / RESET or conductance changes in the positive and negative arrays, respectively.

[0029] Furthermore, the target conductance change in the external product state update can be expressed as:

[0030] ;

[0031] in, Write the scaling factor to the state; if pulse count encoding is used, then the first... Line 1 The number of write pulses at the column intersection can be approximately determined by the following formula:

[0032] ;

[0033] in, This represents the unit conductance change caused by a single effective pulse. When When positive, perform a SET write to the positive array or a RESET erase to the negative array; when... When the value is negative, a SET write operation is performed on the negative array or a RESET erase operation is performed on the positive array.

[0034] Furthermore, the outer product state update is implemented using a random pulse coincidence mechanism; the absolute value of the residual vector is encoded as the probability of row pulse occurrence. The absolute value of the key vector is encoded as the probability of column pulse occurrence. ,exist If row and column pulses are generated independently within each time slice, then the first... Line 1 The expected number of pulse coincidences at column intersections is:

[0035] ;

[0036] When each overlapping pulse causes a unit conductance change At that time, there were:

[0037] ;

[0038] because and Proportional and Proportional, therefore the expected conductance update at the crossover point is related to Proportional. Thus, this invention achieves external product state writing by means of pulse overlap number or probability, rather than requiring the direct multiplication of two analog voltages at a single crossover point.

[0039] Furthermore, attenuation gate This is mapped to state hold, bleed, normalized scaling, or erase pulse control. For example, when When it approaches 1, the state Keep; when When the state is small, a controlled RESET pulse is applied to the state array or scaling is performed in the digital domain to decay the old state. When an erase-write decoupled gating structure is used, the erase gate and the write gate control the erase intensity associated with the old state and the write intensity of the new residual information, respectively.

[0040] Furthermore, the write gate controls one or more of the write pulse amplitude, pulse width, pulse count, or pulse occurrence probability of the memory device state array.

[0041] Furthermore, the attenuation gate or erase gate controls one or more of the following: RESET pulse, state hold ratio, state attenuation ratio, or digital scaling factor of the memory device state array.

[0042] This invention also provides a gated Delta-state linear attention deployment system for in-memory computing, comprising:

[0043] The storage device weight plane is used to store the static weights of query projection, key projection, value projection, gated projection and output projection, and to perform the corresponding matrix-vector multiplication to obtain the current query vector, current key vector, current value vector, write gate and decay gate;

[0044] The state storage plane is used to store the runtime state matrix of gated Delta-state linear attention. The state storage plane may include a state array, an SRAM state buffer, or a combination of both.

[0045] State reading unit, used based on the current key vector From the old state matrix Read the current value vector from Predicted value Based on the current query vector Read the updated state matrix , obtain attention output ;

[0046] The residual generation and gating unit is used to generate and gate the current value vector. and the predicted value Generate prediction residuals and based on write gates Generate the residual to be written ;

[0047] The pulse coding and driving unit is used to encode the residual to be written into a row-direction pulse sequence, encode the current key vector into a column-direction pulse sequence, and control the change of cross-point conductance according to the sign, amplitude, pulse width, pulse number or pulse probability.

[0048] The erase and hold control unit is used to control the hold, decay, or RESET operation of the state array according to the decay gate or erase gate;

[0049] An output projection unit is used to perform output projection on the attention output to obtain the final output of the current token;

[0050] The control and scheduling unit is used to coordinate the timing dependencies between state readout, residual calculation, outer product writing, erase and hold, and output readout.

[0051] The beneficial effects of this invention are as follows:

[0052] 1. This invention reconstructs the dynamic attention graph in Transformer into a read-write update of a fixed-size state matrix, reducing explicit attention operations. Hardware burden of attention matrix and Softmax main path;

[0053] 2. This invention maps the prediction residual outer product term in the gated Delta state update to a pulsed writing process of a cross array, so that the algorithm state update matches the hardware physical writing mechanism;

[0054] 3. This invention improves the controllability of state updates and hardware interpretability by controlling the write intensity, erase intensity and hold ratio of the state array through write gate, attenuation gate or write-erase decoupling gating.

[0055] 4. This invention employs differential state representation and threshold write control, which can support positive and negative states, sparse writing, and half-selection cell disturbance suppression;

[0056] 5. This invention can serve as a dedicated in-memory computational hybrid operator for long-sequence Transformer inference, providing a foundation for subsequent quantization, non-ideal modeling, state-resident scheduling, and hardware-software co-optimization. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This invention provides a heterogeneous execution framework for the CPU control domain and the in-memory accelerator.

[0059] Figure 2 This is a schematic diagram of the linear attention calculation process for the gated Delta state of the present invention;

[0060] Figure 3 This is a schematic diagram of resource mapping and token-by-token timing of state-resident linear attention in in-memory computing according to the present invention.

[0061] Figure 4 This is a schematic diagram of the differential conductance mapping and readout of the signed matrix in this invention;

[0062] Figure 5 This is a schematic diagram of the row and column pulse overlap outer product writing of the present invention. Detailed Implementation

[0063] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0064] This invention provides a hardware-accelerated deployment method for a gated linear attention model, comprising the following steps:

[0065] Perform projection calculation on the input vector of the current token to obtain the current query vector. Current key vector Current value vector Write gate and attenuation gate .

[0066] The runtime state matrix in gated Delta-state linear attention It is mapped to an on-chip state storage plane.

[0067] Based on the current key vector Read old state , obtain the old state For the current value vector Predicted value .

[0068] Based on the current value vector and the predicted value Generate prediction residuals; based on the write gate Modulate the predicted residual to obtain the residual vector to be written. .

[0069] Based on the residual vector to be written With the current key vector Generate outer product state update quantity .

[0070] The outer product state update amount The row and column pulses converted into a state array are written as control signals; and based on the attenuation gate... Alternatively, an erase gate can be used to control the preservation, decay, or erasure of the old state, resulting in an updated state matrix. .

[0071] Based on query vector Read the updated state Get the attention output of the current token. The final output is obtained through output projection. .

[0072] As a preferred embodiment, to implement the above method, the present invention also provides a gated Delta-state linear attention deployment system for in-memory computing, comprising:

[0073] The storage device weight plane is used to store the static weights of query projection, key projection, value projection, gated projection, and output projection, and to perform the corresponding matrix-vector multiplication.

[0074] The state storage plane is used to store the runtime state matrix of gated Delta-state linear attention. The state storage plane may include a state array, an SRAM state buffer, or a combination of both.

[0075] The state reading unit is used to read the current key vector. Read old state and obtain the old state. For the current value vector Predicted value and based on the query vector Read update status And obtain attention output .

[0076] The residual generation and gating unit is used to generate and gate the current value vector. and the predicted value Generate prediction residuals and based on write gates Generate the residual to be written .

[0077] The pulse coding and driving unit is used to encode the residual to be written into a row-direction pulse sequence, encode the current key vector into a column-direction pulse sequence, and control the change of cross-point conductance according to the sign, amplitude, pulse width, pulse number or pulse probability.

[0078] The erase / hold control unit is used to control the hold, decay, or reset operation of the state array according to the decay gate or erase gate.

[0079] The control and scheduling unit is used to coordinate the timing dependencies between state readout, residual calculation, outer product writing, erase and hold, and output readout.

[0080] Example 1: Gated Delta-State Linear Attention Mapping Based on State Array

[0081] In this embodiment, the Transformer layer to be deployed uses gated Delta-state linear attention to replace or partially replace standard Softmax attention. For the input token... First, the query, key, value, and gated projection are executed in parallel in the weight plane to obtain the current query vector. Current key vector Current value vector Write gate and attenuation gate Since the aforementioned projection weights are static parameters during the inference phase, they are suitable for infrequent writing and long-term residence on the weight plane. The overall heterogeneous execution framework is as follows: Figure 1 As shown, the specific mapping between the weight plane and the state plane, and the token-by-token execution process are as follows: Figure 2 and Figure 3 As shown.

[0082] State matrix It is stored as runtime sequence memory in the state storage plane. For the current key vector The status reading unit will The voltage is converted to an analog input voltage or a segmented input and applied to the corresponding port of the state array. The predicted value is obtained by utilizing the current accumulation characteristic of the cross array of the memory devices. .

[0083] The digital control unit receives the current value vector. and predicted value Calculate the difference between the two. - and write gate Modulate the difference to obtain the residual to be written. The residual represents the prediction error of the old state's association with the current key value, and is used to guide local corrections to the state matrix.

[0084] Subsequently, the pulse coding and driving unit will and These are converted into pulse sequences in the row and column directions, respectively. Row-direction pulses represent the "written content" or the value dimension to be corrected, while column-direction pulses represent the "written position" or the key dimension. When row and column pulses coincide at the intersection point in the same time slice, the voltage across the memory cell exceeds the write threshold, resulting in a change in conductance. After accumulating over multiple time slices, the conductance change at the intersection point... Proportional, thus completing the outer product state update.

[0085] Before or after writing, the erase / retain control unit according to The old state can be preserved, decayed, or erased. The final state is updated to... It can be used to read the output of the current token and to recursively calculate the state of the next token.

[0086] In this embodiment, the basic computational relationship of the gated Delta state linear attention can be specifically expressed as follows:

[0087] ;

[0088] ;

[0089] ;

[0090] ;

[0091] ;

[0092] in, For the current input, , , and These are the query, key, value, and output projection weights, respectively. To write the gate, For attenuation gate, For the first Step state matrix, The outer product increment of the state matrix to be written.

[0093] Example 2: Differential Conductivity State Representation and Symbol Update

[0094] In this embodiment, as Figure 4 As shown, to support positive and negative values ​​in the state matrix, each state element... It can be represented by two device conductivity units or two sets of subarrays, i.e. .

[0095] in, and The first and second lines in the positive and negative arrays, respectively. Line number The conductivity value of the column, is the scaling factor. Differential representation can support positive and negative values ​​in the state matrix, and positive and negative updates can be mapped to SET / RESET or conductance changes in the positive and negative arrays, respectively.

[0096] When the target update amount When positive, the system can increase or reduce Achieve; when the target update amount When it is negative, the system can increase or reduce accomplish.

[0097] To reduce writing complexity, a phase-by-phase writing method can be used: the first phase is processed... The intersection, the second phase processing The intersection point; alternatively, a sign-separated row and column pulse coding method can be used to map the positive residual, negative residual, positive key, and negative key to different pulse channels or different differential arrays respectively.

[0098] In this embodiment, the target conductance change of the outer product state update can be expressed as:

[0099] ;

[0100] in, Write the scaling factor to the state; if pulse count encoding is used, then the first... Line number The number of write pulses at the column intersection can be approximately determined by the following formula:

[0101] ;

[0102] in, This represents the unit conductance change caused by a single effective pulse. When When positive, perform a SET write to the positive array or a RESET erase to the negative array; when... When the value is negative, a SET write operation is performed on the negative array or a RESET erase operation is performed on the positive array.

[0103] Example 3: Pulse coincidence outer product writing and threshold anti-disturbance

[0104] In this embodiment, the instantaneous voltage across the memory device unit is determined by the difference between the row line voltage and the column line voltage, i.e. Where i represents the row index of the state array, j represents the column index of the state array, and t represents the pulse time slice; This represents the instantaneous voltage across the storage device at the intersection of the i-th row and j-th column. This represents the voltage of the i-th row line at time t. This represents the voltage of the j-th line at time t.

[0105] This invention does not require a single crossover point to directly perform multiplication on two analog input voltages. Instead, it uses the overlap probability or overlap count of row and column pulses to approximate the outer product update.

[0106] Specifically, at the selected intersection, when both row and column pulses occur simultaneously, a voltage difference exceeding the write threshold is formed; in the half-selection state with only row pulses or only column pulses, the voltage difference is below the write threshold, thereby preventing erroneous writing to unselected cells. The change in cell conductance can be adjusted by controlling the pulse amplitude, width, number of pulses, or probability of occurrence.

[0107] In the implementation of random pulses, residual components Control the first Line pulse occurrence probability, key components Control the first The probability of column pulses occurring. If row and column pulses are independent, the probability of them coinciding in the same time slice is the product of their probabilities. Therefore, the expected conductance change at the intersection point after multiple time slices is related to... Proportional. The outer product of overlapping row and column pulses is written to the structure as follows: Figure 5 As shown.

[0108] In a further implementation, the absolute value of the residual vector is encoded as the probability of line pulse occurrence. The absolute value of the key vector is encoded as the probability of column pulse occurrence. ,exist If row and column pulses are generated independently within each time slice, then the first... Line number The expected number of pulse coincidences at column intersections is:

[0109] ;

[0110] When each overlapping pulse causes a unit conductance change At that time, there were:

[0111] ;

[0112] because and Proportional and Proportional, therefore the expected conductance update at the crossover point is related to Proportional. Thus, this invention achieves external product state writing by means of pulse overlap number or probability, rather than requiring the direct multiplication of two analog voltages at a single crossover point.

[0113] Example 4: Extended Implementation of Erase / Write Decoupling Gating

[0114] In this embodiment, the gated Delta state linear attention can be extended to a structure with decoupled erase and write gates. The write gate controls the intensity of residual information being written to the state array, while the erase gate controls the removal or attenuation of old key-value associations. Correspondingly, in the in-memory device state array, the write gate can adjust the amplitude, width, number, or probability of the SET pulse, while the erase gate can adjust the intensity of the RESET pulse or digital scaling.

[0115] The above-described extension does not change the basic technical idea of ​​the present invention, namely, obtaining the predicted value through state reading, determining the written content through the predicted residual, determining the write position through the key vector, and controlling the state erasure intensity through the gating signal.

[0116] Example 5: Differences from Standard Attention Mapping

[0117] Standard Softmax attention requires explicit formation Dynamic attention graphs rely on operations such as exponentiation, normalization, maximum value comparison, and key-value cache reads. Direct deployment to an in-memory computing platform typically requires additional digital logic, lookup tables, comparison and selection circuitry, and intermediate result write-back.

[0118] This invention does not focus on hardware implementation of Softmax, but rather reconstructs the dynamic token-token correlation matrix into a fixed-size state matrix from the perspective of the attention mechanism. The state matrix can reside on the on-chip state storage plane, and both state reading and state writing have a clear in-memory computing mapping method, thus forming a state-type sequence hybrid operator that is friendly to in-memory computing.

[0119] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A hardware-accelerated deployment method for a gated linear attention model, characterized in that, include: Perform projection calculation on the input vector of the current token to obtain the current query vector, the current key vector, the current value vector, the write gate, and the decay gate; Map the runtime state matrix in gated Delta-state linear attention to an on-chip state storage plane; The predicted value of the current value vector is read from the old state matrix based on the current key vector; Generate prediction residuals based on the current value vector and the predicted values; The predicted residual vector to be written is obtained by modulating the write gate based on the write gate; Generate an outer product state update quantity based on the residual vector to be written and the current key vector; Converting the external product state update quantity into row and column pulse write control signals for the state array includes: encoding the residual to be written into a row direction pulse sequence of the in-memory device state array, encoding the current key vector into a column direction pulse sequence of the in-memory device state array, and performing conductance write on the in-memory device cell corresponding to the intersection point when the row direction pulse and the column direction pulse coincide at the intersection point; the number of pulse coincidences or the coincidence probability at the intersection point is proportional to the product of the residual component to be written and the key vector component, so that the conductance change at the intersection point is proportional to the corresponding element of the external product state update quantity; and controlling the retention, decay, or erasure of the old state based on the attenuation gate or the erase gate to obtain the updated state matrix; Based on the current query vector, the updated state matrix is ​​read to obtain the attention output of the current token, and the attention output is projected to obtain the final output of the current token.

2. The method according to claim 1, characterized in that, The projection calculation on the input vector of the current token includes: mapping the query projection, key projection, value projection, gated projection and output projection parameters in the Transformer layer to the weight plane, and performing the corresponding matrix-vector multiplication through the cross array.

3. The method according to claim 1, characterized in that, The state storage plane is implemented by a state array of a memory computing device, or by a combination of a state array of a memory computing device and an SRAM state buffer.

4. The method according to claim 1, characterized in that, The outer product state update amount is determined by the outer product of the residual vector to be written and the current key vector.

5. The method according to claim 1, characterized in that, The runtime state matrix in the gated Delta state linear attention is represented by a differential conductance array, where each state element is represented by the difference between the positive and negative conductance units.

6. The method according to claim 1, characterized in that, The write gate controls one or more of the write pulse amplitude, pulse width, pulse count, or pulse occurrence probability of the memory device state array.

7. The method according to claim 1, characterized in that, The attenuation gate or erase gate controls one or more of the following: RESET pulse, state hold ratio, state attenuation ratio, or digital scaling factor of the in-memory device state array.

8. A gated Delta-state linear attention deployment system for in-memory computing, used to execute the method according to any one of claims 1 to 7, characterized in that, include: The storage device weight plane is used to store the static weights of query projection, key projection, value projection, gated projection and output projection, and to perform the corresponding matrix-vector multiplication to obtain the current query vector, current key vector, current value vector, write gate and decay gate; The state storage plane is used to store the runtime state matrix of gated Delta-state linear attention; The state reading unit is used to read the predicted value of the current value vector from the old state matrix based on the current key vector; And based on the current query vector, read the updated state matrix to obtain the attention output; The residual generation and gating unit is used to generate a prediction residual based on the current value vector and the prediction value, and to generate a write residual based on the write gate. The pulse coding and driving unit is used to encode the residual to be written into a row-direction pulse sequence, encode the current key vector into a column-direction pulse sequence, and control the change of cross-point conductance according to the sign, amplitude, pulse width, pulse number or pulse probability. The erase and hold control unit is used to control the hold, decay, or erase of the state array according to the decay gate or erase gate; An output projection unit is used to perform output projection on the attention output to obtain the final output of the current token; The control and scheduling unit is used to coordinate the timing dependencies between state readout, residual calculation, outer product writing, erase and hold, and output readout.

Images