A method, apparatus, and chip for in-memory computing of selective state-space models based on scaling labels and low-rank residual buffers.

Abstract

This invention discloses a method, apparatus, and chip for in-memory computation of selective state-space models based on scaling labels and low-rank residual buffers. The method is applied to an accelerator comprising a differential RRAM main state-in-memory array, a row-level scaling label controller, and an SRAM low-rank residual buffer. The method includes: storing the main body of the state matrix of the same layer and head of the selective state-space model in the differential RRAM main state-in-memory array; storing the cumulative decay factors corresponding one-to-one with each row of the state matrix as row-level scaling labels; and storing multiple outer product update terms not yet merged into the differential RRAM main state-in-memory array as vector pairs in the SRAM low-rank residual buffer. This invention eliminates the dependence of selective decay on element-by-element writing back to the main state matrix and eliminates the requirement for outer product updates to immediately fill the entire main state matrix at each time step, while maintaining the semantics of the model output.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit power consumption modeling and electronic design automation technology, and in particular to a selective state-space model in-memory computing method, device and chip based on scaling labels and low-rank residual buffers. Background Technology

[0002] In long sequence modeling tasks, Selective State Space Models (SSMs) typically use a recursive approach with a state matrix to retain historical information. For each time step, the state matrix undergoes row-by-row decay related to the selective decay vector, as well as writing the outer product formed by the update vector and the value vector. Furthermore, in the output phase, the state matrix is ​​multiplied by the query vector. While existing Computing-in-Memory (CIM) solutions can efficiently perform matrix-vector multiplication, when the state matrix resides directly with real values, the time-step decay and outer product updates still trigger large-scale read-write operations, leading to high foreground write bandwidth, high array write frequency, and high state access latency.

[0003] Especially in SSM, selective decay typically involves scaling each row of the state matrix with different coefficients. If the implementation of element-wise multiplication and write-back of the state matrix is ​​still used, the entire state matrix still needs to be accessed for the decay process, even if the main state matrix already resides in the in-memory array. At the same time, if the outer product update is performed by writing the entire state matrix word by word in real time, the array write path will be in a busy state for a long time, thus offsetting the throughput advantage of the in-memory array in matrix-vector multiplication.

[0004] Therefore, a new state representation is needed so that selective decay no longer depends on writing back the master state matrix element by element, and that outer product updates no longer require the entire master state matrix to be filled immediately at each time step, while maintaining the semantics of the model output. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, and chip for in-memory computing of selective state-space models based on scaling labels and low-rank residual buffers, in order to solve the aforementioned technical problems. The various technical effects of the preferred solutions among the many technical solutions provided by this invention are detailed below.

[0006] To achieve the above objectives, the present invention provides the following technical solution: This invention provides a selective state-space model in-memory computing method based on scaling tags and low-rank residual buffers, applicable to an accelerator including a differential RRAM main state in-memory array, a row-level scaling tag controller, and an SRAM low-rank residual buffer. The method includes: For the state matrix of the same layer and the same head of the selective state space model, the main part of the state matrix is ​​stored in the differential RRAM main state storage array, the cumulative decay factor corresponding to each row of the state matrix is ​​stored as row-level scaling label, and multiple outer product update terms that have not yet been merged into the differential RRAM main state storage array are stored in the SRAM low-rank residual buffer in the form of vector pairs. Generate a selective decay vector, an update vector, a value vector, and a query vector based on the current lexical unit, and update the row-level scaling label row by row based on the selective decay vector; The update vector is reverse scaled according to the updated row-level scaling label to obtain a normalized update vector. The normalized update vector and the value vector are written into the SRAM low-rank residual buffer as the current outer product update term. The query vector is forward scaled according to the row-level scaling label to obtain the scaled query vector; the differential RRAM master state memory array performs matrix-vector multiplication on the scaled query vector to obtain the master state output; The residual compensation output is calculated based on the scaled query vector and the multiple outer product update terms. The main state output is then added to the residual compensation output to obtain the current output.

[0007] In some embodiments, after updating the row-level scaling label row by row according to the selective decay vector, element-wise decay write-back is not performed on the master state matrix stored in the differential RRAM master state memory array.

[0008] In some embodiments, after writing the normalized update vector and the value vector as the current outer product update term into the SRAM low-rank residual buffer, the outer product result of the normalized update vector and the value vector is not immediately written into the differential RRAM master state storage array.

[0009] In some embodiments, the SRAM low-rank residual buffer includes a first buffer matrix for storing multiple normalized update vectors, a second buffer matrix for storing multiple value vectors, and a valid bit register; each valid row of the first buffer matrix is ​​used to store a normalized update vector, the corresponding valid row of the second buffer matrix is ​​used to store a value vector, and the low-rank residual matrix is ​​formed by accumulating the vector pairs in all valid buffer slots.

[0010] In some embodiments, when the integer portion of any row-level scaling label reaches the saturation warning boundary of the label format's representable integer range, a renormalization operation is performed, the steps of which include: Take the integer part of the line-level scaling label as Update the row-level scaling label based on the integer part; synchronously multiply the corresponding row of the master state matrix and the corresponding components of each effective normalized update vector in the SRAM low-rank residual buffer by... .

[0011] In some embodiments, the differential RRAM master state memory array uses positive and negative differential conductance cells to represent the signed master state matrix; during background merging, the positive value part of the incremental matrix is ​​converted into a positive write pulse sequence, the negative value part is converted into a negative write pulse sequence, and a verification write is performed after each write to correct the nonlinear write error of the RRAM device.

[0012] In some embodiments, it also includes: When the SRAM low-rank residual buffer reaches a preset maximum depth or the storage bank containing the differential RRAM main state memory array is in an idle state, multiple outer product update terms cached in the SRAM low-rank residual buffer are batch merged into the differential RRAM main state memory array.

[0013] According to another aspect of the present invention, a selective state-space model in-memory computing device based on scaling labels and low-rank residual buffers is also provided, comprising: The parameter generation unit is used to generate selective decay vectors, update vectors, value vectors, and query vectors based on the current lexical units. Differential RRAM master state memory array is used to store the master state matrix and perform matrix-vector multiplication; A row-level scaling label controller is used to store and update the row-level scaling labels corresponding to each row of the main state matrix; A normalized outer product generation unit is used to perform inverse scaling on the update vector according to the row-level scaling label to obtain a normalized update vector; A low-rank SRAM residual buffer is used to cache multiple outer product update terms formed by the normalized update vector and the value vector; A scaling query generation unit is used to perform forward scaling on the query vector according to the row-level scaling label to obtain a scaled query vector. The residual compensation calculation unit is used to calculate the residual compensation output based on the scaled query vector and the multiple outer product update terms. The background merging unit is used to batch merge the multiple outer product update items into the differential RRAM main state memory array when the SRAM low-rank residual buffer reaches a preset maximum depth or the memory where the differential RRAM main state memory array is located is in an idle state. The renormalization unit is used to migrate the exponential factor between the row-level scaling label, the master state matrix, and the normalization update vector when the row-level scaling label reaches the saturation warning boundary.

[0014] In some embodiments, the parameter generation unit is an on-chip digital projection unit; the on-chip digital projection unit receives the hidden vector of the current word and outputs the selective decay vector, the update vector, the value vector and the query vector; the normalized outer product generation unit and the scaling query generation unit adjust the exponent of the vector using block floating-point data format.

[0015] According to another aspect of the present invention, a chip is also provided, including the selective state-space model in-memory computing device based on scaling tags and low-rank residual buffers as described above, and further including an on-chip interconnect network for transmitting data corresponding to the current lexical unit among the differential RRAM main state-memory array, the row-level scaling tag controller, the SRAM low-rank residual buffer and the parameter generation unit.

[0016] Implementing one of the above-described technical solutions of the present invention has the following advantages or beneficial effects: This invention transforms selective decay from matrix-level element-wise multiplication and write-back to row-level scaling label updates, changing the decay front-end path from accessing the entire state matrix to accessing only row-level labels; it absorbs cumulative decay to the query vector side, allowing the differential RRAM master state storage array to continue performing its preferred matrix-vector multiplication, and stabilizes the dynamic range of row-level scaling labels and master state matrix through a renormalization mechanism, adapting to long sequence inference scenarios. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 This is a flowchart of a selective state-space model in-memory computation method based on scaling labels and low-rank residual buffers, according to an embodiment of the present invention. Figure 2 This is a structural block diagram of a selective state-space model in-memory computing device based on scaling labels and low-rank residual buffers, according to an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the present invention clearer, various exemplary embodiments described below will be referenced to the accompanying drawings, which form part of the exemplary embodiments, illustrating various exemplary embodiments that may be used to implement the present invention. Unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. It should be understood that they are merely examples of processes, methods, and apparatuses consistent with some aspects of the present invention disclosed as detailed in the appended claims, and other embodiments may be used, or structural and functional modifications may be made to the embodiments listed herein without departing from the scope and spirit of the present invention.

[0019] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," etc., indicate the orientation or positional relationship based on the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the referred element must have a specific orientation, or be constructed and operated in a specific orientation. The terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. The term "multiple" means two or more. The terms "connected" and "linked" should be interpreted broadly, for example, they can be fixed connections, detachable connections, integral connections, mechanical connections, electrical connections, communication connections, direct connections, indirect connections through an intermediate medium, and can be the internal connection of two elements or the interaction relationship between two elements. The term "and / or" includes any and all combinations of one or more of the related listed items. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0020] To illustrate the technical solution described in this invention, specific embodiments are described below, showing only the parts related to the embodiments of this invention.

[0021] Example 1: like Figure 1 As shown in this embodiment, a selective state-space model in-memory computing method based on scaling tags and low-rank residual buffers is provided. This method is applied to an accelerator including a differential RRAM main state-in-memory array, a row-level scaling tag controller, and an SRAM low-rank residual buffer. The method includes: S100. For the state matrix of the same layer and the same head of the selective state space model, store the main part of the state matrix in the differential RRAM master state storage array, store the cumulative decay factor corresponding to each row of the state matrix as row-level scaling labels, and store multiple outer product update terms that have not yet been merged into the differential RRAM master state storage array in the form of vector pairs in the SRAM low-rank residual buffer.

[0022] This step enables the true state matrix S of the same layer and the same head of the selective state-space model to be represented in a joint form of the row scaling matrix, the master state matrix, and the low-rank residual matrix.

[0023] In a specific embodiment, the true state matrix S is not directly written into the differential RRAM master state memory array as its actual values. Instead, it is interpreted as a joint form of the master state matrix, row-level scaling labels, and low-rank residuals. Specifically, the true state matrix S and the row-level scaling matrix satisfy the following relationship: ; in, Represents the master state matrix. Represents a low-rank residual matrix. This represents a row-level scaling matrix composed of row-level scaling labels. ‎ indicates row-level scaling of the label vector. Each element in the array is subjected to a base-2 exponentiation operation. This means converting the input vector into a diagonal matrix.

[0024] Therefore, each row of the true state matrix is ​​first summed by the corresponding row of the master state matrix and the corresponding row of the low-rank residual, and then uniformly scaled by the cumulative decay factor given by the row-level scaling label.

[0025] Furthermore, the low-rank residual matrix P is formed by accumulating multiple outer product update terms in the SRAM low-rank residual buffer. The SRAM low-rank residual buffer includes a first buffer matrix A, a second buffer matrix V, and a valid bit register.

[0026] The b-th valid buffer slot in the SRAM low-rank residual buffer is used to store the b-th normalized update vector and the b-th value vector, respectively. That is, the τ-th valid row (buffer slot) of A is used to store a normalized update vector a. τ The τth valid row (buffer slot) of V is used to store a value vector v. τ Therefore, the low-rank residual matrix satisfies the following relationship: ; in, This represents the summation over all effective buffer slots.

[0027] S200: Generate a selective decay vector, an update vector, a value vector, and a query vector based on the current terminology, and update the row-level scaling labels row by row based on the selective decay vector.

[0028] In a specific embodiment, the on-chip digital projection unit receives the hidden vector x of the current word and outputs the selective decay vector d, the update vector k, the value vector v, and the query vector q.

[0029] Furthermore, during word-by-word meta-processing, the line-level scaling tags are updated first. The line-level scaling tag controller executes line by line based on the current selective decay vector d. The line-level scaling tag vector... The i-th component Update as follows: ; in, Scaling label vectors by row The i-th component The quantity before the update; It is the smallest positive number that the accelerator can represent; This is the i-th component (or the i-th row) of the selective decay vector.

[0030] In this embodiment, when the selective decay vector d is generated by the time step vector Δ and the state transition parameter A, the row-level scaling label controller directly generates the label increment of the i-th row according to Δ(i)A(i) / ln2, thereby avoiding the need to calculate the exponential decay value first and then take the logarithm.

[0031] In some embodiments, after updating the row-level scaling labels row by row according to the selective decay vector, element-wise decay write-back is not performed on the master state matrix stored in the differential RRAM master state memory array. Since this step only updates the same number of labels as the number of rows in the state matrix, and does not access all elements of the master state matrix, the selective decay is transformed from the original whole matrix read-write path to a label-level update path.

[0032] S300. Perform reverse scaling on the update vector according to the updated row-level scaling label to obtain the normalized update vector. Write the normalized update vector and the value vector as the current outer product update term into the SRAM low-rank residual buffer.

[0033] After updating the row-level scaling labels, the normalized outer product generation unit performs reverse scaling on the update vector k based on the updated row-level scaling labels to obtain the normalized update vector a, which satisfies the following relationship: ; Where a(i) is the i-th component (row) of the normalized update vector, and k(i) is the i-th component (row) of the update vector.

[0034] In this embodiment, the update vector k and the query vector q adopt the Block Floating Point (BFP) format. The normalized outer product generation unit achieves reverse scaling by adjusting the shared exponent and mantissa compensation lookup table, thereby avoiding the need to configure a full floating-point multiplier for each element.

[0035] In some embodiments, after writing the normalized update vector and the value vector as the current outer product update term into the SRAM low-rank residual buffer, the outer product result of the normalized update vector and the value vector is not immediately written into the differential RRAM master state storage array.

[0036] After the normalized update vector 'a' is generated, it is written together with the current value vector 'v' into a free slot in the low-rank residual buffer of the SRAM. During the write process, A stores 'a', V stores 'v', and the valid bit register marks the corresponding buffer slot as valid. At this time, the outer product of the normalized update vector 'a' and the value vector 'v' is not immediately written into the differential RRAM master state storage array, thus avoiding triggering a write operation for the entire master state matrix at every time step.

[0037] In some embodiments, when the integer portion of any row-level scaling label reaches the saturation warning boundary of the label format's representable integer range, a renormalization operation is performed, the steps of which include: Take the integer part of the line-level scaling label as Update the row-level scaling label based on the integer part; synchronously multiply the corresponding components of the corresponding row of the master state matrix and each effective normalized update vector in the SRAM low-rank residual buffer by... . Specifically, To prevent the continuous accumulation of integer parts of row-level scaling labels in long sequences, which could lead to dynamic range imbalance, this embodiment also includes a renormalization unit. When the integer part of any row-level scaling label reaches the saturation warning boundary of the integer range that the label format can represent, the renormalization unit takes the integer part of that row-level scaling label as... and execute: Subsequently, the corresponding rows of the master state matrix and the corresponding components of each effective normalized update vector in the SRAM low-rank residual buffer are simultaneously multiplied by... In this way, the corresponding rows of the real state matrix remain unchanged, while the main state matrix and row scaling labels are both pulled back to a dynamic range suitable for the current numerical format.

[0038] S400: Perform forward scaling on the query vector according to the row-level scaling label to obtain the scaled query vector; perform matrix-vector multiplication on the scaled query vector by the differential RRAM master state storage array to obtain the master state output.

[0039] During the output phase, the scaling query generation unit performs forward scaling on the query vector q based on the updated row-level scaling labels, resulting in a scaled query vector q′, which satisfies the following relationship: ; in, The i-th component (row) of the scaled query vector. Query the i-th component (row) of the vector.

[0040] The differential RRAM master state storage array receives the scaled query vector q′ and performs matrix-vector multiplication to obtain the master state output. Simultaneously, the residual compensation calculation unit reads all valid buffer slots in the SRAM low-rank residual buffer, first calculating the inner product of the normalized update vector and the scaled query vector in each valid buffer slot, then using the resulting inner product as weights to perform a weighted summation of the corresponding value vectors to obtain the residual compensation output. The final output satisfies the following relationship: ; in, Master state matrix transpose, To update the normalized vector The transpose of .

[0041] Since the out-of-product update terms that are not written back have been added to the final result through residual compensation output, the semantics of the model output remain consistent with those of reading directly from the real state matrix, even though the low-rank residuals have not yet been merged into the master state matrix.

[0042] S500: Calculate the residual compensation output based on the scaled query vector and multiple outer product update terms, and add the main state output and the residual compensation output to obtain the current output.

[0043] In some embodiments, it also includes: When the SRAM low-rank residual buffer reaches the preset maximum depth or the memory bank where the differential RRAM main state memory array is located is in an idle state, multiple outer product update items cached in the SRAM low-rank residual buffer are batch merged into the differential RRAM main state memory array.

[0044] In this embodiment, the background merging unit is triggered in two situations: first, when the SRAM low-rank residual buffer reaches a preset maximum depth. Secondly, the memory bank containing the differential RRAM master state in-memory array is in an idle state. After the background merging unit is triggered, it reads the normalized update vector and value vector stored in all valid buffer slots and calculates the master state matrix increment using the following formula: .

[0045] Since the differential RRAM master state memory array uses positive and negative differential conductance cells to represent the signed master state matrix, the background merging unit first decomposes the increment matrix into positive and negative parts, and then converts them into positive and negative write pulse sequences, respectively, and writes them to the corresponding differential cells. A verification write is performed after each write operation to reduce the nonlinear write error of the RRAM device. After batch merging is completed, the background merging unit clears the valid bits of the corresponding buffer slots, and the corresponding positions in A and V can then be reused in subsequent time steps.

[0046] In summary, this method transforms selective decay from matrix-level element-wise multiplication and write-back to row-level scaling label updates, changing the decay front-end path from accessing the entire state matrix to accessing only row-level labels; it transforms the word-by-word complete outer product write-back into a combined path of front-end residual compensation and back-end batch merging, reducing front-end write bandwidth and array write frequency; it absorbs cumulative decay to the query vector side, allowing the differential RRAM master state storage array to continue performing its preferred matrix-vector multiplication; and it stabilizes the dynamic range of row-level scaling labels and the master state matrix through a renormalization mechanism, adapting to long sequence inference scenarios.

[0047] Example 2: like Figure 2 As shown, this embodiment also provides an in-memory computing device for selective state-space models based on scaling labels and low-rank residual buffers, including: The parameter generation unit is used to generate selective decay vectors, update vectors, value vectors, and query vectors based on the current lexical.

[0048] Differential RRAM master state memory array is used to store the master state matrix and perform matrix-vector multiplication; The row scaling label controller is used to store and update the row scaling labels corresponding to each row of the master state matrix.

[0049] The normalized outer product generation unit is used to perform inverse scaling on the update vector according to the row scaling label to obtain the normalized update vector.

[0050] The SRAM low-rank residual buffer is used to cache multiple outer product update terms formed by the normalized update vector and the value vector.

[0051] The scaling query generation unit is used to perform forward scaling on the query vector based on the row-level scaling labels to obtain the scaled query vector.

[0052] The residual compensation calculation unit is used to calculate the residual compensation output based on the scaled query vector and multiple outer product update terms.

[0053] The background merging unit is used to batch merge multiple outer product update items into the differential RRAM main state memory array when the SRAM low-rank residual buffer reaches a preset maximum depth or the memory where the differential RRAM main state memory array is located is in an idle state.

[0054] The renormalization unit is used to migrate the exponential factor between the row-level scaling label, the master state matrix, and the normalization update vector when the row-level scaling label reaches the saturation warning boundary.

[0055] In some embodiments, the parameter generation unit is an on-chip digital projection unit; the on-chip digital projection unit receives the hidden vector of the current word and outputs a selective decay vector, an update vector, a value vector, and a query vector; the normalized outer product generation unit and the scaling query generation unit adjust the exponent of the vector using a block floating-point data format.

[0056] The device in this embodiment can realize the above-mentioned selective state-space model in-store computation method based on scaling labels and low-rank residual buffers. The technical features of the specific method are the same as those in Embodiment 1, and can be referred to in Embodiment 1 for details, which will not be repeated here.

[0057] Example 3: This embodiment also provides a chip for executing the selective state-space model in-memory computing method based on scaling tags and low-rank residual buffers described in Embodiment 1. The chip includes the selective state-space model in-memory computing device based on scaling tags and low-rank residual buffers described in Embodiment 2, and further includes an on-chip interconnect network for transmitting data corresponding to the current term among the differential RRAM main state-memory array, the row-level scaling tag controller, the SRAM low-rank residual buffer, and the parameter generation unit.

[0058] In one embodiment, the chip includes an on-chip digital projection unit, multiple state-residence units, and an on-chip interconnect network. The multiple state-residence units correspond to state matrices of different layers and heads of the SSM. The on-chip digital projection unit receives the hidden vector x of the current term and outputs a selective decay vector d, an update vector k, a value vector v, and a query vector q. The on-chip interconnect network routes the selective decay vector d, update vector k, value vector v, and query vector q to the target state-residence unit, respectively. The row-level scaling tag controller, normalized outer product generation unit, scaling query generation unit, residual compensation calculation unit, and background merging unit within the target state-residence unit work collaboratively in the order described in Embodiment 2.

[0059] In this embodiment, the differential RRAM master state memory array is divided into multiple sub-array blocks according to row blocks and column blocks. Each sub-array block is responsible for a fixed sub-block of the master state matrix. Local matrix-vector multiplication is performed within the block, and the blocks are converged through an on-chip digital reduction network to obtain the complete master state output. The SRAM low-rank residual buffer and the corresponding differential RRAM master state memory array are physically paired to reduce the on-chip transport distance of the normalized update vector, value vector, and scaled query vector.

[0060] With the chip structure adopted in this embodiment, the main state matrix primarily undertakes matrix-vector multiplication tasks in the foreground path, selectively decaying and shifting to the row-level scaling label update path, while outer product updates are shifted to the SRAM low-rank residual buffer, residual compensation, and background batch merging paths. Therefore, this chip can reduce the write pressure on the differential RRAM main state memory array in the foreground path and improve overall inference throughput without changing the SSM recursive semantics.

[0061] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the processes of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0062] The above description is merely a preferred embodiment of the present invention. Those skilled in the art will understand that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the present invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A method for in-memory computation of selective state-space models based on scaling labels and low-rank residual buffers, characterized in that, The method, applied to an accelerator including a differential RRAM main state memory array, a row-level scaling tag controller, and an SRAM low-rank residual buffer, comprises: For the state matrix of the same layer and the same head of the selective state space model, the main part of the state matrix is ​​stored in the differential RRAM main state storage array, the cumulative decay factor corresponding to each row of the state matrix is ​​stored as row-level scaling label, and multiple outer product update terms that have not yet been merged into the differential RRAM main state storage array are stored in the SRAM low-rank residual buffer in the form of vector pairs. Generate a selective decay vector, an update vector, a value vector, and a query vector based on the current lexical unit, and update the row-level scaling label row by row based on the selective decay vector; The update vector is reverse scaled according to the updated row-level scaling label to obtain a normalized update vector. The normalized update vector and the value vector are written into the SRAM low-rank residual buffer as the current outer product update term. The query vector is forward scaled according to the row-level scaling label to obtain the scaled query vector; the differential RRAM master state memory array performs matrix-vector multiplication on the scaled query vector to obtain the master state output; The residual compensation output is calculated based on the scaled query vector and the multiple outer product update terms. The main state output is then added to the residual compensation output to obtain the current output.

2. The in-memory computation method for selective state-space models based on scaling labels and low-rank residual buffers as described in claim 1, characterized in that, After updating the row-level scaling label row by row according to the selective decay vector, no element-wise decay write-back is performed on the master state matrix stored in the differential RRAM master state storage array.

3. The in-memory computation method for selective state-space models based on scaling labels and low-rank residual buffers according to claim 1, characterized in that, After writing the normalized update vector and the value vector as the current outer product update term into the SRAM low-rank residual buffer, the outer product result of the normalized update vector and the value vector is not immediately written into the differential RRAM master state storage array.

4. The in-memory computation method for selective state-space models based on scaling labels and low-rank residual buffers as described in claim 1, characterized in that, The SRAM low-rank residual buffer includes a first buffer matrix for storing multiple normalized update vectors, a second buffer matrix for storing multiple value vectors, and a valid bit register; each valid row of the first buffer matrix is ​​used to store a normalized update vector, the corresponding valid row of the second buffer matrix is ​​used to store a value vector, and the low-rank residual matrix is ​​formed by accumulating the vector pairs in all valid buffer slots.

5. The in-memory computation method for selective state-space models based on scaling labels and low-rank residual buffers according to claim 1, characterized in that, When the integer part of any row-level scaling label reaches the saturation warning boundary of the integer range that the label format can represent, a renormalization operation is performed, the steps of which include: Take the integer part of the line-level scaling label as Update the row-level scaling label based on the integer part; synchronously multiply the corresponding row of the master state matrix and the corresponding components of each effective normalized update vector in the SRAM low-rank residual buffer by... .

6. The in-memory computation method for selective state-space models based on scaling labels and low-rank residual buffers according to claim 1, characterized in that, The differential RRAM master state memory array uses positive and negative differential conductance cells to represent the signed master state matrix. During background merging, the positive value part of the incremental matrix is ​​converted into a positive write pulse sequence, and the negative value part is converted into a negative write pulse sequence. After each write, a verification write is performed to correct the nonlinear write error of the RRAM device.

7. A method for in-memory computation of a selective state-space model based on scaling labels and low-rank residual buffers, as described in any one of claims 1-6, characterized in that, Also includes: When the SRAM low-rank residual buffer reaches a preset maximum depth or the storage bank containing the differential RRAM main state memory array is in an idle state, multiple outer product update terms cached in the SRAM low-rank residual buffer are batch merged into the differential RRAM main state memory array.

8. A selective state-space model in-memory computing device based on scaling labels and low-rank residual buffers, characterized in that, include: The parameter generation unit is used to generate selective decay vectors, update vectors, value vectors, and query vectors based on the current lexical units. Differential RRAM master state memory array is used to store the master state matrix and perform matrix-vector multiplication; A row-level scaling label controller is used to store and update the row-level scaling labels corresponding to each row of the main state matrix; A normalized outer product generation unit is used to perform inverse scaling on the update vector according to the row-level scaling label to obtain a normalized update vector; A low-rank SRAM residual buffer is used to cache multiple outer product update terms formed by the normalized update vector and the value vector; A scaling query generation unit is used to perform forward scaling on the query vector according to the row-level scaling label to obtain a scaled query vector. The residual compensation calculation unit is used to calculate the residual compensation output based on the scaled query vector and the multiple outer product update terms. The background merging unit is used to batch merge the multiple outer product update items into the differential RRAM main state memory array when the SRAM low-rank residual buffer reaches a preset maximum depth or the memory where the differential RRAM main state memory array is located is in an idle state. The renormalization unit is used to migrate the exponential factor between the row-level scaling label, the master state matrix, and the normalization update vector when the row-level scaling label reaches the saturation warning boundary.

9. The in-memory computing device for selective state-space models based on scaling labels and low-rank residual buffers according to claim 8, characterized in that, The parameter generation unit is an on-chip digital projection unit; the on-chip digital projection unit receives the hidden vector of the current word and outputs the selective decay vector, the update vector, the value vector and the query vector; the normalized outer product generation unit and the scaling query generation unit adjust the exponent of the vector using block floating-point data format.

10. A chip, characterized in that, The selective state-space model in-memory computing device based on scaling tags and low-rank residual buffers, as described in any one of claims 8-9, further includes an on-chip interconnect network for transmitting data corresponding to the current lexical unit among the differential RRAM main state-memory array, the row-level scaling tag controller, the SRAM low-rank residual buffer, and the parameter generation unit.

Images