In-memory architecture based on resistive random access memory and matrix decomposition acceleration algorithm

Abstract

The application discloses an in-memory architecture based on a resistive switching memory and a matrix decomposition acceleration algorithm, which is used for accelerating a Transformer neural network. Firstly, the application optimizes a self-attention calculation process, decomposes a weight matrix, and reduces calculation and write operation numbers; then, a softmax calculation array based on a selection and comparison logic structure of the resistive switching memory is used to further reduce overall power consumption. On the basis of the Re-Transformer, the application proposes an optimized matrix multiplication calculation, which uses matrix decomposition in scaled dot-product attention to further eliminate data dependency and reduce calculation delay. Meanwhile, the application uses a hybrid softmax based on the resistive switching memory to save power consumption.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence algorithms and hardware, and in particular to an in-memory architecture for Transformer acceleration based on resistive random access memory and matrix factorization acceleration algorithms. Background Technology

[0002] The Transformer has become a popular deep neural network (DNN) model in Neural Language Processing (NLP) applications, demonstrating excellent performance in areas such as neural machine translation and entity recognition. Transformer-based models, such as Generative Pre-trained Transformer (GPT), Vision Transformer (ViT), and Swin-Transformer, represent one of the most significant advancements in artificial intelligence. These models achieve higher accuracy than traditional convolutional neural networks (CNNs) and have challenged the dominance of CNNs in various AI tasks. Self-attention is a crucial mechanism in Transformer models, used to compute self-attention for each element in the input sequence, resulting in a self-attention representation for each element. This mechanism captures dependencies between elements in the sequence, enabling long-distance dependency modeling, multi-head attention mechanisms, and reduced computational complexity, making it one of the core elements of the Transformer model. Therefore, its computation is also one of the core mathematical operations of the Transformer model.

[0003] Unfortunately, the superior accuracy of Transformer-based models comes at the cost of more operations, which existing processors designed for CNNs cannot efficiently handle. While researchers have successfully applied process-in-memory (PIM) architectures based on resistive random access memory (ReRAM) to accelerate convolutional neural networks (CNNs) and recurrent neural networks (RNNs), the unique computational process of scaling dot product attention in the Transformer makes direct application of these designs difficult. Furthermore, the Transformer must perform numerous matrix-matrix multiplication (MatMul) operations, where both matrices are intermediate results from previous layers. Previous work required writing these intermediate results to the computing device, an operation that can pause the computation process and reduce speed and energy efficiency. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing an in-memory architecture based on resistive random access memory and matrix factorization acceleration algorithms. This invention utilizes the properties of symmetric matrices to transform matrix multiplication into vector multiplication and dot product by factoring the matrix. Furthermore, by introducing hardware architecture to handle the computation process, the self-attention computation flow is further optimized based on the Re-Transformer.

[0005] The objective of this invention is achieved through the following technical solution: an in-memory architecture based on resistive random access memory and matrix factorization acceleration algorithm, comprising:

[0006] Pre-training unit: The weight matrix W of the three network parameters of the Transformer neural network trained using the dataset. Q W K W V ;

[0007] Decomposition unit: The weight matrix W obtained from the pre-trained unit. Q and W K Perform matrix decomposition, the decomposition formula is W Q ·W K T =W qk =A+B, where W qk Let A be a real square matrix, B be a real symmetric matrix, and A = W. qk +W qk T / 2, B = W qk -W qk T / 2; Based on the properties of real symmetric matrices and antisymmetric matrices, the following decomposition is made: A=P·D·P T B = U·C·U T , where P is an orthogonal matrix, D is a diagonal matrix, U is an invertible matrix, and C is a block diagonal matrix;

[0008] Preprocessing unit: Processes the input matrix X and the matrices P, D, U, C obtained from the decomposition unit by row and column respectively, and inputs them into the calculation unit after misalignment;

[0009] The computing unit uses resistive random access memory (RRAM) as the computing element. It performs multiplication and addition calculations on the matrix output from the preprocessing unit to obtain the product matrix, with the formula Out = P'·D·P'. T +U'·C·U' T Where P' = X·P, U' = X·U, and Out is the attention score;

[0010] Cache unit: Temporarily store intermediate results calculated in the computation unit in the cache unit;

[0011] Self-attention mechanism unit: Utilizing a hybrid softmax computation array based on resistive random access memory (RRAM) in-memory logic, the product matrix Out obtained by the computation unit is normalized to obtain the data softmax[(P'·D·P')] T +U'·C·U' T ) / dk 0.5 )], where dk is the dimension of the key matrix K, and the obtained data is softmax[(P'·D·P' T +U'·C·U' T ) / dk 0.5 [ ] with input matrix X and weight matrix W V Matrix multiplication is performed to obtain the self-attention weight coefficients.

[0012] Furthermore, when performing self-attention calculations, the initial Transformer neural network randomly initializes three network parameter weight matrices W. Q W K W V The query matrix Q, key matrix K, and value matrix V are obtained by multiplying the input matrix X after adding location information.

[0013] Furthermore, the self-attention calculation formula for the Transformer neural network is softmax(Q·K). T / dk 0.5 )·V, where attention score Out=Q·K T Expand into Q·K T =(X·W Q )·(X·W K ) T =X·(W Q ·W K T )X T ;

[0014] Based on W Q ·W K T The product result W qk Given a square matrix, W is determined according to the properties of square matrices. qk It can be decomposed into the sum of a real symmetric matrix A and an antisymmetric matrix B, where A = W. qk +W qk T / 2, B = W qk -W qk T / 2;

[0015] Based on the properties of real symmetric and antisymmetric matrices, using the orthogonal decomposition of real symmetric matrices and the Schur decomposition of antisymmetric matrices, the decomposition result is obtained: A = P·D·P T B = U·C·U T And D and C are in the following forms: D = diag(λ1, λ2, ..., λ i ), λ i Let be the eigenvalues ​​of the real symmetric matrix A, i∈{1,2,...,m}, where m is the number of rows in the weight matrix;

[0016] Furthermore, after obtaining the orthogonal matrix P, the invertible matrix U, the diagonal matrix D, and the block diagonal matrix C, the attention score Out is expressed as:

[0017] Out = X·(W) Q ·W K T )·X T =X·A·X T +X·B·X T =(X·P)·D·(X·P) T +(X·U)·C·(X·U) T ;

[0018] Let P' = X·P, U' = X·U, the simplified formula is:

[0019] Out = P'·D·P' T +U'·C·U' T .

[0020] Furthermore, after obtaining the attention score, a hybrid softmax computation array based on resistive random access memory (RRAM) is used to obtain the data softmax[(P'·D·P']]. T +U'·C·U' T ) / dk 0.5 Then multiply by the input matrix X and the weight matrix W in succession. V The final output self-attention weight coefficient Z is obtained as follows:

[0021] Z = softmax[(P'·D·P')] T +U'·C·U' T ) / dk 0.5 )]·X·W V .

[0022] Furthermore, by utilizing the relevant properties of symmetric matrices, the complex matrix multiplication operation can be simplified to matrix multiplication of vectors and dot product between vectors.

[0023] Furthermore, resistive random access memory is utilized in the Transformer for matrix-vector and softmax operations.

[0024] The beneficial effects of this invention are as follows: This invention utilizes certain properties of matrices to further optimize the self-attention calculation process based on Re-Transformer, simplifying the original matrix-matrix operation to matrix-vector multiplication and dot products between vectors, and using in-memory computing arrays, such as resistive random access memory (RANM), to calculate intermediate results. Because this scheme optimizes the matrix multiplication calculation process and utilizes RANM for in-memory computation, it saves many write and storage requirements, thus further reducing power consumption, latency, and the space required to store intermediate results. Furthermore, sparse array design optimization is an effective optimization method that can improve computational efficiency while reducing storage space. Therefore, this invention utilizes the properties of matrices to decompose part of the original weight matrix, thereby reducing the sparsity of the weight matrix and improving the array utilization and accuracy of in-memory computation. Attached Figure Description

[0025] In accompanying drawings that are not necessarily drawn to scale, images can clearly and vividly illustrate relevant examples and solutions in place of some text. The accompanying drawings primarily serve as schematic diagrams to illustrate the specification and claims, rather than as precise and detailed references.

[0026] Figure 1 The computational steps for self-attention in a traditional Transformer are shown.

[0027] Figure 2 The flowchart of the computation process for traditional Transformer self-attention is shown.

[0028] Figure 3 The computation steps for Re-Transformer self-attention for ReRAM in-memory computation are shown.

[0029] Figure 4 The flowchart of the Re-Transformer self-attention calculation is shown.

[0030] Figure 5 This illustrates the core idea of ​​the present invention, Eigen-Transformer, which simplifies Self-Attention computation using matrix factorization.

[0031] Figure 6 The calculation method and steps for Eigen-Transformer self-attention are shown.

[0032] Figure 7 An in-memory acceleration method for matrix-vector multiplication based on resistive random access memory is shown.

[0033] Figure 8 The flowchart of the Eigen-Transformer self-attention computation is shown.

[0034] Figure 9 This paper illustrates three methods for calculating Transformer self-attention involved in this invention (including a comparison of the required cache space). Detailed Implementation

[0035] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and specific examples, but this is not intended to limit the present invention. If there is no necessary sequential relationship between the various steps described herein, the order in which they are described as examples should not be considered a limitation. Those skilled in the art should know that the order can be adjusted, as long as it does not disrupt the algorithmic logic between them and render the entire process impossible.

[0036] Explanation of relevant technical terms:

[0037] Transformer neural network:

[0038] From a macroscopic perspective, the Transformer is a seq2seq model based on an attention mechanism. The encoder is responsible for processing the input sequence (x1, ..., x...). a Mapped to a continuous sequence z = (z1, ..., z2) a For a given z, the decoder generates an output sequence (y1, ..., y) in one pass. b In each step, the model is autoregressive. The Transformer follows this overall architecture, unlike traditional Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). The entire network is composed entirely of a self-attention mechanism and a feed-forward neural network. Generally, a Transformer neural network consists of two parts: an encoder and a decoder.

[0039] The Encoder consists of N identical encoder blocks stacked together, and the Decoder also consists of N identical decoder blocks stacked together. The Encoder's role is to calculate the correlations between the input sequences. Each Encoder block contains a Multi-Head Attention layer to calculate the global scope information of the input data. Because Self-Attention employs a parallel computing strategy, processing all input sequence data simultaneously, the model's computational efficiency is very high. Stacking multiple Encoder layers can better uncover the potential relationships between the sequences. The Decoder's role is to combine the global correlations between previous sequences and output the sequence to be predicted.

[0040] Self-attention mechanism:

[0041] Traditional attention mechanisms operate between elements of the target set (Target) and the source set (Source). Simply put, the calculation of weights in an attention mechanism requires the participation of the Target. That is, in an Encoder-Decoder model, the calculation of attention weights requires the hidden states of both the Encoder and Decoder. In contrast, the Self-Attention mechanism calculates the weight relationships between each vector in the input sequence and all other vectors in parallel, representing a global relationship. Therefore, only the matrix operations at the Source are needed, without requiring information from the Target. The computation steps of a traditional self-attention mechanism are as follows: Figure 1 As shown.

[0042] When the initial Transformer neural network performs self-attention calculation, it randomly initializes three network parameter weight matrices W. Q W K W V The query matrix Q, key matrix K, and value matrix V are obtained by multiplying the input matrix X after adding location information, which assists in the calculation of self-attention.

[0043] Q = X·W Q

[0044] K = X·W K

[0045] V = X·W V

[0046] W Q W K W V∈R mxn

[0047] Out = Q·K T

[0048] Where m and n are the number of rows and columns of the weight matrix, respectively; Out is the attention score; and the superscript T indicates the transpose of the matrix.

[0049] The self-attention weight coefficient Z is calculated using three matrices: Q, K, and V. The calculation formula is as follows:

[0050] Z=Attention(Q,K,V)=softmax(Q·K T / dk 0.5 )·V

[0051] Here, dk represents the dimension of the key-value matrix K, and Attention is the self-attention function. To prevent the Q-K inner product from becoming too large, which could cause the self-attention weight coefficients to become biased to extremes, it is divided by dk during calculation. 0.5 It acts as a buffer. The Attention function is used to calculate the degree of attention of a single data point in the input sequence relative to all data points. The higher the correlation between two elements, the greater the corresponding attention weight. The order of the input vectors does not matter during the entire calculation process; parallel computation is achieved through matrix representation. Figure 2 The relevant calculation process is demonstrated.

[0052] In their paper "Re-Transformer: ReRAM-based processing-in-memory architecture for transformer acceleration," Yang et al. proposed a ReRAM-based PIM architecture for accelerating Transformers. The core ideas of this architecture mainly include:

[0053] In matrix linear transformation, the weight matrix W K Change to W K T Everything else remains the same, and Q is calculated simultaneously; compared to the initial Transformer, this step only calculates Q;

[0054] When calculating attention scores, the formula (Q·W) is used. K T )·X T This step requires an additional cache for X. T ;

[0055] Each attention score undergoes a hybrid softmax operation, which applies ReRAM selection and comparison logic;

[0056] The softmax operation result is then right-multiplied successively by the input matrix X and the weight matrix W. V The self-attention weight coefficient Z is obtained. The overall calculation steps are as follows: Figure 3 As shown.

[0057] The specific calculation process is as follows: Figure 4 As shown, it mainly includes:

[0058] Q = X·W Q

[0059] R = Q·W K T

[0060] Out = R·X T

[0061] Z = [softmax(R·X)] T / dk 0.5 )·X]·W V

[0062] Where R is the intermediate matrix.

[0063] This invention provides an in-memory architecture based on resistive random access memory and matrix factorization acceleration algorithm for accelerating Transformer neural networks, denoted as Eigen-Transformer. The main innovations are in the self-attention acceleration algorithm and hardware architecture.

[0064] Regarding the self-attention acceleration algorithm, based on Re-Transformer, further optimizations were made to the self-attention calculation of the initial Transformer by using matrix factorization and some properties of symmetric matrices. The core idea is as follows: Figure 5 As shown, the main sources are:

[0065] Any real square matrix can be decomposed into the sum of a real symmetric matrix and an antisymmetric matrix;

[0066] Any real symmetric matrix is ​​orthogonally similar to a diagonal matrix, and the elements on the main diagonal of the diagonal matrix are all the eigenvalues ​​of the real symmetric matrix.

[0067] Any antisymmetric matrix can be transformed into a block diagonal matrix through Schur decomposition;

[0068] Specifically:

[0069] The self-attention calculation of the traditional Transformer is softmax(Q·K). T / dk 0.5 )·V, where Q·K T Expanded to:

[0070] Out = Q·K T =(X·W Q )·(X·W K ) T =X·(W Q ·W K T )X T

[0071] W Q ·W K T =W qk =A+B

[0072] Among them, W qk Let A be a real square matrix, B be a real symmetric matrix, and A = W. qk +W qk T / 2, B = W qk -W qk T / 2;

[0073] Based on the properties of real symmetric matrices and antisymmetric matrices, they are decomposed as follows:

[0074] A = P·D·P T B = U·C·U T

[0075] Where P is an orthogonal matrix, D is a diagonal matrix, U is an invertible matrix, and C is a block diagonal matrix, and D and C can be written in the following forms:

[0076] D = diag(λ1, λ2, ..., λ i ), λ i Let A be the eigenvalues ​​of a real symmetric matrix A, i∈{1,2,...,m};

[0077]

[0078] Based on the decomposition results, the original weights W in the self-attention calculation of the initial Transformer are... Q W K If we replace the matrix with an orthogonal matrix P, an invertible matrix U, a diagonal matrix D, and a block diagonal matrix C, then the calculation steps become as follows: Figure 6 As shown, specifically:

[0079] Out = X·(W) Q ·W K T )·X T =X·A·X T +X·B·XT =(X·P)·D·(X·P) T +(X·U)·C·(X·U) T

[0080] Let P' = X·P, U' = X·U. This calculation step conforms to the general in-memory computation format—matrix multiplication by an input vector. Therefore, this invention attempts to calculate matrix-matrix multiplication and matrix-vector multiplication using an in-memory computation array represented by resistive random access memory (RRAM). Essentially, it uses physical quantities (e.g., voltage, current) to encode input, weights, and output data, and constructs circuits to ensure that the input data, weights, and output data satisfy the mathematical relationship of matrix-vector multiplication, thus performing such calculations in a highly parallel manner in the analog domain. Related computational paradigms are as follows: Figure 7 As shown.

[0081] Furthermore, the self-attention calculation is simplified to:

[0082] Out = P'·D·P' T +U'·C·U' T

[0083] Furthermore, using the hybrid softmax operation based on ReRAM in-memory logic proposed in the Re-Transformer work, the following operation results were obtained:

[0084] Softmax[(P'·D·P' T +U'·C·U' T ) / dk 0.5 )]

[0085] D = diag(λ1, λ2, ... λ) i ), λ i Let A be the eigenvalues ​​of a real symmetric matrix A, i∈{1,2,...,m};

[0086] Furthermore, the result is successively multiplied by the input matrix X and the weight matrix W. V The final output self-attention weight coefficient Z can then be obtained, specifically expressed as:

[0087] Z = softmax[(P'·D·P')] T +U'·C·U' T ) / dk 0.5 )]·X·W V

[0088] Based on the above process, draw the corresponding flowchart as follows: Figure 8 As shown.

[0089] In this invention, self-attention calculation mainly focuses on P', U', P'·D·P' Tand U'·C·U' T superior.

[0090] With P'·D·P' T For example, due to the properties of the diagonal matrix D, the result of the calculation in the i-th row and j-th column is O. ij Just use P' i and P' j Perform the inner product, P' i and P' j Let i and j be the row vectors of matrix P', i ≠ j; then multiply by the diagonal elements of the diagonal matrix D to obtain the output; furthermore, due to the properties of symmetric matrices, we only need to perform upper or lower triangular operations to obtain the entire output matrix of this calculation.

[0091] In the same way, U'·C·U' T To calculate the result, we utilize the properties of antisymmetric matrices and block diagonal matrices, calculating only the upper or lower triangular elements to obtain the overall result, thus significantly reducing the number of calculations. Simultaneously, the entire process only requires caching matrices P' and U', as well as the diagonal elements of diagonal matrices D and C, further reducing cache space.

[0092] In terms of hardware architecture, this invention attempts to utilize Resistive Random Access Memory (ReRAM) in the Transformer. Based on optimizations made to the self-attention acceleration algorithm, corresponding hardware measures are proposed to improve computational acceleration, which mainly include:

[0093] Based on the self-attention acceleration algorithm optimization made by this invention, its calculation process is mainly transformed from the original matrix multiplication operation to matrix multiplication of vector and dot product between vectors. Therefore, this invention attempts to use the in-memory logic based on ReRAM to calculate the relevant process, store the elements in the matrix and vector in ReRAM, and use the calculation logic of ReRAM to perform the operation, which can greatly reduce the calculation power consumption and cache space.

[0094] Secondly, in the softmax operation, the hybrid softmax proposed based on the selection and comparison logic of ReRAM is also used to transform the original softmax operation. The operation process is simplified by using the selection and comparison logic structure of ReRAM. The relevant measures are described in detail in the Re-Transformer paper.

[0095] Based on the above analysis, the in-memory architecture based on resistive random access memory and matrix factorization acceleration algorithm provided by the present invention includes the following units:

[0096] Pre-training unit: The weight matrix W of the three network parameters of the Transformer neural network trained using the dataset.Q W K W V ;

[0097] Decomposition unit: The weight matrix W obtained from the pre-trained unit. Q and W K Perform matrix decomposition, the decomposition formula is W Q ·W K T =W qk =A+B, where W qk Let A be a real square matrix, B be a real symmetric matrix, and A = W. qk +W qk T / 2, B = W qk -W qk T / 2; Based on the properties of real symmetric matrices and antisymmetric matrices, the following decomposition is made: A=P·D·P T B = U·C·U T , where P is an orthogonal matrix, D is a diagonal matrix, U is an invertible matrix, and C is a block diagonal matrix;

[0098] Preprocessing unit: Processes the input matrix X and the matrices P, D, U, C obtained from the decomposition unit by row and column respectively, and inputs them into the calculation unit after being staggered.

[0099] The computing unit uses resistive random access memory (RRAM) as the computing element. It performs multiplication and addition calculations on the matrix output from the preprocessing unit to obtain the product matrix, with the formula Out = P'·D·P'. T +U'·C·U' T Where P' = X·P, U' = X·U, and Out is the attention score;

[0100] Cache unit: Temporarily store intermediate results of calculations in the calculation unit in the cache unit for easy use in subsequent calculations;

[0101] Self-attention mechanism unit: Utilizing a hybrid softmax computation array based on resistive random access memory (RRAM) in-memory logic, the product matrix Out obtained by the computation unit is normalized to obtain the data softmax[(P'·D·P')] T +U'·C·U' T ) / dk 0.5 )], where dk is the dimension of the key matrix K, and the obtained data is softmax[(P'·D·P' T +U'·C·U' T ) / dk 0.5 [ ] with input matrix X and weight matrix W VPerform matrix multiplication to obtain the self-attention weight coefficients, and output the self-attention weight coefficients through the data output module.

[0102] A comparison is made between the calculation methods and initial cache space of the initial Transformer, Re-Transformer, and the Eigen-Transformer of this invention, and the corresponding results are as follows: Figure 9 As shown.

[0103] The softmax operation in this invention adopts the hybrid softmax proposed in Re-Transformer - softmax based on ReRAM in-memory logic, and therefore will not be described in detail in this invention.

[0104] This invention aims to optimize the self-attention computation process to reduce computation and write operations. Then, it utilizes a softmax operation based on a ReRAM-based selection and comparison logic structure to further reduce overall power consumption.

[0105] Furthermore, the general flow and concept of the invention have been described herein, and its scope includes any and all algorithms and architectures upon which the invention is based. The operational methods described in the claims will be specifically interpreted based on the language used in the claims.

[0106] The above description is intended to be illustrative. Those skilled in the art can conduct preliminary theoretical investigations and use auxiliary tools to perform preliminary simulations upon reading this description. The main core idea of ​​this invention is to optimize self-attention calculation by utilizing matrix properties and to attempt to calculate the transition matrix used in this scheme using ReRAM in-memory logic. Therefore, those skilled in the art can conduct further investigations and optimizations.

Claims

1. An in-memory architecture based on resistive random access memory and matrix factorization acceleration algorithm, characterized in that, include: Pre-training unit: The weight matrix W of the three network parameters of the Transformer neural network trained using the dataset. Q W K W V ; Decomposition unit: The weight matrix W obtained from the pre-trained unit. Q and W K Perform matrix decomposition, the decomposition formula is W Q ·W K T =W qk =A+B, where W qk Let A be a real square matrix, B be a real symmetric matrix, and A = W. qk +W qk T / 2, B = W qk -W qk T / 2; Based on the properties of real symmetric matrices and antisymmetric matrices, the following decomposition is made: A=P·D·P T B = U·C·U T , where P is an orthogonal matrix, D is a diagonal matrix, U is an invertible matrix, and C is a block diagonal matrix; Preprocessing unit: Processes the input matrix X and the matrices P, D, U, C obtained from the decomposition unit by row and column respectively, and inputs them into the calculation unit after being staggered. The computing unit uses resistive random access memory (RRAM) as the computing element. It performs multiplication and addition calculations on the matrix output from the preprocessing unit to obtain the product matrix, with the formula Out = P'·D·P'. T +U'·C·U' T Where P' = X·P, U' = X·U, and Out is the attention score; Cache unit: Temporarily store intermediate results calculated in the computation unit in the cache unit; Self-attention mechanism unit: Utilizing a hybrid softmax computation array based on resistive random access memory (RRAM) in-memory logic, the product matrix Out obtained by the computation unit is normalized to obtain the data softmax[(P'·D·P')] T +U'·C·U' T ) / dk 0.5 )], where dk is the dimension of the key matrix K, and the obtained data is softmax[(P'·D·P' T +U'·C·U' T ) / dk 0.5 [ ] with input matrix X and weight matrix W V Matrix multiplication is performed to obtain the self-attention weight coefficients.

2. The in-memory architecture based on resistive random access memory and matrix factorization acceleration algorithm according to claim 1, characterized in that, When performing self-attention calculations in the initial Transformer neural network, three network parameter weight matrices W are randomly initialized. Q W K W V The query matrix Q, key matrix K, and value matrix V are obtained by multiplying the input matrix X after adding location information.

3. The in-memory architecture based on resistive random access memory and matrix factorization acceleration algorithm according to claim 2, characterized in that, The formula for calculating the self-attention of a Transformer neural network is softmax(Q·K). T / dk 0.5 )·V, where attention score Out=Q·K T Expand into Q·K T =(X·W Q )·(X·W K ) T =X·(W Q ·W K T )X T ; Based on W Q ·W K T The product result W qk Given a square matrix, W is determined according to the properties of square matrices. qk It can be decomposed into the sum of a real symmetric matrix A and an antisymmetric matrix B, where A = W. qk +W qk T / 2, B = W qk -W qk T / 2; Based on the properties of real symmetric and antisymmetric matrices, using the orthogonal decomposition of real symmetric matrices and the Schur decomposition of antisymmetric matrices, the decomposition result is obtained: A = P·D·P T B = U·C·U T And D and C are in the following forms: D = diag(λ1, λ2, ..., λ i ), λ i Let be the eigenvalues ​​of the real symmetric matrix A, i∈{1,2,...,m}, where m is the number of rows in the weight matrix; 4. The in-memory architecture based on resistive random access memory and matrix factorization acceleration algorithm according to claim 1, characterized in that, After obtaining the orthogonal matrix P, the invertible matrix U, the diagonal matrix D, and the block diagonal matrix C, the attention score Out is expressed as: Out=X·(W Q ·W K T )·X T =X·A·X T +X·B·X T =(X·P)·D·(X·P) T +(X·U)·C·(X·U) T ; Let P' = X·P, U' = X·U, the simplified formula is: Out=P’·D·P’ T +U’·C·U’ T 。 5. The in-memory architecture based on resistive random access memory and matrix factorization acceleration algorithm according to claim 1, characterized in that, After obtaining the attention score, a hybrid softmax computation array based on resistive random access memory (RRAM) is used to obtain the data softmax[(P'·D·P'). T +U'·C·U' T ) / dk 0.5 Then multiply by the input matrix X and the weight matrix W in succession. V The final output self-attention weight coefficient Z is obtained as follows: Z=softmax[(P'·D·P' T +U·C·U T ) / min 0.5 )]·X·W V 。 6. The in-memory architecture based on resistive random access memory and matrix factorization acceleration algorithm according to claim 1, characterized in that, By utilizing the properties of symmetric matrices, complex matrix multiplication operations can be simplified to matrix multiplication of vectors and dot products between vectors.

7. The in-memory architecture based on resistive random access memory and matrix factorization acceleration algorithm according to claim 1, characterized in that, Resistive random access memory is used in Transformer for matrix-vector and softmax operations.

Images