Privacy protection LSTM reasoning method based on fully homomorphic encryption

Abstract

The invention discloses a privacy protection LSTM (Long Short Term Memory) reasoning method based on fully homomorphic encryption, which comprises the following steps that: a client encrypts data to be reasoned and uploads the data to the client, and a server performs LSTM sequence reasoning on ciphertext input of the client by adopting fully homomorphic encryption; firstly, linear calculation of the LSTM is realized by adopting homomorphic matrix multiplication; secondly, high-precision fitting of a nonlinear function is achieved through an optimization method; and then, the calculation in the LSTM Cell is aggregated, so that the times of homomorphic calculation under the ciphertext are reduced, and the calculation efficiency of the LSTM Cell under the ciphertext is improved.

Description

Technical Field

[0001] The present invention relates to a privacy - protected LSTM inference method based on fully homomorphic encryption, belonging to the field of artificial intelligence security. Background Art

[0002] With the advent of the era of artificial intelligence (AI), machine learning as a service (MLaaS) products, such as ChatGPT and Claude, have become household names and integrated into our daily lives. However, while these products provide convenience, they also bring serious privacy problems. Users need to upload personal data to query prediction results from third parties. Despite rules such as the Health Insurance Portability and Accountability Act (HIPAA), the General Data Protection Regulation (GDPR), and the Data Security Law, considering the powerful artificial intelligence technology, users are more concerned about their privacy than ever. Therefore, there is an urgent need to develop privacy - protected machine learning (PPML) methods to protect users' private data.

[0003] Recurrent neural networks (RNNs) have the ability to capture sequence dependencies and are of great significance in sequence learning tasks. Architectures such as RWKV based on RNNs have been used to reshape transformers, which are key components of MLaaS products such as ChatGPT and Gemini. The LSTM network is a typical RNN that can extract long - term dependencies in sequences and balance the influence of long - term and short - term on the LSTM model. It is widely used in applications such as text processing, activity recognition, and state detection. To protect users' private input sequence data, we designed a new privacy - protected LSTM inference scheme.

[0004] PPML has been studied for nearly a decade. Readers can refer to survey papers for more details. Encryption is usually the first choice for protecting data. However, encrypting complex computational processes forces the adoption of the idea of secure multi - party computation (SMC) to implement PPML, which requires online interaction among parties. Some SMC methods are not applicable to non - cooperative scenarios of resource - constrained participants. In addition, for MLaaS products like ChatGPT, users have no willingness to participate in interactive computations. Therefore, the design goal of the present invention is non - interactive and user - friendly, where users only need to upload data sequences and download prediction results. In addition, users may be unhappy with the large latency and reduced accuracy after embedding privacy functions. Therefore, the present invention should have a design that protects privacy and has low latency.

[0005] To achieve the above goals, in this paper, we use a fully homomorphic encryption (FHE) scheme to implement CryptoLSTM. However, there are challenges in implementing a non-interactive and low-latency privacy-preserving inference structure using this technology. First, due to the long process of the input sequence, the continuous proliferation of LSTM cells may generate intractable noise, reducing the prediction accuracy. Second, the LSTM unit contains non-linear activation functions (such as Sigmoid and tanh), which are not supported by FHE homomorphic operations. Although the activation function can be approximated by polynomials, it will increase the number of multiplications and cause noise accumulation. Third, bootstrapping is usually used to refresh the ciphertext and reduce noise, but it brings a high computational overhead. Summary of the Invention

[0006] Design a privacy-preserving LSTM inference method based on fully homomorphic encryption to overcome the deficiencies of the prior art.

[0007] The technical solution of the present invention is as follows: Provide a privacy-preserving LSTM inference method based on fully homomorphic encryption, and the method includes the following steps:

[0008] The user generates a private key and an evaluation key, divides the time-series data to be inferred into multiple vectors in chronological order using fully homomorphic encryption, and encrypts each vector to generate ciphertext;

[0009] The user uploads the ciphertext and the evaluation key to the server to directly perform LSTM neural network inference.

[0010] Further, the calculation process of the LSTM Cell in the LSTM neural network inference where the user uploads the ciphertext and the evaluation key to the server directly includes the following steps:

[0011] S1. Integrate the short-term state and the input into a single ciphertext [C XH using homomorphic rotation and homomorphic addition, perform homomorphic matrix multiplication on [C XH and the merged weight matrix W using homomorphic multiplication, and add the bias to obtain [C figo ;

[0012] S2. Divide [C figo into [C fio and [C g through the homomorphic data shunting algorithm, and perform homomorphic Sigmoid function calculation and homomorphic Tanh function calculation on [C fio and [C g respectively to obtain [C′ fio and [g];

[0013] S3. Use the data shunting algorithm to divide from the ciphertext [C′ fioDivide [f], [i], and [o] from it, and use homomorphic addition and homomorphic multiplication to update the long-term state [S] at time step t t to obtain the long-term state [S] at time step t+1 t+1 ;

[0014] S4. Operate on the long-term state at time step t using the homomorphic Tanh function, and then perform homomorphic multiplication with [o] to obtain the short-term state [H] at time step t+1 t+1 .

[0015] Furthermore, the homomorphic matrix multiplication is specifically:

[0016]

[0017] where

[0018]

[0019] [X] k = HomRot([X], k, EVK)

[0020]

[0021] HomMatMul represents the matrix multiplication operation under homomorphic encryption, EVK represents the evaluation key, [X] represents the ciphertext uploaded by the user, W ∈ R n×n , [X] ∈ R n ; W represents the weight matrix, [Y] represents the result obtained by performing matrix multiplication on [X], HomAdd represents the homomorphic addition operation, and HomMul represents the homomorphic multiplication operation; represents rotating the elements of the vector by -j·n 2 positions; represents the vector composed of the elements on the j·n 2 + k diagonal lines of the matrix W; HomRot represents the vector rotation operation under homomorphic encryption.

[0022] Furthermore, the homomorphic Sigmoid function and the homomorphic Tanh function use polynomial approximation for non-linear operations.

[0023] Furthermore, the method of using polynomial approximation for non-linear operations is:

[0024] Obtain an optimal polynomial by solving the following optimization problem:

[0025]

[0026] Solve the optimization problem through the Remez algorithm to obtain an optimal polynomial P(x), where, Denote the set of polynomials with degree not exceeding \(k\), \(x\) is the independent variable of \(P(x)\), \([a, b]\) represents the optimization interval, and \(f(x)\) represents the target activation function.

[0027] Furthermore, the method of integrating the short-term state and the input into a single ciphertext \([C XH by using homomorphic rotation and homomorphic addition, and performing homomorphic matrix multiplication on \([C XH and the combined weight matrix \(W\) and adding the bias to obtain \([C figo specifically includes:

[0028] During the process of performing matrix multiplication on the short-term state \(H t and the input \(X t , merge the weight matrices of the two into \(W\), that is:

[0029]

[0030] Perform homomorphic rotation on \(H t and perform homomorphic addition operation with \(X t , where \(\alpha\) and \(\beta\) are the input and output dimensions of the LSTM layer, and \(j\) and \(k\) respectively represent the row index and column index of the weight matrix,

[0031] [C XH = HomAdd([X t , HomRot([H t , -\(\alpha\))

[0032] Perform homomorphic multiplication operation on \([C XH and \(W\) and use homomorphic addition to add the bias to obtain \([C figo ,

[0033] [C figo = HomAdd(HomMatMul([C XH , \(W\), EVK), bais, EVK)

[0034] where, bais represents the bias, HomAdd represents homomorphic addition, HomRot represents homomorphic rotation, HomMatMul represents homomorphic multiplication, and EVK represents the evaluation key.

[0035] Furthermore, the method of using the data splitting algorithm to divide \([C figo into \([C fio and \([C g is:

[0036] Construct a vector index \(P 1 ,

[0037]

[0038] Multiply homomorphically [C figo with index P 1 to obtain [C fio .

[0039] [C fio = HomMul([C figo , P 1 , EVK)

[0040] Construct an index P 2 ,

[0041]

[0042] Multiply [C figo with index P 2 to obtain the ciphertext [V 2 of vector V 2 , and then homomorphically rotate it to obtain the ciphertext [C g of vector V g .

[0043] [V 2 = HomMul([C figo , P 2 , EVK)

[0044] [C g = HomRot([V 2 , -2β, EVK).

[0046] Furthermore, the method for performing homomorphic Sigmoid function calculation on [C fio to obtain [C′ fio is as follows:

[0047] Perform homomorphic activation operation on [C fio .

[0048] [C′ fio = HomSigmoid([C fio , EVK)

[0049] HomSigmoid represents the homomorphic Sigmoid function, and EVK represents the evaluation key.

[0050] Furthermore, the method for partitioning [f], [i], and [o] from [C′ fio is as follows:

[0051] Construct an index P 3 ,

[0052]

[0053] Multiply with index P through [C′ fio to obtain vector [f], 3

[0054] [f] = HomMul([C′ fio , P 3 , EVK)

[0055] Construct an index P at the corresponding position 4 ,

[0056]

[0057] Multiply with index P through [C′ fio to obtain the ciphertext [V 4 of vector V 4 , and then perform homomorphic rotation on it to obtain ciphertext [i], 4

[0058] [V 4 = HomMul([C′ fio , P 4 , EVK)

[0059] [I] = HomRot([V 4 , -β, EVK)

[0060] Construct an index P at the corresponding position 5 ,

[0061]

[0062] Multiply with index P through [C′ fio to obtain the ciphertext [V 5 of vector V 5 , and then perform homomorphic rotation on it to obtain ciphertext [o], 5

[0063] [V 5 = HomMul([C′ fio , P 5 , EVK)

[0064] [o] = HomRot([V 5 , -3β, EVK).

[0066] The beneficial effects of the present invention are: Compared with the prior art,

[0067] ​​​1) In the homomorphic encryption environment, all operations of the present invention are performed on ciphertext, ensuring the effective protection of data privacy. Homomorphic encryption allows calculations to be performed on encrypted data, and the calculation results remain in an encrypted state. Only those with the decryption key can decrypt to obtain the final result. This means that even when calculations are performed in an untrusted environment, the privacy of the data can be guaranteed, realizing non-interactive and secure LSTM network inference;

[0068] 2) The present invention proposes a matrix multiplication in LSTM for the homomorphic encryption environment. By combining multiple operations into a single homomorphic encryption operation, the calculation steps are reduced, and the processing efficiency is improved, realizing the linear calculation process in LSTM inference in homomorphic matrix multiplication;

[0069] 3) For the matrix multiplication in LSTM, in the homomorphic encryption environment, data can be calculated without decryption, ensuring data privacy and at the same time realizing the matrix multiplication in LSTM inference;

[0070] 4) The present invention performs non-linear polynomial approximation through an optimization method. Experiments show that this method has better approximation accuracy compared to Taylor expansion and least mean square error. The linear transformation method of the present invention can make full use of the mapping space, adopt an optimized method to linearize the activation function, and integrate the same operation in the LSTM cell, which can reduce noise accumulation while improving the prediction accuracy and at the same time reducing the calculation overhead;

[0071] 5) The present invention optimizes the calculation process of the LSTM Cell under ciphertext. By aggregating matrix multiplication and aggregation activation function, the number of homomorphic calculations is reduced, and the calculation efficiency of the LSTM Cell under ciphertext is improved;

[0072] 6) The present invention conducts experimental analysis on non-linear polynomial approximation and aggregation optimization. Experiments prove that polynomial approximation can achieve extremely high approximation accuracy, and aggregation optimization can bring about an improvement in calculation efficiency. At the same time, the present invention conducts experimental analysis on 4 data sets, and the experiments prove that the present scheme has extremely high calculation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 is the execution framework diagram of the present invention;

[0074] Figure 2 is the effect diagram of polynomial approximation of the present invention;

[0075] Figure 3 is the calculation diagram before aggregation optimization in the Cell under ciphertext of the present invention;

[0076] Figure 4 is the calculation diagram after aggregation optimization in the Cell under ciphertext of the present invention;

[0077] Figure 5 Shunt calculation example diagram under ciphertext for the present invention;

[0078] Figure 6 Cell execution flowchart under ciphertext for the present invention;

[0079] Figure 7 Flowchart of the present invention. Detailed implementation manners

[0080] To make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the drawings of this specification.

[0081] Reference Figures 1-7 , a privacy-preserving LSTM inference method based on fully homomorphic encryption, the method comprising: a user generates a private key and an evaluation key, divides the time series data to be inferred into multiple vectors in chronological order by using fully homomorphic encryption, and encrypts each vector to generate ciphertext;

[0082] The user uploads the ciphertext and the evaluation key to the server to directly perform LSTM neural network inference.

[0083] The service-side data user uploads data for privacy-preserving LSTM inference. The server needs to reconstruct the above calculation process on the ciphertext to implement homomorphic LSTM CELL calculation (HomCell), that is:

[0084]

[0085] Among them, the calculation process of Cell is:

[0086] f = σ(W xf X t + W hf H t + b f )

[0087] i = σ(W xi X t + W hi H t + b i )

[0088] g = φ(W xg X t + W hg H t + b g )

[0089] o = σ(W xo X t + W ho H t + b o )

[0090] S t+1 = f·S t + g·i

[0091] H t+1 = φ(S t+1 )·o,

[0092] It is necessary to construct homomorphic MatMul and activation function calculations.

[0093] The service party performs matrix multiplication on the user-uploaded data (Y = WX, where W ∈ R n×n , X ∈ R n ), that is:

[0094]

[0095] Therefore, the constructed homomorphic MatMul is:

[0096]

[0097] Among them,

[0098]

[0099] [X] k = HomRot([X], k, EVK),

[0100]

[0101] HomMatMul represents the matrix multiplication operation under homomorphic encryption, EVK represents the evaluation key, [X] represents the ciphertext uploaded by the user, W ∈ R n×n , [X] ∈ R n ; W represents the weight matrix, [Y] represents the ciphertext result obtained after performing HomMatMul on [X], and HomAdd represents the matrix addition operation under homomorphic encryption; represents rotating the elements of the vector by -j·n 2 positions; represents the vector composed of the elements on the j·n 2 + k-th diagonal of the matrix W; HomRot represents the vector rotation operation under homomorphic encryption.

[0102] The polynomial approximation method of the non-linear function described above. By solving the following optimization problem:

[0103]

[0104] To obtain an optimal polynomial and approximate the calculation of a non - linear activation function through homomorphic polynomial calculation. The present invention uses the Remez algorithm to solve the optimization problem and obtain an optimal polynomial P(x), where, represents the set of polynomials with degree not exceeding k, x is the independent variable of P(x), [a, b] represents the optimization interval, and f(x) represents the target activation function. Taking Sigmoid as an example, the homomorphic Sigmoid calculation can be expressed as:

[0105]

[0106] LSTM Cell aggregation optimization method. Utilizing the idea of aggregation calculation, by putting multiple input data or output data into a single ciphertext and implementing multiple identical operation calculations through a single ciphertext homomorphic calculation, the calculation efficiency of the LSTM Cell under ciphertext is improved. Among them, {c 0 , c 1 , …, c n-1} is the set composed of the coefficients of the optimal polynomial P(x), and Tanh uses the same method for homomorphic calculation, Figure 2 shows the approximation of the Sigmoid function.

[0107] For the similar structures in the LSTM Cell, the present invention conducts aggregation optimization. By putting multiple input data or output data into a single ciphertext and implementing multiple identical operation calculations through a single ciphertext homomorphic calculation, the calculation efficiency of the LSTM Cell under ciphertext is improved.

[0108] During the process of matrix multiplication of the short - term state H t and the input X t , the weight matrices of the two can be combined into W f , that is:

[0109]

[0110] where α and β are the input - output dimensions of the LSTM layer, and the matrix multiplication calculation of H t and X t is achieved through a single HomMatMul calculation, that is:

[0111] [C XH = HomAdd([X t , HomRot([H t , -α, EVK), EVK)

[0112] [C f = HomMatMul([C XH , W f , EVK)

[0113] Reduce the number of ciphertext calculations and speed up the calculation of the Cell. Further, this method can be extended to all matrix multiplications in the Cell, i.e.:

[0114]

[0115] All matrix multiplication calculations of H t and X t in a single Cell are realized through a single HomMatMul calculation, i.e.:

[0116] [C XH = HomAdd([X t , HomRot([H t , -α, EVK), EVK)

[0117] [C figo = HomMatMul([C XH , W, EVK).

[0118] Meanwhile, during the activation function calculation stage, similar calculations are also found. The present invention reduces the repeated activation function calculation process by adopting an aggregation method, i.e.:

[0119] [C′ fio = HomSigmoid([C fio , EVK)

[0120] where [C fio represents the ciphertext form containing the vectors W xf X t + W hf H t + b f , W xi X t + W hi H t + b i and W xo X t + W ho H t + b o , [C′ fio represents the ciphertext form of the vector C′ fio containing f, i, and o. Since different operations need to be performed on f, i, g, and o to update the long - short - term states S t and H t . Therefore, the ciphertext forms [f], [i], and [o] of f, i, and o need to be partitioned from [C′ fio to fioTake [o] as an example to construct an index P at the corresponding position 2 , through [C′ fio and the index P 2 Perform homomorphic multiplication to obtain the ciphertext [V] of the vector V, and then perform homomorphic rotation on it to obtain the ciphertext [o] of the vector o

[0121]

[0122] [V]=HomMul(C′ figo ,P 2 ,EVK)

[0123] [o]=HomRot([V], - 3β, EVK)

[0124] That is, the relationship between the vector V and the vector o is:

[0125]

[0126] Therefore, construct a homomorphic data shunting algorithm (HomShunt) to partition the ciphertext, that is:

[0127]

[0128] Figure 3 Is the process diagram of matrix multiplication and activation function calculation before aggregation, Figure 4 Is the process diagram of matrix multiplication and activation function calculation after aggregation, Figure 5 Gives an example diagram of HomShunt.

[0129] To more clearly represent the LSTM Cell calculation process, Algorithm 1 and Figure 6 Gives the calculation process of LSTMCell under ciphertext. The following introduces the execution process of the algorithm:

[0130] Step 1: Perform matrix multiplication on the short-term state H t and the input X t . First, use HomRotate and HomAdd to integrate the short-term state [H t and the input data [X t into a single ciphertext [C XH (line 2). Secondly, use HomMatMul to perform homomorphic matrix multiplication on [C XH and use HomAdd to add the bias to obtain [C figo (line 3).

[0131] Step 2: Calculate the activation function. First, use HomShunt to partition [C figo from the ciphertext [Cfio and [C g (lines 5 and 6). Secondly, perform homomorphic Sigmoid and homomorphic Tanh calculations on [C fio and [C g respectively to obtain [C′ fio and [g].

[0132] Step 3:. Update the long-term state S t . First, use HomShunt to divide [f], [i], and [o] from the ciphertext [C′ fio (lines 10 - 12). Secondly, use homomorphic addition HomAdd and homomorphic multiplication HomMul to update the long-term state to obtain [S t+1 (line 10).

[0133] Step 4: Update the short-term state H t . Use HomTanh and HomMul to update the short-term state [H t+1 .

[0134]

[0135] Test results:

[0136] Table 1 Precision of the sigmoid approximation function calculated by three methods (the approximation range is [-4, 4]).

[0137]

[0138]

[0139] First, we will evaluate the precision of approximating the spike function. By comparison, we adopted three methods to approximate sigmoid, including Taylor series expansion, least squares approximation, and Remez algorithm approximation. The approximation range is [-4, 4]. The maximum absolute error (MAE) between the approximate function value and the sigmoid function value is used as the metric. The results are shown in Table 1. For the 7th-order polynomial, the MAEs of these three methods are 0.14319, 2.13598, and 0.0127 respectively. For the 15th-order polynomial, their MAEs are 0.02516, 14.75238, and 0.0000044 respectively. The results show that the Remez algorithm provides the most accurate approximation, especially when the order of the polynomial increases. Therefore, we use the Remez algorithm to implement CryptoLSTM.

[0140] Table 2 compares the computational efficiency of aggregated and non-aggregated operations in the LSTM Cell under ciphertext.

[0141]

[0142] Secondly, we verified the efficiency improvement effect by the integration method. The experimental results are shown in Table 2. We can see that when the input and output sizes (α, β) of the LSTM are (32, 32), the aggregation of BLT improves the computational efficiency by 154%, the integration of activation functions improves the computational efficiency by 5.9%, and the joint aggregation of matrix multiplication and activation functions improves the computational efficiency by 219%. Similar results are obtained when the values of (α, β) are (64, 16) and (100, 32). The aggregation of matrix multiplication reduces the computational overhead much more than the integration of activation functions, and the joint aggregation achieves better performance than single integration. The main reason is that integrating matrix multiplication can reduce many ciphertext multiplications and automorphic deformation operations, while integrating activation functions can reduce the number of polynomial calculations under ciphertext. Therefore, the aggregation improves the computational efficiency of this scheme.

[0143] Finally, to verify the scalability of this scheme, we applied this scheme to the classification task. When performing binary classification on the datasets IMDB and YELP, we set the lengths of the input sequences to 100, 150, and 200 respectively. For multi-classification on the datasets AGNews and DBPedia, the lengths of the input sequences are set to 30, 50, and 70 respectively. The LSTM dimensions (α, β) are uniformly set to (40, 50). In these 4 datasets, the network structure consists of an embedding layer with 40 units, an LSTM layer with 50 units, a fully connected layer with 20 units, and output layers with 2, 2, 4, and 9 units respectively.

[0144] Table 3 Text Classification Experiment Test Table.

[0145]

[0146] As can be seen from Table 3, on all datasets, the maximum difference between the inference accuracy of plaintext and the inference accuracy under ciphertext is 0.98%, which indicates that this scheme has a very satisfactory accuracy rate for any dataset and input sequence length.

[0147] Details not described in this invention are all well-known technologies in the technical field. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the purpose and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A privacy-preserving LSTM inference method based on fully homomorphic encryption, characterized in that: The method comprises the following steps: The user generates a private key and an evaluation key, uses fully homomorphic encryption to divide the time series data to be inferred into multiple vectors in chronological order, and encrypts each vector to generate ciphertext; The user uploads the ciphertext and evaluation key to the server to directly execute LSTM neural network inference.

2. The privacy-preserving LSTM reasoning method based on fully homomorphic encryption according to claim 1 is characterized in that: The user uploads the ciphertext and the evaluation key to the server to directly execute the calculation process of the LSTM Cell in the LSTM neural network reasoning, which includes the following steps: S1. Use homomorphic rotation and homomorphic addition to integrate short-term state and input into a single ciphertext [C XH ], homomorphic multiplication is used to [C XH ] performs homomorphic matrix multiplication with the combined weight matrix W and adds the bias to obtain [C figo ]; S2, through the homomorphic data diversion algorithm from [C figo ] is divided into [C fio ] and [C g ], respectively for [C fio ] and [C g ] and perform homomorphic Sigmoid function calculation and homomorphic Tanh function calculation to obtain [C′ fio ] and [g]; S3, use the data diversion algorithm to extract the ciphertext [C′ fio ] to divide [f], [i] and [o], and use homomorphic addition and homomorphic multiplication to calculate the long-term state [S t ] is updated to obtain the long-term state [S t+1 ]; S4. Use the homomorphic Tanh function to operate on the long-term state of time step t, and then perform homomorphic multiplication with [o] to obtain the short-term state [H] of time step t+1 t+1 ].

3. The privacy-preserving LSTM reasoning method based on fully homomorphic encryption according to claim 2 is characterized in that: The homomorphic matrix multiplication is specifically: in, [X] k =HomRot([X],k,EVK) HomMatMul represents the matrix multiplication operation under homomorphic encryption, EVK represents the evaluation key, [X] represents the ciphertext uploaded by the user, W∈R n×n , [X]∈R n ; W represents the weight matrix, [Y] represents the result of matrix multiplication of [X], HomAdd represents homomorphic addition operation, and HomMul represents homomorphic multiplication operation; Represents a vector The elements of are rotated by -j·n2 positions; Represents the vector consisting of the elements on the j·n2+kth diagonal of the matrix W; HomRot represents the vector rotation operation under homomorphic encryption.

4. The privacy-preserving LSTM reasoning method based on fully homomorphic encryption according to claim 2 is characterized in that: The homomorphic Sigmoid function and the homomorphic Tanh function use polynomial approximation to perform nonlinear operations.

5. The privacy-preserving LSTM reasoning method based on fully homomorphic encryption according to claim 4 is characterized in that: The method of using polynomial approximation to perform nonlinear operations is: An optimal polynomial is obtained by solving the following optimization problem: The optimization problem is solved by the Remez algorithm to obtain an optimal polynomial P(x), where represents a set of polynomials of degree not exceeding k, x is the independent variable of P(x), [a,b] represents the optimization interval, and f(x) represents the target activation function.

6. The privacy-preserving LSTM reasoning method based on fully homomorphic encryption according to claim 2, characterized in that: The method uses homomorphic rotation and homomorphic addition to integrate short-term state and input into a single ciphertext [C XH ], homomorphic multiplication is used to [C XH ] performs homomorphic matrix multiplication with the combined weight matrix W and adds the bias to obtain [C figo The method specifically includes: In the short-term state H t and input X t During the matrix multiplication, the weight matrices of the two are combined into W, that is: H t Perform homomorphic rotation and combine with X t Perform a homomorphic addition operation, where α and β represent the input latitude and output dimension of the LSTM layer, j and k represent the row index and column index of the weight matrix, respectively. [C XH ]=HomAdd([X t ],HomRot([H t ],-α,EVK),EVK) Yes [C XH ] and W are homomorphically multiplied and the bias is added using homomorphic addition to obtain the ciphertext [C figo ], [C figo ]=HomAdd(HomMatMul([C XH ],W,EVK),bais,EVK) Among them, bais represents bias, Homadd represents homomorphic addition, HomRot represents homomorphic rotation, HomMatMul represents homomorphic multiplication, and EVK represents evaluation key.

7. The privacy-preserving LSTM reasoning method based on fully homomorphic encryption according to claim 6 is characterized in that: The data diversion algorithm is used to separate the ciphertext [C figo ] is divided into [C fio ] and [C g ] is: Construct a vector index P1 corresponding to the position, By [C figo ] is homomorphically multiplied with index P1 to obtain [C fio ], [C fio ]=HomMul([C figo ],P1,EVK) Construct an index P2 corresponding to the position, By [C figo ] is multiplied by the index P2 to obtain the ciphertext of vector V2 [V2], and then it is homomorphically rotated to obtain the vector V g The ciphertext [C g ], [V2]=HomMul([C figo ],P2,EVK) [C g ]=HomRot([V2],-2β,EVK)。 8. The privacy-preserving LSTM reasoning method based on fully homomorphic encryption according to claim 7, characterized in that: The pair [C fio ] is calculated by homomorphic Sigmoid function to obtain [C′ fio ] is: Yes [C fio ] performs homomorphic activation operations, [C′ fio ]=HomSigmoid([C fio ],EVK) HomSigmoid denotes homomorphic sigmoid function, and EVK denotes evaluation key.

9. The privacy-preserving LSTM reasoning method based on fully homomorphic encryption according to claim 8, characterized in that: From [C′ fio ] The method of dividing [f], [i] and [o] is: Construct an index P3 corresponding to the position, By [C′ fio ] is multiplied by index P3 to obtain vector [f], [f]=HomMul([C′ fio ],P3,EVK) Construct an index P4 corresponding to the position, By [C′ fio ] is multiplied by the index P4 to obtain the ciphertext of vector V4 [V4], and then it is homomorphically rotated to obtain the ciphertext of vector i [i] [V4]=HomMul([C′ fip ],P4,EVK) [i]=HomRot([V4],-β,EVK) Construct an index P5 corresponding to the position, By [C′ fio ] is multiplied by the index P5 to obtain the ciphertext of vector V5 [V5], and then it is homomorphically rotated to obtain the ciphertext of vector i [o], [V5]=HomMul([C′ fio ],P5,EVK) [o] = HomRot([V5], -β, EVK).

Images