Data processing system and data processing method for reducing neural network homomorphic computation error

Abstract

The application discloses a data processing system for reducing homomorphic computation error of a neural network, comprising a plaintext neural network module, a ciphertext neural network module, a homomorphic ciphertext decryption module, an error correction layer training data collection module, an error correction layer training module and an error correction module. The application improves the structure of a typical homomorphic computation neural network, and adds an error correction layer after the output layer of the ciphertext neural network. The error correction layer reduces the error caused by factors such as a polynomial approximation activation function by performing inverse operation opposite to the error mode, thereby improving the accuracy of homomorphic computation of the neural network. The application further discloses a data processing method for reducing homomorphic computation error of a neural network.

Description

Technical Field

[0001] This invention relates to a big data processing system, specifically a data processing system for reducing homomorphic computation errors in neural networks. This invention also relates to a data processing method for reducing homomorphic computation errors in neural networks. Background Technology

[0002] With the development of big data and artificial intelligence technologies, more and more enterprises are using various AI-based data mining and analysis methods to improve their operations and management. By uncovering patterns and information in the data, they aim to create business value, enabling better production organization and more precise marketing. However, many enterprises lack the technical capabilities and hardware / software environment for data mining and analysis, necessitating outsourcing these activities to third parties or even cloud platforms. This significantly increases the risk of leakage of sensitive operational data. Therefore, data privacy protection technologies have emerged. These technologies can protect data confidentiality and reduce the risk of sensitive data leakage while performing data analysis.

[0003] Homomorphic encryption is a fundamental data privacy protection technology. Homomorphically encrypted data can undergo the same algebraic operations as the plaintext data, thus achieving "computable but invisible data" and preventing data leakage. However, existing homomorphic encryption schemes only support addition and multiplication operations. For certain operations in neural network models, such as the calculation of nonlinear activation functions, they often cannot be directly performed on homomorphically encrypted data and must be replaced with calculations that can be performed homomorphically. For example, CryptNet (a dense neural network) uses the squared activation function instead of common activation functions such as the Logistic function. This is because the squared activation function is nonlinear and has a low degree, making it easy to perform homomorphic calculations. However, the squared activation function is not conducive to model training; therefore, compared to an equivalent plaintext neural network, its fitting or classification ability is usually slightly worse.

[0004] To reduce the error caused by homomorphic computation in neural network models and improve their fitting and classification performance, polynomial approximation of activation functions can be used. For example, CryptDL (dense-state deep learning) uses polynomial approximation to approximate the activation function, achieving more accurate classification results than CryptNet (dense-state neural network). Additionally, some researchers have used weighted least squares to fit the activation function, also achieving good approximation. However, polynomial approximation inevitably introduces approximation errors. These errors propagate and accumulate across different layers of the neural network, eventually forming complex error patterns that are difficult to represent analytically. These errors ultimately degrade model performance, making the performance of a dense-text neural network inferior to that of a plain-text neural network for a given classification or fitting problem.

[0005] Currently, the following methods can typically be used to improve the accuracy of homomorphic computation in neural networks:

[0006] First, the approximation error can be reduced by increasing the order of the approximation polynomial. However, increasing the polynomial order increases the required multiplication depth and also increases the risk of Runge's phenomenon.

[0007] Second, batch normalization is also a method that helps improve the accuracy of homomorphic computation in neural networks. Its basic principle is to process the input data of each layer into standard normal distribution data, so that the distribution of the input data of subsequent neurons is located in a relatively ideal approximation range of the activation function.

[0008] Third, using secure multi-party computation mechanisms, which involve client-side cooperation to perform homomorphic computation on the client side, is a common method to improve and simplify homomorphic computation in neural networks. However, this approach requires client cooperation and involves significant client-side interaction, which often impacts computational performance.

[0009] The methods described above, when performing homomorphic computation on ciphertext neural networks, basically follow the structure of plaintext neural networks, only making corresponding improvements to some parts of plaintext neural networks that are unsuitable for homomorphic computation. However, due to the complex sources of error in homomorphic ciphertext neural network computation, including approximation error, homomorphic computation error, and changes in error patterns and scales caused by the propagation of various errors in the neural network, it is quite difficult to further reduce errors and improve model performance by optimizing a single part. Summary of the Invention

[0010] The technical problem to be solved by the present invention is to provide a data processing system for reducing the error of homomorphic computation in neural networks. It can improve the accuracy of homomorphic computation in neural networks, reduce the difference between ciphertext and plaintext computation in neural networks, so that the fitting and classification capabilities of homomorphic computation neural networks are as close as possible to their corresponding plaintext models.

[0011] To address the aforementioned technical problems, the technical solution of this invention for a data processing system used to reduce homomorphic computation errors in neural networks is as follows:

[0012] The system includes a plaintext neural network module, a ciphertext neural network module, a homomorphic ciphertext decryption module, an error correction layer training data collection module, an error correction layer training module, and an error correction module. The plaintext neural network module includes a plaintext neural network capable of being trained based on input plaintext data. The ciphertext neural network module includes a ciphertext neural network employing a polynomial approximation activation function suitable for computation on homomorphic ciphertext. The input of the homomorphic ciphertext decryption module is connected to the output layer of the ciphertext neural network module, used to decrypt the output of the ciphertext neural network for training the error correction layer. The input of the error correction layer training data collection module is connected to the output layer of the plaintext neural network module and the output of the homomorphic ciphertext decryption module. This is used to collect the output data of the plaintext neural network and the homomorphic encrypted neural network as training data. This data will be used to train the error correction layer training module. The input of the error correction layer training module is connected to the output of the error correction layer training data collection module. Using the training data output by the error correction layer training data collection module, a polynomial function is used for fitting. The error correction module includes a homomorphic computation error correction layer. The input of the homomorphic computation error correction layer is connected to the output of the error correction layer training module. The homomorphic computation error correction layer corrects the error of the homomorphic encrypted computation of the neural network. The output of the homomorphic computation error correction layer is connected to the encrypted neural network output layer of the encrypted neural network module. The output of the homomorphic computation error correction layer is the final output of the homomorphic encrypted neural network.

[0013] In another embodiment, the error correction layer training module uses a polynomial-based function approximation method to fit y. i With y i The relationship between ′; then:

[0014] y i =φ(x i )

[0015] Among them, y i This refers to the textual model based on the i-th input vector x in the dataset. i The output, vector x i Belonging to a given dataset x = {x1, x2, ..., x}n This dataset is used to train the error correction module;

[0016] φ(·) is a function used to describe the computation process of the plaintext neural network model;

[0017] c i =φ HE (x i )

[0018] Where, φ HE (·) represents a homomorphic ciphertext computation neural network model equivalent to φ(·) for homomorphic computation.

[0019] c i For y i The corresponding ciphertext;

[0020] y i ′ refers to the plaintext result after decryption of the output of the ciphertext model, which is equivalent to the plaintext model;

[0021] y i ≈y i ′=Dec sk (c i )

[0022] Simplifying the above notation, without considering the subscript i, c is the ciphertext result output by the homomorphic computation model for a given dataset x. Decrypting it yields the final plaintext result y′ generated by the homomorphic computation model.

[0023] Dec sk (·) indicates the decryption operation;

[0024] Then y i With y i The error of ′ is:

[0025] ||e HE ||=||yy′||=||y-Dec sk (c)‖

[0026] Among them, e HE This refers to model φ(·) and its equivalent homomorphic computation model φ HE (·) Homomorphic computation error for a given input x.

[0027] In another embodiment, the plaintext neural network includes a plaintext neural network input layer, a plaintext neural network hidden layer, and a plaintext neural network output layer connected in sequence; the plaintext neural network input layer is used to input a given input dataset, and the plaintext neural network output layer is used to obtain the output dataset of the plaintext neural network;

[0028] In another embodiment, the encrypted neural network includes a encrypted neural network input layer, a encrypted neural network hidden layer, and an encrypted neural network output layer connected in sequence; the encrypted neural network has the same network structure and weight parameters as the plaintext neural network; the encrypted neural network input layer is used to input the same input dataset as the plaintext neural network, and the encrypted neural network output layer is used to obtain the output dataset of the encrypted neural network, which is an encrypted state;

[0029] The ciphertext neural network uses the same input as the plaintext neural network and employs a polynomial approximation method to approximate the activation function in the plaintext neural network, outputting a ciphertext state equivalent to that of the plaintext neural network.

[0030] In another embodiment, the error correction module uses the polynomial error correction function fitted by the error correction layer training module. An error correction layer is used in a encrypted neural network to perform error correction calculations on the output data of the output layer of the homomorphic encrypted neural network.

[0031] In another embodiment, the homomorphic computation error correction layer employs an error correction function g(·); then

[0032] ||e HE ′‖=||y-Dec sk (g(c))‖

[0033] ||e HE ′‖<‖e HE ||

[0034] Among them, e HE This refers to model φ(·) and its equivalent homomorphic computation model φ HE (·) The error for a given input x;

[0035] e HE ′ represents the error after error correction, and the error after correction is less than the error before correction.

[0036] In another embodiment, the error correction function g(·) adopts the following polynomial function P. n (x):

[0037] P n (x)=a n x n +a n-1 x n-1 +...+a1x+a0

[0038] Wherein, the polynomial coefficient a i The error correction layer training module uses polynomial function fitting to obtain the result.

[0039] In another embodiment, the polynomial function P n (x) is a polynomial that minimizes the error shown in the following expression:

[0040]

[0041] Among them, y i For plaintext model output,

[0042] P n Let be an nth-order polynomial function used for error correction.

[0043] vector x i Belonging to a given dataset x = {x1, x2, ..., x} n} represents the i-th input vector in the dataset.

[0044] c i For a given input x i The ciphertext output of the output layer of a homomorphic computation neural network.

[0045] Dec sk (·) represents the homomorphic decryption operation.

[0046] E2 is the total error term, which is the sum of the squares of the errors of each element in the dataset;

[0047] Alternatively, the polynomial function P n (x) is a Chebyshev polynomial as shown below:

[0048] T n (x) = cos(n·arccosx);

[0049] Where n is the order of the Chebyshev polynomial;

[0050] The weight function w(x) corresponding to the Chebyshev polynomial is:

[0051] w(x)=(1-x 2 ) -1 / 2 .

[0052] In another embodiment, the output of the homomorphic computation error correction layer is:

[0053]

[0054] Where n is the number of neurons in the previous layer.

[0055] f(·) represents the output layer activation function.

[0056] a′ is the output value of the homomorphic computation error correction layer.

[0057] z represents the net input value of the previous layer.

[0058]

[0059] w i For connection weights,

[0060] x i x is the input to the neuron. i Belonging to a given dataset x = {x1, x2, ..., x} n},

[0061] b is the bias parameter.

[0062] This is the optimal error correction polynomial function of order n obtained through training.

[0063] This invention also provides a data processing method for reducing homomorphic computation errors in neural networks, the technical solution of which includes the following steps:

[0064] Step 1: Train the plaintext neural network;

[0065] Plaintext neural networks are trained using plaintext data to obtain trained plaintext neural networks; the trained plaintext neural networks are used as the benchmark and reference for the corresponding encrypted neural networks;

[0066] Step 2: Obtain the ciphertext neural network equivalent to the plaintext neural network;

[0067] After training, a plaintext neural network is obtained; the model parameters of the trained plaintext neural network are used as the model parameters of the encrypted neural network; the encrypted neural network uses a polynomial approximation activation function.

[0068] Step 3: Obtain the output of the plaintext neural network;

[0069] Using a given input dataset, the output dataset of a plaintext neural network is obtained; the output dataset of the plaintext neural network is used for error correction in homomorphic ciphertext computation.

[0070] Step 4: Obtain the output of the encrypted neural network;

[0071] Using the same input dataset as in step 3, obtain the output dataset of the encrypted neural network;

[0072] Step 5: Decrypt the ciphertext data to obtain the output dataset of the decrypted ciphertext neural network;

[0073] Step 6: Fit the relationship between the output of the encrypted neural network and the output of the plaintext neural network to obtain the fitting polynomial represented by the polynomial vector coefficients;

[0074] Using a polynomial approximation activation function, the output dataset of the decrypted ciphertext neural network obtained in step 5 and the output dataset of the plaintext neural network obtained in step 3 are fitted together.

[0075] Step 7: Perform error correction;

[0076] Using the fitting polynomial obtained in step 6, i.e., the optimal error correction polynomial function, the output dataset of the decrypted ciphertext neural network is calculated to obtain the error-corrected ciphertext neural network; the error correction uses the following formula:

[0077] P n (x)=a n x n +a n-1 x n-1 +...+a1x+a0

[0078] Wherein, the polynomial coefficient a i Obtained through step 6;

[0079] Step 8: Obtain the output of the encrypted neural network after error correction;

[0080] Using a given input dataset, obtain the output dataset of the encrypted neural network after error correction; this output dataset is the final output of the encrypted neural network.

[0081]

[0082] Where n is the number of neurons in the previous layer.

[0083] f(·) represents the output layer activation function.

[0084] a′ is the output value of the homomorphic computation error correction layer.

[0085] z represents the net input value of the previous layer.

[0086]

[0087] w i For connection weights,

[0088] x i x is the input to the neuron. i Belonging to a given dataset x = {x1, x2, ..., x} n},

[0089] b is the bias parameter.

[0090] This is the optimal error correction polynomial function of order n obtained through training.

[0091] In another embodiment, the fitting method in step 6 employs the following error term:

[0092]

[0093] Among them, y i For plaintext model output,

[0094] P n Let be an nth-order polynomial function used for error correction.

[0095] vector x i Belonging to a given dataset x = {x1, x2, ..., x} n} represents the i-th input vector in the dataset.

[0096] c i For a given input x i The ciphertext output of the output layer of a homomorphic computation neural network.

[0097] Dec sk (·) represents the homomorphic decryption operation.

[0098] E2 is the total error term, which is the sum of the squares of the errors of each element in the dataset;

[0099] Alternatively, the fitting method in step 6 can be approximated using Chebyshev polynomials as shown below:

[0100] T n (x) = cos(n·arccosx);

[0101] Where n is the order of the Chebyshev polynomial;

[0102] The weight function w(x) corresponding to the Chebyshev polynomial is:

[0103] w(x)=(1-x 2 ) -1 / 2 .

[0104] The technical effects that this invention can achieve are:

[0105] This invention improves the structure of existing typical homomorphic computation neural networks by adding an error correction layer after the output layer of the encrypted neural network. The function of this layer is to reduce the error caused by factors such as polynomial approximation activation function by performing the inverse operation opposite to the error pattern, thereby improving the accuracy of the neural network in homomorphic computation.

[0106] This invention corrects the output of the encrypted neural network output layer through an error correction module to reduce errors; the output of the homomorphic computation error correction layer is the final output of the encrypted neural network.

[0107] This invention has advantages such as low computational cost, ability to modify encrypted data, no need for client participation in computation, and interpretability of the model. Attached Figure Description

[0108] Those skilled in the art will understand that the following description is merely illustrative of the principles of the invention, which can be applied in various ways to achieve many different alternative implementations. These descriptions are intended only to illustrate the general principles of the teachings of the invention and are not intended to limit the inventive concept disclosed herein.

[0109] Embodiments of the invention are illustrated in conjunction with the accompanying drawings, which are incorporated in and form part of this specification, and together with the foregoing general description and the following detailed description of the drawings, serve to explain the principles of the invention.

[0110] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0111] Figure 1 This is a schematic diagram of the data processing system of the present invention for reducing homomorphic computation errors in neural networks;

[0112] Figure 2 This is a flowchart illustrating the data processing method of the present invention for reducing homomorphic computation errors in neural networks;

[0113] Figure 3 This is a schematic diagram comparing the effects of error correction before and after using the present invention. Detailed Implementation

[0114] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the described embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Unless otherwise defined, the technical or scientific terms used herein should have the ordinary meaning understood by those skilled in the art to which this invention pertains. The terms "first," "second," and similar terms used herein do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" mean that the element or object preceding the word covers the element or object listed following the word and its equivalents, but does not exclude other elements or objects.

[0115] like Figure 1As shown, the data processing system of the present invention for reducing homomorphic computation error in neural networks includes a plaintext neural network module, a ciphertext neural network module, a homomorphic ciphertext decryption module, an error correction layer training data collection module, an error correction layer training module, and an error correction module.

[0116] The plaintext neural network module includes a plaintext neural network that can be trained based on the input plaintext data; the trained plaintext neural network serves as the benchmark and reference for the corresponding homomorphic encrypted neural network; the plaintext neural network is existing technology and will not be described in detail here.

[0117] A plaintext neural network consists of a plaintext neural network input layer, a plaintext neural network hidden layer, and a plaintext neural network output layer connected in sequence. The plaintext neural network input layer is used to input a given input dataset, and the plaintext neural network output layer is used to obtain the output dataset of the plaintext neural network.

[0118] The ciphertext neural network module includes a ciphertext neural network, which employs an activation function that is suitable for computation on homomorphic ciphertexts using a polynomial approximation.

[0119] A encrypted neural network consists of a encrypted neural network input layer, a encrypted neural network hidden layer, and an encrypted neural network output layer connected in sequence. The encrypted neural network has the same network structure and weight parameters as the plaintext neural network. The encrypted neural network input layer is used to input the same input dataset as the plaintext neural network, and the encrypted neural network output layer is used to obtain the output dataset of the encrypted neural network, which is the encrypted state. Therefore, based on the same input, the encrypted neural network uses a polynomial approximation method to approximate the activation function in the plaintext neural network, and can output a result equivalent to the encrypted state of the plaintext neural network.

[0120] The homomorphic ciphertext decryption module has its input connected to the output layer of the ciphertext neural network module. It is used to decrypt the output of the ciphertext neural network in order to train the error correction layer.

[0121] The error correction layer training data collection module has its input connected to the plaintext neural network output layer of the plaintext neural network module and the output of the homomorphic ciphertext decryption module; it is used to collect the output data of the plaintext neural network and the homomorphic ciphertext neural network as training data, which will be used to train the error correction layer training module.

[0122] The error correction layer training module has its input connected to the output of the error correction layer training data collection module; the training data output by the error correction layer training data collection module is used to fit a given polynomial function.

[0123] The error correction layer training module uses a polynomial-based function approximation method to fit y.i With y i The relationship between ′;

[0124] y i This refers to the textual model based on the i-th input vector x in the dataset. i The output, y i ′ refers to the plaintext result obtained after decryption from the output of the ciphertext model, which is equivalent to the plaintext model; then:

[0125] y i =φ(x i )

[0126] Where, vector x i Belonging to a given dataset x = {x1, x2, ..., x} n This dataset is used to train the error correction module;

[0127] φ(·) is a function used to describe the computation process of the plaintext neural network model;

[0128] c i =φ HE (x i )

[0129] Where, φ HE (·) represents a homomorphic ciphertext computation neural network model equivalent to φ(·) for homomorphic computation.

[0130] c i For y i The corresponding ciphertext is:

[0131] y i ≈y i ′=Dec sk (c i )

[0132] Simplifying the above notation, without considering the subscript i, c is the ciphertext result output by the homomorphic computation model for a given dataset x. Decrypting it yields the final plaintext result y' generated by the homomorphic computation model.

[0133] Dec sk (·) indicates the decryption operation.

[0134] Ideally, the computation process of the ciphertext model should be a perfect reproduction of the computation process of the plaintext model on the homomorphic ciphertext. In the absence of computational errors, y should have... i =y i That is, f(x) = x.

[0135] However, due to approximation errors and error propagation, y actually... i ≠yi Furthermore, the two often exhibit a non-linear relationship, and this error can be described by the following expression:

[0136] ||e HE ||=||yy′||=||y-Dec sk (c)‖

[0137] Among them, e HE This refers to model φ(·) and its equivalent homomorphic computation model φ HE (·) Homomorphic computation error for a given input x;

[0138] The training of the error correction layer training module of this invention involves finding the error correction function g(·) in the function space, i.e., y. i =g(y i ′), so that ||y i -Dec sk (g(c i The process of minimizing φ; that is, the function can minimize the homomorphic computation model φ. HE The output domain of (·) is mapped to the output domain of the original plaintext computation model φ(·), so that the result of the ciphertext model is as close as possible to the result of the plaintext model, thereby enabling error correction.

[0139] The error correction module includes a homomorphic computation error correction layer. The input of the homomorphic computation error correction layer is connected to the output of the error correction layer training module. The homomorphic computation error correction layer corrects the error of homomorphic ciphertext computation in the neural network. The output of the homomorphic computation error correction layer is connected to the ciphertext neural network output layer of the ciphertext neural network module. The output of the output of the homomorphic computation error correction layer is the final output of the homomorphic ciphertext neural network.

[0140] The homomorphic computation error correction layer can correct errors in the homomorphic ciphertext computation of neural networks caused by various factors, thereby reducing computational errors and improving model performance; the error correction module uses the polynomial error correction function fitted by the error correction layer training module. An error correction layer is applied to the encrypted neural network to perform error correction calculations on the output data of the homomorphic encrypted neural network output layer;

[0141] Homomorphic computation error correction layers use error correction functions g(·). If the error correction function g(·) satisfies the following conditions, then g(·) is called an effective error correction layer; if there is a function g * If the error after correction by (·) is smaller than the error after correction by other functions in the function space, then the function g is called a function g. * (x) is the optimal error correction function; then

[0142] ||eHE ′‖=||y-Dec sk (g(c))||

[0143] ||e HE ′‖<‖e HE ||

[0144] Among them, e HE This refers to model φ(·) and its equivalent homomorphic computation model φ HE (·) The error for a given input x;

[0145] e HE ′ represents the error after error correction, meaning the error after correction must be less than the error before correction.

[0146] Because of c i Since it is in a ciphertext state, the function g(·) must be able to support homomorphic computation, that is, it can be completed by using only addition and multiplication.

[0147] The g(·) function that satisfies this condition can be a univariate real coefficient polynomial function P. n 9x (abbreviated as polynomial function). A univariate real coefficient polynomial function is a function of the following form:

[0148] P n (x)=a n x n +a n-1 x n-1 +...+a1x+a0

[0149] in, It is the set of real numbers;

[0150] Because of the polynomial function P n (x) only requires addition and multiplication operations, making it suitable for homomorphic ciphertext computation. If a polynomial function is used as the error correction function for homomorphic computation, then according to the properties of homomorphic computation, the learning process of this error correction layer is to find the optimal polynomial error correction function in the polynomial function space. Come to y i and y i The process of polynomial approximation can be carried out using polynomial least squares and Chebyshev polynomial approximation methods, depending on user requirements and specified parameters.

[0151] Polynomial least squares approximation is an approximation method that minimizes the overall error. Specifically, it uses the following error term:

[0152]

[0153] Among them, y i For plaintext model output,

[0154] P n Let be an nth-order polynomial function used for error correction.

[0155] vector x i Belonging to a given dataset x = {x1, x2, ..., x} n} represents the i-th input vector in the dataset.

[0156] c i For a given input x i The ciphertext output of the output layer of a homomorphic computation neural network.

[0157] Dec sk (·) represents the homomorphic decryption operation.

[0158] E2 is the total error term, which is the sum of the squares of the errors of each element in the dataset.

[0159] Chebyshev polynomials of the first kind (hereinafter referred to as Chebyshev polynomials) T n Let w(x) be an orthogonal polynomial of weight function w(x) on the interval [-1,1]. The maximum error can be minimized by approximating with Chebyshev polynomials.

[0160] w(x)=(1-x 2 ) -1 / 2

[0161] T n (x) = cos(n·arccosx);

[0162] The output of the error correction module is:

[0163]

[0164] Where n is the number of neurons in the previous layer.

[0165] f(·) represents the activation function.

[0166] a′ is the output value of the homomorphic computation error correction layer.

[0167] z represents the net input value of the previous layer.

[0168]

[0169] w i For connection weights,

[0170] x i x is the input to the neuron. i Belonging to a given dataset x = {x1, x2, ..., x}n},

[0171] b is the bias parameter.

[0172] This is the optimal error correction polynomial function of order n obtained through training.

[0173] like Figure 2 As shown, the data processing method of the present invention for reducing homomorphic computation error in neural networks includes the following steps:

[0174] Step 1: Train the plaintext neural network;

[0175] A plaintext neural network is trained using plaintext data to obtain a trained plaintext neural network; the trained plaintext neural network is used as the benchmark and reference for the corresponding encrypted neural network; the method of training a plaintext neural network using plaintext data is existing technology and will not be described in detail here.

[0176] Step 2: Obtain the ciphertext neural network equivalent to the plaintext neural network;

[0177] After training, a plaintext neural network is obtained. The model parameters of the trained plaintext neural network, such as the weight matrices of each layer, are used as the model parameters of the ciphertext neural network. Then, the ciphertext neural network has the same network structure and weight parameters as the plaintext neural network, and the ciphertext neural network uses a polynomial approximation activation function that is suitable for computation on homomorphic ciphertext.

[0178] Step 3: Obtain the output of the plaintext neural network;

[0179] Using a given input dataset, the output dataset of a plaintext neural network is obtained; the output dataset of the plaintext neural network is used for error correction in homomorphic ciphertext computation.

[0180] Step 4: Obtain the output of the encrypted neural network;

[0181] Using the same input dataset as in step 3, obtain the output dataset of the encrypted neural network;

[0182] Step 5: Decrypt the ciphertext data to obtain the output dataset of the decrypted ciphertext neural network;

[0183] Since the output dataset of the ciphertext neural network obtained in step 4 is ciphertext data in an encrypted state, it needs to be decrypted before using the data to train the error correction function. The decryption operation is the inverse of the encryption operation performed on the ciphertext data input to the homomorphic ciphertext neural network. The decryption operation of the existing homomorphic encryption scheme can be used, which will not be elaborated here.

[0184] Step 6: Fit the relationship between the output of the encrypted neural network and the output of the plaintext neural network to obtain the fitting polynomial represented by the polynomial vector coefficients;

[0185] Using a polynomial approximation activation function, the output dataset of the decrypted ciphertext neural network obtained in step 5 and the output dataset of the plaintext neural network obtained in step 3 are fitted together.

[0186] In this dataset, the ciphertext data in the output dataset of the decrypted ciphertext neural network is the independent variable, and the data in the output dataset of the plaintext neural network is the dependent variable.

[0187] Specific fitting methods can include polynomial least squares fitting, Chebyshev polynomial fitting, etc.

[0188] The polynomial least squares approximation method uses the following error term:

[0189]

[0190] Among them, y i For plaintext model output,

[0191] P n Let be an nth-order polynomial function used for error correction.

[0192] vector x i Belonging to a given dataset x = {x1, x2, ..., x} n} represents the i-th input vector in the dataset.

[0193] c i For a given input x i The ciphertext output of the output layer of a homomorphic computation neural network.

[0194] Dec sk (·) represents the homomorphic decryption operation.

[0195] E2 is the total error term, which is the sum of the squares of the errors of each element in the dataset.

[0196] Chebyshev polynomial T n (x) is expressed by the following formula:

[0197] T n (x) = cos(n·arccosx);

[0198] Where n is the order of the Chebyshev polynomial;

[0199] The weight function w(x) corresponding to the Chebyshev polynomial is:

[0200] w(x)=(1-x 2 )-1 / 2 .

[0201] The core function of step 6 of this invention is to obtain the coefficient vector a = (a n ,a n-1 The optimal error correction polynomial function is represented by (a1, a0), ..., a1, a0.

[0202] Step 7: Perform error correction;

[0203] Using the fitting polynomial obtained in step 6, i.e., the optimal error correction polynomial function, the output dataset of the decrypted ciphertext neural network is calculated to obtain the error-corrected ciphertext neural network; the error correction uses the following formula:

[0204]

[0205] Wherein, the polynomial coefficient a i Obtained through step 6;

[0206] Step 8: Obtain the output of the encrypted neural network after error correction;

[0207] Using a given input dataset, obtain the output dataset of the encrypted neural network after error correction; this output dataset is the final output of the encrypted neural network.

[0208]

[0209] Where n is the number of neurons in the previous layer.

[0210] f(·) represents the output layer activation function.

[0211] a′ is the output value of the homomorphic computation error correction layer.

[0212] z represents the net input value of the previous layer.

[0213]

[0214] w i For connection weights,

[0215] x i x is the input to the neuron. i Belonging to a given dataset x = {x1, x2, ..., x} n},

[0216] b is the bias parameter.

[0217] This is the optimal error correction polynomial function of order n obtained through training.

[0218] Figure 3This image shows a comparison of the effects of adding a homomorphic computation error correction layer before and after a study using a homomorphic encrypted neural network regressor for time series analysis of computing power costs. The blue scatter points represent values ​​before error correction, and the red scatter points represent values ​​after error correction. It can be seen that after adding the error correction layer, the red scatter points are closer to f(x) = x than the previous blue scatter points, indicating a smaller error.

[0219]

[0220] Table 1

[0221] Table 1 presents the specific performance metrics of the comparative models in this study. In this study, compared to the plaintext multilayer perceptron model, the performance metrics of the encrypted multilayer perceptron model performing homomorphic operations show a significant degradation due to factors such as errors in homomorphic computation. After introducing an error correction layer into the encrypted multilayer perceptron model using the method of this invention for error correction calculation, the model performance metrics are improved to some extent. Although still lower than the performance of the original plaintext multilayer perceptron model, the degradation is effectively mitigated. It is evident that this invention has a significant effect on reducing the errors in homomorphic computation of specific types of neural network models.

[0222] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A data processing system for reducing homomorphic computation errors in neural networks, characterized in that, include: The plaintext neural network module includes a plaintext neural network that can be trained based on input plaintext data. The encrypted neural network module includes an encrypted neural network that employs an activation function that is suitable for computation on homomorphic ciphertexts using a polynomial approximation. The homomorphic ciphertext decryption module has its input connected to the output layer of the ciphertext neural network module. It is used to decrypt the output of the ciphertext neural network in order to train the error correction layer. The error correction layer training data collection module has its input connected to the plaintext neural network output layer of the plaintext neural network module and the output of the homomorphic ciphertext decryption module. This data is used to collect the output data of plaintext neural networks and homomorphic encrypted neural networks as training data, which will be used to train the error correction layer training module. The error correction layer training module has its input connected to the output of the error correction layer training data collection module; a polynomial function is used to fit the training data output by the error correction layer training data collection module. The error correction module includes a homomorphic computation error correction layer. The input of the homomorphic computation error correction layer is connected to the output of the error correction layer training module. The homomorphic computation error correction layer corrects the error of homomorphic ciphertext computation in the neural network. The output of the homomorphic computation error correction layer is connected to the ciphertext neural network output layer of the ciphertext neural network module. The output of the homomorphic computation error correction layer is the final output of the homomorphic ciphertext neural network. The homomorphic computation error correction layer employs an error correction function. The error correction function Using the following polynomial function : Among them, polynomial coefficients The error correction layer training module uses polynomial function fitting to obtain the result. The output of the homomorphic computation error correction layer is: in, This represents the number of neurons in the previous layer. Represents the activation function of the output layer. This is the output value of the homomorphic computation error correction layer. Represents the net input value of the previous layer. For connection weights, For neuron input, Belongs to a given dataset , For bias parameters, This is the optimal error correction polynomial function of order n obtained through training; The error correction function Homomorphic computation model The output domain is mapped to the original plaintext computation model. The output domain.

2. The data processing system for reducing homomorphic computation errors in neural networks according to claim 1, characterized in that, The error correction layer training module uses a polynomial-based function approximation method to fit the data. and Relationship; but: in, This refers to the textual model based on the i-th input vector in the dataset. The output vector Belongs to a given dataset This dataset is used to train the error correction module; This is a function used to describe the computation process of a plaintext neural network model; in, To and An equivalent homomorphic ciphertext computation neural network model that performs homomorphic computation; for The corresponding ciphertext; It refers to the plaintext result obtained after decryption from the output of the ciphertext model, which is equivalent to the plaintext model; Simplifying the above notation, without considering the subscript i... For a homomorphic computation model, given a dataset The output ciphertext result, when decrypted, yields the final plaintext result generated by the homomorphic computation model. ; Indicates decryption operation; but and The error is: in, Refers to the model and its equivalent homomorphic computation model For a given input The homomorphic calculation error.

3. The data processing system for reducing homomorphic computation errors in neural networks according to claim 1, characterized in that, The plaintext neural network includes a plaintext neural network input layer, a plaintext neural network hidden layer, and a plaintext neural network output layer connected in sequence; The plaintext neural network input layer is used to input a given input dataset, and the plaintext neural network output layer is used to obtain the output dataset of the plaintext neural network; And / or, the encrypted neural network includes a encrypted neural network input layer, a encrypted neural network hidden layer, and an encrypted neural network output layer connected in sequence; the encrypted neural network has the same network structure and weight parameters as the plaintext neural network; the encrypted neural network input layer is used to input the same input dataset as the plaintext neural network, and the encrypted neural network output layer is used to obtain the output dataset of the encrypted neural network, wherein the output dataset of the encrypted neural network is an encrypted state; The ciphertext neural network uses the same input as the plaintext neural network and employs a polynomial approximation method to approximate the activation function in the plaintext neural network, outputting a ciphertext state equivalent to that of the plaintext neural network.

4. The data processing system for reducing homomorphic computation errors in neural networks according to claim 1, characterized in that, The error correction module uses the polynomial error correction function fitted by the error correction layer training module. An error correction layer is used in a encrypted neural network to perform error correction calculations on the output data of the output layer of the homomorphic encrypted neural network.

5. The data processing system for reducing homomorphic computation errors in neural networks according to claim 1, characterized in that, The homomorphic computation error correction layer employs an error correction function. ;but in, Refers to the model and its equivalent homomorphic computation model For a given input The error; This indicates the error after error correction; the corrected error is smaller than the original error.

6. The data processing system for reducing homomorphic computation errors in neural networks according to claim 5, characterized in that, The polynomial function The polynomial that minimizes the error shown in the following expression: in, For plaintext model output, Let be an nth-order polynomial function used for error correction. vector Belongs to a given dataset , where is the i-th input vector in the dataset. For a given input The ciphertext output of the output layer of a homomorphic computation neural network For homomorphic decryption operations, This is the total error term, which is the sum of the squares of the errors of each element in the dataset; Alternatively, the polynomial function The Chebyshev polynomial is as follows: ； in, Let be the order of the Chebyshev polynomial; Weight function corresponding to Chebyshev polynomial for: 。 7. A data processing method for reducing homomorphic computation errors in neural networks, characterized in that, Includes the following steps: Step 1: Train the plaintext neural network; A plaintext neural network is trained using plaintext data to obtain a trained plaintext neural network; The trained plaintext neural network is used as the benchmark and reference for the corresponding encrypted neural network; Step 2: Obtain the ciphertext neural network equivalent to the plaintext neural network; After training, a plaintext neural network is obtained; The model parameters of the trained plaintext neural network are used as the model parameters of the encrypted neural network. Encrypted neural networks employ activation functions that are approximated by polynomials; Step 3: Obtain the output of the plaintext neural network; Using a given input dataset, obtain the output dataset of a plaintext neural network; The output dataset of the plaintext neural network is used for error correction in homomorphic ciphertext computation; Step 4: Obtain the output of the encrypted neural network; Using the same input dataset as in step 3, obtain the output dataset of the encrypted neural network; Step 5: Decrypt the ciphertext data to obtain the output dataset of the decrypted ciphertext neural network; Step 6: Fit the relationship between the output of the encrypted neural network and the output of the plaintext neural network to obtain the fitting polynomial represented by the polynomial vector coefficients; Using a polynomial approximation activation function, the output dataset of the decrypted ciphertext neural network obtained in step 5 and the output dataset of the plaintext neural network obtained in step 3 are fitted together. Step 7: Perform error correction; Using the fitting polynomial obtained in step 6, i.e., the optimal error correction polynomial function, the output dataset of the decrypted ciphertext neural network is calculated to obtain the error-corrected ciphertext neural network; the error correction uses the following error correction function: Among them, polynomial coefficients Obtained through step 6; Step 8: Obtain the output of the encrypted neural network after error correction; Using a given input dataset, obtain the output dataset of the encrypted neural network after error correction; this output dataset is the final output of the encrypted neural network. in, This represents the number of neurons in the previous layer. Represents the activation function of the output layer. This is the output value of the homomorphic computation error correction layer. Represents the net input value of the previous layer. For connection weights, For neuron input, Belongs to a given dataset , For bias parameters, This is the optimal error correction polynomial function of order n obtained through training; The error correction function can map the output domain of the homomorphic computation model to the output domain of the original plaintext computation model.

8. The data processing method for reducing homomorphic computation errors in neural networks according to claim 7, characterized in that, The fitting method in step 6 uses the following error term: in, For plaintext model output, Let be an nth-order polynomial function used for error correction. vector Belongs to a given dataset , where is the i-th input vector in the dataset. For a given input The ciphertext output of the output layer of a homomorphic computation neural network For homomorphic decryption operations, This is the total error term, which is the sum of the squares of the errors of each element in the dataset; Alternatively, the fitting method in step 6 can be approximated using Chebyshev polynomials as shown below: ； in, Let be the order of the Chebyshev polynomial; Weight function corresponding to Chebyshev polynomial for: 。

Images