Model training method and apparatus, device, storage medium, and computer program product
By encrypting the local dataset using a fully homomorphic encryption algorithm and training the neural network in the encrypted domain, the problems of data privacy protection and computational efficiency in federated learning are solved, and an efficient model training process is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA MOBILE (SUZHOU) SOFTWARE TECH CO LTD
- Filing Date
- 2026-04-09
- Publication Date
- 2026-07-31
AI Technical Summary
Existing federated learning models have shortcomings in terms of data privacy protection and computational efficiency. Traditional security measures cannot effectively protect user data privacy, and fully homomorphic encryption algorithms have high complexity when computing large-scale data.
The local plaintext dataset is encrypted using a fully homomorphic encryption algorithm to generate a ciphertext dataset. The neural network is then trained in the encrypted domain. A rotation addition algorithm is used to simplify the summation process of the fully connected layers. The central server aggregates the local model update parameters to generate a global model.
While protecting user data privacy, it improves the efficiency and accuracy of model training and simplifies the computational operations of encrypted data during model training.
Smart Images
Figure CN122491382A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to a model training method, apparatus, device, storage medium, and computer program product. Background Technology
[0002] Model training is the core process of machine learning and deep learning. Model training uses a large amount of datasets and algorithms to optimize model parameters so that the model can make accurate predictions and classifications on new datasets.
[0003] Currently, federated learning is widely used in model training; however, traditional security measures such as encryption and access control are insufficient to ensure user data privacy. Furthermore, while homomorphic encryption algorithms allow computation on encrypted data, some computational operators are not yet implemented. Fully homomorphic encryption algorithms satisfy the homomorphism of arbitrary operations, but their computational complexity on encrypted data is high, making them unsuitable for large-scale encrypted computation. Summary of the Invention
[0004] The purpose of this invention is to provide a model training method, apparatus, device, storage medium, and computer program product that can protect user data privacy and security while performing distributed model training based on encrypted datasets, ultimately obtaining a global model update, and effectively improving the efficiency and accuracy of data processing.
[0005] To achieve the above objectives, embodiments of the present invention provide a model training method applied to a client, the method comprising: The local plaintext dataset is encrypted using a fully homomorphic encryption algorithm to obtain a ciphertext dataset. The encrypted dataset is input into a preset neural network model for local model training to obtain local model update parameters; The local model update parameters are uploaded to the server; wherein the local model update parameters are transmitted in an encrypted state. The system receives a global model from the server and performs personalized training based on the global model; wherein the global model is generated by the server by aggregating the local model update parameters uploaded by each client.
[0006] As an improvement to the above scheme, the encryption of the local plaintext dataset based on a fully homomorphic encryption algorithm to obtain a ciphertext dataset includes: The input matrix and weight matrix of the local plaintext dataset are zero-padded to expand the matrix dimension to a power of 2. The processed local plaintext dataset is encrypted using the HEAAN fully homomorphic encryption algorithm to obtain the ciphertext dataset.
[0007] As an improvement to the above scheme, the step of inputting the encrypted dataset into a preset neural network model for local model training to obtain local model update parameters includes: Receive the initial neural network model parameters distributed by the server; Based on the initial neural network model parameters, forward and backward propagation calculations of the neural network are performed in the encrypted domain until a preset iteration stopping condition is reached, thereby obtaining local model update parameters.
[0008] As an improvement to the above scheme, when performing forward and backward propagation calculations of the neural network in the encrypted domain, a ciphertext summation algorithm based on rotation addition is adopted, including: Perform the following steps on the original ciphertext matrix of the ciphertext dataset: The iteration proceeds in steps 1, where n is the size of the original ciphertext matrix and is an integer power of 2. Each iteration divides the current ciphertext matrix into at least one partition of length 2. i+1 The paragraphs, each paragraph containing the first 2 i The elements and the last 2 i The positions of each element are swapped to obtain a rotation matrix. The current ciphertext matrix is then subjected to a fully homomorphic addition operation with the rotation matrix. The result is used as the current ciphertext matrix for the next iteration, with i taking values sequentially. , 3, 2, 1…0; go through After several iterations, a ciphertext element sum matrix is obtained; wherein, the element value at each position in the ciphertext element sum matrix is the sum of all original elements.
[0009] This invention also provides a model training method applied to a server, the method comprising: Receive local model update parameters uploaded by each client; wherein, the local model update parameters are obtained by the client encrypting the local plaintext dataset based on the fully homomorphic encryption algorithm to obtain the ciphertext dataset, inputting the ciphertext dataset into the preset neural network model for local model training, and the local model update parameters are transmitted in an encrypted state. The update parameters of the local models uploaded by each client are aggregated to generate a global model; The global model is distributed to the client so that the client can perform personalized training based on the global model.
[0010] This invention also provides a model training device for use on a client side, the device comprising: The data encryption module is used to encrypt the local plaintext dataset based on the fully homomorphic encryption algorithm to obtain the ciphertext dataset; The local training module is used to input the encrypted dataset into a preset neural network model for local model training to obtain local model update parameters; The model upload module is used to upload the local model update parameters to the server; wherein the local model update parameters are transmitted in an encrypted state. The model training module is used to receive the global model sent by the server and perform personalized training based on the global model; wherein, the global model is generated by the server by aggregating the local model update parameters uploaded by each client.
[0011] This invention also provides a model training device for use on a server, the device comprising: The model receiving module is used to receive local model update parameters uploaded by each client; wherein, the local model update parameters are obtained by the client encrypting the local plaintext dataset based on a fully homomorphic encryption algorithm to obtain a ciphertext dataset, and inputting the ciphertext dataset into a preset neural network model for local model training; The model generation module is used to aggregate the update parameters of local models uploaded by each client and generate a global model. The model distribution module is used to distribute the global model to the client so that the client can perform personalized training based on the global model.
[0012] This invention also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the model training method described in any of the preceding claims.
[0013] This invention also provides a computer-readable storage medium, which includes a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to execute any of the above-described model training methods.
[0014] This invention also provides a computer program product, which includes a computer program or computer instructions. When the computer program or computer instructions are executed by a processor, they implement the model training method described above.
[0015] Compared to existing technologies, the beneficial effects of the model training method, apparatus, device, storage medium, and computer program product provided in this invention are as follows: User nodes encrypt their local plaintext datasets using a fully homomorphic encryption algorithm to obtain ciphertext datasets for model training, thus protecting user data privacy. After fully homomorphic encryption, each user's private data is used locally for local model training using the ciphertext data. The central server aggregates the local model updates uploaded by user nodes to generate a global model. During training, the global model only collects model updates obtained by users training locally using the encrypted dataset, thereby improving the training efficiency of the global model while protecting user data privacy. Furthermore, in the process of model training using the ciphertext dataset, this invention applies deep neural networks to the computation of the fully homomorphically encrypted ciphertext data. Fully connected layers are used in the encrypted domain to implement matrix multiplication and addition operations, and a fully homomorphic encrypted ciphertext summation algorithm based on rotation addition simplifies the summation process within the fully connected layer nodes, thereby improving the computational efficiency of the ciphertext data during model training. Attached Figure Description
[0016] Figure 1 This is a flowchart illustrating a preferred embodiment of a model training method provided by the present invention; Figure 2 This is a schematic diagram of the overall process of a model training method provided by the present invention; Figure 3 This is a schematic diagram of the distributed model training process in a model training method provided by the present invention; Figure 4 This is a flowchart illustrating another preferred embodiment of a model training method provided by the present invention; Figure 5 This is a schematic diagram of a preferred embodiment of a model training device provided by the present invention; Figure 6 This is a schematic diagram of another preferred embodiment of a model training device provided by the present invention; Figure 7 This is a schematic diagram of a preferred embodiment of a terminal device provided by the present invention. Detailed Implementation
[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0018] Please see Figure 1, Figure 1 This is a flowchart illustrating a preferred embodiment of a model training method provided by the present invention. The model training method is applied to a client and includes: S11, encrypt the local plaintext dataset using a fully homomorphic encryption algorithm to obtain a ciphertext dataset; S12, the encrypted dataset is input into a preset neural network model for local model training to obtain local model update parameters; S13, the local model update parameters are uploaded to the server; wherein the local model update parameters are transmitted in an encrypted state; S14, Receive the global model sent by the server, and perform personalized training based on the global model; wherein, the global model is generated by the server by aggregating the local model update parameters uploaded by each client.
[0019] Specifically, this embodiment of the invention provides a model training method applied to a client, such as a user node. Please refer to... Figure 2 and Figure 3 , Figure 2 This is a schematic diagram of the overall process of a model training method provided by the present invention. Figure 3 This is a flowchart illustrating the distributed model training process in a model training method provided by the present invention. The embodiment of the invention includes three modules: data encryption, distributed training, and model generation. User nodes encrypt their local plaintext datasets using a fully homomorphic encryption algorithm to obtain ciphertext datasets. Federated learning allows multiple users to jointly train a global model without sharing their local raw data. In this embodiment, users' privacy data is fully homomorphically encrypted, and each user trains a local model locally using the ciphertext data, collecting local model update parameters. After local training is complete, user nodes upload the local model update parameters to a server, such as a central server. It should be noted that the local model update parameters are always transmitted in an encrypted state. After receiving the local model update parameters uploaded by each user node, the central server aggregates the uploaded parameters to generate a global model and distributes the global model to each user node. After receiving the global model from the central server, user nodes perform personalized training based on the global model to adapt the model to their local data distribution. Personalized training includes fine-tuning using the local dataset and adding extra layers to the global model and training these extra layers using local data.
[0020] This invention uses a fully homomorphic encryption algorithm to directly encrypt the user's plaintext dataset. This allows for addition and multiplication operations on the ciphertext data, enabling its use as a training dataset for distributed model training. Compared to homomorphically encrypting the trained weights, this avoids privacy leaks and improves data security.
[0021] In another preferred embodiment, step S11 involves encrypting the local plaintext dataset using a fully homomorphic encryption algorithm to obtain a ciphertext dataset, comprising: S111, zero-padding is performed on the input matrix and weight matrix of the local plaintext dataset to expand the matrix dimension to a power of 2. S112, the processed local plaintext dataset is encrypted using the HEAAN fully homomorphic encryption algorithm to obtain the ciphertext dataset.
[0022] Specifically, in this embodiment of the invention, when encrypting a local plaintext dataset using a fully homomorphic encryption algorithm, because the HEAAN library requires operations on a polynomial ring, and the polynomial degree / dimension must be a power of 2, the input feature matrix and neural network weight matrix of the local plaintext dataset are first zero-padded to expand the matrix dimensions to a power of 2 to fit the polynomial ring dimension requirement of HEAAN. Then, the HEAAN fully homomorphic encryption algorithm is used to encrypt the processed local plaintext dataset to obtain the ciphertext dataset. For example, assuming the dimension of the input vector is 784 (28x28 image), and the polynomial degree N set by the HEAAN algorithm is 1024 (2^32^28^28), the ciphertext dataset is obtained. 10 Then, 1024-784=240 zeros need to be padded to the end of the vector to make it a vector of length 1024 before encryption.
[0023] In another preferred embodiment, step S12, which involves inputting the encrypted dataset into a preset neural network model for local model training to obtain local model update parameters, includes: S121, Receive the initial neural network model parameters distributed by the server; S122, based on the initial neural network model parameters, perform forward and backward propagation calculations of the neural network in the encrypted domain until the preset iteration stopping condition is reached, and obtain the local model update parameters.
[0024] Specifically, in this embodiment of the invention, users receive initial model parameters distributed by a central server through a client, and train the encrypted data using different neural network models according to actual needs. These neural network models include VGG16, VGG19, ResNet50, ResNet152, or other neural network models distributed by the central server. During local training, users can use different local optimization algorithms, such as stochastic gradient descent (SGD), to perform forward and backward propagation calculations in the encrypted domain until a preset iteration stopping condition is reached. The user's local model generates model update parameters and saves training weights, ensuring that the weights do not leak the user's personal privacy data. Each user's model training dataset is generated and stored locally and is not transmitted to the central server. After local training is complete, the user's local model update parameters, such as parameter gradients and training weights, are uploaded to the central server for storage and merging to form a global model update.
[0025] After aggregating the local model update parameters uploaded by all user nodes, the central server uses the FedMax algorithm as a feature selection algorithm to select important and relevant model updates. After federated learning, this minimizes: ; ; ; in, The local objective trained locally on the user node is determined by the model parameters. The loss function used for prediction; The number of user nodes / model updates obtained by the central server after filtering. The proportion of user nodes selected for the central server out of all user nodes. This represents the total number of user nodes. The number of model updates generated during training for the k-th user node. The number of model updates generated for all user nodes. The FedMax algorithm, applied to filter model updates, ensures that different user nodes have similar activation vectors. ; in, It is the cross-entropy loss on local data. Let i be the activation vector at the input of the last fully connected layer on user node k. The micro batch size for training the dataset for user nodes. It is a hyperparameter used to control the scale of entropy loss. For Kullback-Leibler divergence, The activation vectors are uniformly distributed.
[0026] The FedMax algorithm considers the local model performance of each user node and assigns a weight factor, allowing users with the best performance to contribute more to the global model. The model update parameters selected by the central server from the user's local model updates are stored as part of the master data, categorized by different models. The central server also selects different model weights to distinguish models based on the training weights. For example, an update trained using the VGG16 model is structurally named VGG16-CKPT, and the central server weights and merges them according to the different model categories before storing them in the database.
[0027] After the central server completes weight merging and global model updates, it sends the global model to the user client. Users then perform personalized training based on the updated global model, adapting the model to their local data distribution. This personalized training includes fine-tuning using a local dataset and adding extra layers to the global model, training these extra layers using local data.
[0028] This invention provides a local neural network model that allows users to train their local model using a encrypted dataset and different neural network models. After the global model is updated, users can use the global model and perform personalized training again based on the local encrypted dataset to improve the performance of the local model.
[0029] In yet another preferred embodiment, when performing forward and backward propagation calculations of the neural network in the encrypted domain, a ciphertext summation algorithm based on rotation addition is employed, including: Perform the following steps on the original ciphertext matrix of the ciphertext dataset: The iteration proceeds in steps 1, where n is the size of the original ciphertext matrix and is an integer power of 2. Each iteration divides the current ciphertext matrix into at least one partition of length 2. i+1 The paragraphs, each paragraph containing the first 2 i The elements and the last 2 i The positions of each element are swapped to obtain a rotation matrix. The current ciphertext matrix is then subjected to a fully homomorphic addition operation with the rotation matrix. The result is used as the current ciphertext matrix for the next iteration, with i taking values sequentially. , 3, 2, 1…0; go through After several iterations, a ciphertext element sum matrix is obtained; wherein, the element value at each position in the ciphertext element sum matrix is the sum of all original elements.
[0030] Specifically, fully homomorphic encryption algorithms ensure the security and privacy of user data by entrusting the key to the encryptor (i.e., the data owner). Subsequent computations are performed on the encrypted data until the resulting ciphertext is returned to the user for decryption, thus maintaining confidentiality throughout the process. In this embodiment of the invention, the model training process based on the ciphertext dataset applies a deep neural network to the computation of the fully homomorphically encrypted ciphertext data, using fully connected layers to implement matrix multiplication operations in the encryption domain. Matrix multiplication includes multiplication and summation operations, and since fully homomorphic encryption algorithms do not allow access to individual elements of the ciphertext, this embodiment proposes a fully homomorphic encryption ciphertext summation algorithm based on rotation addition to simplify the summation process within fully connected layer nodes during model training. An exemplary ciphertext element summation process based on rotation addition is as follows: To perform the addition following matrix-weight multiplication, we first obtain the original ciphertext matrix of size n. : ; Then, the original ciphertext matrix Rotate by n / 2, that is, permutate the positions of n / 2 ciphertext elements before and after. ;in, *) represents the permutation operation of the elements in the matrix: ; Will Add to the original ciphertext matrix From , making The first n / 2 elements and the last n / 2 elements are symmetrical; among them, : ; right The first n / 2 elements and the last n / 2 elements are then subjected to rotation and addition operations again to obtain the result. : ; To achieve the sum of all elements, this process needs to be repeated. Next, after After several iterations of rotation and ciphertext addition, the total matrix of ciphertext elements is obtained. Complete the calculation of the sum of all positions in the ciphertext: ; in, .
[0031] It should be noted that in the linear layers of a deep neural network, the individual output of each node is used to construct the ciphertext for subsequent layers, and node position multiplication is essential for the output (i.e., multiplying by a zero vector that is only 1 at the node index). This calculation is performed for each node, and the results are added using the addition property available in the CKKS fully homomorphic encryption algorithm. An exemplary algorithm for ciphertext rotation addition in the encrypted domain is shown in the table below:
[0032] In the above algorithm flow, i starts from k-1 (k= Each time a Rot operation is performed, i is decremented by 1 until i = 0. Rot(c,2) i ) of 2 i This indicates the length of the sequence in this Rot (n=8, i=2, length is 4; i=1, length is 2, i.e., n / 4; i=0, length is 1, i.e., n / 8). A matrix of length n will be... Subsequent substitution.
[0033] For example, when n=8, the first permutation involves swapping the first 4 and the last 4; the second permutation swaps the 1st and 2nd positions and the 3rd and 4th positions in the first 4, and the 1st and 2nd positions and the 3rd and 4th positions in the last 4. Each permutation subsequence is half the length of the subsequence in the previous round, and the permutation stops when the subsequence length is 1.
[0034] Assumption: The original ciphertext matrix C1 is [A1,A2,A3,A4,A5,A6,A7,A8], n=8, k= =3; In the first iteration, i = k - 1 = 2, Rot(C, 2) 2 ) = [A5,A6,A7,A8,A1,A2,A3,A4]; C2=[A1,5, A2,6, A3,7, A4,8, A5,1, A6,2, A7,3, A8,4]; In the second iteration, i = k - 2 = 1, Rot(C, 2 1 ) = [A3,7, A4,8,A1,5, A2,6,A7,3, A8,4,A5,1,A6,2]; C3=[A1,5,3,7, A2,6,4,8, A3,7,1,5, A4,8,2,6, A5,1,7,3, A6,2,8,4, A7,3,5,1, A8,4,6,2]; In the third iteration, i = k - 3 = 0, Rot(C, 2) 0) = [A2,6,4,8, A1,5,3,7, A4,8,2,6, A3,7,1,5, A6,2,8,4, A5,1,7,3, A8,4,6,2, A7,3,5,1]; C4=[A1,5,3,7,2,6,4,8, A2,6,4,8,1,5,3,7 A3,7,1,5,4,8,2,6, A4,8,2,6,3,7,1,5, A5,1,7,3,6,2,8,4, A6,2,8,4,5,7,1,3, A7,3,5,1,8,4,6,2, A8,4,6,2,7,3,5,1]; As can be seen, after three iterations, the element value at each position in the resulting ciphertext element sum matrix C4 is the sum of all the original elements.
[0035] In the process of model training using ciphertext datasets, this invention applies deep neural networks to the computation of fully homomorphically encrypted ciphertext data. Fully connected layers are used in the encryption domain to implement matrix multiplication and addition operations, and a fully homomorphically encrypted ciphertext summation algorithm based on rotation addition simplifies the summation process within the fully connected layer nodes, thereby improving the computational efficiency of ciphertext data during model training.
[0036] Please see Figure 4 , Figure 4 This is a flowchart illustrating a preferred embodiment of a model training method provided by the present invention. The model training method is applied to a server and includes: S21, receive local model update parameters uploaded by each client; wherein, the local model update parameters are obtained by the client encrypting the local plaintext dataset based on the fully homomorphic encryption algorithm to obtain the ciphertext dataset, inputting the ciphertext dataset into the preset neural network model for local model training, and the local model update parameters are transmitted in an encrypted state. S22, aggregate the local model update parameters uploaded by each client to generate a global model; S23, the global model is sent to the client so that the client can perform personalized training based on the global model.
[0037] Specifically, this invention provides a model training method applied to a server, such as a central server. The central server receives local model update parameters uploaded by various clients, such as user nodes. These local model update parameters are obtained by each user node encrypting its local plaintext dataset using a fully homomorphic encryption algorithm to obtain a ciphertext dataset, and then training its local model locally using the ciphertext data. It should be noted that the local model update parameters are transmitted while always in an encrypted state. After receiving the local model update parameters uploaded by each client, the central server aggregates these parameters to generate a global model, and distributes the global model to each user node. This allows user nodes to perform personalized training based on the global model, adapting the model to their local data distribution. Personalized training includes fine-tuning using the local dataset and adding additional layers to the global model and training these additional layers using local data.
[0038] This invention employs a fully homomorphic encryption algorithm to encrypt a user's local plaintext dataset to obtain a ciphertext dataset, which is then used for model training, thus protecting user data privacy. During training, the global model only collects updates obtained from local training using the encrypted dataset, thereby improving the training efficiency of the global model while protecting user data privacy.
[0039] Accordingly, the present invention also provides a model training apparatus capable of implementing all the processes of the model training method in the above embodiments.
[0040] Please see Figure 5 , Figure 5 This is a schematic diagram of a preferred embodiment of a model training device provided by the present invention. The model training device is applied to a client and includes: The data encryption module 501 is used to encrypt the local plaintext dataset based on the fully homomorphic encryption algorithm to obtain the ciphertext dataset; The local training module 502 is used to input the encrypted dataset into a preset neural network model for local model training to obtain local model update parameters; The model upload module 503 is used to upload the local model update parameters to the server; wherein the local model update parameters are transmitted in an encrypted state. The model training module 504 is used to receive the global model sent by the server and perform personalized training based on the global model; wherein, the global model is generated by the server by aggregating the local model update parameters uploaded by each client.
[0041] Preferably, the data encryption module 501 is specifically used for: The input matrix and weight matrix of the local plaintext dataset are zero-padded to expand the matrix dimension to a power of 2. The processed local plaintext dataset is encrypted using the HEAAN fully homomorphic encryption algorithm to obtain the ciphertext dataset.
[0042] Preferably, the local training module 502 is specifically used for: Receive the initial neural network model parameters distributed by the server; Based on the initial neural network model parameters, forward and backward propagation calculations of the neural network are performed in the encrypted domain until a preset iteration stopping condition is reached, thereby obtaining local model update parameters.
[0043] Preferably, when performing forward and backward propagation calculations of the neural network in the encrypted domain, a ciphertext summation algorithm based on rotation addition is employed, including: Perform the following steps on the original ciphertext matrix of the ciphertext dataset: The iteration proceeds in steps 1, where n is the size of the original ciphertext matrix and is an integer power of 2. Each iteration divides the current ciphertext matrix into at least one partition of length 2. i+1 The paragraphs, each paragraph containing the first 2 i The elements and the last 2 i The positions of each element are swapped to obtain a rotation matrix. The current ciphertext matrix is then subjected to a fully homomorphic addition operation with the rotation matrix. The result is used as the current ciphertext matrix for the next iteration, with i taking values sequentially. , 3, 2, 1…0; go through After several iterations, a ciphertext element sum matrix is obtained; wherein, the element value at each position in the ciphertext element sum matrix is the sum of all original elements.
[0044] Please see Figure 6 , Figure 6 This is a schematic diagram of a preferred embodiment of a model training device provided by the present invention. The model training device is applied to a server, and the device includes: The model receiving module 601 is used to receive local model update parameters uploaded by each client; wherein, the local model update parameters are obtained by the client encrypting the local plaintext dataset based on the fully homomorphic encryption algorithm to obtain the ciphertext dataset, and inputting the ciphertext dataset into the preset neural network model for local model training; The model generation module 602 is used to aggregate the local model update parameters uploaded by each client to generate a global model; The model distribution module 603 is used to distribute the global model to the client so that the client can perform personalized training based on the global model.
[0045] In specific implementation, the working principle, control process and technical effects of the model training device provided in the embodiments of the present invention are the same as those of the model training method in the above embodiments, and will not be repeated here.
[0046] Please see Figure 7 , Figure 7 This is a schematic diagram of a preferred embodiment of a terminal device provided by the present invention. The terminal device includes a processor 701, a memory 702, and a computer program stored in the memory 702 and configured to be executed by the processor 701. When the processor 701 executes the computer program, it implements the model training method described in any of the above embodiments.
[0047] Preferably, the computer program can be divided into one or more modules / units (such as computer program 1, computer program 2, ...), and the one or more modules / units are stored in the memory 702 and executed by the processor 701 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device.
[0048] The processor 701 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor 701 can be any conventional processor. The processor 701 is the control center of the terminal device, connecting various parts of the terminal device through various interfaces and lines.
[0049] The memory 702 mainly includes a program storage area and a data storage area. The program storage area can store the operating system, applications required for at least one function, etc., and the data storage area can store related data, etc. In addition, the memory 702 can be a high-speed random access memory, or a non-volatile memory, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, and a flash card, etc., or the memory 702 can also be other volatile solid-state storage devices.
[0050] It should be noted that the aforementioned terminal devices may include, but are not limited to, processors and memory, as will be understood by those skilled in the art. Figure 7 The structural diagram is merely an example of the terminal device described above and does not constitute a limitation on the terminal device described above. It may include more or fewer components than shown in the diagram, or combine certain components, or use different components.
[0051] This invention also provides a computer-readable storage medium, which includes a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to execute the model training method described in any of the above embodiments.
[0052] This invention also provides a computer program product, which includes a computer program or computer instructions. When the computer program or computer instructions are executed by a processor, they implement the model training method described in any of the above embodiments.
[0053] This invention provides a model training method, apparatus, device, storage medium, and computer program product. User nodes encrypt their local plaintext datasets using a fully homomorphic encryption algorithm to obtain ciphertext datasets for model training, thus protecting user data privacy. After fully homomorphic encryption, each user's private data is used locally for local model training. A central server aggregates the local model updates uploaded by user nodes to generate a global model. During training, the global model only collects model updates obtained by users training locally using the encrypted datasets, thereby improving the training efficiency of the global model while protecting user data privacy. Furthermore, during model training using the ciphertext dataset, this invention applies deep neural networks to the computation of the fully homomorphically encrypted ciphertext data. Fully connected layers are used in the encrypted domain to implement matrix multiplication and addition operations, and a fully homomorphic encrypted ciphertext summation algorithm based on rotational addition simplifies the summation process within the fully connected layer nodes, thereby improving the computational efficiency of the ciphertext data during model training.
[0054] It should be noted that the system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the system embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.
[0055] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.
Claims
1. A model training method, characterized in that, Applied to a client, the method includes: The local plaintext dataset is encrypted using a fully homomorphic encryption algorithm to obtain a ciphertext dataset. The encrypted dataset is input into a preset neural network model for local model training to obtain local model update parameters; The local model update parameters are uploaded to the server; wherein the local model update parameters are transmitted in an encrypted state. The system receives a global model from the server and performs personalized training based on the global model; wherein the global model is generated by the server by aggregating the local model update parameters uploaded by each client.
2. The model training method as described in claim 1, characterized in that, The encryption of the local plaintext dataset based on the fully homomorphic encryption algorithm to obtain the ciphertext dataset includes: The input matrix and weight matrix of the local plaintext dataset are zero-padded to expand the matrix dimension to a power of 2. The processed local plaintext dataset is encrypted using the HEAAN fully homomorphic encryption algorithm to obtain the ciphertext dataset.
3. The model training method as described in claim 2, characterized in that, The step of inputting the encrypted dataset into a preset neural network model for local model training to obtain local model update parameters includes: Receive the initial neural network model parameters distributed by the server; Based on the initial neural network model parameters, forward and backward propagation calculations of the neural network are performed in the encrypted domain until a preset iteration stopping condition is reached, thereby obtaining local model update parameters.
4. The model training method as described in claim 3, characterized in that, When performing forward and backward propagation calculations of the neural network in the encrypted domain, a ciphertext summation algorithm based on rotation addition is employed, including: Perform the following steps on the original ciphertext matrix of the ciphertext dataset: The iteration proceeds in steps 1, where n is the size of the original ciphertext matrix and is an integer power of 2. Each iteration divides the current ciphertext matrix into at least one partition of length 2. i+1 The paragraphs, each paragraph containing the first 2 i The elements and the last 2 i The positions of each element are swapped to obtain a rotation matrix. The current ciphertext matrix is then subjected to a fully homomorphic addition operation with the rotation matrix. The result is used as the current ciphertext matrix for the next iteration, with i taking values sequentially. , 3, 2, 1…0; go through After several iterations, a ciphertext element sum matrix is obtained; wherein, the element value at each position in the ciphertext element sum matrix is the sum of all original elements.
5. A model training method, characterized in that, Applied to the server side, the method includes: Receive local model update parameters uploaded by each client; wherein, the local model update parameters are obtained by the client encrypting the local plaintext dataset based on the fully homomorphic encryption algorithm to obtain the ciphertext dataset, inputting the ciphertext dataset into the preset neural network model for local model training, and the local model update parameters are transmitted in an encrypted state. The update parameters of the local models uploaded by each client are aggregated to generate a global model; The global model is distributed to the client so that the client can perform personalized training based on the global model.
6. A model training device, characterized in that, Applied to a client, the device includes: The data encryption module is used to encrypt the local plaintext dataset based on the fully homomorphic encryption algorithm to obtain the ciphertext dataset; The local training module is used to input the encrypted dataset into a preset neural network model for local model training to obtain local model update parameters; The model upload module is used to upload the local model update parameters to the server; wherein the local model update parameters are transmitted in an encrypted state. The model training module is used to receive the global model sent by the server and perform personalized training based on the global model; wherein, the global model is generated by the server by aggregating the local model update parameters uploaded by each client.
7. A model training device, characterized in that, Applied to the server side, the device includes: The model receiving module is used to receive local model update parameters uploaded by each client; wherein, the local model update parameters are obtained by the client encrypting the local plaintext dataset based on the fully homomorphic encryption algorithm to obtain the ciphertext dataset, and inputting the ciphertext dataset into the preset neural network model for local model training; The model generation module is used to aggregate the update parameters of local models uploaded by each client and generate a global model. The model distribution module is used to distribute the global model to the client so that the client can perform personalized training based on the global model.
8. A terminal device, characterized in that, The system includes a processor and a memory, wherein the memory stores a computer program and the computer program is configured to be executed by the processor, wherein the processor, when executing the computer program, implements the model training method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the device containing the computer-readable storage medium executes the computer program, it implements the model training method as described in any one of claims 1 to 6.
10. A computer program product, characterized in that, The computer program product includes a computer program or computer instructions, which, when executed by a processor, implement the model training method as described in any one of claims 1 to 6.