A secure and lightweight data interaction system based on modular exponentiation optimization
By adopting a secure and lightweight data interaction system with mode power optimization in the power system, combining federated learning and transfer learning, a personalized power privacy data classification model is built, and the power is solved through pruning algorithm and mode power optimization homomorphic encryption. The problems of privacy leakage and high computing overhead in terminal data interaction have achieved improvements in stability and security.
Patent Information
- Application Number
- CN202411046675.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-08-01
AI Technical Summary
In power systems, data interaction between distributed terminals and central nodes has problems such as privacy leakage risks and high computing overhead, making it difficult to take into account both security and lightweight.
Adopt a secure and lightweight data interaction system based on model power optimization, through the collaborative work of intelligent power terminals, edge nodes and cloud servers, use federated learning and transfer learning to build a personalized power privacy data classification model, and reduce the model through pruning algorithm Parameter transmission volume, use a lightweight homomorphic encryption algorithm optimized by modulo exponentialization to reduce computational overhead.
It effectively improves the stability and security of model aggregation and data interaction processes, reduces transmission overhead and computing overhead, meets the demand for personalized models of power terminals, and alleviates the operating pressure caused by resource constraints.
Smart Images

Figure CN118573707B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of terminal interaction, and in particular to a secure and lightweight data interaction system based on modular exponentiation optimization. Background Art
[0002] As the scale and type of data in the power system continue to expand, the proportion of distributed energy such as photovoltaics and wind power increases, and the data collected by terminals becomes more heterogeneous and complex. Federated learning provides more possibilities for the secure interaction of distributed node parameters, and to a certain extent realizes the feature aggregation of power terminal data, solving the problem of sharing between heterogeneous data. However, training data such as model parameters and gradients need to be frequently transmitted between power distributed terminals and central nodes. This process poses a risk of privacy leakage, and it is difficult to prevent malicious participants from stealing or tampering with data interaction information, which seriously undermines the stability of the model.
[0003] Different from the traditional centralized machine learning paradigm, federated transfer learning uses homomorphic encryption and polynomial approximation to replace differential privacy technology, and builds personalized models for terminal devices while protecting data privacy. However, in order to meet the stringent requirements of high security, the encryption process often introduces high computing overhead and parameter communication overhead. At the same time, the lack of terminal device resources also makes it difficult for data interaction and information sharing processes to balance security and lightweight. Therefore, a personalized and lightweight transformation solution is urgently needed to solve the lightweight collaborative computing problem of large-scale heterogeneous power terminal computing models. Summary of the invention
[0004] In order to solve the above problems, the purpose of the present invention is to provide a safe and lightweight data interaction system based on modular power optimization, which solves the difficult problem of high transmission overhead and computing overhead in scenarios such as heterogeneous power terminal data and significant demand differences, and effectively improves the stability and security of model aggregation and data interaction processes.
[0005] To achieve the above object, the present invention adopts the following technical solutions:
[0006] A secure and lightweight data interaction system based on modular exponentiation optimization, comprising:
[0007] Smart power terminals, used to collect power-related data;
[0008] The cloud server is responsible for collecting the model parameter data transmitted by the edge nodes, aggregating the model parameter data according to the security aggregation algorithm, decrypting the encrypted data in the learning process, and calculating the new model parameter data for updating the global model;
[0009] The edge node obtains the edge-side local model from the cloud server, and trains the local model using the data obtained by the smart power terminal; the edge node encrypts the trained model parameters and sends them to the cloud server; the edge node trains the power privacy data classification model based on the global model, local model and power-related data updated by the cloud server, and in the training process of the power privacy data classification model, uses the idea of transfer learning to perform personalized model training.
[0010] Furthermore, the cloud server is the top layer, connected to the edge nodes in all regions, and builds models and shares parameters with the edge nodes through federated learning. The federated learning includes two parts: global model and local model learning, as follows:
[0011] Training data on the cloud and terminal data For training, the model learning objective of the cloud server is expressed as:
[0012] (1)
[0013] in, represents the parameters to be learned of the global deep neural network, represents the cloud model to be learned, represents the total number of samples of cloud training data, represents the loss function, Represents sample data for cloud training;
[0014] When the global model After the training is completed, the cloud server sends the trained model to each edge node;
[0015] When the global model training is completed, the cloud server sends the trained model to each edge node;
[0016] Edge Node The learning objectives are:
[0017] (2)
[0018] in, represents the parameters to be learned of the local deep neural network, represents the local model to be learned, Represents an edge node The total number of samples in the training data, represents the loss function, Represents an edge node Sample data for training;
[0019] When all local models After convergence, its model parameters will be uploaded to the cloud server for aggregation; the federated averaging algorithm FeDAvg algorithm is used to align local models;
[0020] In each round of training, The average value of the local models is used to update the global model. The updated global model It can be expressed as:
[0021] (3)
[0022] in, The weights of the neural network are calculated, and the global and local model parameter update and learning processes are iterated to finally obtain a personalized power privacy data classification model.
[0023] Furthermore, edge nodes are specifically used for:
[0024] First, the public dataset used to train the global model on the cloud server is used as the source domain, and the private data used to train the local model on the edge node is used as the target domain;
[0025] The basic neural network used to train the global model and the local model is divided into feature extraction layers. , Data alignment layer And the resulting output layer ;
[0026] Feature extraction layer: This layer consists of 2 convolutional layers and 2 normalization layers, which is used for global feature extraction;
[0027] Data alignment layer: This layer consists of a fully connected layer FC, which uses the maximum mean variance kernel learning method, expressed as:
[0028] (4)
[0029] in, represents the Frobenius norm of the matrix, represents the total number of samples of cloud training data, Represents an edge node The total number of samples in the training data, Represents a nonlinear mapping function; the larger the MMD, the greater the difference between the source domain and the target domain; using the source domain data and the target domain data to calculate the MMD, the loss function of the local model can be expressed as:
[0030] (5)
[0031] in, is a parameter that balances the classification task and the MMD ratio; the goal is to minimize the total loss;
[0032] Result output layer: This layer consists of 1 fully connected layer and 1 Softmax layer, which is used to output the classification results of smart power terminal data.
[0033] Furthermore, the fully connected layer focuses on the learning task and the specific features of the smart power terminal data, so the parameters are updated during the model training process. Softmax is used as the classification function, and its calculation formula is:
[0034] (6)
[0035] in, Represents an edge node The power consumption data of the smart power terminal input on is the total amount of electricity consumption data during training, Represents the output result of one round of network forward propagation, Indicates the data classification result.
[0036] Furthermore, in the process of federated transfer learning, a model compression algorithm based on a pruning algorithm is used to reduce the amount of model parameter transmission, as follows:
[0037] The feature extraction layer structure is Conv+BN+Relu. After the model training, the Conv layer usually contains two parameters: weight and bias, while the BN layer contains four parameters: weight, bias, mean, and variance. The activation function only works during the training process and has no parameters after training.
[0038] Assume that the weight obtained by Conv layer training is , the bias is ; The weight obtained by BN layer training is , the bias is , the mean is , the variance is , assuming that in this structure, the data input is , the intermediate output after the Conv layer , and then after the BN layer and the Relu layer, the output is , then under normal circumstances, the calculation formula after the convolution layer is:
[0039] (7)
[0040] The calculation formula after the BN layer is:
[0041] (8)
[0042] Will Substituting into the above formula, we get:
[0043] (9)
[0044] Simplified:
[0045] (10)
[0046] From the structure of the above formula, we can see that its form is the same as that of a single convolutional layer, so the new weights can be defined according to the combined formula and bias :
[0047] (11)
[0048] Based on the above formula, the original Conv+BN+Relu structure is converted to the Conv structure.
[0049] Furthermore, a key generation center is provided to generate key pairs used in the lightweight homomorphic encryption process and distribute them to the corresponding edge nodes or cloud servers. The key generation center generates keys based on the lightweight homomorphic encryption algorithm optimized by modular exponentiation, as follows:
[0050] Using Carmichael's theorem to add two privacy parameters and , the definitions of these two parameters are as follows:
[0051] (13)
[0052] (14)
[0053] in ,and ;
[0054] Initialize a random prime number ,make ,calculate ,when and are mutually prime, or satisfy When, return ,otherwise , traverse and continue searching;
[0055] Modular exponentiation optimization:
[0056] First define ,make:
[0057] (15)
[0058] because , then
[0059] (16)
[0060] The value of T is given by Finally, we calculate The value of Calculated;
[0061] By pre-calculating the settings , and save it as one of the private keys.
[0062] A secure and lightweight data interaction method based on modular exponentiation optimization, comprising:
[0063] First, the edge node uses the public dataset from the cloud server for training the global model as the source domain, and the private data from the edge node for training the local model as the target domain;
[0064] The basic neural network used to train the global model and the local model is divided into feature extraction layers. , Data alignment layer And the resulting output layer ;
[0065] Feature extraction layer: This layer consists of 2 convolutional layers and 2 normalization layers, which is used for global feature extraction;
[0066] Data alignment layer: This layer consists of a fully connected layer FC, which uses the maximum mean variance kernel learning method, expressed as:
[0067] (4)
[0068] in, represents the Frobenius norm of the matrix, represents the total number of samples of cloud training data, Represents an edge node The total number of samples in the training data, Represents a nonlinear mapping function; the larger the MMD, the greater the difference between the source domain and the target domain; using the source domain data and the target domain data to calculate the MMD, the loss function of the local model can be expressed as:
[0069] (5)
[0070] in, is a parameter that balances the classification task and the MMD ratio; the goal is to minimize the total loss;
[0071] Result output layer: This layer consists of 1 fully connected layer and 1 Softmax layer, which is used to output the classification results of smart power terminal data.
[0072] Furthermore, the server is the top layer, connected to the edge nodes in all regions, and builds models and shares parameters with the edge nodes through federated learning. The federated learning includes two parts: global model and local model learning, as follows:
[0073] Training data on the cloud and terminal data For training, the model learning objective of the cloud server is expressed as:
[0074] (1)
[0075] in, represents the parameters to be learned of the global deep neural network, represents the cloud model to be learned, represents the total number of samples of cloud training data, represents the loss function, Represents sample data for cloud training;
[0076] When the global model After the training is completed, the cloud server sends the trained model to each edge node;
[0077] When the global model training is completed, the cloud server sends the trained model to each edge node;
[0078] Edge Node The learning objectives are:
[0079] (2)
[0080] in, represents the parameters to be learned of the local deep neural network, represents the local model to be learned, Represents an edge node The total number of samples in the training data, represents the loss function, Represents an edge node Sample data for training;
[0081] When all local models After convergence, its model parameters will be uploaded to the cloud server for aggregation; the federated averaging algorithm FeDAvg algorithm is used to align local models;
[0082] In each round of training, The average value of the local models is used to update the global model. The updated global model It can be expressed as:
[0083] (3)
[0084] in, The weights of the neural network are iterated, and the global and local model parameter update and learning process are iterated to finally obtain a personalized power privacy data classification model.
[0085] Furthermore, the fully connected layer focuses on the learning task and the specific features of the smart power terminal data, so the parameters are updated during the model training process. Softmax is used as the classification function, and its calculation formula is:
[0086] (6)
[0087] in, Represents an edge node The power consumption data of the smart power terminal input on is the total amount of electricity consumption data during training, Represents the output result of one round of network forward propagation, Indicates the data classification result.
[0088] Furthermore, in the process of federated transfer learning, a model compression algorithm based on a pruning algorithm is used to reduce the amount of model parameter transmission, as follows:
[0089] The feature extraction layer structure is Conv+BN+Relu. After the model training, the Conv layer usually contains two parameters: weight and bias, while the BN layer contains four parameters: weight, bias, mean, and variance. The activation function only works during the training process and has no parameters after training.
[0090] Assume that the weight obtained by Conv layer training is , the bias is ; The weight obtained by BN layer training is , the bias is , the mean is , the variance is , assuming that in this structure, the data input is , the intermediate output after the Conv layer , and then after the BN layer and the Relu layer, the output is , then under normal circumstances, the calculation formula after the convolution layer is:
[0091] (7)
[0092] The calculation formula after the BN layer is:
[0093] (8)
[0094] Will Substituting into the above formula, we get:
[0095] (9)
[0096] Simplified:
[0097] (10)
[0098] From the structure of the above formula, we can see that its form is the same as that of a single convolutional layer, so the new weights can be defined according to the combined formula and bias :
[0099] (11)
[0100] Based on the above formula, the original Conv+BN+Relu structure is converted to the Conv structure.
[0101] The present invention has the following beneficial effects:
[0102] 1. The present invention constructs an edge-end collaborative power network architecture; based on full consideration of data characteristics and network conditions, a federated transfer learning framework is introduced and a privacy data classification model is designed; the model covers key processes such as model training, updating and parameter encryption, and can effectively aggregate source data from multiple power terminals, thereby shielding the data heterogeneity caused by differences in underlying terminals in the power system; at the same time, the model meets the actual needs of power terminals for personalized models;
[0103] 2. Based on the transfer learning framework, the present invention takes into account that the large number of neural network model parameters is the main factor restricting the model convergence speed, and constructs a model compression scheme based on the pruning algorithm. By reducing the transmission amount of model parameters, the problem of large transmission overhead in the federated transfer learning process is effectively solved, and the current situation of poor application of terminals in the data interaction process due to resource limitations is significantly alleviated;
[0104] 3. Aiming at the specific scenario where power terminal resources are limited, the present invention uses homomorphic encryption as the privacy protection algorithm for data interaction, improves the encryption operation logic, and proposes a lightweight homomorphic encryption algorithm based on modular exponentiation optimization, which effectively solves the problem of high computational overhead in the homomorphic encryption process and further alleviates the operating pressure caused by limited terminal resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0105] Figure 1 The system architecture of the present invention;
[0106] Figure 2 A basic neural network architecture in one embodiment of the present invention;
[0107] Figure 3 is a feature extraction layer structure in one embodiment of the present invention;
[0108] Figure 4 Schematic diagram of model compression in one embodiment of the present invention. DETAILED DESCRIPTION
[0109] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:
[0110] refer to Figure 1 ,In this embodiment, a secure and lightweight data interaction system based on modular exponentiation optimization is provided, including a smart power terminal, an edge node, a cloud server and a key generation center;
[0111] The smart power terminal is the bottom layer. According to the density of smart power terminals, the terminal layer is divided into multiple areas. Each area corresponds to an upper edge node. The smart power terminals of users in the area will collect power-related data and transmit the data to the connected edge nodes.
[0112] The edge node establishes a connection with the cloud server through the network and is used to build a power privacy data classification model;
[0113] The cloud server is the top layer, connected to the edge nodes in all regions, and builds models and shares parameters with the edge nodes through federated learning;
[0114] The key generation center is responsible for generating key pairs used in the parameter sharing process between the cloud and the edge, and distributing them to the corresponding edge nodes or cloud servers.
[0115] In this embodiment, smart power terminal: There are many types of smart power terminals, which are terminal devices that collect all power-related data online. As the perception layer, the smart power terminal is mainly responsible for sensing and collecting work, reflecting the operating status of the power system and various variables around the equipment, which are closely related to the operation of the power system. The storage resources and computing power of the smart power terminal itself are limited, and the data needs to be transmitted to the corresponding edge node for storage and model training, but the data it holds is highly sensitive, and the corresponding algorithm needs to be used to encrypt the data to ensure the security of the transmission process.
[0116] Based on federated learning, the edge node first obtains the edge-side local model from the cloud server, and then uses the terminal data to train the local model; during the entire learning process, the edge node adopts lightweight homomorphic encryption to encrypt the trained model parameters and send them to the cloud server; finally, the edge node trains the power privacy data classification model based on the updated global model, local model and terminal data; in the training process of the power privacy data classification model, the idea of transfer learning is used to perform personalized model training.
[0117] Cloud server: The cloud server has the storage, computing and transmission capabilities that meet the model requirements, plays the role of a data processing center, and establishes connections with all edge nodes in each region. In this system model, the cloud server is responsible for collecting model parameter data transmitted by edge nodes, aggregating parameters according to a secure aggregation algorithm, decrypting encrypted data during the learning process, calculating new parameters for training the global model, and storing and using the new global model.
[0118] Key Generation Center: The key generation center is a fully trusted entity with key generation capabilities. During the model initialization phase, it is responsible for generating and distributing public and private keys to each node. The key generation center only participates in system initialization and is not involved in subsequent data transmission and encryption and decryption processes.
[0119] In this embodiment, it is assumed that Edge Nodes , the smart power terminal data set obtained by each edge node is In order to train a power privacy data classification model on the edge node, based on federated transfer learning, each edge node is trained to obtain a privacy-preserving, personalized power privacy data classification model PPDCM without uploading the smart power terminal data to the cloud server.
[0120] In this embodiment, preferably, federated learning includes two parts: global model learning and local model learning, as follows:
[0121] Training data on the cloud and terminal data For training, the model learning objective of the cloud server is expressed as:
[0122] (1)
[0123] in, represents the parameters to be learned of the global deep neural network, represents the cloud model to be learned, represents the total number of samples of cloud training data, represents the loss function, Represents sample data for cloud training;
[0124] When the global model After the training is completed, the cloud server sends the trained model to each edge node;
[0125] When the global model training is completed, the cloud server sends the trained model to each edge node;
[0126] Edge Node The learning objectives are:
[0127] (2)
[0128] in, represents the parameters to be learned of the local deep neural network, represents the local model to be learned, Represents an edge node The total number of samples in the training data, represents the loss function, Represents an edge node Sample data for training;
[0129] When all local models After convergence, its model parameters will be uploaded to the cloud server for aggregation; the federated averaging algorithm FeDAvg algorithm is used to align local models;
[0130] In each round of training, The average value of the local models is used to update the global model. The updated global model It can be expressed as:
[0131] (3)
[0132] in, The weights of the neural network are calculated, and the global and local model parameter update and learning processes are iterated to finally obtain a personalized power privacy data classification model.
[0133] In this embodiment, although the edge node can directly use the global model trained by the cloud server, the data on the power terminal device is highly personalized, and the main reason for this phenomenon is the distribution difference between the terminal data and the cloud data. The global model only learns the coarse-grained features of all terminals, but cannot learn the fine-grained information of a specific terminal. In the conventional deep learning-based data classification model training process, as the neural network transitions from low-level to high-level, the portability of data features gradually decreases. Therefore, in the federated learning in the power Internet of Things scenario, the performance of directly applying the federated learning network is poor. To address this problem, the MPO-SLDEM proposed in this chapter combines the transfer learning theory to build a separate personalized power privacy data classification model for each edge node.
[0134] refer to Figure 2 In the training process of the power privacy data classification model, the idea of transfer learning is used to perform personalized model training, as follows:
[0135] First, the public dataset used to train the global model on the cloud server is used as the source domain, and the private data used to train the local model on the edge node is used as the target domain;
[0136] The basic neural network used to train the global model and the local model is divided into feature extraction layers. , Data alignment layer And the resulting output layer ;
[0137] Feature extraction layer: This layer consists of 2 convolutional layers and 2 normalization layers, which is used for global feature extraction;
[0138] Data alignment layer: This layer consists of a fully connected layer FC, which uses the maximum mean variance kernel learning method, expressed as:
[0139] (4)
[0140] in, represents the Frobenius norm of the matrix, represents the total number of samples of cloud training data, Represents an edge node The total number of samples in the training data, Represents a nonlinear mapping function; the larger the MMD, the greater the difference between the source domain and the target domain; using the source domain data and the target domain data to calculate the MMD, the loss function of the local model can be expressed as:
[0141] (5)
[0142] in, is a parameter that balances the classification task and the MMD ratio; the goal is to minimize the total loss;
[0143] Result output layer: This layer consists of 1 fully connected layer and 1 Softmax layer, which is used to output the classification results of smart power terminal data.
[0144] In this embodiment, the fully connected layer focuses on the learning task and the specific features of the smart power terminal data, so the parameters are updated during the model training process. Softmax is used as the classification function, and its calculation formula is:
[0145] (6)
[0146] in, Represents an edge node The power consumption data of the smart power terminal input on is the total amount of electricity consumption data during training, Represents the output result of one round of network forward propagation, Indicates the data classification result.
[0147] During the training of the local model, the parameters of the feature extraction layer and the data alignment layer are frozen. Considering that directly transferring parameters will result in a large amount of communication overhead, this chapter can appropriately reduce the parameters of the first two parts during the migration process. To this end, the model compression process is added to reduce the amount of model parameter transmission during the migration process. The parameters of the output layer are not migrated, but are fine-tuned and trained using the target domain's own dataset.
[0148] In this embodiment, reference Figure 3 ,In the process of federated transfer learning, the model compression algorithm based on the pruning algorithm reduces the transmission amount of model parameters, as follows:
[0149] The feature extraction layer structure is Conv+BN+Relu. After the model training, the Conv layer usually contains two parameters: weight and bias, while the BN layer contains four parameters: weight, bias, mean, and variance. The activation function only works during the training process and has no parameters after training, as shown in the following table:
[0150] Table 1 Parameters
[0151]
[0152] Assume that the weight obtained by Conv layer training is , the bias is ; The weight obtained by BN layer training is , the bias is , the mean is , the variance is , assuming that in this structure, the data input is , the intermediate output after the Conv layer , and then after the BN layer and the Relu layer, the output is , then under normal circumstances, the calculation formula after the convolution layer is:
[0153] (7)
[0154] The calculation formula after the BN layer is:
[0155] (8)
[0156] Will Substituting into the above formula, we get:
[0157] (9)
[0158] Simplified:
[0159] (10)
[0160] From the structure of the above formula, we can see that its form is the same as that of a single convolutional layer, so the new weights can be defined according to the combined formula and bias :
[0161] (11)
[0162] Based on the above formula, the original Conv+BN+Relu structure is converted to the Conv structure.
[0163] In this embodiment, by introducing parameters, the calculation of is converted into the calculation of, and the time-consuming part in the formula can be optimized. The equation is established according to the following mathematical theorem:
[0164] If a polynomial equation modulo a given prime number has a single root, then this root can be used to find the roots of the equation in higher powers of the modulo.
[0165] Based on the above transformation relationship, the modular exponentiation operation in homomorphic encryption is optimized:
[0166] Using Carmichael's theorem to add two privacy parameters and , the definitions of these two parameters are as follows:
[0167] (13)
[0168] (14)
[0169] in ,and , the key generation algorithm is as follows:
[0170] Table 2 Key generation algorithm
[0171]
[0172] Place The range is , depending on the performance of the operator, there is a high probability that a positive integer that meets the conditions can be found .when and are mutually prime, or satisfy When, return , and store it as the private key. Otherwise, the traversal continues to look for modular exponentiation optimization:
[0173] First define ,make:
[0174] (15)
[0175] because , then
[0176] (16)
[0177] The value of T can be given by Finally, we calculate The value of can be Calculated.
[0178] By pre-calculating , and save it as one of the private keys.
[0179] In this embodiment, the accuracy of this solution is verified:
[0180] First of all Expanding it, we get:
[0181] (17)
[0182] When calculating When the value is stored in advance ,because can be converted to:
[0183] (18)
[0184] Since it is a modulus calculation, Must be less than , so we can directly get .set up , then
[0185] (19)
[0186] After conversion, we can get
[0187] (20)
[0188] because can be precomputed, and
[0189] (twenty one)
[0190] Can be directly obtained .
[0191] In this embodiment, in order to verify the improvement of algorithm performance by eliminating modular exponentiation, the running time of the algorithm is measured by using the single precision multiplication (SPM) in the CIOS algorithm. The number of words long and The operation time is considered to be SPM, then two The modulus is also When the word length is Therefore, for ,in Both Bytes long, for Bit length, if calculated using CIOS algorithm, the average required Next SPM.
[0192] by For example, the modulus Considered to be Therefore, in traditional homomorphic encryption, the time required for the decryption algorithm is times SPM, of which for The bit length of When is large enough, the formula can be simplified to:
[0193] (6-22)
[0194] For the algorithm proposed in this invention, for The bit length of for The bit length of the modulus Still considered to be The modulus Considered to be The required time of the algorithm proposed by the present invention can be expressed as:
[0195] (6-23)
[0196] The formula can be simplified to:
[0197] (6-24)
[0198] It can be obtained that when Less than ,Right now The present invention can save running time.
[0199] Preferably, in this embodiment, the local model and the global model can be continuously updated for the new user data that continuously appears on the smart power terminal device, including the following steps:
[0200] Enter the Electricity Privacy Dataset , cloud servers use public datasets Training CNN global model ; The key generation center distributes public and private keys. The cloud server will Distribute to edge nodes;
[0201] The edge nodes train local models based on the following formula :
[0202] ;
[0203] in, represents the parameters to be learned of the local deep neural network, represents the local model to be learned, Represents an edge node The total number of samples in the training data, represents the loss function, Represents an edge node Sample data for training;
[0204] The edge nodes will Divide into , and ;
[0205] right Model compression is performed;
[0206] A lightweight homomorphic encryption algorithm based on modular exponentiation optimization and The parameters are encrypted;
[0207] Perform model aggregation according to the following formula and update the global model ;
[0208] ;
[0209] in, is the weight of the neural network, s is the number of local models;
[0210] The cloud server will update the global model Distribute to edge nodes;
[0211] Transfer learning is performed according to the following formula to obtain the power privacy data classification model :
[0212]
[0213] in, It is a parameter that balances the classification task and the MMD ratio; is the total loss, Represents an edge node The power consumption data of the smart power terminal input on is the total amount of electricity consumption data during training, Represents the output result of one round of network forward propagation, Indicates the data classification result.
[0214] In this embodiment, a secure and lightweight data interaction method based on modular exponentiation optimization is also provided, including:
[0215] First, the edge node uses the public dataset from the cloud server for training the global model as the source domain, and the private data from the edge node for training the local model as the target domain;
[0216] The basic neural network used to train the global model and the local model is divided into feature extraction layers. , Data alignment layer And the resulting output layer ;
[0217] Feature extraction layer: This layer consists of 2 convolutional layers and 2 normalization layers, which is used for global feature extraction;
[0218] Data alignment layer: This layer consists of a fully connected layer FC, which uses the maximum mean variance kernel learning method, expressed as:
[0219] (4)
[0220] in, represents the Frobenius norm of the matrix, represents the total number of samples of cloud training data, Represents an edge node The total number of samples in the training data, Represents a nonlinear mapping function; the larger the MMD, the greater the difference between the source domain and the target domain; using the source domain data and the target domain data to calculate the MMD, the loss function of the local model can be expressed as:
[0221] (5)
[0222] in, is a parameter that balances the classification task and the MMD ratio; the goal is to minimize the total loss;
[0223] Result output layer: This layer consists of 1 fully connected layer and 1 Softmax layer, which is used to output the classification results of smart power terminal data.
[0224] Furthermore, the server is the top layer, connected to the edge nodes in all regions, and builds models and shares parameters with the edge nodes through federated learning. The federated learning includes two parts: global model and local model learning, as follows:
[0225] Training data on the cloud and terminal data For training, the model learning objective of the cloud server is expressed as:
[0226] (1)
[0227] in, represents the parameters to be learned of the global deep neural network, represents the cloud model to be learned, represents the total number of samples of cloud training data, represents the loss function, Represents sample data for cloud training;
[0228] When the global model After the training is completed, the cloud server sends the trained model to each edge node;
[0229] When the global model training is completed, the cloud server sends the trained model to each edge node;
[0230] Edge Node The learning objectives are:
[0231] (2)
[0232] in, represents the parameters to be learned of the local deep neural network, represents the local model to be learned, Represents an edge node The total number of samples in the training data, represents the loss function, Represents an edge node Sample data for training;
[0233] When all local models After convergence, its model parameters will be uploaded to the cloud server for aggregation; the federated averaging algorithm FeDAvg algorithm is used to align local models;
[0234] In each round of training, The average value of the local models is used to update the global model. The updated global model It can be expressed as:
[0235] (3)
[0236] in, The weights of the neural network are iterated, and the global and local model parameter update and learning process are iterated to finally obtain a personalized power privacy data classification model.
[0237] Furthermore, the fully connected layer focuses on the learning task and the specific features of the smart power terminal data, so the parameters are updated during the model training process. Softmax is used as the classification function, and its calculation formula is:
[0238] (6)
[0239] in, Represents an edge node The power consumption data of the smart power terminal input on is the total amount of electricity consumption data during training, Represents the output result of one round of network forward propagation, Indicates the data classification result.
[0240] Furthermore, in the process of federated transfer learning, a model compression algorithm based on a pruning algorithm is used to reduce the amount of model parameter transmission, as follows:
[0241] The feature extraction layer structure is Conv+BN+Relu. After the model training, the Conv layer usually contains two parameters: weight and bias, while the BN layer contains four parameters: weight, bias, mean, and variance. The activation function only works during the training process and has no parameters after training.
[0242] Assume that the weight obtained by Conv layer training is , the bias is ; The weight obtained by BN layer training is , the bias is , the mean is , the variance is , assuming that in this structure, the data input is , the intermediate output after the Conv layer , and then after the BN layer and the Relu layer, the output is , then under normal circumstances, the calculation formula after the convolution layer is:
[0243] (7)
[0244] The calculation formula after the BN layer is:
[0245] (8)
[0246] Will Substituting into the above formula, we get:
[0247] (9)
[0248] Simplified:
[0249] (10)
[0250] From the structure of the above formula, we can see that its form is the same as that of a single convolutional layer, so the new weights can be defined according to the combined formula and bias :
[0251] (11)
[0252] Based on the above formula, the original Conv+BN+Relu structure is converted to the Conv structure.
[0253] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0254] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0255] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0256] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0257] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.
Claims
1. A secure and lightweight data interaction system based on modular exponentiation optimization, characterized in that: include: Smart power terminals, used to collect power-related data; The cloud server is responsible for collecting the model parameter data transmitted by the edge nodes, aggregating the model parameter data according to the security aggregation algorithm, decrypting the encrypted data in the learning process, and calculating the new model parameter data for updating the global model; The edge node obtains the edge-side local model from the cloud server and trains the local model using the data obtained by the smart power terminal; the edge node encrypts the trained model parameters and sends them to the cloud server; the edge node trains the power privacy data classification model based on the global model, local model and power-related data updated by the cloud server, and in the training process of the power privacy data classification model, uses the idea of transfer learning to perform personalized model training; The cloud server is the top layer, connected to the edge nodes in all regions, and builds models and shares parameters with the edge nodes through federated learning. The federated learning includes two parts: global model and local model learning, as follows: Training data on the cloud and terminal data For training, the model learning objective of the cloud server is expressed as: (1) in, represents the parameters to be learned of the global deep neural network, represents the cloud model to be learned, represents the total number of samples of cloud training data, represents the loss function, Represents sample data for cloud training; When the global model After the training is completed, the cloud server sends the trained model to each edge node; When the global model training is completed, the cloud server sends the trained model to each edge node; Edge Node The learning objectives are: (2) in, represents the parameters to be learned of the local deep neural network, represents the local model to be learned, Represents an edge node The total number of samples in the training data, represents the loss function, Represents an edge node Sample data for training; When all local models After convergence, its model parameters will be uploaded to the cloud server for aggregation; the federated averaging algorithm FeDAvg algorithm is used to align local models; In each round of training, The average value of the local models is used to update the global model. The updated global model It can be expressed as: (3) in, The weights of the neural network are calculated, and the global and local model parameter update and learning processes are iterated to finally obtain a personalized power privacy data classification model.
2. The secure and lightweight data interaction system based on modular exponentiation optimization according to claim 1 is characterized in that: The edge node is specifically used for: First, the public dataset used to train the global model on the cloud server is used as the source domain, and the private data used to train the local model on the edge node is used as the target domain; The basic neural network used to train the global model and the local model is divided into feature extraction layers. , Data alignment layer And the resulting output layer ; Feature extraction layer: This layer consists of 2 convolutional layers and 2 normalization layers, which is used for global feature extraction; Data alignment layer: This layer consists of a fully connected layer FC, which uses the maximum mean variance kernel learning method, expressed as: (4) in, represents the Frobenius norm of the matrix, represents the total number of samples of cloud training data, Represents an edge node The total number of samples in the training data, Represents a nonlinear mapping function; the larger the MMD, the greater the difference between the source domain and the target domain; using the source domain data and the target domain data to calculate the MMD, the loss function of the local model can be expressed as: (5) in, is a parameter that balances the classification task and the MMD ratio; the goal is to minimize the total loss; Result output layer: This layer consists of 1 fully connected layer and 1 Softmax layer, which is used to output the classification results of smart power terminal data.
3. The secure and lightweight data interaction system based on modular exponentiation optimization according to claim 2 is characterized in that: The fully connected layer focuses on the learning task and the specific features of the smart power terminal data, so the parameters are updated during the model training process. Softmax is used as the classification function, and its calculation formula is: (6) in, Represents an edge node The power consumption data of the smart power terminal input on is the total amount of electricity consumption data during training, Represents the output result of one round of network forward propagation, Indicates the data classification result.
4. The secure and lightweight data interaction system based on modular exponentiation optimization according to claim 2 is characterized in that: In the process of federated transfer learning, the model compression algorithm based on the pruning algorithm reduces the amount of model parameter transmission, as follows: The feature extraction layer structure is Conv+BN+Relu. After the model training, the Conv layer usually contains two parameters: weight and bias, while the BN layer contains four parameters: weight, bias, mean, and variance. The activation function only works during the training process and has no parameters after training. Assume that the weight obtained by Conv layer training is , the bias is ; The weight obtained by BN layer training is , the bias is , the mean is , the variance is , assuming that in this structure, the data input is , the intermediate output after the Conv layer , and then after the BN layer and the Relu layer, the output is , then under normal circumstances, the calculation formula after the convolution layer is: (7) The calculation formula after the BN layer is: (8) Will Substituting into the above formula, we get: (9) Simplified: (10) From the structure of the above formula, we can see that its form is the same as that of a single convolutional layer, so the new weights can be defined according to the combined formula and bias : (11) Based on the above formula, the original Conv+BN+Relu structure is converted to the Conv structure.
5. The secure and lightweight data interaction system based on modular exponentiation optimization according to claim 1 is characterized in that: There is also a key generation center, which is responsible for generating key pairs used in the lightweight homomorphic encryption process and distributing them to the corresponding edge nodes or cloud servers. The key generation center generates keys based on the lightweight homomorphic encryption algorithm optimized by modular exponentiation, as follows: Using Carmichael's theorem to add two privacy parameters and , the definitions of these two parameters are as follows: (13) (14) in ,and ; Initialize a random prime number ,make ,calculate ,when and are mutually prime, or satisfy When, return ,otherwise , traverse and continue searching; Modular exponentiation optimization: First define ,make: (15) because , then (16) The value of T is given by Finally, we calculate The value of Calculated; By pre-calculating the settings , and save it as one of the private keys.
6. A secure and lightweight data interaction method based on modular exponentiation optimization, characterized in that: include: First, the edge node uses the public dataset from the cloud server for training the global model as the source domain, and the private data from the edge node for training the local model as the target domain; The basic neural network used to train the global model and the local model is divided into feature extraction layers. , Data alignment layer And the resulting output layer ; Feature extraction layer: This layer consists of 2 convolutional layers and 2 normalization layers, which is used for global feature extraction; Data alignment layer: This layer consists of a fully connected layer FC, which uses the maximum mean variance kernel learning method, expressed as: (4) in, represents the Frobenius norm of the matrix, represents the total number of samples of cloud training data, Represents an edge node The total number of samples in the training data, Represents a nonlinear mapping function; the larger the MMD, the greater the difference between the source domain and the target domain; using the source domain data and the target domain data to calculate the MMD, the loss function of the local model can be expressed as: (5) in, is a parameter that balances the classification task and the MMD ratio; the goal is to minimize the total loss; Result output layer: This layer consists of 1 fully connected layer and 1 Softmax layer, which is used to output the classification results of smart power terminal data; The cloud server is the top layer, connected to the edge nodes in all regions, and builds models and shares parameters with the edge nodes through federated learning. The federated learning includes two parts: global model and local model learning, as follows: Training data on the cloud and terminal data For training, the model learning objective of the cloud server is expressed as: (1) in, represents the parameters to be learned of the global deep neural network, represents the cloud model to be learned, represents the total number of samples of cloud training data, represents the loss function, Represents sample data for cloud training; When the global model After the training is completed, the cloud server sends the trained model to each edge node; When the global model training is completed, the cloud server sends the trained model to each edge node; Edge Node The learning objectives are: (2) in, represents the parameters to be learned of the local deep neural network, represents the local model to be learned, Represents an edge node The total number of samples in the training data, represents the loss function, Represents an edge node Sample data for training; When all local models After convergence, its model parameters will be uploaded to the cloud server for aggregation; the federated averaging algorithm FeDAvg algorithm is used to align local models; In each round of training, The average value of the local models is used to update the global model. The updated global model It can be expressed as: (3) in, The weights of the neural network are iterated, and the global and local model parameter update and learning process are iterated to finally obtain a personalized power privacy data classification model.
7. The secure and lightweight data interaction method based on modular exponentiation optimization according to claim 6 is characterized in that: The fully connected layer focuses on the learning task and the specific features of the smart power terminal data, so the parameters are updated during the model training process. Softmax is used as the classification function, and its calculation formula is: (6) in, Represents an edge node The power consumption data of the smart power terminal input on is the total amount of electricity consumption data during training, Represents the output result of one round of network forward propagation, Indicates the data classification result.
8. The secure and lightweight data interaction method based on modular exponentiation optimization according to claim 7 is characterized in that: In the process of federated transfer learning, the model compression algorithm based on the pruning algorithm reduces the amount of model parameter transmission, as follows: The feature extraction layer structure is Conv+BN+Relu. After the model training, the Conv layer usually contains two parameters: weight and bias, while the BN layer contains four parameters: weight, bias, mean, and variance. The activation function only works during the training process and has no parameters after training. Assume that the weight obtained by Conv layer training is , the bias is ; The weight obtained by BN layer training is , the bias is , the mean is , the variance is , assuming that in this structure, the data input is , the intermediate output after the Conv layer , and then after the BN layer and the Relu layer, the output is , then under normal circumstances, the calculation formula after the convolution layer is: (7) The calculation formula after the BN layer is: (8) Will Substituting into the above formula, we get: (9) Simplified: (10) From the structure of the above formula, we can see that its form is the same as that of a single convolutional layer, so the new weights can be defined according to the combined formula and bias : (11) Based on the above formula, the original Conv+BN+Relu structure is converted to the Conv structure.
