A federated learning optimization method based on privacy protection of neural network models

Abstract

The present invention discloses a federated learning optimization method based on privacy protection of a neural network model, including S1, deploying distributed nodes, each node containing a partial data set; S2, applying differential privacy technology to pre-process data on each node to ensure the security of data in the local processing process; S3, utilizing a secure multi-party computing mechanism to securely aggregate model parameters between nodes; S4, optimizing the gradient descent process through an improved gradient aggregation strategy; S5, using an adaptive algorithm to adjust the training parameters and learning rate of each node; S6, implementing an efficient data encoding and transmission strategy to reduce the amount of data exchange between nodes; S7, applying an adaptive learning mechanism based on data and environmental feedback to automatically adjust the learning strategy and resource allocation. The present invention has the advantages of optimized data privacy protection, improved learning efficiency, reduced communication costs, and enhanced model adaptability.

Description

Technical Field

[0001] The present invention relates to the field of privacy protection technology, and in particular to a federated learning optimization method based on privacy protection of a neural network model. Background Art

[0002] With the rapid development of information technology and data science, federated learning, as an emerging distributed machine learning method, allows multiple nodes to jointly train models without exchanging their original data, thus solving the key issues of data privacy and security. However, in the process of implementing federated learning, data privacy protection, computational efficiency, and optimization of model performance still face severe challenges. Especially in a multi-node environment, the dispersion and heterogeneity of data make the model training process complex and inefficient, and it is difficult to ensure the privacy and security of data at each node.

[0003] In existing technologies, although federated learning can avoid the privacy risks brought by centralized data storage to a certain extent, there are still potential security vulnerabilities in the process of data transmission and processing. In addition, existing technologies show obvious deficiencies in the following aspects:

[0004] 1. Insufficient privacy protection mechanism: Although federated learning does not require direct data sharing, sensitive information may still be leaked during the exchange of model parameters and update information. Existing privacy protection technologies such as differential privacy and secure multi-party computing provide certain security guarantees, but in practical applications they often lead to excessive consumption of computing resources or impaired model performance.

[0005] 2. Inefficiency of federated learning algorithms: Existing federated learning algorithms have failed to effectively solve the problems of low learning efficiency and slow model convergence caused by uneven data distribution. In addition, the communication cost between nodes is high, especially in large-scale networks, and the network congestion and delay problems caused by frequent data exchange have not been effectively solved.

[0006] 3. Unreasonable resource allocation and scheduling: In a multi-node federated learning environment, how to reasonably allocate resources based on the data characteristics and computing power of each node is the key to improving overall training efficiency. Existing methods are difficult to achieve optimal resource allocation in a dynamic environment, resulting in resource waste and computing bottlenecks.

[0007] 4. Insufficient model performance and versatility: It is difficult for existing federated learning models to balance privacy protection and model performance at the same time, and they lack the necessary adaptability when facing new data distribution or changes, which affects the model's generalization ability and practical application value.

[0008] Therefore, how to provide a federated learning optimization method based on privacy protection of neural network models is an urgent problem that technicians in this field need to solve. Summary of the invention

[0009] One object of the present invention is to propose a method for optimizing federated learning based on privacy protection of a neural network model. The present invention describes in detail a method for optimizing the federated learning process under the premise of protecting data privacy by combining advanced privacy protection technologies such as differential privacy and secure multi-party computing (SMC). This method introduces an improved gradient aggregation strategy, optimizes the federated learning algorithm, and improves the training efficiency and convergence speed of the model in a distributed environment. At the same time, through efficient communication strategies and adaptive learning mechanisms, the amount of data exchange between nodes is reduced, and the learning strategy and resource allocation are dynamically adjusted, thereby speeding up the overall training process. The present invention has the advantages of optimized data privacy protection, improved learning efficiency, reduced communication costs, and enhanced model adaptability.

[0010] A federated learning optimization method based on privacy protection of a neural network model according to an embodiment of the present invention is characterized by comprising the following steps:

[0011] S1. Deploy distributed nodes, each of which contains part of the data set, and the information in the data set does not leave the original environment;

[0012] S2, apply differential privacy technology to pre-process the data at each node to make the data safe during local processing and blur the sensitive features in the data set by adding noise;

[0013] S3. Use secure multi-party computing mechanism to securely aggregate model parameters between nodes, so that no data information of any single node is leaked during model training;

[0014] S4. Through the improved gradient aggregation strategy, the gradient descent process is optimized to reduce the deviation and performance degradation problems caused by uneven data distribution in model training;

[0015] S5, using adaptive algorithms to adjust the training parameters and learning rate of each node, dynamically optimizing the training process based on the computing power of the node and data distribution;

[0016] S6. Implement efficient data encoding and transmission strategies to reduce the amount of data exchange between nodes, reduce communication overhead, and speed up the overall training of the model;

[0017] S7. Apply adaptive learning mechanisms based on data and environmental feedback to automatically adjust learning strategies and resource allocation.

[0018] Optionally, the S1 specifically includes:

[0019] S11. Select a distributed computing environment, calculate the optimal layout of node deployment based on the number and geographical location of devices participating in federated learning, and determine the node layout using the objective function of minimizing network latency and data transmission cost:

[0020]

[0021] Among them, n represents the total number of nodes, d ij is the physical distance between node i and node j, c ij is the data transmission cost between node i and node j;

[0022] S12. Configure the hardware and software resources of each node so that each node has sufficient resources to process the local dataset and participate in federated learning:

[0023] r k =β·s k +γ·p k ;

[0024] Among them, r k represents the total amount of resources of node k, s k is the storage capacity, p k is the processing capacity, β and γ are adjustment factors;

[0025] S13. Deploy privacy protection algorithms on each node, including differential privacy and secure multi-party computing protocols, so that each node can complete data preprocessing and model training tasks while protecting data privacy:

[0026]

[0027] Among them, ∈ ij represents the risk measure of privacy leakage between nodes i and j, e ij is the amount of information exchanged between two nodes, L is the privacy protection level, and δ is the privacy protection strength coefficient;

[0028] S14. Use TLS / SSL encryption technology to establish a secure communication protocol between nodes:

[0029]

[0030] Among them, σ represents the overall security of the system, α i is the encryption strength of node i, c i is the communication frequency of node i;

[0031] S15. Implement a data synchronization mechanism to enable data updates on each node to be synchronized in real time during the federated learning process:

[0032] T sync =maxi≠j |t i -t j |;

[0033] Among them, T sync Indicates the maximum time difference of data synchronization, t i and t j is the data update timestamp of nodes i and j;

[0034] S16. Perform initial data cleaning and format standardization on the data set of each node:

[0035]

[0036] Among them, Q norm is the average normalized score of data quality in the whole system, q i is the score of the data quality of node i.

[0037] Optionally, the S2 specifically includes:

[0038] S21. Implement differential privacy technology on each distributed node to protect the privacy of data by adding a specific amount of noise to the data set of each node:

[0039] Noise(μ,σ 2 )=μ+σ×N(0,1);

[0040] Where μ is the mean value of the noise, σ is the standard deviation of the noise, and N(0,1) represents the standard normal distribution;

[0041] S22. Calculate the privacy budget in each data set to control the balance between the strength of privacy protection and data practicality:

[0042]

[0043] Among them, ∈ total represents the total privacy budget, k is the number of operations involved in the dataset, ∈ i is the privacy budget of the ith operation;

[0044] S23. Set privacy parameters for each data operation:

[0045]

[0046] Among them, δ i is the privacy protection ratio of the ith operation, which quantifies the weight of the operation in the overall privacy budget;

[0047] S24. Apply secure multi-party computing mechanism in the data preprocessing stage to achieve secure data aggregation without exposing the specific data content of any node:

[0048]

[0049] Among them, x i is the data of the ith node, Enc(x i ) is the data encrypted using the secure multi-party computing protocol.

[0050] Optionally, the S3 specifically includes:

[0051] S31. In the process of federated learning, the secure multi-party computing mechanism is used to securely aggregate the model parameters between distributed nodes, and homomorphic encryption technology is used to prevent the specific data information of any single node from being leaked during the aggregation process:

[0052]

[0053] Among them, x i represents the model parameters of the i-th node, Enc(x i ) and Dec represent encryption and decryption functions respectively;

[0054] S32. Implement a threshold-based security detection mechanism. At any given point in time, the aggregate model parameters of all participating nodes meet the preset security standards:

[0055]

[0056] Among them, T represents the safety threshold to determine whether the model parameters after aggregation are within the safety range;

[0057] S33. Optimize the weight adjustment strategy used in secure multi-party computing to improve the efficiency and accuracy of the aggregation process, and balance the influence of each node in the aggregation by adjusting the weight of each node:

[0058]

[0059] Among them, w i is the weight of node i, n is the total number of nodes participating in the aggregation, σ i is the data quality standard deviation of node i, ∈ is a small constant to prevent division by zero errors;

[0060] S34. After data aggregation is completed, use the feedback adjustment mechanism based on model performance to dynamically adjust the parameters and strategies of subsequent aggregation:

[0061] FeedbackAdjustment(m)=m+α·(target-performance(m));

[0062] Among them, m represents the current aggregated model, α is the learning rate, target is the target performance indicator, and performance(m) is the performance evaluation of the current model.

[0063] Optionally, the S4 specifically includes:

[0064] S41. In the federated learning environment, the gradient descent optimization is implemented through an improved gradient aggregation strategy. The calculation and aggregation of gradients take into account the distribution characteristics of the data and the computing power of the nodes:

[0065]

[0066] Where n is the total number of nodes involved in the calculation, w i is the weight determined by the amount and quality of data of node i, g i is the gradient calculated by node i;

[0067] S42. Implement gradient clipping technology to limit the size of the gradient transmitted by each node and reduce the instability of model training caused by extreme gradient values:

[0068] g i '=Clip(g i ,τ)=min(max(g i ,-τ),τ);

[0069] Among them, g i is the original gradient, τ is the clipping threshold, g i ' is the clipped gradient;

[0070] S43, using an adaptive learning rate adjustment mechanism to optimize the learning rate during gradient descent, automatically adjusting the learning rate according to the current stage of model training and the data characteristics of the node:

[0071]

[0072] Among them, η t is the learning rate at the tth iteration, η 0 is the initial learning rate, δ is the decay rate, and t is the number of iterations;

[0073] S44. After each iteration, the global model is updated based on the aggregated gradients returned by each node to speed up model convergence and improve training efficiency:

[0074] θ t+1 =θ t -η t ·Gradient agg ;

[0075] Among them, θ t and θt+1 Represent the values ​​of the model parameters before and after the tth and t+1th iterations, respectively;

[0076] S45. Periodically evaluate parameter updates during model training to monitor model performance improvements and detect potential overfitting issues:

[0077]

[0078] ΔE val =E val (t)-E val (t-1);

[0079] Among them, E val (t) is the average validation error of the t-th evaluation cycle, L is the loss function, θ t is the model parameter of the current period, x j and j They are the features and labels in the validation set, m is the number of samples in the validation set, and the error change ΔE val to determine the degree of improvement or overfitting risk of the model, where t-1 and t represent consecutive evaluation cycles.

[0080] Optionally, the S5 specifically includes:

[0081] S51, using an adaptive algorithm to adjust the training parameters of each node, dynamically adjusting the step size of gradient descent and the model update frequency according to the computing power and data characteristics of each node:

[0082] λ i =λ 0 ·exp(-κ·Var(D i ));

[0083] Among them, λ i is the adjusted learning rate of node i, λ 0 is the basic learning rate, κ is the adjustment sensitivity parameter, Var(D i ) is the variance of the data set of node i;

[0084] S52. Implement parameter optimization based on node performance feedback to optimize the learning efficiency and model performance of the entire network:

[0085] θ i,new =θ i,old +γ·(Perf i -Target);

[0086] Among them, θ i,new and θ i,old They represent the model parameters of node i before and after adjustment, γ is the learning rate, and Perfi is the current performance of node i, Target is the performance target to be achieved;

[0087] S53. Automatically adjust the amount of data involved in model training according to the computing power of the node to balance the load of data processing and model training tasks:

[0088] N i =C i ·R;

[0089] Among them, N i is the number of data samples assigned to node i, C i is the computing power index of node i, and R is the proportional factor of the total amount of global data samples;

[0090] S54. Apply resource allocation algorithms to optimize the allocation of resources and training tasks between nodes to minimize overall training time and cost:

[0091]

[0092] Among them, S opt is the optimized system efficiency, T i is the time required for node i to complete the training task, C i is the running cost of the node;

[0093] S55. Regularly evaluate and adjust resource allocation and parameter settings during federated learning, and optimize the system based on real-time performance feedback and data distribution changes:

[0094]

[0095] Among them, Adjust system (t) represents the system adjustment amount in the tth time period, P t is the parameter setting of the previous cycle, α is the adjustment coefficient, E i,t is the performance evaluation value of the i-th node in the t-th cycle, n is the total number of nodes, E target is the target performance threshold.

[0096] Optionally, the S6 specifically includes:

[0097] S61. Implement efficient data encoding and transmission strategies to reduce the amount of data transmitted between participating nodes and reduce communication costs and time:

[0098]

[0099] Among them, S o Indicates the original data size, S e Indicates the size of the encoded data, Efficiencyencode Indicates the efficiency of the encoding;

[0100] S62. Optimize data transmission protocols to enable fast and secure data transmission in the network:

[0101]

[0102] Where D is the amount of data to be transmitted, T is the time required for data transmission, and Rate transfer Indicates the data transmission rate;

[0103] S63. Deploy data compression mechanisms to reduce network congestion and increased transmission costs caused by large amounts of data:

[0104]

[0105] Among them, U is the size of uncompressed data, C is the size of compressed data, and CompressionRatio represents the compression ratio;

[0106] S64. Use distributed data storage technology to reduce the load on a single node and improve data reliability:

[0107]

[0108] Among them, L max , L min and L avg They represent the maximum, minimum, and average data loads of all nodes respectively, and Balance represents the balance of data distribution;

[0109] S65. Keep all nodes synchronized in terms of data integrity and consistency by implementing data synchronization protocols:

[0110]

[0111] Among them, N s is the number of nodes that have synchronized data, N t is the total number of nodes, and SyncEfficiency indicates the efficiency of data synchronization.

[0112] Optionally, the S7 specifically includes:

[0113] S71. Implement an adaptive learning mechanism based on data or environmental feedback to dynamically adjust learning strategies and resource allocation by analyzing the performance feedback of each node in real time:

[0114] Adjust learn (i) = η i ·(1+β·(Perf current (i)-Perftarget ));

[0115] Among them, η i is the initial learning rate of node i, β is the adjustment coefficient, and Perf current (i) is the current performance score of node i, Perf target is the predetermined target performance level;

[0116] S72. Automatically adjust algorithm parameters according to the data characteristics and computing power of each node to optimize the training efficiency and effect of the model:

[0117] θ new (i) = θ old (i)+α·Δθ(i);

[0118] Among them, θ new (i) and θ old (i) are the parameters of node i before and after adjustment, α is the learning rate, Δθ(i) is the parameter update amount calculated based on performance feedback;

[0119] S73. Apply environmental monitoring and data analysis tools to evaluate the load and data processing capacity of each node and dynamically adjust resource allocation at the system level:

[0120]

[0121] Among them, R alloc (i) is the amount of resources allocated to node i, C i is the computing power score of node i, R total is the total system resources, n is the total number of nodes;

[0122] S74. Implement a dynamic adjustment algorithm to adjust the data exchange and processing strategy during the learning process based on the real-time monitored network status and node feedback:

[0123] Policy dynamic (i) = γ·Load(i) + δ·Data rate (i);

[0124] Among them, Policy dynamic (i) represents the dynamic strategy adjustment value of node i, γ and δ are weight factors, Load(i) is the current load of node i, Data rate (i) is the data processing rate of node i;

[0125] S75. Regularly evaluate the performance of the federated learning network to ensure that all adjustments meet the optimization goals of efficiency and effectiveness:

[0126]

[0127] Among them, Performance eval is the overall performance evaluation value of the network, E i is the efficiency score of node i, and n is the total number of nodes.

[0128] The beneficial effects of the present invention are:

[0129] (1) The present invention provides a federated learning method that protects data privacy and maintains data transmission and processing security by combining differential privacy technology and secure multi-party computing (SMC). This method can effectively prevent any sensitive information leakage during the learning process and ensure data security even when exchanging model updates between participating nodes. Through this combination, the present invention greatly reduces the potential risks of privacy infringement and data leakage and enhances the security protection capabilities of the entire system.

[0130] (2) The present invention not only optimizes the training efficiency of the model in a distributed environment, but also improves the convergence speed of the model through an improved gradient aggregation strategy and adaptive learning mechanism. This optimization reduces the performance degradation problem caused by uneven data distribution, enables the model to adapt to various data environments more quickly, and improves the efficiency of the training process and the accuracy of the model.

[0131] (3) The present invention significantly reduces the amount of data that needs to be exchanged between nodes by implementing efficient data encoding and transmission strategies, thereby reducing communication costs and alleviating network congestion. In addition, the optimized data exchange protocol and dynamic resource allocation further speed up data processing and model updating, making the entire federated learning process more efficient.

[0132] (4) The present invention dynamically adjusts learning parameters and resource allocation by real-time monitoring and evaluation of the performance and load of each node. This not only enables the system to be optimized according to the actual data distribution and node capabilities, but also ensures optimal performance under different operating environments, improving the efficiency and economy of resource use.

[0133] (5) The present invention ensures that the federated learning model is continuously optimized in the long run and adapts to new data environments and challenges through regular system performance evaluation and continuous parameter adjustment. This continuous optimization not only improves the generalization ability of the model, but also lays the foundation for coping with possible changes and challenges in the future. BRIEF DESCRIPTION OF THE DRAWINGS

[0134] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0135] Figure 1This is an overall flow chart of a federated learning optimization method based on privacy protection of a neural network model proposed by the present invention;

[0136] Figure 2 A schematic diagram of the combined use of differential privacy and secure multi-party computing for a federated learning optimization method based on privacy protection of a neural network model proposed by the present invention;

[0137] Figure 3 A schematic diagram of an improved gradient aggregation strategy for a federated learning optimization method based on privacy protection of a neural network model proposed in the present invention; DETAILED DESCRIPTION

[0138] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0139] refer to Figure 1-Figure 3 A federated learning optimization method based on privacy protection of a neural network model is characterized by comprising the following steps:

[0140] S1. Deploy distributed nodes, each of which contains part of the data set, and the information in the data set does not leave the original environment;

[0141] In this implementation, S1 specifically includes:

[0142] S11. Select a distributed computing environment, calculate the optimal layout of node deployment based on the number and geographical location of devices participating in federated learning, and determine the node layout using the objective function of minimizing network latency and data transmission cost:

[0143]

[0144] Among them, n represents the total number of nodes, d ij is the physical distance between node i and node j, c ij is the data transmission cost between node i and node j;

[0145] S12. Configure the hardware and software resources of each node so that each node has sufficient resources to process the local dataset and participate in federated learning:

[0146] r k =β·s k +γ·p k ;

[0147] Among them, r k represents the total amount of resources of node k, s k is the storage capacity, p k is the processing capacity, β and γ are adjustment factors;

[0148] S13. Deploy privacy protection algorithms on each node, including differential privacy and secure multi-party computing protocols, so that each node can complete data preprocessing and model training tasks while protecting data privacy:

[0149]

[0150] Among them, ∈ ij represents the risk measure of privacy leakage between nodes i and j, e ij is the amount of information exchanged between two nodes, L is the privacy protection level, and δ is the privacy protection strength coefficient;

[0151] S14. Use TLS / SSL encryption technology to establish a secure communication protocol between nodes:

[0152]

[0153] Among them, σ represents the overall security of the system, α i is the encryption strength of node i, c i is the communication frequency of node i;

[0154] S15. Implement a data synchronization mechanism to enable data updates on each node to be synchronized in real time during the federated learning process:

[0155] T sync =max i≠j |t i -t j |;

[0156] Among them, T sync Indicates the maximum time difference of data synchronization, t i and t j is the data update timestamp of nodes i and j;

[0157] S16. Perform initial data cleaning and format standardization on the data set of each node:

[0158]

[0159] Among them, Q norm is the average normalized score of data quality in the whole system, q i is the score of the data quality of node i.

[0160] S2, apply differential privacy technology to pre-process the data at each node to make the data safe during local processing and blur the sensitive features in the data set by adding noise;

[0161] In this implementation, S2 specifically includes:

[0162] S21. Implement differential privacy technology on each distributed node to protect the privacy of data by adding a specific amount of noise to the data set of each node:

[0163] Noise(μ,σ 2 )=μ+σ×N(0,1);

[0164] Where μ is the mean value of the noise, σ is the standard deviation of the noise, and N(0,1) represents the standard normal distribution;

[0165] S22. Calculate the privacy budget in each data set to control the balance between the strength of privacy protection and data practicality:

[0166]

[0167] Among them, ∈ total represents the total privacy budget, k is the number of operations involved in the dataset, ∈ i is the privacy budget of the ith operation;

[0168] S23. Set privacy parameters for each data operation:

[0169]

[0170] Among them, δ i is the privacy protection ratio of the ith operation, which quantifies the weight of the operation in the overall privacy budget;

[0171] S24. Apply secure multi-party computing mechanism in the data preprocessing stage to achieve secure data aggregation without exposing the specific data content of any node:

[0172]

[0173] Among them, x i is the data of the ith node, Enc(x i ) is the data encrypted using the secure multi-party computing protocol.

[0174] S3. Use secure multi-party computing mechanism to securely aggregate model parameters between nodes, so that no data information of any single node is leaked during model training;

[0175] In this implementation, S3 specifically includes:

[0176] S31. In the process of federated learning, the secure multi-party computing mechanism is used to securely aggregate the model parameters between distributed nodes, and homomorphic encryption technology is used to prevent the specific data information of any single node from being leaked during the aggregation process:

[0177]

[0178] Among them, x i represents the model parameters of the i-th node, Enc(x i ) and Dec represent encryption and decryption functions respectively;

[0179] S32. Implement a threshold-based security detection mechanism. At any given point in time, the aggregate model parameters of all participating nodes meet the preset security standards:

[0180]

[0181] Among them, T represents the safety threshold to determine whether the model parameters after aggregation are within the safety range;

[0182] S33. Optimize the weight adjustment strategy used in secure multi-party computing to improve the efficiency and accuracy of the aggregation process, and balance the influence of each node in the aggregation by adjusting the weight of each node:

[0183]

[0184] Among them, w i is the weight of node i, n is the total number of nodes participating in the aggregation, σ i is the data quality standard deviation of node i, ∈ is a small constant to prevent division by zero errors;

[0185] S34. After data aggregation is completed, use the feedback adjustment mechanism based on model performance to dynamically adjust the parameters and strategies of subsequent aggregation:

[0186] Feedback Adjustment(m)=m+α·(target-performance(m));

[0187] Among them, m represents the current aggregated model, α is the learning rate, target is the target performance indicator, and performance(m) is the performance evaluation of the current model.

[0188] S4. Through the improved gradient aggregation strategy, the gradient descent process is optimized to reduce the deviation and performance degradation problems caused by uneven data distribution in model training;

[0189] In this implementation, S4 specifically includes:

[0190] S41. In the federated learning environment, the gradient descent optimization is implemented through an improved gradient aggregation strategy. The calculation and aggregation of gradients take into account the distribution characteristics of the data and the computing power of the nodes:

[0191]

[0192] Where n is the total number of nodes involved in the calculation, w i is the weight determined by the amount and quality of data of node i, g i is the gradient calculated by node i;

[0193] S42. Implement gradient clipping technology to limit the size of the gradient transmitted by each node and reduce the instability of model training caused by extreme gradient values:

[0194] g i '=Clip(g i ,τ)=min(max(g i ,-τ),τ);

[0195] Among them, g i is the original gradient, τ is the clipping threshold, g i ' is the clipped gradient;

[0196] S43, using an adaptive learning rate adjustment mechanism to optimize the learning rate during gradient descent, automatically adjusting the learning rate according to the current stage of model training and the data characteristics of the node:

[0197]

[0198] Among them, η t is the learning rate at the tth iteration, η 0 is the initial learning rate, δ is the decay rate, and t is the number of iterations;

[0199] S44. After each iteration, the global model is updated based on the aggregated gradients returned by each node to speed up model convergence and improve training efficiency:

[0200] θ t+1 =θ t -η t ·Gradient agg ;

[0201] Among them, θ t and θ t+1 Represent the values ​​of the model parameters before and after the tth and t+1th iterations, respectively;

[0202] S45. Periodically evaluate parameter updates during model training to monitor model performance improvements and detect potential overfitting issues:

[0203]

[0204] ΔE val =E val (t)-E val (t-1);

[0205] Among them, E val (t) is the average validation error of the t-th evaluation cycle, L is the loss function, θ t is the model parameter of the current period, x j and j They are the features and labels in the validation set, m is the number of samples in the validation set, and the error change ΔE val to determine the degree of improvement or overfitting risk of the model, where t-1 and t represent consecutive evaluation cycles.

[0206] S5, using adaptive algorithms to adjust the training parameters and learning rate of each node, dynamically optimizing the training process based on the computing power of the node and data distribution;

[0207] In this implementation, S5 specifically includes:

[0208] S51, using an adaptive algorithm to adjust the training parameters of each node, dynamically adjusting the step size of gradient descent and the model update frequency according to the computing power and data characteristics of each node:

[0209] λ i =λ 0 ·exp(-κ·Var(D i ));

[0210] Among them, λ i is the adjusted learning rate of node i, λ 0 is the basic learning rate, κ is the adjustment sensitivity parameter, Var(D i ) is the variance of the data set of node i;

[0211] S52. Implement parameter optimization based on node performance feedback to optimize the learning efficiency and model performance of the entire network:

[0212] θ i,new =θ i,old +γ·(Perf i -Target);

[0213] Among them, θ i,new and θ i,old They represent the model parameters of node i before and after adjustment, γ is the learning rate, and Perf i is the current performance of node i, Target is the performance target to be achieved;

[0214] S53. Automatically adjust the amount of data involved in model training according to the computing power of the node to balance the load of data processing and model training tasks:

[0215] N i =C i ·R;

[0216] Among them, N i is the number of data samples assigned to node i, C i is the computing power index of node i, and R is the proportional factor of the total amount of global data samples;

[0217] S54. Apply resource allocation algorithms to optimize the allocation of resources and training tasks between nodes to minimize overall training time and cost:

[0218]

[0219] Among them, S opt is the optimized system efficiency, T i is the time required for node i to complete the training task, C i is the running cost of the node;

[0220] S55. Regularly evaluate and adjust resource allocation and parameter settings during federated learning, and optimize the system based on real-time performance feedback and data distribution changes:

[0221]

[0222] Among them, Adjust system (t) represents the system adjustment amount in the tth time period, P t is the parameter setting of the previous cycle, α is the adjustment coefficient, E i,t is the performance evaluation value of the i-th node in the t-th cycle, n is the total number of nodes, E target is the target performance threshold.

[0223] S6. Implement efficient data encoding and transmission strategies to reduce the amount of data exchange between nodes, reduce communication overhead, and speed up the overall training of the model;

[0224] In this implementation, S6 specifically includes:

[0225] S61. Implement efficient data encoding and transmission strategies to reduce the amount of data transmitted between participating nodes and reduce communication costs and time:

[0226]

[0227] Among them, S o Indicates the original data size, S e Indicates the size of the encoded data, Efficiency encode Indicates the efficiency of the encoding;

[0228] S62. Optimize data transmission protocols to enable fast and secure data transmission in the network:

[0229]

[0230] Where D is the amount of data to be transmitted, T is the time required for data transmission, and Rate transfer Indicates the data transmission rate;

[0231] S63. Deploy data compression mechanisms to reduce network congestion and increased transmission costs caused by large amounts of data:

[0232]

[0233] Among them, U is the size of uncompressed data, C is the size of compressed data, and CompressionRatio represents the compression ratio;

[0234] S64. Use distributed data storage technology to reduce the load on a single node and improve data reliability:

[0235]

[0236] Among them, L max , L min and L avg They represent the maximum, minimum, and average data loads of all nodes respectively, and Balance represents the balance of data distribution;

[0237] S65. Keep all nodes synchronized in terms of data integrity and consistency by implementing data synchronization protocols:

[0238]

[0239] Among them, N s is the number of nodes that have synchronized data, N t is the total number of nodes, and SyncEfficiency indicates the efficiency of data synchronization.

[0240] S7. Apply adaptive learning mechanisms based on data and environmental feedback to automatically adjust learning strategies and resource allocation.

[0241] In this implementation, S7 specifically includes:

[0242] S71. Implement an adaptive learning mechanism based on data or environmental feedback to dynamically adjust learning strategies and resource allocation by analyzing the performance feedback of each node in real time:

[0243]

[0244] Among them, η i is the initial learning rate of node i, β is the adjustment coefficient, and Perf current (i) is the current performance score of node i, Perftarget is the predetermined target performance level;

[0245] S72. Automatically adjust algorithm parameters according to the data characteristics and computing power of each node to optimize the training efficiency and effect of the model:

[0246] θ new (i) = θ old (i)+α·Δθ(i);

[0247] Among them, θ new (i) and θ old (i) are the parameters of node i before and after adjustment, α is the learning rate, Δθ(i) is the parameter update amount calculated based on performance feedback;

[0248] S73. Apply environmental monitoring and data analysis tools to evaluate the load and data processing capacity of each node and dynamically adjust resource allocation at the system level:

[0249]

[0250] Among them, R alloc (i) is the amount of resources allocated to node i, C i is the computing power score of node i, R total is the total system resources, n is the total number of nodes;

[0251] S74. Implement a dynamic adjustment algorithm to adjust the data exchange and processing strategy during the learning process based on the real-time monitored network status and node feedback:

[0252] Policy dynamic (i) = γ·Load(i) + δ·Data rate (i);

[0253] Among them, Policy dynamic (i) represents the dynamic strategy adjustment value of node i, γ and δ are weight factors, Load(i) is the current load of node i, Data rate (i0 is the data processing rate of node i;

[0254] S75. Regularly evaluate the performance of the federated learning network to ensure that all adjustments meet the optimization goals of efficiency and effectiveness:

[0255]

[0256] Among them, Performance eval is the overall performance evaluation value of the network, E i is the efficiency score of node i, and n is the total number of nodes.

[0257] Embodiment 1;

[0258] In order to verify the effectiveness and practicality of this invention, we chose an international medical group as the implementation scenario. This medical group is distributed in many countries around the world and processes and stores a large amount of sensitive medical data. Due to the data protection regulations of many countries and the high attention of patients to privacy protection, ensuring data privacy and security is crucial during the processing and analysis process.

[0259] In this scenario, the federated learning optimization method based on privacy protection of neural network models of the present invention has been specifically applied. Specifically, this method allows distributed data nodes in various places to participate in the training of a global neural network model without directly exchanging original data. Each node applies differential privacy technology and secure multi-party computing (SMC) to protect data. While maintaining data privacy, it can also share insights gained from learning and optimize overall medical services and operational efficiency.

[0260] The following is a comparison of specific data before and after the implementation of this technology. The key indicators before and after the implementation are collected and analyzed to demonstrate the beneficial effects of the present invention.

[0261] Table 1 Performance improvement report of medical data processing center

[0262]

[0263]

[0264] It can be seen from this table that after deploying the system of the present invention, the data processing capacity of the medical data processing center has been significantly improved, and the processing speed has been increased from 5TB / hour to 7TB / hour, and more data can be processed every month. The number of privacy protection events has been significantly reduced from 15 to 2, showing the effect of enhanced data privacy and security measures. Real-time monitoring and adaptive learning mechanisms have also significantly improved the system's response speed and intelligent adjustment capabilities, reducing communication costs by 30%, increasing processing accuracy by about 6.2 percentage points, and shortening response time from 40 minutes to 5 minutes, greatly enhancing the efficiency and security of medical data processing.

[0265] Through the above detailed data comparison, the specific effects and benefits of the implementation of the present invention can be further explained. First, the increase in processing speed allows the medical data center to process more data in the same amount of time, which provides strong technical support for responding to sudden large amounts of data needs, such as data collected during public health emergencies. In addition, the significant reduction in privacy protection incidents shows that the present invention can effectively resist external security threats and internal data leakage risks, and ensure the privacy security of patient data, which is crucial for compliance with international data protection regulations such as GDPR.

[0266] The introduction of real-time monitoring and adaptive learning mechanisms not only enhances the system's ability to detect abnormal behavior, but also improves the system's overall response speed and accuracy by intelligently adjusting response strategies. This dynamic adjustment capability enables data centers to flexibly respond to various operating situations, optimize resource allocation, and reduce unnecessary resource waste.

[0267] The reduction of communication costs is achieved through efficient data encoding and transmission strategies, which not only reduces the time delay in the data transmission process, but also optimizes the use of network resources and reduces operating costs. In addition, the improvement of data processing accuracy directly affects the quality and efficiency of medical decision-making. Higher accuracy means more reliable medical analysis results, which has a direct positive impact on improving patient treatment effects and the quality of medical services.

[0268] The significant reduction in response time from 40 minutes to 5 minutes is particularly important. In the medical field, fast response time is often related to life and health. The ability to complete data processing and analysis in a short period of time helps medical personnel make timely diagnosis and treatment decisions. Especially in emergency medical or emergency response, this ability can undoubtedly save lives.

[0269] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A federated learning optimization method based on privacy protection of neural network model, characterized in that: The steps include: S1. Deploy distributed nodes, each of which contains part of the data set, and the information in the data set does not leave the original environment; S2, apply differential privacy technology to pre-process the data at each node to make the data safe during local processing and blur the sensitive features in the data set by adding noise; S3. Use secure multi-party computing mechanism to securely aggregate model parameters between nodes, so that no data information of any single node is leaked during model training; S4. Through the improved gradient aggregation strategy, the gradient descent process is optimized to reduce the deviation and performance degradation problems caused by uneven data distribution in model training; S5, using adaptive algorithms to adjust the training parameters and learning rate of each node, dynamically optimizing the training process based on the computing power of the node and data distribution; S6. Implement efficient data encoding and transmission strategies to reduce the amount of data exchange between nodes, reduce communication overhead, and speed up the overall training of the model; S7. Apply adaptive learning mechanisms based on data and environmental feedback to automatically adjust learning strategies and resource allocation; The S4 specifically includes: S41. In the federated learning environment, the gradient descent optimization is implemented through an improved gradient aggregation strategy. The calculation and aggregation of gradients take into account the distribution characteristics of the data and the computing power of the nodes: Where n is the total number of nodes involved in the calculation, w i is the weight determined by the amount and quality of data of node i, g i is the gradient calculated by node i; S42. Implement gradient clipping technology to limit the size of the gradient transmitted by each node and reduce the instability of model training caused by extreme gradient values: g i '=Clip(g i ,τ)=min(max(g i ,-t),t); Among them, g i is the original gradient, τ is the clipping threshold, g i ' is the clipped gradient; S43, using an adaptive learning rate adjustment mechanism to optimize the learning rate during gradient descent, automatically adjusting the learning rate according to the current stage of model training and the data characteristics of the node: Among them, η t is the learning rate of the tth iteration, η0 is the initial learning rate, δ is the decay rate, and t is the number of iterations; S44. After each iteration, the global model is updated based on the aggregated gradients returned by each node to speed up model convergence and improve training efficiency: i t+1 =θ t -or t ·Gradient agg ; Among them, θ t and θ t+1 Represent the values ​​of the model parameters before and after the tth and t+1th iterations, respectively; S45. Periodically evaluate parameter updates during model training to monitor model performance improvements and detect potential overfitting issues: ΔE val =E val (t)-E val (t-1); Among them, E val (t) is the average validation error of the t-th evaluation cycle, L is the loss function, θ t is the model parameter of the current period, x j and j They are the features and labels in the validation set, m is the number of samples in the validation set, and the error change ΔE val To determine the degree of improvement or overfitting risk of the model, where t-1 and t represent consecutive evaluation cycles; The S5 specifically includes: S51, using an adaptive algorithm to adjust the training parameters of each node, dynamically adjusting the step size of gradient descent and the model update frequency according to the computing power and data characteristics of each node: λ i =λ0·exp(-κ·Var(D i )); Among them, λ i is the adjusted learning rate of node i, λ0 is the base learning rate, κ is the adjustment sensitivity parameter, Var(D i ) is the variance of the node i data set; S52. Implement parameter optimization based on node performance feedback to optimize the learning efficiency and model performance of the entire network: i i,new =θ i,old +γ·(Perf i -Target); Among them, θ i,new and θ i,old They represent the model parameters of node i before and after adjustment, γ is the learning rate, and Perf i is the current performance of node i, Target is the performance target to be achieved; S53. Automatically adjust the amount of data involved in model training according to the computing power of the node to balance the load of data processing and model training tasks: N i =C i ·R; Among them, N i is the number of data samples assigned to node i, C i is the computing power index of node i, and R is the proportional factor of the total amount of global data samples; S54. Apply resource allocation algorithms to optimize the allocation of resources and training tasks between nodes to minimize overall training time and cost: Among them, S opt is the optimized system efficiency, T i is the time required for node i to complete the training task, C i is the running cost of the node; S55. Regularly evaluate and adjust resource allocation and parameter settings during federated learning, and optimize the system based on real-time performance feedback and data distribution changes: Among them, Adjust system (t) represents the system adjustment amount in the tth time period, P t is the parameter setting of the previous cycle, α is the adjustment coefficient, E i,t is the performance evaluation value of the i-th node in the t-th cycle, n is the total number of nodes, E target is the target performance threshold; The S6 specifically includes: S61. Implement efficient data encoding and transmission strategies to reduce the amount of data transmitted between participating nodes and reduce communication costs and time: Among them, S o Indicates the original data size, S e Indicates the size of the encoded data, Efficiency encode Indicates the efficiency of the encoding; S62. Optimize data transmission protocols to enable fast and secure data transmission in the network: Where D is the amount of data to be transmitted, T is the time required for data transmission, and Rate transfer Indicates the data transmission rate; S63. Deploy data compression mechanisms to reduce network congestion and increased transmission costs caused by large amounts of data: Among them, U is the size of uncompressed data, C is the size of compressed data, and CompressionRatio represents the compression ratio; S64. Use distributed data storage technology to reduce the load on a single node and improve data reliability: Among them, L max , L min and L avg They represent the maximum, minimum, and average data loads of all nodes respectively, and Balance represents the balance of data distribution; S65. Keep all nodes synchronized in terms of data integrity and consistency by implementing data synchronization protocols: Among them, N s is the number of nodes that have synchronized data, N t is the total number of nodes, and SyncEfficiency indicates the efficiency of data synchronization.

2. According to claim 1, a federated learning optimization method based on privacy protection of a neural network model is characterized in that: The S1 specifically includes: S11. Select a distributed computing environment, calculate the optimal layout of node deployment based on the number and geographical location of devices participating in federated learning, and use the objective function of minimizing network latency and data transmission cost to determine the node layout: Among them, n represents the total number of nodes, d ij is the physical distance between node i and node j, c ij is the data transmission cost between node i and node j; S12. Configure the hardware and software resources of each node so that each node has sufficient resources to process the local dataset and participate in federated learning: r k =β·s k +γ·p k ; Among them, r k represents the total amount of resources of node k, s k is the storage capacity, p k is the processing capacity, β and γ are adjustment factors; S13. Deploy privacy protection algorithms on each node, including differential privacy and secure multi-party computing protocols, so that each node can complete data preprocessing and model training tasks while protecting data privacy: Among them, ∈ ij represents the risk measure of privacy leakage between nodes i and j, e ij is the amount of information exchanged between two nodes, L is the privacy protection level, and δ is the privacy protection strength coefficient; S14. Use TLS / SSL encryption technology to establish a secure communication protocol between nodes: Among them, σ represents the overall security of the system, α i is the encryption strength of node i, c i is the communication frequency of node i; S15. Implement a data synchronization mechanism to enable data updates on each node to be synchronized in real time during the federated learning process: T sync =max i≠j |t i -t j |; Among them, T sync Indicates the maximum time difference of data synchronization, t i and t j is the data update timestamp of nodes i and j; S16. Perform initial data cleaning and format standardization on the data set of each node: Among them, Q norm is the average normalized score of data quality in the whole system, q i is the score of the data quality of node i.

3. A federated learning optimization method based on privacy protection of a neural network model according to claim 2, characterized in that: The S2 specifically includes: S21. Implement differential privacy technology on each distributed node to protect the privacy of data by adding a specific amount of noise to the data set of each node: Noise(μ,σ 2 )=μ+σ×N(0,1); Where μ is the mean value of the noise, σ is the standard deviation of the noise, and N(0,1) represents the standard normal distribution; S22. Calculate the privacy budget in each data set to control the balance between the strength of privacy protection and data practicality: Among them, ∈ total represents the total privacy budget, k is the number of operations involved in the dataset, ∈ i is the privacy budget of the ith operation; S23. Set privacy parameters for each data operation: Among them, δ i is the privacy protection ratio of the ith operation, which quantifies the weight of the operation in the overall privacy budget; S24. Apply secure multi-party computing mechanism in the data preprocessing stage to achieve secure data aggregation without exposing the specific data content of any node: Among them, x i is the data of the ith node, Enc(x i ) is the data encrypted using the secure multi-party computing protocol.

4. The method for optimizing federated learning based on privacy protection of a neural network model according to claim 3, characterized in that: The S3 specifically includes: S31. In the process of federated learning, the secure multi-party computing mechanism is used to securely aggregate the model parameters between distributed nodes, and homomorphic encryption technology is used to prevent the specific data information of any single node from being leaked during the aggregation process: Among them, x i represents the model parameters of the i-th node, Enc(x i ) and Dec represent encryption and decryption functions respectively; S32. Implement a threshold-based security detection mechanism. At any given point in time, the aggregate model parameters of all participating nodes meet the preset security standards: Among them, T represents the safety threshold to determine whether the model parameters after aggregation are within the safety range; S33. Optimize the weight adjustment strategy used in secure multi-party computing to improve the efficiency and accuracy of the aggregation process, and balance the influence of each node in the aggregation by adjusting the weight of each node: Among them, w i is the weight of node i, n is the total number of nodes participating in the aggregation, σ i is the data quality standard deviation of node i, ∈ is a small constant to prevent division by zero errors; S34. After data aggregation is completed, use the feedback adjustment mechanism based on model performance to dynamically adjust the parameters and strategies of subsequent aggregation: FeedbackAdjustment(m)=m+α·(target-performance(m)); Among them, m represents the current aggregated model, α is the learning rate, target is the target performance indicator, and performance(m) is the performance evaluation of the current model.

5. The method for optimizing federated learning based on privacy protection of a neural network model according to claim 1, characterized in that: The S7 specifically includes: S71. Implement an adaptive learning mechanism based on data or environmental feedback to dynamically adjust learning strategies and resource allocation by analyzing the performance feedback of each node in real time: Adjust learn (i)=η i ·(1+β·(Perf current (i)-Perf target )); Among them, η i is the initial learning rate of node i, β is the adjustment coefficient, and Perf current (i) is the current performance score of node i, Perf target is the predetermined target performance level; S72. Automatically adjust algorithm parameters according to the data characteristics and computing power of each node to optimize the training efficiency and effect of the model: i new (i)=θ old (i)+α·Δθ(i); Among them, θ new (i) and θ old (i) are the parameters of node i before and after adjustment, α is the learning rate, Δθ(i) is the parameter update amount calculated based on performance feedback; S73. Apply environmental monitoring and data analysis tools to evaluate the load and data processing capacity of each node and dynamically adjust resource allocation at the system level: Among them, R alloc (i) is the amount of resources allocated to node i, C i is the computing power score of node i, R total is the total system resources, n is the total number of nodes; S74. Implement a dynamic adjustment algorithm to adjust the data exchange and processing strategy during the learning process based on the real-time monitored network status and node feedback: Policy dynamic (i)=γ·Load(i)+δ·Data rate (i); Among them, Policy dynamic (i) represents the dynamic strategy adjustment value of node i, γ and δ are weight factors, Load(i) is the current load of node i, Data rate (i) is the data processing rate of node i; S75. Regularly evaluate the performance of the federated learning network to ensure that all adjustments meet the optimization goals of efficiency and effectiveness: Among them, Performance eval is the overall performance evaluation value of the network, E i is the efficiency score of node i, and n is the total number of nodes.

Images