A federated modeling method and system

Abstract

The present application relates to the technical field of federated learning and privacy computing, and specifically provides a federated modeling method and system, which is based on sample sharing and parameter isolation, first performs federated modeling task preparation, then performs federated modeling task training, and finally performs federated modeling model optimization. Compared with the prior art, the present application accelerates the training process and improves the performance and generalization ability of the model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of federated learning and privacy computing, and specifically provides a federated modeling method and system. BACKGROUND

[0002] With the explosive growth of data and the increasing demand for privacy protection, traditional centralized machine learning methods face a series of challenges in processing distributed data. Traditional methods usually require all data to be centralized in a central location for model training, but this centralized approach may lead to the risk of data privacy leakage and increase the cost of data transmission and the burden of computing resources.

[0003] To overcome these problems, federated learning emerges as a distributed machine learning method. Federated learning achieves the goal of model training and knowledge sharing on distributed data by training models on local devices and only sharing model parameters. This distributed approach can avoid centralized data collection, protect data privacy, reduce data transmission costs, and improve computing efficiency.

[0004] However, existing federated learning methods still face some challenges. Each participant (such as a mobile device or an edge node) has different data distribution and sample size, and data sample imbalance can negatively affect model training. In addition, the distribution of data samples may be biased, i.e., the local training data of different participants exhibits non-independent and identically distributed (non-IID) characteristics. Inconsistent data distribution can lead to decreased model performance, failing to meet the actual needs of federated modeling.

[0005] Therefore, there is a need to address the problem of inconsistent data distribution among participants and improve model training effectiveness and data privacy protection. In this case, how to use sample sharing and parameter isolation and other means to form an efficient federated modeling scheme to solve the problems of data sample imbalance and data distribution bias, thereby improving model performance and improving model training effectiveness and data privacy protection, becomes a problem that needs to be solved. SUMMARY

[0006] The present application is aimed at the deficiencies of the prior art and provides a practical federated modeling method.

[0007] The further technical task of the present application is to provide a federated modeling system with reasonable design and safety.

[0008] The technical solution adopted by the present application to solve its technical problems is:

[0009] A federated modeling method, based on sample sharing and parameter isolation, first prepares the federated modeling task, then trains the federated modeling task, and finally optimizes the federated modeling model. The preparation of the federated modeling task includes the following steps:

[0010] S101. Create a federated modeling task based on the actual needs of the federated modeling prediction task;

[0011] S102. Confirm the selected federated modeling participants and design a shared training dataset;

[0012] S103. Set the number of samples in the shared feature dataset, randomly sample samples, and generate a shared feature vector dataset through the pre-trained model PretrainEncoder.

[0013] S104. The federated modeling task initiator sends the shared feature vector dataset and the pre-trained model PretrainEncoder to the federated aggregation node to form a local data feature vector dataset.

[0014] S105. Calculate the bulldozer distance between the selected local data feature vector dataset and the shared feature vector dataset, and continuously replace and adjust the selected local data feature vector dataset samples to meet the bulldozer distance requirements.

[0015] Furthermore, in step S101, based on the actual needs of the federated modeling prediction task, the federated modeling initiator determines the data feature structure of the training dataset, sets the initial network model architecture, training strategy, and federated modeling aggregation method, and creates the federated modeling task.

[0016] Participating nodes in the federated modeling process confirm the requirements of the federated modeling task, negotiate to determine whether their local training data and local resources meet the task requirements, and negotiate to confirm their participation in this task.

[0017] Furthermore, in step S102, after preliminary consultation on the federated modeling task, the initiator of the federated modeling task confirms the selected federated modeling participants and designs a shared training dataset based on the preliminary consultation results.

[0018] The initiator of the federated modeling task selects a publicly available pre-trained model with an encoder-decoder structure, and fine-tunes the parameters of the pre-trained model using the reconstruction error of real raw data according to actual business requirements. The encoder part of the model network is used as the pre-trained model PretrainEncoder.

[0019] The initiator of the federal modeling task uses the raw data to perform data anonymization processing and randomly selects data to form an anonymized dataset that represents the actual business data.

[0020] The federated modeling task initiator uses a generative network to generate a virtual dataset that conforms to the distribution of the original data.

[0021] Furthermore, in step S104, the federated modeling task initiator sends the shared feature vector dataset and the pre-trained model PretrainEncoder to the federated aggregation node.

[0022] The federated aggregation node uses the task management module to negotiate and confirm the federated modeling task;

[0023] The federated aggregation node and each participating node establish a trusted transmission channel through the privacy protection module, and distribute the shared feature vector dataset and the pre-trained model PretrainEncoder to the nodes participating in federated modeling.

[0024] The participating nodes in the federated modeling use the sample selection module LSSM to extract features from the local training data using the PretrainEncoder, forming a local data feature vector dataset.

[0025] Furthermore, in step S105, a threshold for bulldozer distance is set, random sampling is performed in the local data feature vector dataset, and the same number of samples as the shared feature vector dataset are selected. The bulldozer distance between the selected local data feature vector dataset and the shared feature vector dataset is then calculated.

[0026] A greedy algorithm is used to continuously replace and adjust the selected local data feature vector dataset samples to meet the bulldozer distance requirements.

[0027] Repeat the process to select multiple sets of local data feature vector datasets that meet the requirements, and use the corresponding node local data as the local dataset for federated modeling training.

[0028] Furthermore, training for the federated modeling task includes the following steps:

[0029] S201. First-stage federated modeling training based on the selected local dataset;

[0030] S202. Based on the full local dataset, conduct the second stage of federated modeling training;

[0031] Furthermore, in step S201, the following steps are included:

[0032] (1) The aggregation node distributes the initial parameters of the federated modeling neural network model to the participating nodes in the federated modeling;

[0033] (2) The federated modeling node uses the model training and update module to complete the local computation of federated model training based on the selected local dataset, including forward computation, label error calculation and gradient calculation.

[0034] (3) The federated modeling nodes aggregate the calculated model gradients to the federated aggregation nodes through a trusted network channel;

[0035] (4) The federated aggregation node uses the parameter aggregation module to complete the gradient aggregation of the neural network model, and updates the neural network model parameters and distributes them to the federated modeling node;

[0036] (5) In accordance with security requirements, homomorphic encryption is adopted, gradient aggregation and accumulation are calculated using ciphertext, and the homomorphic encryption key is generated by secret sharing and the decryption key is used to decrypt the aggregation result.

[0037] (6) The federated aggregation node receives the updated neural network model and repeats steps (1) to (4) for N rounds until the aggregation model parameters converge. It records the gradient update changes of the model parameters in the last two rounds (N-1, N) of iterative training.

[0038] Furthermore, in step S202, the following steps are performed:

[0039] (1) Divide the parameters according to the parameter layers of the deep neural network model, sort the changes of the gradient update weighted average of the last two rounds of parameters of each layer, and set a threshold. For parameters whose gradient update values ​​in the two rounds are less than the threshold, select a fixed parameter update strategy.

[0040] (2) The federated modeling node uses the model training and update module, which is based on the full local dataset and adopts a parameter isolation strategy. Some parameters are fixed and not updated, and local computation of federated model training is completed, including forward computation, label error calculation and gradient calculation.

[0041] (3) The federated modeling nodes aggregate the calculated model gradients to the federated aggregation nodes through trusted network channels;

[0042] (4) The federated aggregation node uses the parameter aggregation module to complete the gradient aggregation of the neural network model, and updates the neural network model parameters and distributes them to the federated modeling node;

[0043] (5) Based on the ratio of the filtered dataset to the local dataset, set the number of loops M and S. The federated aggregation node receives the updated neural network model and repeats steps (2) to (5) for M loops.

[0044] Federated modeling training is performed based on the full local data, and then steps (2) to (5) are repeated for S loops to perform federated modeling training based on the selected local dataset.

[0045] (6) The central server uses the federated modeling task management module to monitor the execution process of the federated modeling task until the model converges and the final federated modeling neural network model is obtained.

[0046] Furthermore, the optimization of the federated model involves the following steps:

[0047] S301. The federal task initiator deploys the neural network model formed by the federal modeling task to the business system for model inference.

[0048] S202. Continuously collect feedback data on model inference results, continuously optimize the real dataset, and use the feedback data to optimize and improve the model.

[0049] A federated modeling system, based on the actual needs of the federated modeling task, designs a virtual dataset and uses a pre-trained model to extract features from the data, thereby forming a feature data distribution that meets the needs of the real scenario. The feature data distribution is shared with the client nodes participating in the modeling through the sample sharing module of the central server.

[0050] The participating modeling nodes perform data distribution calculations on the local training data through the sample selection module, and select local training data with similar feature distributions to the shared data for node modeling calculations in federated modeling. The entire federated modeling process is divided into two stages: first, federated modeling is performed using the formed independent and identically distributed data, and the parameter update method in the federated modeling process is adjusted. After multiple iterations, the federated modeling model is finally formed.

[0051] Compared with existing technologies, the federated modeling method and system of this invention have the following outstanding advantages:

[0052] This invention differs from traditional federated modeling methods by employing a two-stage modeling approach. First, federated modeling is performed using the resulting independent and identically distributed data. Then, the parameter update method during the federated modeling process is adjusted through a parameter isolation module. The two-stage training is performed alternately, which fully considers the representativeness of the shared dataset and the diversity of global data, accelerates the training process, and improves the model's performance and generalization ability.

[0053] By generating feature vectors through pre-trained models and sharing datasets, the risk of original data leakage is eliminated. This allows for the collaborative construction of high-performance models while protecting data privacy.

[0054] Meanwhile, multiple privacy protection measures, including channel encryption, homomorphic encryption, access control, and permission management, are employed to ensure data security during transmission, storage, and computation, preventing unauthorized access and leakage, and thus protecting data privacy. This method is applicable to various federated learning scenarios, such as medical data analysis, financial risk control, and intelligent transportation, and has high practical value and broad application prospects. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Appendix Figure 1 This is a flowchart illustrating a federated modeling approach. Detailed Implementation

[0057] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to specific embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] The following is a preferred embodiment:

[0059] like Figure 1 As shown, this embodiment of the federated modeling method, based on sample sharing and parameter isolation, first prepares the federated modeling task, then trains the federated modeling task, and finally optimizes the federated modeling model. The preparation of the federated modeling task includes the following steps:

[0060] S101. Based on the actual needs of the federated modeling prediction task, the federated modeling initiator determines the data feature structure of the training dataset, sets the initial network model architecture, training strategy, and federated modeling aggregation method, and creates the federated modeling task.

[0061] S102. Participating nodes in the federated modeling process confirm the requirements of the federated modeling task, negotiate to determine whether their local training data and local resources meet the task requirements, and negotiate to confirm their participation in this task.

[0062] S103. After preliminary consultation on the federated modeling task, the initiator of the federated modeling task confirms the selected federated modeling participants and designs a shared training dataset based on the preliminary consultation results.

[0063] S104. The initiator of the federated modeling task selects a publicly available pre-trained model with an encoder-decoder structure, and fine-tunes the parameters of the pre-trained model using the reconstruction error of real raw data according to actual business requirements. The encoder part of the model network is used as the pre-trained model PretrainEncoder.

[0064] S105. (Optional) The initiator of the federated modeling task uses the original data to perform data anonymization processing and randomly selects data to form an anonymized dataset that represents the actual business data.

[0065] S106. (Optional) The federated modeling task initiator uses a generative network to generate a virtual dataset that conforms to the distribution of the original data based on the original data.

[0066] S107. The federated aggregation node uses the Task Management Module (TMM) to complete the negotiation and confirmation of the federated modeling task;

[0067] S108: The federated aggregation node and each participating node establish a trusted transmission channel through the privacy protection module, and distribute the shared feature vector dataset and the pre-trained model PretrainEncoder to the nodes participating in federated modeling.

[0068] S109. The federated modeling participating nodes use the sample selection module LSSM to extract features from the local training data using the PretrainEncoder, forming a local data feature vector dataset.

[0069] S110. Set a threshold for bulldozer distance, randomly sample from the local data feature vector dataset, select the same number of samples as the shared feature vector dataset, and calculate the bulldozer distance between the selected local data feature vector dataset and the shared feature vector dataset.

[0070] S111. Using a greedy algorithm, the selected local data feature vector dataset samples are continuously replaced and adjusted to meet the bulldozer distance requirements.

[0071] S112. Repeat steps S110 to S111 to select multiple sets of local data feature vector datasets that meet the requirements, and use the corresponding node local data as the local dataset for federated modeling training.

[0072] Training for federated modeling tasks includes the following steps:

[0073] S201. First-stage federated modeling training based on the selected local dataset;

[0074] (1) The aggregation node distributes the initial parameters of the federated modeling neural network model to the participating nodes in the federated modeling;

[0075] (2) The federated modeling node uses the model training and update module to complete the local computation of federated model training based on the selected local dataset, including forward computation, label error calculation and gradient calculation.

[0076] (3) The federated modeling nodes aggregate the calculated model gradients to the federated aggregation nodes through a trusted network channel;

[0077] (4) The federated aggregation node uses the parameter aggregation module to complete the gradient aggregation of the neural network model, and updates the neural network model parameters and distributes them to the federated modeling node;

[0078] (5) In accordance with security requirements, homomorphic encryption is adopted, gradient aggregation and accumulation are calculated using ciphertext, and the homomorphic encryption key is generated by secret sharing and the decryption key is used to decrypt the aggregation result.

[0079] (6) The federated aggregation node receives the updated neural network model and repeats steps (1) to (4) for N rounds until the aggregation model parameters converge. It records the gradient update changes of the model parameters in the last two rounds (N-1, N) of iterative training.

[0080] S202. Based on the full local dataset, conduct the second stage of federated modeling training;

[0081] (1) Divide the parameters according to the parameter layers of the deep neural network model, sort the changes of the gradient update weighted average of the last two rounds of parameters of each layer, and set a threshold. For parameters whose gradient update values ​​in the two rounds are less than the threshold, select a fixed parameter update strategy.

[0082] (2) The federated modeling node uses the model training and update module, which is based on the full local dataset and adopts a parameter isolation strategy. Some parameters are fixed and not updated, and local computation of federated model training is completed, including forward computation, label error calculation and gradient calculation.

[0083] (3) The federated modeling nodes aggregate the calculated model gradients to the federated aggregation nodes through trusted network channels;

[0084] (4) The federated aggregation node uses the parameter aggregation module to complete the gradient aggregation of the neural network model, and updates the neural network model parameters and distributes them to the federated modeling node;

[0085] (5) Based on the ratio of the filtered dataset to the local dataset, set the number of loops M and S (generally set M / S = 30). The federated aggregation node receives the updated neural network model and repeats steps (2) to (5) for M loops.

[0086] Federated modeling training is performed based on the full local data, and then steps (2) to (5) are repeated for S loops to perform federated modeling training based on the selected local dataset.

[0087] (6) The central server uses the federated modeling task management module to monitor the execution process of the federated modeling task until the model converges and the final federated modeling neural network model is obtained.

[0088] The following steps are involved in optimizing a federated model:

[0089] S301. The federal task initiator deploys the neural network model formed by the federal modeling task to the business system for model inference.

[0090] S202. Continuously collect feedback data on model inference results, continuously optimize the real dataset, and use the feedback data to optimize and improve the model in order to improve prediction accuracy and performance.

[0091] Based on the above method, a federated modeling system in this embodiment, according to the actual needs of the federated modeling task, designs a virtual dataset and uses a pre-trained model to extract features from the data, thereby forming a feature data distribution that meets the needs of the real scene. The feature data distribution is shared with the client nodes participating in the modeling through the sample sharing module of the central server.

[0092] The participating modeling nodes perform data distribution calculations on the local training data through the sample selection module, and select local training data with similar feature distributions to the shared data for node modeling calculations in federated modeling. The entire federated modeling process is divided into two stages: first, federated modeling is performed using the formed independent and identically distributed data, and the parameter update method in the federated modeling process is adjusted. After multiple iterations, the federated modeling model is finally formed.

[0093] Federated modeling tasks involve multiple parties jointly training a deep learning model. The task initiator creates the task and negotiates its content. Each participating node in the federated modeling task has its own dataset and computing resources, and they coordinate and aggregate through the federated modeling aggregation node to jointly complete the federated modeling task.

[0094] The federated modeling aggregation node is deployed on the central server and builds services based on computing, storage, and network resource infrastructure. It is responsible for coordinating, aggregating, and managing federated modeling training tasks and consists of functional modules such as task management, sample sharing, parameter aggregation, and privacy protection.

[0095] The Task Management Module (TMM) enables task creation, negotiation, execution, and monitoring.

[0096] The Sample Sharing Module (SSM) is a feature vector dataset generated by the initiator of the federated modeling task based on the actual situation of the task, using a pre-trained model, and sharing the feature vector dataset and the pre-trained model with the participating modeling nodes.

[0097] PretrainEncoder is a publicly available feature extraction model or a feature extraction model fine-tuned using publicly available pretrained models. The core of the model is a neural network with an Encoder and Decoder structure, which extracts features from data samples and forms a low-dimensional feature representation of the samples.

[0098] Feature vector datasets are collections of sample vectors representing low-order features generated using pre-trained models based on virtual datasets constructed according to actual business operations. They are used for global sample sharing and to represent the distribution of input data for actual task operations.

[0099] Virtual datasets are virtual datasets designed and constructed by task initiators based on their real business requirements. They can use privacy protection modules in various ways to protect the privacy of the real original data, such as data anonymization of the original data, or data anonymization by combining feature vectors constructed from pre-trained models.

[0100] The Parameter Aggregation Module (PAM) is responsible for aggregating the training gradients of each participating node. Depending on the aggregation algorithm, the homomorphic encryption module of the privacy protection module can be used to implement ciphertext computation to enhance the security of aggregation.

[0101] The Privacy Protection Module (PPM) is responsible for protecting data privacy during the federated modeling training process. It employs various privacy protection methods, such as channel encryption, homomorphic encryption, access control, and permission management, to ensure the security of data during transmission, storage, and computation, prevent unauthorized access and leakage, and achieve data privacy protection.

[0102] Federated modeling involves nodes distributed across local nodes (Client Nodes) of each participating modeling task. These nodes possess computing, storage, and network resources. Training data for the modeling task is stored locally on each node and does not leave the domain during training. This enables local model training computation across distributed nodes in federated modeling. Based on local resource infrastructure, it provides functions such as a sample selection module, a parameter isolation module, a model training and update module, and a privacy protection module.

[0103] The Local Sample Selection Module (LSSM) is responsible for selecting the local dataset for training the federated model based on the shared feature vector dataset and the pre-trained model, using bulldozer distance to measure the difference between the distribution of the local dataset and the shared feature dataset.

[0104] The Parameter Isolation Module (PIM) freezes some parameters based on gradient changes by setting a parameter update strategy, thus achieving more efficient and reasonable model parameter updates.

[0105] The Model Training Module (MTM) performs local model forward computation and gradient computation based on the federated modeling two-stage training algorithm.

[0106] The Privacy Protection Module (LPPM) is primarily responsible for protecting the privacy of local data and computations of participating nodes.

[0107] The specific embodiments described above are merely specific examples of the present invention. The patent protection scope of the present invention includes, but is not limited to, the specific embodiments described above. Any appropriate changes or substitutions made by a person skilled in the art that conform to the claims of the present invention regarding a federal modeling method and system should fall within the patent protection scope of the present invention.

[0108] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A federated modeling method, characterized in that, Based on sample sharing and parameter isolation, the federated modeling task is first prepared, then trained, and finally optimized. The preparation of the federated modeling task includes the following steps: S101. Create a federated modeling task based on the actual needs of the federated modeling prediction task; Based on the actual needs of the federated modeling prediction task, the federated modeling initiator determines the data feature structure of the training dataset, sets the initial network model architecture, training strategy, and federated modeling aggregation method, and creates the federated modeling task. Participating nodes in the federated modeling task confirm the requirements of the federated modeling task, negotiate to determine whether their local training data and local resources meet the task requirements, and negotiate to confirm their participation in this task. S102. Confirm the selected federated modeling participants and design a shared training dataset; Following initial consultations regarding the federated modeling task, the initiator of the federated modeling task confirmed the selected federated modeling participants and designed a shared training dataset based on the results of the initial consultations. The initiator of the federated modeling task selects a publicly available pre-trained model with an encoder-decoder structure, and fine-tunes the parameters of the pre-trained model using the reconstruction error of real raw data according to actual business requirements. The encoder part of the model network is used as the pre-trained model PretrainEncoder. The initiator of the federated modeling task uses the raw data to perform data anonymization processing and randomly selects data to form an anonymized dataset that represents the actual business data. The federated modeling task initiator uses a generative network to generate a virtual dataset that conforms to the distribution of the original data based on the original data; S103. Set the number of samples in the shared feature dataset, randomly sample samples, and generate a shared feature vector dataset through the pre-trained model PretrainEncoder. S104. The federated modeling task initiator sends the shared feature vector dataset and the pre-trained model PretrainEncoder to the federated aggregation node to form a local data feature vector dataset. The federated modeling task initiator sends the shared feature vector dataset and the pre-trained model PretrainEncoder to the federated aggregation node; The federated aggregation node uses the task management module to negotiate and confirm the federated modeling task; The federated aggregation node and each participating node establish a trusted transmission channel through the privacy protection module, and distribute the shared feature vector dataset and the pre-trained model PretrainEncoder to the nodes participating in federated modeling. The participating nodes in the federated modeling use the sample selection module LSSM to extract features from the local training data using the PretrainEncoder, forming a local data feature vector dataset. S105. Calculate the bulldozer distance between the selected local data feature vector dataset and the shared feature vector dataset, and continuously replace and adjust the selected local data feature vector dataset samples to meet the bulldozer distance requirements. Set a threshold for bulldozer distance, randomly sample from the local data feature vector dataset, select the same number of samples as the shared feature vector dataset, and calculate the bulldozer distance between the selected local data feature vector dataset and the shared feature vector dataset. A greedy algorithm is used to continuously replace and adjust the selected local data feature vector dataset samples to meet the bulldozer distance requirements. Repeat the process to select multiple sets of local data feature vector datasets that meet the requirements, and use the corresponding node local data as the local dataset for federated modeling training. Training for federated modeling tasks includes the following steps: S201. First-stage federated modeling training based on the selected local dataset; It has the following steps: (1) The aggregation node distributes the initial parameters of the federated modeling neural network model to the participating nodes in the federated modeling; (2) The federated modeling node uses the model training and update module to complete the local computation of federated model training based on the selected local dataset, including forward computation, label error calculation and gradient calculation; (3) The federated modeling nodes aggregate the calculated model gradients to the federated aggregation nodes through a trusted network channel; (4) The federated aggregation node uses the parameter aggregation module to complete the gradient aggregation of the neural network model, and updates the neural network model parameters and distributes them to the federated modeling node; (5) In accordance with security requirements, homomorphic encryption is adopted, gradient aggregation and accumulation are calculated using ciphertext, and the homomorphic encryption key is generated by secret sharing and the decryption key is used to decrypt the aggregation result; (6) The federated aggregation node receives the updated neural network model and repeats steps (1) to (4) for N rounds until the aggregation model parameters converge. It records the gradient update changes of the model parameters in the last two rounds (N-1, N) of iterative training. S202. Based on the full local dataset, conduct the second stage of federated modeling training.

2. The federated modeling method according to claim 1, characterized in that, In step S202, the following steps are performed: (1) Divide the parameters of the deep neural network model into layers, sort the changes of the gradient update weighted average of the last two rounds of parameters of each layer, and set a threshold. For parameters whose gradient update values ​​of the two rounds are less than the threshold, select a fixed update strategy for some of the parameters. (2) The federated modeling node uses the model training and update module, which is based on the full local dataset and adopts a parameter isolation strategy. Some parameters are fixed and not updated, and local computation of federated model training is completed, including forward computation, label error calculation and gradient calculation. (3) The federated modeling nodes aggregate the calculated model gradients to the federated aggregation nodes through trusted network channels; (4) The federated aggregation node uses the parameter aggregation module to complete the gradient aggregation of the neural network model, and updates the neural network model parameters and distributes them to the federated modeling node; (5) Based on the ratio of the filtered dataset to the local dataset, set the number of loops M and S. The federated aggregation node receives the updated neural network model and repeats steps (2) to (5) for M loops. Federated modeling training is performed based on the full local data, and then steps (2) to (5) are repeated for S loops to perform federated modeling training based on the selected local dataset. (6) The central server uses the federated modeling task management module to monitor the execution process of the federated modeling task until the model converges and the final neural network model of the federated modeling is obtained.

3. The federated modeling method according to claim 2, characterized in that, The following steps are involved in optimizing a federated model: S301. The federal task initiator deploys the neural network model formed by the federal modeling task to the business system for model inference. S202. Continuously collect feedback data on model inference results, continuously optimize the real dataset, and use the feedback data to optimize and improve the model.

4. A federated modeling system, characterized in that, Based on the actual needs of the federated modeling task, a virtual dataset is designed and a pre-trained model is used to extract features from the data, thereby forming a feature data distribution that meets the needs of the real scenario. The feature data distribution is shared with the client nodes participating in the modeling through the sample sharing module of the central server. The participating modeling nodes perform data distribution calculations on the local training data through the sample selection module, and select local training data with similar feature distribution to the shared data for node modeling calculations in federated modeling. The entire federated modeling process is divided into two stages: first, federated modeling is performed using the formed independent and identically distributed data, and the parameter update method in the federated modeling process is adjusted. After multiple iterations, the federated modeling model is finally formed. Preparation for the federated modeling task includes the following steps: S101. Create a federated modeling task based on the actual needs of the federated modeling prediction task; Based on the actual needs of the federated modeling prediction task, the federated modeling initiator determines the data feature structure of the training dataset, sets the initial network model architecture, training strategy, and federated modeling aggregation method, and creates the federated modeling task. Participating nodes in the federated modeling task confirm the requirements of the federated modeling task, negotiate to determine whether their local training data and local resources meet the task requirements, and negotiate to confirm their participation in this task. S102. Confirm the selected federated modeling participants and design a shared training dataset; Following initial consultations regarding the federated modeling task, the initiator of the federated modeling task confirmed the selected federated modeling participants and designed a shared training dataset based on the results of the initial consultations. The initiator of the federated modeling task selects a publicly available pre-trained model with an encoder-decoder structure, and fine-tunes the parameters of the pre-trained model using the reconstruction error of real raw data according to actual business requirements. The encoder part of the model network is used as the pre-trained model PretrainEncoder. The initiator of the federated modeling task uses the raw data to perform data anonymization processing and randomly selects data to form an anonymized dataset that represents the actual business data. The federated modeling task initiator uses a generative network to generate a virtual dataset that conforms to the distribution of the original data based on the original data; S103. Set the number of samples in the shared feature dataset, randomly sample samples, and generate a shared feature vector dataset through the pre-trained model PretrainEncoder. S104. The federated modeling task initiator sends the shared feature vector dataset and the pre-trained model PretrainEncoder to the federated aggregation node to form a local data feature vector dataset. The federated modeling task initiator sends the shared feature vector dataset and the pre-trained model PretrainEncoder to the federated aggregation node; The federated aggregation node uses the task management module to negotiate and confirm the federated modeling task; The federated aggregation node and each participating node establish a trusted transmission channel through the privacy protection module, and distribute the shared feature vector dataset and the pre-trained model PretrainEncoder to the nodes participating in federated modeling. The participating nodes in the federated modeling use the sample selection module LSSM to extract features from the local training data using the PretrainEncoder, forming a local data feature vector dataset. S105. Calculate the bulldozer distance between the selected local data feature vector dataset and the shared feature vector dataset, and continuously replace and adjust the selected local data feature vector dataset samples to meet the bulldozer distance requirements. Set a threshold for bulldozer distance, randomly sample from the local data feature vector dataset, select the same number of samples as the shared feature vector dataset, and calculate the bulldozer distance between the selected local data feature vector dataset and the shared feature vector dataset. A greedy algorithm is used to continuously replace and adjust the selected local data feature vector dataset samples to meet the bulldozer distance requirements. Repeat the process to select multiple sets of local data feature vector datasets that meet the requirements, and use the corresponding node local data as the local dataset for federated modeling training. Training for federated modeling tasks includes the following steps: S201. First-stage federated modeling training based on the selected local dataset; It has the following steps: (1) The aggregation node distributes the initial parameters of the federated modeling neural network model to the participating nodes in the federated modeling; (2) The federated modeling node uses the model training and update module to complete the local computation of federated model training based on the selected local dataset, including forward computation, label error calculation and gradient calculation; (3) The federated modeling nodes aggregate the calculated model gradients to the federated aggregation nodes through a trusted network channel; (4) The federated aggregation node uses the parameter aggregation module to complete the gradient aggregation of the neural network model, and updates the neural network model parameters and distributes them to the federated modeling node; (5) In accordance with security requirements, homomorphic encryption is adopted, gradient aggregation and accumulation are calculated using ciphertext, and the homomorphic encryption key is generated by secret sharing and the decryption key is used to decrypt the aggregation result; (6) The federated aggregation node receives the updated neural network model and repeats steps (1) to (4) for N rounds until the aggregation model parameters converge. It records the gradient update changes of the model parameters in the last two rounds (N-1, N) of iterative training. S202. Based on the full local dataset, conduct the second stage of federated modeling training.

Images