Federal learning model learning prediction method and system based on PSI technology
By training heterogeneous models in federated learning and using PSI technology to calculate data intersection, the problem of difference in model prediction results is solved, the reliability and adaptability of predictions are improved, and a high-correlation federated learning model is generated.
Patent Information
- Application Number
- CN202411359112.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-27
- Publication Date
- 2025-01-10
AI Technical Summary
In federated learning, because the model has undergone multiple iterative training, the final obtained model cannot obtain predicted results that meet the expectations due to the differences in data sources.
By training heterogeneous models on each participant device and calculating data intersections using PSI technology, determining correlation sorting and weighting, multiple heterogeneous models are integrated to form a federated learning model for data prediction.
Improve the adaptability to participant data, ensure the reliability of predicted data, and filter and empower relevant participant devices through heterogeneous model integration and PSI technology, and ultimately generate a highly correlated federated learning model.
Smart Images
Figure CN119294557A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of federated learning prediction technology, and more specifically, to a federated learning model learning prediction method and system based on PSI technology. Background Art
[0002] Federated learning is a distributed machine learning method that protects data privacy and improves data utilization efficiency. It allows multiple participants (usually devices or servers) to jointly train a shared machine learning model while maintaining data privacy and localization. In federated learning, data does not need to be transmitted to a central server, thereby reducing the risk of data leakage. Participants independently train the model using their own data, and then send model updates (such as gradients or model parameters) to the central server.
[0003] The central server is responsible for collecting model updates from all participants and aggregating these updates using certain algorithms (such as averaging, weighted averaging, etc.) to form a global model. The aggregated global model updates are sent back to the participants, who use these updates to improve their own models.
[0004] Federated learning is an iterative process, in which the model is gradually optimized through multiple rounds of training and updating; its core advantage is that it does not share original data, but uses decentralized data to train models, making it suitable for a variety of scenarios.
[0005] In the prediction stage, since the model has gone through multiple rounds of iterations, the party with prediction needs can make data predictions based on the trained model. However, due to the differences in the data sources involved in the training, the final model often cannot produce expected results. A federated learning model learning prediction method and system based on PSI technology is needed to solve this problem. Summary of the invention
[0006] The technical problem to be solved by the present invention is that, in view of the above-mentioned defects of the prior art, a federated learning model learning prediction method based on PSI technology is provided, and a federated learning model learning prediction system based on PSI technology is also provided.
[0007] The technical solution adopted by the present invention to solve its technical problem is:
[0008] A federated learning model learning prediction method based on PSI technology is constructed, wherein the method comprises the steps of:
[0009] Train different heterogeneous models on each participant’s device;
[0010] Receive a data prediction request from any participant, and filter other participants with the same type of data based on the type of data involved in training of the participant who issued the prediction request;
[0011] Calculate the data intersection of the participant data that issued the request and the participant data obtained by screening through PSI technology;
[0012] The relevance ranking is determined by the number of data intersections and the corresponding weights are assigned. Multiple heterogeneous models are integrated according to the weights to obtain a federated learning model.
[0013] The PSI-based federated learning model learning prediction method of the present invention further comprises the steps of:
[0014] The prediction data input by the participant who issued the prediction request is predicted through the federated learning model.
[0015] The PSI-based federated learning model learning prediction method of the present invention, wherein the method of training different heterogeneous models on each participant device is as follows:
[0016] The server receives a federated learning participation request from a participant;
[0017] Provide appropriate model structure or training method, hyperparameter setting, and data preprocessing method based on the data distribution in the federated learning request and the computing power of the participant's equipment;
[0018] Train heterogeneous models based on the provided model structure or training method.
[0019] The PSI-based federated learning model learning prediction method of the present invention, wherein the method of receiving a data prediction request from any participant and screening other participants with the same type of data according to the type of data involved in the training of the participant who issued the prediction request adopts:
[0020] The server establishes a data type database based on the data type in the federated learning reference request;
[0021] The server associates the data type, participants, and heterogeneous model information and stores them in the data type database;
[0022] After receiving the data prediction request from the participant, the server matches the data type corresponding to the requester in the data type database;
[0023] The data type obtained by matching is matched with participants corresponding to the same type of data in the data type database;
[0024] The corresponding heterogeneous model information is matched through the participant information, and the corresponding participant data is queried through the participant information.
[0025] The PSI-based federated learning model learning prediction method of the present invention, wherein the correlation ranking is determined by the number of data intersections and the corresponding weights are assigned, and multiple heterogeneous models are integrated according to the weights to obtain a federated learning model adopting the method:
[0026] Calculate the percentage of the number of data intersections with each participant's data to the total number of data intersections, and use the percentage as the weight;
[0027] The output of each heterogeneous model is extracted as knowledge, and the quantity ratio of the knowledge extracted by each heterogeneous model is adjusted according to the weight;
[0028] Using the extracted knowledge as the target, a unified federated learning model is trained.
[0029] A federated learning model learning prediction system based on PSI technology, wherein the system includes a plurality of participant terminals and a server; the server is communicatively connected with the plurality of participant terminals;
[0030] The participant terminal is used to train the heterogeneous model;
[0031] The server is used to receive a data prediction request from any participant, and screen other participants with the same type of data according to the type of data involved in the training of the participant who issued the prediction request; calculate the data intersection of the data of the participant who issued the request and the screened participant data through the PSI technology; determine the relevance ranking according to the number of data intersections and assign corresponding weights, and integrate multiple heterogeneous models according to the weights to obtain a federated learning model.
[0032] The federated learning model learning prediction system based on PSI technology described in the present invention, wherein the participant terminal is also used to predict the prediction data input by the participant who issued the prediction request through the federated learning model.
[0033] The federated learning model learning prediction system based on PSI technology of the present invention, wherein the participant terminal trains the heterogeneous model using the method:
[0034] The server receives a federated learning participation request from a participant;
[0035] Provide appropriate model structure or training method, hyperparameter setting, and data preprocessing method based on the data distribution in the federated learning request and the computing power of the participant's equipment;
[0036] Train heterogeneous models based on the provided model structure or training method.
[0037] The PSI-based federated learning model learning prediction system of the present invention, wherein the method of receiving a data prediction request from any participant and screening other participants with the same type of data according to the type of data involved in training of the participant who issued the prediction request is as follows:
[0038] The server establishes a data type database based on the data type in the federated learning reference request;
[0039] The server associates the data type, participants, and heterogeneous model information and stores them in the data type database;
[0040] After receiving the data prediction request from the participant, the server matches the data type corresponding to the requester in the data type database;
[0041] The data type obtained by matching is matched with participants corresponding to the same type of data in the data type database;
[0042] The corresponding heterogeneous model information is matched through the participant information, and the corresponding participant data is queried through the participant information.
[0043] The PSI-based federated learning model learning prediction system of the present invention, wherein the correlation ranking is determined by the number of data intersections and the corresponding weights are assigned, and multiple heterogeneous models are integrated according to the weights to obtain a federated learning model using the following method:
[0044] Calculate the percentage of the number of data intersections with each participant's data to the total number of data intersections, and use the percentage as the weight;
[0045] The output of each heterogeneous model is extracted as knowledge, and the quantity ratio of the knowledge extracted by each heterogeneous model is adjusted according to the weight;
[0046] Using the extracted knowledge as the target, a unified federated learning model is trained.
[0047] The beneficial effects of the present invention are: applying the methods of the present application and adopting heterogeneous models to integrate the federated learning model can greatly improve the adaptability of the participant data. At the same time, on this basis, the PSI technology is combined to perform calculation screening and weighting to match other related participant devices, and finally a federated learning model with a high correlation with the predicted data is obtained, thereby ensuring the reliability of the predicted data. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be further described below in conjunction with the accompanying drawings and embodiments. The drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work:
[0049] Figure 1 This is a flow chart of a federated learning model learning prediction method based on PSI technology in a preferred embodiment of the present invention;
[0050] Figure 2 It is a principle block diagram of a federated learning model learning prediction system based on PSI technology in a preferred embodiment of the present invention. DETAILED DESCRIPTION
[0051] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the following will be described clearly and completely in combination with the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are partial embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.
[0052] The federated learning model learning prediction method based on PSI technology in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the steps include:
[0053] S01: Train different heterogeneous models on each participant’s device;
[0054] S02: receiving a data prediction request from any participant, and screening other participants with the same type of data according to the type of data involved in training of the participant who issued the prediction request;
[0055] S03: Calculate the data intersection of the participant data that issued the request and the participant data obtained by screening through the PSI technology;
[0056] S04: Determine the relevance ranking by the number of data intersections and assign corresponding weights, and integrate multiple heterogeneous models according to the weights to obtain a federated learning model;
[0057] S05: Use the federated learning model to predict the prediction data input by the participant who issued the prediction request;
[0058] By applying the methods and approaches of this application and using heterogeneous models to integrate federated learning models, the adaptability to participant data can be greatly improved. At the same time, on this basis, PSI technology is combined for calculation, screening and weighting to match other related participant devices, and finally a federated learning model with a high correlation with the predicted data is obtained, thereby ensuring the reliability of the predicted data.
[0059] Among them, the following methods can be used to train different heterogeneous models on each participant's device:
[0060] The server receives a federated learning participation request from a participant;
[0061] Provide appropriate model structure or training method, hyperparameter setting, and data preprocessing method based on the data distribution in the federated learning request and the computing power of the participant's equipment;
[0062] Train heterogeneous models based on the provided model structure or training method;
[0063] In this way, a local model that is closer to the participant's data can be generated. At the same time, the server will assign which model to generate without the need for intervention from the participant's device.
[0064] Preferably, a data prediction request from any participant is received, and other participants having the same type of data are screened according to the type of data involved in training of the participant who issued the prediction request. The method is as follows:
[0065] The server establishes a data type database based on the data type in the federated learning reference request;
[0066] The server associates the data type, participants, and heterogeneous model information and stores them in the data type database;
[0067] After receiving the data prediction request from the participant, the server matches the data type corresponding to the requester in the data type database;
[0068] The data type obtained by matching is matched with participants corresponding to the same type of data in the data type database;
[0069] Match the corresponding heterogeneous model information through the participant information, and query the corresponding participant data through the participant information;
[0070] The data type can be divided according to the industry of the participants, or according to one or more specific attributes of the participant data, or a combination of the two, and can be set according to actual needs;
[0071] Preferably, the relevance ranking is determined by the number of data intersections and the corresponding weights are assigned, and multiple heterogeneous models are integrated according to the weights to obtain a federated learning model using the following method:
[0072] Calculate the percentage of the number of data intersections with each participant's data to the total number of data intersections, and use the percentage as the weight;
[0073] The output of each heterogeneous model is extracted as knowledge, and the quantity ratio of the knowledge extracted by each heterogeneous model is adjusted according to the weight;
[0074] Use the extracted knowledge as the target to train a unified federated learning model;
[0075] The weights are counted according to percentages, and the amount of knowledge involved in the integration is adjusted (increased or decreased) based on the weights, thereby achieving the purpose of generating a federated learning model based on relevance.
[0076] A federated learning model learning prediction system based on PSI technology, such as Figure 2 As shown, the system includes a plurality of participant terminals 100 and a server 101; the server 101 is communicatively connected with the plurality of participant terminals 100;
[0077] Participant terminal 100, used for training heterogeneous models;
[0078] Server 101 is used to receive a data prediction request from any participant, and screen other participants with the same type of data according to the type of data involved in training of the participant who issued the prediction request; calculate the data intersection of the data of the participant who issued the request and the screened participant data through PSI technology; determine the relevance ranking according to the number of data intersections and assign corresponding weights, and integrate multiple heterogeneous models according to the weights to obtain a federated learning model;
[0079] By applying the system of the present application and integrating the federated learning model with heterogeneous models, the adaptability to the participant data can be greatly improved. At the same time, on this basis, the PSI technology is combined for calculation, screening and weighting to match other related participant devices, and finally a federated learning model with a high correlation with the predicted data is obtained, thereby ensuring the reliability of the predicted data.
[0080] Preferably, the participant terminal 100 is also used to predict the prediction data input by the participant who issued the prediction request through the federated learning model.
[0081] Preferably, the participant terminal 100 trains the heterogeneous model using the following method:
[0082] The server 101 receives a federated learning participation request from a participant;
[0083] Provide appropriate model structure or training method, hyperparameter setting, and data preprocessing method based on the data distribution in the federated learning request and the computing power of the participant's equipment;
[0084] Train heterogeneous models based on the provided model structure or training method;
[0085] In this way, a local model that is closer to the participant's data can be generated. At the same time, the server will assign which model to generate without the need for intervention from the participant's device.
[0086] Preferably, a data prediction request from any participant is received, and other participants having the same type of data are screened according to the type of data involved in training of the participant who issued the prediction request. The method is as follows:
[0087] The server establishes a data type database based on the data type in the federated learning reference request;
[0088] The server associates the data type, participants, and heterogeneous model information and stores them in the data type database;
[0089] After receiving the data prediction request from the participant, the server matches the data type corresponding to the requester in the data type database;
[0090] The data type obtained by matching is matched with participants corresponding to the same type of data in the data type database;
[0091] Match the corresponding heterogeneous model information through the participant information, and query the corresponding participant data through the participant information;
[0092] The data type can be divided according to the industry of the participants, or according to one or more specific attributes of the participant data, or a combination of the two, and can be set according to actual needs;
[0093] Preferably, the relevance ranking is determined by the number of data intersections and the corresponding weights are assigned, and multiple heterogeneous models are integrated according to the weights to obtain a federated learning model using the following method:
[0094] Calculate the percentage of the number of data intersections with each participant's data to the total number of data intersections, and use the percentage as the weight;
[0095] The output of each heterogeneous model is extracted as knowledge, and the quantity ratio of the knowledge extracted by each heterogeneous model is adjusted according to the weight;
[0096] Use the extracted knowledge as the target to train a unified federated learning model;
[0097] The weights are calculated based on percentages, and the amount of knowledge involved in the integration is adjusted (increased or decreased) based on the weights, thereby achieving the purpose of generating a federated learning model based on relevance.
[0098] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.
Claims
1. A federated learning model learning prediction method based on PSI technology, characterized in that: The method comprises the steps of: Train different heterogeneous models on each participant’s device; Receive a data prediction request from any participant, and filter other participants with the same type of data based on the type of data involved in training of the participant who issued the prediction request; Calculate the data intersection of the participant data that issued the request and the participant data obtained by screening through PSI technology; The relevance ranking is determined by the number of data intersections and the corresponding weights are assigned. Multiple heterogeneous models are integrated according to the weights to obtain a federated learning model.
2. The method for learning and predicting a federated learning model based on PSI technology according to claim 1, characterized in that: The method further comprises the steps of: The prediction data input by the participant who issued the prediction request is predicted through the federated learning model.
3. The method for learning and predicting a federated learning model based on PSI technology according to claim 1, characterized in that: The method of training different heterogeneous models on each participant's device is as follows: The server receives a federated learning participation request from a participant; Provide appropriate model structure or training method, hyperparameter setting, and data preprocessing method based on the data distribution in the federated learning request and the computing power of the participant's equipment; Train heterogeneous models based on the provided model structure or training method.
4. The method for learning and predicting a federated learning model based on PSI technology according to claim 3 is characterized in that: The method of receiving a data prediction request from any participant and screening other participants with the same type of data according to the type of data involved in training of the participant who issued the prediction request is as follows: The server establishes a data type database based on the data type in the federated learning reference request; The server associates the data type, participants, and heterogeneous model information and stores them in the data type database; After receiving the data prediction request from the participant, the server matches the data type corresponding to the requester in the data type database; The data type obtained by matching is matched with participants corresponding to the same type of data in the data type database; The corresponding heterogeneous model information is matched through the participant information, and the corresponding participant data is queried through the participant information.
5. The method for learning and predicting a federated learning model based on PSI technology according to claim 4 is characterized in that: The method of determining the relevance ranking by the number of data intersections and assigning corresponding weights, and integrating multiple heterogeneous models according to the weights to obtain a federated learning model adopts: Calculate the percentage of the number of data intersections with each participant's data to the total number of data intersections, and use the percentage as the weight; The output of each heterogeneous model is extracted as knowledge, and the quantity ratio of the knowledge extracted by each heterogeneous model is adjusted according to the weight; Using the extracted knowledge as the target, a unified federated learning model is trained.
6. A federated learning model learning prediction system based on PSI technology, characterized in that: The system includes a plurality of participant terminals and a server; the server is communicatively connected with the plurality of participant terminals; The participant terminal is used to train the heterogeneous model; The server is used to receive a data prediction request from any participant, and screen other participants with the same type of data according to the type of data involved in the training of the participant who issued the prediction request; calculate the data intersection of the data of the participant who issued the request and the screened participant data through the PSI technology; determine the relevance ranking according to the number of data intersections and assign corresponding weights, and integrate multiple heterogeneous models according to the weights to obtain a federated learning model.
7. The PSI-based federated learning model learning prediction system according to claim 6, characterized in that: The participant terminal is also used to predict the prediction data input by the participant who issued the prediction request through the federated learning model.
8. The PSI-based federated learning model learning prediction system according to claim 6, characterized in that: The participant terminal training heterogeneous model adopts the method: The server receives a federated learning participation request from a participant; Provide appropriate model structure or training method, hyperparameter setting, and data preprocessing method based on the data distribution in the federated learning request and the computing power of the participant's equipment; Train heterogeneous models based on the provided model structure or training method.
9. The PSI-based federated learning model learning prediction system according to claim 8, characterized in that: The method of receiving a data prediction request from any participant and screening other participants with the same type of data according to the type of data involved in training of the participant who issued the prediction request is as follows: The server establishes a data type database based on the data type in the federated learning reference request; The server associates the data type, participants, and heterogeneous model information and stores them in the data type database; After receiving the data prediction request from the participant, the server matches the data type corresponding to the requester in the data type database; The data type obtained by matching is matched with participants corresponding to the same type of data in the data type database; The corresponding heterogeneous model information is matched through the participant information, and the corresponding participant data is queried through the participant information.
10. The PSI-based federated learning model learning prediction system according to claim 9, characterized in that: The method of determining the relevance ranking by the number of data intersections and assigning corresponding weights, and integrating multiple heterogeneous models according to the weights to obtain a federated learning model adopts: Calculate the percentage of the number of data intersections with each participant's data to the total number of data intersections, and use the percentage as the weight; The output of each heterogeneous model is extracted as knowledge, and the quantity ratio of the knowledge extracted by each heterogeneous model is adjusted according to the weight; Using the extracted knowledge as the target, a unified federated learning model is trained.