Personalized federated learning dimension reduction method and system based on conditional calculation

Abstract

The invention discloses a personalized federal learning dimension reduction method and system based on conditional calculation. The method comprises the following steps: firstly, selecting a dimension reduction model with enough flexibility and expression ability, wherein the dimension reduction model can learn and capture personalized features of high-dimensional patient data on each client; secondly, performing feature extraction on original patient data features through the dimensionality reduction model, and mapping high-dimensional data into a low-dimensional space; the low-dimensional data features are further converted into global features and personalized features; and finally, training the model by using the global features and the personalized features, and optimizing the performance of the model through collaborative learning between the clients and aggregation operation of the server until the model converges. According to the method, an auto-encoder model is used as a basic dimension reduction model, original patient data features are converted into global features and personalized features, and a client side learns the global features and the personalized features in a collaborative manner, so that the personalized purpose is achieved.

Description

Technical Field

[0001] The present invention belongs to the field of personalized federated learning, and specifically relates to a personalized federated learning dimensionality reduction method and system based on conditional computing. Background Art

[0002] Federated learning is a distributed machine learning method that aims to train models without centralizing the participants' original data sets in one place. Instead, the participants' data is kept locally, the model is trained locally, and model updates are delivered to the central server through encryption and secure aggregation technology. This approach protects the data privacy of the participants and reduces the need for data transmission.

[0003] Hospitals or health management companies need to accurately analyze data from different patients (such as genetic data). The health data between patients not only has high data dimensions but also has great differences. The complexity of data processing and calculation in high-dimensional space increases significantly, and the curse of dimensionality has a negative impact on the performance of machine learning and data mining models. The global model is sent to each hospital by the server, and the model used by each hospital is the same. Obviously, for hospitals with different patient data, the global model cannot be universal. If the dimensionality reduction method is trained from ordinary federated learning alone, it can reduce the dimension well for some data types and retain the personalized information of the data, but for data that is not of these types, its personalized information may be lost. Especially in the early prediction of diseases or the recommendation of personalized treatment plans, traditional models cannot take into account the personalized needs of everyone at the same time. Summary of the invention

[0004] In response to the dimensionality disaster problem caused by the high-dimensional data of current hospital patients, the present invention provides a personalized federated learning dimensionality reduction method and system based on conditional computing.

[0005] In a first aspect, the present invention provides a personalized federated learning dimensionality reduction method based on conditional computing, comprising:

[0006] Step (1) selecting a dimension reduction model with sufficient flexibility and expressiveness, wherein the dimension reduction model can learn and capture personalized features of high-dimensional patient data on each client, wherein the high-dimensional patient data includes but is not limited to genetic data, electronic medical record data, and medical imaging data;

[0007] Step (2) extracting features of the original patient data through the dimensionality reduction model, mapping the high-dimensional data into a low-dimensional space, while retaining the key features of the data to reduce computational complexity and avoid dimensionality disaster;

[0008] Step (3) further converting the low-dimensional data features obtained in step (2) into global features and personalized features, wherein the global features are used to capture the common information of all clients, and the personalized features are used to capture the specific information of each client;

[0009] Step (4) trains the model based on the global features and personalized features obtained in step (3), and optimizes the model performance through collaborative learning between clients and aggregation operations on the server until the model converges.

[0010] In a second aspect, the present invention provides a personalized federated learning dimensionality reduction system based on conditional computing, comprising:

[0011] A dimensionality reduction model selection module, used to select a suitable dimensionality reduction model, which can learn and capture personalized features of high-dimensional patient data on each client;

[0012] A feature extraction module, used for extracting features of original patient data through the dimension reduction model;

[0013] Feature conversion module, used to convert low-dimensional data features into global features and personalized features;

[0014] The model training module is used to train the model based on global features and personalized features until the model converges.

[0015] In a third aspect, the present invention provides an application of a personalized federated learning dimensionality reduction method based on conditional computing in intelligent medical care.

[0016] Beneficial effects of the invention: The invention proposes a new PFL dimensionality reduction method, which is based on the autoencoder model and converts the original patient data features into global features and personalized features. The client collaboratively learns the global features and personalized features to achieve the purpose of personalization. Experimental results show that the method proposed by the invention has better personalized effect than other personalized federated learning dimensionality reduction methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of the method of the embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, the embodiment of the present application provides a personalized federated learning dimensionality reduction method based on conditional calculation, and the method comprises the following steps:

[0020] (1) Select an appropriate dimensionality reduction model, ensuring that the selected model has sufficient flexibility and expressiveness to learn and capture personalized features on each client;

[0021] (2) extracting features from the original data using the dimensionality reduction model selected in step (1);

[0022] (3) Converting the low-dimensional data features obtained in step (2) into global features and personalized features;

[0023] (4) Train the model based on the global features and personalized features obtained in step (3) until the model converges.

[0024] In one embodiment, in step (1):

[0025] The dimensionality reduction model uses an autoencoder model. The AE model is a very good dimensionality reduction model with good data reconstruction capabilities and nonlinear feature extraction capabilities, and is suitable for personalized learning tasks. The model is divided into three parts: input layer, hidden layer, and output layer. The Encoder part maps the original data to a low-dimensional space to obtain low-dimensional data, and the Decoder part increases the low-dimensional data to the same dimension as the original data.

[0026] In one embodiment, in step (2):

[0027] To extract the original features, firstly, the original data x is converted into i Dimensionality reduction to obtain h i , where e is the parameter of the encoder, x i ∈R D ,h i ∈R E , R represents the dimension vector, and in the usual case D>>E.

[0028] In one embodiment, step (3) comprises the following steps:

[0029] (3-1) Obtain global feature information parameter g and personalized feature information parameter p through Kmeans clustering algorithm i :

[0030]

[0031] Where |G| is the total number of samples in the category, K is the number of class centers, and x j is the sample feature vector, Indicates the proportion of category u in the total number of samples;

[0032] (3-2) The global feature information parameter g and the personalized feature information parameter p obtained according to (3-1) i Combined with the conditional calculation valve (Cov), γ, β, and γ are calculated. i , β i :

[0033] {γ,β}=Cov(h i ,g,V)

[0034] {γ i ,β i = Cov(h i ,p i ,V)

[0035] Cov consists of two submodules Cov γ and Cov β composition, they have the same structure but different parameters, specifically Cov γ By changing g or p i Input a fully connected layer, a ReLU activation layer, and a layer normalization layer to generate γ / γ i , Cov β Generate β / β in the same way i ,h i It is the low-dimensional data feature after feature extraction in step (2);

[0036] (3-3) According to the parameters obtained from (3-1) and (3-2), the extracted low-dimensional data features h i Convert to Global Features and personalized features

[0037]

[0038] Where σ is the RELU activation function, 1 and h i have the same dimension and all values ​​are 1, ⊙ represents the Hadamard product.

[0039] In one embodiment, step (4) comprises the following steps:

[0040] (4-1) Each client uses local data and then obtains the personalized feature vector according to the operation in step (3) It is used to train the local personalized dimensionality reduction model, and its training loss function is:

[0041]

[0042] Where n represents the number of samples, and the parameters e and d represent the encoder parameters and decoder parameters in the autoencoder model respectively;

[0043] (4-2) The global feature vector obtained in the collaborative learning step (3) between clients Its loss function is:

[0044]

[0045] Among them C k is a cluster point of the cluster center obtained by the Kmeans clustering algorithm, and this cluster point is as close as possible to the global feature vector

[0046] (4-3) Combining the loss functions in (4-1) and (4-2), we can get the client local loss function as:

[0047]

[0048] Among them, λ and μ are hyperparameters in the federated learning training process;

[0049] (4-4) Client uploads model parameters To the server;

[0050] (4-5) The server aggregates model parameters using a weighted average algorithm:

[0051]

[0052] Symbol T t represents the client set in round t iteration, Represents the size of the global dataset in round t iterations.

[0053] (4-6) Repeat steps (4-1) to (4-5) until the model converges.

[0054] The present application also provides a system corresponding to the method, the system comprising:

[0055] A dimensionality reduction model selection module, used to select a suitable dimensionality reduction model, which can learn and capture personalized features of high-dimensional patient data on each client;

[0056] A feature extraction module, used for extracting features of original patient data through the dimension reduction model;

[0057] Feature conversion module, used to convert low-dimensional data features into global features and personalized features;

[0058] The model training module is used to train the model based on global features and personalized features until the model converges.

[0059] The embodiment of the present application also provides an application of the method in intelligent medical treatment.

[0060] Since the patient’s medical information is extremely confidential, other public data sets are used to replace the patient’s data information. The trained personalized autoencoder was used to conduct experiments on the four data sets of MNIST, FMNIST, CIFAR10, and CIFAR100. In order to highlight the personalized performance of the method proposed in this application, the method proposed in this application is compared with the four personalized federated learning methods. In order to highlight the effectiveness of dimensionality reduction in the method proposed in this application, a newer autoencoder model is also trained in a federated learning environment, and clustering operations and FNN classification operations are performed on the reduced-dimensional data. At the same time, three commonly used clustering indicators, namely the silhouette coefficient, the variance ratio criterion, and the DB index, are used to judge the clustering effect. It is believed that the smaller the intra-class distance and the larger the inter-class distance, the better the personalized effect. At the same time, the accuracy (FNN_Accuracy) is used to judge the quality of the classification. In this application, in order to facilitate the observation of the quality of the proposed method, the dimension of the data after dimensionality reduction is closest to the size of the original data dimension. The autoencoder model of the proposed algorithm was used to experimentally test the four indicators mentioned above, and the most commonly used algorithm FEDAVG in federated learning was used as a benchmark to compare the advantages and disadvantages of other algorithms. The results are shown in Table 1.

[0061] Table 1. Data index results after dimensionality reduction

[0062]

[0063] The data marked in bold in Table 1 represent the optimal data results obtained by the personalized dimensionality reduction method in federated learning in the current indicators. The experimental results show that whether from the perspective of different data sets or from the perspective of different personalized federated learning methods, the method proposed in this application is dominant in most cases.

[0064] The above is only a preferred embodiment of the present invention. Although the present invention has been disclosed as a preferred embodiment, it is not intended to limit the present invention. Any technician familiar with the art can make many possible changes and modifications to the technical solution of the present invention by using the above disclosed methods and technical contents without departing from the scope of the technical solution of the present invention, or modify it into an equivalent embodiment of equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of protection of the technical solution of the present invention.

Claims

1. A personalized federated learning dimensionality reduction method based on conditional computing, characterized by The method comprises the following steps: Step (1) selecting a dimension reduction model with sufficient flexibility and expressiveness, wherein the dimension reduction model can learn and capture personalized features of high-dimensional patient data on each client, wherein the high-dimensional patient data includes but is not limited to genetic data, electronic medical record data, and medical imaging data; Step (2) extracting features of the original patient data through the dimensionality reduction model, mapping the high-dimensional data into a low-dimensional space, while retaining the key features of the data to reduce computational complexity and avoid dimensionality disaster; Step (3) further converting the low-dimensional data features obtained in step (2) into global features and personalized features, wherein the global features are used to capture the common information of all clients, and the personalized features are used to capture the specific information of each client; Step (4) trains the model based on the global features and personalized features obtained in step (3), and optimizes the model performance through collaborative learning between clients and aggregation operations on the server until the model converges.

2. The personalized federated learning dimensionality reduction method based on conditional computing according to claim 1, characterized in that: The dimensionality reduction model is an autoencoder model, which includes an input layer, a hidden layer and an output layer, wherein the encoder part maps the original data to a low-dimensional space to obtain low-dimensional data, and the decoder part increases the low-dimensional data to the same dimension as the original data.

3. The personalized federated learning dimensionality reduction method based on conditional computing according to claim 2 is characterized in that: The feature extraction in step (2) includes the following steps: (2-1) Input the original high-dimensional patient data into the encoder part of the autoencoder model; (2-2) Mapping high-dimensional data to low-dimensional space through the encoder’s multi-layer neural network to obtain low-dimensional data feature representation; (2-3) The decoder part restores the low-dimensional data features to the same dimension as the original data to verify the effectiveness of feature extraction.

4. The personalized federated learning dimensionality reduction method based on conditional computing according to claim 1, characterized in that: Step (3) includes the following steps: (3-1) Perform cluster analysis on low-dimensional data features through Kmeans clustering algorithm to obtain global feature information parameters and personalized feature information parameters; (3-2) Calculating weight parameters of global features and personalized features based on the global feature information parameters and personalized feature information parameters obtained in step (3-1) and in combination with a conditional calculation valve; (3-3) According to the parameters obtained in steps (3-1) and (3-2), the low-dimensional data features are converted into global features and personalized features.

5. The personalized federated learning dimensionality reduction method based on conditional computing according to claim 1, characterized in that: Step (4) includes the following steps: (4-1) Each client uses local data to train a local personalized dimensionality reduction model based on the personalized feature vector obtained in step (3); (4-2) The global feature vector obtained in the collaborative learning step (3) between clients is shared and optimized through the federated learning framework; (4-3) Merge the loss functions in steps (4-1) and (4-2) to obtain the client local loss function, and minimize the loss function through the optimization algorithm; (4-4) The client uploads model parameters to the server; (4-5) The server aggregates model parameters through a weighted average algorithm and updates the global model; (4-6) Repeat steps (4-1) to (4-5) until the model converges.

6. The personalized federated learning dimensionality reduction method based on conditional computing according to claim 1, characterized in that: It also includes clustering and classification operations on the data after dimensionality reduction to verify the dimensionality reduction effect and the degree of retention of personalized features. The clustering operation uses three commonly used clustering indicators, namely silhouette coefficient, variance ratio criterion, and DB index, to judge the clustering effect. The classification operation uses accuracy to judge the quality of classification.

7. Application of the personalized federated learning dimensionality reduction method based on conditional computing according to any one of claims 1-6 in intelligent medical care.

8. A personalized federated learning dimensionality reduction system based on conditional computing, characterized in that: include: A dimensionality reduction model selection module, used to select a suitable dimensionality reduction model, which can learn and capture personalized features of high-dimensional patient data on each client; A feature extraction module, used for extracting features of original patient data through the dimension reduction model; Feature conversion module, used to convert low-dimensional data features into global features and personalized features; The model training module is used to train the model based on global features and personalized features until the model converges.

9. The personalized federated learning dimensionality reduction system based on conditional computing according to claim 8, characterized in that: Also includes: The clustering analysis module is used to perform clustering operations on the data after dimensionality reduction to verify the dimensionality reduction effect and the degree of retention of personalized features; The classification verification module is used to perform classification operations on the data after dimensionality reduction to verify the classification effect; Data privacy protection module, used to ensure the privacy of patient data during data processing and model training.

Images