Alzheimer's disease federal learning method based on sample importance and gradient relevance

Abstract

The invention relates to a senile dementia federal learning method based on sample importance and gradient relevance. The method comprises the following steps: firstly, collecting MRI (Magnetic Resonance Imaging) scanning results not suffering from Alzheimer's disease and suffering from Alzheimer's disease, and generating a coded representation by using an encoder; training a prototype network by using local data, and dividing samples into important samples, boundary samples and edge samples by calculating the distance between the samples and a prototype; the terminals share important samples through federal learning, jointly train a global prototype network, and issue the global prototype network to the terminals to initialize subsequent fine tuning. And finally, on the terminal, performing fine tuning by using the weighted important sample and the boundary sample, endowing the boundary sample with a larger gradient weight, calculating the main direction of the gradient in the fine tuning process by using principal component analysis, and selecting a terminal with a similar gradient to perform combined fine tuning, thereby reducing the overfitting problem caused by insufficient boundary samples. The method can quickly adapt to different sample distribution of each terminal, and effectively solves the problems of terminal data distribution difference and small sample overfitting in federated learning.

Description

Technical Field

[0001] The present invention belongs to the field of federated learning, and more specifically, relates to a federated learning method for Alzheimer's disease based on sample importance and gradient correlation. Background Art

[0002] Alzheimer's disease is a common neurodegenerative disease that affects the cognitive function of the elderly. With the intensification of global aging, the prevalence of Alzheimer's disease has increased year by year, bringing heavy economic and emotional burdens to society and families. Early diagnosis and intervention are important means to slow the progression of the disease. Currently, magnetic resonance imaging is widely used in the diagnosis of Alzheimer's disease. Automatic analysis of magnetic resonance images through machine learning and deep learning technology can effectively improve the accuracy of early diagnosis. However, training a high-precision deep learning model usually requires large-scale labeled data, and these data are often scattered in different medical institutions and are difficult to share due to privacy protection and security requirements.

[0003] Federated learning, as a distributed learning framework, allows various institutions to build a global model through joint training without sharing the original data, thereby protecting data privacy. However, in federated learning, the local data distribution of each terminal often varies greatly, which may cause the performance of the global model on each terminal to be inconsistent and difficult to adapt to the specific data distribution of each terminal. In addition, due to the limited boundary samples, the model is prone to overfitting problems, especially in the case of small sample sizes and unbalanced data, which further affects the generalization ability of the model.

[0004] In view of the above problems, the present invention proposes a federated learning method for Alzheimer's disease based on sample importance and gradient correlation. In this method, the importance of samples is first divided, and the samples are divided into important samples and boundary samples, where important samples contain general knowledge and are suitable for the training of global models, while boundary samples have a greater impact on the decision boundary. Through federated learning, important samples are jointly trained on the server side to generate a global model that integrates the general knowledge of each terminal. This global model can quickly adapt to the different sample distributions of each terminal through subsequent fine-tuning, effectively solving the problem that federated learning does not fully consider the differences in terminal data distribution.

[0005] In order to further improve the stability of the model and solve the overfitting problem, the present invention introduces a joint fine-tuning method based on gradient correlation. During the fine-tuning process, the terminals with similar gradient directions are jointly trained by analyzing the correlation of the gradient directions of each terminal. Since the data distribution of these terminals is relatively similar, joint fine-tuning can alleviate the risk of overfitting and improve the overall accuracy and convergence speed of the model. This method can not only improve the generalization ability of the model, but also ensure the consistency and stability of the model on different terminals, thereby better supporting the early diagnosis of Alzheimer's disease. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and propose a federated learning method based on sample importance and gradient correlation to achieve rapid and accurate analysis of Alzheimer's disease.

[0007] To achieve the above-mentioned object of the invention, the federated learning method for Alzheimer's disease based on sample importance and gradient correlation of the present invention is characterized by comprising the following steps:

[0008] (1) Collecting MRI scan results of patients with and without Alzheimer's disease as a data set for model training, and obtaining corresponding encoding representations through an encoder;

[0009] (2) For each terminal, use its own local data to train the prototype network stored in the terminal, and then calculate the distance between the sample and the prototype of Alzheimer's disease and the prototype of non-Alzheimer's disease, such as the squared Euclidean distance or cosine distance in Bregman divergence, etc., and divide the samples into three types according to the distance: important samples (points that are close to one prototype and far from the other prototype), boundary samples (points with similar distances from the two prototypes), and edge samples (points that are far from both prototypes);

[0010] (3) Each terminal shares important samples through federated learning and jointly trains a global prototype network with broad adaptability. After the training is completed, the optimized global prototype network is sent to each terminal to initialize the subsequent fine-tuning stage of the terminal;

[0011] (4) On the terminal, the global prototype network received from the server is used to perform fine-tuning using weighted important samples and boundary samples. During the fine-tuning process, a larger weight is given to the gradient change of the boundary samples, while a smaller weight is given to the gradient change of the important samples. Subsequently, the main direction of the gradient during the fine-tuning process is calculated by principal component analysis, and the correlation of the gradients between different terminals is further calculated. According to the similarity of the gradients, terminals with similar gradients are selected for joint fine-tuning to alleviate the overfitting problem caused by insufficient boundary samples;

[0012] The object of the present invention is achieved in this way.

[0013] The federated learning method for Alzheimer's disease based on sample importance and gradient correlation of the present invention divides samples into three categories by dividing the sample importance: boundary samples that have a greater impact on the decision boundary, important samples that are easier to classify and contain general knowledge, and edge samples that may be noise. Through federated learning, each terminal is jointly trained based on important samples to generate a server-side model that integrates the general knowledge of important samples of each terminal. The global model has generalization capabilities and can quickly adapt to the different data distributions of each terminal through fine-tuning, thereby effectively solving the model adaptability problem caused by the differences in data distribution of each terminal.

[0014] In order to further alleviate the overfitting problem of the model caused by insufficient boundary samples, the present invention introduces a joint fine-tuning method based on gradient correlation. During the fine-tuning process, the similarity of gradients between different terminals is calculated, and terminals with similar gradient directions are jointly trained. This method can jointly optimize terminals with similar distributions, thereby reducing the risk of overfitting and improving the overall accuracy while ensuring the stability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flowchart of a specific implementation method of the federated learning method for Alzheimer's disease based on sample importance and gradient correlation of the present invention;

[0016] Figure 2 is a schematic diagram of sample division in the present invention, wherein each triangle represents an edge sample divided into noise, each star represents an important sample divided into generality, each square represents a boundary sample divided into an impact boundary, and the blue point and the green point represent the prototype of Alzheimer's disease and the prototype of non-Alzheimer's disease, respectively;

[0017] Figure 3 is a schematic diagram of the gradient correlation detection based on principal component analysis in the present invention, wherein each circle is the gradient obtained by the terminal in a certain round, d1 and d2 respectively represent the first principal component directions of the gradients when the two terminals are fine-tuned, and θ represents the angle between them; DETAILED DESCRIPTION

[0018] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0019] Figure 1 It is a flowchart of a specific implementation of the federated learning method for Alzheimer's disease based on sample importance and gradient correlation of the present invention.

[0020] In this embodiment, if Figure 1 As shown, the federated learning method for Alzheimer's disease based on sample importance and gradient correlation of the present invention comprises the following steps:

[0021] S1: Collecting and processing data

[0022] At each terminal, magnetic resonance imaging data samples are first collected and divided into two groups according to whether the samples suffer from Alzheimer's disease: an Alzheimer's disease group and a non-Alzheimer's disease group. In order to extract the characteristic information of the samples, an encoder is used to encode each nuclear magnetic resonance image sample to obtain its corresponding encoded representation. In this embodiment, the encoder can use an unpre-trained network structure (such as a convolutional neural network, CNN), or an encoder that has been pre-trained in an image classification task (such as a Vision Transformer, VIT). The encoder is responsible for mapping high-dimensional image data into low-dimensional feature representations for use by subsequent models.

[0023] S2: Sample importance division

[0024] In order to classify more effectively, the present invention solves the problem of data distribution differences between different terminals by dividing the sample importance. The specific implementation steps are as follows:

[0025] 2.1) On each terminal, the data set is first divided into a support set and a query set, which are used to train the prototype network and calculate the distance between the sample and the prototype respectively. For all samples of Alzheimer's disease and non-Alzheimer's disease in the support set, the mean of their encoding representation is calculated as the prototype representation of these two categories:

[0026]

[0027] Among them, x c1 and x c2 are the prototype representations of Alzheimer's disease and non-Alzheimer's disease, N1 and N2 represent the number of samples in the two categories, respectively. i and x j For each sample, the encoding representation is x. k , respectively calculate it and the two prototypes x c1 and x c2 Distance between:

[0028] d1=dist(x k ,x c1 )

[0029] d2=dist(x k ,x c2 )

[0030] Where dist is the distance function. In this embodiment, Euclidean distance or cosine distance can be used. The smaller the distance, the closer the sample is to the prototype of the class, indicating that it is more likely to belong to the class; then, we use the SoftMax function to normalize the distance and calculate the probability representation of the sample belonging to each prototype; then based on this probability, back propagation is performed through the cross entropy loss function to update the encoder parameters, and iterate the step 2.1) until the prototype converges.

[0031] 2.2) Once the local prototype network is trained, we can use it to divide the importance of samples. Figure 2 As shown in the figure, the samples are divided into three categories by the Euclidean distance between the sample encoding representation and each type of prototype: important samples close to the prototype, boundary samples located at the decision boundary between the two prototypes, and edge samples far from both prototypes. In this example, after we convert the distance into a probability vector [y1, y2], we first regard the samples whose y1 and y2 are both less than the threshold as edge samples, and then use the absolute value of the difference between the predicted results as the criterion for dividing important samples and boundary samples: the smaller the absolute value, the more similar the distance to the two prototypes, and the more likely it is a boundary sample; the larger the absolute value, the closer the distance to a prototype, and the more likely it is an important sample.

[0032] S3: Jointly training general prototypes

[0033] Using the federated learning method, each terminal downloads the latest model from the server, then uses the important samples obtained by local division to train the model, and then uploads the gradient to the server. The server uses the gradient averaging algorithm to aggregate the gradient:

[0034]

[0035] in, is the global model parameter matrix of the server at round t+1, is the global model parameter matrix of the server in the tth round, η is the global learning rate, K is the number of terminals participating in federated learning, and N i is the number of important samples for each terminal, N is the total number of important samples for all terminals, ΔW i t is the gradient matrix of the parameters obtained in the tth round of joint fine-tuning. The joint gradient is then used to update the server model in a joint gradient descent manner, and the updated model is used as the initialization parameter for the next cycle. The joint training is continued in this way, and finally a global prototype model with general knowledge is obtained. This prototype model integrates important sample information of each terminal, has strong generalization ability, and can quickly adapt to the data distribution of each terminal through subsequent fine-tuning.

[0036] S4: Joint fine-tuning based on gradient correlation

[0037] Although the federated learning approach enables the model to indirectly train all data using the shared gradient method without directly sharing samples, in actual scenarios, samples from different terminals may have personalized distributions with local characteristics, which makes the jointly trained model unable to predict boundary samples of different terminals well. To solve this problem, in this example, the gradient correlation joint fine-tuning method is used to further fine-tune the terminal model, which is mainly divided into the following steps:

[0038] 4.1) Weighted Local Fine-tuning

[0039] The boundary samples are located between the two prototypes, near the decision boundary, and often have a greater impact on the division of the decision boundary. They are representative samples that can represent the local sample distribution of different terminals. In this example, the parameters of the general prototype of joint training are first used as the initialization parameters for fine-tuning of each terminal, and then the divided boundary samples and important samples are used to weight fine-tune the terminal model:

[0040]

[0041]

[0042] in, is the terminal model parameter matrix of the t+1th round, is the terminal model parameter matrix of the tth round, ΔW e t is the gradient matrix obtained by the important samples, α is the gradient weight given to the important samples, is the gradient matrix obtained by the boundary samples, and β is the gradient weight assigned to the boundary samples; by adjusting the values ​​of α and β, the influence of boundary samples and important samples on the decision boundary can be adjusted respectively.

[0043] 4.2) Joint fine-tuning based on similarity

[0044] Since the boundary samples of each terminal are limited, it is necessary to use the federated learning method to jointly train the boundary samples of each terminal. However, the boundary samples of each terminal represent the sample distribution characteristics of the terminal itself. The traditional federated learning method will make the trained model unable to solve the heterogeneity of different sample distributions of different terminals. To solve this problem, in this example, we no longer use all boundary samples for joint training, but instead jointly train terminals with similar distributions. The specific steps are as follows:

[0045] 4.2.1) The gradient obtained by the i-th terminal during the t-th round of fine-tuning is recorded as W i (t), select the gradient matrix G obtained in T rounds of fine-tuning to represent the terminal i , and then use the principal component analysis method to obtain the first principal component of each terminal as the convergence direction of each terminal;

[0046] 4.2.2) Figure 3 As shown, the difference between the first principal component directions d1 and d2 of the two terminal matrices is compared. Here, the angle is used for calculation. Assuming that the first principal component of the first terminal gradient matrix is ​​V1 and the first principal component of the other terminal matrix is ​​V2, the similarity ε is calculated using the following formula:

[0047]

[0048]

[0049] The larger the angle θ is, the more different the convergence directions of the two terminals are, and the closer the similarity ε is to 0, and vice versa. α is a hyperparameter for adjusting the decay speed. The larger α is, the faster the function decays at a larger angle. For example, if it is set to Or bigger.

[0050] 4.2.3) Jointly fine-tune the terminals with similar distributions. The specific formula is as follows;

[0051]

[0052] Among them, W j t is the model parameter of the jth terminal at the tth round of fine-tuning, η is the learning rate, K is the total number of terminals, ΔW i t is the gradient matrix of the i-th terminal during the t-th round of local fine-tuning.

[0053] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A federated learning method for Alzheimer's disease based on sample importance and gradient correlation, characterized in that: The following steps are involved: (1) Collecting MRI scan results of patients with and without Alzheimer's disease as a data set for model training, and obtaining corresponding encoding representations through an encoder; (2) For each terminal, use its own local data to train the prototype network stored in the terminal, and then calculate the distance between the sample and the prototype of Alzheimer's disease and the prototype of non-Alzheimer's disease, such as the squared Euclidean distance or cosine distance in Bregman divergence, etc., and divide the samples into three types according to the distance: important samples (points that are close to one prototype and far from the other prototype), boundary samples (points with similar distances from the two prototypes), and edge samples (points that are far from both prototypes); (3) Each terminal shares important samples through federated learning and jointly trains a global prototype network with broad adaptability. After the training is completed, the optimized global prototype network is sent to each terminal to initialize the subsequent fine-tuning stage of the terminal; (4) On the terminal, the global prototype network received from the server is used to perform fine-tuning using weighted important samples and boundary samples. During the fine-tuning process, a larger weight is given to the gradient change of the boundary samples, while a smaller weight is given to the gradient change of the important samples. Subsequently, the main direction of the gradient during the fine-tuning process is calculated by principal component analysis, and the correlation of the gradients between different terminals is further calculated. According to the similarity of the gradients, terminals with similar gradients are selected for joint fine-tuning to alleviate the overfitting problem caused by insufficient boundary samples; 2. The method for detecting Alzheimer's disease according to claim 1, characterized in that in step (3) and step (4), the global prototype network joint training based on sample importance and the terminal model fine-tuning based on gradient relevance, wherein the method for dividing the sample importance is: 2.1) On each terminal, the data set is first divided into a support set and a query set, which are used to train the prototype network and calculate the distance between the sample and the prototype respectively. For all samples of Alzheimer's disease and non-Alzheimer's disease in the support set, the mean of their encoding representation is calculated as the prototype representation of these two categories: Among them, x c1 and x c2 are the prototype representations of Alzheimer's disease and non-Alzheimer's disease, N1 and N2 represent the number of samples in the two categories, respectively. i and x j For each sample, the encoding representation is x. k , respectively calculate it and the two prototypes x c1 and x c2 Distance between: d1=dist(x k ,x c1 ) d2=dist(x k ,x c2 ) Where dist is the distance function. In this embodiment, Euclidean distance or cosine distance can be used. The smaller the distance, the closer the sample is to the prototype of the class, indicating that it is more likely to belong to the class; then, we use the SoftMax function to normalize the distance and calculate the probability representation of the sample belonging to each prototype; then based on this probability, back propagation is performed through the cross entropy loss function to update the encoder parameters, and iterate the step 2.1) until the prototype converges. 2.2) Once the local prototype network is trained, we can divide the importance of samples through the prototype network. As shown in Figure 2, the samples are divided into three categories by the Euclidean distance between the sample encoding representation and various prototypes: important samples close to the prototype, boundary samples located at the decision boundary between the two prototypes, and edge samples far from both prototypes. In this example, after we convert the distance into a probability vector [y1, y2], we first regard the samples whose y1 and y2 are both less than the threshold as edge samples, and then use the absolute value of the difference between the predicted results as the criterion for dividing important samples and boundary samples: the smaller the absolute value, the more similar the distance to the two prototypes, and the more likely it is a boundary sample; the larger the absolute value, the closer the distance to a prototype, and the more likely it is an important sample. The calculation method of correlation is: 4.1) Weighted Local Fine-tuning The boundary samples are located between the two prototypes, near the decision boundary, and often have a greater impact on the division of the decision boundary. They are representative samples that can represent the local sample distribution of different terminals. In this example, the parameters of the general prototype of joint training are first used as the initialization parameters for fine-tuning of each terminal, and then the divided boundary samples and important samples are used to weight fine-tune the terminal model: in, is the terminal model parameter matrix of the t+1th round, is the terminal model parameter matrix of round t, is the gradient matrix obtained by the important samples, α is the gradient weight given to the important samples, is the gradient matrix obtained by the boundary samples, and β is the gradient weight assigned to the boundary samples; by adjusting the values ​​of α and β, the influence of boundary samples and important samples on the decision boundary can be adjusted respectively. 4.2) Joint fine-tuning based on similarity Since the boundary samples of each terminal are limited, it is necessary to use the federated learning method to jointly train the boundary samples of each terminal. However, the boundary samples of each terminal represent the sample distribution characteristics of the terminal itself. The traditional federated learning method will make the trained model unable to solve the heterogeneity of different sample distributions of different terminals. To solve this problem, in this example, we no longer use all boundary samples for joint training, but instead jointly train terminals with similar distributions. The specific steps are as follows: 4.2.1) The gradient obtained by the i-th terminal during the t-th round of fine-tuning is recorded as W i (t) , select the gradient matrix G obtained in T rounds of fine-tuning to represent the terminal i , and then use the principal component analysis method to obtain the first principal component of each terminal as the convergence direction of each terminal; 4.2.2), as shown in Figure 3, compare the differences in the first principal component directions d1 and d2 of the two terminal matrices. Here, the angle is used for calculation. Assuming that the first principal component of the first terminal gradient matrix is ​​V1, and the first principal component of the other terminal matrix is ​​V2, the similarity ε is calculated using the following formula: in, The larger the angle θ is, the more different the convergence directions of the two terminals are, and the closer the similarity ε is to 0, and vice versa. α is a hyperparameter for adjusting the decay speed. The larger α is, the faster the function decays at a larger angle. For example, if it is set to Or bigger. 4.2.3) Jointly fine-tune the terminals with similar distributions. The specific formula is as follows; in, W j t is the model parameter of the jth terminal in the tth round of fine-tuning, η is the learning rate, K is the total number of terminals, ΔW i t is the gradient matrix of the i-th terminal during the t-th round of local fine-tuning.

Images