Multi-center diffusion magnetic resonance data analysis method based on federal learning

Abstract

The invention belongs to the technical field of medical image data processing, and particularly relates to a multi-center diffusion magnetic resonance data analysis method based on federal learning. According to the multi-center diffusion magnetic resonance data analysis method based on federated learning, an improved model and a new federated learning training algorithm are provided for the problem of non-independent identical distribution of diffusion magnetic resonance brain image data in a multi-center data conjoint analysis scene, and multi-center diffusion magnetic resonance data analysis is achieved. And reference data is provided for subsequent diagnosis.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image data processing, and specifically is a multi-center diffusion magnetic resonance data analysis method based on federated learning. Background Art

[0002] Federated Learning (FL) is a distributed machine learning method and one of the solutions to the joint analysis of multi-center data. It allows multiple participants (such as hospitals, companies, or personal devices) to collaboratively train a global model without sharing their local data to improve model performance. It solves the problem of data shortage caused by less data in a single center and data transmission between centers is not allowed. Unlike traditional centralized methods, federated learning disperses the model training process to where the data is located. The data never leaves the local device or server, and only shares the model updates (such as gradients or model parameters), thereby protecting data privacy and being able to use more data for model training to improve the accuracy of the model.

[0003] The process of federated learning usually includes the following steps: 1) Initialize the global model: The central server initializes a global model and sends it to each participant. 2) Local training: Each participant uses its local data to perform local training based on the global model and calculates the model update. 3) Upload updates: Each participant sends the model update parameters (not the original data) back to the central server. 4) Model aggregation: The central server summarizes the model update parameters of all participants and updates the global model. 5) Iteration process: The above process is repeated until the model converges or reaches a predetermined accuracy.

[0004] In the medical field, due to the restrictions of laws and regulations and the protection of patient privacy, medical data is often not shared across institutions, which limits the application of artificial intelligence in the medical field. Federated learning can solve the problem of medical data privacy and develop a new paradigm for the application of artificial intelligence. Through federated learning, hospitals can use their own patient data to train models locally without exchanging sensitive patient data. Ultimately, all centers are aggregated into a global model with excellent performance, and each center obtains better model performance.

[0005] In recent years, the related technologies of federated learning include: 1) FedAvg (Federated Averaging) is one of the most commonly used and basic algorithms in federated learning. It updates the global model by simply averaging the local model updates, thereby realizing distributed training of the model. However, the limitation of FedAvg is that the problem of data heterogeneity has not been well solved. The local data distribution of the participants is usually different (data heterogeneity), and the data feature distribution, data types, and the number of each type owned by different centers are different, which may cause the global model to be unable to adapt well to the needs of all participants. 2) FedBN (Federated Batch Normalization) is an improved algorithm for the problem of data heterogeneity in a federated learning environment. In order to deal with the problem of non-independent and identically distributed data, in FedBN, each participant in federated learning maintains the normalization layer parameters (such as mean and variance) of its own local model, and does not share these parameters with other participants. The limitation of FedBN is that it only considers the characteristics of non-independent and identically distributed data, and does not consider the scale and value of data owned by different centers. In extreme cases, some centers have a large amount of data, but some centers have very little data, and the quality of the data cannot be evaluated. The contribution of each center in the global model training process cannot be considered. Previous studies have achieved the joint training of global models by different centers in a multi-center scenario, and have also tried to solve the data heterogeneity between different centers to obtain better global model performance. However, at present, the problem has not been completely solved. Even if each center trains its own normalization layer, it still does not adapt well to the characteristics of diffusion MRI data.

[0006] At present, there is no solution for the joint analysis of multi-center diffusion MRI data in a multi-center scenario. Diffusion MRI, as a type of magnetic resonance imaging, is currently the only method that can use fiber bundle reconstruction technology to observe brain nerve fiber bundles in the living brain, providing a new perspective for the analysis of brain and nerve diseases. Taking the neurological disease autism as an example, this disease spectrum is diverse and clinical diagnosis is difficult. No relevant biomarkers have been found. Studies have shown that artificial intelligence technology can assist in the analysis of clinical disease data. Previous studies have attempted to use diffusion MRI to analyze autism through machine learning methods, but only a single center has not been combined with multiple centers for joint analysis, and it is unable to deal with the problem of non-independent and identically distributed data. Summary of the invention

[0007] In response to the above problems, the present invention proposes a multi-center diffusion magnetic resonance imaging data analysis method based on federated learning. In the scenario of multi-center data joint analysis, an improved model and a new federated learning training algorithm are proposed for the non-independent and identically distributed problem of diffusion magnetic resonance brain imaging data, so as to realize multi-center diffusion magnetic resonance imaging data analysis and provide reference data for clinical disease analysis.

[0008] The technical solution of the present invention is:

[0009] A multi-center diffusion magnetic resonance data analysis method based on federated learning comprises the following steps:

[0010] S1. Build a multi-center data analysis system consisting of one server and multiple centers, and each center obtains a specified type of diffusion magnetic resonance data locally as training data, and each center exchanges data with the server separately;

[0011] S2. Each center preprocesses the input training data, specifically obtaining the whole-brain fiber bundle reconstruction results of each individual based on the input diffusion magnetic resonance data. Each individual is segmented into 1516 fiber bundle regions by the ORG template atlas, and 26 microscopic diffusion indices of the fiber bundles in each fiber bundle region are obtained, including the anisotropy fraction FA, the homogeneity fraction MD, and the number of fiber bundles. Finally, a 1516*26 feature matrix is ​​obtained for each individual.

[0012] S3. Build a deep learning model with the same structure in each center. The model includes the first convolution layer, the first normalization layer, the first attention module, the second convolution layer, the second normalization layer, the second attention module, the third convolution layer, the third normalization layer, the third attention module, the first fully connected layer, the fourth normalization layer, the second fully connected layer, and the classification layer, which are connected in sequence. The input of the model is the feature matrix obtained in S2, and the prediction result category is obtained through the classification layer.

[0013] S4. Use the federated algorithm to train the model of each center in the multi-center data analysis system, including:

[0014] After each center obtains the feature matrix, it determines whether the data needs to be repaired. The judgment method is to determine whether the fiber bundle detection value is less than 75%. If so, the repair strategy is started, otherwise it goes directly to the next step; the detection value refers to the number of fiber bundle areas with a fiber bundle number greater than 10 divided by the number of all fiber bundle areas;

[0015]

[0016] detection is the detection value, count is the number of calculated fiber bundle areas, cluster ≥10 Cluster is a fiber bundle region containing more than 10 fiber bundles. all is the total number of fiber bundle regions.

[0017] The patching strategy is specifically as follows: for each feature of a fiber bundle region with a value of 0, according to the geometric distance information and the anatomical structure information, find the intersection of 30 neighboring fiber bundle regions with the closest geometric distance to the fiber bundle region and all fiber bundle regions connected with two common anatomical positions, and then average their feature values ​​as the value of the feature of the fiber bundle region, until all the features of all fiber bundle regions with a value of 0 are patched;

[0018] The central server inquires about the data size and data value of each center. Each center performs principal component analysis (PCA) on the healthy individual data to reduce the dimension to two dimensions and calculates the variance of all data. The central server calculates the weight of the i-th center training based on the reported data using the formula:

[0019]

[0020] scale i is the data size of the ith center, scale i is the data value of the ith center;

[0021] The server initializes the global model;

[0022] The server distributes the global model to each center;

[0023] Each center initializes the local model according to the global model parameters, but retains its own locally adapted normalization layer and trains for 10 rounds using local data;

[0024] After each center completes training, it returns the rest of the model parameters to the server except for the normalization layer;

[0025] The server multiplies the training weight of each center by the model parameter of the corresponding center, sums them up and assigns them to the global model;

[0026] After reaching the set number of times, each center uses the model with the best performance during the training process as the model for prediction by the center;

[0027] S5. Each center analyzes the input diffusion magnetic resonance data based on the obtained prediction model.

[0028] The beneficial effect of the present invention is that the present invention proposes for the first time a multi-center diffusion magnetic resonance data analysis method based on federated learning, which can combine multi-center diffusion magnetic resonance data without transmitting the original data to protect privacy, and perform deep learning model training through improved models and federated learning training algorithms to realize the analysis of diffusion magnetic resonance data. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Schematic diagram of the deep learning model structure in the present invention.

[0030] Figure 2 Schematic diagram of the structure of the attention module in the deep learning model.

[0031] Figure 3 This is a schematic diagram of the multi-center topology structure of the federal system.

[0032] Figure 4 This is a flowchart of the multi-center data federation analysis strategy. DETAILED DESCRIPTION

[0033] The scheme of the present invention is further described below in conjunction with the accompanying drawings.

[0034] The present invention mainly includes the following aspects:

[0035] 1. Data preprocessing of each center

[0036] Data preprocessing: Each center uses the white matter analysis pipeline to process data. The specific steps are as follows: 1) The input is the diffusion magnetic resonance data of the individual to obtain the whole-brain fiber bundle reconstruction results of each individual. 2) The whole-brain fiber bundle reconstruction results of each individual are divided into 1516 fiber bundle regions by the ORG brain template atlas in the tool (of which 716 regions can be divided into 716*2=1432 regions due to the symmetry of the left and right hemispheres, and the other 84 regions are shared by the left and right brains, a total of 1516 regions), and 26 microscopic diffusion indicators of the fiber bundles in each region are obtained, such as various anisotropy fractions FA, various homogeneity fractions MD, and steamline number of fiber bundles. Finally, each individual obtains a 1516*26 feature matrix.

[0037] 2. Construction of each center model

[0038] Each center uses the same deep learning model, which is a modified version of 1DCNN, consisting of convolutional layers, attention modules, normalization layers, and fully connected layers. The network structure is as follows: Figure 1 As shown in Figure 1, it includes convolutional layer, normalization layer, attention module, fully connected layer and classification layer. The structure of the attention module is as follows: Figure 2 As shown in the figure, it includes global average pooling, fully connected layer, activation layer, fully connected layer and activation layer connected in sequence. The network input is a feature matrix of 1516 (1516 brain white matter fiber bundle areas) * 26 (26 microscopic indicators of each fiber bundle). Finally, the classification layer outputs the predicted probability of two categories (autism, non-autism), and the one with a larger probability is the prediction result.

[0039] 3. Federated Model Training

[0040] Federated algorithm: When training the model, federated learning is performed according to the federated strategy. The topology of the federated device is as follows: Figure 3 The star structure is shown in Figure 1. Each center communicates with the central server only to transmit model parameters. The purpose is to use federated training to obtain better model training results without sharing the private data of each center. Figure 4 As shown, the algorithm steps are as follows:

[0041] a) Each center determines whether the data of the center needs to be repaired. Focus on the number of fiber bundles in the micro-indicators. For an individual, the ratio of the number of fiber bundle areas (ckusters) with less than 10 fiber bundles to the number of all fiber bundle areas is called the detection value. If the average detection value of all individuals in a center is less than 75%, then the repair strategy is initiated.

[0042]

[0043] Repair strategy: For each feature with a value of 0 in a fiber bundle region, according to the geometric distance information and anatomical structure information, find the intersection of the 30 neighboring fiber bundle regions with the closest geometric distance to the fiber bundle region and all fiber bundle regions with two common anatomical positions, and then average their feature values ​​as the feature value of the fiber bundle region. This process continues until all the feature values ​​with a value of 0 in all fiber bundle regions are repaired.

[0044] b) The central server asks each center about the data scale and data value. Each center performs PCA dimensionality reduction on the healthy individual data, and calculates the variance of all data after reducing it to two dimensions. The central server uses the following formula to calculate the weight of each center during training based on the reported data.

[0045]

[0046] c) The central server initializes the global model.

[0047] d) The central server distributes the global model to each center.

[0048] e) Each center initializes the local model according to the global model parameters, uses its own local normalization layer, and trains for 10 rounds using local data.

[0049] f) After each center completes training, except for the normalization layer, the trained model parameters are returned to the central server

[0050] g) The central server aggregates the model parameters of all centers according to the training weight of each center and assigns them to the global model.

[0051] h) Repeat the above steps dg until the global round reaches the target round.

[0052] 4. Data Analysis

[0053] Each center uses the best performing model in the training process as the prediction model to analyze the diffusion magnetic resonance imaging data of local patients. The federated model training is restarted regularly according to the growth rate of the center's data.

Claims

1. A multi-center diffusion magnetic resonance data analysis method based on federated learning, characterized in that: The following steps are involved: S1. Build a multi-center data analysis system consisting of one server and multiple centers, and each center obtains a specified type of diffusion magnetic resonance data locally as training data, and each center exchanges data with the server separately; S2. Each center preprocesses the input training data, specifically obtaining the whole-brain fiber bundle reconstruction results of each individual based on the input diffusion magnetic resonance data. Each individual is divided into 1516 clusters by the ORG template atlas, and the 26 microscopic diffusion indices of the fiber bundles of each cluster, including the anisotropy fraction FA, the homogeneity fraction MD, and the number of fiber bundles, are obtained. Finally, a 1516*26 feature matrix is ​​obtained for each individual. S3. Build a deep learning model with the same structure in each center. The model includes the first convolution layer, the first normalization layer, the first attention module, the second convolution layer, the second normalization layer, the second attention module, the third convolution layer, the third normalization layer, the third attention module, the first fully connected layer, the fourth normalization layer, the second fully connected layer, and the classification layer, which are connected in sequence. The input of the model is the feature matrix obtained in S2, and the prediction result category is obtained through the classification layer. S4. Use the federated algorithm to train the model of each center in the multi-center data analysis system, including: After each center obtains the feature matrix, it determines whether the data needs to be repaired. The judgment method is to determine whether the detection value is less than 75%. If so, the repair strategy is started, otherwise it goes directly to the next step; the detection value refers to the ratio of the number of clusters with less than 10 fiber bundles to the total number of clusters; the repair strategy is specifically: for each cluster with a value of 0, according to the geometric distance information and anatomical structure information, find the intersection of the 30 neighbor clusters with the closest geometric distance to the cluster and all clusters connected by two common anatomical positions, and then average their feature values ​​as the value of the cluster feature, until all the features with a value of 0 in all clusters are repaired; The central server inquires about the data size and data value of each center. Each center performs PCA dimensionality reduction on the healthy individual data, and calculates the variance of all data after reducing it to two dimensions. The central server calculates the weight of the i-th center training based on the reported data using the formula: scale i is the data size of the i-th center, value i is the data value of the ith center; The server initializes the global model; The server distributes the global model to each center; Each center initializes the local model according to the global model parameters, adapts its own local batchnorm layer, and trains 10 epochs using local data; After each center completes training, except for the batchnorm layer, the trained model parameters are returned to the server; The server aggregates the model parameters of all centers according to the training weight of each center and assigns them to the global model; After reaching the set number of times, each center uses the model with the best performance during the training process as the prediction model; S5. Analyze the input diffusion magnetic resonance data according to the obtained prediction model.

Images