A training method and an application method of a distributed medical image processing model

Abstract

The application discloses a kind of training method and application method of distributed medical image processing model, comprising: collecting the medical image data from multiple medical centers after being labeled and carrying out data preprocessing, obtain the local data set of each medical center, generate global data set according to local data set;According to local data set, generate local adversarial image sample;Based on variational bayes framework, construct initial distributed medical image processing model;Local data set and local adversarial image sample are used to carry out local training to local model, when each iteration reaches preset round, each local model is globally aggregated and processed, and global model is formed;When the local loss function of each local model converges or the iteration number reaches the preset second iteration number threshold, the target distributed medical image processing model is obtained.The present application can train large-scale data, improve the robustness of model accuracy model, and can be widely applied in image processing technical field.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a training method and application method of a distributed medical image processing model. BACKGROUND

[0002] In recent years, with the continuous improvement of medical image diagnosis performance requirements and the development of deep learning, a large number of AI-assisted diagnosis systems for medical images can improve the accuracy of diagnosis and protect the privacy of patients to some extent. However, they also have the following shortcomings:

[0003] 1. The medical data of each hospital is scattered, and the sharing degree is not high, making it difficult to centrally train large-scale data. AI needs a large amount of data to achieve high accuracy, but in fact, the medical data available for training is very scattered, and the data volume is unbalanced in cities and remote areas.

[0004] 2. The model is not robust enough, the clinical trust is low, and the actual deployment of the model is difficult to promote. Existing models are often based on various deep neural networks, and research shows that these models will expose weaknesses under carefully designed adversarial perturbations. The data quality problem and potential adversarial attacks collected in remote areas are more difficult to detect, so the robustness and generalization performance of the model cannot meet the requirements.

[0005] 3. The model lacks explainability, and it is not possible to determine whether the model is trained according to the requirements. Some AI algorithm models often lack sufficient explainability when making predictions or diagnoses, and cannot generate reasonable predictions that doctors can explain. This makes it difficult for doctors to understand the decision-making process of the model, and thus doubts or distrust the results. SUMMARY

[0006] The present application aims to at least solve one of the technical problems in the related art. To this end, the present application proposes a training method and application method of a distributed medical image processing model suitable for training large-scale scattered data and capable of improving the accuracy of the model.

[0007] In one aspect, the present application embodiment provides a training method of a distributed medical image processing model, comprising:

[0008] Collecting labeled medical image data from multiple medical centers, pre-processing the medical image data to obtain local data sets of each medical center, and generating a global data set according to the local data sets;

[0009] Generating local adversarial image samples according to the local data sets;

[0010] constructing an initial distributed medical image processing model based on a variational Bayesian framework; wherein the initial distributed medical image processing model comprises a plurality of local models corresponding to the medical centers one by one;

[0011] performing local training on the local models using the local data sets and the local adversarial image samples, and performing global aggregation processing on each of the local models when the number of iterations reaches a preset number of rounds, to form a global model;

[0012] obtaining a target distributed medical image processing model when the local loss function of each of the local models converges or the number of iterations reaches a preset second iteration threshold; wherein the target distributed medical image processing model comprises a global model and a plurality of local models corresponding to the medical centers one by one.

[0013] Optionally, the data preprocessing of the medical image data to obtain a local data set, and generating a global data set according to the local data set, comprises:

[0014] performing a morphological operation of random combination on the medical image data of a plurality of medical centers to obtain a local data set; wherein the morphological operation comprises at least one of the following: an opening operation, a closing operation, a random dilation operation or a random erosion operation;

[0015] determining a plurality of samples from the local data set to generate a global data set;

[0016] performing data enhancement on the local data set and the global data set.

[0017] Optionally, the generating of the local adversarial image sample according to the local data set comprises:

[0018] configuring a perturbation term;

[0019] adding the perturbation term to the local data set to generate a first adversarial image sample;

[0020] iteratively processing the perturbation term according to the first adversarial image sample;

[0021] using a gradient descent algorithm, repeatedly performing the step of adding the perturbation term to the local data set to generate a first adversarial image sample in a small step gradient descent manner according to a preset time interval, until the number of iterations reaches a preset first iteration threshold, to obtain an optimal perturbation value;

[0022] adding the optimal perturbation value to the local data set to generate a local adversarial image sample.

[0023] Optionally, the global adversarial image sample is generated according to the global data set, the perturbation term is added to the local data set, and a calculation formula for generating the first adversarial image sample is:

[0024]

[0025] wherein x' represents the first adversarial image sample, epsilon represents a positive number approaching 0, sign() represents a sign function, L() represents a loss function of an original model, that is, a gradient calculation of the loss function with respect to input x; x represents an original sample, and y represents a sample output after the original sample is operated by the model.

[0026] Optionally, in the step of constructing the initial distributed medical image processing model based on the variational Bayesian framework, the following steps of constructing a local model are included:

[0027] A segmentation network is constructed for generating a probability distribution prediction of general data; wherein the segmentation network is a convolutional neural network;

[0028] A prior encoder and a posterior encoder are constructed; wherein the prior encoder is used to calculate a global loss function of a global model; and the posterior encoder is used to calculate a local loss function of a local model;

[0029] A distribution adaptive network model is constructed for outputting an adaptive matrix to calculate a final prediction result.

[0030] Optionally, the local training of the local model by using the local data set and the local adversarial image sample includes:

[0031] The local loss function is configured based on a gradient descent algorithm and an adversarial loss;

[0032] The local data set and the local adversarial image sample are combined as a training set;

[0033] The distributed medical image processing model is locally trained by using the training set to obtain a model training result;

[0034] Based on the local loss function, a loss value is calculated according to the model training result.

[0035] Optionally, the expression of the local loss function is:

[0036] loss new =l CE +l NR +αl TR +βl KL +λl ADV

[0037] wherein, l CE is a cross-entropy loss; l NR is a non-overlapping loss; l TR is a constraint value; l KL represents a difference between a prior distribution and a posterior distribution; and a, b and l are hyperparameters; l ADV is a loss function of the generative adversarial network.

[0038] In another aspect, the embodiment of the present application also provides an application method of the distributed medical image processing model, comprising:

[0039] obtaining a medical image to be processed;

[0040] inputting the medical image to be processed into a target distributed medical image processing model trained by the training method provided by the embodiment of the present application for segmentation prediction processing, and obtaining an image processing result.

[0041] In another aspect, the embodiment of the present application also provides a training system of a distributed medical image processing model, comprising:

[0042] A first module is configured to collect labeled medical image data from a plurality of medical centers, perform data preprocessing on the medical image data, obtain a local data set of each medical center, and generate a global data set according to the local data set;

[0043] A second module is configured to generate local adversarial image samples according to the local data set;

[0044] A third module is configured to construct an initial distributed medical image processing model based on a variational Bayesian framework, wherein the initial distributed medical image processing model comprises a plurality of local models corresponding to the medical centers one by one;

[0045] A fourth module is configured to perform local training on the local models using the local data set and the local adversarial image samples, perform global aggregation processing on each local model to form a global model when the number of iterations reaches a preset number of rounds, and obtain a target distributed medical image processing model when the local loss function of each local model converges or the number of iterations reaches a preset second iteration threshold, wherein the target distributed medical image processing model comprises a global model and a plurality of local models corresponding to the medical centers one by one.

[0046] In another aspect, the embodiment of the present application also provides an electronic device, comprising a processor and a memory, wherein the memory is configured to store a program, and the processor is configured to execute the program to implement the method described above.

[0047] In another aspect, the embodiment of the present application also provides a computer storage medium, which stores a processor-executable program, and the processor-executable program is used to implement the method as described above when executed by a processor.

[0048] The embodiment of the present application has the following beneficial effects: a general global model can be trained based on the respective data of different medical centers, and the prediction results of the global model are corrected by training local models of the medical centers, which is suitable for training large-scale distributed data and can improve the accuracy of the model; at the same time, since data aggregation is not required, the data privacy security in the training process can also be improved. By adding the adversarial training of the adversarial image samples, the robustness of the distributed medical image processing model can also be improved. BRIEF DESCRIPTION OF DRAWINGS

[0049] The accompanying drawings are used to provide a further understanding of the technical solutions of the present application, and constitute a part of the specification, and are used to explain the technical solutions of the present application together with the embodiments of the present application, and do not constitute a limitation on the technical solutions of the present application.

[0050] Figure 1 is a step diagram of the training method of the distributed medical image processing model provided by the embodiment of the present application;

[0051] Figure 2 is a flowchart of the training method of the distributed medical image processing model provided by the embodiment of the present application;

[0052] Figure 3 is a model structure diagram of the distributed medical image processing model provided by the embodiment of the present application;

[0053] Figure 4 is a step diagram of the application method of the distributed medical image processing model provided by the embodiment of the present application;

[0054] Figure 5 is a training system structure diagram of the distributed medical image processing model provided by the embodiment of the present application;

[0055] Figure 6 is an electronic device structure diagram provided by the embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and do not limit the present application.

[0057] It is to be understood that, even though a function module division is made in the system schematic diagram, and a logical sequence is shown in the flow chart, in some cases, the steps shown or described can be performed in a manner different from the module division in the system, or the sequence in the flow chart. The terms "first / S100", "second / S200", etc. in the specification and claims and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific sequence or order.

[0058] Reference herein to "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase that the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of other embodiments. It is explicitly contemplated that embodiments described herein can be combined with other embodiments.

[0059] Before introducing, first introduce the meaning of robust optimization:

[0060] Robust Optimization: Robust optimization is a kind of programming method whose purpose is to find a solution that is suitable for all constraints, so that the objective function is optimal in the worst case. Mathematical programming needs to input data to establish a model under the condition of accurate knowledge, and to obtain the optimal solution by using existing mathematical programming methods, which do not consider the influence of data uncertainty. Robust optimization establishes an uncertainty set for uncertain variables in programming, and uses related optimization theory such as duality theory to transform it, so that the uncertainty problem is transformed into a solvable robust counterpart problem, and the robust optimal solution is given.

[0061] With reference to Figure 1 and Figure 2 The method comprises the following steps:

[0062] S100, collecting medical image data of a plurality of medical centers, pre-processing the medical image data to obtain a local data set of each medical center, and generating a global data set according to the local data set.

[0063] Optionally, the step S100 comprises the following steps S110-S130.

[0064] S110, performing a random combination morphological operation on the medical image data of a plurality of medical centers to obtain a local data set; wherein the morphological operation comprises at least one of the following: opening operation, closing operation, random dilation operation or random erosion operation.

[0065] Erosion operation is to shrink objects in an image by using a structuring element (also known as kernel). The operation convolves the structuring element with the original image and sets the pixel value of the convolved image to the minimum pixel value within the structuring element. Erosion operation can be used to remove noise in the image, shrink the size of objects, separate connected objects, or detect boundaries. It makes objects thinner or shrinks them by removing their edges.

[0066] While dilation operation is the opposite of erosion operation, it expands objects in an image by using a structuring element. The operation sets the pixel value after convolution to the maximum pixel value within the structuring element. Dilation operation makes objects thicker or expands them by extending pixels at their edges, and can be used to connect incomplete objects or increase the size of objects.

[0067] Based on this, the morphological operation performed by the embodiment of the present application on medical image data can be:

[0068] Opening operation: a combination of first erosion operation and then dilation operation. It is mainly used to remove small noise points or small discontinuous regions in the image, and can smooth the boundaries of the image.

[0069] Closing operation: a combination of first dilation operation and then erosion operation. It is mainly used to fill small holes in the image or connect incomplete objects.

[0070] Random dilation operation: a variant of dilation operation that introduces randomness. While the usual dilation operation uses a fixed size structuring element to expand objects in the image, the random dilation operation randomly selects the size and shape of the structuring element for dilation, which can simulate the natural shape changes of objects or the influence of noise to some extent. Random erosion operation is similar to random dilation operation, which is a variant of erosion operation that introduces randomness, and both have similar functions.

[0071] The morphological operations can be randomly combined from at least one of the above, and the medical image data is morphologically processed according to the randomly combined operation to simulate the common bias and changes in labeling between different medical centers.

[0072] S120, determining a plurality of samples from the local data set to generate a global data set.

[0073] Each local data set corresponds to a medical center, and the data volume can be very large, so the embodiment of the present application selects and determines a plurality of samples from the local data set to form a global data set for verifying the performance of the global model.

[0074] S130, performing data augmentation on the local data set and the global data set.

[0075] In some embodiments, all processed medical image data can be resampled to a certain resolution, for example, 0.6x0.6x1.25mm, and then cropped to an appropriate size, for example, 256x256, with the region to be segmented as the center. Then the cropped image data is subjected to Z-score normalization. In the training process, the embodiments of the present application use methods such as random rotation, flipping, elastic deformation and adding Gaussian noise to perform data augmentation on the data.

[0076] S200, generating a local adversarial image sample according to the local data set.

[0077] Specifically, step S200 can include steps S210-S240.

[0078] S210, configuring a perturbation term.

[0079] In some embodiments, the configured perturbation term can be:

[0080]

[0081] wherein η is the added perturbation term. ε represents a positive number approaching 0; sign() represents a sign function, which outputs 1 when the input is ≥0, and otherwise outputs -1; L() represents the loss function of the original model, i.e., the gradient calculation of the loss function with respect to the original sample x.

[0082] In order to make the subsequently generated adversarial image sample not be recognized by the machine, η should be small enough. Here, the infinite order norm is used to express that η is small enough, as shown in the following formula:

[0083] ||η|| ∞ <ε#(2)

[0084] S220, adding the perturbation term to the local data set to generate a first adversarial image sample.

[0085] Let x be the original sample, i.e., the original image data of each medical center; x' be the first adversarial image sample. The calculation formula for generating the first adversarial image sample is:

[0086]

[0087] The way of generating adversarial image samples in steps S210-S220 can be explained as a single-step attack of internal expectation maximization, but from the perspective of improving robustness, multi-step attacks will be more effective in improving the robustness of the model, because the distributed training framework constructed subsequently by the embodiments of the present application is a complex nonlinear mapping.

[0088] Therefore, after adding the adversarial sample in the sample training, the PGD algorithm is used to explore most cases of the loss function in a small step gradient descent manner, and is iterated multiple times step by step in a predetermined time interval to find the most suitable adversarial sample.

[0089] PGD (Projected Gradient Descent) is a gradient descent algorithm in adversarial training. In deep learning, after careful training of the neural network, the classification accuracy can be very good, but the robustness may be poor, and it may be easily broken by adversarial attacks. That is, by making a small perturbation to the input picture, the classification accuracy of the neural network can be greatly reduced under the premise that there is almost no difference in the naked eye. The adversarial training algorithm adopts a small step and multiple walks strategy to attack, aiming to find an optimal solution to make the model more robust in the adversarial environment. The core idea is to add a small amount of perturbation to the input data at each training step to maximize the interference or misjudgment of the original model. These perturbations are based on the gradient calculation of the model. The PGD algorithm maximizes the influence of the adversarial by iteratively updating the perturbation. The goal is to increase the robustness of the model to adversarial samples during training, thereby improving the security and credibility of the model. By constantly making the model face adversarial samples and optimizing training, the PGD algorithm can help the model better recognize and process input data with certain perturbations, so that the model can better cope with input data with certain attacks.

[0090] Thus, steps S230-S240 are obtained.

[0091] S230, iteratively processing the perturbation term according to the first adversarial image sample.

[0092] The perturbation iteration expression of the embodiment of the application is:

[0093]

[0094] wherein the input at t+1 time is based on the input at t time and the gradient of the loss function with respect to x between t+1 time and t time is solved. means: first calculate the loss gradient of the original image sample to obtain the adversarial image sample, and then subtract the original image sample from the adversarial image sample to obtain the perturbation value; if the perturbation value exceeds a certain range, it needs to be mapped back to the specified range S. After multiple iterations, the generation of the adversarial image sample x' can achieve the optimal solution, that is, the strongest attack effect, which is beneficial to improve the robustness of the model after training.

[0095] S240, using a gradient descent algorithm, in a small step gradient descent manner, repeatedly performing the step of adding the perturbation term to the local data set according to a preset time interval to generate a first adversarial image sample until the number of iterations reaches a preset iteration threshold, and obtaining an optimal perturbation value.

[0096] S250, adding the optimal perturbation value to the local data set to generate a local adversarial image sample.

[0097] S300, constructing an initial distributed medical image processing model based on a variational Bayesian framework; wherein the initial distributed medical image processing model includes a plurality of local models corresponding to the medical centers one by one.

[0098] Non-identically Distributed Data refers to the data in a system or data set not meeting the same probability distribution or data generation process. This means that different data samples may have different probability distribution characteristics, such as statistical characteristics, distribution patterns, and data generation methods. This may pose challenges for machine learning and data analysis tasks, as traditional methods based on the assumption of independent and identically distributed data may no longer be applicable, and the non-identically distributed nature of the data needs to be considered.

[0099] Decoupling prediction aims to decompose complex prediction problems into multiple sub-problems and model and predict each sub-problem independently. The purpose is to improve the flexibility and overall performance of the prediction model by decoupling the dependencies between sub-problems. In decoupling prediction, first, multiple related but independent sub-problems need to be defined, and then independent models are established for each sub-problem. Each model can use different machine learning algorithms, feature selection methods or parameter settings to best solve the specific sub-problem. These models can be trained in parallel without interfering with each other. In the prediction stage, the decoupling prediction method passes the new input sample to each sub-model to generate one or more prediction results, and finally combines these prediction results to produce the final overall prediction. The advantage of decoupling prediction is that it can effectively solve the coupling and dependency in complex problems, by decomposing the problem into independent sub-problems, it can improve the interpretability and stability of the model, and it is easier to adjust and optimize each sub-model.

[0100] The embodiment of the present application constructs a distributed medical image processing model based on a variational Bayesian framework. The model is a distributed learning decoupling prediction model, which connects global methods and personalized methods, and can be used for segmentation of medical image data that does not meet the independent and identically distributed assumption (NDD). The variational Bayesian framework measures the difference between the fitted distribution and the true posterior distribution by minimizing the variational divergence. The variational divergence is a distance measure between two probability distributions. Commonly used variational divergences include KL divergence and Hellinger distance. In order to minimize the variational divergence, an optimization problem can be defined to minimize the variational divergence by adjusting the parameters of the variational distribution. This optimization problem can usually be solved by an iterative algorithm. The global method refers to the aggregation of data from all medical centers in distributed learning to train a global model, and then applying the model to medical image processing in all medical centers. The personalized method refers to training a local model in each medical center, which is trained only using local data of the medical center and is only applied to image processing in the medical center. The distributed medical image processing model of the embodiment of the present application has a distributed conditional adaptive network, which can decouple the prediction sample and the label corresponding to the prediction sample during training, and adaptively predict according to the local distribution during testing.

[0101] Based on this, referring to Figure 3 , the initial distributed medical image processing model constructed by the embodiment of the present application can include a plurality of local models corresponding to medical centers one by one, and the global model is generated by a global aggregation operation during the training process of the model. Figure 3 In the learning task, the left side of the dashed line is the learning task, and the right side is the learning framework of the local model.

[0102] In the learning task, Figure 5 , task 1 is to aggregate and train a global model f(θ) for non-IDD data from multiple medical centers; and task 2 is to find a local model for each medical center. The global model of the embodiment of the present application realizes unified segmentation of general data distribution p(x,y), that is, the medical image of the medical center that is not used for training can also be effectively segmented. On the other hand, the local model can modify the prediction result of the global model, so that the prediction result is more inclined to the local distribution, for example, the kth center p(x k ,y k ).

[0103] The construction steps of the local model can include the following steps a-c.

[0104] a) Construct a segmentation network for generating probability distribution prediction of general data; wherein the segmentation network is a convolutional neural network.

[0105] The segmentation network is used to generate the prediction of the general data. It is a convolutional neural network which takes x as input and outputs a probability distribution p(y|x). The probability distribution represents the probability distribution of the output y given the input x.

[0106] In the embodiment of the present application, the general data is data which can be applied to different local models and global model training, and the data has wide applicability and is usually formed by selecting data of a certain medical center in multiple centers, that is, the general data is data extracted from a certain medical center.

[0107] The convolutional neural network (Convolutional Neural Network) is a deep feedforward neural network with local connection and weight sharing, which is good at processing images, especially image recognition and other related machine learning problems, such as image classification, target detection, image segmentation and various visual tasks, which has a significant improvement effect, and is one of the most widely used models. It has the ability of representation learning, can perform translation-invariant classification on input information according to its hierarchical structure, can perform supervised learning and unsupervised learning, and the convolution kernel parameters in the hidden layer are shared and the sparsity of the interlayer connection makes the convolutional neural network can learn the lattice features such as pixels and audio with small amount of calculation, has stable effect and no additional feature engineering requirement for data.

[0108] b) constructing a priori encoder and posteriori encoder, wherein the a priori encoder is used to calculate the global loss function of the global model; and the posteriori encoder is used to calculate the local loss function of the local model.

[0109] In the embodiment of the present application, the joint data refers to a set of data from different centers. These data are stored in different places and may not be concentrated in one place for processing, so this kind of data is specially called joint data, that is, the joint data refers to the data of all medical centers.

[0110] The a priori encoder and the posteriori encoder can be modeled through a latent variable z k The joint data distribution p(x k ,y k ) and q(x k ,y j ) are modeled, that is, p ψk and in the figure. The two encoders are a kind of neural network, and the functions of both are to take the input x and output y as input and output a latent variable z; the latent variable z represents the relationship between the input x and the output y. The difference between the a priori encoder and the posteriori encoder is that the output result of the former is used to calculate the loss function of the global model, and the result of the latter is used to calculate the loss function of the personalized model (i.e. the local model).

[0111] c) constructing a distribution adaptive network model for outputting an adaptive matrix to calculate a final prediction result.

[0112] The distribution adaptive network model of the embodiment of the application is composed of a multi-layer perceptron (MLP), the input of which is the output results of the prior encoder and the posterior encoder and the prediction result of the general data generated by the segmentation network. The output of the MLP is an adaptive matrix W, which is used to adapt the prediction result of the general data to the distribution of the local data. W is calculated according to the relationship between the distribution of the local data and the prediction result of the general data.

[0113] Based on this, the output results of the above steps a and b can be used as the input of the DA network (distribution adaptive network) model. According to these new parameters and the distribution of the local data, the DA network can output an adaptive matrix W k . Finally, the prediction of the local data is obtained by multiplying the adaptive matrix W k and the prediction of the global data.

[0114] S400, locally training the local model using the local data set and the local adversarial image sample, and when the number of iterations reaches a preset number of rounds, performing global aggregation processing on each local model to form a global model.

[0115] The embodiment of the application trains a distributed medical image processing model based on swarm learning, including model global aggregation and local model training according to a certain period.

[0116] Swarm learning: Swarm learning is a distributed and decentralized machine learning method that completes the training and prediction tasks of the model through the cooperation and information sharing between edge devices, has the advantages of protecting privacy and avoiding data centralization. In swarm learning, each edge device is a learning node that has its own data set and computing power. First, all devices independently initialize and train the model locally; then, each edge device exchanges model parameters with each other through the network and combines local data for combined update. This exchange and combination of parameters can protect the security of data through encryption and de-privatization technology, which enables the model to learn in a distributed manner between devices, avoiding the problem of data centralization, and since data does not need to leave the device for centralized storage and processing, swarm learning has the advantage of protecting user privacy on the basis of fully utilizing the data sets of each node.

[0117] In global aggregation, each medical center k collects local update parameters of the global model from other medical centers and the model aggregation is weighted by the training size (n k ) of each medical center. The specific expression of global aggregation is:

[0118]

[0119] where, denotes the local model, which is kept in the local during the stage of global aggregation. While training locally, each medical center trains its own local model, and the global model is initialized as the aggregation result θ at this time.

[0120] In step S400, the step of locally training the local model using the local data set and the local adversarial image sample can include steps S410-S440.

[0121] S410, configure the local loss function based on the gradient descent algorithm and the adversarial loss.

[0122] The traditional loss function when joining the adversarial training algorithm can be determined in advance:

[0123] loss=l CE +l NR +αl TR +βl KL #(6)

[0124] where, l CE is the cross-entropy loss, which is applied to train the segmentation network and the DA network, and is used to calculate the cross-entropy loss between the product of the outputs of the two networks and the label.

[0125] l NR is the non-overlapping loss, which acts on the local model: in the data set of each medical center, the segmentation bias is regarded as noise, and it is added to the training data as a pseudo label, which can increase the amount of training data and balance the noise tolerance and convergence speed, thereby improving the generalization ability and robustness of the local model on multiple center data sets.

[0126] l TR is used to constrain the size and sparsity of the adaptive matrix W k , so as to avoid model overfitting, so that the global model is more consistent with the distribution characteristics of the local data set.

[0127] l KL is used to learn the difference between the prior distribution p(z) and the posterior distribution q(z|(x k , y k ), so as to better model the data distribution.

[0128] And alpha and beta are hyperparameters, used to balance the influence of matrix regularization and KL divergence loss on loss.

[0129] Further, the loss function loss is optimized based on the added PGD algorithm to further evaluate the robust performance of the model. The above optimization method can be: using the discriminator loss function l ADV of the adversarial generation network (GAN) to encourage the model to learn the ability to improve robustness, and the embodiment of the application assumes that D(x) represents the output of the discriminator network for the input x, and the adversarial loss can be determined as:

[0130] l ADV = -log(D(x')) (7)

[0131] By minimizing l ADV , the adversarial sample can be made more difficult to be distinguished by the discriminator network, thereby improving the robustness of the model.

[0132] Based on this, the expression of the optimized loss function loss new is:

[0133] loss new = l CE + l NR + alpha l TR + beta l KL + lambda l ADV # (8)

[0134] Where lambda is a hyperparameter used to balance the weight between adversarial loss and other losses.

[0135] S420, combine the local data set and the local adversarial image sample as a training set.

[0136] S430, using the training set to locally train the distributed medical image processing model to obtain a model training result.

[0137] Before model training, in addition to the regularization parameter, some necessary hyperparameters need to be set to make the model run, such as the number of iterations, the learning rate, and the perturbation gradient descent step.

[0138] S440, based on the local loss function, calculate the loss value according to the model training result.

[0139] S500, when the local loss function of each local model converges or the number of iterations reaches a preset second iteration threshold, a target distributed medical image processing model is obtained; wherein the target distributed medical image processing model includes a global model and a plurality of local models corresponding to the medical centers one by one.

[0140] The distributed training framework with the model robustness improvement strategy and the optimized loss function can be used to evaluate the performance of the framework. In some embodiments, the same sample is input into the model of the present embodiment and other conventional framework models, and the evaluation and comparison are performed in task 1 and task 2, respectively.

[0141] The distributed medical image processing model trained by the training method can be applied to various medical image processing scenarios, such as segmenting medical images, identifying target objects from medical images, etc. The training data set and training task input according to actual needs can be determined.

[0142] The training method of the present embodiment has the following advantages:

[0143] 1. The training method of the present embodiment can train a general global model based on the data of different medical centers, and correct the prediction results of the global model by training local models of each medical center, which is suitable for training large-scale dispersed data and can improve the accuracy of the model. At the same time, since data aggregation is not required, the data privacy security in the training process can be improved.

[0144] 2. The training method of the present embodiment is beneficial to breaking the data silos between hospitals and promoting data collaboration. By integrating medical image data of multiple remote area hospitals, pathological diagnosis can be more comprehensive, and the reliability and applicability of the medical image assisted diagnosis platform in pathological diagnosis can be improved. This provides convenience for patient referral, and makes medical resources more reasonably allocated and utilized.

[0145] 3. Robustness and reliability are improved: through the group learning technology and the adversarial training framework, the robustness and clinical trust of the medical image processing model are improved. The group learning technology allows multiple nodes to learn locally and then aggregate and share knowledge, expanding the data scale of model training and improving the robustness and reliability of the model. The adversarial training framework further improves the robustness and generalization performance of the model.

[0146] 4. Balance between individualization and generalization: the present embodiment takes into account the local particularity of the health status of different remote area populations. Through methods such as random combination of morphological operations in the data preprocessing process, the present embodiment balances the requirements of the model for generalization and individualization, which is beneficial to the applicability and accuracy of the trained model in different regions.

[0147] The following further introduces an example of the training method of the present embodiment in the application scenario of processing chest magnetic resonance imaging (MRI) medical images:

[0148] (1) collecting medical image data of multiple medical centers, performing data preprocessing on the medical image data to obtain a local data set of each medical center, and generating a global data set according to the local data set.

[0149] First, medical images of chest magnetic resonance imaging (MRI) are collected from different medical centers, and the medical images need to be labeled with left atrial segmentation regions by medical experts. For ease of understanding, the MRI medical image data of three medical centers is exemplarily collected in the embodiment of the application, and in actual situations, the medical centers can be several.

[0150] Two of the centers are named center A and center B, and for centers A and B, 15 samples are selected for model training, and 5 samples are selected as local test data. For the data set of one of the centers, two groups of 35 samples are randomly selected as center C and center D, and the training and test ratio is divided as 6:1.

[0151] In the embodiment of the application, in order to simulate the common bias and changes in labeling between different centers, morphological operations are performed on the training labels of center C, that is, open and random erosion operations are performed on the training labels of center C; closed and random dilation operations are performed on the training labels of center D; open and closed operations are performed on the local test data of centers C and D.

[0152] In addition, exemplarily, the embodiment of the application selects 10 case sample data from each medical center to generate a global data test set. The local test data is used to verify the performance of the personalized method, and the global data test set is used to verify the performance of the global model.

[0153] Then, all the MRI medical image data is resampled to a resolution of 0.6x0.6x1.25mm, and the heart region is centered and cropped to a size of 256x256, and then Z-score normalization processing is performed. In the training process, methods such as random rotation, flipping, elastic deformation and adding Gaussian noise are applied to data augmentation.

[0154] It should be noted that the above sample number is only an example, and the embodiment of the application does not limit this.

[0155] (2) generating local adversarial image samples according to the local data set.

[0156] The method of step S200 is used to generate, which will not be repeated here.

[0157] (3) constructing a distributed medical image processing model; wherein the medical image processing model comprises a global model, a plurality of local models corresponding to the medical centers one by one, and a distributed learning decoupling prediction model constructed based on a variational Bayesian framework.

[0158] In this example, a distributed learning framework for left atrial segmentation is built based on a variational Bayesian framework, which serves as a bridge connecting global methods and personalized methods and can be used for the segmentation of MRI image data that does not meet the independent and identically distributed assumption (IDD). The global method refers to a method in which all data of the centers are collected together in distributed learning, a global model is trained, and then the model is applied to the test data of all medical centers. The personalized method refers to a method in which a local model is trained in each medical center, the model is trained only using local data, and is only applied to the test data of the medical center. The embodiment of the application also proposes a distributed conditional adaptive network, which decouples the prediction sample and the label corresponding to it during training, and performs adaptive prediction according to the local distribution during testing.

[0159] (4) decoupling the prediction sample and the label by using the distributed learning decoupling prediction model, and performing global aggregation processing on a global model using the global data set.

[0160] Specifically, step S400 is used for global aggregation processing.

[0161] (5) using the local data set and the local adversarial image sample to locally train the distributed medical image processing model, and obtaining a target distributed medical image processing model when the loss function converges or reaches a preset second iteration threshold.

[0162] Specifically, steps S400-S500 are used for model training.

[0163] On the other hand, with reference to Figure 4 The embodiment of the application provides an application method of a distributed medical image processing model, comprising the following steps S600-S700.

[0164] S600, obtaining a medical image to be processed.

[0165] S700, inputting the medical image to be processed into a target distributed medical image processing model trained by using the training method of claim 1 to perform segmentation and prediction processing, and obtaining an image processing result.

[0166] Specifically, the obtained image result is determined according to the target distributed medical image processing model trained according to actual requirements.

[0167] On the other hand, as Figure 5As shown, the embodiment of the present application provides a training system of a distributed medical image processing model, comprising:

[0168] A first module is configured to collect annotated medical image data from a plurality of medical centers, perform data preprocessing on the medical image data, obtain a local data set of each medical center, and generate a global data set according to the local data set;

[0169] A second module is configured to generate a local adversarial image sample according to the local data set;

[0170] A third module is configured to construct an initial distributed medical image processing model based on a variational Bayesian framework, wherein the initial distributed medical image processing model comprises a plurality of local models corresponding to the medical centers one by one;

[0171] A fourth module is configured to perform local training on the local models by using the local data set and the local adversarial image sample, perform global aggregation processing on each local model to form a global model when the number of iterations reaches a preset round, and obtain a target distributed medical image processing model when a local loss function of each local model converges or the number of iterations reaches a preset second iteration threshold, wherein the target distributed medical image processing model comprises a global model and a plurality of local models corresponding to the medical centers one by one.

[0172] On the other hand, as Figure 6 As shown, the embodiment of the present application further provides an electronic device, comprising a processor and a memory, the memory is configured to store a program, and the processor is configured to execute the program to implement the method as described above.

[0173] On the other hand, the embodiment of the present application further provides a computer storage medium, wherein the computer storage medium stores a program executable by a processor, and the program executable by the processor is used to implement the method as described above when executed by the processor.

[0174] In some alternative embodiments, the functions / operations mentioned in the block diagram can not occur in the order mentioned in the operation diagram. For example, depending on the functions / operations involved, two blocks shown in succession can actually be executed substantially simultaneously or the blocks can sometimes be executed in reverse order. In addition, the embodiments presented and described in the flowcharts of the present application are provided by way of example, and the purpose is to provide a more comprehensive understanding of the technology. The disclosed method is not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and in which sub-operations described as part of larger operations are independently executed.

[0175] The logic and / or steps represented in the flow diagrams or otherwise described herein, for example, can be considered as a sequence of executable instructions, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical connections), a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and stored in a computer memory.

[0176] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical connections), a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and stored in a computer memory.

[0177] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific feature, structure, material or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. Descriptive expressions of the above terms in the present specification do not necessarily refer to the same embodiment or example. Also, the specific feature, structure, material or characteristic described can be combined in any one or more embodiments or examples in an appropriate manner.

[0178] Although embodiments of the present application have been shown and described, it would be recognized by those of ordinary skill in the art that various changes, modifications, alternatives, and variations can be made thereto without departing from the spirit and scope of the application, which should be limited only by the scope of the claims and the equivalents thereof.

[0179] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the described embodiments, and those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application, and these equivalent modifications or replacements are included in the scope defined by the claims of the present application.

Claims

1. A training method for a distributed medical image processing model, characterized in that, The method comprises the following steps: Collecting annotated medical image data from multiple medical centers, pre-processing the medical image data to obtain local data sets of each medical center, and generating a global data set according to the local data sets; Generating local adversarial image samples according to the local data sets; Building an initial distributed medical image processing model based on a variational Bayesian framework; wherein the initial distributed medical image processing model comprises a plurality of local models corresponding to the medical centers one by one; Local training of the local model using the local data set and the local adversarial image sample, and when the iteration number reaches a preset round, global aggregation processing is performed on each local model to form a global model; When the local loss function of each local model converges or the iteration number reaches a preset second iteration number threshold, a target distributed medical image processing model is obtained; wherein the target distributed medical image processing model comprises a global model and a plurality of local models corresponding to the medical centers one by one; The step of generating local adversarial image samples according to the local data sets comprises: Configuring a perturbation term; Adding the perturbation term to the local data set to generate a first adversarial image sample; Iterative processing of the perturbation term according to the first adversarial image sample; Using a gradient descent algorithm, repeatedly performing the step of adding the perturbation term to the local data set to generate a first adversarial image sample in a small step gradient descent manner according to a preset time interval until the iteration number reaches a preset first iteration number threshold, and obtaining an optimal perturbation value; Adding the optimal perturbation value to the local data set to generate a local adversarial image sample; The step of local training of the local model using the local data set and the local adversarial image sample comprises: Configuring a local loss function based on a gradient descent algorithm and an adversarial loss; Combining the local data set and the local adversarial image sample as a training set; Local training of the distributed medical image processing model using the training set to obtain a model training result; Calculating a loss value based on the local loss function according to the model training result; The expression of the local loss function is: wherein, is a cross-entropy loss; is a non-overlapping loss; is a constraint value; denotes a difference between a prior distribution and a posterior distribution; , and are hyperparameters; is a loss function for the adversarial generator. 2.The method of claim 1, wherein, The step of pre-processing the medical image data to obtain a local data set and generating a global data set according to the local data set comprises: Performing a random combination morphological operation on the medical image data of a plurality of medical centers to obtain a local data set; wherein the morphological operation comprises at least one of the following: opening operation, closing operation, random dilation operation or random erosion operation; Determining a plurality of samples from the local data set to generate a global data set; Data augmentation is performed on the local data set and the global data set. 3.The method of claim 1, wherein, The calculation formula of generating a global adversarial image sample according to the global data set and adding the perturbation term to the local data set to generate a first adversarial image sample is: wherein, represents a first pair of adversarial image samples, represents a positive number approaching 0; sign() represents a sign function; L() represents a loss function of an original model, i.e., represents a gradient calculation of the loss function with respect to the input x; x represents an original sample, and y represents a sample output after the original sample is operated by the model. 4.The method of claim 1, wherein, In the step of building an initial distributed medical image processing model based on a variational Bayesian framework, the following steps of building a local model are included: A segmentation network is constructed for generating a probability distribution prediction of general data, wherein the segmentation network is a convolutional neural network. A prior encoder and a posterior encoder are constructed, wherein the prior encoder is used to calculate a global loss function of a global model, and the posterior encoder is used to calculate a local loss function of a local model. A distribution adaptive network model is constructed for outputting an adaptive matrix to calculate a final prediction result.

5. An application method of a distributed medical image processing model, characterized by, The method comprises the following steps: Obtaining a medical image to be processed; Inputting the medical image to be processed into a target distributed medical image processing model trained by the training method of claim 1 for segmentation prediction processing to obtain an image processing result.

6. A system for implementing the training method of the distributed medical image processing model according to any one of claims 1-4, characterized in that, The method comprises the following steps: A first module is configured to collect labeled medical image data from a plurality of medical centers, perform data preprocessing on the medical image data, obtain a local data set of each medical center, and generate a global data set according to the local data set; A second module is configured to generate local adversarial image samples according to the local data set; A third module is configured to construct an initial distributed medical image processing model based on a variational Bayesian framework, wherein the initial distributed medical image processing model comprises a plurality of local models corresponding to the medical centers one by one; A fourth module is configured to perform local training on the local models using the local data set and the local adversarial image samples, perform global aggregation processing on each local model when the number of iterations reaches a preset number of rounds, and form a global model; the fourth module is also configured to obtain a target distributed medical image processing model when the local loss function of each local model converges or the number of iterations reaches a preset second iteration threshold; wherein the target distributed medical image processing model comprises a global model and a plurality of local models corresponding to the medical centers one by one.

7. An electronic device, comprising: The device comprises a processor and a memory; The memory is used to store a program; The processor executes the program to implement the method of any one of claims 1 to 5.

Images