A method for enhancing the generalization ability of multi-center deep learning for image processing

Abstract

The present invention provides a method for enhancing the generalization ability of deep learning multi-center for processing images, comprising the following steps: S1: acquiring multi-center multi-center data of multiple centers; wherein the multi-center data includes single-center data and cross-center data; S2: constructing a style transfer network based on the multi-center data, so that the generalization ability of deep learning multi-center is enhanced; step S2 comprises: taking a number of single-center data and cross-center data as sample data, inputting the sample data into an initial CG-DDPM diffusion model for training, predicting the noise data to be removed in the training process through a noise prediction network, calculating the center loss according to the noise data to be removed, until the center loss converges, and obtaining a style transfer network; wherein the noise prediction network includes a single-center noise prediction network and a cross-center noise prediction network.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for enhancing the generalization ability of multi-center deep learning for image processing. Background Art

[0002] At present, surgical resection and radiotherapy are still the main means of treating cancer patients. Before surgery, the three-dimensional understanding of the location of the patient's lesions and the anatomical variations of key organs has been increasingly valued by clinicians. However, manual segmentation of CT or MRI image data is currently time-consuming and labor-intensive, and has poor consistency. In contrast, with the development of big data, large computing power, and artificial intelligence algorithms in recent years, automatic segmentation algorithms have brought hope for segmentation automation. In the process of building automatic segmentation algorithms, on the one hand, data acquisition is not as easy as natural image scenes, and on the other hand, data annotation also requires professionals.

[0003] Currently, deep learning models are trained based on the single-center data of a single center, which results in poor generalization ability of deep learning models based on single-center data to process imaging data across centers. Directly applying the deep learning model to segment and process imaging data across centers will result in a decrease in model performance at best and impaired clinical functions at worst. Existing processing methods in the industry generally use federated learning to build models across centers and build models for specific centers separately. However, both of these methods require re-labeling of the data before training, which results in high usage costs. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a method for enhancing the generalization ability of multi-center deep learning for image processing, which solves the above-mentioned problems in the prior art.

[0005] According to an embodiment of the present invention, a method for enhancing the generalization ability of deep learning multi-centers for processing images includes the following steps:

[0006] S1: Obtain multicenter data from multiple centers; multicenter data includes single-center data and cross-center data;

[0007] S2: Building a style transfer network based on multi-center data to enhance the generalization ability of deep learning multi-center;

[0008] Step S2 includes:

[0009] Using several single-center data and cross-center data as sample data, the sample data is input into the initial CG-DDPM diffusion model for training. The noise prediction network is used to predict the noise data that should be removed during the training process. The center loss is calculated based on the noise data that should be removed until the center loss converges to obtain the style transfer network.

[0010] Among them, the noise prediction network includes a single-center noise prediction network and a cross-center noise prediction network.

[0011] As an embodiment of the present invention, the sample data is input into the initial CG-DDPM diffusion model for training, including:

[0012] S21: Label some single-center data and cross-center data to obtain some initial single-center data and initial cross-center data , select an initial cross-center data It is the original cross-center data;

[0013] S22: Randomly add to the original cross-center data step by step The noise is obtained The cross-center data generated after adding noise , The calculation formula is as follows:

[0014]

[0015]

[0016] in, , , The mean is 0 and the variance is A random variable that follows a Gaussian distribution;

[0017] S23: The cross-center data generated after adding noise Use the preset single center denoising network step by step De-noise the first time and get the first generated single center data , The calculation formula is as follows:

[0018]

[0019] in, is the single-center data generated after denoising once. The noise that should be removed for the nth time predicted by the single-center noise prediction network, = ;

[0020] S24: Generate the first single center data As the original order center data;

[0021] S25: Randomly and gradually add to the original order center data The noise is obtained After adding noise, the first single-center data is generated , The calculation formula is as follows:

[0022]

[0023] in, ;

[0024] S26: After adding noise, the first single-center data is generated Using the preset cross-center denoising network step by step De-noising, and get the first generated cross-center data , The calculation formula is as follows:

[0025]

[0026] in, It is the cross-center data generated after denoising once. The noise that should be removed for the nth time predicted by the cross-center noise prediction network, = ;

[0027] S27: Select an initial single center data , based on the initial single-center data Get the second generation single center data .

[0028] As an embodiment of the present invention, step S27 includes:

[0029] S271: Select an initial single center data As the original single-center data, repeat steps S25-S26 to obtain Single center data after adding noise and secondly generate cross-center data ;

[0030] S272: Generate cross-center data with the second As the original cross-center data, repeat steps S22-S23 to obtain The second step is to generate cross-center data after adding noise and secondly generate single-center data , complete a training session.

[0031] As an embodiment of the present invention, the noise prediction network predicts the data to be removed from the training process, and the center loss is calculated according to the data to be removed from the noise, including:

[0032] After completing a training, get the After adding noise, the first single-center data is generated Denoising to get the first generated cross-center data In the process, random noise removal is predicted by the cross-center prediction noise network Generate cross-center data The first one should remove the noise data ;

[0033] Computational denoising Generate cross-center data The first actual noise data actually contained in , The calculation formula is as follows:

[0034]

[0035] Based on the first actual noise data Calculate the cross-center loss, the calculation formula is as follows:

[0036]

[0037] in, is the cross-center loss.

[0038] As an embodiment of the present invention, the noise prediction network predicts the data to be removed from the training process, and the center loss is calculated according to the data to be removed from the noise, and further includes:

[0039] Get the The second step is to generate cross-center data after adding noise Denoising to obtain the second generation single center data In the process, random noise removal is predicted by a single-center prediction noise network Generate cross-center data The second one should remove the noisy data ;

[0040] Computational denoising Generate single-center data The second actual noise data actually contained in , The calculation formula is as follows:

[0041]

[0042] Based on the second actual noise data Calculate the cross-center loss, the calculation formula is as follows:

[0043]

[0044] in, is the single center loss.

[0045] As an embodiment of the present invention, until the center loss converges, a style transfer network is obtained, including:

[0046] Determine the single center loss and cross-center loss Is it less than the preset loss value? If the single center loss and cross-center loss If both are smaller than the preset loss value, it is determined that the center loss converges and the style transfer network is obtained;

[0047] If a single center is lost or cross-center loss If the loss is greater than or equal to the preset loss value, an initial cross-center data and an initial single-center data are reselected for training.

[0048] Compared with the prior art, the present invention has the following beneficial effects: a style transfer network is obtained by training with the CG-DDPM diffusion model, and the cross-center data can be converted into single-center data through the style transfer network; and in the process of training the style transfer network, the training can be completed without re-labeling the cross-center data, thereby reducing the cost of use. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of generating single-center data from cross-center data in an embodiment of the present invention;

[0050] Figure 2 A diagram of a single-center noise prediction network architecture according to another embodiment of the present invention;

[0051] Figure 3 A flow chart of obtaining network training data generated by cross-center data circulation diffusion according to another embodiment of the present invention;

[0052] Figure 4 This is a flowchart of obtaining network training data generated by single-center data cyclic diffusion according to another embodiment of the present invention. DETAILED DESCRIPTION

[0053] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0054] like Figures 1 to 4As shown, the embodiment of the present invention proposes a method for enhancing the generalization ability of deep learning multi-centers for processing images, comprising the following steps:

[0055] S1: Obtain multicenter data from multiple centers; multicenter data includes single-center data and cross-center data;

[0056] S2: Building a style transfer network based on multi-center data to enhance the generalization ability of deep learning multi-center;

[0057] Step S2 includes:

[0058] Using several single-center data and cross-center data as sample data, the sample data is input into the initial CG-DDPM diffusion model for training. The noise prediction network is used to predict the noise data that should be removed during the training process. The center loss is calculated based on the noise data that should be removed until the center loss converges to obtain the style transfer network.

[0059] Among them, the noise prediction network includes a single-center noise prediction network and a cross-center noise prediction network.

[0060] The working principle of the above technical solution is as follows: in actual use, the collected cross-center data is converted into single-center data through a trained style transfer network, thereby improving the generalization performance of deep learning multi-center; when training the style transfer network, 500 single-center data and 500 cross-center data are collected as sample data, and it is necessary to cover as much data variability as possible; the sample data is input into the initial CG-DDPM diffusion model for training, and the noise prediction network is used to predict the noise data that should be removed during the training process, and the center loss is calculated based on the noise data that should be removed until the center loss converges to obtain the style transfer network; wherein, the selection of the CG-DDPM diffusion model can, on the one hand, avoid the loss of details in the style transfer achieved by the traditional GAN ​​network; on the other hand, it avoids the risk of loss and modification of the original image information when using a non-cyclic generation network; wherein, the center loss includes single-center loss and cross-center loss; the noise data that should be removed includes the first noise data that should be removed and the second noise data that should be removed;

[0061] The beneficial effects of the above technical solution are as follows: a style transfer network is obtained by training with the CG-DDPM diffusion model, and the cross-center data can be converted into single-center data through the style transfer network; and in the process of training the style transfer network, the training can be completed without re-labeling the cross-center data, thereby reducing the cost of use.

[0062] In one embodiment, the sample data is input into the initial CG-DDPM diffusion model for training, including:

[0063] S21: Label some single-center data and cross-center data to obtain some initial single-center data and initial cross-center data , select an initial cross-center data It is the original cross-center data;

[0064] S22: Randomly add to the original cross-center data step by step The noise is obtained The cross-center data generated after adding noise , The calculation formula is as follows:

[0065]

[0066]

[0067] in, , , The mean is 0 and the variance is A random variable that follows a Gaussian distribution;

[0068] S23: The cross-center data generated after adding noise Use the preset single center denoising network step by step De-noise the first time and get the first generated single center data , The calculation formula is as follows:

[0069]

[0070] in, is the single-center data generated after denoising once. The noise that should be removed for the nth time predicted by the single-center noise prediction network, = ;

[0071] S24: Generate the first single center data As the original order center data;

[0072] S25: Randomly and gradually add to the original order center data The noise is obtained After adding noise, the first single-center data is generated , The calculation formula is as follows:

[0073]

[0074] in, ;

[0075] S26: After adding noise, the first single-center data is generated Using the preset cross-center denoising network step by step De-noising, and get the first generated cross-center data , The calculation formula is as follows:

[0076]

[0077] in, It is the cross-center data generated after denoising once. The noise that should be removed for the nth time predicted by the cross-center noise prediction network, = ;

[0078] S27: Select an initial single center data , based on the initial single-center data Get the second generation single center data ;

[0079] Step S27 includes:

[0080] S271: Select an initial single center data As the original single-center data, repeat steps S25-S26 to obtain Single center data after adding noise and secondly generate cross-center data ;

[0081] S272: Generate cross-center data with the second As the original cross-center data, repeat steps S22-S23 to obtain After adding noise, generate cross-center data and secondly generate single-center data , complete a training session.

[0082] Working principle and technical effect of the above technical solution: In actual use, by marking a number of single-center data and cross-center data, a number of initial single-center data are obtained. and initial cross-center data ; An initial cross-center data will be selected by mark As the original cross-center data, randomly and gradually add The noise is obtained The cross-center data generated after adding noise ; for The cross-center data generated after adding noise Use the preset single-center denoising network (single-center Unetsingle) step by step De-noise the first time and get the first generated single center data ; Then generate the first single center data As the original order center data; randomly and gradually add to the original order center data The noise is obtained After adding noise, the first single-center data is generated ; for After adding noise, the first single-center data is generated Using the preset cross-center denoising network step by step De-noising, and get the first generated cross-center data ;

[0083] Then click on the mark to select an initial single center data , based on the initial single-center data As the original single center data, repeat step S25 to obtain Single center data after adding noise Repeat step S26 to obtain the second generated cross-center data ; Then generate cross-center data with the second As the original cross-center data, repeat steps S22-S23 to obtain The second step is to generate cross-center data after adding noise and secondly generate single-center data , complete one training; after completing one training, reselect an initial single-center data and an initial cross-center data according to the mark, repeat the training, and traverse all the marks to complete a cycle.

[0084] In one embodiment, the noise prediction network predicts the data that should be removed from the noise during the training process, and the center loss is calculated based on the data that should be removed from the noise, including:

[0085] After completing a training, get the After adding noise, the first single-center data is generated Denoising to get the first generated cross-center data In the process, random noise removal is predicted by the cross-center prediction noise network Generate cross-center data The first one should remove the noise data ;

[0086] Computational denoising Generate cross-center data The first actual noise data actually contained in , The calculation formula is as follows:

[0087]

[0088] Based on the first actual noise data Calculate the cross-center loss, the calculation formula is as follows:

[0089]

[0090] in, for inter-center losses;

[0091] Also includes:

[0092] Get the After adding noise, generate cross-center data Denoising to obtain the second generation single center data In the process, random noise removal is predicted by a single-center prediction noise network Generate cross-center data The second one should remove the noisy data ;

[0093] Computational denoising Generate single-center data The second actual noise data actually contained in , The calculation formula is as follows:

[0094]

[0095] Based on the second actual noise data Calculate the cross-center loss, the calculation formula is as follows:

[0096]

[0097] in, is the single center loss.

[0098] The working principle and technical effect of the above technical solution: After completing one training, the cross-center prediction noise network and the cross-center prediction noise network prediction are respectively predicted. and , and then calculate the single center loss Post-cross-center loss ; The whole framework is to send the cross-center data after the nth noise addition and the single-center data generated after the mth denoising into the Unet network to estimate the noise that should be removed for the nth time; Figure 2 As shown in Figure 2, the key difference is that the time series n needs to be embedded into the network features of each layer through cosine coding.

[0099] In one embodiment, until the center loss converges, a style transfer network is obtained, including:

[0100] Determine the single center loss and cross-center loss Is it less than the preset loss value? If the single center loss and cross-center loss If both are smaller than the preset loss value, it is determined that the center loss converges and the style transfer network is obtained;

[0101] If a single center is lost or cross-center loss If the loss is greater than or equal to the preset loss value, an initial cross-center data and an initial single-center data are reselected for training.

[0102] The working principle and beneficial effects of the above technical solution: After completing a training, the single center loss is determined and cross-center loss Is it less than the preset loss value? If the single center loss and cross-center loss If both are smaller than the preset loss value, the center loss is determined to have converged, and after completing a cycle of training, the style transfer network is obtained; if the single center loss or cross-center loss If the loss is greater than or equal to the preset loss value, an initial cross-center data and an initial single-center data are re-selected for training; the batch size of the training process is 1, the patch size is 128*128*128, the initial learning rate is 0.01, the optimizer is adam, and the learning rate decays linearly according to the training epoch; MSE represents the mean square error calculation;

[0103] The data resolution was resampled to 0.8*0.8*0.8mm, and the grayscale was standardized to a mean of 0 and a standard deviation of 1.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A method for enhancing the generalization ability of multi-center deep learning for image processing, characterized in that: The following steps are involved: S1: Obtain multicenter data from multiple centers; The multicenter data include single-center data and cross-center data; S2: Building a style transfer network based on multi-center data enhances the generalization ability of deep learning multi-center; Step S2 includes: Using several single-center data and cross-center data as sample data, the sample data is input into the initial CG-DDPM diffusion model for training. The noise prediction network is used to predict the noise data that should be removed during the training process. The center loss is calculated based on the noise data that should be removed until the center loss converges to obtain the style transfer network. Among them, the noise prediction network includes a single-center noise prediction network and a cross-center noise prediction network; The sample data is input into the initial CG-DDPM diffusion model for training, including: S21: Label some single-center data and cross-center data to obtain some initial single-center data and initial cross-center data , select an initial cross-center data It is the original cross-center data; S22: Randomly add to the original cross-center data step by step The noise is obtained The cross-center data generated after adding noise , The calculation formula is as follows: in, , , The mean is 0 and the variance is A random variable that follows a Gaussian distribution; S23: The cross-center data generated after adding noise Use the preset single center denoising network step by step De-noise the first time and get the first generated single center data , The calculation formula is as follows: in, is the single-center data generated after denoising once. The noise that should be removed for the nth time predicted by the single-center noise prediction network, = ; S24: Generate the first single center data As the original order center data; S25: Randomly and gradually add to the original order center data The noise is obtained After adding noise, the first single-center data is generated , The calculation formula is as follows: in, ; S26: After adding noise, the first single-center data is generated Using the preset cross-center denoising network step by step De-noising, and get the first generated cross-center data , The calculation formula is as follows: in, It is the cross-center data generated after denoising once. The noise that should be removed for the nth time predicted by the cross-center noise prediction network, = ; S27: Select an initial single center data , based on the initial single-center data Get the second generation single center data .

2. According to claim 1, a method for enhancing the generalization ability of deep learning multi-centers for image processing is characterized in that: Step S27 includes: S271: Select an initial single center data As the original single-center data, repeat steps S25-S26 to obtain Single center data after adding noise and secondly generate cross-center data ; S272: Generate cross-center data with the second As the original cross-center data, repeat steps S22-S23 to obtain The second step is to generate cross-center data after adding noise and secondly generate single-center data , complete a training session.

3. The method for enhancing the generalization ability of multi-center deep learning for image processing according to claim 1, characterized in that: The noise prediction network is used to predict the data that should be removed from the noise during the training process, and the center loss is calculated based on the data that should be removed from the noise, including: After completing a training, get the After adding noise, the first single-center data is generated Denoising to get the first generated cross-center data In the process, random noise removal is predicted by the cross-center prediction noise network Generate cross-center data The first one should remove the noise data ; Computational denoising Generate cross-center data The first actual noise data actually contained in , The calculation formula is as follows: Based on the first actual noise data Calculate the cross-center loss, the calculation formula is as follows: in, is the cross-center loss.

4. The method for enhancing the generalization ability of multi-center deep learning for image processing according to claim 2, characterized in that: The noise prediction network predicts the data that should be removed during the training process, and the center loss is calculated based on the data that should be removed. It also includes: Get the The second step is to generate cross-center data after adding noise Denoising to obtain the second generation single center data In the process, random noise removal is predicted by a single-center prediction noise network Generate cross-center data The second one should remove the noisy data ; Computational denoising Generate single-center data The second actual noise data actually contained in , The calculation formula is as follows: Based on the second actual noise data Calculate the cross-center loss, the calculation formula is as follows: in, is the single center loss.

5. The method for enhancing the generalization ability of multi-center deep learning for image processing according to claim 4, characterized in that: Until the center loss converges, the style transfer network is obtained, including: Determine the single center loss and cross-center loss Is it less than the preset loss value? If the single center loss and cross-center loss If both are smaller than the preset loss value, it is determined that the center loss converges and the style transfer network is obtained; If a single center is lost or cross-center loss If the loss is greater than or equal to the preset loss value, an initial cross-center data and an initial single-center data are reselected for training.

Images