Tumor diagnosis method based on manifold confrontation field self-adaption

Abstract

The invention discloses a manifold confrontation field self-adaption-based tumor diagnosis method, which comprises the following steps of: firstly, pre-training, and using a model pre-trained by source domain data as a model initialization parameter; then, manifold confrontation field self-adaption is carried out, a confrontation learning network framework is designed, a discriminant manifold propagation method is introduced to adjust structural parameters of a manifold module, manifold learning is used for assisting confrontation learning to further minimize the source domain and target domain feature mapping distance, and a target domain feature extractor is trained; and finally, testing, inputting a target domain test tumor image into a feature extractor after adversarial training to extract features, and then classifying data by using a trained classifier to obtain a tumor image classification result. The objective of the invention is to provide a manifold confrontation field-based adaptive tumor diagnosis method, which solves the dependence of existing deep learning on a large amount of labeled data, reduces the sample labeling cost in tumor diagnosis, reduces the labor intensity of doctors and experts, and realizes automatic and high-precision diagnosis classification of non-labeled medical tumor images.

Description

Technical Field

[0001] The present invention belongs to the technical field of tumor diagnosis and domain adaptive learning, and relates to a tumor diagnosis method based on manifold adversarial domain adaptation. Background Art

[0002] Deep learning is an important research direction in the era of artificial intelligence. Its research results have been widely applied to various fields and have played an increasingly important role in people's daily lives. In the field of medical image processing, deep learning, especially deep convolutional neural networks, can automatically learn to extract implicit disease diagnostic features from medical image big data, and has quickly become a research hotspot in medical image analysis. [1][2] Although computer-aided diagnosis based on deep learning has achieved great results, it has the following problems: 1) Deep learning requires a large amount of labeled training sample data, which is very expensive for medical experts to label, and some cases are rare; 2) Deep learning requires a large amount of memory and computing resources, otherwise the training process will be extremely time-consuming; 3) Training deep neural networks often becomes very complicated due to overfitting and convergence problems, and it is often necessary to repeatedly adjust the learning parameters in the framework or network. The high performance of deep learning is closely related to a large number of labeled samples. The high requirements for the hardware configuration lead to high hardware costs. At the same time, the large amount of labeled data requires high labeling costs. Therefore, how to use the existing labeled domain data knowledge to label unlabeled domain data for classification is of great significance in various application fields, especially in the medical field.

[0003] Domain Adaptation is one of the main methods to solve the above domain knowledge migration. It can overcome the differences in data distribution and label space between the two domains, migrate the labeled source domain knowledge to the unlabeled target domain, and complete the learning task of the target domain. [3] ) can learn a model with good performance on the source domain data, adapt to the target domain data with different distribution or even different category label space, and solve the learning task of the target domain data with no labeled samples or a small number of labeled samples. At present, domain adaptation can be roughly divided into two categories. One method is based on statistical moment alignment, which uses the distance function to minimize the distribution difference between the source domain and the target domain, including the maximum mean deviation (MMD) [4] , Center moment deviation (CMD) [5] and second-order statistical alignment. The other is an adversarial approach that shortens the distance between the two domains through adversarial training and obtains effective cross-domain features. Existing adversarial domain adaptation methods are mainly divided into three categories: [3]: A method based on gradient reversal, a method based on minimax optimization, and a method based on generative adversarial networks. The first two methods minimize the domain distribution difference through feature-level domain confusion under the supervision of a domain discriminator, and the latter method achieves pixel-level domain migration by synthesizing indistinguishable target domain images. Inspired by manifold learning, a manifold adversarial domain adaptation method is proposed for tumor diagnosis and classification, which can not only solve the problem of insufficient image annotation samples in tumor diagnosis applications, but also achieve high-precision tumor diagnosis and classification.

[0004] [1] Tian Juanxiu, Liu Guocai, Gu Shanshan, Ju Zhongjian, Liu Jinguang, Gu Dongdong. Research and challenges of deep learning methods for medical image analysis[J]. Acta Automatica Sinica, 2018, 44(3): 401-424.

[0005] [2] Chen Shihui, Liu Weixiang, Qin Jing, Chen Liangliang, Bingguo, Zhou Yuxiang, Wang Tianfu, Huang Bingsheng. Research progress of computer-aided diagnosis of cancer based on deep learning and medical images[J]. Journal of Biomedical Engineering, 2017, 34(02): 314-319.

[0006] [3] L. Zhang,

[0007] [4]Tzeng, E., Hoffman, J., Zhang, et al. Deep domain confusion: Maximizing for domain invariance[J]. Computing Research Repository, 2014.

[0008] [5]Zellinger W, Grubinger T, Lughofer, E, et al. Central momentdiscrepancy (CMD) for domain-invariant representation learning [C]. International Conference on Learning Representations, 2017. Summary of the invention

[0009] The purpose of the present invention is to provide a tumor diagnosis method based on manifold adversarial domain adaptation, which solves the existing deep learning dependence on a large amount of labeled data, reduces the sample labeling cost in tumor diagnosis, reduces the labor intensity of doctors and experts, and realizes automatic and high-precision diagnostic classification of unlabeled medical tumor images.

[0010] The technical solution adopted by the present invention is

[0011] The method for tumor diagnosis based on manifold adversarial domain adaptation is characterized in that pre-training is first performed, and the model pre-trained with source domain data is used as the model initialization parameter. It is characterized in that manifold adversarial domain adaptation is then performed, an adversarial learning network framework is designed, and the discriminative manifold propagation (Discriminative Manifold Propagation, DMP) method is introduced to adjust the structural parameters of the manifold module to adapt it to the adversarial learning network framework, and manifold learning is used to assist adversarial learning to further minimize the feature mapping distance between the source domain and the target domain, and the target domain feature extractor is trained. Its final feature is to carry out testing, input the target domain test tumor image data into the adversarially trained feature extractor to extract features, and then use the trained classifier to classify the data to obtain the final tumor image classification result.

[0012] The beneficial effects of the present invention are

[0013] 1. Construct a manifold adversarial domain adaptive tumor diagnosis method, and transfer the labeled source domain knowledge to the unlabeled target domain data through the manifold adversarial domain adaptive model to achieve cross-domain knowledge transfer.

[0014] 2. Construct a tumor diagnosis method that is adaptive to the manifold adversarial domain, which can achieve high-precision unsupervised tumor image diagnosis in the absence of labeled samples.

[0015] 3. Construct a manifold adversarial domain-adaptive tumor diagnosis method to achieve cross-domain knowledge transfer and high-precision tumor image diagnosis classification at a lower hardware cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a pre-training framework of the method of the present invention;

[0017] Figure 2 It is the manifold adversarial domain adaptive framework of the method of the present invention;

[0018] Figure 3 It is the test framework of the method of the present invention. DETAILED DESCRIPTION

[0019] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] The entire training process of the method is divided into three stages: pre-training, manifold adversarial domain adaptation and testing.

[0021] The first step is pre-training; design a double hidden neural network layer as the feature extractor F S and a classifier C, using the source domain training data {X S ,y S} Train the classifier C and obtain the feature extractor F S The hyperparameter M S The steps of pre-training are as follows Figure 1 As shown, the labeled source domain training data is first input into the feature extractor F S The features are extracted from the model, and then the classifier C is used for classification to train the model, where the feature extractor consists of two fully connected layers and an activation function, and the classifier consists of two fully connected layers and one activation function.

[0022] In this process, the classifier loss function is defined as:

[0023] L C (X S ,y S , F S , C) = min[log C(F S (X S ), y S )] (1)

[0024] Then the second step is manifold adversarial domain adaptation; in this stage, the adversarial learning network framework is designed, and the structural parameters of the manifold module M are adjusted according to the DMP method to adapt it to the adversarial learning network framework. Manifold learning is used to assist adversarial learning to further minimize the feature mapping distance between the source domain and the target domain, and the target domain feature extractor F is trained. T The steps of this stage are as follows Figure 2 As shown, the steps include at least the following:

[0025] First, we use the pre-trained feature extractor as the feature extractor for the source domain and the target domain, and use the pre-trained feature extractor hyperparameters M S Initialize the source domain F S and the target domain feature extractor F T , and then preliminarily extract the data features of the two domains to obtain the feature mapping matrix H(X S ) and H(X T ).

[0026] H=G(a ij , b j , x i ), a ij ∈Rn×L , b j ∈R 1×L (2) Among them, G(a ij , b j , x i ) is a nonlinear function, a ij and b j are weight and bias respectively.

[0027] Next, fix the source domain mapping and design the discriminator of the adversarial learning network. Use an appropriate loss function, input the source domain feature map and the target domain feature map into module M for inter-class alignment, and output the source domain weight W S and the target domain weight W T , where module M consists of three fully connected layers using different activation functions to form a Riemannian manifold alignment network F M It is composed of a three-layer linear layer auxiliary classifier C2 for calculating weights, and its weight calculation method is as follows:

[0028] W S =M(X S )=C2(F M (X S )) (3)

[0029] W T =M(X T )=C2(F M (X T )) (4)

[0030] Then, the weight is multiplied by the features corresponding to the source domain and the target domain respectively, and used as the discriminator input X S ', X T ', conduct adversarial training to reduce the difference between domains. Through iterative training, the trained target domain feature extractor F' is obtained T .

[0031] X S '=W S ·H(X S ) (5)

[0032] X T '=W T ·H(X T ) (6)

[0033] The loss function of this process is defined as:

[0034]

[0035]

[0036]

[0037] Therefore, the total loss at this stage is defined as:

[0038]

[0039] The final third step, testing, is to test the target domain data X T Input to the adversarially trained F T Features are extracted from the dataset, and then the classifier C in the first step is used to classify the data to obtain the final tumor image classification result.

[0040] In this process, the classifier loss function is defined as:

[0041] L C (X T , F T , C) = min[log C(F T (X T ))] (11).

Claims

1. A tumor diagnosis method based on manifold adversarial domain adaptation, characterized by First, pre-training is performed; the method does not use source domain data, but trains the source domain model. The source model is pre-trained using labeled source samples. The main steps are as follows: Design a double hidden neural network layer as feature extractor F S and a classifier C, using the source domain training data {X S ,y S } Train the classifier C and obtain the feature extractor F S The hyperparameter M S ; The pre-training steps are shown in Figure 1. First, the labeled source domain training data is input into the feature extractor F S Extract features from the model, and then use classifier C for classification to train the model, where the feature extractor consists of two fully connected layers and an activation function, and the classifier consists of two fully connected layers and one activation function; In this process, the classifier loss function is defined as: L C (X S ,y S ,F S ,C)=min[logC(F S (X S ),y S )] (1) 2. A tumor diagnosis method based on manifold adversarial domain adaptation, characterized by Secondly, manifold adversarial domain adaptation is performed; In this stage, the adversarial learning network framework is designed, and the structural parameters of the manifold module M are adjusted according to the discriminative manifold propagation (DMP) method to adapt it to the adversarial learning network framework. Manifold learning is used to assist adversarial learning to further minimize the feature mapping distance between the source domain and the target domain, and the target domain feature extractor F is trained. T The steps of this stage are shown in Figure 2, and the steps at least include the following: (1) First, use the pre-trained feature extractor as the feature extractor for the source domain and the target domain, and use the pre-trained feature extractor hyperparameters M S Initialize the source domain F S and the target domain feature extractor F T , and then preliminarily extract the data features of the two domains to obtain the feature mapping matrix H(X S ) and H(X T ); H=G(a ij ,b j ,x i ),a ij ∈R n×L ,b j ∈R 1×L (2) Where G(a ij ,b j ,x i ) is a nonlinear function, a ij and b j They are weight and bias respectively; (2) Then, the source domain mapping is fixed and the discriminator of the adversarial learning network is designed. Using an appropriate loss function, the source domain feature map and the target domain feature map are input into the module M for inter-class alignment, and the source domain weight W is output. S and the target domain weight W T , where module M consists of three fully connected layers using different activation functions to form a Riemannian manifold alignment network F M It is composed of a three-layer linear layer auxiliary classifier C2 for calculating weights, and its weight calculation method is as follows: W S =M(X S )=C2(F M (X S )) (3) W T =M(X T )=C2(F M (X T )) (4) (3) Secondly, the weight is multiplied by the features corresponding to the source domain and the target domain respectively, and used as the discriminator input X S ′,X T ′, conduct adversarial training to reduce the difference between domains; through iterative training, the trained target domain feature extractor F′ is obtained T ; X S ′=W S ·H(X S ) (5) X T ′=W T ·H(X T ) (6) The loss function of this process is defined as: Therefore, the total loss at this stage is defined as:

3. A tumor diagnosis method based on manifold adversarial domain adaptation, characterized by Finally, the test is performed and the target domain test data X T Input to the adversarially trained F T Extract features, and then use the classifier C in the first step to classify the data to obtain the final tumor image classification result; In this process, the classifier loss function is defined as: L C (X T ,F T ,C)=min[lgo C(F T (X T ))]。 (11)。

Images