A medical image classification method based on deep reweighted training
Through deep reweighting training technology, the problems of low interpretability and robustness in medical image classification are solved, the processing ability of imbalance and noise data sets is improved, the interpretability of features and samples is enhanced, and the medical image classification is improved. accuracy and reliability.
Patent Information
- Application Number
- CN202410784388.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2023-12-01
- Filing Date
- 2024-06-18
- Publication Date
- 2024-12-20
- Estimated Expiration
- 2044-06-18
AI Technical Summary
Existing medical image classification techniques face the problems of low interpretability and low robustness for data sets containing label noise and imbalanced data sets, especially when dealing with complex and diverse medical image data.
Using a deep reweighted training method, the training of deep neural networks is divided into early training and late training. The high confidence sample set and noise sample set are obtained through early training. The noise sample and high confidence sample are used for replication in the later training. Weighted training, dynamically update feature masks and sample weights to improve the interpretability and robustness of the model.
Improve the robustness of the model to unbalanced data sets and labeled noise data sets, and enhance the feature interpretability and sample interpretability through dynamically changing sample weights and feature weights, and improve the accuracy of medical image classification. and reliability.
Smart Images
Figure CN118691883B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of medical image classification, and more specifically, relates to a medical image classification method based on deep reweighted training. Background Art
[0002] As a powerful machine learning technology, deep learning has shown great potential and application prospects in various fields. In the field of medical image classification technology, the application of deep learning has also attracted widespread attention and research. Medical image classification is a crucial task in the medical field and is of great significance for accurately and quickly identifying and analyzing lesions and abnormalities in medical images.
[0003] Traditional medical image classification usually relies on manually designed feature extraction and classification algorithms, which often require a lot of manual participation and expertise. However, due to the complexity and diversity of medical images, traditional methods face challenges in processing large-scale data and complex case medical images. The emergence of deep learning has brought new opportunities and breakthroughs to medical image classification.
[0004] The core of deep learning is to automatically learn feature representation and pattern recognition by building deep neural networks. Compared with traditional methods, deep learning can learn richer and more abstract feature representations from large-scale medical image data, thereby improving the accuracy and efficiency of medical image classification and helping doctors diagnose diseases.
[0005] The application of deep learning in medical image classification has achieved a series of remarkable results, which is helpful for medical activities such as lung nodule detection, breast cancer diagnosis, brain disease analysis, etc. However, deep learning still faces some challenges and problems in medical image classification. First, medical image data usually has label noise and high class imbalance, which puts forward requirements for the robustness of deep learning models. Secondly, the interpretability and reliability of deep learning models are also the focus of medical field, because accurate diagnosis results require reliable explanations and basis. Summary of the invention
[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a medical image classification method based on deep reweighted training to solve the problem of low interpretability of existing medical image classification and the problem of low robustness of existing medical image processing classification to labeled noisy data sets and data sets with unbalanced number of category samples, so as to cope with the particularity and challenges of medical image data.
[0007] To achieve the above-mentioned object of the invention, the medical image classification method based on deep reweighted training of the present invention is characterized by comprising the following steps:
[0008] (1) Medical image data preprocessing
[0009] Obtain the annotated original medical image data and enter the data preprocessing module to obtain a training set X containing m medical image samples, each of which is represented by n features;
[0010] (2) Early training
[0011] Early deep neural networks were defined as DNNs E , based on the early deep neural network DNN initialized with the training set X E To train:
[0012] 2.1) Obtain a batch of medical image samples of size b from the training set X and train the early deep neural network DNN E Perform training, define the prediction value corresponding to the category marked in the label vector of each medical image sample as the confidence and record the confidence of each medical image sample;
[0013] 2.2) After repeating the training of all medical image samples, a training round is completed. The training set X is classified according to categories. The k medical image samples with the highest confidence in each category are defined as high-confidence samples and added to the high-confidence sample pool;
[0014] 2.3) Repeat steps 2.1 and 2.2 for T rounds to end the early training phase. The early deep neural network DNN E Transformed into a later deep neural network DNN L , the high confidence sample pool contains m1 high confidence samples, forming a high confidence sample set X C , except for m1 high confidence samples in the training set X, the rest are noise samples, the number is m-m1, and they constitute the noise sample set X N ;
[0015] (3) Post-training
[0016] Based on the late deep neural network DNN L , high confidence sample set X C And the noise sample set X N Start reweighted training:
[0017] 3.1) Noise sample set X N Get a batch of noise samples (x N_i ,Y N_i ), i = 1, 2, ..., b, and input it into the later deep neural network DNN L , and then calculate the cross entropy loss for each noise sample. The cross entropy loss of all noise samples constitutes a noise sample loss vector L(X N), initialize the sample weight vector W0 with dimension b;
[0018] 3.2) For the noise sample loss vector L(X N ) and the sample weight vector W0 to obtain b noise samples (x N_i ,Y N_i ), i=1,2,…,b, based on the total loss of the later deep neural network DNN L The network parameters are calculated to obtain the gradient, and a pre-update is performed to obtain the later deep neural network DNN′ L , the pre-update does not update the later deep neural network DNN L , is based on the later deep neural network DNN L A gradient descent obtains a new deep neural network DNN′ L ;
[0019] 3.3) For the high confidence sample set X C The high confidence sample x in C_i ,i=1,2,…,m1, based on the category feature mask after the previous iteration High confidence samples of the corresponding category Perform point multiplication to obtain high confidence samples Where j represents the jth category, j = 1, 2, ..., c, c is the total number of categories, m j Represents the high confidence sample set X C The number of high confidence samples of the j-th category in t represents the t-th iteration. If the current iteration is the first iteration, that is, t = 1, the feature mask matrix of the j-th category is initialized. Feature mask matrix The feature dimension of the medical image sample is consistent with n, and the category feature mask code Each element of represents a high confidence sample of the jth category The weight of each feature;
[0020] The high confidence samples Input deep neural network DNN′ L Calculate the mean cross entropy loss and mask the category features based on the mean cross entropy loss Perform a gradient descent to obtain the category feature mask
[0021] 3.4) Based on category feature mask High confidence samples of the corresponding category Perform point multiplication to obtain high confidence samples
[0022] Input deep neural network DNN′L , the feature weighted loss L is calculated according to the following formula F :
[0023]
[0024] in, Indicates high confidence samples Through the deep neural network DNN′ L Get the predicted value of category j;
[0025] 3.5) Calculate the feature weighted loss L F Gradient of the initial sample weight W0 Then calculate the new sample weight W1:
[0026]
[0027] in, The function is to avoid the denominator being zero. hour In other cases
[0028] 3.6) Based on the noise sample loss vector L(X N ) and the new sample weight W1, calculate the sample weighted loss L S :
[0029] L s = <L(x N ),W1>
[0030] in, <L(x N ),W1> represents the inner product of two vectors;
[0031] Based on the sample weighted loss L S For the later deep neural network DNN L Perform a gradient descent to obtain the updated later deep neural network DNN L , complete a reweighted training.
[0032] 3.7) Repeat steps 3.1) to 3.6) until the noise sample set X N All noise samples are taken out to complete a training round, t = t + 1;
[0033] 3.8) Repeat T′ training rounds to obtain the converged late-stage deep neural network DNN L , denoted as deep neural network DNN F ;
[0034] (4) Classification of medical image samples
[0035] Obtain the original medical image data, enter the data preprocessing module, obtain medical image samples, and send them to the deep neural network DNN F , and obtain its predicted value in each category. The category with the largest predicted value is the category of the medical image sample.
[0036] The object of the present invention is achieved in that:
[0037] The present invention is based on a medical image classification method based on deep reweighted training. The training of deep neural network is divided into early training and late training. The early training obtains the late deep neural network DNN. L At the same time, the training set is divided into a high-confidence sample set and a noise sample set for later training. When the later deep neural network is trained, a pre-updated deep neural network is obtained through the noise sample set, which is used to obtain a real-time updated feature mask, and the feature weight of each category feature can be obtained. The higher the weight, the higher the importance of the feature to the classification, and the feature is interpretable. At the same time, by giving dynamically changing sample weights, the present invention can select key samples with high importance for classification, and has sample interpretability. On the other hand, the present invention uses high-confidence samples and pre-updated deep neural networks to give dynamically changing sample weights to training samples, which can increase the weights of samples with a small number of categories and samples with correct labels, thereby correcting the adverse effects of unbalanced data sets of sample categories or data sets containing labeled noise on model training, and improving the robustness of deep neural networks to unbalanced data sets and data sets containing labeled noise. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 is a flow chart of a medical image classification method based on deep reweighted training of the present invention;
[0039] Figure 2 This is a schematic diagram of Alzheimer's disease classification based on fMRI data;
[0040] Figure 3 yes Figure 2 Schematic diagram of one training of the reweighted training module shown;
[0041] Figure 4 It is a comparison chart of the accuracy curves of the graph convolutional neural network (GCN) and the present invention on the classification task of the Alzheimer's disease fMRI dataset containing 30% symmetric label noise;
[0042] Figure 5 It is a comparison chart of the accuracy curves of the graph convolutional neural network (GCN) and the present invention on the classification task of the Alzheimer's disease fMRI dataset containing 50% symmetric label noise. DETAILED DESCRIPTION
[0043] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.
[0044] Figure 1 It is a flow chart of the medical image classification method based on deep reweighted training of the present invention.
[0045] In this embodiment, if Figure 1 As shown, the medical image classification method based on deep reweighted training of the present invention comprises the following steps:
[0046] Step S1: Medical image data preprocessing
[0047] like Figure 2 As shown, the labeled original medical image data is obtained and enters the data preprocessing module to obtain a training set X containing m medical image samples, each of which is represented by n features.
[0048] In this embodiment, the medical image data is fMRI data, i.e., functional magnetic resonance imaging (fMRI) data, which is used to assist in the diagnosis of Alzheimer's disease. The original fMRI data is preprocessed including spatial distortion correction, skull dissection, and cerebellum removal. The fMRI data is then segmented into gray matter, white matter, and cerebrospinal fluid, and 93 regions of interest (ROIs) are acquired using the Jacob brain partitioning template "Kabani NJ, MacDonald DJ, Holmes CJ, et al. 3D anatomical atlas of the human brain [J] / / NeuroImage, 1998, 7 (4): S717." The Alzheimer's disease dataset used in this embodiment has a total of 412 samples, including 226 normal controls (NC) and 186 Alzheimer's disease patients (AD), i.e., two categories. It is divided into a training set X containing m = 320 medical image samples and a test set containing m′ = 92 medical image samples. Each medical image sample is represented by n=93 region of interest (ROIs) features.
[0049] Step S2: Early training
[0050] Early deep neural networks were defined as DNNs E , based on the early deep neural network DNN initialized with the training set X E In this embodiment, Figure 2As shown, the early training is carried out in the training module, and the result of the training is to obtain the later deep neural network DNN L And the high confidence sample set X C , noise sample set X N .
[0051] Step S2.1: Train the early deep neural network as DNN E And record the confidence
[0052] Get a batch of medical image samples of size b in the training set X and train the early deep neural network DNN E Perform training, define the predicted value corresponding to the category marked in the label vector of each medical image sample as the confidence and record the confidence of each medical image sample.
[0053] In this embodiment, a medical image sample x with b=32 is obtained. i ,i=1,2,…,32 and the corresponding label Y i , where x i represents the i-th medical image sample, y ij is the jth label value in the label vector. If it belongs to the jth category, the value is 1, otherwise it is 0. v Enter the early deep neural network DNN E , get the predicted value p of c categories j (x i ),j=1,2,…,c, and record the confidence of each medical image sample, where the sample confidence is the predicted value corresponding to the labeled category of each medical image sample label vector, and the category corresponding to the position where the label value is 1 in the medical image sample label vector is the labeled category.
[0054] Calculate the cross entropy loss L CE :
[0055]
[0056] Then, according to the cross entropy loss L CE Early deep neural networks (DNNs) E Perform a gradient descent and complete a training.
[0057] In this embodiment, the total number of categories c=2, namely, two categories: normal and Alzheimer's disease patients.
[0058] Step S2.2: After completing a training round, add high-confidence medical image samples to the high-confidence sample pool according to the confidence level.
[0059] After all medical image samples are repeatedly trained, a training round is completed. The training set X is classified according to categories, and the k medical image samples with the highest confidence in each category are defined as high-confidence samples and added to the high-confidence sample pool.
[0060] After all samples are trained repeatedly, a training round is completed. The k=2 samples with the highest confidence in each category are defined as high-confidence samples and added to the high-confidence sample pool. In this embodiment, k=2. In this way, since there are 2 categories, 4 high-confidence samples are obtained in each training round.
[0061] Step S2.3: Repeat T = 5 rounds to obtain the late deep neural network DNN L , divide the training set X into high confidence sample sets X C And the noise sample set X N
[0062] Repeat step S2.1: and step S2.2 for 5 rounds to end the early training phase and the early deep neural network DNN E Transformed into a later deep neural network DNN L , the high confidence sample pool contains m1=20 high confidence samples, forming a high confidence sample set X C In the training set X, except for m1=20 high confidence samples, the rest are noise samples, the number of which is m-m1=320-20=300, and they constitute the noise sample set X N .
[0063] Step S3: Post-training
[0064] Based on the late deep neural network DNN L , high confidence sample set X C And the noise sample set X N Start reweighted training. In this embodiment, Figure 2 As shown, the later training is carried out in the reweighted training module in the training module to obtain the converged later deep neural network DNN L And as a deep neural network DNN for classification F .
[0065] In this embodiment, the schematic diagram of the training process of the reweighted training module is as follows: Figure 3 As shown, including:
[0066] Step S3.1: Based on a batch of noise samples, construct a noise sample loss vector L(X N ) and initialize the sample weight vector W0
[0067] Noise sample set X N Get a batch of noise samples (x) with a number of b = 32N_i ,Y N_i ), i = 1, 2, ..., 32, and input it into the later deep neural network DNN L , and then calculate the cross entropy loss for each noise sample. The cross entropy loss of all noise samples constitutes a noise sample loss vector L(X) of dimension 32. N ), initialize the sample weight vector W0 with dimension 32.
[0068] Step S3.2: According to the noise sample loss vector L(X N ) and the inner product of the sample weight vector W0 is the total loss, and the later deep neural network DNN′ is obtained L
[0069] For the noise sample loss vector L(X N ) and the sample weight vector W0 to obtain 32 noise samples (x N_i ,Y N_i ), i=1,2,…,b, based on the total loss of the later deep neural network DNN L The network parameters are calculated to obtain the gradient, and a pre-update is performed to obtain the later deep neural network DNN′ L , the pre-update does not update the later deep neural network DNN L , is based on the later deep neural network DNN L A gradient descent obtains a new deep neural network DNN′ L .
[0070] Step S3.3: Based on the high confidence sample set X C Compute category feature masks
[0071] For the high confidence sample set X C The high confidence sample x in C_i ,i=1,2,…,m1, based on the category feature mask after the previous iteration High confidence samples of the corresponding category Perform point multiplication to obtain high confidence samples Where j represents the jth category, j = 1, 2, ..., c, c is the total number of categories, m j Represents the high confidence sample set X C The number of high confidence samples of the j-th category in t represents the t-th iteration. If the current iteration is the first iteration, that is, t = 1, the feature mask matrix of the j-th category is initialized. Feature mask matrix The feature dimension of the medical image sample is consistent with n = 93, and the category feature mask code Each element of represents a high confidence sample of the jth category The weight of each feature.
[0072] The high confidence samples Input deep neural network DNN′ L Calculate the mean cross entropy loss and mask the category features based on the mean cross entropy loss Perform a gradient descent to obtain the category feature mask
[0073] Step S3.4: Calculate the feature weighted loss L F
[0074] Class-based feature mask High confidence samples of the corresponding category Perform point multiplication to obtain high confidence samples
[0075] Input deep neural network DNN′ L , the feature weighted loss L is calculated according to the following formula F :
[0076]
[0077] in, Indicates high confidence samples Through the deep neural network DNN′ L Get the predicted value for category j.
[0078] Step S3.5: Calculate new sample weight W1
[0079] Calculate the feature weighted loss L F Gradient of the initial sample weight W0 Then calculate the new sample weight W1:
[0080]
[0081] in, The function is to avoid the denominator being zero. hour In other cases
[0082] Step S3.6: Calculate the sample weighted loss L S
[0083] Based on the noise sample loss vector L(X N ) and the new sample weight W1, according to the calculated sample weighted loss L S :
[0084] L D = <L(x N ),W1>
[0085] in, <L(x N ),W1> represents the inner product of two vectors.
[0086] Based on the sample weighted loss L S For the later deep neural network DNN L Perform a gradient descent to obtain the updated later deep neural network DNN L , complete a reweighted training.
[0087] Step S3.7: Repeat steps S3.1 to S3.6 until the noise sample set X N All noise samples in are taken out to complete a training round, t=t+1.
[0088] Step S3.8: Repeat T′=1995 training rounds to obtain the converged late deep neural network DNN L , denoted as deep neural network DNN F .
[0089] Step S4: Use deep neural network DNN F Classifying medical image samples
[0090] Get the original medical image data, enter the data preprocessing module, get the medical image sample x, and send it to the deep neural network DNN F , and obtain its predicted value in each category. The category with the largest predicted value is the category of the medical image sample.
[0091] In this embodiment, the test set The deep neural network DNN fed into the classification module F , the deep neural network DNN after convergence F , enter the classification module, and obtain the diagnostic results and classification accuracy of the test set. The present invention and the graph convolutional neural network respectively classify and compare the Alzheimer's disease fMRI dataset with 30% symmetrical label noise and the Alzheimer's disease fMRI dataset with 50% symmetrical label noise. Figure 4 , 5 As shown, the classification accuracy of the present invention is always higher than that of the graph convolutional neural network during the entire training process.
[0092] By providing a real-time updated feature mask, the present invention can obtain the feature weight of each category feature. The higher the weight, the more important the feature is to the classification, and the feature is interpretable. At the same time, by providing a dynamically changing sample weight, the present invention can select key samples with high importance to the classification, and the sample is interpretable.
[0093] By assigning dynamically changing sample weights to training samples through high-confidence samples and pre-updated networks, the present invention can increase the weights of samples with fewer categories and samples with correct labels, thereby correcting the adverse effects of unbalanced datasets of sample categories or datasets containing label noise on model training, and improving the robustness of the model to unbalanced datasets and datasets containing label noise.
[0094] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.
Claims
1. A medical image classification method based on deep reweighted training, characterized in that: The following steps are involved: (1) Medical image data preprocessing Obtain the annotated original medical image data and enter the data preprocessing module to obtain a training set X containing m medical image samples, each of which is represented by n features; (2) Early training Early deep neural networks were defined as DNNs E , based on the early deep neural network DNN initialized with the training set X E To train: 2.1) Obtain a batch of medical image samples of number b in the training set X and train the early deep neural network DNN E Perform training, define the prediction value corresponding to the category marked in the label vector of each medical image sample as the confidence and record the confidence of each medical image sample; 2.2) After repeating the training of all medical image samples, a training round is completed. The training set X is classified according to categories. The k medical image samples with the highest confidence in each category are defined as high-confidence samples and added to the high-confidence sample pool; 2.3) Repeat steps 2.1 and 2.2 for T rounds to end the early training phase. The early deep neural network DNN E Transformed into a later deep neural network DNN L , the high confidence sample pool contains m1 high confidence samples, forming a high confidence sample set X C , except for m1 high confidence samples in the training set X, the rest are noise samples, the number is m-m1, and they constitute the noise sample set X N ; (3) Post-training Based on the late deep neural network DNN L , high confidence sample set X C And the noise sample set X N Start reweighted training: 3.1) Noise sample set X N Get a batch of noise samples (x N_i ,Y N_i ), i = 1, 2, ..., b, and input it into the later deep neural network DNN L , and then calculate the cross entropy loss for each noise sample. The cross entropy loss of all noise samples constitutes a noise sample loss vector L(X N ), initialize the sample weight vector W0 with dimension b; 3.2) For the noise sample loss vector L(X N ) and the sample weight vector W0 to obtain b noise samples (x N_i ,Y N_i ), i=1,2,…,b, based on the total loss of the later deep neural network DNN L The network parameters are calculated to obtain the gradient, and a pre-update is performed to obtain the later deep neural network DNN′ L , the pre-update does not update the later deep neural network DNN L , is based on the later deep neural network DNN L A gradient descent obtains a new deep neural network DNN′ L ; 3.3) For the high confidence sample set X C The high confidence sample x in C_i ,i=1,2,…,m1, based on the category feature mask after the previous iteration High confidence samples of the corresponding category Perform point multiplication to obtain high confidence samples in, j represents the jth category, j = 1, 2, ..., c, c is the total number of categories, m j Represents the high confidence sample set X C The number of high confidence samples of the j-th category in t represents the t-th iteration. If the current iteration is the first iteration, that is, t = 1, the feature mask matrix of the j-th category is initialized. Feature mask matrix The feature dimension of the medical image sample is consistent with n, and the category feature mask code Each element of represents a high confidence sample of the jth category The weight of each feature; The high confidence samples Input deep neural network DNN′ L Calculate the mean cross entropy loss and mask the category features based on the mean cross entropy loss Perform a gradient descent to obtain the category feature mask 3.4) Based on category feature mask High confidence samples of the corresponding category Perform point multiplication to obtain high confidence samples Input deep neural network DNN′ L , the feature weighted loss L is calculated according to the following formula F : in, Indicates high confidence samples Through the deep neural network DNN′ L Get the predicted value of category j; 3.5) Calculate the feature weighted loss L F Gradient of the initial sample weight W0 Then calculate the new sample weight W1: in, The function is to avoid the denominator being zero. hour In other cases 3.6) Based on the noise sample loss vector L(X N ) and the new sample weight W1, calculate the sample weighted loss L S : L S = <L(x N ),W1> in, <L(x N ),W1> represents the inner product of two vectors; Based on the sample weighted loss L S For the later deep neural network DNN L Perform a gradient descent to obtain the updated later deep neural network DNN L , complete a reweighted training; 3.7) Repeat steps 3.1) to 3.6) until the noise sample set X N All noise samples are taken out to complete a training round, t = t + 1; 3.8) Repeat T′ training rounds to obtain the converged late-stage deep neural network DNN L , denoted as deep neural network DNN F ; (4) Classification of medical image samples Obtain the original medical image data, enter the data preprocessing module, obtain medical image samples, and send them to the deep neural network DNN F , and obtain its predicted value in each category. The category with the largest predicted value is the category of the medical image sample.
2. The medical image classification method based on deep reweighted training according to claim 1, characterized in that: The medical image data is functional magnetic resonance imaging data. The data is preprocessed including spatial distortion correction, skull stripping and cerebellum removal. The functional magnetic resonance imaging data is then segmented into gray matter, white matter and cerebrospinal fluid, and 93 regions of interest are obtained using the Jacob brain partition template. Each medical image sample is represented by n=93 region of interest features, b=32, and the total number of categories c=2, namely two categories: normal and Alzheimer's patients.