A fiber segmentation method based on co-teaching and noisy label learning

Abstract

The present application belongs to the technical field of medical image processing, and particularly relates to a fiber segmentation method based on co-teaching and noise label learning. The method comprises the following steps: constructing a training data set; pre-training two Piont Net networks using expert-labeled brain fiber picture labels; inputting the training data set into the two pre-trained networks to label each sample with a pseudo label, and inputting the labeled samples into the two pre-trained networks respectively; dividing all the labeled samples into high-quality labels and noise labels, and allowing the two networks to teach each other based on the co-teaching idea; randomly selecting part of the noise labels corresponding to the data from the noise labels screened from the two networks for data enhancement processing and predicting the output; calculating the loss function of the two networks during the training process, and updating the parameters of the two networks; correcting the noise labels that meet the conditions using the predicted output after the data enhancement processing; and continuously iterating to obtain the label of the fiber that needs to be segmented finally.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical image processing, and particularly relates to a fiber segmentation method based on co-teaching and noise label learning. BACKGROUND

[0002] Automatic fiber segmentation plays a vital role in brain science research. First, automatic human brain fiber segmentation technology can help neuroscientists to quantitatively analyze brain nerve axons in research and make medical diagnosis based on the same. In addition, when planning brain surgery through medical images, doctors can use segmented fiber bundle maps to achieve micro-precision surgical treatment, which will ensure minimal interference and optimal results. Zhangfan et al. proposed a fast and stable fiber segmentation method DeepWMA (Deep white matter analysis), which encodes the spatial information of each fiber bundle to form a three-channel two-dimensional image FiberMap, and uses the two-dimensional image FiberMap as the input of the neural network to classify the fibers by using the neural network.

[0003] DeepWMA belongs to full-supervised learning and relies on a large training data set with high-quality artificial annotation. Manual annotation of brain fibers is time-consuming, costly and difficult to annotate, and is difficult to obtain in many clinical applications, so in practical applications, it is difficult to obtain and construct the training data set. SUMMARY

[0004] The purpose of the application is to provide a fiber segmentation method based on co-teaching and noise label learning, which uses fewer expert annotated labels to train the fiber segmentation framework, and the trained fiber segmentation framework can perform fiber segmentation processing on unannotated brain fiber images, effectively alleviating the problem of high cost and difficulty of manual annotation of brain fiber images by experts.

[0005] To achieve the above purpose, the application adopts the following technical solutions:

[0006] A fiber segmentation method based on co-teaching and noise label learning, comprising:

[0007] S1, obtaining a whole brain fiber image without association as a data sample and constructing a training data set;

[0008] S2, using the expert annotated brain fiber image label as input data to pre-train Piont Net1 and Piont Net2;

[0009] S3, input the training data set into the pre-trained Piont Net1 and Piont Net2 respectively, the pre-trained Piont Net1 and Piont Net2 both label each sample data in the training data set with a pseudo label (positive voxel and negative voxel) through a classifier, and input the sample data with the label into the pre-trained Piont Net1 and Piont Net2 respectively;

[0010] S4, the Piont Net1 and Piont Net2 both filter the sample data with the pseudo label and the sample data with the label according to a threshold, and divide the label of the sample data into a high-quality label and a noise label according to the threshold;

[0011] S5, input the high-quality label and noise label filtered by the Piont Net1 into the Piont Net2 for re-discrimination, and input the high-quality label and noise label filtered by the Piont Net2 into the Piont Net1 for re-discrimination, to obtain the final filtering result of the Piont Net1 and the final filtering result of the Piont Net2, adopt a parallel network by using the co-teaching idea, reduce the accumulation and propagation of errors, and make the Piont Net1 and Piont Net2 have high robustness;

[0012] S6, randomly select a part of the noise label from the noise label finally filtered by the Piont Net1 and Piont Net2 respectively, perform data enhancement processing on the data corresponding to the noise label, and predict the output after the data enhancement processing, to mine more potential features from the image and reduce data loss;

[0013] S7, calculate the loss function of the Piont Net1 and Piont Net2 in the training process, and update the parameters of the Piont Net1 and Piont Net2 according to the loss function;

[0014] S8, after the Piont Net1 and Piont Net2 are updated, correct the noise label meeting a preset condition using the predicted output after the data enhancement processing;

[0015] S9, continuously iterate to obtain the label of the fiber finally needing to be segmented.

[0016] A part of the noise labels selected randomly from the noise labels screened out finally from Piont Net1 and Piont Net2 are subjected to data enhancement, and then prediction is performed again instead of discarding all the noise labels, so that more potential features can be mined from the images, and data loss can be reduced. The threshold for dividing high-quality labels and noise labels is continuously improved as the training proceeds.

[0017] The application trains a fiber segmentation framework by using fewer expert-labeled labels, and the fiber segmentation framework trained can perform fiber segmentation processing on unlabeled brain fiber images, effectively alleviating the problems of high labeling cost and difficult labeling when experts manually label human brain fiber images.

[0018] As a preferred solution, Piont Net1 and Piont Net2 each consist of three fully connected layers and one max-pooling layer connected in sequence, the neurons used in the fully connected layers share parameters, each fully connected layer uses Batch Normalization technology to facilitate network convergence, and each fully connected layer uses ReLU as an activation function.

[0019] As a preferred solution, when pre-training Piont Net1 and Piont Net2, segmentation loss is used as a loss function.

[0020] As a preferred solution, the classifier is a two-layer fully connected network, used for binary classification of positive fiber bundles and negative fiber bundles, and cross-entropy loss is used as a loss function.

[0021] As a preferred solution, the data corresponding to a part of the noise labels selected randomly is subjected to data enhancement processing, specifically, the data corresponding to the noise labels selected randomly is subjected to rotation or inversion processing to mine potential features existing in the images.

[0022] As a preferred solution, in step S7, the loss function is obtained by weighted calculation of consistency loss and segmentation loss.

[0023] As a preferred solution, the consistency loss is obtained by mean square loss, used to measure the difference between the predicted value y' and the true value y, and the calculation formula of the consistency loss is:

[0024]

[0025] wherein y i represents the true value of the i-th label in a batch, y' i represents the predicted value of the i-th label in a batch, N represents the total number of labels in a batch, and L con represents the consistency loss of y' about y.

[0026] The segmentation loss is calculated by weighting the Dice loss and the cross-entropy loss, and the calculation formula of the segmentation loss is as follows:

[0027]

[0028] wherein L Dice represents the Dice loss, L CE represents the cross-entropy loss, a represents a weighting coefficient, and e represents a constant ensuring the stability of the loss function value, L seg represents the segmentation loss of y' with respect to y.

[0029] As a preferred solution, the loss function of Piont Net1 is as follows:

[0030]

[0031] The loss function of Piont Net2 is as follows:

[0032]

[0033] wherein N i (y|d j ; wi) N i represents training by Piont Neti, d j represents the input sample data for training, wi represents the network parameters of N i , d hi , d li , respectively represent data corresponding to high-quality labels, data corresponding to noise labels, and enhanced data, respectively represent high-quality labels, noise labels, and noise labels generated after data enhancement screened by Piont Neti, N1 represents the total number of labels in a batch in Piont Net1, N2 represents the total number of labels in a batch in Piont Net2, w1 and w2 respectively represent the network parameters of Piont Net1 and Piont Net2, L1 and L2 respectively represent the loss functions of Piont Net1 and Piont Net2, and l q represents a hyperparameter, and q w represents a warm-up training period, and q represents the current training period.

[0034] As a preferred solution, the network parameters w1 of Piont Net1 and the network parameters w2 of Piont Net2 are calculated according to the gradient descent principle, and the calculation formula is as follows:

[0035] w1=w1'-lr▽L1,

[0036] w2=w2'-lr▽L2,

[0037] wherein w1, w2 represent the updated network parameters, w1', w2' represent the network parameters before updating, lr represents the learning rate, represents the loss change gradient of Piont Net1, represents the loss change gradient of Piont Net2.

[0038] As a preferred solution, in step S8, the output after data enhancement processing is used to correct the 25% of the noise labels with the lowest Dice score, and the Dice score L Dice The calculation formula is:

[0039]

[0040] wherein TP represents the number of true positive predictions, FP represents the number of false positive predictions, FN represents the number of false negative predictions, L Dice represents the Dice score.

[0041] The beneficial effects of the present application are:

[0042] 1. The co-teaching and pseudo-label ideas are improved and applied to fiber segmentation, so that the result of fiber segmentation is more accurate.

[0043] 2. A part of the noise labels corresponding to the data are randomly selected for data enhancement from the noise labels finally screened out from Piont Net1 and Piont Net2, and then re-predicted, instead of discarding all the noise labels, which can mine more potential features from the image and reduce data loss.

[0044] 3. The present application uses fewer expert labeled labels to train the fiber segmentation framework, and the trained fiber segmentation framework can perform fiber segmentation processing on unlabeled brain fiber images, effectively alleviating the problem that the cost of manual labeling of brain fiber images by experts is high and the labeling is difficult. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0046] Figure 1 is a flowchart of a fiber segmentation method based on co-teaching and noise label learning. DETAILED DESCRIPTION

[0047] The present application is herein described, by way of example only, with the comprehension that the advantages and utility thereof are not confined thereto. It will be clear that, in the light of the foregoing disclosure, additional embodiments of the application will suggest themselves to those skilled in the art. Where necessary, the reference numerals included in the following description have their significance as set forth in the following description. Where not in conflict, the embodiments described in the following description can also be combined with one another in an arbitrary manner.

[0048] Reference Figure 1 A fiber segmentation method based on co-teaching and noise label learning, comprising:

[0049] S1, obtain a whole brain fiber image irrelevant to association as a data sample, and construct a training data set. Collect whole brain fiber images irrelevant to the fiber segmentation task, and group them into a training data set to prepare for subsequent fiber segmentation framework training.

[0050] S2, input the brain fiber picture label annotated by an expert as input data, and pre-train Piont Net1 and Piont Net2; this helps Piont Net1 and Piont Net2 network learn the features and structure of brain fibers.

[0051] S3, input the training data set into the pre-trained Piont Net1 and Piont Net2 respectively, and the pre-trained Piont Net1 and Piont Net2 both label the sample data in the training data set with pseudo labels (positive voxel and negative voxel) through classifier, and input the sample data with labels into the pre-trained Piont Net1 and Piont Net2 respectively.

[0052] S4, Piont Net1 and Piont Net2 both filter the sample data with pseudo labels and the sample data with labels according to the threshold, and divide the labels of the sample data into high-quality labels and noise labels according to the threshold. Noise label learning uses the characteristics of DNN (Deep Neural Network) to preferentially remember "simple" (high-quality) samples, and then fit "complex" (noise) samples, so a relatively high threshold will be set when filtering labels to try to filter out high-quality labels. The threshold for dividing high-quality labels and noise labels will be continuously improved as the training progresses.

[0053] S5, Piont Net1, Piont Net2 based on co-teaching idea mutual teaching, to get Piont Net1 final screening result and Piont Net2 final screening result.Piont Net1 screened high-quality label, noise label is input into Piont Net2 again discrimination, Piont Net2 screened high-quality label, noise label is input into Piont Net1 again discrimination, utilize co-teaching idea adopts parallel network, reduces the accumulation and propagation of error, makes Piont Net1, Piont Net2 all have higher robustness.

[0054] S6, randomly select a part of noise labels from the noise labels screened out by Piont Net1 and Piont Net2 respectively, perform data enhancement processing on the data corresponding to the noise labels, and predict the output after data enhancement processing;From the noise labels screened out by Piont Net1 and Piont Net2, a part of noise labels corresponding to the data are randomly selected for data enhancement, and then re-predicted, instead of discarding all noise labels, which can extract more potential features from the image and reduce data loss.

[0055] S7, calculate the loss function of Piont Net1 and Piont Net2 in the training process, and update the parameters of Piont Net1 and Piont Net2 according to the loss function.

[0056] S8, after updating Piont Net1 and Piont Net2, use the prediction output after data enhancement processing to correct the noise labels that meet the preset conditions, so that the fiber segmentation framework has the ability of self-evolution.

[0057] S9, continuously iterate to obtain the label of the fiber that needs to be segmented finally.

[0058] The application trains the fiber segmentation framework with fewer expert annotated labels, and the trained fiber segmentation framework can perform fiber segmentation processing on unannotated brain fiber images, effectively alleviating the problem that manual annotation of brain fiber images by experts is high in cost and difficult to annotate.

[0059] Further, Piont Net1 and Piont Net2 are each composed of three fully connected layers (FC) and a max-pooling layer connected in sequence, and each neuron used in the fully connected layer shares parameters. Each fully connected layer uses the Batch Normalization technology to facilitate network convergence, and each fully connected layer uses ReLU as an activation function. Adding a BN layer in Piont Net can effectively solve the problem of internal covariate shift in the network, improve the training speed and accuracy of the network. At the same time, the BN technology also has a certain regularization effect, which can alleviate the problem of overfitting.

[0060] Further, when pre-training Piont Net1 and Piont Net2, a segmentation loss is used as a loss function.

[0061] Further, the classifier is a two-layer fully connected network, which is used for binary classification of positive and negative fiber bundles, and a cross-entropy loss is used as a loss function.

[0062] Further, a part of the data corresponding to the noise labels is randomly selected for data enhancement processing, specifically, the data corresponding to the randomly selected noise labels is rotated or inverted to mine potential features in the image. A part of the noise labels is randomly selected from the noise labels finally screened out by Piont Net1 and Piont Net2, the data corresponding to the noise labels is enhanced, and then Piont Net1 and Piont Net2 are used for re-prediction. This method similar to data distillation selectively puts the noise labels into training, rather than discarding all noise labels, and utilizes the network characteristics of the DNN to make the originally unlabeled data better utilized, to mine more potential features from the image, and to reduce data loss.

[0063] Further, the loss function is calculated by weighting the consistency loss and the segmentation loss. The introduction of the segmentation loss can avoid the negative impact of high noise labels. The introduction of the consistency loss can predict the consistency loss of the noise labels before and after data enhancement, and the noise labels without data enhancement are sharpened by temperature before consistency loss prediction to prevent overfitting during training.

[0064] The consistency loss is obtained by a mean square loss, which is used to measure the difference between the predicted value y' and the true value y, and the calculation formula of the consistency loss is:

[0065]

[0066] where yi represents the true value of the i-th label in a batch, y i represents the predicted value of the i-th label in a batch, N represents the total number of labels in a batch, L con represents the consistency loss of y' with respect to y;

[0067] The segmentation loss is calculated by weighting the Dice loss and the cross-entropy loss, and the calculation formula of the segmentation loss is:

[0068]

[0069] wherein, L Dice represents the Dice loss; L CE represents the cross-entropy loss; a represents a weighting coefficient, which is set to 1, so that the Dice loss and the cross-entropy loss are equally weighted; and e represents a constant to ensure the stability of the loss function value, which is set to 1; L seg represents the segmentation loss of y' with respect to y.

[0070] By controlling the hyperparameter λ q , the consistency loss and the segmentation loss are weighted to obtain the loss functions of Piont Net1 and Piont Net2 respectively, so that the proportion of the consistency loss of the noise label gradually decreases in the training, and the proportion of the consistency loss of the noise label screened after data augmentation gradually increases in the training. This can reduce the side effects of the noise label and make the most of the effective information in the noise label.

[0071] The loss function of Piont Net1 is:

[0072]

[0073] The loss function of Piont Net2 is:

[0074]

[0075]

[0076] wherein, N i (y|d j ; wi) in which N i represents training by Piont Neti, d j represents the input sample data for training, wi represents the network parameters of N i , d hi , d li , respectively represent the data corresponding to the high-quality label, the data corresponding to the noise label, and the data after augmentation, respectively represent the high-quality label, the noise label, and the noise label generated after data augmentation obtained by Piont Neti, N1 represents the total number of labels in one batch in Piont Net1, N2 represents the total number of labels in one batch in Piont Net2, w1 and w2 respectively represent the network parameters of Piont Net1 and Piont Net2, L1 and L2 respectively represent the loss functions of Piont Net1 and Piont Net2, and λ q represents a hyperparameter, and q w represents a preheating training period, and q represents a current training period. This step mainly calculates the segmentation loss and consistency loss of the second output label and the first output label.

[0077] Further, the network parameters w1 of Piont Net1 and the network parameters w2 of Piont Net2 are calculated according to the gradient descent principle, and the calculation formula is as follows:

[0078]

[0079]

[0080] wherein w1 and w2 represent updated network parameters, w1' and w2' represent network parameters before updating, lr represents a learning rate, represents the loss change gradient of Piont Net1, represents the loss change gradient of Piont Net2.

[0081] Further, in step S8, the output after data augmentation is used to correct the 25% of the noise labels with the lowest Dice score, and the calculation formula of the Dice score L Dice is as follows:

[0082]

[0083] wherein TP represents the number of true positive predictions, FP represents the number of false positive predictions, FN represents the number of false negative predictions, and L Dice represents the Dice score.

[0084] In the early stage of training, the label is updated once every training period before reaching the preset preheating training period q w ; thereafter, the label only needs to be updated once every 10 training periods. The corrected label is iterated constantly, and finally the label required for fiber segmentation is obtained. The fiber segmentation framework only performs segmentation on one pair of fiber tracts each time, and the output result in the drawing is the effect obtained after multiple segmentations.

[0085] The application trains a fiber segmentation framework by using less expert-labeled tags, and the trained fiber segmentation framework can perform fiber segmentation processing on unlabeled brain fiber images, effectively alleviating the problems of high labeling cost and difficult labeling when experts manually label human brain fiber images.

[0086] The above-described embodiments are merely preferred embodiments of the present application and are not intended to limit the scope of the present application. Various modifications and improvements to the technical solutions of the present application made by those of ordinary skill in the art without departing from the design spirit of the present application shall fall within the protection scope of the present application.

Claims

1. A fiber segmentation method based on co-teaching and noise label learning, characterized in that, include: S1. Obtain unrelated whole-brain fiber images as data samples and construct a training dataset; S2. Use expert-annotated brain fiber image labels as input data to pre-train PinotNet1 and PinotNet2; S3. Input the training dataset into the pre-trained PinotNet1 and PinotNet2 respectively. Both PinotNet1 and PinotNet2 use classifiers to assign pseudo-labels to each sample data in the training dataset, and input the labeled sample data into the pre-trained PinotNet1 and PinotNet2 respectively. S4, PinotNet1, and PinotNet2 all use thresholds to filter sample data with pseudo-labels and labeled sample data, and classify the labels of sample data into high-quality labels and noisy labels according to the thresholds. S5, PinotNet1, and PinotNet2 teach each other based on the idea of ​​co-teaching to obtain the final selection results of PinotNet1 and PinotNet2. S6. During mutual teaching, randomly select a portion of the data corresponding to the noise labels for data augmentation, and predict the output after data augmentation. S7. Calculate the loss functions of PinotNet1 and PinotNet2 during training, and update the parameters of PinotNet1 and PinotNet2 based on the loss functions; S8. After updating Pinot Net1 and Pinot Net2, use the prediction output after data augmentation to correct the noise labels that meet the preset conditions. S9. Iterate continuously to obtain the labels for the fibers that need to be split in the end; In step S7, the loss function is calculated by weighting the consistency loss and the segmentation loss; The formula for calculating consistency loss is: Among them, y i Let y' represent the true value of the i-th label in a batch. i Let L represent the predicted value of the i-th label in a batch, N represent the total number of labels in a batch, and L represent the predicted value of the i-th label in a batch. con This represents the consistency loss of y' with respect to y; The formula for calculating the segmentation loss is: Among them, L Dice L represents Dice loss. CE Let L represent the cross-entropy loss, α represent the weighting coefficient, ε represent the constant that ensures the stability of the loss function value, and L represent the cross-entropy loss. seg Let y' represent the segmentation loss of y with respect to y.

2. The fiber segmentation method based on co-teaching and noise label learning according to claim 1, characterized in that, Both PinotNet1 and PinotNet2 consist of three fully connected layers and a max pooling layer connected sequentially. Each fully connected layer uses BatchNormalization to promote network convergence, and ReLU is used as the activation function for each fully connected layer.

3. The fiber segmentation method based on co-teaching and noise label learning according to claim 1, characterized in that, When pre-training PinotNet1 and PinotNet2, segmentation loss is used as the loss function.

4. The fiber segmentation method based on co-teaching and noise label learning according to claim 1, characterized in that, The classifier is a two-layer fully connected network used for binary classification of positive and negative fiber bundles, and uses cross-entropy loss as the loss function.

5. The fiber segmentation method based on co-teaching and noise label learning according to claim 1, characterized in that, The data augmentation process involves randomly selecting a portion of the data corresponding to the noise labels and rotating or reversing the data to uncover potential features in the image.

6. The fiber segmentation method based on co-teaching and noise label learning according to claim 1, characterized in that, The loss function of PiontNet1 is: The loss function of Piont Net2 is: Where, N i (y|d j ;wi) in N i This indicates that it is trained using PinotNeti, d j Let wi represent the input training sample data, where N is the input. i Network parameters, d hi d li , These represent the data corresponding to high-quality labels, the data corresponding to noisy labels, and the enhanced data, respectively. Let N1 and N2 represent the high-quality labels, noisy labels, and noisy labels generated after data augmentation, respectively, selected by Pinot Neti. Let N1 represent the total number of labels in a batch of Pinot Net1, and N2 represent the total number of labels in a batch of Pinot Net2. Let w1 and w2 represent the network parameters of Pinot Net1 and Pinot Net2, respectively, and let L1 and L2 represent the loss functions of Pinot Net1 and Pinot Net2, respectively. q Represents the hyperparameter, q w q represents the warm-up training period, and q represents the current training period.

7. The fiber segmentation method based on co-teaching and noise label learning according to claim 6, characterized in that, The parameters w1 of the PinotNet1 network and w2 of the PinotNet2 network are calculated based on the gradient descent principle. The calculation formula is as follows: w1=w1'-lr▽L1, w2=w2'-lr▽L2, Where w1 and w2 represent the updated network parameters, w1' and w2' represent the original network parameters, lr represents the learning rate, ▽L1 represents the gradient of the loss change of PinotNet1, and ▽L2 represents the gradient of the loss change of PinotNet2.

8. The fiber segmentation method based on co-teaching and noise label learning according to claim 1, characterized in that, In step S8, the output after data augmentation is used to correct the noise labels in the lowest 25% of the Dice scores, where the Dice score L is [not specified]. Dice The calculation formula is: Where TP represents the number of voxels predicted as true positives, FP represents the number of voxels predicted as false positives, FN represents the number of voxels predicted as false negatives, and L... Dice This indicates Dice's score.

Images