Intelligent early screening system for lung cancer

Abstract

The invention provides an intelligent early screening system for lung cancer. The intelligent early screening system comprises a data enhancement module, a discriminative feature extraction module and a fine adjustment module, the data enhancement module is used for generating a weak enhancement sample and a strong enhancement sample; the discriminative feature extraction module is used for extracting corresponding strong and weak enhanced sample feature vectors and classification vectors, respectively performing comparative learning and discriminative learning on the strong and weak enhanced sample feature vectors and classification vectors, shortening the distance between semantically related samples and excluding semantically unrelated samples; and the fine tuning module is used for optimizing the feature extractor and enhancing the accuracy of model classification through comparative learning of an original image with label data and a corresponding weak enhancement sample.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent medical care, relates to the technology of semi-supervised deep network models, and specifically relates to an early screening system for lung cancer. Background Art

[0002] Lung cancer is the leading cause of cancer death worldwide today, and the early detection of lung cancer is of great significance for reducing lung cancer-related mortality. CT imaging is currently an internationally recognized method for early screening of lung cancer. Among them, lung nodules are a common manifestation in lung CT imaging examinations. These nodules may be benign, such as caused by inflammation, infection, tuberculosis, or scars left after healing of old injuries; or they may be malignant, which is an early manifestation of lung cancer. Therefore, the identification of benign and malignant lung nodules has become a key link in the early detection and intervention of lung cancer.

[0003] At present, the diagnosis of clinical lung cancer mainly relies on the radiologists' careful review and analysis of CT images. This process is not only time-consuming, but also in a long-term, high-intensity working environment, even experienced radiologists may become fatigued and their judgment may decline, which in turn affects the accuracy of the diagnosis. In addition, the heterogeneity of lung nodules poses a great challenge to the diagnosis of lung cancer. Even experienced radiologists sometimes find it difficult to accurately distinguish between benign and malignant nodules. Intelligent diagnostic methods based on deep learning can quickly and accurately perform quantitative analysis of CT images, and have shown good development prospects in the field of benign and malignant lung nodule identification.

[0004] Most of the existing deep learning methods for lung nodule recognition are supervised learning methods. Although they have achieved certain effectiveness, their limitations are that they require a large amount of labeled data and have poor generalization when faced with new samples. In the medical field, imaging data comes from expensive medical equipment, and labels come from time-consuming and laborious analysis by multiple medical experts, making data acquisition extremely difficult. In addition, data labels may contain some sensitive information that requires privacy protection.

[0005] In the face of the above problems, semi-supervised deep learning methods have emerged. However, the research on benign and malignant lung nodules identification methods based on semi-supervised deep learning is still in its early stages and has some shortcomings. On the one hand, these semi-supervised learning lung nodule classification methods use data augmentation methods to expand the data of the same sample, trying to obtain more detailed information about the lung nodules, but do not consider the relevant information between different samples of the same category, ignoring the diversity and consistency between samples of the same category, resulting in the misclassification of benign and malignant nodules, and the model accuracy is reduced. On the other hand, these methods use affine transformation and histogram equalization as data augmentation methods. This type of data augmentation method is more suitable for natural images and has not been improved and designed according to the characteristics of lung nodules, resulting in the potential information between samples not being properly utilized, resulting in weak generalization ability of the model.

[0006] In summary, how to ensure the accuracy of the model while improving the generalization ability of the model under the premise of limited labeled data and a large amount of unlabeled data is an urgent problem to be solved in the current intelligent early screening system for lung cancer.

[0007] Related Literature

[0008] [1] Armato III SG, McLennan G, Bidaut L, et al. The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans [J]. Medical physics, 2011, 38 (2): 915-931.

[0009] [2]Xie Y, Zhang J, Xia Y. Semi-supervised adversarial model for benign–malignant lung nodule classification on chest CT[J]. Medical image analysis, 2019,57:237-248.

[0010] [3] Apostolopoulos ID, Papathanasiou ND, Panayiotakis GS. Classification of lung nodule malignancy in computed tomography imaging utilizing generative adversarial networks and semi-supervised transfer learning [J]. Biocybernetics and Biomedical Engineering, 2021, 41(4): 1243-1257.

[0011] [4]Fu Y,Xue P,Xiao T,et al.Semi-supervised adversarial learning forimproving the diagnosis ofpulmonary nodules[J].IEEE Journal ofBiomedicalandHealth Informatics,2022,27(1):109-120.

[0012] [5]Zhu W,Liu C,Fan W,et al.Deeplung:Deep 3d dual path nets forautomated pulmonary noduledetection and classification[C] / / 2018IEEE winterconference on applications of computer vision(WACV).IEEE,2018:673-681.

[0013] [6]Shen W,Zhou M,Yang F,et al.Multi-scale convolutional neuralnetworks for lung nodule classification[C] / / Information Processing in MedicalImaging:24th International Conference,IPMI 2015,Sabhal Mor Ostaig,Isle ofSkye,UK,June 28-July 3,2015,Proceedings 24.Springer International Publishing,2015:588-599.

[0014] [7]Xie Y,XiaY,Zhang J,et al.Knowledge-based collaborative deeplearning for benign-malignant lung nodule classification on chest CT[J].IEEEtransactions on medical imaging,2018,38(4):991-1004.

[0015] [8] Jiang H, Shen F, Gao F, et al. Learning efficient, explainable and discriminative representations for pulmonary nodules classification [J]. Pattern Recognition, 2021,113:107825.

[0016] [9]Xu

[0017]

[10] Al-Shabi M,Shak K,Tan M.ProCAN: Progressive growing channel attentive non-local network for lung nodule classification[J].PatternRecognition,2022,122:108309.

[0018]

[11] Chen T, Kornblith S, Norouzi M, et al. A simple framework for contrastive learning of visual representations[C] / / International conferenceon machine learning.PMLR, 2020:1597-1607.

[0019]

[12] Van der Maaten L,Hinton G.Visualizing data using t-SNE[J].Journalofmachine learning research,2008,9(11). Summary of the invention

[0020] In view of the above problems, the present invention proposes an intelligent early screening system for lung cancer, which can be used to realize the classification of benign and malignant lung nodules based on semi-supervised deep learning. The technical solution of the present invention is as follows:

[0021] A lung cancer intelligent early screening system includes a data enhancement module, a discriminative feature extraction module and a fine-tuning module; the data enhancement module is used to generate weakly enhanced samples and strongly enhanced samples; the discriminative feature extraction module is used to extract the corresponding strong and weak enhanced sample feature vectors and classification vectors, and respectively perform comparative learning and discriminative learning on the two, narrowing the distance between semantically related samples and excluding semantically irrelevant samples; the fine-tuning module is used to optimize the feature extractor, and enhance the accuracy of model classification through comparative learning of original images with labeled data and corresponding weakly enhanced samples; wherein,

[0022] The discriminative feature extraction module includes a feature contrast learning submodule based on high-confidence pseudo-labels, a classification vector contrast learning submodule based on enhanced anchoring, and a discriminative learning submodule. Unlabeled weakly enhanced samples, strongly enhanced samples, and labeled weakly enhanced samples are input into the backbone network, and then the feature vectors and classification vectors of the unlabeled samples are taken out and input into the feature contrast learning submodule based on high-confidence pseudo-labels and the classification vector contrast learning submodule based on enhanced anchoring, and the classification vectors of the labeled samples and the classification vectors of the unlabeled strongly enhanced samples are sent to the discriminative learning submodule.

[0023] In the feature contrast learning submodule based on high-confidence pseudo-labels, unlabeled weakly enhanced samples and strongly enhanced samples are respectively pseudo-labeled through the classification layer of the backbone network; unlabeled weakly enhanced samples and strongly enhanced samples are constructed into sample pairs based on the pseudo-labels;

[0024] Based on the enhanced anchoring classification vector comparison learning submodule, unlabeled weakly enhanced samples and strongly enhanced samples obtain corresponding classification vectors through the backbone network, which are used to construct ordered tuple pairs; by pulling semantically related classification vectors in the ordered tuple pairs closer and pulling semantically irrelevant classification vectors farther apart, the knowledge between the classification vectors of unlabeled nodules at the classification level is learned;

[0025] The discriminative learning submodule uses the cross entropy loss function to compare the classification vector of the labeled weakly enhanced sample with the true label to obtain the supervised loss, and compares the classification vector of the unlabeled strongly enhanced sample with its corresponding pseudo label to obtain the unsupervised loss, thereby realizing the classification of benign and malignant pulmonary nodules.

[0026] The fine-tuning module is based on the model obtained after the semi-supervised learning training is completed. Only labeled nodules are used to construct the original image of the lung nodule and the weakly enhanced image sample pair. Based on the real labels of the samples, the similarities between samples of the same type and the differences between samples of different types are learned to correct some errors in semi-supervised learning and improve network performance.

[0027] The fine-tuning module adopts a supervised contrastive learning method. At the feature level of labeled nodules, the original image of the labeled nodules is retained, and the weakly enhanced image of the labeled nodules is obtained through the data enhancement module. The true labels of the labeled nodules are used and assigned to the weakly enhanced samples of the labeled samples.

[0028] Furthermore, the data enhancement module includes a weak enhancement submodule and a strong enhancement submodule; the weak enhancement submodule T weak The random horizontal flipping and random cropping methods are used on the lung nodule samples to obtain the corresponding weakly enhanced samples; the strong enhancer module T strong On the basis of the weak enhancement submodule, random rotation, contrast enhancement and CutOut methods are added to obtain strong enhancement samples; in the labeled dataset X, a data sample x i Only through the weak enhancer module T weak Get the weakly enhanced sample of the data sample, that is, In the unlabeled dataset U, a data sample u i , respectively through the weak enhancer module T weak and strong enhancer module T strong Get the weakly enhanced sample and the strongly enhanced sample of the data sample, that is,

[0029] Furthermore, EfficientNet is selected as the backbone network.

[0030] Furthermore, in the feature contrast learning submodule based on high-confidence pseudo-labels, for a feature vector of a specific sample When a batch has 2N unlabeled weakly enhanced samples and strongly enhanced samples, 2N-1 ordered sample pairs are matched, among which the samples with the same pseudo-label constitute positive sample pairs, and the rest constitute negative sample pairs. The potential relationship between unlabeled nodules is used at the feature level; the loss function calculation formula for feature contrast learning is as follows:

[0031]

[0032] Among them, U contains the 2N unlabeled sample augmentation data in an input batch, · represents the inner product, P(i) represents all positive samples with the same pseudo-label corresponding to sample i in the current input batch, A(i) represents all negative samples with different pseudo-labels in the current input batch, and z i , z p , z a represents the unlabeled data augmentation sample feature vector, M T Represents the confidence mask, and its specific mathematical expression is as follows:

[0033]

[0034] Among them, T c represents the confidence threshold, Indicates the threshold value is T c A strict threshold function is used to filter out samples with high pseudo-label confidence to avoid erroneous positive and negative sample pairs that hinder contrastive learning.

[0035] Furthermore, in the classification vector contrast learning submodule based on enhanced anchoring, the method of constructing ordered tuples is as follows: the feature vector of the unlabeled data augmented sample in an input batch The corresponding classification vector is obtained through the classification layer of the backbone network Weakly enhanced classification vector for unlabeled samples Combine two by two to construct ordered tuple pairs.

[0036] Furthermore, the loss function of the classification vector contrast learning based on enhanced anchoring is expressed as:

[0037]

[0038] Where N is the number of samples in the batch, f sim (·) represents the similarity threshold measurement function

[0039] like The highest class prediction value in exceeds the confidence threshold T c , and in the ordered tuple pair if p i If it is located in the main sequence position, it is used as an "anchor point", and the similarity measurement function calculates the similarity between the main sequence and the secondary order prediction vectors in all tuple pairs whose main sequence elements are "anchor points", so as to gather the secondary order classification vectors related to the main sequence category to the main sequence classification vector;

[0040] f dist (·) represents the distance function, which is used to represent the distance between the classification vector of the strongly enhanced sample corresponding to the secondary sample and the classification vector of the main sequence "anchor point", so as to align the classification vector of the strongly enhanced sample with the "anchor point";

[0041] T s represents the similarity threshold, Indicates the threshold value is T s A strict threshold function is used to screen reliable tuple pairs, that is, if the similarity between the primary order and the secondary order prediction vector in the tuple pair is lower than the preset similarity threshold T s , indicating that the main sequence samples and the secondary sequence samples may be heterogeneous samples, and the similarity mask is used to remove the tuple pair.

[0042] Furthermore, the overall loss L of the semi-supervised deep learning semi Including supervised discriminative loss L x, the unsupervised loss L u , the feature contrast loss L feature And the classification vector contrast loss L p , expressed as:

[0043] L semi =L x +L u +λ f L feature +λ p L p

[0044] where λ f and λ p Control L feature and L p The coefficients are set to 0.001 and 1 respectively, which are used to weigh the influence of contrastive learning and discriminative learning in the model training process;

[0045] Furthermore, in the construction of the sample pair, the fine-tuning module calculates the feature vector z of a specific sample. i , the samples with the same true labels constitute the positive sample pairs {z i ,z}(y i =y), and the rest constitute negative sample pairs, and the cosine distance is used to calculate the similarity between sample pairs to obtain the supervised contrast loss L SCL , expressed as:

[0046]

[0047] Among them, P(i) represents all positive samples with the same pseudo-label corresponding to sample i in the current input batch, and A(i) represents all negative samples with different pseudo-labels in the current input batch;

[0048] The loss function of the fine-tuning module is expressed as:

[0049] L=L x +λ s L SCL

[0050] Among them, λ s A hyperparameter for the trade-off between supervised discriminative loss and supervised contrastive loss.

[0051] The beneficial effects of the present invention are as follows:

[0052] 1. A semi-supervised deep learning framework is proposed. First, the diversity and consistency between data are mined through unsupervised contrastive learning methods to construct high-quality data representation. Secondly, supervised discriminant learning is used to optimize model parameters to minimize the difference between the predicted results and the actual labels. Finally, supervised contrastive learning is used to fine-tune and correct some errors in semi-supervised learning to improve network performance.

[0053] 2. A data enhancement technology for lung nodules is proposed, which effectively avoids image distortion and ensures that the quality and diagnostic value of the original medical images are maintained while increasing the diversity of training data, thereby helping to improve the accuracy and robustness of the model.

[0054] 3. Considering the interaction between samples, a method is proposed to extract the discriminative features of benign and malignant lung nodules from a global perspective. By constructing feature sample pairs and category sample pairs, the potential associations between lung nodules are effectively learned, thereby improving the accuracy and reliability of model classification.

[0055] 4. A supervised contrastive learning fine-tuning method is proposed. Based on semi-supervised learning, the real labels of the samples are used to enhance the similarities between samples of the same type and highlight the differences between heterogeneous samples, so as to correct some errors in semi-supervised learning and improve the accuracy of the model in the task of classifying benign and malignant lung nodules. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 The overall framework of the present invention

[0057] Figure 2 This is the module structure diagram of the feature comparison learning submodule based on high confidence pseudo labels in the present invention

[0058] Figure 3 This is the module structure diagram of the classification vector contrast learning submodule based on enhanced anchoring in the present invention

[0059] Figure 4 Comparison curve of the accuracy of different strong enhancement methods

[0060] Figure 5 t-SNE visualization comparison chart for different model features DETAILED DESCRIPTION

[0061] The present invention proposes a method for classifying benign and malignant pulmonary nodules based on semi-supervised deep learning, characterized in that the method includes a data enhancement module, a discriminative feature extraction module and a fine-tuning module; the data enhancement module is used to generate strong and weak enhanced samples; the discriminative feature extraction module is the core part of semi-supervised deep learning, and is used to extract corresponding strong and weak enhanced sample feature vectors and classification vectors, and perform comparative learning and discriminative learning on the two respectively, so as to shorten the distance between semantically related samples and exclude semantically irrelevant samples; the fine-tuning module is used to optimize the feature extractor, and enhance the accuracy of model classification through comparative learning of original images with labeled data and corresponding weakly enhanced samples.

[0062] The data enhancement module includes a weak enhancer module and a strong enhancer module. weak The random horizontal flipping and random cropping methods are used on the lung nodule samples to obtain the corresponding weakly enhanced samples. Strong enhancer module T strong On the basis of the weak enhancement submodule, random rotation, contrast enhancement and CutOut methods are added to obtain strongly enhanced samples.

[0063] In a labeled dataset X, a data sample x i Only through the weak enhancer module T weak Get the weakly enhanced sample of the sample, that is, In the unlabeled dataset U, a data sample u i , respectively through the weak enhancer module T weak and strong enhancer module T strong Get the weakly enhanced sample and the strongly enhanced sample of the sample, that is,

[0064] The discriminative feature extraction module consists of a feature contrast learning submodule based on high-confidence pseudo-labels, a classification vector contrast learning submodule based on enhanced anchoring, and a discriminative learning submodule. EfficientNet is selected as the backbone network, and the unlabeled weak enhancement samples and strong enhancement samples and labeled weak enhancement samples are input into the backbone network. Then the feature vector and classification vector of the unlabeled samples are taken out and input into the feature contrast learning submodule based on high-confidence pseudo-labels and the classification vector contrast learning submodule based on enhanced anchoring respectively, and the classification vector of the labeled samples and the classification vector of the unlabeled strong enhancement samples are sent to the discriminative learning submodule.

[0065] The module structure diagram of the feature contrast learning submodule based on high-confidence pseudo-labels is as follows Figure 2 As shown in the figure, unlabeled weak enhancement samples and strong enhancement samples are respectively pseudo-labeled through the classification layer of the backbone network, and the pseudo-labels are assigned feature vectors of strong and weak enhancement samples. The pseudo-label calculation formula is as follows:

[0066]

[0067] Among them, Classifier represents the classification layer of the backbone network, that is, the last fully connected layer, θ′ represents the network parameters of the classification layer, Model represents the backbone network without the classification layer, and θ represents its network parameters.

[0068] Unlabeled weakly enhanced samples and strongly enhanced samples construct sample pairs based on pseudo labels. For a specific sample’s feature vector When a batch has 2N unlabeled weakly enhanced samples and strongly enhanced samples, 2N-1 ordered sample pairs are matched, among which the samples with the same pseudo-label constitute the positive sample pairs. The rest constitute negative sample pairs, which exploit the potential relationship between unlabeled nodules at the feature level. The loss function calculation formula for feature contrast learning is as follows:

[0069]

[0070] Among them, U contains the 2N unlabeled sample augmentation data in an input batch, · represents the inner product, P(i) represents all positive samples with the same pseudo-label corresponding to sample i in the current input batch, A(i) represents all negative samples with different pseudo-labels in the current input batch, and M T Represents the confidence mask, and its specific mathematical expression is as follows:

[0071]

[0072] Among them, T c represents the confidence threshold, which is set to 0.95. Indicates the threshold value is T c The strict threshold function of is as follows:

[0073]

[0074] Among them, p i Represents the predicted value of the highest class of the sample pseudo-label. This strict threshold function is used to filter out samples with high pseudo-label confidence to avoid incorrect positive and negative sample pairs that hinder contrastive learning.

[0075] The structure diagram of the classification vector contrast learning submodule based on enhanced anchoring is as follows Figure 3 As shown in Figure 1, unlabeled weakly enhanced samples and strongly enhanced samples obtain corresponding classification vectors through the backbone network, which are used to construct ordered tuple pairs. The specific construction of ordered tuple pairs can be expressed as: the feature vector of unlabeled data enhancement samples in an input batch The corresponding classification vector is obtained through the classification layer of the backbone network Weakly enhanced classification vector for unlabeled samples Combine two by two to construct ordered tuple pairs, such as First, use Construct an ordered set of sample pairs: Define the element at the first position in the ordered sample pair as the main order (such as In ), the elements in the next position are in order (such as In ). Then, the class distribution probability vector of the strong enhanced sample corresponding to the weakly enhanced sample in the order is added to the sample pair to form the final ordered tuple pair: By bringing semantically related classification vectors closer and semantically unrelated classification vectors apart in ordered tuple pairs, we learn the knowledge between the classification vectors of unlabeled nodes at the classification level. So far, the loss function of classification vector contrast learning based on enhanced anchoring can be expressed as:

[0076]

[0077] Where N is the number of samples in the batch, f sim (·) represents the similarity threshold measurement function, and its specific mathematical expression is as follows:

[0078]

[0079] like The highest class prediction value in exceeds the confidence threshold T c , and in the ordered tuple pair if p i If it is located in the main sequence position, it is used as an "anchor point", and the similarity measurement function calculates the similarity between the main sequence and the secondary order prediction vectors in all tuple pairs with the main sequence elements as "anchor points", thereby gathering the secondary order classification vectors related to the main sequence category to the main sequence classification vector.

[0080] f dist (·) represents the distance function, which is used to represent the distance between the classification vector of the strongly enhanced sample corresponding to the secondary sample and the classification vector of the main sequence "anchor point", so as to align the classification vector of the strongly enhanced sample with the "anchor point". Its specific mathematical expression is as follows:

[0081] T s Represents the similarity threshold, set to 0.9, Indicates the threshold value is T s A strict threshold function is used to screen reliable tuple pairs, that is, if the similarity between the primary order and the secondary order prediction vector in the tuple pair is lower than the preset similarity threshold T s , indicating that the main sequence samples and the secondary sequence samples may be heterogeneous samples, and the similarity mask is used to remove the tuple pair.

[0082] The discriminant learning submodule uses the cross entropy loss function to compare the classification vector of the labeled weakly enhanced sample with the true label to obtain the supervised discriminant loss. At the same time, the classification vector of the unlabeled strongly enhanced sample is compared with its corresponding pseudo label to obtain the unsupervised loss, thereby realizing the classification of benign and malignant pulmonary nodules. x and the unsupervised loss L u It can be expressed by mathematical formulas as follows:

[0083]

[0084] Where B represents the number of labeled nodules in a batch, H(·) represents the cross entropy loss function, and μ represents the ratio of the number of unlabeled nodules to labeled nodules in a batch, which is set to 7. represents the pseudo labels of unlabeled samples.

[0085] So far, the overall loss L of semi-supervised deep learning is semi Including supervised discriminative loss L x , unsupervised loss L u , feature contrast loss L feature And the classification vector contrast loss L p , specifically expressed as:

[0086] L semi =L x +L u +λ f L feature +λ p L p

[0087] where λ f and λ p Control L feature and L p The coefficients are set to 0.001 and 1 respectively to weigh the influence of contrastive learning and discriminative learning in the model training process.

[0088] The fine-tuning module is performed on the basis of the model obtained after the semi-supervised learning training is completed. Only labeled nodules are used to construct sample pairs of lung nodule original images and weakly enhanced images. Based on the true labels of the samples, the similarities between samples of the same type and the differences between samples of different types are learned to correct some errors in semi-supervised learning and improve network performance.

[0089] The fine-tuning module uses a supervised contrastive learning method. At the feature level of labeled nodules, the original image of the labeled nodules is retained, and the weakly enhanced image of the labeled nodules is obtained through the data enhancement module. The real labels of the labeled nodules are used and assigned to the weakly enhanced samples of the labeled samples. Then, in the construction of the sample pair, for a specific sample, the feature vector zi , the samples with the same true labels constitute the positive sample pairs {z i ,z}(y i =y), and the rest constitute negative sample pairs, and the cosine distance is used to calculate the similarity between sample pairs, thereby obtaining the supervised contrast loss L SCL , which can be specifically expressed as:

[0090]

[0091] Among them, P(i) represents all positive samples with the same pseudo-label corresponding to sample i in the current input batch, and A(i) represents all negative samples with different pseudo-labels in the current input batch.

[0092] Furthermore, the specific mathematical expression of the loss function of the fine-tuning module is as follows:

[0093] L=L x +λ s L SCL

[0094] Among them, L x Denoted as supervised discriminative loss, λ s To balance the supervised discriminative loss and the supervised contrastive loss, the hyperparameter is set to 0.001.

[0095] In order to test the effectiveness of this system, the model in this system is applied to the LIDC-IDRI dataset [1]. The LIDC-IDRI dataset is the largest publicly available lung CT image dataset. The dataset contains 1018 cases, each of which includes images and labeling files from clinical chest CT scans. For each image in an instance, four experienced chest radiologists perform a two-stage diagnostic annotation. In the first stage, each physician independently diagnoses and annotates the location of the lesion. In the second stage, each physician independently reviews the annotations of the other three physicians and gives his or her final diagnosis. This two-stage annotation can avoid inconsistency in annotation results caused by subjective differences between doctors.

[0096] In order to improve the stability and reliability of the test results, a ten-fold cross-validation experiment was performed. The data set was almost evenly divided into ten non-overlapping subsets, and the nodules in different subsets were guaranteed to come from different patients to avoid deviations in measurement performance. Each of the ten subsets was taken as a test set, and the remaining nine subsets of the data were taken as training sets.

[0097] (1) The classification results of the ten-fold cross validation can be seen in Table 1. As can be seen from Table 1, the present invention can achieve a ten-fold average accuracy of 93.73%, a sensitivity of 88.20%, a specificity of 97.33%, an accuracy of 95.62%, an AUC of 94.35%, and an F1 score of 91.63%, indicating that the present invention has achieved good performance in the classification of lung nodules and has a certain robustness.

[0098] Table 1 Ten-fold cross validation evaluation results

[0099] Fold Accuracy Sensitivity Specificity Accuracy AUC F1 score 1 92.50 87.50 95.83 93.33 92.84 90.32 2 93.75 87.50 97.92 96.55 95.05 91.80 3 92.50 84.38 97.92 96.43 94.73 90.00 4 92.50 87.10 95.92 93.10 93.48 90.00 5 95.00 93.55 95.92 93.55 95.13 93.55 6 93.75 87.10 97.96 96.43 95.72 91.53 7 96.25 96.77 95.92 93.75 95.13 95.24 8 96.25 93.55 97.96 96.67 97.43 95.08 9 91.14 77.42 100.00 100.00 93.62 87.27 10 93.67 87.10 97.92 96.43 90.32 91.53 average 93.73±1.69 88.20±5.44 97.33±1.38 95.62±2.17 94.35±1.92 91.63±2.47

[0100] (2) The performance of the present invention is compared with the most advanced semi-supervised model on the LIDC-IDRI dataset, and the comparison results are shown in Table 2. The results show that the present invention achieves the highest accuracy, specificity, precision and F1 score, which are improved by 0.52%, 0.74%, 4.8% and 2.08% respectively compared with the highest indicators of other methods listed, clearly demonstrating the effectiveness of the present invention.

[0101] Table 2 Comparison results with semi-supervised pulmonary nodule classification methods

[0102]

[0103] (3) The present invention is compared with the most advanced supervised model on the LIDC-IDRI dataset, and the comparison results are shown in Table 3. As can be seen from Table 3, the present invention achieves the highest specificity and accuracy. Through this comparative experiment, it can be seen that there is still a gap between the present invention and the most advanced supervised pulmonary nodule classification method, but through various indicators, it can be seen that the performance of the present invention is relatively close to the existing supervised pulmonary nodule classification method, and the present invention has low dependence on high-quality labeled data, so it is more scalable and practical than supervised learning.

[0104] Table 3 Comparison results with supervised pulmonary nodule classification methods

[0105]

[0106] (4) In order to verify the effectiveness of each module in the present invention, the present invention has conducted extensive ablation experiments on the effective design part of the model, including ablation experiments on three modules: feature contrast learning submodule M1 based on high-confidence pseudo-labels, classification vector contrast learning submodule M2 based on enhanced anchoring, and fine-tuning module M3. The results are shown in Table 4, and "√" indicates that the corresponding module is used in the current method. First, the feature contrast learning submodule based on high-confidence pseudo-labels is added to the basic framework, and the accuracy, sensitivity, and specificity are improved by 0.76%, 1.02%, and 0.61%, respectively; then the classification vector contrast learning submodule based on enhanced anchoring is added, and the accuracy, sensitivity, and specificity are improved by 0.65%, 0.9%, and 0.42%, respectively. This shows that feature contrast learning and classification vector contrast learning effectively utilize the information between unlabeled samples; in addition, the use of the fine-tuning module after semi-supervised training improves the accuracy, sensitivity, and specificity by 0.73%, 1.6%, and 0.21%, respectively. This fully proves that supervised contrast fine-tuning can correct some errors of unsupervised contrast learning and improve model performance by using real label information to construct positive and negative sample pairs.

[0107] Table 4 Ablation experiment results of effective modules in the system

[0108] M1 M2 M3 Accuracy Sensitivity Specificity Accuracy AUC F1 score 91.59 84.68 96.09 93.53 93.78 88.68 √ 92.35 85.70 96.70 94.44 93.91 90.06 √ √ 93.00 86.60 97.12 95.23 94.33 90.96 √ √ √ 93.73 88.20 97.33 95.62 94.35 91.63

[0109] At the same time, in order to verify the impact of different data enhancement strategies on the performance of the benign and malignant lung nodule classification task, the strong enhancement methods in the two data enhancement strategies were compared, one was the RandAugment method, and the other was the strong enhancer module in the present invention. In this comparative experiment, both strong enhancement methods acted on the basis of the weak enhancer module, and in order to keep the number of enhancements consistent, the RandAugment used randomly selected 3 enhancement methods from the predefined enhancement method pool to act on the image. All additional modules (discriminative feature extraction module and fine-tuning module) were removed, and the feature extraction network and other hyperparameters were kept unchanged, and the comparative experimental results were obtained as shown in Table 5. It can be seen from the data in the table that the strong enhancer module of the present invention achieved higher accuracy and sensitivity, which were increased by 0.25% and 1.6% respectively.

[0110] Table 5 Comparison results of image enhancement methods

[0111]

[0112]

[0113] In order to further analyze the impact of different strong enhancement methods on the model accuracy, the experiment visualized the accuracy curve of one fold of the model in the table. The accuracy comparison curve is as follows: Figure 4 As shown. Figure 4As can be seen in the model using the strong enhancer module of the present invention ( Figure 4 The model with RandAugment as the strong enhancer module (yellow line in the middle) converges faster and can reach the highest accuracy on this fold after about 10,000 iterations. Figure 4 The middle blue line) cannot reach the highest accuracy during the entire training process. At the same time, the model using RandAugment as the strong enhancer module has a large fluctuation in model training because some enhancement methods in its enhancement pool will seriously distort the nodules. Figure 4 There are large accuracy fluctuations around the 5000th and 15000th iterations. The model using the strong enhancer module of the present invention has a smaller accuracy curve fluctuation and is more stable overall. From the comparison results, it can be seen that the strong enhancer module of the data enhancement module of the present invention is more suitable for the task of lung nodule classification, and can better retain the effective information of the nodules while enhancing, avoiding excessive distortion of the nodules and affecting the training effect.

[0114] In addition, the experiment analyzes the impact of the contrastive learning methods based on the feature level and the classification vector level on the performance of the pulmonary nodule classification task. First, in order to verify the impact of different feature contrastive learning methods on the performance of the pulmonary nodule classification task, the experiment compares two feature contrastive learning methods, one is a single positive example contrastive learning method similar to SimCLR

[11] , and the other is a feature contrastive learning submodule based on high-confidence pseudo labels of the present invention. In this comparative experiment, the classification vector contrastive learning submodule and the fine-tuning module based on enhanced anchoring are eliminated, and only the contrastive learning method in the feature contrastive learning and the strong enhancement submodule of the data enhancement module are changed. Four models are designed for comparative experiments. The four models are: Model I uses RandAugment as a strong enhancement submodule, and the feature contrastive learning method uses a single positive example contrastive learning method. Model II uses RandAugment as a strong enhancement submodule, and the feature contrastive learning method uses the feature contrastive learning submodule based on high-confidence pseudo labels of the present invention. Model III uses the strong enhancement submodule of the present invention, and the feature contrastive learning method uses a single positive example contrastive learning method. The comparative experimental results are shown in Table 6. It can be seen from the comparative experimental model III and the present invention that the feature contrast learning submodule based on high-confidence pseudo-labels of the present invention has achieved the highest values ​​in all indicators compared with the single positive example contrast learning method similar to SimCLR, among which the accuracy rate is improved by 0.62%, the specificity is improved by 1.01%, and the sensitivity is improved by 0.04%. In the binary classification task, the sample pair construction method of a single positive example may expand the problem that potential similar samples are classified as negative samples. The feature contrast learning submodule based on high-confidence pseudo-labels constructs positive and negative sample pairs through high-confidence pseudo-labels, expands the positive sample pairs, and achieves targeted contrast learning as a whole, which alleviates this problem to a certain extent.

[0115] Table 6 Results of different comparative learning strategies

[0116]

[0117]

[0118] From the results of the comparative experimental models I and II, it can be analyzed that when the feature contrast learning submodule based on high-confidence pseudo-labels of the present invention uses RandAugment as a strong enhancement submodule, compared with the contrast learning method of a single positive example, the accuracy rate decreases by 0.12%, the sensitivity decreases by 0.37%, and the precision rate decreases by 0.24%.

[0119] Finally, in order to observe the influence of the feature contrast learning submodule based on high-confidence pseudo-labels and the classification vector contrast learning submodule based on enhanced anchoring on the model characteristics, as well as to compare the influence of the feature contrast learning method of a single positive example and the feature contrast learning submodule based on high-confidence pseudo-labels on the model characteristics, the experiment uses the t-SNE

[12] algorithm to visualize the dimensionality reduction of the final classification features of the following four models. The four models are: (a) based on the consistency regular basic framework alone. (b) adding a single positive example contrast learning method to the basic framework to act on the feature level. (c) adding the feature contrast learning submodule based on high-confidence pseudo-labels of the present invention to the basic framework. (d) adding the feature contrast learning submodule based on high-confidence pseudo-labels and the classification vector contrast learning submodule based on enhanced anchoring of the present invention to the basic framework. The t-SNE visualization results are shown in the figure. Figure 5 As shown. Figure 5 As can be seen from (a) in the figure, the classification features of the model without feature contrast learning have obvious outliers, and the intra-class distance optimization is insufficient. When the model is added with the feature contrast learning submodule based on high-confidence pseudo-labels of the present invention, that is, Figure 5 (c), outliers disappear, and the distance within the class is shortened. Compared with the model (b) using a single positive example contrast learning strategy, the feature distribution of the model based on a single positive example contrast learning is loose after dimensionality reduction and is not compact enough. Finally, the model (d) that simultaneously uses the feature contrast learning submodule based on high-confidence pseudo labels and the classification vector contrast learning submodule based on enhanced anchoring of the present invention, while maintaining the compactness within the class, further widens the distance between classes, and further optimizes the decision boundary.

Claims

1. A smart early screening system for lung cancer, comprising a data enhancement module, a discriminative feature extraction module and a fine-tuning module; the data enhancement module is used to generate weakly enhanced samples and strongly enhanced samples; the discriminative feature extraction module is used to extract corresponding strong and weak enhanced sample feature vectors and classification vectors, and perform comparative learning and discriminative learning on the two respectively, narrowing the distance between semantically related samples and excluding semantically irrelevant samples; the fine-tuning module is used to optimize the feature extractor, and enhance the accuracy of model classification through comparative learning of original images with labeled data and corresponding weakly enhanced samples; in, The discriminative feature extraction module includes a feature contrast learning submodule based on high-confidence pseudo-labels, a classification vector contrast learning submodule based on enhanced anchoring, and a discriminative learning submodule. Unlabeled weakly enhanced samples, strongly enhanced samples, and labeled weakly enhanced samples are input into the backbone network, and then the feature vectors and classification vectors of the unlabeled samples are taken out and input into the feature contrast learning submodule based on high-confidence pseudo-labels and the classification vector contrast learning submodule based on enhanced anchoring, and the classification vectors of the labeled samples and the classification vectors of the unlabeled strongly enhanced samples are sent to the discriminative learning submodule. In the feature contrast learning submodule based on high-confidence pseudo-labels, unlabeled weakly enhanced samples and strongly enhanced samples are respectively pseudo-labeled through the classification layer of the backbone network; unlabeled weakly enhanced samples and strongly enhanced samples are constructed into sample pairs based on the pseudo-labels; Based on the enhanced anchoring classification vector comparison learning submodule, unlabeled weakly enhanced samples and strongly enhanced samples obtain corresponding classification vectors through the backbone network, which are used to construct ordered tuple pairs; by pulling semantically related classification vectors in the ordered tuple pairs closer and pulling semantically irrelevant classification vectors farther apart, the knowledge between the classification vectors of unlabeled nodules at the classification level is learned; The discriminative learning submodule uses the cross entropy loss function to compare the classification vector of the labeled weakly enhanced sample with the true label to obtain the supervised loss, and compares the classification vector of the unlabeled strongly enhanced sample with its corresponding pseudo label to obtain the unsupervised loss, thereby realizing the classification of benign and malignant pulmonary nodules. The fine-tuning module is based on the model obtained after the semi-supervised learning training is completed. Only labeled nodules are used to construct the original image of the lung nodule and the weakly enhanced image sample pair. Based on the real labels of the samples, the similarities between samples of the same type and the differences between samples of different types are learned to correct some errors in semi-supervised learning and improve network performance. The fine-tuning module adopts a supervised contrastive learning method. At the feature level of labeled nodules, the original image of the labeled nodules is retained, and the weakly enhanced image of the labeled nodules is obtained through the data enhancement module. The true labels of the labeled nodules are used and assigned to the weakly enhanced samples of the labeled samples.

2. The intelligent early screening system for lung cancer according to claim 1, characterized in that: The data enhancement module includes a weak enhancer module and a strong enhancer module; the weak enhancer module T weak The random horizontal flipping and random cropping methods are used on the lung nodule samples to obtain the corresponding weakly enhanced samples; the strong enhancer module T strong On the basis of the weak enhancement submodule, random rotation, contrast enhancement and CutOut methods are added to obtain strong enhancement samples; in the labeled dataset X, a data sample x i Only through the weak enhancer module T weak Get the weakly enhanced sample of the data sample, that is, In the unlabeled dataset U, a data sample u i , respectively through the weak enhancer module T weak and strong enhancer module T strong Get the weakly enhanced sample and the strongly enhanced sample of the data sample, that is, 3. The intelligent early screening system for lung cancer according to claim 1, characterized in that: Select EfficientNet as the backbone network.

4. The intelligent early screening system for lung cancer according to claim 1, characterized in that: In the feature contrast learning submodule based on high-confidence pseudo-labels, for a feature vector of a specific sample When a batch has 2N unlabeled weakly enhanced samples and strongly enhanced samples, 2N-1 ordered sample pairs are matched, among which the samples with the same pseudo-label constitute positive sample pairs, and the rest constitute negative sample pairs. The potential relationship between unlabeled nodules is used at the feature level; the loss function calculation formula for feature contrast learning is as follows: Among them, U contains the 2N unlabeled sample augmentation data in an input batch, · represents the inner product, P(i) represents all positive samples with the same pseudo-label corresponding to sample i in the current input batch, A(i) represents all negative samples with different pseudo-labels in the current input batch, and z i , z p , z a represents the unlabeled data augmentation sample feature vector, M T Represents the confidence mask, and its specific mathematical expression is as follows: Among them, T c represents the confidence threshold, Indicates the threshold value is T c A strict threshold function is used to filter out samples with high pseudo-label confidence to avoid erroneous positive and negative sample pairs that hinder contrastive learning.

5. The intelligent early screening system for lung cancer according to claim 4, characterized in that: T c Set to 0.

95.

6. The intelligent early screening system for lung cancer according to claim 4, characterized in that: In the classification vector contrast learning submodule based on enhanced anchoring, the method of constructing ordered tuples is as follows: the feature vector of the unlabeled data augmented sample in an input batch The corresponding classification vector is obtained through the classification layer of the backbone network Weakly enhanced classification vector for unlabeled samples Combine two by two to construct ordered tuple pairs.

7. The intelligent early screening system for lung cancer according to claim 6, characterized in that: The loss function of the classification vector contrastive learning based on enhanced anchoring is expressed as: Where N is the number of samples in the batch, f sim (·) represents the similarity threshold measurement function like The highest class prediction value in exceeds the confidence threshold T c , and in the ordered tuple pair if p i If it is located in the main sequence position, it is used as an "anchor point", and the similarity measurement function calculates the similarity between the main sequence and the secondary order prediction vectors in all tuple pairs whose main sequence elements are "anchor points", so as to gather the secondary order classification vectors related to the main sequence category to the main sequence classification vector; f dist (·) represents the distance function, which is used to represent the distance between the classification vector of the strongly enhanced sample corresponding to the secondary sample and the classification vector of the main sequence "anchor point", so as to align the classification vector of the strongly enhanced sample with the "anchor point"; T s represents the similarity threshold, Indicates the threshold value is T s A strict threshold function is used to screen reliable tuple pairs, that is, if the similarity between the primary order and the secondary order prediction vector in the tuple pair is lower than the preset similarity threshold T s , indicating that the main sequence samples and the secondary sequence samples may be heterogeneous samples, and the similarity mask is used to remove the tuple pair.

8. The intelligent early screening system for lung cancer according to claim 7, characterized in that: The overall loss L of the semi-supervised deep learning semi Including supervised discriminative loss L x , the unsupervised loss L u , the feature contrast loss Lf eature And the classification vector contrast loss L p , expressed as: L semi =L x +L u +λ f L feature +λ p L p where λ f and λ p Control L feature and L p The coefficients are set to 0.001 and 1 respectively to weigh the influence of contrastive learning and discriminative learning in the model training process.

9. The intelligent early screening system for lung cancer according to claim 8, characterized in that: In the construction of sample pairs, the fine-tuning module takes the feature vector z of a specific sample as i , the samples with the same true labels constitute the positive sample pairs {z i ,z}(y i =y), and the rest constitute negative sample pairs, and the cosine distance is used to calculate the similarity between sample pairs to obtain the supervised contrast loss L SCL , expressed as: Among them, P(i) represents all positive samples with the same pseudo-label corresponding to sample i in the current input batch, and A(i) represents all negative samples with different pseudo-labels in the current input batch; The loss function of the fine-tuning module is expressed as: L=L x +λ s L SCL Among them, λ s A hyperparameter for the trade-off between supervised discriminative loss and supervised contrastive loss.

10. The intelligent early screening system for lung cancer according to claim 9, characterized in that: λ f and λ p are set to 0.001 and 1 respectively; s Set to 0.001.

Images