Intelligent early screening system for lung cancer

Abstract

The application provides a lung cancer intelligent early screening system, comprising a data enhancement module, a discriminative feature extraction module and a fine-tuning module; the data enhancement module is used for generating weak enhancement samples and strong enhancement samples; the discriminative feature extraction module is used for extracting corresponding strong and weak enhancement sample feature vectors and classification vectors, and performing contrast learning and discriminative learning on the two respectively, narrowing the distance between samples related in semantics and excluding samples irrelevant in semantics; the fine-tuning module is used for optimizing the feature extractor, and the contrast learning of the original image of the labeled data and the corresponding weak enhancement sample is used to strengthen the accuracy of model classification.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent medical care, relates to the technology of semi-supervised deep network models, and specifically relates to an early screening system for lung cancer. Background Art

[0002] Lung cancer is the leading cause of cancer death worldwide, and early detection of lung cancer is crucial for reducing lung cancer-related mortality. CT imaging is currently an internationally recognized method for early lung cancer screening. Pulmonary nodules are a common finding on lung CT scans. These nodules can be benign, such as those caused by inflammation, infection, tuberculosis, or scarring from healing wounds, or they can be malignant, an early manifestation of lung cancer. Therefore, distinguishing between benign and malignant lung nodules is a key step in the early detection and intervention of lung cancer.

[0003] Currently, the diagnosis of clinical lung cancer primarily relies on radiologists' meticulous review and analysis of CT images. This process is not only time-consuming, but even experienced radiologists can become fatigued and their judgment impaired during long, intense work environments, thus affecting diagnostic accuracy. Furthermore, the heterogeneity of lung nodules poses significant challenges to the diagnosis of lung cancer, and even experienced radiologists sometimes struggle to accurately distinguish between benign and malignant nodules. Intelligent diagnostic methods based on deep learning can quickly and accurately perform quantitative analysis of CT images and have shown promising development prospects in the field of benign and malignant lung nodule identification.

[0004] Most existing deep learning methods for lung nodule identification are supervised learning methods. While they have achieved some effectiveness, their limitations lie in the requirement for large amounts of labeled data and poor generalization to new samples. In the medical field, imaging data comes from expensive medical equipment, and labels are the result of time-consuming and labor-intensive analysis by multiple medical experts, making data acquisition extremely difficult. Furthermore, data labels may contain sensitive information that requires privacy protection.

[0005] In response to the above problems, semi-supervised deep learning methods have emerged. However, research on methods for identifying benign and malignant lung nodules based on semi-supervised deep learning is still in its early stages and has some shortcomings. On the one hand, these semi-supervised learning lung nodule classification methods use data augmentation methods to expand the data of the same sample, attempting to obtain more detailed information about the lung nodules. However, they do not consider the relevant information between different samples of the same category, ignoring the diversity and consistency between similar samples, resulting in the misclassification of benign and malignant nodules and reduced model accuracy. On the other hand, these methods use affine transformation and histogram equalization as data augmentation methods. These data augmentation methods are more suitable for natural images and have not been improved and designed based on the characteristics of lung nodules. As a result, the potential information between samples cannot be properly utilized, resulting in weak generalization ability of the model.

[0006] In summary, how to ensure model accuracy while improving the generalization ability of the model under the premise of limited labeled data and a large amount of unlabeled data is an urgent problem to be solved in the current intelligent early screening system for lung cancer.

[0007] Related Literature

[0008] [1] Armato III SG, McLennan G, Bidaut L, et al. The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans [J]. Medical physics, 2011, 38 (2): 915-931.

[0009] [2]Xie Y, Zhang J, Xia Y. Semi-supervised adversarial model for benign–malignant lung nodule classification on chest CT[J]. Medical image analysis, 2019,57:237-248.

[0010] [3] Apostolopoulos ID, Papathanasiou ND, Panayiotakis GS. Classification of lung nodule malignancy in computed tomography imaging utilizing generative adversarial networks and semi-supervised transfer learning [J]. Biocybernetics and Biomedical Engineering, 2021, 41(4): 1243-1257.

[0011] [4]Fu Y,Xue P,Xiao T,et al.Semi-supervised adversarial learning forimproving the diagnosis ofpulmonary nodules[J].IEEE Journal ofBiomedicalandHealth Informatics,2022,27(1):109-120.

[0012] [5]Zhu W,Liu C,Fan W,et al.Deeplung:Deep 3d dual path nets forautomated pulmonary noduledetection and classification[C] / / 2018IEEE winterconference on applications of computer vision(WACV).IEEE,2018:673-681.

[0013] [6]Shen W,Zhou M,Yang F,et al.Multi-scale convolutional neuralnetworks for lung nodule classification[C] / / Information Processing in MedicalImaging:24th International Conference,IPMI 2015,Sabhal Mor Ostaig,Isle ofSkye,UK,June 28-July 3,2015,Proceedings 24.Springer International Publishing,2015:588-599.

[0014] [7]Xie Y,XiaY,Zhang J,et al.Knowledge-based collaborative deeplearning for benign-malignant lung nodule classification on chest CT[J].IEEEtransactions on medical imaging,2018,38(4):991-1004.

[0015] [8]Jiang H, Shen F, Gao F, et al. Learning efficient, explainable and discriminative representations for pulmonary nodules classification[J]. Pattern Recognition, 2021,113:107825.

[0016] [9]Xu

[0017]

[10] Al-Shabi M,Shak K,Tan M.ProCAN: Progressive growing channel attentive non-local network for lung nodule classification[J].PatternRecognition,2022,122:108309.

[0018]

[11] Chen T, Kornblith S, Norouzi M, et al. A simple framework for contrastive learning of visual representations[C] / / International conferenceon machine learning.PMLR, 2020:1597-1607.

[0019]

[12] Van der Maaten L,Hinton G.Visualizing data using t-SNE[J].Journalofmachine learning research,2008,9(11). Summary of the Invention

[0020] To address the above issues, the present invention proposes an intelligent early screening system for lung cancer, which can be used to classify benign and malignant lung nodules based on semi-supervised deep learning. The technical solution of the present invention is as follows:

[0021] A smart early screening system for lung cancer includes a data enhancement module, a discriminative feature extraction module, and a fine-tuning module; the data enhancement module is used to generate weakly enhanced samples and strongly enhanced samples; the discriminative feature extraction module is used to extract the corresponding feature vectors and classification vectors of the strong and weak enhanced samples, and perform comparative learning and discriminative learning on the two respectively, narrowing the distance between semantically related samples and rejecting semantically irrelevant samples; the fine-tuning module is used to optimize the feature extractor, and enhance the accuracy of model classification through comparative learning between the original image with labeled data and the corresponding weakly enhanced samples; wherein,

[0022] The discriminative feature extraction module includes a feature contrast learning submodule based on high-confidence pseudo-labels, a classification vector contrast learning submodule based on enhanced anchoring, and a discriminative learning submodule. Unlabeled weakly enhanced samples, strongly enhanced samples, and labeled weakly enhanced samples are input into the backbone network. The feature vectors and classification vectors of the unlabeled samples are then taken out and input into the feature contrast learning submodule based on high-confidence pseudo-labels and the classification vector contrast learning submodule based on enhanced anchoring, respectively. The classification vectors of the labeled samples and the classification vectors of the unlabeled strongly enhanced samples are then sent to the discriminative learning submodule.

[0023] In the feature contrast learning submodule based on high-confidence pseudo-labels, unlabeled weakly enhanced samples and strongly enhanced samples are respectively pseudo-labeled through the classification layer of the backbone network; unlabeled weakly enhanced samples and strongly enhanced samples are constructed into sample pairs based on the pseudo-labels;

[0024] In the enhanced anchor-based classification vector comparison learning submodule, unlabeled weakly enhanced samples and strongly enhanced samples are fed into the backbone network to obtain corresponding classification vectors, which are used to construct ordered tuple pairs. By bringing semantically related classification vectors closer together and spreading semantically unrelated classification vectors further apart in the ordered tuple pairs, knowledge between the classification vectors of unlabeled nodules at the classification level is learned.

[0025] The discriminative learning submodule uses the cross-entropy loss function to compare the classification vectors of labeled weakly enhanced samples with the true labels to obtain supervised loss. At the same time, it compares the classification vectors of unlabeled strongly enhanced samples with their corresponding pseudo labels to obtain unsupervised loss, thereby achieving benign and malignant lung nodule classification.

[0026] The fine-tuning module is based on the model obtained after the semi-supervised learning training. It only uses labeled nodules to construct sample pairs of lung nodule original images and weakly enhanced images. Based on the true labels of the samples, it learns the similarities between samples of the same type and the differences between samples of different types to correct some errors in semi-supervised learning and improve network performance.

[0027] The fine-tuning module adopts a supervised contrastive learning method. At the feature level of labeled nodules, it retains the original image of the labeled nodules, obtains the weakly enhanced image of the labeled nodules through the data enhancement module, and uses the true label of the labeled nodules and assigns it to the weakly enhanced sample of the labeled sample.

[0028] Furthermore, the data enhancement module includes a weak enhancement submodule and a strong enhancement submodule; the weak enhancement submodule T weak The random horizontal flipping and random cropping methods are used on the lung nodule samples to obtain the corresponding weak enhancement samples; the strong enhancer module T strong On the basis of the weak enhancement submodule, random rotation, contrast enhancement and CutOut methods are added to obtain strong enhancement samples; in the labeled dataset X, a data sample x i Only through the weak enhancer module T weak Get the weakly enhanced sample of the data sample, that is, In an unlabeled dataset U, a data sample u i , respectively through the weak enhancer module T weak and strong enhancer module T strong Get the weak enhancement sample and strong enhancement sample of the data sample, that is,

[0029] Furthermore, EfficientNet is selected as the backbone network.

[0030] Furthermore, in the feature contrast learning submodule based on high-confidence pseudo-labels, for a specific sample’s feature vector When a batch contains 2N unlabeled weakly enhanced samples and strongly enhanced samples, 2N-1 ordered sample pairs are matched. Among them, the samples with the same pseudo-label constitute the positive sample pairs, and the rest constitute the negative sample pairs. The potential relationship between unlabeled nodules is used at the feature level. The loss function calculation formula for feature contrast learning is as follows:

[0031]

[0032] Among them, U contains the 2N unlabeled sample augmentation data in an input batch, · represents the inner product, P(i) represents all positive samples with the same pseudo label corresponding to sample i in the current input batch, A(i) represents all negative samples with different pseudo labels in the current input batch, z i , z p , z a Represents the unlabeled data augmentation sample feature vector, M T Represents the confidence mask, and its specific mathematical expression is as follows:

[0033]

[0034] Among them, T c represents the confidence threshold, Indicates the threshold value is T c A strict threshold function is used to filter out samples with high pseudo-label confidence to avoid incorrect positive and negative sample pairs hindering contrastive learning.

[0035] Furthermore, in the enhanced anchor-based classification vector contrast learning submodule, the method of constructing ordered tuples is as follows: the unlabeled data in an input batch is enhanced with the sample feature vector The corresponding classification vector is obtained through the classification layer of the backbone network Weakly enhanced classification vector for unlabeled samples Combine two by two to construct ordered tuple pairs.

[0036] Furthermore, the loss function of the classification vector contrast learning based on enhanced anchoring is expressed as:

[0037]

[0038] Where N is the number of samples in the batch, f sim (·) represents the similarity threshold measurement function

[0039] like The highest class prediction value in exceeds the confidence threshold T c , and in the ordered tuple pair if p i If it is located in the main sequence position, it will be used as an "anchor point", and the similarity measurement function will calculate the similarity between the main sequence and the sub-order prediction vectors in all tuple pairs with the main sequence element as the "anchor point", so as to gather the sub-order classification vectors related to the main sequence category towards the main sequence classification vector;

[0040] f dist (·) represents the distance function, which is used to express the distance between the classification vector of the strongly enhanced sample corresponding to the order sample and the classification vector of the main order "anchor point", so as to align the classification vector of the strongly enhanced sample with the "anchor point";

[0041] T s represents the similarity threshold, Indicates the threshold value is T s A strict threshold function is used to screen reliable tuple pairs, that is, if the similarity between the main order and the order prediction vector in the tuple pair is lower than the preset similarity threshold T s , indicating that the main sequence samples and the secondary sequence samples may be heterogeneous samples, and the similarity mask is used to remove the tuple pair.

[0042] Furthermore, the overall loss L of the semi-supervised deep learning semi Including supervised discriminative loss L x, the unsupervised loss L u , the feature contrast loss L feature and the classification vector contrast loss L p , expressed as:

[0043] L semi =L x +L u +λ f L feature +λ p L p

[0044] where λ f and λ p Control L feature and L p The coefficients of are set to 0.001 and 1 respectively, which are used to weigh the influence of contrastive learning and discriminative learning in the model training process;

[0045] Furthermore, in the construction of the sample pair, the fine-tuning module calculates the feature vector z of a specific sample. i , the samples with the same true labels constitute the positive sample pairs {z i ,z}(y i =y), and the rest constitute negative sample pairs, and the cosine distance is used to calculate the similarity between sample pairs to obtain the supervised contrast loss L SCL , expressed as:

[0046]

[0047] Among them, P(i) represents all positive samples with the same pseudo-label corresponding to sample i in the current input batch, and A(i) represents all negative samples with different pseudo-labels in the current input batch;

[0048] The loss function of the fine-tuning module is expressed as:

[0049] L=L x +λ s L SCL

[0050] Among them, λ s A hyperparameter that balances supervised discriminative loss and supervised contrastive loss.

[0051] The beneficial effects of the present invention are as follows:

[0052] 1. We propose a semi-supervised deep learning framework. First, we exploit the diversity and consistency of data through unsupervised contrastive learning to construct a high-quality data representation. Second, we use supervised discriminant learning to optimize model parameters to minimize the discrepancy between predicted results and actual labels. Finally, we use supervised contrastive learning to fine-tune the model, correcting errors in semi-supervised learning and improving network performance.

[0053] 2. A data augmentation technology for lung nodules is proposed, which effectively avoids image distortion and ensures that the quality and diagnostic value of the original medical images are maintained while increasing the diversity of training data, thereby helping to improve the accuracy and robustness of the model.

[0054] 3. Considering the interaction between samples, a method is proposed to extract the discriminative features of benign and malignant lung nodules from a global perspective. By constructing feature sample pairs and category sample pairs, the potential associations between lung nodules are effectively learned, thereby improving the accuracy and reliability of model classification.

[0055] 4. A supervised contrastive learning fine-tuning method is proposed. Based on semi-supervised learning, the real labels of the samples are used to enhance the similarities between samples of the same type and highlight the differences between heterogeneous samples, so as to correct some errors in semi-supervised learning and improve the accuracy of the model in the task of classifying benign and malignant lung nodules. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 The overall framework of the present invention

[0057] Figure 2 This is the module structure diagram of the feature contrast learning submodule based on high confidence pseudo labels in the present invention

[0058] Figure 3 This is the module structure diagram of the classification vector contrast learning submodule based on enhanced anchoring in the present invention

[0059] Figure 4 Comparison curve of the accuracy of different strong enhancement methods

[0060] Figure 5 t-SNE visualization comparison chart for different model features DETAILED DESCRIPTION

[0061] The present invention proposes a method for classifying benign and malignant lung nodules based on semi-supervised deep learning, characterized in that the method includes a data enhancement module, a discriminative feature extraction module and a fine-tuning module; the data enhancement module is used to generate strong and weak enhanced samples; the discriminative feature extraction module is the core part of semi-supervised deep learning, and is used to extract corresponding strong and weak enhanced sample feature vectors and classification vectors, and perform comparative learning and discriminative learning on the two respectively, to shorten the distance between semantically related samples and exclude semantically irrelevant samples; the fine-tuning module is used to optimize the feature extractor, and enhance the accuracy of model classification through comparative learning of original images with labeled data and corresponding weakly enhanced samples.

[0062] The data enhancement module includes a weak enhancer module and a strong enhancer module. weak Random horizontal flipping and random cropping methods are used on lung nodule samples to obtain corresponding weak enhancement samples. Strong enhancer module T strong Based on the weak enhancement submodule, random rotation, contrast enhancement and CutOut methods are added to obtain strongly enhanced samples.

[0063] In a labeled dataset X, a data sample x i Only through the weak enhancer module T weak Get the weakly enhanced sample of the sample, that is In an unlabeled dataset U, a data sample u i , respectively through the weak enhancer module T weak and strong enhancer module T strong Get the weak enhancement sample and strong enhancement sample of the sample, that is,

[0064] The discriminative feature extraction module consists of a feature contrast learning submodule based on high-confidence pseudo labels, a classification vector contrast learning submodule based on enhanced anchors, and a discriminative learning submodule. EfficientNet is selected as the backbone network, and the unlabeled weak enhancement samples, strong enhancement samples, and labeled weak enhancement samples are input into the backbone network. Then, the feature vectors and classification vectors of the unlabeled samples are taken out and input into the feature contrast learning submodule based on high-confidence pseudo labels and the classification vector contrast learning submodule based on enhanced anchors respectively. The classification vectors of the labeled samples and the classification vectors of the unlabeled strongly enhanced samples are sent to the discriminative learning submodule.

[0065] The module structure diagram of the feature contrast learning submodule based on high confidence pseudo labels is as follows Figure 2 As shown in Figure 2, unlabeled weakly enhanced samples and strongly enhanced samples are respectively pseudo-labeled through the classification layer of the backbone network, and the pseudo-labels are assigned to the feature vectors of the strong and weak enhanced samples. The pseudo-label calculation formula is as follows:

[0066]

[0067] Where Classifier represents the classification layer of the backbone network, that is, the last fully connected layer, θ′ represents the network parameters of the classification layer, Model represents the backbone network excluding the classification layer, and θ represents its network parameters.

[0068] Unlabeled weakly enhanced samples and strongly enhanced samples construct sample pairs based on pseudo labels. For a specific sample’s feature vector When a batch contains 2N unlabeled weakly enhanced samples and strongly enhanced samples, 2N-1 ordered sample pairs are matched, among which the samples with the same pseudo-label constitute the positive sample pairs. The rest constitute negative sample pairs, which exploit the potential relationship between unlabeled nodules at the feature level. The loss function calculation formula for feature contrast learning is as follows:

[0069]

[0070] Among them, U contains the 2N unlabeled sample augmentation data in an input batch, represents the inner product, P(i) represents all positive samples with the same pseudo label corresponding to sample i in the current input batch, A(i) represents all negative samples with different pseudo labels in the current input batch, and M T Represents the confidence mask, and its specific mathematical expression is as follows:

[0071]

[0072] Among them, T c represents the confidence threshold, which is set to 0.95. Indicates the threshold value is T c The strict threshold function of is expressed as follows:

[0073]

[0074] Among them, p i Represents the predicted value of the highest class of the sample pseudo-label. This strict threshold function is used to filter out samples with high pseudo-label confidence to avoid incorrect positive and negative sample pairs that hinder contrastive learning.

[0075] The structure diagram of the classification vector contrast learning submodule based on enhanced anchoring is as follows Figure 3 As shown, the unlabeled weakly enhanced samples and the strongly enhanced samples are fed with the backbone network to obtain corresponding classification vectors, which are used to construct ordered tuple pairs. The specific construction of ordered tuple pairs can be expressed as: the unlabeled data enhancement sample feature vector in an input batch The corresponding classification vector is obtained through the classification layer of the backbone network Weakly enhanced classification vector for unlabeled samples Combine two by two to construct ordered tuple pairs, such as First, use Construct an ordered set of sample pairs: Define the element at the first position in the ordered sample pair as the main order (such as in ), the elements in the second position are in order (such as in ). Then, the class distribution probability vector of the strong enhancement sample corresponding to the weak enhancement sample in the order is added to the sample pair to form the final ordered tuple pair: By bringing semantically related classification vectors closer together and pulling semantically unrelated classification vectors further apart in ordered tuple pairs, we learn the knowledge between the classification vectors of unlabeled nodes at the classification level. The loss function for contrastive learning of classification vectors based on enhanced anchoring can be expressed as:

[0076]

[0077] Where N is the number of samples in the batch, f sim (·) represents the similarity threshold measurement function, and its specific mathematical expression is as follows:

[0078]

[0079] like The highest class prediction value in exceeds the confidence threshold T c , and in the ordered tuple pair if p i If it is located in the main sequence position, it will be used as an "anchor point", and the similarity measurement function calculates the similarity between the main sequence and the sub-order prediction vectors in all tuple pairs with the main sequence elements as "anchor points", so as to gather the sub-order classification vectors related to the main sequence category to the main sequence classification vector.

[0080] f dist (·) represents the distance function, which is used to express the distance between the classification vector of the strongly enhanced sample corresponding to the order sample and the classification vector of the main order "anchor point", thereby aligning the classification vector of the strongly enhanced sample with the "anchor point". Its specific mathematical expression is as follows:

[0081] T s Represents the similarity threshold, set to 0.9, Indicates the threshold value is T s A strict threshold function is used to screen reliable tuple pairs, that is, if the similarity between the main order and the order prediction vector in the tuple pair is lower than the preset similarity threshold T s , indicating that the main sequence samples and the secondary sequence samples may be heterogeneous samples, and the similarity mask is used to remove the tuple pair.

[0082] The discriminant learning submodule uses the cross entropy loss function to compare the classification vector of the labeled weakly enhanced sample with the true label to obtain the supervised discriminant loss. At the same time, the classification vector of the unlabeled strongly enhanced sample is compared with its corresponding pseudo label to obtain the unsupervised loss, thereby achieving the classification of benign and malignant lung nodules. The specific supervised loss L x and the unsupervised loss L u It can be expressed by mathematical formulas as follows:

[0083]

[0084] Where B represents the number of labeled nodes in a batch, H(·) represents the cross entropy loss function, and μ represents the ratio of the number of unlabeled nodes to the number of labeled nodes in a batch, which is set to 7. represents the pseudo labels of unlabeled samples.

[0085] So far, the overall loss L of semi-supervised deep learning semi Including supervised discriminative loss L x , unsupervised loss L u , feature contrast loss L feature And the classification vector contrast loss L p , specifically expressed as:

[0086] L semi =L x +L u +λ f L feature +λ p L p

[0087] where λ f and λ p Control L feature and L p The coefficients are set to 0.001 and 1 respectively to weigh the influence of contrastive learning and discriminative learning in the model training process.

[0088] The fine-tuning module is performed based on the model obtained after the semi-supervised learning training is completed. It only uses labeled nodules to construct sample pairs of lung nodule original images and weakly enhanced images. Based on the true labels of the samples, it learns the similarities between samples of the same type and the differences between samples of different types to correct some errors in semi-supervised learning and improve network performance.

[0089] The fine-tuning module adopts a supervised contrastive learning method. At the feature level of labeled nodules, the original image of the labeled nodules is retained, and the weakly enhanced image of the labeled nodules is obtained through the data enhancement module. The real labels of the labeled nodules are used and assigned to the weakly enhanced samples of the labeled samples. Then, in the construction of the sample pair, for a specific sample, the feature vector z i , the samples with the same true labels constitute the positive sample pairs {z i ,z}(y i =y), and the rest constitute negative sample pairs, and the cosine distance is used to calculate the similarity between sample pairs, thereby obtaining the supervised contrast loss L SCL , which can be specifically expressed as:

[0090]

[0091] Among them, P(i) represents all positive samples with the same pseudo-label corresponding to sample i in the current input batch, and A(i) represents all negative samples with different pseudo-labels in the current input batch.

[0092] Furthermore, the specific mathematical expression of the loss function of the fine-tuning module is as follows:

[0093] L=L x +λ s L SCL

[0094] Among them, L x Denoted as supervised discriminative loss, λ s To balance the hyperparameters of supervised discriminative loss and supervised contrastive loss, it is set to 0.001.

[0095] In order to test the effectiveness of this system, the model in this system was applied to the LIDC-IDRI dataset [1]. The LIDC-IDRI dataset is the largest publicly available lung CT image dataset. The dataset contains 1018 cases, each of which includes images and labeling files from clinical chest CT scans. For each image in the instance, four experienced chest radiologists performed a two-stage diagnostic annotation process. In the first stage, each physician independently diagnosed and annotated the lesion location. In the second stage, each physician independently reviewed the annotations of the other three physicians and gave their final diagnosis. This two-stage annotation process can avoid inconsistencies in the annotation results caused by subjective differences between doctors.

[0096] To improve the stability and reliability of the test results, a ten-fold cross-validation experiment was performed. The dataset was divided almost evenly into ten disjoint subsets, ensuring that the nodules in different subsets came from different patients to avoid bias in measurement performance. Each of these ten subsets was used as a test set, and the remaining nine subsets were used as training sets.

[0097] (1) The classification results of the ten-fold cross validation can be seen in Table 1. As can be seen from Table 1, the present invention can achieve a ten-fold average accuracy of 93.73%, a sensitivity of 88.20%, a specificity of 97.33%, an accuracy of 95.62%, an AUC of 94.35%, and an F1 score of 91.63%, indicating that the present invention has achieved good performance in lung nodule classification and has certain robustness.

[0098] Table 1 Ten-fold cross validation evaluation results

[0099] Fold Accuracy Sensitivity Specificity Accuracy AUC F1 score 1 92.50 87.50 95.83 93.33 92.84 90.32 2 93.75 87.50 97.92 96.55 95.05 91.80 3 92.50 84.38 97.92 96.43 94.73 90.00 4 92.50 87.10 95.92 93.10 93.48 90.00 5 95.00 93.55 95.92 93.55 95.13 93.55 6 93.75 87.10 97.96 96.43 95.72 91.53 7 96.25 96.77 95.92 93.75 95.13 95.24 8 96.25 93.55 97.96 96.67 97.43 95.08 9 91.14 77.42 100.00 100.00 93.62 87.27 10 93.67 87.10 97.92 96.43 90.32 91.53 average 93.73±1.69 88.20±5.44 97.33±1.38 95.62±2.17 94.35±1.92 91.63±2.47

[0100] (2) The performance of the present invention is compared with the most advanced semi-supervised model on the LIDC-IDRI dataset, and the comparison results are shown in Table 2. The results show that the present invention achieves the highest accuracy, specificity, precision and F1 score, which are improved by 0.52%, 0.74%, 4.8% and 2.08% respectively compared with the highest indicators of other methods listed, clearly demonstrating the effectiveness of the present invention.

[0101] Table 2 Comparison results with semi-supervised lung nodule classification methods

[0102]

[0103] (3) The present invention was compared with the most advanced supervised model on the LIDC-IDRI dataset, and the comparison results are shown in Table 3. As can be seen from Table 3, the present invention achieved the highest specificity and accuracy. Through this comparative experiment, it can be seen that there is still a gap between the present invention and the most advanced supervised lung nodule classification method, but it can be seen from various indicators that the performance of the present invention is relatively close to the existing supervised lung nodule classification method, and the present invention has low dependence on high-quality labeled data, so it is more scalable and practical than supervised learning.

[0104] Table 3 Comparison results with supervised pulmonary nodule classification methods

[0105]

[0106] (4) In order to verify the effectiveness of each module in the present invention, the present invention conducted extensive ablation experiments on the effective design part of the model, including ablation experiments on three modules: the feature contrast learning submodule M1 based on high-confidence pseudo-labels, the classification vector contrast learning submodule M2 based on enhanced anchoring, and the fine-tuning module M3. The results are shown in Table 4, where "√" indicates that the corresponding module is used in the current method. First, the feature contrast learning submodule based on high-confidence pseudo-labels is added to the basic framework, and the accuracy, sensitivity, and specificity are improved by 0.76%, 1.02%, and 0.61% respectively; then the classification vector contrast learning submodule based on enhanced anchoring is added, and the accuracy, sensitivity, and specificity are improved by 0.65%, 0.9%, and 0.42% respectively. This shows that feature contrastive learning and classification vector contrastive learning effectively utilize information between unlabeled samples. In addition, using the fine-tuning module after semi-supervised training improves accuracy, sensitivity, and specificity by 0.73%, 1.6%, and 0.21%, respectively. This fully demonstrates that supervised contrastive fine-tuning can correct some errors in unsupervised contrastive learning by constructing positive and negative sample pairs using real label information, thereby improving model performance.

[0107] Table 4 Ablation test results of effective modules in the system

[0108] M1 M2 M3 Accuracy Sensitivity Specificity Accuracy AUC F1 score 91.59 84.68 96.09 93.53 93.78 88.68 √ 92.35 85.70 96.70 94.44 93.91 90.06 √ √ 93.00 86.60 97.12 95.23 94.33 90.96 √ √ √ 93.73 88.20 97.33 95.62 94.35 91.63

[0109] At the same time, in order to verify the impact of different data enhancement strategies on the performance of the benign and malignant lung nodule classification task, the strong enhancement methods in the two data enhancement strategies were compared, one was the RandAugment method, and the other was the strong enhancer module in the present invention. In this comparative experiment, both strong enhancement methods acted on the basis of the weak enhancer module, and in order to keep the number of enhancements consistent, the RandAugment used randomly selected 3 enhancement methods from the predefined enhancement method pool to act on the image. All additional modules (discriminative feature extraction module and fine-tuning module) were removed, and the feature extraction network and other hyperparameters were kept unchanged, and the comparative experimental results were obtained as shown in Table 5. It can be seen from the data in the table that the strong enhancer module of the present invention achieved higher accuracy and sensitivity, which were increased by 0.25% and 1.6% respectively.

[0110] Table 5 Comparison results of image enhancement methods

[0111]

[0112]

[0113] In order to further analyze the impact of different strong enhancement methods on the model accuracy, the experiment visualized the accuracy curve of one fold of the model in the table. The accuracy comparison curve is as follows: Figure 4 As shown. Figure 4As can be seen in the model using the strong enhancer module of the present invention ( Figure 4 The model with RandAugment as the strong enhancer module ( yellow line in the middle) converges faster and can reach the highest accuracy on this fold after about 10,000 iterations. Figure 4 The middle blue line) cannot reach the highest accuracy during the entire training process. At the same time, the model using RandAugment as a strong enhancer module has a large fluctuation in model training due to the serious distortion of nodules by some of the enhancement methods in its enhancement pool. Figure 4 Large accuracy fluctuations occurred around the 5000th and 15000th iterations. However, the accuracy curve of the model using the strong enhancer module of the present invention had smaller fluctuations and was more stable overall. From this comparison, it can be seen that the strong enhancer module of the data enhancement module of the present invention is more suitable for the task of lung nodule classification. It can better retain the effective information of the nodules while enhancing them, avoiding excessive distortion of the nodules that affects the training effect.

[0114] In addition, the experiment analyzes the impact of the contrastive learning methods based on the feature level and the classification vector level on the performance of the lung nodule classification task. First, in order to verify the impact of different feature contrastive learning methods on the performance of the lung nodule classification task, the experiment compares two feature contrastive learning methods, one is a single positive example contrastive learning method similar to SimCLR

[11] , and the other is the feature contrastive learning submodule based on high-confidence pseudo labels of the present invention. In this comparative experiment, the classification vector contrastive learning submodule and the fine-tuning module based on enhanced anchoring are eliminated, and only the contrastive learning method in the feature contrastive learning and the strong enhancer module of the data enhancement module are changed. Four models are designed for comparative experiments. The four models are: Model I uses RandAugment as a strong enhancer module, and the feature contrastive learning method uses a single positive example contrastive learning method. Model II uses RandAugment as a strong enhancer module, and the feature contrastive learning method uses the feature contrastive learning submodule based on high-confidence pseudo labels of the present invention. Model III uses the strong enhancer module of the present invention, and the feature contrastive learning method uses a single positive example contrastive learning method. The comparative experimental results are shown in Table 6. Comparing Experimental Model III with the present invention demonstrates that the present invention's feature contrastive learning submodule, based on high-confidence pseudo-labels, achieves the highest performance across all metrics compared to a single positive example contrastive learning method similar to SimCLR, with improvements of 0.62% in accuracy, 1.01% in specificity, and 0.04% in sensitivity. In binary classification tasks, methods that construct sample pairs based on a single positive example can exacerbate the problem of potentially similar samples being classified as negative. The feature contrastive learning submodule, based on high-confidence pseudo-labels, constructs positive and negative sample pairs using high-confidence pseudo-labels, expanding the number of positive sample pairs and enabling targeted contrastive learning overall, alleviating this problem to a certain extent.

[0115] Table 6 Results of different contrastive learning strategies

[0116]

[0117]

[0118] The results of the comparative experimental models I and II show that when the feature contrastive learning submodule based on high-confidence pseudo-labels of the present invention uses RandAugment as a strong enhancer module, the accuracy decreases by 0.12%, the sensitivity decreases by 0.37%, and the precision decreases by 0.24% compared with the contrastive learning method of a single positive example.

[0119] Finally, in order to observe the influence of the feature contrast learning submodule based on high-confidence pseudo-labels and the classification vector contrast learning submodule based on enhanced anchoring on the model features of the present invention, and to compare the influence of the feature contrast learning method of a single positive example and the feature contrast learning submodule based on high-confidence pseudo-labels of the present invention on the model features, the experiment uses the t-SNE

[12] algorithm to perform dimensionality reduction visualization on the final classification features of the following four models. The four models are: (a) based on the consistency regular basic framework alone. (b) adding a single positive example contrast learning method to the basic framework to act on the feature level. (c) adding the feature contrast learning submodule based on high-confidence pseudo-labels of the present invention to the basic framework. (d) adding the feature contrast learning submodule based on high-confidence pseudo-labels and the classification vector contrast learning submodule based on enhanced anchoring to the basic framework. The t-SNE visualization results are as follows: Figure 5 As shown. Figure 5 As can be seen from (a) in the figure, the classification features of the model without feature contrast learning have obvious outliers, and the intra-class distance optimization is insufficient. When the model is added with the feature contrast learning submodule based on high-confidence pseudo-labels of the present invention, that is, Figure 5 In (c), outliers disappear and intra-class distances are shortened. Compared with model (b) using a single positive example contrastive learning strategy, the feature distribution of the model based on single positive example contrastive learning is loose after dimensionality reduction and is not compact enough. Finally, model (d), which simultaneously uses the feature contrastive learning submodule based on high-confidence pseudo-labels and the classification vector contrastive learning submodule based on enhanced anchoring of the present invention, further increases the distance between classes while maintaining intra-class compactness, further optimizing the decision boundary.

Claims

1. A smart early screening system for lung cancer, comprising a data augmentation module, a discriminative feature extraction module, and a fine-tuning module. The data augmentation module generates weakly enhanced samples and strongly enhanced samples. The discriminative feature extraction module extracts feature vectors and classification vectors for the corresponding strong and weakly enhanced samples, and performs comparative learning and discriminative learning on the two, respectively, to narrow the distance between semantically related samples and exclude semantically unrelated samples. The fine-tuning module optimizes the feature extractor and enhances the accuracy of the model classification through comparative learning between the original image with labeled data and the corresponding weakly enhanced samples. in, The discriminative feature extraction module includes a feature contrast learning submodule based on high-confidence pseudo-labels, a classification vector contrast learning submodule based on enhanced anchoring, and a discriminative learning submodule. Unlabeled weakly enhanced samples, strongly enhanced samples, and labeled weakly enhanced samples are input into the backbone network. The feature vectors and classification vectors of the unlabeled samples are then taken out and input into the feature contrast learning submodule based on high-confidence pseudo-labels and the classification vector contrast learning submodule based on enhanced anchoring, respectively. The classification vectors of the labeled samples and the classification vectors of the unlabeled strongly enhanced samples are then sent to the discriminative learning submodule. In the feature contrast learning submodule based on high-confidence pseudo-labels, unlabeled weakly enhanced samples and strongly enhanced samples are respectively pseudo-labeled through the classification layer of the backbone network; unlabeled weakly enhanced samples and strongly enhanced samples are constructed into sample pairs based on the pseudo-labels; In the enhanced anchor-based classification vector comparison learning submodule, unlabeled weakly enhanced samples and strongly enhanced samples are fed into the backbone network to obtain corresponding classification vectors, which are used to construct ordered tuple pairs. By bringing semantically related classification vectors closer together and spreading semantically unrelated classification vectors further apart in the ordered tuple pairs, knowledge between the classification vectors of unlabeled nodules at the classification level is learned. The discriminative learning submodule uses the cross-entropy loss function to compare the classification vectors of labeled weakly enhanced samples with the true labels to obtain supervised loss. At the same time, it compares the classification vectors of unlabeled strongly enhanced samples with their corresponding pseudo labels to obtain unsupervised loss, thereby achieving benign and malignant lung nodule classification. The fine-tuning module is based on the model obtained after the semi-supervised learning training. It only uses labeled nodules to construct sample pairs of lung nodule original images and weakly enhanced images. Based on the true labels of the samples, it learns the similarities between samples of the same type and the differences between samples of different types to correct some errors in semi-supervised learning and improve network performance. The fine-tuning module adopts a supervised contrastive learning method. At the feature level of labeled nodules, it retains the original image of the labeled nodules, obtains the weakly enhanced image of the labeled nodules through the data enhancement module, and uses the true label of the labeled nodules and assigns it to the weakly enhanced sample of the labeled sample.

2. The intelligent early screening system for lung cancer according to claim 1, characterized in that: The data enhancement module includes a weak enhancer module and a strong enhancer module; the weak enhancer module T weak The random horizontal flipping and random cropping methods are used on the lung nodule samples to obtain the corresponding weak enhancement samples; the strong enhancer module T strong On the basis of the weak enhancement submodule, random rotation, contrast enhancement and CutOut methods are added to obtain strong enhancement samples; in the labeled dataset X, a data sample x i Only through the weak enhancer module T weak Get the weakly enhanced sample of the data sample, that is, In an unlabeled dataset U, a data sample u i , respectively through the weak enhancer module T weak and strong enhancer module T strong Get the weak enhancement sample and strong enhancement sample of the data sample, that is, 3. The intelligent early screening system for lung cancer according to claim 1, characterized in that: EfficientNet is selected as the backbone network.

4. The intelligent early screening system for lung cancer according to claim 1, characterized in that: In the feature contrast learning submodule based on high-confidence pseudo-labels, for a specific sample’s feature vector When a batch contains 2N unlabeled weakly enhanced samples and strongly enhanced samples, 2N-1 ordered sample pairs are matched. Among them, the samples with the same pseudo-label constitute the positive sample pairs, and the rest constitute the negative sample pairs. The potential relationship between unlabeled nodules is used at the feature level. The loss function calculation formula for feature contrast learning is as follows: Among them, U contains the 2N unlabeled sample augmentation data in an input batch, · represents the inner product, P(i) represents all positive samples with the same pseudo label corresponding to sample i in the current input batch, A(i) represents all negative samples with different pseudo labels in the current input batch, z i , z p , z a Represents the unlabeled data augmentation sample feature vector, M T Represents the confidence mask, and its specific mathematical expression is as follows: Among them, T c represents the confidence threshold, Indicates the threshold value is T c A strict threshold function is used to filter out samples with high pseudo-label confidence to avoid incorrect positive and negative sample pairs hindering contrastive learning.

5. The intelligent early screening system for lung cancer according to claim 4, characterized in that: T c Set to 0.

95.

6. The intelligent early screening system for lung cancer according to claim 4, characterized in that: In the classification vector contrast learning submodule based on enhanced anchoring, the method of constructing ordered tuples is as follows: the feature vector of the unlabeled data augmented sample in an input batch The corresponding classification vector is obtained through the classification layer of the backbone network Weakly enhanced classification vector for unlabeled samples Combine two by two to construct ordered tuple pairs.

7. The intelligent early screening system for lung cancer according to claim 6, characterized in that: The loss function of the classification vector contrastive learning based on enhanced anchoring is expressed as: Where N is the number of samples in the batch, f sim (·) represents the similarity threshold measurement function like The highest class prediction value in exceeds the confidence threshold T c , and in the ordered tuple pair if p i If it is located in the main sequence position, it will be used as an "anchor point", and the similarity measurement function will calculate the similarity between the main sequence and the sub-order prediction vectors in all tuple pairs with the main sequence element as the "anchor point", so as to gather the sub-order classification vectors related to the main sequence category towards the main sequence classification vector; f dist (·) represents the distance function, which is used to express the distance between the classification vector of the strongly enhanced sample corresponding to the order sample and the classification vector of the main order "anchor point", so as to align the classification vector of the strongly enhanced sample with the "anchor point"; T s represents the similarity threshold, Indicates the threshold value is T s A strict threshold function is used to screen reliable tuple pairs, that is, if the similarity between the main order and the order prediction vector in the tuple pair is lower than the preset similarity threshold T s , indicating that the main sequence samples and the secondary sequence samples may be heterogeneous samples, and the similarity mask is used to remove the tuple pair.

8. The intelligent early screening system for lung cancer according to claim 7, characterized in that: The overall loss L of the semi-supervised deep learning semi Including supervised discriminative loss L x , the unsupervised loss L u , the feature contrast loss Lf eature and the classification vector contrast loss L p , expressed as: L semi =L x +L u +λ f L feature +λ p L p where λ f and λ p Control L feature and L p The coefficients are set to 0.001 and 1 respectively to weigh the influence of contrastive learning and discriminative learning in the model training process.

9. The intelligent early screening system for lung cancer according to claim 8, characterized in that: In the construction of the sample pair, the fine-tuning module takes the feature vector z of a specific sample as i , the samples with the same true labels constitute the positive sample pairs {z i ,z}(y i =y), and the rest constitute negative sample pairs, and the cosine distance is used to calculate the similarity between sample pairs to obtain the supervised contrast loss L SCL , expressed as: Among them, P(i) represents all positive samples with the same pseudo-label corresponding to sample i in the current input batch, and A(i) represents all negative samples with different pseudo-labels in the current input batch; The loss function of the fine-tuning module is expressed as: L=L x +λ s L SCL Among them, λ s A hyperparameter that balances supervised discriminative loss and supervised contrastive loss.

10. The intelligent early screening system for lung cancer according to claim 9, characterized in that: λ f and λ p are set to 0.001 and 1 respectively; λ s Set to 0.001.

Images