Intelligent diagnosis method for benign and malignant pulmonary nodules based on radiomics

Abstract

The invention relates to an intelligent diagnosis method for benign and malignant pulmonary nodules based on radiomics, and belongs to the technical field of medical diagnosis, and the method comprises the following steps: S1, building a high-quality data annotation model; s2, a model training data preprocessing stage; s3, a model training optimization stage; s4, improving a nodule benign and malignant prediction model; and S5, performing application expansion in a prediction stage. According to the image omics-based pulmonary nodule benign and malignant intelligent diagnosis method, through a triple guarantee mechanism of back-to-back double-person labeling, DICE threshold value control and high-annuity expert auditing, the accuracy and consistency of labeling data are ensured, through adoption of a 3D-CNN deep learning model, the feature dimension is expanded to million dimensions, and the accuracy and the consistency of the labeling data are ensured. The features are automatically mined and screened through the deep neural network, the limitation that a traditional artificial feature extraction method is low in dimension and high in subjectivity is broken through, richer and more objective feature expression is achieved, and the diagnostic ability of the model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical diagnosis, and particularly to an intelligent diagnosis method for benign and malignant pulmonary nodules based on radiomics. Background Art

[0002] Lung cancer is one of the malignant tumors with the highest incidence and mortality rates globally. Early detection and accurate diagnosis of pulmonary nodules are of great significance for improving the prognosis of patients. With the popularization of low-dose CT screening, the number of pulmonary nodules detected annually has shown an explosive growth. Traditional diagnostic methods mainly rely on radiologists to subjectively judge CT images based on personal experience, suffering from problems such as low diagnostic efficiency and poor inter-doctor consistency. In recent years, with the development of artificial intelligence technology, computer-aided diagnosis (CAD) systems based on deep learning have shown great potential in the field of pulmonary nodule diagnosis.

[0003] Traditional radiomics uses manually predefined feature extraction methods, mainly relying on classical medical image processing algorithms. These algorithms require medical experts and engineers to jointly design feature extraction rules and determine key parameters based on clinical experience. The entire feature extraction process highly depends on manual experience, requires repeated debugging and verification, and is difficult to dynamically adjust once the extraction rules are determined. The features extracted by this method mainly include four categories: first-order features reflect the basic attributes of the lesion such as volume, density, etc.; morphological features describe the geometric characteristics of the lesion such as roundness, irregularity; texture features describe the internal structure of the lesion through methods such as gray-level co-occurrence matrix; statistical features describe the gray-scale distribution characteristics from a mathematical perspective. These features are designed based on traditional image processing theory and have clear mathematical definitions and physical meanings.

[0004] Although traditional intelligent diagnosis methods have mathematical definitions and physical meanings, they still have the following problems: 1. In the existing technology, the annotation of pulmonary nodule image data mainly relies on a single doctor to complete, lacking a systematic quality control mechanism. This annotation method is not only easily affected by personal subjective experience, resulting in poor consistency of annotation results, but also the amount of annotated data is generally insufficient, especially lacking benign case data confirmed by long-term follow-up, seriously affecting the quality and reliability of model training. 2. The convolutional neural network structure adopted in the existing technology is relatively simple, and the feature dimension is usually only in the range of thousands to tens of thousands of dimensions, with obvious insufficient feature expression ability. This limitation causes the model to be unable to fully capture the complex features of pulmonary nodules. Especially when dealing with lesions with complex morphological features such as ground-glass nodules, it is difficult to achieve accurate feature extraction and expression, directly affecting the diagnostic accuracy. 3. The existing technology mostly adopts a simple binary classification output method, only giving the judgment results of benign or malignant, lacking the probability quantification of the prediction results. This rough output method cannot provide more detailed reference information for clinicians and is also difficult to reflect the confidence level of the model in the prediction results, greatly reducing the clinical application value and credibility. The reliability and applicability of traditional intelligent diagnosis methods in actual clinical applications are severely limited and difficult to meet the needs of clinical practice. Therefore, this application proposes an intelligent diagnosis method for the benign and malignant of pulmonary nodules based on radiomics to solve the technical problems proposed above. Summary of the Invention

[0005] Aiming at the deficiencies of the existing technology, the present invention provides an intelligent diagnosis method for the benign and malignant of pulmonary nodules based on radiomics, which has the advantages of high diagnostic accuracy, high reliability of prediction results and wide application range, and solves the problem that the existing diagnosis methods lack benign case data confirmed by long-term follow-up, seriously affecting the quality and reliability of model training.

[0006] To achieve the above object, the present invention provides the following technical solutions: An intelligent diagnosis method for the benign and malignant of pulmonary nodules based on radiomics, comprising the following steps: S1. Establish a high-quality data annotation model: First, in the data annotation link, use mid-senior imaging experts to outline the nodule contours and adopt a back-to-back double annotation method; when the DICE of the double annotation overlap is less than 90%, it is reviewed and outlined again by senior imaging experts for confirmation, and the visible ground-glass contour is used as the boundary during the outlining. S2. Preprocessing stage of model training data: In the model training stage, first crop the CT data, retain the bounding box of the lungs to reduce the computational complexity, and then perform data preprocessing by means of flipping, translation, adding Gaussian noise and salt-and-pepper noise. S3. Model training and optimization stage: Next, the Region Proposal Network (RPN) of the object detection Faster-RCNN is used in combination with 3D-ResNet18 to detect candidate nodules. The overall model adopts a U-shaped architecture to automatically outline the position and scope of the nodules; S4. Improvement of the nodule benign and malignant prediction model: The nodule benign and malignant prediction model uses the cutting-edge three-dimensional convolutional neural network (3D-CNN) technology. With the preprocessed nodule position image information as the input, during the training process, the convolutional neural network is used to automatically mine and screen the most significant features for benign and malignant judgment. The network feature dimension can reach one million dimensions, which is more abundant and complete than the features extracted based on experience traditionally. Through the training of a large-scale dataset, the feature weights are continuously adjusted to make the model prediction results converge to the pathological gold standard; S5. Application expansion in the prediction stage: In the prediction stage, the model can apply the knowledge learned from a large amount of data to the identification of the benign and malignant of new lesions, and give a malignant probability of 0%-100%. In the retrospective study of more than 900 external surgical patients, the positive predictive value of the identification reaches more than 95%.

[0007] Furthermore, in the data annotation link of S1, more than 40,000 cases of domestic and foreign pulmonary nodule patient data are accumulated for annotation. Among them, more than 18,000 cases of data have definite pathological outcomes, and the remaining more than 20,000 cases of data have CT images with no changes in three consecutive years of follow-up as benign labels.

[0008] Furthermore, in the data annotation link of S1, a semi-automatic annotation tool needs to be further used to assist mid-level seniority imaging experts in outlining the nodule contours to improve the annotation efficiency and accuracy; when the DICE of the double-person annotation overlap is lower than 90%, in addition to being reviewed by senior imaging experts, a third-party independent review mechanism is also introduced to ensure the reliability of the annotation results.

[0009] Furthermore, in the data preprocessing stage of S2, in addition to flipping, translation, adding Gaussian noise and salt-and-pepper noise, rotation, scaling and contrast adjustment are also used to further enhance the generalization ability of the model.

[0010] Furthermore, in S2, targeted preprocessing strategies need to be adopted for different types of CT data to improve the applicability of the model. The CT data includes low-dose CT and high-resolution CT.

[0011] Furthermore, when using the Region Proposal Network (RPN) of Faster-RCNN in combination with 3D-ResNet18 for candidate nodule detection in S3, an attention mechanism needs to be introduced to improve the model's attention to key features.

[0012] Furthermore, on the basis of the overall U-shaped architecture of the model in S3, skip connections are introduced to reduce information loss and improve the contour accuracy of the model.

[0013] Furthermore, in S4, the large-scale dataset contains more than 40,000 cases. When using the three-dimensional convolutional neural network (3D-CNN) technology in S4, a deep learning optimization algorithm is introduced to accelerate model convergence and improve prediction accuracy. The deep learning optimization algorithm can be any one of Adam or RMSprop.

[0014] Furthermore, in S4, in addition to automatically mining and screening features using the convolutional neural network, clinical information needs to be combined for multimodal fusion prediction to improve the prediction performance of the model; the clinical information includes patient age, gender, and smoking history.

[0015] Furthermore, in S5, the prediction stage also provides quantitative information on the volume and density of nodules to assist doctors in making more comprehensive diagnoses; when applying the model to the identification of the benign and malignant nature of new lesions in S5, a real-time feedback mechanism needs to be combined to continuously optimize the model to adapt to changing clinical needs.

[0016] Compared with the prior art, the present invention provides an intelligent diagnosis method for the benign and malignant nature of pulmonary nodules based on radiomics, having the following beneficial effects: 1. The intelligent diagnosis method for the benign and malignant nature of pulmonary nodules based on radiomics, through a triple guarantee mechanism of back-to-back double annotation, DICE threshold control, and senior expert review, ensures the accuracy and consistency of the annotated data. At the same time, a large-scale standardized dataset of more than 40,000 cases is established, which contains label data verified by pathological diagnosis and long-term follow-up, effectively solving the problem of unreliable data quality.

[0017] 2. The intelligent diagnosis method for the benign and malignant nature of pulmonary nodules based on radiomics, by adopting a 3D-CNN deep learning model, expands the feature dimension to one million dimensions, and automatically mines and screens features through a deep neural network, breaking through the limitations of traditional manual feature extraction methods with low dimensions and strong subjectivity, achieving a richer and more objective feature expression, and improving the diagnostic ability of the model.

[0018] 3. The intelligent diagnosis method for the benign and malignant nature of pulmonary nodules based on radiomics, by combining the RPN of Faster R-CNN with 3D-ResNet18, adopting a U-shaped network architecture, and through multi-scale feature fusion, improves the accuracy of nodule detection and localization, especially for the boundary delineation of ground-glass nodules is more accurate.

[0019] 4. The intelligent diagnosis method for the benign and malignant of pulmonary nodules based on radiomics provides a more fine-grained quantitative reference basis for clinicians by outputting continuous probability prediction values from 0% to 100%, which is different from the traditional binary classification results, improves the interpretability of the prediction results and the clinical practical value, and achieves a positive prediction value of more than 95% in the external validation set.

[0020] 5. The intelligent diagnosis method for the benign and malignant of pulmonary nodules based on radiomics significantly improves the generalization ability of the model through diversified data augmentation methods such as flipping, translation, adding Gaussian noise and salt-and-pepper noise. At the same time, the large-scale training data set also ensures the stable performance of the model in different scenarios. Detailed implementation manners

[0021] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0022] An intelligent diagnosis method for the benign and malignant of pulmonary nodules in this embodiment includes the following steps: S1. Establish a high-quality data annotation model: First, in the data annotation link, middle-seniority imaging experts are used to outline the nodule contours, and the back-to-back double annotation method is adopted; when the DICE of the double annotation overlap is lower than 90%, senior imaging experts are used to review and outline again for confirmation, and the visible ground-glass contour is used as the boundary during the outlining. S2. Preprocessing stage of the model training data: In the model training stage, first crop the CT data, retain the bounding box of the lungs to reduce the computational complexity, and then perform data preprocessing by flipping, translation, adding Gaussian noise and salt-and-pepper noise. S3. Model training optimization stage: Then, the region proposal network (RPN) of the object detection Faster-RCNN is combined with 3D-ResNet18 to detect candidate nodules, and the overall model adopts a U-shaped architecture to automatically outline the position and range of the nodules. S4. Improvement of the prediction model for the benign and malignant of nodules: The prediction model for the benign and malignant of nodules adopts the advanced three-dimensional convolutional neural network (3D-CNN) technology, uses the preprocessed nodule position image information as the input, and automatically mines and screens the most significant features for the judgment of benign and malignant during the training process. The network feature dimension can reach one million dimensions, which is more rich and complete than the features extracted based on experience in the past. Through the training of a large-scale data set, the feature weights are continuously adjusted to make the model prediction results converge to the pathological gold standard. S5. Application Expansion in the Prediction Stage: In the prediction stage, the model can apply the knowledge learned from massive data to the identification of the benign or malignant nature of new lesions, giving a malignant probability of 0% - 100%. In a retrospective study of more than 900 external surgical patients, the positive predictive value of the identification reached over 95%.

[0023] Specifically, in this embodiment, in the data annotation step of S1, a total of more than 40,000 cases of lung nodule patients at home and abroad were annotated. Among them, more than 18,000 cases of data had definite pathological outcomes, and the remaining more than 20,000 cases of data had CT scans without changes during three consecutive years of follow-up as benign labels.

[0024] Specifically, in this embodiment, in the data annotation step of S1, a semi-automatic annotation tool is further used to assist mid-level seniority imaging experts in outlining the nodule contours to improve the annotation efficiency and accuracy; when the DICE of double-person annotation overlap is lower than 90%, in addition to being reviewed by senior imaging experts, a third-party independent review mechanism is also introduced to ensure the reliability of the annotation results.

[0025] Specifically, in this embodiment, in the data preprocessing stage of S2, in addition to flipping, translation, adding Gaussian noise and salt-and-pepper noise, rotation, scaling and contrast adjustment are also used to further enhance the generalization ability of the model.

[0026] Specifically, in this embodiment, targeted preprocessing strategies need to be adopted for different types of CT data to improve the applicability of the model. The CT data includes low-dose CT and high-resolution CT.

[0027] Specifically, in this embodiment, when using the Region Proposal Network (RPN) of Faster-RCNN combined with 3D-ResNet18 for candidate nodule detection in S3, an attention mechanism needs to be introduced to improve the model's attention to key features.

[0028] Specifically, in this embodiment, on the basis of the overall U-shaped architecture of the model in S3, skip connections are introduced to reduce information loss and improve the outlining accuracy of the model.

[0029] Specifically, in this embodiment, when using the three-dimensional convolutional neural network (3D-CNN) technology in S4, a deep learning optimization algorithm is introduced to accelerate the model convergence and improve the prediction accuracy.

[0030] It should be noted that in this embodiment, the large-scale dataset is more than 40,000 cases.

[0031] It should be noted that the deep learning optimization algorithm can be any one of Adam or RMSprop.

[0032] Specifically, in this embodiment, in S4, in addition to automatically mining and screening features using a convolutional neural network, clinical information also needs to be combined for multimodal fusion prediction to improve the prediction performance of the model; the clinical information includes patient age, gender, and smoking history.

[0033] Specifically, in this embodiment, in S5, during the prediction stage, quantitative information on the volume and density of the nodules is also provided to assist doctors in making a more comprehensive diagnosis; when applying the model to the identification of the benign or malignant nature of new lesions in S5, a real-time feedback mechanism needs to be combined to continuously optimize the model to adapt to the changing clinical needs.

[0034] The advantages of the above embodiment are as follows:

[0035] 1. Data quality and scale advantages: In the present invention: By establishing a large-scale standardized dataset of more than 40,000 cases, including more than 18,000 cases of pathologically diagnosed data and more than 20,000 cases of benign data confirmed by long-term follow-up, and adopting a strict quality control system of back-to-back double annotation and expert review.

[0036] In the prior art: Mainly relying on the Kaggle competition dataset, the data scale is small, and there is a lack of a strict annotation quality control mechanism.

[0037] Advantage manifestation: The data scale is larger and the quality is higher, which can cover the clinical actual scenarios more comprehensively and provide a more reliable basis for model training.

[0038] 2. Model architecture innovation: In the present invention: By combining the RPN of Faster R-CNN with 3D-ResNet18 and adopting a U-shaped network architecture, a two-stage intelligent diagnosis model is realized.

[0039] In the prior art: Only using a basic CNN network architecture and adopting a single end-to-end training method.

[0040] Advantage manifestation: The model structure is more optimized, the feature extraction is more comprehensive, especially showing better performance when dealing with complex pulmonary nodule morphological features.

[0041] 3. Feature dimension breakthrough: In the present invention: The feature space is extended to one million dimensions, and richer feature expressions are realized through 3D-CNN.

[0042] In the prior art: Mainly relying on the feature extraction ability of traditional CNN, the feature dimension is relatively limited.

[0043] Advantage manifestation: It can capture more subtle image features, provide more comprehensive feature expressions, and improve the diagnostic accuracy.

[0044] 4. Prediction result accuracy: The present invention: outputs continuous probability prediction values from 0% to 100%, and achieves a positive predictive value (PPV) of over 95% in an external validation set.

[0045] Prior art: usually adopts simple classification output and lacks probability prediction ability.

[0046] Advantage manifestation: provides more accurate prediction results and offers a more valuable reference basis for clinical decision-making.

[0047] 5. Clinical practicability: The present invention: demonstrates the generalization ability of the model through large-scale external validation and provides interpretable probability prediction results.

[0048] Prior art: mainly focuses on algorithm performance and lacks systematic clinical validation.

[0049] Advantage manifestation: more suitable for actual clinical applications and has higher practical value.

[0050] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent diagnosis method for benign and malignant pulmonary nodules based on radiomics, characterized in that: The following steps are involved: S1. Establish a high-quality data annotation model: First, in the data annotation stage, middle-aged imaging experts are used to outline the nodules, using a back-to-back double-annotation method; when the double-annotation overlap DICE is less than 90%, a senior imaging expert will review and confirm the nodule outline again, using the visible ground-glass outline as the boundary; S2, model training data preprocessing stage: In the model training stage, the CT data is first cropped to retain the lung bounding box to reduce the computational complexity, and then the data is preprocessed by flipping, translating, adding Gaussian noise and salt and pepper noise; S3, model training and optimization stage: Then, the target detection Faster-RCNN's region proposal network (RPN) combined with 3D-ResNet18 is used to detect candidate nodules. The overall model adopts a U-shaped architecture to automatically outline the location and range of nodules; S4. Improvement of the nodule benign and malignant prediction model: The nodule benign and malignant prediction model adopts the cutting-edge three-dimensional convolutional neural network (3D-CNN) technology, with pre-processed nodule location image information as input. During the training process, the convolutional neural network is used to automatically mine and screen the most significant features for benign and malignant judgment. The network feature dimension can reach millions of dimensions, which is richer and more complete than the traditional experience-based feature extraction. Through the training of large-scale data sets, the feature weights are continuously adjusted to make the model prediction results converge to the pathology gold standard; S5. Application expansion in the prediction stage: In the prediction stage, the model can be applied to the identification of new lesions based on the knowledge learned from massive data, giving a malignancy probability of 0%-100%. In a retrospective study of more than 900 patients undergoing external surgery, the positive predictive value of identification reached more than 95%.

2. The method for intelligent diagnosis of benign and malignant pulmonary nodules based on radiomics according to claim 1, characterized in that: The data annotation process in S1 has annotated more than 40,000 data of patients with lung nodules at home and abroad, of which more than 18,000 cases have definite pathological outcomes, and the remaining more than 20,000 cases have CT scans with no changes in three consecutive years of follow-up as benign labels.

3. The method for intelligent diagnosis of benign and malignant pulmonary nodules based on radiomics according to claim 1, characterized in that: The data annotation process in S1 needs to further use semi-automatic annotation tools to assist middle-aged imaging experts in outlining nodules to improve annotation efficiency and accuracy; when the double annotation overlap DICE is less than 90%, in addition to the review by senior imaging experts, a third-party independent review mechanism is also introduced to ensure the reliability of the annotation results.

4. The method for intelligent diagnosis of benign and malignant pulmonary nodules based on radiomics according to claim 1, characterized in that: In the data preprocessing stage of S2, in addition to flipping, translating, adding Gaussian noise and salt and pepper noise, rotation, scaling and contrast adjustment are also used to further enhance the generalization ability of the model.

5. The method for intelligent diagnosis of benign and malignant pulmonary nodules based on radiomics according to claim 1, characterized in that: In S2, targeted preprocessing strategies need to be adopted for different types of CT data to improve the applicability of the model. The CT data includes low-dose CT and high-resolution CT.

6. The method for intelligent diagnosis of benign and malignant pulmonary nodules based on radiomics according to claim 1, characterized in that: In S3, when using the Faster-RCNN region proposal network (RPN) combined with 3D-ResNet18 to detect candidate nodules, it is necessary to introduce an attention mechanism to increase the model's attention to key features.

7. The method for intelligent diagnosis of benign and malignant pulmonary nodules based on radiomics according to claim 1, characterized in that: The S3 model adopts a U-shaped architecture as a whole, and introduces skip connections to reduce information loss and improve the model's outlining accuracy.

8. The method for intelligent diagnosis of benign and malignant pulmonary nodules based on radiomics according to claim 1, characterized in that: The large-scale data set in S4 is more than 40,000 cases. When using the three-dimensional convolutional neural network (3D-CNN) technology in S4, a deep learning optimization algorithm is introduced to accelerate model convergence and improve prediction accuracy. The deep learning optimization algorithm can be any one of Adam or RMSprop.

9. The method for intelligent diagnosis of benign and malignant pulmonary nodules based on radiomics according to claim 1, characterized in that: In addition to using convolutional neural networks to automatically mine and screen features, S4 also needs to combine clinical information for multimodal fusion prediction to improve the prediction performance of the model; the clinical information includes the patient's age, gender, and smoking history.

10. The method for intelligent diagnosis of benign and malignant pulmonary nodules based on radiomics according to claim 1, characterized in that: The prediction stage in S5 also provides quantitative information on the volume and density of nodules to assist doctors in making a more comprehensive diagnosis. When the model is applied to the identification of new benign and malignant lesions in S5, the model needs to be continuously optimized in combination with a real-time feedback mechanism to adapt to changing clinical needs.