Method and device for identifying non-small cell lung cancer subcarina lymph node metastasis and prognosis prediction based on dual-region radiomics and deep learning model, storage medium and electronic equipment

Abstract

The embodiment of the invention provides a method and device for identifying lymph node metastasis and prognosis prediction under non-small cell lung cancer carina based on dual-region radiomics and a deep learning model, a storage medium and electronic equipment, and relates to the technical field of lung cancer diagnosis. Compared with a traditional scheme for judging lymph node metastasis of a non-small cell lung cancer patient based on physician experience, an artificial intelligence model is constructed by using dual-region CT radiomics, and a highly reliable prediction result is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of lung cancer diagnosis. Specifically, it relates to a method, device, storage medium and electronic device for differentiating subcarinal lymph node metastasis and prognostic prediction of non-small cell lung cancer based on dual-region radiomics and deep learning models. Background Art

[0002] Due to the dense lymphatic vessels in the lungs, lung cancer often metastasizes to lymph nodes. Important lymph nodes such as subcarinal lymph nodes, as important sites for pulmonary lymphatic drainage, their nature is an important factor in evaluating the stage of lung cancer and treatment prognosis. Therefore, accurate preoperative prediction of the nature of lymph nodes is crucial for formulating appropriate treatment strategies and improving the prognosis of patients with non-small cell lung cancer (NSCLC). In the past, lymph node exploration was mainly carried out by invasive methods such as transbronchial needle aspiration biopsy, which has high requirements for technology and various complications. At present, with the progress of medical technology, the diagnostic method of lymph node metastasis has changed to mainly image diagnosis. The quality of traditional image diagnosis mainly depends on the personal experience of radiologists and the imaging quality of imaging equipment. This method is time-consuming and laborious, and the diagnosis is inaccurate. Therefore, there is an urgent need for a method to assist physicians in diagnosis. Summary of the Invention

[0003] Embodiments of the present invention provide a method, device, storage medium and electronic device for differentiating subcarinal lymph node metastasis and prognostic prediction of non-small cell lung cancer based on dual-region radiomics and deep learning models to solve the technical problems existing in the prior art.

[0004] Other features and advantages of the present invention will become apparent through the following detailed description, or be learned in part through the practice of the present invention.

[0005] According to the first aspect of the embodiments of the present invention, there is provided a method for differentiating subcarinal lymph node metastasis and prognostic prediction of non-small cell lung cancer based on dual-region radiomics and deep learning models, including: Collecting the desensitized clinical data and CT images of patients; Extracting the region of interest based on the CT images; Using high-throughput feature extraction technology to extract tumor-SLN radiomics features based on the region of interest, where the tumor-SLN radiomics features include: intensity features, texture features, shape features and wavelet transform features; Using a deep neural network to extract deep learning features based on the CT images; Using the min-max normalization method to normalize the tumor-SLN radiomics features and the deep learning features; Multiple tumor-SLN dual-region radiomics models are constructed based on the processed tumor-SLN radiomics features; Multiple tumor-SLN dual-region deep learning models are constructed based on the processed deep learning features; Multiple tumor-SLN dual-region radiomics models and multiple tumor-SLN dual-region deep learning models are verified and evaluated, and an optimal model is selected as the prediction model; The prediction model is used to identify subcarinal lymph node metastasis and prognosis prediction of non-small cell lung cancer.

[0006] In some embodiments of the present invention, based on the foregoing solution, the regions of interest extracted based on the CT images include: Based on the CT images, three-dimensional reconstruction of the tumor is performed to obtain preoperative images; Based on professional medical knowledge, the tumor region and the SLN region are outlined from the preoperative images as regions of interest.

[0007] In some embodiments of the present invention, based on the foregoing solution, the tumor-SLN radiomics features extracted based on the regions of interest by using high-throughput feature extraction technology include: The sizes of the regions of interest in each group are unified using standardized volume; Using high-throughput feature extraction technology, tumor radiomics features and SLN radiomics features are extracted from the regions of interest; The tumor radiomics features and the SLN radiomics features are fused to obtain tumor-SLN radiomics features.

[0008] In some embodiments of the present invention, based on the foregoing solution, the deep learning features extracted based on the CT images by using a deep neural network include: The CT images are multiplied by the gold standard mask, cropped into voxel blocks, normalized and resized; The resized CT image voxel blocks are input into a 3D VGG network for feature extraction processing, and the process includes: The resolution of the CT image voxel blocks is gradually reduced through max pooling; Tumor feature vectors and SLN feature vectors are generated through global average pooling; The tumor feature vectors and the SLN feature vectors are merged through a Concatenate operation and input into a fully connected layer for further fusion to obtain deep learning features.

[0009] In some embodiments of the present invention, based on the foregoing solution, the construction of multiple tumor-SLN dual-region radiomics models based on the processed tumor-SLN radiomics features includes: The reliability of each feature was tested according to the intra-class correlation coefficient of the tumor-SLN radiomics features, and the features with intra-class correlation coefficients less than the set threshold were screened out; Using statistical methods, the imaging feature significance and clinical feature significance were calculated based on the tumor-SLN radiomics features and the de-identified clinical data, and the features that did not meet the significance requirements were screened out according to the imaging feature significance and clinical feature significance; Six algorithms, namely random forest, Boruta, Relief, LASSO, IG, and RFE, were used to screen the features again; According to the finally screened features, five machine learning algorithms, namely support vector machine, K-nearest neighbor algorithm, naive Bayes, random forest, and artificial neural network, were used to construct an artificial intelligence model, and a variety of tumor-SLN dual-region radiomics models were obtained.

[0010] In some embodiments of the present invention, based on the foregoing solution, a variety of tumor-SLN dual-region deep learning models were constructed based on the processed deep learning features, including: Convolutional neural network was used to extract and screen the deep learning features related to tumors and SLNs from the deep learning features; The screened deep learning features were fused to obtain tumor-SLN deep learning features; A variety of tumor-SLN dual-region deep learning models were constructed using the tumor-SLN deep learning features.

[0011] In some embodiments of the present invention, based on the foregoing solution, the verification and evaluation of a variety of tumor-SLN dual-region radiomics models and a variety of tumor-SLN dual-region deep learning models were carried out, and an optimal model was selected as the prediction model, including: The area under the curve algorithm was used to evaluate the ability of each model in a variety of tumor-SLN dual-region radiomics models and a variety of tumor-SLN dual-region deep learning models to predict extrapulmonary lymph nodes; The Brier score was used to evaluate the accuracy of each model; Five-fold cross-validation was used to perform internal validation on the performance of each model; Kaplan-Meier survival curve, Log-rank test, and Cox regression risk ratio were used to perform prognostic analysis on each model; According to the clinical applicability of each model, the ability to predict subcarinal lymph nodes, the model accuracy, the model performance, and the prognostic analysis results, a comparison was made, and an optimal model was selected as the prediction model.

[0012] According to the second aspect of the embodiments of the present invention, there is provided an apparatus for identifying subcarinal lymph node metastasis and prognostic prediction of non-small cell lung cancer based on dual-region radiomics and deep learning models, including: An acquisition unit, configured to acquire de-identified clinical data and CT images of a patient; A first extraction unit, configured to extract a region of interest based on the CT images; A second extraction unit, configured to extract tumor-SLN radiomics features based on the region of interest by using high-throughput feature extraction technology, wherein the tumor-SLN radiomics features include: intensity features, texture features, shape features, and wavelet transform features; A third extraction unit, configured to extract deep learning features based on the CT images by using a deep neural network; A processing unit, configured to perform normalization processing on the tumor-SLN radiomics features and the deep learning features by using a min-max normalization method; A first construction unit, configured to construct multiple tumor-SLN dual-region radiomics models based on the processed tumor-SLN radiomics features; A second construction unit, configured to construct multiple tumor-SLN dual-region deep learning models based on the processed deep learning features; A screening unit, configured to perform verification and evaluation on multiple tumor-SLN dual-region radiomics models and multiple tumor-SLN dual-region deep learning models, and screen out an optimal model as a prediction model; A prediction unit, configured to use the prediction model to identify subcarinal lymph node metastasis and prognostic prediction of non-small cell lung cancer.

[0013] According to the second aspect of the embodiments of the present invention, there is provided a computer-readable storage medium, in which computer instructions are stored, and when the computer instructions run on a computer, the computer is enabled to execute the method as described in the first aspect.

[0014] According to the second aspect of the embodiments of the present invention, there is provided an electronic device, including: a memory and a processor; The memory is configured to store computer instructions; The processor is configured to call the computer instructions stored in the memory, so that the electronic device executes the method as described in the first aspect.

[0015] The technical solution of the present invention has the following beneficial effects: (1)High-precision prediction: Verified and trained with a large amount of clinical data, this solution can accurately predict the risk of subcarinal lymph node metastasis in patients with non-small cell lung cancer (AUC: 0.880, Brier score 0.093). Among them, AUC represents the area under the curve, which is an index to measure the performance of a binary classification model. The Brier score represents the Brier score, which is an important index for measuring the accuracy of probability prediction, indicating the difference between the predicted probability and the actual result. The lower the value, the more accurate the prediction. At the same time, an artificial intelligence model is innovatively constructed using dual-region CT radiomics. Compared with the traditional method of judging lymph node metastasis in patients with non-small cell lung cancer based on doctors' experience, it has highly reliable prediction results.

[0016] (2)Non-invasive: Compared with traditional invasive diagnostic methods such as lymph node biopsy, this solution is completely based on CT images and patients' clinical indicators, without the need for additional surgery or biopsy, greatly reducing the pain and risk of patients.

[0017] (3)Personalized risk assessment: This solution takes into account the individual differences of each patient, covering multiple aspects such as clinical characteristics and imaging features, and can provide personalized risk assessment for each patient to help doctors formulate targeted treatment plans.

[0018] (4)Dynamic update: With the continuous expansion of clinical data in the later stage, the technical team can dynamically update and optimize the prediction scheme system to ensure the continuous improvement of performance.

[0019] (5)Rapid detection: Traditional image reading usually takes a long time to obtain results (>30 minutes), while this solution can complete the prediction in a short time (<10 minutes), improving the diagnostic efficiency. This is particularly important for improving the turnover efficiency of the radiology department and relieving the pressure on radiologists.

[0020] (6)Repeatability: Traditional diagnostic methods may be affected by various factors, resulting in unstable results. This solution is based on objective medical image data and machine learning algorithms, and its prediction results have better repeatability and stability.

[0021] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present invention. Description of the Drawings

[0022] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. In the drawings: Figure 1 It shows a schematic flowchart of a method for identifying subcarinal lymph node metastasis and prognosis prediction of non-small cell lung cancer based on dual-region radiomics and deep learning model according to an embodiment of the present invention; Figure 2 It shows a block diagram of a device for identifying subcarinal lymph node metastasis and prognosis prediction of non-small cell lung cancer based on dual-region radiomics and deep learning model according to an embodiment of the present invention; Figure 3 It shows a block diagram of an electronic device according to an embodiment of the present invention; Figure 4 It shows a schematic structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present invention. Detailed implementation manners

[0023] Now, example embodiments will be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present invention will be more complete and comprehensive, and the concept of the example embodiments will be fully conveyed to those skilled in the art.

[0024] In addition, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to give a thorough understanding of the embodiments of the present invention. However, those skilled in the art will realize that the technical solutions of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present invention.

[0025] The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0026] The flowcharts shown in the accompanying drawings are merely illustrative and not necessarily include all content and operations / steps, nor are they necessarily executed in the order described. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined. Therefore, the actual execution order may change according to the actual situation.

[0027] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above accompanying drawings are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the objects so used can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described.

[0028] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, rather than all, embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0029] The following will describe in detail some embodiments of the present invention with reference to the accompanying drawings. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0030] See Figure 1 , which shows a schematic flowchart of a method for identifying subcarinal lymph node metastasis and prognostic prediction of non-small cell lung cancer based on dual-region radiomics and deep learning models according to an embodiment of the present invention.

[0031] As Figure 1 shown, a method for identifying subcarinal lymph node metastasis and prognostic prediction of non-small cell lung cancer based on dual-region radiomics and deep learning models is presented, which specifically includes steps S100 to S900.

[0032] Refer to Figure 1 , step S100, collect the de-identified clinical data and CT images of the patient.

[0033] It can be understood that in this embodiment, first, a standardized inclusion and exclusion criteria are established based on medical experience and professional knowledge, then the patients meeting the requirements are determined according to the inclusion and exclusion criteria, and then the de-identified clinical data and CT images of the patients are collected.

[0034] For example, the patient needs to meet the following conditions: (1) Pathologically confirmed primary non-small cell lung cancer; (2) Underwent pulmonary resection and extra-pulmonary lymph node dissection; (3) Chest CT images within 4 weeks before surgery can be retrieved.

[0035] Patients with the following conditions: (1) receiving neoadjuvant therapy (which may interfere with the tumor nature and thus interfere with the study); (2) having been diagnosed with distant metastasis (the prognosis of the patients is poor and deviates from the study purpose); (3) not having undergone extra-pulmonary lymph node dissection (data missing) are not included in the collection scope.

[0036] It should be noted that in this embodiment, the de-identified clinical data refers to the characteristic data after de-identifying the clinical data. Exemplarily, in this embodiment, the de-identified clinical data collected from patients includes: patient age, gender, body mass index, TNM stage (according to the eighth edition of the "Cancer TNM Staging System" jointly released by the Union for International Cancer Control (UICC) and the American Joint Committee on Cancer (AJCC)), surgical method, tumor location, smoking history, blood type, histological type, tumor size, preoperative hematological indexes (carcinoembryonic antigen, squamous cell carcinoma antigen, cytokeratin 19 fragment, pro-gastrin-releasing peptide, and neuron-specific enolase), etc., which are information for analyzing basic clinical characteristics.

[0037] Continue to refer to Figure 1 , step S200, to extract the region of interest based on the CT image.

[0038] It can be understood that the region of interest established in this embodiment is related to the tumor region and the SLN region. Among them, SLN refers to subcarinal lymph nodes, that is, lymph nodes under the carina.

[0039] In some feasible embodiments, based on the foregoing solution, the extracting the region of interest based on the CT image includes: Performing three-dimensional reconstruction of the tumor based on the CT image to obtain a preoperative image; Outlining the tumor region and the SLN region from the preoperative image as the region of interest based on professional medical knowledge.

[0040] Exemplarily, 64-slice multi-detector row spiral CT (GE LightSpeed VCT) was used to perform three-dimensional reconstruction of tumors under the conditions of tube voltage 120 kVp, tube current 200 mA (using automatic exposure control), tube rotation time 0.5 s, slice thickness 1 - 5 mm, collimation 64 × 0.6 mm, pitch 0.9, matrix 512 × 512 mm, and minimum slice thickness interval to obtain preoperative images. Two physicians with rich imaging experience (the latter verified the work of the former) used 3D Slicer software to delineate the tumor region (delineate the outer contour of the tumor at the maximum diameter level) and the SLN region (mark the boundary of subcarinal lymph nodes) of the CT images as the regions of interest. Quality control was independently delineated by the two physicians, and consensus was reached through consultation when there was disagreement. The intraclass correlation coefficient (ICC>0.75) was calculated to ensure inter-observer consistency.

[0041] Continue to refer to Figure 1 , step S300, using high-throughput feature extraction technology to extract tumor-SLN radiomics features based on the regions of interest, wherein the tumor-SLN radiomics features include: intensity features, texture features, shape features, and wavelet transform features.

[0042] It should be noted that in this embodiment, the intensity feature refers to the feature based on the pixel values of the image, which directly reflects the brightness or gray level of each point in the image; the texture feature describes the complexity and its distribution pattern of the patterns and structures in the image, characterized by the spatial arrangement of pixel intensities within the image region, reflecting attributes such as the uniformity, roughness, and directionality inside the image; the shape feature can focus on the geometric attributes of the objects in the image and is used to identify and analyze specific anatomical structures; the wavelet transform feature is used to decompose the image into frequency components at different scales and directions, and can extract features of the image at different resolutions, including detail and contour information, covering the multi-scale structural information in the processed image, which is conducive to finding tiny abnormal structures.

[0043] In some feasible embodiments, based on the foregoing solution, the using high-throughput feature extraction technology to extract tumor-SLN radiomics features based on the regions of interest includes: Using standardized volume to unify the sizes of the regions of interest in each group; Using high-throughput feature extraction technology to extract tumor radiomics features and SLN radiomics features from the regions of interest; Fusing the tumor radiomics features and SLN radiomics features to obtain tumor-SLN radiomics features.

[0044] It can be understood that the purpose of using standardized volume to unify the sizes of the regions of interest in each group is to eliminate the natural differences existing between different individuals.

[0045] Exemplarily, in this embodiment, the radiomics features are extracted from the delineated region of interest using the PyRadiomics package in Python. Finally, 851 feature data can be extracted from the tumor region and the SLN region respectively, and then the extracted feature data are fused to obtain multiple tumor-SLN radiomics features.

[0046] Continue to refer to Figure 1 , step S400, deep learning features are extracted based on the CT images using a deep neural network.

[0047] In some feasible embodiments, based on the foregoing solution, the extracting deep learning features based on the CT images using a deep neural network includes: Multiply the CT image by the gold standard mask, crop it into voxel blocks, normalize and adjust the size; The voxel blocks of the CT image after adjusting the size are input into a 3D VGG network for feature extraction processing, and the process includes: Gradually reduce the resolution of the CT image voxel blocks through max pooling; Generate a tumor feature vector and an SLN feature vector through global average pooling; Merge the tumor feature vector and the SLN feature vector through a Concatenate operation, and input them into a fully connected layer for further fusion to obtain deep learning features.

[0048] It can be understood that fusing the tumor feature vector and the SLN feature vector using a deep neural network improves the model's ability to model the "tumor-lymph node" interaction relationship in the subsequent modeling process.

[0049] Continue to refer to Figure 1 , step S500, use the min-max normalization method to normalize the tumor-SLN radiomics features and the deep learning features.

[0050] Exemplarily, linear transformation is performed on the tumor-SLN radiomics features and the deep learning features so that all feature values are scaled within the range of 0 to 1, achieving unified data scale and maintaining the original relationship of the data.

[0051] Continue to refer to Figure 1 , step S600, multiple tumor-SLN dual-region radiomics models are constructed based on the processed tumor-SLN radiomics features.

[0052] In some feasible embodiments, based on the foregoing solution, the constructing multiple tumor-SLN dual-region radiomics models based on the processed tumor-SLN radiomics features includes: The reliability of each feature is tested according to the intra-class correlation coefficient of the tumor-SLN radiomics features, and the features with intra-class correlation coefficients less than the set threshold are screened out; Using statistical methods, the imaging feature significance and clinical feature significance are calculated based on the tumor-SLN radiomics features and the de-identified clinical data, and the features that do not meet the significance requirements are screened out according to the imaging feature significance and clinical feature significance; The features are screened again by six algorithms: random forest, Boruta, Relief, LASSO, IG, and RFE; According to the finally screened features, five machine learning algorithms, namely support vector machine, K-nearest neighbor algorithm, naive Bayes, random forest, and artificial neural network, are used to construct an artificial intelligence model to obtain multiple tumor-SLN dual-region radiomics models.

[0053] Exemplarily, in the process of screening according to the imaging feature significance and clinical feature significance, the condition for a feature to be included in the clinical feature significance is: the set threshold p < 0.05, and the condition for a feature to be included in the imaging feature significance is: the set threshold p < 0.01.

[0054] Exemplarily, after the above three screenings, 3 groups of features * 6 algorithms, a total of 18 groups of feature values, regarding the tumor location, SLN location, and tumor-SLN location will be obtained, and then 5 machine learning algorithms, namely support vector machine, K-nearest neighbor algorithm, naive Bayes, random forest, and artificial neural network, are applied to develop 5 * 18 = 90 machine learning models.

[0055] Continue to refer to Figure 1 , step S700, multiple tumor-SLN dual-region deep learning models are constructed based on the processed deep learning features.

[0056] In some feasible embodiments, based on the foregoing solution, the construction of multiple tumor-SLN dual-region deep learning models based on the processed deep learning features includes: Extract and screen the deep learning features related to the tumor and SLN from the deep learning features through a convolutional neural network; Fuse the screened deep learning features to obtain tumor-SLN deep learning features; Use the tumor-SLN deep learning features to construct multiple tumor-SLN dual-region deep learning models.

[0057] Continue to refer to Figure 1 , step S800, verify and evaluate multiple tumor-SLN dual-region radiomics models and multiple tumor-SLN dual-region deep learning models, and screen out an optimal model as the prediction model.

[0058] In some feasible embodiments, based on the foregoing solution, validating and evaluating the multiple tumor-SLN dual-region radiomics models and multiple tumor-SLN dual-region deep learning models, and screening out an optimal model as the prediction model, including: Using the area under the curve algorithm to evaluate the ability of each model in the multiple tumor-SLN dual-region radiomics models and multiple tumor-SLN dual-region deep learning models to predict subcarinal lymph nodes; Using the Brier score to evaluate the accuracy of each model; Adopting five-fold cross-validation to perform internal validation on the performance of each model; Performing prognostic analysis on each model using the Kaplan-Meier survival curve, Log-rank test, and Cox regression risk ratio; Comparing according to the clinical applicability of each model, the ability to predict extrapulmonary lymph nodes, the model accuracy, the model performance, and the prognostic analysis results, and screening out an optimal model as the prediction model.

[0059] It should be noted that there is a risk that the model deviates from the actual clinical needs in the development work, so the applicability of the model needs to be added during the comparison process. Applying the model to clinical practice to test the actual efficacy can improve the clinical application matching degree, enhance the clinical adaptability of the model, and thus better meet the clinical needs.

[0060] Continue to refer to Figure 1 , step S900, using the prediction model to identify subcarinal lymph node metastasis and prognostic prediction of non-small cell lung cancer.

[0061] In summary, the technical solution of the present invention has the following characteristics: (1) Innovatively propose to use dual-region imaging information for model establishment. Compared with the traditional single-region machine learning model that only focuses on the primary tumor region, the machine learning model and deep learning model based on dual-region and clinical feature data show higher performance in predicting extrapulmonary lymph nodes, can comprehensively utilize all information, and have higher prediction value.

[0062] (2) Innovatively provide personalized risk assessment for patients. This solution takes into account the individual differences of each patient, covers multiple aspects such as clinical features and imaging features, and can provide personalized risk assessment for each patient. (3) Innovatively introduce dynamic update. In clinical applications, the model can be continuously updated and optimized dynamically to ensure continuous improvement of efficacy.

[0063] The following describes the device embodiments of the present invention, which can be used to execute a method for identifying subcarinal lymph node metastasis and prognostic prediction of non-small cell lung cancer based on dual-region radiomics and deep learning models in the above embodiments of the present invention. For details not disclosed in the device embodiments of the present invention, please refer to the method embodiments of the present invention above.

[0064] Referring Figure 2 As shown, a device 200 for identifying subcarinal lymph node metastasis and prognostic prediction of non-small cell lung cancer based on dual-region radiomics and deep learning models according to an embodiment of the present invention includes: An acquisition unit 201 for acquiring de-identified clinical data and CT images of a patient; A first extraction unit 202 for extracting regions of interest based on the CT images; A second extraction unit 203 for extracting tumor-SLN radiomics features based on the regions of interest by using high-throughput feature extraction techniques, wherein the tumor-SLN radiomics features include: intensity features, texture features, shape features, and wavelet transform features; A third extraction unit 204 for extracting deep learning features based on the CT images by using a deep neural network; A processing unit 205 for performing normalization processing on the tumor-SLN radiomics features and the deep learning features by using a min-max normalization method; A first construction unit 206 for constructing various tumor-SLN dual-region radiomics models based on the processed tumor-SLN radiomics features; A second construction unit 207 for constructing various tumor-SLN dual-region deep learning models based on the processed deep learning features; A screening unit 208 for verifying and evaluating various tumor-SLN dual-region radiomics models and various tumor-SLN dual-region deep learning models, and screening out an optimal model as a prediction model; A prediction unit 209 for using the prediction model to identify subcarinal lymph node metastasis and prognostic prediction of non-small cell lung cancer.

[0065] As Figure 3 shown, an embodiment of the present invention further provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor. When the processor 320 executes the computer program 311, the steps of the above method for identifying subcarinal lymph node metastasis and prognostic prediction of non-small cell lung cancer based on dual-region radiomics and deep learning models are implemented.

[0066] Since the electronic device introduced in this embodiment is the device used in the device for identifying subcarinal lymph node metastasis and prognosis prediction of non-small cell lung cancer based on dual-region radiomics and deep learning models in the embodiments of the present invention, based on the methods introduced in the embodiments of the present invention, those skilled in the art can understand the specific implementation manners of the electronic device in this embodiment and their various variations. Therefore, the specific implementation of how this electronic device implements the methods in the embodiments of the present invention will not be described in detail here. As long as the devices used by those skilled in the art to implement the methods in the embodiments of the present invention fall within the scope of protection of the present invention.

[0067] In the specific implementation process, when the computer program 311 is executed by the processor, it can implement any implementation manner in the corresponding embodiments of the first aspect.

[0068] Figure 4 The structural schematic diagram of the computer system of the electronic device suitable for implementing the embodiments of the present invention is shown.

[0069] It should be noted that Figure 4 The computer system 400 of the electronic device shown is only an example and should not bring any limitation to the functions and usage scope of the embodiments of the present invention.

[0070] As Figure 4 shown, the computer system 400 includes a central processing unit 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory 402 or the program loaded from the storage part 408 into the random access memory 403, such as executing the methods described in the above embodiments. In the random access memory 403, various programs and data required for system operation are also stored. The central processing unit 401, the read-only memory 402, and the random access memory 403 are connected to each other through a bus 404. The input / output interface 405 is also connected to the bus 404.

[0071] The following components are connected to the input / output interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the input / output interface 405 as required. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 410 as required so that a computer program read therefrom is installed into the storage section 408 as required.

[0072] Specifically, according to an embodiment of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present invention includes a computer program product which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 409, and / or installed from the removable medium 411. When the computer program is executed by the central processing unit 401, various functions defined in the system of the present invention are executed.

[0073] It should be noted that the computer-readable medium shown in the embodiments of the present invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present invention, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0074] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. Among them, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0075] The units involved in the embodiments of the present invention can be implemented in software or in hardware, and the described units can also be provided in a processor. In some cases, the names of these units do not constitute a limitation on the units themselves.

[0076] As another aspect, the present invention also provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes a method for identifying subcarinal lymph node metastasis and prognosis prediction of non-small cell lung cancer based on dual-region radiomics and a deep learning model as described in the above embodiments.

[0077] As another aspect, the present invention also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or may exist alone without being assembled into the electronic device. The above computer-readable medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device realizes a method for identifying subcarinal lymph node metastasis and prognosis prediction of non-small cell lung cancer based on dual-region radiomics and a deep learning model as described in the above embodiments.

[0078] It should be noted that although several modules or units of a device for action execution are mentioned in the above detailed description, such a division is not mandatory. In fact, according to the embodiments of the present invention, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0079] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented in software or in a manner combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (such as a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present invention.

[0080] Other embodiments of the present invention will be readily apparent to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the invention following the general principles of the invention and including known common general knowledge or conventional technical means in the technical field not disclosed by the present invention. It should be understood that the present invention is not limited to the precise structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A method for identifying subcarinal lymph node metastasis and prognostic prediction of non-small cell lung cancer based on dual-region radiomics and deep learning model, characterized in that, Including: Collecting desensitized clinical data and CT images of patients; Extracting regions of interest based on the CT images; Using high-throughput feature extraction technology to extract tumor-SLN radiomics features based on the regions of interest, where the tumor-SLN radiomics features include: intensity features, texture features, shape features, and wavelet transform features; Using a deep neural network to extract deep learning features based on the CT images; Using the min-max normalization method to normalize the tumor-SLN radiomics features and the deep learning features; Constructing multiple tumor-SLN dual-region radiomics models based on the processed tumor-SLN radiomics features; Constructing multiple tumor-SLN dual-region deep learning models based on the processed deep learning features; Validating and evaluating multiple tumor-SLN dual-region radiomics models and multiple tumor-SLN dual-region deep learning models, and screening out an optimal model as the prediction model; Using the prediction model to identify subcarinal lymph node metastasis and prognosis prediction of non-small cell lung cancer.

2. The method according to claim 1, wherein The extracting regions of interest based on the CT images includes: Performing three-dimensional reconstruction of the tumor based on the CT images to obtain preoperative images; Outlining the tumor region and the SLN region from the preoperative images as regions of interest based on professional medical knowledge.

3. The method according to claim 2, characterized in that The using high-throughput feature extraction technology to extract tumor-SLN radiomics features based on the regions of interest includes: Using a standardized volume to unify the sizes of the regions of interest in each group; Using high-throughput feature extraction technology to extract tumor radiomics features and SLN radiomics features from the regions of interest; Fusing the tumor radiomics features and the SLN radiomics features to obtain tumor-SLN radiomics features.

4. The method according to claim 1, characterized in that, The using a deep neural network to extract deep learning features based on the CT images includes: Multiplying the CT images by a gold standard mask, cropping them into voxel blocks, normalizing, and adjusting the size; Inputting the voxel blocks of the CT images after adjusting the size into a 3D VGG network for feature extraction processing, and the process includes: Gradually reducing the resolution of the voxel blocks of the CT images through max pooling; Generating a tumor feature vector and an SLN feature vector through global average pooling; Merging the tumor feature vector and the SLN feature vector through a Concatenate operation, inputting them into a fully connected layer for further fusion, and obtaining deep learning features.

5. The method according to claim 1, characterized in that, The constructing multiple tumor-SLN dual-region radiomics models based on the processed tumor-SLN radiomics features includes: Testing the reliability of each feature according to the intra-class correlation coefficient of the tumor-SLN radiomics features, and screening out features with an intra-class correlation coefficient less than a set threshold; Using a statistical method to calculate the imaging feature significance and the clinical feature significance based on the tumor-SLN radiomics features and the desensitized clinical data, and screening out features that do not meet the significance requirements according to the imaging feature significance and the clinical feature significance; Six algorithms, namely random forest, Boruta, Relief, LASSO, IG, and RFE, are used to screen features again respectively; Based on the finally screened features, five machine learning algorithms, namely support vector machine, K-nearest neighbor algorithm, naive Bayes, random forest, and artificial neural network, are used to construct an artificial intelligence model, and a variety of tumor-SLN dual-region radiomics models are obtained.

6. The method according to claim 4, characterized in that, The multiple tumor-SLN dual-region deep learning models are constructed based on the processed deep learning features, including: Convolutional neural network is used to extract and screen deep learning features related to tumors and SLNs from the deep learning features; The screened deep learning features are fused to obtain tumor-SLN deep learning features; Multiple tumor-SLN dual-region deep learning models are constructed using the tumor-SLN deep learning features.

7. The method according to claim 1, characterized in that, The verification and evaluation of multiple tumor-SLN dual-region radiomics models and multiple tumor-SLN dual-region deep learning models are carried out, and a best model is selected as the prediction model, including: The area under the curve algorithm is used to evaluate the ability of each model in multiple tumor-SLN dual-region radiomics models and multiple tumor-SLN dual-region deep learning models to predict extrapulmonary lymph nodes; The Brier score is used to evaluate the accuracy of each model; Five-fold cross-validation is adopted to perform internal validation on the performance of each model; The Kaplan-Meier survival curve, Log-rank test, and Cox regression risk ratio are used to perform prognostic analysis on each model; According to the clinical applicability of each model, the ability to predict subcarinal lymph nodes, the model accuracy, the model performance, and the results of prognostic analysis, a best model is selected as the prediction model.

8. An apparatus for differentiating subcarinal lymph node metastasis and prognostic prediction of non-small cell lung cancer based on dual-region radiomics and deep learning model, characterized in that, Including: An acquisition unit for acquiring the de-identified clinical data and CT images of patients; A first extraction unit for extracting the region of interest based on the CT images; A second extraction unit for extracting tumor-SLN radiomics features based on the region of interest using high-throughput feature extraction technology, wherein the tumor-SLN radiomics features include: intensity features, texture features, shape features, and wavelet transform features; A third extraction unit for extracting deep learning features based on the CT images using a deep neural network; A processing unit for using the min-max normalization method to normalize the tumor-SLN radiomics features and the deep learning features; A first construction unit for constructing multiple tumor-SLN dual-region radiomics models based on the processed tumor-SLN radiomics features; A second construction unit for constructing multiple tumor-SLN dual-region deep learning models based on the processed deep learning features; A screening unit for verifying and evaluating multiple tumor-SLN dual-region radiomics models and multiple tumor-SLN dual-region deep learning models, and selecting a best model as the prediction model; A prediction unit for using the prediction model to identify subcarinal lymph node metastasis and prognostic prediction of non-small cell lung cancer.

9. A computer-readable storage medium, characterized in that, The storage medium stores computer instructions, which, when running on a computer, cause the computer to execute the method according to any one of claims 1-7.

10. An electronic device, characterized in that, Including: A memory and a processor; The memory is used to store computer instructions; The processor is used to call the computer instructions stored in the memory, so that the electronic device executes the method according to any one of claims 1-7.

Images