Machine learning to predict cancer invasion of lymph nodes

JP2023551335A5Inactive Publication Date: 2024-11-26JOHNSON & JOHNSON ENTERPRISE INNOVATION INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2023533338
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2020-12-01
Filing Date
2021-11-30
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Current methods for determining lung cancer staging, particularly the presence of lung cancer in thoracic lymph nodes, involve invasive procedures like PET scans and biopsies, which are costly, time-consuming, and increase the risk of metastasis due to prolonged diagnosis times.

Method used

A machine learning-based approach using chest CT scans to analyze lymph nodes, employing a risk model trained on radiomics features to predict the likelihood of metastatic cancer, allowing for non-invasive differentiation between high-risk and low-risk patients, thereby streamlining treatment decisions.

Benefits of technology

This method reduces the need for invasive tests in low-risk patients, accelerates treatment initiation, and improves survival rates by minimizing interval metastasis through early and accurate risk assessment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

Disclosed herein are methods for determining subject-level risk of metastatic cancer, comprising training and / or deploying a model to determine 1) an individual's lymph node-level risk of lymph node involvement and / or 2) a subject-level risk of lymph node involvement. Thus, the methods can identify patients at high or low risk of having nodal disease, optionally allowing for guided intervention in cancer patients, e.g., via therapy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims the benefit and priority of U.S. Provisional Patent Application No. 63 / 120,102, filed on December 1, 2020, the entire disclosure of which is incorporated by reference in its entirety for all purposes.

Background Art

[0002] Chest computed tomographic (CT) imaging has become the clinical standard for evaluating people at risk of lung cancer. Such efforts focus on the detection of pulmonary nodules, which may represent the disease at its earliest and most curable stage. The detection of such pulmonary nodules serves as the basis for a more extensive clinical evaluation to determine whether it is cancerous and, if so, whether it has spread to other structures within the chest and even to remote regions of the body. This is done to stage lung cancer and thereby determine the treatment intervention plan.

[0003] One aspect of lung cancer staging is the determination of the presence or absence of lung cancer in chest lymph nodes (nodular state). The current standard of care involves a step - by - step approach that first determines whether the nodule is benign or malignant and then completes additional tests such as positron emission tomography (PET) scans of the lymph nodes and surgical, endoscopic, or bronchoscope - based sampling. Tissue sampling procedures increase the risk, cost, and time of a definitive examination. There is a need for new non - invasive methods to reduce such risks and shorten the time between diagnosis and treatment. As will be described in more detail herein, machine - learning and deep - learning - based techniques applied to initial chest CT scans used for lung cancer screening can address such needs.

[0004] The detection, staging, and treatment planning for lung cancer is a multi-stage process that can take weeks or months. In addition to the risks and expenses of such detailed clinical examinations, the extra time required for this process causes considerable anxiety for patients and increases the likelihood of interstitial metastasis. Indeed, longer time between diagnosis and treatment has been associated with worse overall survival. Exemplary approaches to mitigate these concerns include enabling near-simultaneous diagnosis / staging of cancer, or, in the case of a bronchoscopic approach to therapy, near-simultaneous diagnosis / treatment. [Overview of the Initiative] [Means for solving the problem]

[0005] Embodiments of the present invention disclosed herein are based on the analysis of chest CT scans used to screen for lung cancer in high-risk populations. In some embodiments, patients with high risk of cancer who have metastasized to at least one lymph node (LN) in the chest and therefore require additional tests such as PET scans and lymph node biopsies are identified. This allows clinicians to prioritize testing for patients who most need it. Conversely, patients suspected of having lung cancer who are at low risk of lymph node metastasis can proceed directly to immediate treatment, such as surgical resection or a combination of bronchoscopy for diagnosis and treatment. This eliminates the time between diagnosis and treatment of the disease. Specifically, embodiments of the present invention disclosed herein enable the identification of patients at high or low risk of nodular disease and therefore allow clinicians to either prioritize additional tests for high-risk individuals or allow lung cancer patients with low risk of metastasis to proceed immediately to a confirmatory bronchoscopy biopsy followed immediately by bronchoscopy treatment.

[0006] This specification discloses a method for determining the target-level risk of metastatic cancer in a subject, the method comprising: acquiring one or more images captured from a subject, including multiple lymph nodes in the subject; and predicting the target-level risk of metastatic cancer by applying a risk model to extracted features of the acquired one or more images, the risk model being trained to predict the lymph node (LN) level risk of metastatic cancer for lymph nodes in the images, the risk model being trained using labels derived from the complemented LN level risk of metastatic cancer for at least one pair of lymph nodes in a reference individual, the complemented LN level risk for one pair of lymph nodes being determined using a complementation model that distinguishes between cancerous and non-cancerous lymph nodes in the training images. In various embodiments, predicting the target level risk of metastatic cancer further includes selecting one or more lymph nodes based on the LN level risk of metastatic cancer predicted by a risk model, and determining the target level risk of metastatic cancer using the LN level risk of metastatic cancer predicted for one or more lymph nodes.

[0007] In various embodiments, selecting one or more lymph nodes from a group includes identifying the lymph node with the highest probability LN level risk, and determining the target level risk for metastatic cancer includes assigning the highest probability LN level risk as the target level risk for metastatic cancer. In various embodiments, the risk model predicts the LN level risk for metastatic cancer having a wider probability range than the complemented LN level risk predicted by the complementation model. In various embodiments, a pair of lymph nodes from a reference individual is selected by determining the median risk value and comparing the complemented LN level risk of the reference individual's lymph nodes to the median risk value. In various embodiments, a pair of lymph nodes from a reference individual is further selected by including one or more lymph nodes with an LN level risk higher than the median risk value of the pair of lymph nodes. In various embodiments, one or more lymph nodes with an LN level risk lower than the median risk value of the pair of lymph nodes are excluded from the pair of lymph nodes.

[0008] In various embodiments, the median risk value is the median risk of LN levels within the subject in a reference individual. In various embodiments, one or more images include computed tomography (CT) images. In various embodiments, one or more images are obtained from a chest CT scan. In various embodiments, the method disclosed herein further includes determining the stage of cancer according to the predicted risk level of metastatic cancer in the subject. In various embodiments, the method disclosed herein further includes selecting a diagnosis or treatment for the subject according to the predicted risk level of metastatic cancer in the subject. In various embodiments, selecting a diagnosis or treatment for the subject includes selecting either surgical tumor resection or a combination of bronchoscopy or endoscopic diagnosis and treatment when the predicted risk level of metastatic cancer in the subject is below a threshold risk value. In various embodiments, selecting a diagnosis or treatment for the subject includes selecting an additional diagnostic test from either or both of performing a PET or PET-CT scan or performing a lymph node biopsy when the predicted risk level of metastatic cancer in the subject is above a threshold risk value.

[0009] In various embodiments, the complementary model is trained using training images obtained from chest CT scans. In various embodiments, the training images obtained from chest CT scans include one or more lymph nodes. In various embodiments, the complementary model is trained using training images from the National Lung Screening Trial (NLST). In various embodiments, the complementary model is trained using training images from a custom dataset.

[0010] In various embodiments, the complementary model is trained by generating two or more radiomics panels from at least one training image. In various embodiments, the two or more radiomics panels include the Laplacian of the Gaussian transform of the training image and the wavelet transform of the training image. In various embodiments, the complementary model is further trained by extracting features from each of the two or more radiomics panels and training the complementary model using at least the extracted features. In various embodiments, the complementary model is further trained by defining a region of interest (ROI) within the radiomics panel, wherein the defined region of interest includes lymph nodes, extracting features from the ROI within the radiomics panel, and training the complementary model using at least the extracted features. In various embodiments, the extracted features include one or more feature categories from among primary features, shape features, gray level co-occurrence matrix (GLCM) features, gray level run length matrix (GLRLM) features, gray level size zone matrix (GLSZM) features, and neighborhood gray tone difference matrix (NGTDM) features.

[0011] In various embodiments, the complementary model is further trained using a baseline ground truth value indicating the presence or absence of metastasis in the training images. In various embodiments, the baseline ground truth value is derived from an N-stage covariate value indicating the stage of cancer, where a cancer stage of "0" or "1" indicates no metastasis, and a cancer stage of "2" or "3" indicates the presence of metastasis. In various embodiments, the baseline ground truth value indicating whether an individual lymph node is cancerous or non-cancerous is calculated backward from the N-stage covariate value based on the characteristics of the tumor or lymph node.

[0012] In various embodiments, the complementary model is a random forest classifier or a LASSO classifier. In various embodiments, the risk model is a random forest classifier or a LASSO classifier. In various embodiments, the risk model is trained using supervised learning techniques. In various embodiments, the multiple lymph nodes include 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, or 21 lymph nodes. In various embodiments, the multiple lymph nodes include 8-20 lymph nodes, 10-19 lymph nodes, 12-18 lymph nodes, or 14-16 lymph nodes. In various embodiments, metastatic cancer is metastatic lung cancer. In various embodiments, the risk model exhibits performance of at least AUC=78%, sensitivity=82%, or specificity=60%.

[0013] In addition, this specification discloses a method for training a risk model according to any one of claims 1 to 35, the method comprising: applying a complementary model to generate complementary LN level risks for a plurality of lymph nodes in a training image; selecting a pair of lymph nodes such that the lymph nodes in the pair have an LN level risk higher than the median risk value; and using the LN level risks of the lymph nodes in the pair of lymph nodes as a reference ground truth for training the risk model such that the risk model can predict an LN level risk having a wider probability range than the complementary LN level risks predicted by the complementary model. In various embodiments, the risk model is further trained by extracting features from one or more radiomics panels and training the risk model using at least the extracted features. In various embodiments, extracting features from two or more radiomics panels includes defining a region of interest (ROI) within one or more radiomics panels, wherein the defined region of interest includes lymph nodes, and extracting features from the ROI within one or more radiomics panels. In various embodiments, the extracted features include one or more feature categories from among primary features, shape features, gray-level co-occurrence matrix 1 (GLCM) features, gray-level run-length matrix (GLRLM) features, gray-level size zone matrix (GLSZM) features, and neighboring gray-tone difference matrix (NGTDM) features.

[0014] In addition, this specification discloses a system for performing the methods described above. In various embodiments, a system for determining the target-level risk of metastatic cancer in a subject comprises an imaging device configured to capture images of one or more subjects, and a computing device, wherein the computing device is configured to perform the steps of: acquiring one or more images captured from a subject, including a plurality of lymph nodes in the subject; and predicting the target-level risk of metastatic cancer by applying a risk model to extracted features of the acquired one or more images, wherein the risk model is trained to predict the lymph node (LN) level risk of metastatic cancer for lymph nodes in the images, the risk model is trained using labels derived from the complemented LN level risk of metastatic cancer for at least one pair of lymph nodes in a reference individual, and the complemented LN level risk for one pair of lymph nodes is determined using a complementation model that distinguishes between cancerous and non-cancerous lymph nodes in the training images.

[0015] In addition, this specification discloses non-temporary computer-readable media for performing the methods described above. In various embodiments, the non-temporary computer-readable media, which includes instructions, causes the processor, when the instructions are executed by the processor, to perform the following: to acquire one or more images captured from a subject, including a plurality of lymph nodes of the subject; and to predict the subject-level risk of metastatic cancer by applying a risk model to extracted features of the acquired one or more images, wherein the risk model is trained to predict the lymph node (LN) level risk of metastatic cancer for lymph nodes in the images, the risk model being trained to use labels derived from the complemented LN level risk of metastatic cancer for at least one pair of lymph nodes of a reference individual, the complemented LN level risk for one pair of lymph nodes being determined using a complementation model that distinguishes cancerous and non-cancerous lymph nodes in the training images. [Brief explanation of the drawing]

[0016] These and other features, aspects and advantages of the present invention will be better understood with reference to the following description and accompanying drawings. [Figure 1A] This describes an overview of a system environment for cancer staging by determining the target level of risk for a target metastatic cancer, according to one embodiment. [Figure 1B] This shows a block diagram of a cancer staging system according to one embodiment. [Figure 2A] This represents an exemplary lymph node map for identifying lymph node stations according to one embodiment. [Figure 2B] This is an exemplary lymph node annotation according to one embodiment. [Figure 3A] This represents an exemplary implementation of a complementary model and a risk model according to one embodiment. [Figure 3B] This section further describes the implementation of the risk model according to one embodiment. [Figure 4] Figures 1A, 1B, 2A, 2B, 3A, and 3B illustrate exemplary computers for implementing the entities shown. [Figure 5A] This represents an exemplary correlation heatmap of different lymph node radiomic features. [Figure 5B] This represents an exemplary correlation heatmap of different lymph node radiomic features. [Figure 5C] This represents an exemplary correlation heatmap of different lymph node radiomic features. [Figure 5D] This represents an exemplary correlation heatmap of different lymph node radiomic features. [Figure 5E] This represents an exemplary correlation heatmap of different lymph node radiomic features. [Figure 5F] This represents an exemplary correlation heatmap of different lymph node radiomic features. [Figure 5G] This represents an exemplary correlation heatmap of different lymph node radiomic features. [Figure 5H]Represents an exemplary correlation heatmap of different lymph node radiomics features. [Figure 5I] Represents an exemplary correlation heatmap of different lymph node radiomics features. [Figure 5J] Represents an exemplary correlation heatmap of different lymph node radiomics features. [Figure 5K] Represents an exemplary correlation heatmap of different lymph node radiomics features. [Figure 5L] Represents an exemplary correlation heatmap of different lymph node radiomics features. [Figure 5M] Represents an exemplary correlation heatmap of different lymph node radiomics features. [Figure 5N] Represents an exemplary correlation heatmap of different lymph node radiomics features. [Figure 5O] Represents an exemplary correlation heatmap of different lymph node radiomics features. [Figure 5P] Represents an exemplary correlation heatmap of different lymph node radiomics features. [Figure 5Q] Represents an exemplary correlation heatmap of different lymph node radiomics features. [Figure 5R] Represents an exemplary correlation heatmap of different lymph node radiomics features. [Figure 6] Represents the lymph node level performance of the complementary model. [Figure 7A] Represents the target level performance using the highest probability score predicted by the LN level classifier. [Figure 7B] Represents the target level performance using the highest probability score predicted by the LN level classifier. [Figure 8] Represents the lymph node level performance of the complementary model in a two-step bootstrap approach. [Figure 9A] Represents the target level performance using the bootstrap approach. [Figure 9B] Represents the target level performance using the bootstrap approach.

[0017] Where possible, similar or identical reference numbers may be used in the drawings, and it should be noted that these reference numbers may indicate similar or identical functionality. For example, letters following a reference number, such as "Individual 315A," indicate that the sentence specifically refers to the element having that particular reference number. Reference numbers in a sentence without following letters, such as "Individual 315," refer to any or all of the elements in the drawing to which that reference number is assigned (for example, "Individual 315" in a sentence refers to the reference numbers "Individual 315A," "Individual 315B," "Individual 315C," and "Individual 315D" in the drawing). [Modes for carrying out the invention]

[0018] I. Definition Unless otherwise specified, terms used in the claims and specification are defined as follows:

[0019] The term "subject" includes cells, tissues, or organisms, whether in vivo, ex vivo, or in vitro, male or female, human or non-human.

[0020] The term “mammal” encompasses both humans and non-humans, including, but is not limited to, humans, non-human primates, dogs, cats, mice, cattle, horses, and pigs.

[0021] The term "sample" in "test sample" may include single or multiple cells, or cell fragments, or aliquots of bodily fluids such as blood samples, taken from a subject by means including venipuncture, excretion, ejaculation, massage, biopsy, needle aspiration, lavage, scraping, surgical incision, or intervention, or other means known in the art. Examples of aliquots of bodily fluids include amniotic fluid, aqueous humor, bile, lymph, breast milk, interstitial fluid, blood, plasma, earwax, Cowper's gland fluid (bulborurethral gland fluid), chyle, ointment, Skene's gland fluid (female ejaculate), menstrual blood, mucus, saliva, urine, vomit, tears, vaginal fluid, sweat, serum, semen, sebum, pus, pleural fluid, cerebrospinal fluid, synovial fluid, intracellular fluids, and vitreous fluid.

[0022] The term "reference population" refers to an individual with a known cancer stage. For example, a reference population may include a healthy individual (e.g., a control population). Another example is that a reference population may include an individual previously diagnosed and staged with cancer. Yet another example is that a reference population may include an individual previously diagnosed with cancer that has metastasized to the lymph nodes, e.g., stage IV cancer.

[0023] The phrase "to obtain one or more images" encompasses obtaining one or more images captured from an object, or obtaining one or more images captured from a sample obtained from an object. Obtaining one or more images may also encompass performing the step of capturing one or more images from or from a sample obtained from an object. This phrase may also encompass receiving one or more images, for example, from a third party who has performed the step of capturing one or more images from an object, or from a sample obtained from an object. One or more images may be obtained by those skilled in the art through various known methods, including being stored in a storage device.

[0024] The terms "lymph node invasion," "lymph node metastasis," or "lymph node cancer" refer to the presence or absence of metastatic cancer in one or more lymph nodes.

[0025] The terms "risk of lymph node metastasis," "risk at the lymph node level," and "risk at the LN level" are used interchangeably and generally refer to the risk of metastatic cancer in a specific lymph node.

[0026] The phrases “target-level risk of metastatic cancer” or “patient-level risk of metastatic cancer” are used interchangeably and refer to the likelihood that the target cancer has metastasized. In various embodiments, the target cancer is lung cancer that has metastasized to one or more lymph nodes of the target.

[0027] The term "cancer stage" refers to the stage of the cancer in question. Cancer stage can refer to the TNM staging system, which includes stages 0-4. Additionally, cancer stage can refer to nodular cancer stages 0-3. Nodular cancer stage 0 shows no nodular invasion, nodular cancer stage 1 shows hilar nodular invasion, and nodular cancer stage 2 or 3 shows mediastinal invasion.

[0028] When used herein, it should be noted that the singular forms "a," "an," and "the" include multiple quotations unless the context clearly indicates otherwise.

[0029] II. Overview of the System Environment Figure 1A shows an overview of a system environment 100 for cancer staging by determining the target level risk of a target metastatic cancer in one embodiment. The system environment 100 provides context for introducing a target 110, an imaging device 120, and a cancer staging system 130 for determining a cancer prediction 140. In various embodiments, the cancer prediction 140 determined by the cancer staging system 130 serves as the cancer staging for the target 110. In various embodiments, the cancer prediction 140 determined by the cancer staging system 130 is the predicted risk level for the target 110 and serves as the basis for determining the cancer staging for the target 110.

[0030] In various embodiments, the subjects are healthy. Therefore, the cancer staging methods described herein may be useful for the early detection of cancer in healthy subjects. In various embodiments, the subjects have been previously diagnosed with cancer. In various embodiments, the subjects are healthy but suspected of having a certain form of cancer (e.g., subjects exhibiting symptoms associated with cancer). In certain embodiments, the type of cancer in the subjects is lung cancer. Therefore, the cancer staging methods described herein may be useful for identifying and differentiating subjects at high risk of metastatic cancer, including lymph node invasion, from subjects suspected of having lung cancer but at low risk of lymph node metastasis. Subjects at high risk of metastatic cancer may undergo additional tests such as PET scans and lymph node biopsies, while subjects at low risk of lymph node metastasis may receive immediate treatment such as surgical resection of lung cancer or a combination of bronchoscopy for diagnosis and treatment.

[0031] In various embodiments, the imaging device 120 captures images from the subject 110 for further analysis. In various embodiments, the imaging device 120 captures images from test samples obtained from the subject 110. Images and / or samples may be obtained by the individual or by a third party, such as a healthcare provider or medical professional. Examples of medical professionals include physicians, emergency medical technicians, nurses, first responders, psychologists, phlebotomists, physiotherapists, nurse practitioners, surgeons, dentists, and any other obvious medical professionals known to those skilled in the art.

[0032] In some embodiments, the imaging device 120 captures images of anatomical locations of the subject 110. Exemplary anatomical locations of the subject may include the lungs, thoracic cavity, kidneys, liver, pancreas, brain, stomach, intestines, hip joint, knee, leg, arm, and face. In various embodiments, the imaging device 120 captures images of the thoracic cavity of the subject 110. In various embodiments, the imaging device 120 captures images of one or more lymph nodes of the subject 110. In various embodiments, the imaging device 120 captures images of the thoracic cavity of the subject 110 including one or more lymph nodes.

[0033] In various embodiments, the imaging device 120 is one of the following: computed tomography (CT) scanner, magnetic resonance imaging (MRI) scanner, positron emission tomography (PET) scanner, X-ray scanner, or ultrasound imaging device. In certain embodiments, the imaging device 120 is a CT scanner that captures one or more images of the subject 110. In certain embodiments, the imaging device 120 is a CT scanner that captures one or more CT images of the thoracic cavity including one or more lymph nodes of the subject 110.

[0034] Generally, the cancer staging system 130 analyzes one or more images (e.g., images captured by the imaging device 120) to generate a cancer prediction 140. In various embodiments, the cancer prediction 140 may be a predicted stage of cancer in subject 110. In various embodiments, the cancer prediction 140 may be a predicted likelihood of cancerous lymph node invasion in subject 110. In various embodiments, the cancer prediction 140 may include a recommended clinical approach (e.g., additional tests such as PET scans and lymph node biopsies) or treatment (e.g., surgical resection and / or drug therapy). The cancer staging system 130 may include one or more computers embodied as a computer system 400, as discussed below with respect to Figure 4. Thus, in various embodiments, the steps described with reference to the cancer staging system 130 are performed in silico.

[0035] In various embodiments, the imaging device 120 and the cancer staging system 130 are used by different parties. For example, a first party operates the imaging device 120 to capture one or more images from the subject 110, and then provides the captured one or more images to a second party that implements the cancer staging system 130 to determine a cancer prediction 140.

[0036] Here, we refer to Figure 1B, which shows a block diagram illustrating the computer logic components of a cancer staging system 130 according to one embodiment. Specifically, the cancer staging system 130 may include a lymph node feature module 145, a complement module 150, a risk training module 155, a risk deployment module 160, a cancer prediction module 165, and a training data store 170.

[0037] Generally, the lymph node feature module 145 extracts features from images derived from a subject or training subject. The complement module 150 trains and deploys a complement model. The complement model analyzes the extracted features to determine the lymph node-level risk of cancer for each lymph node in the image. The risk training module 155 trains the risk model using training data that includes at least the lymph node-level risk of cancer determined by the complement model. The risk deployment module 160 analyzes features extracted from images obtained from a subject (e.g., subject 110 in Figure 1A) to implement the risk model to determine cancer predictions, such as the patient-level metastasis risk or cancer stage prediction for subject 110. In other words, the cancer staging system 130 trains and implements at least two models (e.g., a complement model and a risk model). Thus, through this multi-step implementation involving at least two models, the cancer staging system 130 generates cancer predictions, such as the patient-level metastasis risk or cancer stage prediction for a subject.

[0038] The implementation of both the complementary and risk models leads to improved cancer prediction. Specifically, the complementary model learns from training data of N stages at the target level and applies it to extracted lymph node features to define which lymph nodes are likely to be involved in positive N stages. The risk model then learns more refined lymph node metastasis risk probabilities from the complementary data. Here, the risk model predicts the LN level risk of metastatic cancer with a wider range of probabilities than the complementary LN level risk predicted by the complementary model. This allows for a more accurate determination of the target level risk of metastatic cancer.

[0039] The components of the cancer staging system 130 will be described below with reference to two stages: 1) the training stage and 2) the deployment stage. More specifically, the training stage refers to the construction and training of one or more models based on training data, such as training images captured from reference individuals (e.g., individuals with known cancer staging). Thus, during the deployment stage, the models are trained using the training data so that the implementation of the models enables the prediction of the cancer staging of a subject (e.g., subject 110 in Figure 1A). In various embodiments, a complementary module 150 may also be implemented to train and deploy models, hereinafter referred to as a complementary model. Thus, the complementary module 150 trains a complementary model during the training stage and deploys the complementary model during the deployment stage.

[0040] In some embodiments, components of the cancer staging system 130 are applied during one of the training and deployment phases. For example, the risk training module 155 and the training data store 170 are applied during the training phase to train a risk model. Additionally, the risk deployment module 160 is applied during the deployment phase. In various embodiments, components of the cancer staging system 130 can be executed by different parties, depending on whether the components were applied during the training phase or the deployment phase. In such scenarios, the training and deployment of the predictive model are performed by different parties. For example, the risk training module 155 and the training data store 170 applied during the training phase may be used by a first party (e.g., training a risk model), and the risk deployment module 160 used during the deployment phase may be executed by a second party (e.g., deploying a risk model). Similarly, the complementary module 150 can be used by the first party to train a complementary model, and the complementary module 150 can be used by the second party to deploy a complementary model.

[0041] III. Methods for cancer staging Embodiments described herein include methods for cancer staging by analyzing one or more images captured from a subject and applying one or more models to determine the subject level of metastatic cancer risk, including lymph nodes. Such methods can be performed by the cancer staging system 130 shown in Figure 1B.

[0042] Referring to the modules of the cancer staging system 130 in Figure 1B, the lymph node feature module 145 analyzes one or more images to extract features from one or more images. In various embodiments, the lymph node feature module 145 extracts features from training images obtained from a reference individual. As will be described in more detail below, these features extracted from the training images can be used to train one or more models, such as a complementary model. In various embodiments, the lymph node feature module 145 extracts feature images obtained from a subject (e.g., subject 110 in Figure 1A). As will be described in more detail below, these features extracted from images obtained from a subject can be analyzed by deploying a model, such as a risk model.

[0043] In various embodiments, one or more images or training images include one or more lymph nodes within the image. For example, an image or training image may be an image of the pleural cavity including one or more lymph nodes. In various embodiments, an image includes at least two lymph nodes. In various embodiments, an image includes 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, or 21 lymph nodes. In various embodiments, the images include 1 to 10 lymph nodes, 1 to 15 lymph nodes, 2 to 10 lymph nodes, 2 to 15 lymph nodes, 2 to 20 lymph nodes, 4 to 10 lymph nodes, 4 to 15 lymph nodes, 4 to 20 lymph nodes, 5 to 10 lymph nodes, 5 to 15 lymph nodes, 5 to 20 lymph nodes, 8 to 15 lymph nodes, 8 to 20 lymph nodes, 10 to 15 lymph nodes, 10 to 20 lymph nodes, 12 to 15 lymph nodes, 12 to 20 lymph nodes, or 15 to 20 lymph nodes. In various embodiments, multiple images are captured from a single subject or reference individual. The multiple images may capture different lymph nodes of the subject or reference individual. In various embodiments, the multiple images include at least two lymph nodes. In various embodiments, the images include 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, or 21 lymph nodes. In various embodiments, the images include 8 to 20 lymph nodes. In various embodiments, the images include 10 to 19 lymph nodes, 12 to 18 lymph nodes, or 14 to 16 lymph nodes.

[0044] Figure 2A shows an exemplary lymph node map for identifying lymph node stations according to one embodiment. More specifically, Figure 2A is a depiction of the regional lymph node classification extracted from the American Thoracic Society mapping scheme. Here, lymph node stations are annotated to indicate the location and classification of each lymph node. Specifically, the annotations are as follows: (1) Supraclavicular lymph nodes, (2R / 2L) Supraparatracheal lymph nodes, (3A) Prevascular lymph nodes, (3P) Prevertebral lymph nodes, (4R / 4L) Paratracheal lymph nodes, (5) Infra-aortic lymph nodes, (6) Para-aortic lymph nodes, (7) Infrabronchial lymph nodes, (8) Paraesophageal lymph nodes, (9) Mesopulmonary lymph nodes, (10) Hilar lymph nodes, (11) Interlobar lymph nodes, (12) Lobar lymph nodes, (13) Segmental lymph nodes, and (14) Subsegmental lymph nodes.

[0045] The lymph node feature module 145 annotates one or more lymph nodes in the image. In various embodiments, the lymph node feature module 145 annotates one or more lymph nodes by defining a region around the lymph node. In various embodiments, the region around the defined lymph node is a spherical region. In various embodiments, the spherical region has a radius of X mm. In various embodiments, X is 5 mm. In various embodiments, X is 7.5 mm. In some embodiments, X is 0.5 mm, 1 mm, 1.5 mm, 2 mm, 2.5 mm, 3 mm, 3.5 mm, 4 mm, 4.5 mm, 5.5 mm, 6 mm, 6.5 mm, 7 mm, 8 mm, 8.5 mm, 9 mm, 9.5 mm, 10 mm, 10.5 mm, 11 mm, 11.5 mm, 12 mm, 12.5 mm, 13 mm, 13.5 mm, 14 mm, 14.5 mm, or 15 mm. In various embodiments, X is 1 to 15 mm. In various embodiments, X is 2 to 12 mm. In various embodiments, X is 3 to 10 mm. In various embodiments, X is 4 to 8 mm. In various embodiments, X is 5 to 7.5 mm. In various embodiments, X is 4 to 6 mm. In various embodiments, X is 6 to 8 mm. In various embodiments, X is 7 to 8 mm. In various embodiments, X is 10 to 15 mm.

[0046] In various embodiments, the lymph node feature module 145 annotates one or more lymph nodes by defining two or more spherical regions around the lymph node. For example, the lymph node feature module 145 annotates one or more lymph nodes by defining a first spherical region of 5 mm around the lymph node and a second spherical region of 7.5 mm around the lymph node. Figure 2B shows an exemplary lymph node annotation according to one embodiment. Specifically, Figure 2B shows two spherical regions defined around two 2R superior paratracheal lymph nodes.

[0047] In various embodiments, the lymph node feature module 145 extracts features from regions defined around each lymph node. In various embodiments, the lymph node feature module 145 transforms the original input to generate a transformed image, and then extracts additional features from that image. In one embodiment, the lymph node feature module 145 performs a wavelet transform on the original image. In one embodiment, the lymph node feature module 145 performs a Laplacian of Gaussian (LoG) on the original image. Thus, the lymph node feature module 145 can extract features from regions defined around the lymph nodes in the transformed image. In various embodiments, the lymph node feature module 145 extracts features from regions defined around the lymph nodes in both the original image and the transformed image.

[0048] In certain embodiments, the lymph node feature module 145 performs two transformations on the original image (e.g., both a wavelet transform of the original image and a Laplacian (LoG) transformation of the original image). Thus, the lymph node feature module 145 can extract features from regions defined around the lymph nodes in the first and second transformed images. In various embodiments, the lymph node feature module 145 extracts features from regions defined around the lymph nodes in the original image, the first transformed image, and the second transformed image.

[0049] Exemplary categories of features from the original image, the first transformed image, and / or the second transformed image include primary features, shape features, gray-level co-occurrence matrix 1 (GLCM) features, gray-level run-length matrix (GLRLM) features, gray-level size-zone matrix (GLSZM) features, and neighboring gray-tone difference matrix (NGTDM) features. Primary features refer to features of primary statistics of intensity values. GLCM features are derived from a bar graph of co-occurring grayscale values ​​at a given offset on the image. GLRLM features include statistics of the gray-level run-length matrix. GLSZM features include statistics of the gray-level size-zone matrix. NGTDM features include statistics of the neighboring gray-tone difference matrix. Examples of features in different feature categories include correlation features, LMC2, MCC, differential entropy, bound entropy, total entropy, cluster prominence, differential mean, bound mean, total mean, autocorrelation, differential variance, contrast, cluster tendency, total square, LDMN, LDN, cluster shade, LMC1, inverse variance, LD, LDM, bound energy, and maximum probability.

[0050] In various embodiments, the lymph node feature module 145 extracts at least 50 features from the image. In various embodiments, the lymph node feature module 145 extracts at least 100 features, at least 150 features, at least 200 features, at least 250 features, at least 300 features, at least 350 features, at least 400 features, at least 450 features, at least 500 features, at least 550 features, at least 600 features, at least 650 features, at least 700 features, at least 750 features, at least 800 features, at least 850 features, at least 900 features, at least 950 features, at least Extract 1,000 features, at least 1,100 features, at least 1,200 features, at least 1,300 features, at least 1,400 features, at least 1,500 features, at least 1,600 features, at least 1,700 features, at least 1,800 features, at least 1,900 features, at least 2,000 features, at least 3,000 features, at least 4,000 features, at least 5,000 features, at least 6,000 features, at least 7,000 features, at least 8,000 features, at least 9,000 features, or at least 10,000 features. In various embodiments, the lymph node feature module 145 extracts 10 to 100,000 features. In various embodiments, the lymph node feature module 145 extracts 100 to 1,000 features. In various embodiments, the lymph node feature module 145 extracts 300 to 900 features. In various embodiments, the lymph node feature module 145 extracts 500 to 1000 features. In various embodiments, the lymph node feature module 145 extracts 500 to 900 features. In various embodiments, the lymph node feature module 145 extracts 700 to 1000 features. In various embodiments, the lymph node feature module 145 extracts 700 to 900 features. In various embodiments, the lymph node feature module 145 extracts 800 to 1000 features.In various embodiments, the lymph node feature module 145 extracts 800 to 900 features.

[0051] In various embodiments, the lymph node feature module 145 provides the extracted features to the complement module 150 for training or deploying a complement model. For example, if the extracted features are derived from training images captured from a reference individual, the lymph node feature module 145 provides the extracted features to the complement module 150. In various embodiments, the lymph node feature module 145 provides the extracted features to the risk training module 155 for training a risk model. For example, if the extracted features are derived from training images captured from a reference individual, the lymph node feature module 145 provides the extracted features to the risk training module 155 for training a risk model. In various embodiments, the lymph node feature module 145 provides the extracted features to the risk deployment module 160 for deploying a risk model. For example, if the extracted features are derived from images captured from a subject (e.g., subject 110 in Figure 1A), the lymph node feature module 145 provides the extracted features to the risk deployment module 160.

[0052] III.A. Training of Complementary Models The complementary module 150 trains and deploys the complementary model. Generally, the complementary model is configured to analyze features extracted from an image containing one or more lymph nodes to predict the probability that one or more lymph nodes are cancerous. Thus, based on the features extracted from an image containing one or more lymph nodes, the complementary model complements the probability for each lymph node, and the complemented probability indicates the likelihood that the lymph node is cancerous (e.g., metastatic cancer). In other words, the complementary model distinguishes between cancerous and non-cancerous lymph nodes by determining the risk of metastatic cancer at the lymph node level.

[0053] In various embodiments, the complementary model is one of the following: regression models (e.g., linear regression, logistic regression, or polynomial regression), decision trees, random forests, support vector machines, naive Bayes models, k-means clusters, or neural networks (e.g., feedforward networks, convolutional neural networks (CNNs), deep neural networks (DNNs), autoencoder neural networks, generative adversarial networks, or regressive networks (e.g., long short-term memory networks (LSTMs), bidirectional regressive networks, deep bidirectional regressive networks)), or any combination thereof. In certain embodiments, the complementary model is a random forest classifier or a least absolute shrinkage and selection operator (LASSO) classifier. In various embodiments, the complementary model is a machine learning model. In various embodiments, the complementary model is a boosted gradient machine learning model (e.g., xgboost).

[0054] Complementary models can be trained using machine learning implementations such as linear regression algorithms, logistic regression algorithms, decision tree algorithms, support vector machine classification, naive Bayes classification, K-nearest neighbor classification, random forest algorithms, deep learning algorithms, gradient boosting algorithms, and dimensionality reduction analyses such as manifold learning, principal component analysis, factor analysis, autoencoder normalization, and independent component analysis, or any combination thereof. In various embodiments, complementary models are trained using supervised learning algorithms, unsupervised learning algorithms, semi-supervised learning algorithms (e.g., partially supervised), weakly supervised learning, transfer learning, multitask learning, or any combination thereof.

[0055] In various embodiments, the complementary model has one or more parameters, such as hyperparameters or model parameters. Hyperparameters are generally established before training. Examples of hyperparameters include the learning rate, depth, or leaves of a decision tree, the number of hidden layers in a deep neural network, the number of clusters in a k-means cluster, the penalty in a regression model, and regularization parameters associated with a cost function. Model parameters are generally tuned during training. Examples of model parameters include nodes in layers of a neural network, support vectors in a support vector machine, and weights associated with coefficients in a regression model. The model parameters of the complementary model are trained (e.g., tuned) using the training data to improve the predictive power of the complementary model.

[0056] The complementary module 150 trains a complementary model using training data. The training data can be stored in and / or retrieved from the training data store 170. In various embodiments, the training data includes features extracted from training images, including one or more lymph nodes acquired from a reference individual. In various embodiments, the reference individual includes a healthy individual (e.g., a control individual). In various embodiments, the reference individual includes an individual previously diagnosed with cancer. In various embodiments, the reference individual includes an individual previously diagnosed with cancer that has metastasized to lymph nodes. Therefore, images captured from such individuals include one or more cancerous lymph nodes.

[0057] In various embodiments, the training data used to train the complementary model includes a reference ground truth indicating whether a lymph node is cancerous (hereinafter also referred to as "positive" or "+") or non-cancerous (hereinafter also referred to as "negative" or "-"). In various embodiments, the reference ground truth in the training data is a binary value such as "1" or "0". For example, a positive lymph node, which is cancerous, can be identified with training data having a value of "1", while a negative lymph node, which is non-cancerous, can be identified with training data having a value of "0". In short, the complementary module 150 trains the complementary model to minimize a loss function using the training data so that the complementary model can better predict outcomes (e.g., the probability of lymph node metastasis) based on the input (e.g., extracted features of the lymph node). In various embodiments, the loss function is constructed for either Lasso (least absolute shrinkage and selection operator, LASSO) regression, Ridge regression, or ElasticNet regression.

[0058] In various embodiments, training data can be obtained and / or derived from publicly available databases. For example, training data can be obtained and / or derived from the National Lung Screening Trial (NLST) and include nodular stage (N-stage) cancer information of reference individuals. In some embodiments, training data can be obtained and collected independently from publicly available databases, for example, by capturing images of one or more lymph nodes from multiple reference individuals. For example, such training data may be a custom dataset. Training data for a custom dataset can be obtained during patient care and therefore may have corresponding surgical sampling information for lymph nodes. Training data may include N-stage cancer information of reference individuals, for example, N-stage information previously diagnosed by a clinician based on imaging and / or biopsy results of reference individuals.

[0059] In various embodiments, the reference ground truth indicating whether a lymph node is cancerous is nodule stage (N-stage) information derived from clinical stage data. For example, publicly available NLST datasets include clinical stage information for reference individuals (e.g., stages 1a, 1b, 2a, 2b, 3a, etc.) but do not include information specific to lymph nodes. In other words, NLST datasets do not distinguish whether individual lymph nodes are cancerous or non-cancerous. In such embodiments, the reference ground truth indicating whether a lymph node is cancerous can be calculated backward based on the N-stage information and other characteristics of the cancer in the reference individual (e.g., tumor size, tumor location, tumor shape, and / or tumor density) and / or the N-stage information and other characteristics of the lymph node (e.g., lymph node size, lymph node density, and / or lymph node shape). As a first example, a reference individual with an N stage of 0 or 1 has not yet encountered cancer metastasis, and therefore, the lymph nodes of this reference individual are assigned a “negative” or non-cancerous reference ground truth. As another example, a reference individual with an N stage of 2 or 3 has encountered cancer metastasis. Therefore, based on the characteristics of the tumor and / or lymph nodes, the lymph nodes are assigned a “positive” or “negative” reference ground truth.

[0060] III.B. Deployment of Complementary Models The complementary module 150 deploys a trained complementary model to analyze features extracted from images containing one or more lymph nodes and predict the probability that one or more lymph nodes are cancerous. In various embodiments, the trained complementary model is deployed against a test dataset. In various embodiments, the test and training datasets used to train the complementary model may be derived from a common dataset. For example, the common dataset may be split into two subsets for training and testing the complementary model. In some embodiments, the common dataset undergoes a 50:50 training:test dataset split. In some embodiments, the common dataset undergoes a 60:40 training:test dataset split. In some embodiments, the common dataset undergoes an 80:20 training:test dataset split.

[0061] The deployment of the trained complementary model yields probabilities for lymph nodes (e.g., a sequence of probabilities from 0 to 1), where each probability indicates whether the corresponding lymph node is likely to be cancerous. In various embodiments, the complementary module 150 provides one or more lymph node metastasis risks for lymph nodes predicted by the complementary model used to train the risk model. In various embodiments, the complementary module 150 provides all lymph node metastasis risks for lymph nodes predicted by the complementary model used to train the risk model. In various embodiments, the complementary module 150 performs a selection process to identify a subset of lymph node metastasis risks for lymph nodes predicted by the complementary model. Thus, the risk model is subsequently trained using the subset of lymph node metastasis risks.

[0062] In some embodiments, the complementary module 150 selects a subset of lymph node metastasis risk by comparing the lymph node metastasis risk to a threshold. If the lymph node metastasis risk of a lymph node exceeds the threshold, the lymph node metastasis risk of that lymph node is selected to be included in the subset for training the risk model. If the lymph node metastasis risk of a lymph node is below the threshold, the lymph node metastasis risk of that lymph node is excluded from the subset and therefore not used to train the risk model.

[0063] In various embodiments, the threshold is a constant value. In various embodiments, the threshold is a 50% risk of metastatic cancer. In various embodiments, the threshold is a 30%, 35%, 40%, 45%, 55%, 60%, 65%, 70%, 75%, 80%, 85%, 90%, 95%, or 99% risk of metastatic cancer. In some embodiments, the threshold is 30–70%. In some embodiments, the threshold is 40–60%. In some embodiments, the threshold is 45–55%. In some embodiments, the threshold is 48–52%. In some embodiments, the threshold is 49–51%. In various embodiments, the threshold is variable. In various embodiments, the threshold is the median probability of having cancer within the subject.

[0064] Complementary module 150 provides a subset of lymph node metastasis risk to risk training module 155 for training the risk model.

[0065] III.C. Training of Risk Models The risk training module 155 trains the risk model using training data that includes at least one or more lymph node metastasis risks of lymph nodes predicted by the complementary model. Here, one or more lymph node metastasis risks may be a subset of lymph node metastasis risks selected by the complementary module 150, as described above. The subset of lymph node metastasis risks for training the risk model represents a refined training dataset compared to the training dataset used to train the complementary model. Specifically, the refined training dataset includes a subset of lymph nodes that are more likely to be cancerous, while lymph nodes that are less likely to be cancerous (but which may otherwise represent false positives or be confused with cancerous lymph nodes for training the risk model) are removed from the refined training dataset. Thus, by training the risk model with this refined training dataset, the risk model predicts a wider range of probabilities for all lymph nodes in a subject that have metastatic disease. More specifically, risk models are more likely to assign lymph nodes that are more likely to have cancer with a higher risk of lymph node metastasis, and lymph nodes that are less likely to have cancer with a lower risk of lymph node metastasis.

[0066] In various embodiments, the risk model analyzes features extracted from an image containing one or more lymph nodes to predict the consecutive probability that each of the one or more lymph nodes is cancerous. Thus, based on the features extracted from an image containing one or more lymph nodes, the risk model determines a probability for each lymph node, and the determined probability indicates the likelihood that the lymph node is cancerous (e.g., metastatic cancer). Here, the risk model determines the risk of metastatic cancer at the lymph node level. In various embodiments, the risk model is similar to the complementary model in that both models analyze features extracted from an image to predict the risk of metastatic cancer at the lymph node level. However, the risk model is trained using the predictions of the complementary model, e.g., the risks at one or more lymph node levels predicted by the complementary model. In short, this allows the risk model to predict the LN level risk of metastatic cancer, which has a wider range of probabilities than the complementary LN level risk predicted by the complementary model.

[0067] The risk model is trained using training data that includes at least one or more lymph node metastasis risks predicted by the complementary model. Here, one or more lymph node metastasis risks can serve as a reference ground truth for training the risk model. In various embodiments, the training data may further include features extracted from training images that include one or more lymph nodes obtained from a reference individual. In some embodiments, the training images used to train the risk model may be the same training images previously used to train the complementary model. In some embodiments, the training images used to train the risk model are different from the training images previously used to train the complementary model.

[0068] In various embodiments, the training data includes individual training examples so that a risk model is trained using each training example. For example, each training example may include extracted features of one or more lymph nodes from a specific training image and a baseline ground truth (e.g., the lymph node metastasis risk of the same one or more lymph nodes in a specific training image). Thus, for each training example, the risk model is trained to minimize a loss function (e.g., a loss function for Least Absolute Contraction and Selection (LASSO) regression, ridge regression, or ElasticNet regression).

[0069] In various embodiments, the risk model is one of the following: regression models (e.g., linear regression, logistic regression, or polynomial regression), decision trees, random forests, support vector machines, naive Bayes models, k-means clusters, or neural networks (e.g., feedforward networks, convolutional neural networks (CNNs), deep neural networks (DNNs), autoencoder neural networks, generative adversarial networks, or regressive networks (e.g., long-short-term memory networks (LSTMs), bidirectional regressive networks, deep bidirectional regressive networks), or any combination thereof. In certain embodiments, the risk model is a random forest classifier or a least absolute contraction and selection (LASSO) classifier. In various embodiments, the risk model is a machine learning model.

[0070] Risk models can be trained using machine learning implementations such as linear regression algorithms, logistic regression algorithms, decision tree algorithms, support vector machine classification, naive Bayes classification, K-nearest neighbor classification, random forest algorithms, deep learning algorithms, gradient boosting algorithms, and dimensionality reduction analyses such as manifold learning, principal component analysis, factor analysis, autoencoder normalization, and independent component analysis, or any combination thereof. In various embodiments, risk models are trained using supervised learning algorithms, unsupervised learning algorithms, semi-supervised learning algorithms (e.g., partially supervised), weakly supervised learning, transfer learning, multitask learning, or any combination thereof.

[0071] In various embodiments, a risk model has one or more parameters, such as hyperparameters or model parameters. Hyperparameters are generally established before training. Examples of hyperparameters include the learning rate, depth, or leaves of a decision tree, the number of hidden layers in a deep neural network, the number of clusters in a k-means cluster, the penalty in a regression model, and regularization parameters associated with a cost function. Model parameters are generally adjusted during training. Examples of model parameters include nodes in layers of a neural network, support vectors in a support vector machine, and weights associated with coefficients in a regression model. The model parameters of a risk model are trained (e.g., adjusted) using training data to improve the predictive ability of the risk model.

[0072] Deployment of the Bit.C Risk Model The risk deployment module 160 deploys a risk model to determine the target-level risk of metastatic cancer in a subject (e.g., subject 110 described above in Figure 1A). Here, the risk deployment module 160 provides extracted features from an image containing one or more lymph nodes captured from the subject as input to the trained risk model. The trained risk model analyzes the extracted features and outputs a lymph node probability indicating whether the individual lymph nodes are likely to be cancerous.

[0073] In various embodiments, following the deployment of the risk model, the risk deployment module 160 transitions from the risk of metastatic cancer at individual lymph node levels predicted by the risk model to the target level risk of the target metastatic cancer. In various embodiments, the risk deployment module 160 combines the risks of metastatic cancer at individual lymph node levels to determine the target level risk of the metastatic cancer. In various embodiments, the risk deployment module 160 performs statistical calculations across the overall risks of metastatic cancer at lymph node levels. The statistical calculations can identify the maximum, minimum, mean, median, or mode value across the overall risks of metastatic cancer at individual lymph node levels. In certain embodiments, the statistical calculation is the maximum value across the overall risks of metastatic cancer at individual lymph node levels. Therefore, the lymph node with the highest risk of metastatic disease at the lymph node level is assigned as the target level risk of the target metastatic disease.

[0074] The cancer staging module 165 determines the target cancer stage based on the target level risk of the target metastatic disease. In various embodiments, the cancer staging module 165 determines the target cancer stage by comparing the target level risk of the target metastatic disease with one or more threshold risk values. These threshold risk values ​​can serve as cutoffs to determine whether the target cancer is likely to be stage 0, stage 1, stage 2, or stage 3.

[0075] In some embodiments, if the risk level of the subject exceeds 30%, the subject is categorized as having at least stage 3 cancer. In some embodiments, if the risk level of the subject exceeds 40%, 50%, 60%, 70%, 80%, or 90%, the subject is categorized as having at least stage 3 cancer.

[0076] In various embodiments, the threshold risk value defines a risk range corresponding to one or more cancer stages. Thus, the threshold risk value sets upper and lower limits to the risk range. In some embodiments, if the risk level of the subject is 30% to 100%, the subject is categorized as having nodular stage 3 cancer. In some embodiments, if the risk level of the subject is 40% to 100%, the subject is categorized as having nodular stage 3 cancer. In some embodiments, if the risk level of the subject is 50% to 100%, the subject is categorized as having nodular stage 3 cancer.

[0077] In some embodiments, if the risk level of the subject is between 10% and 40%, the subject is categorized as having nodular stage 2 cancer. In some embodiments, if the risk level of the subject is between 15% and 40%, the subject is categorized as having nodular stage 2 cancer. In some embodiments, if the risk level of the subject is between 20% and 40%, the subject is categorized as having nodular stage 2 cancer. In some embodiments, if the risk level of the subject is between 30% and 40%, the subject is categorized as having nodular stage 2 cancer. In some embodiments, if the risk level of the subject is between 15% and 30%, the subject is categorized as having nodular stage 2 cancer. In some embodiments, if the risk level of the subject is between 20% and 30%, the subject is categorized as having nodular stage 2 cancer.

[0078] In some embodiments, if the risk level of the subject is between 5% and 20%, the subject is categorized as having nodular stage 0 or nodular stage 1 cancer. In some embodiments, if the risk level of the subject is between 10% and 20%, the subject is categorized as having nodular stage 0 or nodular stage 1 cancer. In some embodiments, if the risk level of the subject is between 5% and 15%, the subject is categorized as having nodular stage 0 or nodular stage 1 cancer. In some embodiments, if the risk level of the subject is between 8% and 12%, the subject is categorized as having nodular stage 0 or nodular stage 1 cancer.

[0079] IV. Exemplary methods of cancer staging Figure 3A shows an exemplary implementation of the complementary and risk models according to one embodiment. Here, reference individuals 305A and 305B can be identified by a specific N stage. For example, reference individual 305A is identified as N stage negative, indicating that reference individual 305A does not exhibit lymph node cancer metastasis. Reference individual 305B is identified as N stage positive, indicating that reference individual 305B exhibits lymph node cancer metastasis. However, the control level N stage does not distinguish between specific lymph nodes, or between individual lymph nodes being cancerous or non-cancerous.

[0080] The complementary model 310 complements the metastatic risk of each lymph node (e.g., lymph node metastasis risk), thereby enabling differentiation between lymph nodes with metastatic cancer and those that are not cancerous. In various embodiments, the complementary model 310 predicts the probability of each lymph node being cancerous.

[0081] As shown in Figure 3A, the output of the complementary model 310 can identify that individual 305C (corresponding to reference individual 305A) does not exhibit lymph node invasion. In other words, none of individual 305C's lymph nodes are likely to be cancerous. This confirms the identification of N-stage negativity. Additionally, the output of the complementary model 310 can identify that individual 305D (corresponding to reference individual 305B) exhibits lymph node invasion in one or more lymph nodes. For example, the complementary model 310 can complement the probabilities such that exactly one lymph node of individual 305D is cancerous, while the other lymph nodes of individual 305D are not cancerous.

[0082] In various embodiments, the risk model 320 is trained using one or more of the lymph node metastasis risks predicted by the complementary model 310. In various embodiments, the risk model 320 is trained using all of the lymph node metastasis risks predicted by the complementary model 310. In various embodiments, the risk model 320 is trained using a subset of lymph node metastasis risks corresponding to a subset of lymph nodes, thereby enabling the risk model to predict a wider range of probabilities for all lymph nodes in a subject with metastatic disease.

[0083] Once trained, the risk model 320 is deployed to predict the target-level risk of metastatic cancer for one or more subjects 315. Thus, the risk model 320 is deployed to transition from the lymph node-level risk of metastatic disease to the target-level risk of metastatic disease. As shown in Figure 3A, the risk model 320 is deployed to analyze features extracted from images captured from individuals 315. The risk model 320 predicts the target-level risk of metastatic cancer in the region of individual 315. For example, the risk model 320 predicts a 20% risk of metastatic cancer for individual 315A, a 30% risk of metastatic cancer for individual 315B, a 50% risk of metastatic cancer for individual 315C, and a 60% risk of metastatic cancer for individual 315D. Therefore, individuals 315A and 315B can be categorized as early-stage cancer (e.g., stage 0 or 1), while individuals 315C and 315D can be categorized as late-stage cancer (e.g., stage 2, 3, or 4).

[0084] Individual 315 can receive different treatments according to their target level of risk for metastatic disease. For example, individuals 315A and 315B, who have a low risk of lymph node metastasis, can proceed directly to immediate treatment, such as surgical resection or a combination of bronchoscopy for diagnosis and treatment. As another example, individuals 315C and 315D, who have a higher risk of lymph node metastasis, can undergo additional tests such as PET scans and lymph node biopsies.

[0085] Figure 3B shows in more detail an implementation of the risk model 320 according to one embodiment. Here, the risk model 320 analyzes features extracted from images captured from the subject 350 and outputs the probability of each of one or more lymph nodes in the subject. As shown in Figure 3B, the risk model 320 outputs the probabilities of three lymph nodes. In other embodiments, the risk model 320 can output the probabilities of additional or fewer lymph nodes in the subject 350.

[0086] As shown in Figure 3B, risk model 320 predicts that the first lymph node has a 20% chance of being cancerous and an 80% chance of being non-cancerous. Additionally, risk model 320 predicts that the second lymph node has a 30% chance of being cancerous and a 70% chance of being non-cancerous. Additionally, risk model 320 predicts that the third lymph node has a 60% chance of being cancerous and a 40% chance of being non-cancerous. The probabilities of individual lymph nodes are analyzed to identify the target level risk for metastatic cancer. In this example, the highest probability of metastatic cancer among the lymph nodes is used as the target level probability for metastatic cancer. The third lymph node, with a 60% chance of being cancerous, has the highest probability among the lymph nodes. Therefore, the target level probability for metastatic cancer in target 350 is 60%.

[0087] V. Cancer The methods described herein include cancer staging by determining the target level of risk for the target cancer. In various embodiments, the target cancers may include one or more of the following: lymphoma, B-cell lymphoma, T-cell lymphoma, mycosis fungoides, Hodgkin's disease, myeloid leukemia, bladder cancer, brain tumor, nervous system cancer, head and neck cancer, head and neck squamous cell carcinoma, kidney cancer, lung cancer, neuroblastoma / glioblastoma, ovarian cancer, pancreatic cancer, prostate cancer, skin cancer, liver cancer, melanoma, squamous cell carcinoma of the oral cavity, throat, larynx, and lung, colon cancer, cervical cancer, breast cancer, epithelial cancer, gastrointestinal cancer, kidney cancer, genitourinary cancer, lung cancer, esophageal cancer, stomach cancer, thyroid cancer, head and neck cancer, colorectal cancer, hematopoietic cancer, testicular cancer, colon cancer and / or rectal cancer, uterine cancer, or prostate cancer. Lung bronchioloalveolar carcinoma (BAC), bladder cancer, female reproductive tract malignancies (e.g., serous uterine carcinoma, endometrial cancer, vulvar squamous cell carcinoma, and uterine sarcoma), ovarian surface epithelial carcinoma (e.g., clear cell carcinoma of the ovary, epithelial ovarian cancer, fallopian tube cancer, primary peritoneal cancer), breast cancer, non-small cell lung cancer (NSCLC), male reproductive tract malignancies (e.g., testicular cancer), retroperitoneal or peritoneal cancer, gastroesophageal adenocarcinoma, gastroesophageal junction cancer, hepatocellular carcinoma of the liver, esophageal and gastroesophageal junction cancer, cervical cancer, bile duct cancer, pancreatic adenocarcinoma, extrahepatic bile duct adenocarcinoma, small intestine malignancies, gastric adenocarcinoma, cancer of unknown primary origin. The cancer may be any one of the following: primary (CUP), colorectal adenocarcinoma, esophageal cancer, prostate adenocarcinoma, kidney cancer, head and neck squamous cell carcinoma, thymic cancer, non-melanoma skin cancer, thyroid cancer (e.g., papillary carcinoma), head and neck cancer, anal cancer, non-epithelial ovarian cancer (non-EOC), uveal melanoma, malignant pleural mesothelioma, small cell lung cancer (SCLC), central nervous system cancer, neuroendocrine tumors, and soft tissue tumors. In certain embodiments, the cancers are breast cancer, non-small cell lung cancer, bladder cancer, kidney cancer, colorectal cancer, and melanoma.In some embodiments, the cancer in the subject may be a metastatic cancer, including one of bladder cancer, breast cancer, colon cancer, kidney cancer, lung cancer, melanoma, ovarian cancer, pancreatic cancer, prostate cancer, rectal cancer, stomach cancer, thyroid cancer, or uterine cancer. In certain embodiments, the cancer is lung cancer. In certain embodiments, the cancer is a certain type of lung cancer, including one of small cell lung cancer, non-small cell lung cancer, adenocarcinoma, squamous cell carcinoma, large cell carcinoma, small cell carcinoma, combined small cell carcinoma, pulmonary sarcoma, pulmonary lymphoma, or bronchial carcinoid.

[0088] VI. Guided intervention Embodiments described herein include determining cancer staging by determining the target level risk of metastatic cancer in the subject. In various embodiments, interventions are provided to the subject based on the predicted cancer staging and / or target level risk of metastatic cancer for the subject. Such interventions are hereafter referred to as induced interventions and may be any one or any combination of the following: application of diagnosis, application of therapeutic agent, subsequent testing of the subject, subsequent biopsy (e.g., cancer biopsy or lymph node biopsy, bronchoscopic sampling of lymph nodes), subsequent imaging scan (e.g., PET scan, MRI scan, ultrasound imaging, or X-ray imaging), tumor resection, bronchoscopic diagnosis, selection and / or administration of therapeutic substance, selection / administration of pharmaceutical composition, registration in a clinical trial, non-registration in a clinical trial.

[0089] In various embodiments, the predicted target level risk of metastatic cancer for a subject, or the predicted cancer stage, indicates that the subject is at high risk of metastatic cancer. In various embodiments, subjects at high risk of metastatic cancer have a predicted target level risk of metastatic cancer that exceeds a threshold score. In various embodiments, the threshold score is a metastatic cancer risk greater than 20%. In various embodiments, the threshold score is a metastatic cancer risk greater than 25%. In various embodiments, the threshold score is a metastatic cancer risk greater than 30%, greater than 35%, greater than 40%, greater than 45%, greater than 50%, greater than 55%, greater than 60%, greater than 65%, greater than 70%, greater than 75%, greater than 80%, greater than 85%, greater than 9%, greater than 95%, or greater than 99%. In various embodiments, the threshold score is between 20% and 80%. In some embodiments, the threshold is between 30% and 70%. In some embodiments, the threshold is between 40% and 60%. In some embodiments, the threshold is between 45% and 55%. In some embodiments, the threshold is 20-60%. In some embodiments, the threshold is 30-55%. In some embodiments, the threshold is 35-50%. In some embodiments, the threshold is 40-45%.

[0090] In various embodiments, subjects are at high risk of metastatic cancer if their predicted cancer stage is at least stage 2 (e.g., stage 2 or stage 3). In such embodiments, the induced intervention includes one of the following: a subsequent trial of the subject, a subsequent biopsy of the subject, or a subsequent imaging scan of the subject. This allows the clinical provider to prioritize the subsequent trial, biopsy, or imaging scan of the subject that poses the highest risk.

[0091] In various embodiments, if a patient is predicted to be at high risk of metastatic cancer, treatments unsuitable for late-stage metastatic cancer can be excluded from the patient's treatment. Examples of such treatments to be excluded include tumor resection, lung resection, radiation (e.g., external beam radiation), chemotherapy (e.g., local or systemic chemotherapy), immunomodulatory therapy, and general anesthesia.

[0092] In certain embodiments, if a subject is predicted to be at high risk of metastatic cancer, the subject undergoes a subsequent imaging scan to confirm the prediction. For example, if a subject is predicted to be at high risk of metastatic cancer, the subject undergoes a further PET scan. In such embodiments, if the PET scan returns a negative result (e.g., no lymph node invasion), the subject may undergo further testing. For example, the subject may undergo a subsequent biopsy to definitively sample lymph nodes. In some embodiments, the subsequent biopsy is a bronchoscopic sampling of lymph nodes.

[0093] In certain embodiments, if a subject is predicted to be at high risk of metastatic cancer, the subject undergoes a subsequent biopsy directly to confirm the prediction. For example, the subject does not need to undergo a subsequent imaging scan to confirm the prediction; instead, the subject undergoes a subsequent biopsy directly. For example, if a subject is predicted to be at high risk of metastatic cancer, the subject undergoes further bronchoscopy sampling of lymph nodes.

[0094] As described herein, the risk level of metastatic cancer at a target level can be determined from the risk at the individual lymph node level across the entire lymph node system. Therefore, for a target predicted to be at high risk of metastatic cancer based on the risk level, specific lymph nodes associated with the high-risk target level can be identified. Thus, specific lymph nodes can be targeted for subsequent testing or treatment. In certain embodiments, specific lymph nodes can undergo subsequent imaging scans (e.g., PET scans) to confirm the prediction. In certain embodiments, specific lymph nodes can undergo subsequent biopsies to definitively sample the specific lymph node. Thus, the subsequent biopsy can confirm the presence or absence of metastatic cancer in the specific lymph node.

[0095] In various embodiments, if a subject is predicted to be at high risk of metastatic cancer, the subject may be enrolled in a clinical trial. For example, such a clinical trial may test the therapeutic effect of a treatment for metastatic cancer and, therefore, its eligibility criteria may identify that the registrant has metastatic cancer. In various embodiments, if a subject is predicted to be at high risk of metastatic cancer, the subject's enrollment in a clinical trial is withheld. For example, such a clinical trial may test the therapeutic effect of a treatment for non-metastatic forms of cancer.

[0096] In various embodiments, the predicted target level risk of metastatic cancer for a subject indicates that the subject has a low risk of metastatic cancer. In various embodiments, a subject with a low risk of metastatic cancer has a predicted target level risk that is lower than the threshold score. In various embodiments, the threshold score is a metastatic cancer risk lower than 20%. In various embodiments, the threshold score is a metastatic cancer risk lower than 25%. In various embodiments, the threshold score is a metastatic cancer risk lower than 30%, lower than 35%, lower than 40%, lower than 45%, lower than 50%, lower than 55%, lower than 60%, lower than 65%, lower than 70%, lower than 75%, lower than 80%, lower than 85%, lower than 90%, lower than 95%, or lower than 99%. In some embodiments, the threshold is 5% to 40%. In some embodiments, the threshold is 10% to 30%. In some embodiments, the threshold is 15% to 25%. In some embodiments, the threshold is 18–22%. In various embodiments, subjects have a low risk of metastatic cancer if their predicted cancer stage is lower than stage 2 (e.g., stage 0 or stage 1). In such embodiments, the induced intervention includes one of the following: tumor resection, bronchoscopic diagnosis, selection and / or administration of a therapeutic substance, and / or selection and / or administration of a pharmaceutical composition. This allows for the rapid identification and treatment of subjects in whom metastasis has not yet occurred.

[0097] In various embodiments, therapeutic agents can be selected and / or administered to subjects based on the subject's risk level of metastatic cancer, and the selected therapeutic agent is likely to be effective against the cancer. Exemplary therapeutic agents include chemotherapy, radiation, antigen-specific monoclonal antibodies, or immunotherapy. In various embodiments, therapeutic agents are provided to subjects at high risk of metastatic cancer. In various embodiments, subjects at high risk of metastatic cancer have a predicted subject level of risk of metastatic cancer that exceeds a threshold score. In various embodiments, the threshold score is a risk of metastatic cancer greater than 20%. In various embodiments, the threshold score is a risk of metastatic cancer greater than 25%. In various embodiments, the threshold score is a risk of metastatic cancer greater than 30%, greater than 35%, greater than 40%, greater than 45%, greater than 50%, greater than 55%, greater than 60%, greater than 65%, greater than 70%, greater than 75%, greater than 80%, greater than 85%, greater than 90%, greater than 95%, or greater than 99%. In various embodiments, subjects are at high risk of metastatic cancer if their predicted cancer stage is at least stage 2 (e.g., stage 2, stage 3, or stage 4).

[0098] In various embodiments, therapeutic agents are provided to subjects with a low risk of metastatic cancer. In various embodiments, subjects with a low risk of metastatic cancer have a predicted subject level of risk that is lower than a threshold score. In various embodiments, the threshold score is a metastatic cancer risk of less than 20%. In various embodiments, the threshold score is a metastatic cancer risk of less than 25%. In various embodiments, the threshold score is a metastatic cancer risk of less than 30%, less than 35%, less than 40%, less than 45%, less than 50%, less than 55%, less than 60%, less than 65%, less than 70%, less than 75%, less than 80%, less than 85%, less than 90%, less than 95%, or less than 99%. In various embodiments, a subject has a low risk of metastatic cancer if the subject's predicted cancer stage is lower than stage 2 (e.g., stage 0 or stage 1).

[0099] In various embodiments, if a subject is predicted to have a low risk of metastatic cancer, the subject may receive early-stage cancer treatment. Examples of early-stage cancer treatment include tumor resection, lung resection, chemotherapy (e.g., local or systemic chemotherapy), locally delivered therapies (e.g., transbronchial infusion or catheter-based ablation), or radiation.

[0100] In certain embodiments, if a subject is predicted to have a low risk of metastatic cancer, the subject undergoes a subsequent imaging scan to confirm the prediction. For example, if a subject is predicted to have a low risk of metastatic cancer, the subject undergoes a further PET scan. In such embodiments, if the PET scan returns a negative result (e.g., no lymph node invasion), the PET scan result is consistent with the prediction. However, if the PET scan returns a positive result (e.g., lymph node invasion), the subject may undergo further testing. For example, the subject may undergo a subsequent biopsy to definitively sample lymph nodes. In some embodiments, the subsequent biopsy is a bronchoscopic sampling of lymph nodes.

[0101] In various embodiments, therapeutic agents are biological, such as cytokines, antibodies, soluble cytokine receptors, antisense oligonucleotides, siRNAs, and the like. Such biological agents include mutant proteins and derivatives of biological agents, derivatives of which include, for example, fusion proteins, PEGylated derivatives, cholesterol-coupled derivatives, and the like, known in the art. Antagonists of cytokines and cytokine receptors, such as traps and monoclonal antagonists, are also included. Biosimilar pharmaceuticals of the activators described herein or drugs that are bioequivalent to the activators are also included.

[0102] Lung cancer treatments may include chemotherapy drugs such as docetaxel, cisplatin, carboplatin, gemcitabine, nab-paclitaxel, paclitaxel, pemetrexed, gefitinib, erlotinib, britinib (Alunbrig®), capmatinib (Tabrecta®), serpacatinib (Retevmo®), entrectinib (Rozlytrek®), lorlatinib (Lorbrena®), lalotrectinib (Vitrakvi®), dacomitinib (Vizimpro®), and vinorelbine. Treatments for lung cancer may include antibody therapies such as durvalumab (Imfinzi®), nivolumab (Opdivo®), pembrolizumab (Keytruda®), atezolizumab (Tecentriq®), and ramucirumab.

[0103] In various embodiments, pharmaceutical compositions may be selected and / or administered to a subject based on the risk level of the subject with metastatic cancer, and the selected therapeutic agent is likely to exhibit efficacy against the cancer. The pharmaceutical composition administered to an individual contains an active agent such as the therapeutic agent described above. The active ingredient is present in a therapeutically effective amount, i.e., a sufficient amount when administered to treat the disease or medical condition it mediates. The composition may also contain various other agents to enhance delivery and efficacy in order to enhance the delivery and stability of the active ingredient. Thus, for example, the composition may also contain a pharmaceutically acceptable non-toxic carrier or diluent, depending on the desired formulation, which is defined as a vehicle commonly used to formulate pharmaceutical compositions for animal or human administration. The diluent is selected so as not to affect the biological activity of the combination. Examples of such diluents are distilled water, buffer water, physiological saline, PBS, Ringer's solution, dextrose solution, and Hanks' solution. In addition, the pharmaceutical composition or formulation may contain other carriers, adjuvants, or non-toxic, non-therapeutic, non-immunogenic stabilizers, excipients, etc. The composition may also contain additional substances to approximate physiological conditions, such as pH adjusters and buffers, toxicity modifiers, wetting agents, and detergents. The composition may also contain any of a variety of stabilizers, such as antioxidants.

[0104] The pharmaceutical compositions described herein can be administered in a variety of different ways. Examples include administering compositions containing a pharmaceutically acceptable carrier via oral, intranasal, rectal, topical, intraperitoneal, intravenous, intramuscular, subcutaneous, subdermal, percutaneous, intrathecal, or intracranial methods. In various embodiments, the pharmaceutical compositions can be administered via catheter-based deployments (e.g., using flexible or rigid robot-based bronchoscopes).

[0105] Such pharmaceutical compositions may be administered for therapeutic purposes (for example, after a patient has been diagnosed with lung cancer). In the context of the present invention, prevention, or cessation of a disease or disorder refers to the administration of a composition to prevent the onset, progression, or recurrence of some or all of the symptoms of lung cancer, or to reduce the likelihood of developing lung cancer. Treating, treating, or therapy for lung cancer means slowing, stopping, or reversing the progression of cancer by the administration of a treatment according to the present invention. In a preferred embodiment, treating lung cancer means reversing the progression of cancer to the point of ideally eliminating the cancer itself.

[0106] VII. Computer Implementation The methods of the present invention, including a method for cancer staging by determining the target level of risk for a target metastatic cancer, are performed in some embodiments on one or more computers.

[0107] For example, the construction and deployment of complementary models and / or risk models, as well as database storage devices, can be implemented in hardware, software, or a combination thereof. In one embodiment of the present invention, a machine-readable storage medium is provided, which includes data storage material encoded by machine-readable data, and when used with a machine programmed with instructions for using the data, it can display any of the datasets, executions, and results of the models described herein. The present invention can be implemented by a computer program running on a programmable computer, which includes a processor, a data storage system (including volatile and non-volatile memory and / or storage elements), a graphics adapter, a pointing device, a network adapter, at least one input device, and at least one output device. The display is coupled to the graphics adapter. The program code is applied to the input data to perform the above functions and generate output information. The output information is applied to one or more output devices in known ways. The computer can be, for example, a personal computer, microcomputer, or workstation of a conventional design.

[0108] Each program may be executed in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if necessary, the program may be executed in assembly language or machine language. In either case, the language may be a compiled language or an interpreted language. Each such computer program is preferably stored in a storage medium or device (e.g., ROM or magnetic diskette) that is readable by a general-purpose or dedicated programmable computer to configure and operate the computer when the storage medium or device is read by the computer to perform the procedures described herein. The system may also be considered to be implemented as a computer-readable storage medium configured with computer programs, such a storage medium that causes the computer to operate in a specific predefined manner to perform the functions described herein.

[0109] Signature patterns and their databases can be provided on various media to facilitate their use. “Medium” refers to a product containing the signature pattern information of the present invention. The database of the present invention can be recorded on a computer-readable medium, for example, any medium that a computer can read and directly access. Such media include, but are not limited to, magnetic storage media such as floppy disks, hard disk storage media, and magnetic tapes, optical media such as CD-ROMs, electrical storage media such as RAM and ROM, and hybrids of these categories such as magnetic / optical storage media. Those skilled in the art will readily recognize how a product containing the recording of this database information can be made using any of the currently known computer-readable media. “Recorded” refers to the process of storing information on a computer-readable medium using any such method known in the art. Any convenient data storage structure can be selected based on the means used to access the stored information. Various data processor programs and formats can be used for storage, for example, a word processing text file, a database format, etc.

[0110] In some embodiments, the methods of the present invention, including a method for cancer staging by determining a target level of risk for a target metastatic cancer, are executed on one or more computers in a distributed computing system environment (e.g., within a cloud computing environment). Hereinafter, “cloud computing” is defined as a model for enabling on-demand network access to a shared set of configurable computing resources. Cloud computing can provide on-demand access to a shared set of configurable computing resources. This shared set of configurable computing resources can be rapidly provisioned via virtualization, released with low management effort or service provider interaction, and scaled accordingly. Cloud computing models can be comprised of various characteristics, such as on-demand self-service, broad network access, resource pooling, rapid scalability, and metric services. Cloud computing models can also expose various service models, such as Software as a Service ("SaaS"), Platform as a Service ("PaaS"), and Infrastructure as a Service ("IaaS"). Cloud computing models can also be deployed using different deployment models such as private clouds, community clouds, public clouds, and hybrid clouds. In this specification and claims, “cloud computing environment” is an environment in which cloud computing is used.

[0111] Figure 4 shows an exemplary computer for implementing the entities shown in Figures 1A, 1B, 2A, 2B, 3A, and 3B. Computer 400 includes at least one processor 402 coupled to a chipset 404. The chipset 404 includes a memory controller hub 420 and an input / output (I / O) controller hub 422. Memory 406 and a graphics adapter 412 are coupled to the memory controller hub 420, and a display 418 is coupled to the graphics adapter 412. A storage device 408, an input device 414, and a network adapter 416 are coupled to the I / O controller hub 422. Other embodiments of computer 400 have different architectures.

[0112] The storage device 408 is a non-temporary computer-readable storage medium such as a hard drive, compact disk read-only memory (CD-ROM), DVD, or solid-state memory device. Memory 406 holds instructions and data used by the processor 402. The input interface 414 is a touchscreen interface, mouse, trackball, or other type of pointing device, keyboard, or some combination thereof, used to input data into the computer 400. In some embodiments, the computer 400 may be configured to receive input (e.g., commands) from the input interface 414 via gestures from the user. The graphics adapter 412 displays images and other information on the display 418. The network adapter 416 connects the computer 400 to one or more computer networks.

[0113] Computer 400 is adapted to run computer program modules for providing the functionality described herein. As used herein, the term “module” refers to computer program logic used to provide the specified functionality. Thus, modules can be implemented in hardware, firmware, and / or software. In one embodiment, a program module is stored in a storage device 408, loaded into memory 406, and executed by a processor 402.

[0114] The type of computer 400 used by the entity in Figure 1A or Figure 1B may vary depending on the embodiment and the processing power required by the entity. For example, the cancer staging system 130 may operate with a single computer 400 or with multiple computers 400 communicating with each other via a network such as a server farm. The computer 400 may omit some of the above components, such as the graphics adapter 412 and the display 418.

[0115] VIII. Systems This specification further discloses a system for cancer staging by determining the target level of risk for a target metastatic cancer. In various embodiments, such a system may include at least the cancer staging system 130 described in Figure 1A. In various embodiments, the cancer staging system 130 is embodied as a computer system, such as a computer system having the exemplary computer 400 shown in Figure 4.

[0116] In various embodiments, the system includes an imaging device such as the imaging device 120 described above in Figure 1A. In various embodiments, the system includes both a cancer staging system 130 (e.g., a computer system) and an imaging device. In such embodiments, the cancer staging system 130 can be communicatively coupled with the imaging device 120 to receive images (e.g., CT scans) captured from the subject. The computer system analyzes the images and determines the subject-level risk of nodular disease in the subject by implementing one or both of a complementary model and a risk model within the computer. [Examples]

[0117] The following are examples of specific embodiments for carrying out the present invention. These examples are provided for illustrative purposes only and are not intended to limit the scope of the present invention in any way. While efforts have been made to ensure accuracy with respect to the numerical values ​​used (e.g., quantities, temperatures, etc.), some degree of experimental error and deviation should be acceptable.

[0118] Example 1: Research population and feature extraction This study was conducted using data from the CT arm of the National Lung Screening Trial (NLST). This subcohort of NLST included data from 658 study participants, 301 with good nodules and 357 with lung cancer. Each CT scan of the 658 study participants was examined and annotated to up to 21 lymph node stations (16 ± 6 per person) (9969 LNs were annotated, with 4631 LNs in the control group and 5318 LNs in study participants with lung cancer). From this initial sample, 288 individuals with lung cancer had staging data available for analysis. 287 control subjects were identified to create a well-balanced initial cohort. Lymph node stations are shown in Figure 2A and detailed in Table 1.

[0119] [Table 1]

[0120] For each LN station, a spherical region with radii of 5 mm and 7.5 mm was defined. The center of the sphere was defined by annotation markers. Two examples of LN annotation are shown in Figure 2B.

[0121] These smaller (5mm) and larger (7.5mm) spherical ROIs (Regions of Interest) were used to investigate the extent of images used to build the model. From these spherical ROIs, three distinct radiomics panels were extracted, each with six feature categories. Each panel differed in the number of metrics used and was based on different transformations of the CT. Panel 1 used the original image as input for feature extraction. Panels 2 and 3 used the wavelet transform and Gaussian Laplacian (LoG) transform of the input image, respectively. Feature categories included first-order features, shape features, gray-level co-occurrence matrix 1 (GLCM) features, gray-level run-length matrix (GLRLM) features, gray-level size zone matrix (GLSZM) features, and neighboring gray-tone difference matrix (NGTDM) features. Feature decompositions are shown in Table 3.

[0122] [Table 2] * The shape characterizes areas that are not included in the LN analysis when each station is analyzed within a certain spherical region.

[0123] Figures 5A–5R represent exemplary correlation heatmaps of different lymph node radiomic features across different panels of GLCM feature categories. Additionally, Figures 5A–5R each show specific features. Examples of such features shown in Figures 5A–5R include correlation features, LMC2, MCC, differential entropy, bound entropy, total entropy, cluster prominence, differential mean, bound mean, total mean, autocorrelation, differential variance, contrast, cluster tendency, total square, LDMN, LDN, cluster shade, LMC1, inverse variance, LD, LDM, bound energy, and highest probability. Figures 5C–5R show features related to different filters. Specifically, Figures 5C–5R provide the notation "XYZ," where "X," "Y," and "Z" refer to filters applied in the X, Y, or Z direction of the image, respectively. "H" refers to a high-pass filter, while "L" refers to a low-pass filter. Therefore, the notation "HHH" refers to features derived from a 3D image that has undergone high-pass filtering in the X, Y, and Z directions. The notation "HHL" refers to features derived from a 3D image that has undergone high-pass filtering in the X and Y directions and low-pass filtering in the Z direction.

[0124] Example 2: Complementary model for cancer staging Modeling began with predicting cancer metastasis to each lymph node. To do this, cancer stage (reported in NLST) was added to each annotated LN station (individual LN data was not reported in NLST, so LN invasion was "backed out" from cancer stage). The data was then split 50:50 for training and testing, and the data distribution is provided in Table 2.

[0125] [Table 3]

[0126] The model results were presented in binary format (using an N-stage covariate on the NLST dataset to indicate yes / no for nodular metastasis; "No" was N=0 or N=1, and "Yes" was N=2 or N=3). The metric was normalized (z-score) and used to train and test random forest and LASSO classifiers. For each LN, a continuous probability of being cancerous was generated. For example, there could be an LN with a 20% probability of being cancerous (assigned an 80% probability of not being cancerous by a subtraction mechanism). All LNs within and across the study participants were considered independent data. Figure 6 shows the performance of the initial model. Specifically, the model achieved an Area under the Curve (AUC) of 65.3%, Accuracy (ACC) of 96.6%, Sensitivity of 61.2%, and Specificity of 60.8%.

[0127] A limitation of the NLST data is that information specific to each LN is unavailable, and on average, 16 LNs were annotated and each underwent radiomics-based evaluation. Therefore, this model assumed that all LNs within a patient with metastatic cancer were "+", whereas in clinical medicine, typically only one LN contains cancerous cells (cancer generally spreads to only one or two LNs, rather than all LNs simultaneously). Thus, complementing the invasion of individual LNs from the cancer stage at the target level leads to false positives that confuse model performance at the LN level (again, since all LNs within the range of that person's CT scan are considered "+").

[0128] Next, the probability of N-stage was determined using the highest probability score predicted by the LN level classifier for all LN stations of a given subject. Figures 7A and 7B show the subject-level performance based on the probability of N-stage (AUC = 69.1%, sensitivity = 72.7%, and specificity = 61.6%). Furthermore, as shown in Figure 7B, the distribution of scores (e.g., highest probability) across all N stages was plotted. Here, a higher highest probability was observed for stage 3 cancer (left panel).

[0129] Example 3: Step-up lymph node classification approach: Inter-patient LN bootstrapping A two-step bootstrap approach was implemented to transition from LN-level risk of metastatic disease to target-level risk of metastatic disease. Specifically, as described above with reference to Figure 3A, the two-step bootstrap approach involved the implementation of complementary and risk models. The challenge here was to complement all LN stations with the target N stages in order to train the classifier. Given that most LNs do not contain cancer, this complementation introduces errors at the LN stations.

[0130] The two steps were as follows: 1. Complementary Analysis: For each subject with known metastatic disease, the lymph nodes most likely to be involved in cancer were identified. This was done for each subject by selecting LNs with a probability of being cancerous (output from the final model) that exceeded the median probability within the subjects (a complementary process including the complementary model shown in Figure 3A). LNs with a probability lower than the median probability within the subjects having cancer were classified as negative. 2. Classification: The cohort was split into 50:50 training:test groups. A second classifier (e.g., risk model 320 in Figure 3A) was then trained using a complementary pool of LN results to predict the presence of nodular disease for each subject.

[0131] The output of this model was a continuous probability that each LN was cancerous, similar to the output of the model described above in Example 2. However, the benefit of the additional complementarity and classification steps was that there was a wider range of probabilities assigned to all lymph nodes in subjects with metastatic disease. More specifically, LNs that were more likely to have cancer had a higher probability of being "+", while LNs that were less likely to have cancer had a lower probability of being "+". For the prediction step, a subject level score for LN invasion was constructed based on the LN with the highest probability (Figure 3B). Specifically, this involved LNs taking unique probabilities and providing a unique probability for subjects with LN invasion of lung cancer. In the subjects shown in Figure 3B, the bottom LN was assigned a 60% probability of having cancer (e.g., probability (+) = 0.6). This was the LN with the highest "+" probability among the three LNs. Therefore, a value of 0.6 was assigned to the subject level probability.

[0132] As illustrated in Figure 9A, after training the step-up approach using a random forest classifier, the bootstrap model achieved a target-level performance of AUC = 77.7%, sensitivity = 81.8%, and specificity = 59.9%. Thus, the performance of the two-step bootstrap model was improved compared to the approach described in Example 2. Similar performance was achieved using the LASSO classifier. Finally, this model can be used to plot the distribution of scores (or highest probabilities) across the entire N stage (Figure 9B). Interestingly, this model obtains higher highest probabilities when there are higher N stages. These probability scores can be used in different clinical situations to enable a different patient triage decision tool. Additional data for training and testing can provide a model that predicts N as a categorical covariate rather than a binary covariate.

[0133] All publications, patents, patent applications, and other documents cited in this application are incorporated herein by reference in whole for all purposes to the extent that each individual publication, patent, patent application, and other document is individually indicated to be incorporated by reference for all purposes.

[0134] While various specific embodiments have been illustrated and described, the above specification is not limiting. It will be recognized that various modifications can be made without departing from the spirit and scope of this disclosure. Many modifications will become apparent to those skilled in the art upon examination of this specification.

[0135] Example 4: Exemplary guided intervention using a CT image-based predictive device Image-based devices implementing the methods described herein (e.g., deploying risk models to predict the risk of lymph node (LN) level metastatic cancer) will impact clinical care due to the inadequate performance of conventional methods, including PET scans. A recent examination of 219 NSCLC patients who underwent systematic lymph node dissection / sampling examined the performance of preoperative PET staging (see, for example, Li et al., Implications of False Negative and False Positive Diagnosis in Lymph Node Staging of NSCLC by Means of 18F-FDG PET / CT. PLoS ONE 8(10):e78552, the whole of which is incorporated by reference). The sensitivity, specificity, PPV, and NPV of PET for detecting hilar and mediastinal lymph node metastases were 74.2%, 54.4%, 86.8%, and 73.5%, respectively, with a false-negative rate of 13.2% and a false-positive rate of 45.5%. Therefore, the method described herein represents a non-invasive technique for detecting metastatic disease resulting in lymph node invasion, improving decision-making and clinical care. Examples of the impact of this device on decision-making and clinical care include:

[0136] This encourages further testing in subjects with tumors of a size that poses a low risk of metastasis. Since there is no standard care for the use of preoperative PET scans or lymph node biopsies in subjects with small lung tumors, prediction of lymph node invasion based on CT images captured by a CT device encourages subsequent testing via PET scans or bronchoscopic sampling of the subject's lymph nodes. In contrast, conventional methods involve subjects undergoing surgery, and more advanced disease stages are only recognized during surgery.

[0137] If the PET result is negative, tissue-based lymph node sampling should be encouraged. While there is a general awareness that PET has false-positive and false-negative failure rates, diagnostic trials are heavily influenced by the results of the PET scan. Patients with smaller tumor sizes and negative PET results often undergo surgery without additional LN sampling. This subset of patients would benefit from additional LN sampling before surgery.

[0138] False-positive PET scans are used to stage patients and demonstrate their eligibility for surgical intervention. Currently, patients strongly suspected of having advanced-stage disease based on PET scans receive non-curative systemic / non-surgical treatment without additional confirmatory lymph node sampling. Patients predicted to have lymph node invasion (e.g., based on CT images of lymph nodes) and those with negative PET results instead undergo more limited lymph node sampling before making a decision regarding surgery. Thus, these patients may be candidates for surgical resection and have a higher chance of cure.

[0139] This indicates which lymph nodes (LNs) are most likely to have metastatic disease. Patients at high risk of metastatic disease undergo bronchoscopic or surgical sampling of lymph nodes. Typically, this strategy places the greatest emphasis on LNs proximal to the tumor, with more distant LNs either not being sampled or being sampled less. Hereinafter, the method described herein enables prediction of lymph node invasion and identification of the lymph nodes involved. This guides the clinical team to appropriately focus their sampling strategy on obtaining tissue, maximizing yield, and minimizing the risk of missing a diagnosis.

[0140] We will refine the enrollment of clinical trials that focus on patients without lymph node metastasis. Patients with localized disease often require clinical trials that focus on local delivery (transbronchial intravenous administration of chemotherapy or ablation). Unfortunately, such patients often have more advanced-stage cancer discovered during lymph node sampling, which delays appropriate care. Therefore, implementing the methods described herein (e.g., deploying a risk model to predict the risk of lymph node (LN) level cancer with lymph node metastasis) will enable the identification of patients exhibiting lymphatic invasion, thus eliminating the need to enroll such patients in clinical trials that focus on patients without metastasis.

[0141] [Implementation Method] (1) A method for determining the target level of risk for a target metastatic cancer, wherein the method is Acquiring one or more images captured from the subject, including multiple lymph nodes of the subject, The process includes predicting the target-level risk of metastatic cancer by applying a risk model to extracted features of one or more images obtained, wherein the risk model is trained to predict the lymph node (LN) level risk of metastatic cancer for lymph nodes in the images. A method wherein the risk model is trained using labels derived from the risk of a complemented LN level for metastatic cancer for at least one pair of lymph nodes in a reference individual, and the risk of the complemented LN level for the pair of lymph nodes is determined using a complementation model that distinguishes between cancerous and non-cancerous lymph nodes in training images. (2) Predicting the target level of risk for the aforementioned metastatic cancer is Based on the risk of metastatic cancer at the LN level predicted by the aforementioned risk model, one or more of the aforementioned lymph nodes are selected. The method according to Embodiment 1, further comprising determining the target level risk of the metastatic cancer using the predicted LN level risk of the metastatic cancer for one or more lymph nodes. (3) The method according to Embodiment 2, wherein selecting one or more of the plurality of lymph nodes includes identifying the lymph node having the highest probability LN level risk, and determining the target level risk for metastatic cancer includes assigning the highest probability LN level risk as the target level risk for metastatic cancer. (4) The method according to any one of embodiments 1 to 3, wherein the risk model predicts the risk of LN levels of metastatic cancer, wherein the risk model has a wider probability range than the risk of the complemented LN levels predicted by the complementation model. (5) The set of lymph nodes of the reference individual, Determining the median risk value, The method according to any one of embodiments 1 to 4, selected by comparing the risk of the supplemented LN level of the lymph nodes of the reference individual with the median risk value.

[0142] (6) The set of lymph nodes of the reference individual, The method according to Embodiment 5, further selected by including one or more lymph nodes having a risk of LN levels higher than the median risk value of the aforementioned pair of lymph nodes. (7) The method according to embodiment 5 or 6, wherein one or more lymph nodes having a risk at an LN level lower than the median risk value of the set of lymph nodes are excluded from the set of lymph nodes. (8) The method according to any one of embodiments 5 to 7, wherein the median risk value is the median risk of LN levels within the reference individual. (9) The method according to any one of embodiments 1 to 8, wherein the one or more images include computed tomography (CT) images. (10) The method according to any one of embodiments 1 to 9, wherein the one or more images are obtained from a chest CT scan.

[0143] (11) The method according to any one of embodiments 1 to 10, further comprising determining the stage of the cancer according to the risk level of the predicted metastatic cancer. (12) The method according to any one of embodiments 1 to 11, further comprising selecting a diagnosis or treatment for the subject according to the risk level of the subject for the predicted metastatic cancer. (13) The method according to Embodiment 12, wherein selecting a diagnosis or treatment for the subject includes selecting either surgical tumor resection or a combination of diagnosis and treatment by bronchoscopy or endoscopy when the risk level of the subject for the predicted metastatic cancer is below a threshold risk value. (14) The method according to Embodiment 12, wherein selecting a diagnosis or treatment for the subject includes selecting an additional diagnostic test from one or both of the following: performing a PET or PET-CT scan, or performing a lymph node biopsy, when the risk level of the subject for the predicted metastatic cancer is higher than a threshold risk value. (15) The method of Embodiment 12, wherein selecting a diagnosis or treatment for the subject includes selecting an additional diagnostic test of PET or PET-CT scan when the risk level of the subject for the predicted metastatic cancer is below a threshold risk value.

[0144] (16) The method according to any one of embodiments 1 to 15, wherein the complementary model is trained using training images obtained from a chest CT scan. (17) The method according to embodiment 16, wherein the training image obtained from a chest CT scan includes one or more lymph nodes. (18) The method according to any one of embodiments 1 to 17, wherein the complementary model is trained using training images from the National Lung Screening Trial (NLST). (19) The method according to any one of embodiments 1 to 17, wherein the complementary model is trained using training images from a custom dataset. (20) The method according to any one of embodiments 16 to 19, wherein the complementary model is trained by generating two or more radiomic panels from at least one of the training images.

[0145] (21) The method according to Embodiment 20, wherein the two or more radiomics panels include the Laplacian of the Gaussian transform of the training images and the wavelet transform of the training images. (22) The method according to Embodiment 20 or 21, wherein the complementary model is further trained by extracting features from each of the two or more radiomics panels and training the complementary model using at least the extracted features. (23) The aforementioned complementary model The process involves defining a region of interest (ROI) within the radiomics panel, wherein the defined region of interest includes lymph nodes. Extracting features from the ROI within the radiomics panel, The method according to Embodiment 20 or 21, wherein the complementary model is further trained by training it using at least the extracted features. (24) The method according to Embodiment 22 or 23, wherein the extracted features include one or more feature categories from among primary features, shape features, gray-level co-occurrence matrix 1 (GLCM) features, gray-level run-length matrix (GLRLM) features, gray-level size zone matrix (GLSZM) features, and neighboring gray-tone difference matrix (NGTDM) features. (25) The method according to any one of embodiments 16 to 24, wherein the complementary model is further trained using a reference ground truth value indicating the presence or absence of transfer in the training images.

[0146] (26) The method according to Embodiment 25, wherein the reference ground truth value is derived from an N-stage covariate value indicating the stage of cancer, and a cancer stage of "0" or "1" indicates the absence of metastasis, and a cancer stage of "2" or "3" indicates the presence of metastasis. (27) The method according to Embodiment 26, wherein the reference ground truth value indicates whether an individual lymph node is cancerous or non-cancerous. (28) The method according to Embodiment 27, wherein the reference ground truth value indicating whether an individual lymph node is cancerous or non-cancerous is calculated backward from the N-stage covariate value based on the characteristics of the tumor or lymph node. (29) The method according to any one of embodiments 1 to 28, wherein the complementary model is a random forest classifier or a LASSO classifier. (30) The method according to any one of embodiments 1 to 29, wherein the risk model is a random forest classifier or a LASSO classifier.

[0147] (31) The method according to embodiment 30, wherein the risk model is trained using supervised learning techniques. (32) The method according to any one of embodiments 1 to 31, wherein the plurality of lymph nodes comprises 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, or 21 lymph nodes. (33) The method according to any one of embodiments 1 to 31, wherein the plurality of lymph nodes include 8 to 20 lymph nodes, 10 to 19 lymph nodes, 12 to 18 lymph nodes, or 14 to 16 lymph nodes. (34) The method according to any one of embodiments 1 to 33, wherein the metastatic cancer is metastatic lung cancer. (35) The method according to any one of Embodiments 1 to 34, wherein the risk model exhibits performance of at least AUC = 78%, sensitivity = 82%, or specificity = 60%.

[0148] (36) A method for training a risk model according to any of Embodiments 1 to 35, wherein the method is Applying a complementary model to generate complementary LN level risk for multiple lymph nodes in the training images, The selection of a set of lymph nodes, wherein the lymph nodes within the set have a risk level of LN higher than the median risk value. A method comprising using the risk of the LN level of the lymph nodes in the set of lymph nodes as a reference ground truth for training the risk model, such that the risk model can predict a risk of the LN level having a wider probability range than the complemented LN level risk predicted by the complement model. (37) The method according to Embodiment 36, wherein the risk model is further trained by extracting features from one or more radiomics panels and training the risk model using at least the extracted features. (38) Extracting features from two or more radiomics panels Defining a region of interest (ROI) within one or more radiomic panels, wherein the defined region of interest includes lymph nodes. The method according to Embodiment 37, comprising extracting features from the ROI within one or more radiomic panels. (39) The method according to Embodiment 37 or 38, wherein the extracted features include one or more feature categories from among primary features, shape features, gray-level co-occurrence matrix 1 (GLCM) features, gray-level run-length matrix (GLRLM) features, gray-level size zone matrix (GLSZM) features, and neighboring gray-tone difference matrix (NGTDM) features. (40) A system for determining the target level of risk for a target metastatic cancer, wherein the system An imaging device configured to capture one or more images of the aforementioned target, A computing device and the computing device The steps include obtaining one or more images captured from the target, including multiple lymph nodes of the target, The system is configured to perform the steps of predicting the target level risk of metastatic cancer by applying a risk model to the extracted features of one or more images obtained, wherein the risk model is trained to predict the lymph node (LN) level risk of metastatic cancer for lymph nodes in the images. A system in which the risk model is trained using labels derived from the risk of metastatic cancer at a complemented LN level for at least one pair of lymph nodes of a reference individual, and the risk of the complemented LN level for the pair of lymph nodes is determined using at least a complementation model that distinguishes between cancerous and non-cancerous lymph nodes in training images.

[0149] (41) Predicting the target level of risk for the aforementioned metastatic cancer is Based on the risk of metastatic cancer at the LN level predicted by the aforementioned risk model, one or more of the aforementioned lymph nodes are selected. The system according to Embodiment 40, further comprising determining the target level risk of the metastatic cancer using the predicted LN level risk for one or more lymph nodes. (42) The system according to Embodiment 41, wherein selecting one or more of the plurality of lymph nodes includes identifying the lymph node having the highest probability of LN level risk, and determining the target level risk for metastatic cancer includes assigning the highest probability of LN level risk as the target level risk for metastatic cancer. (43) The system according to any one of embodiments 40 to 42, wherein the risk model predicts the risk of LN levels of metastatic cancer having a wider probability range than the risk of the complemented LN levels predicted by the complementation model. (44) The set of lymph nodes of the reference individual, Determining the median risk value, A system according to any one of embodiments 40 to 43, selected by comparing the risk of the supplemented LN level of the lymph nodes of the reference individual with the median risk value. (45) The set of lymph nodes of the reference individual, The system according to Embodiment 44, further selected by including one or more lymph nodes having a risk of LN levels higher than the median risk value of the aforementioned pair of lymph nodes.

[0150] (46) The system according to embodiment 44 or 45, wherein one or more lymph nodes having a risk at an LN level lower than the median risk value of the set of lymph nodes are excluded from the set of lymph nodes. (47) The system according to any one of embodiments 44 to 46, wherein the median risk value is the median risk of LN levels within the reference individual. (48) The system according to any one of embodiments 40 to 47, wherein the imaging device is one of a computed tomography (CT) scanner, a magnetic resonance imaging (MRI) scanner, a positron emission tomography (PET) scanner, an X-ray scanner, or an ultrasound imaging device. (49) The system according to any one of embodiments 40 to 48, wherein the imaging device is a CT scanner, and the one or more images include computed tomography (CT) images. (50) The system according to any one of embodiments 40 to 49, wherein the one or more images include a chest CT image.

[0151] (51) The system according to any one of embodiments 40 to 50, wherein the computing device is further configured to perform the step of determining the stage of the cancer in accordance with the target level risk of the predicted metastatic cancer. (52) The system according to any one of embodiments 40 to 51, wherein the computing device is further configured to perform the step of selecting a diagnosis or treatment for the subject according to the risk level of the subject of the predicted metastatic cancer. (53) The system according to Embodiment 52, wherein selecting a diagnosis or treatment for the subject includes selecting either surgical tumor resection or a combination of diagnosis and treatment by bronchoscopy or endoscopy when the risk level of the subject for the predicted metastatic cancer is below a threshold risk value. (54) The system according to Embodiment 53, wherein selecting a diagnosis or treatment for the subject includes selecting an additional diagnostic test, either or both of the following: performing a PET or PET-CT scan, or performing a lymph node biopsy, when the risk level of the subject level of the predicted metastatic cancer is higher than a threshold risk value. (55) The system according to Embodiment 53, wherein selecting a diagnosis or treatment for the subject includes selecting an additional diagnostic test of PET or PET-CT scan when the risk level of the subject for the predicted metastatic cancer is below a threshold risk value.

[0152] (56) The system according to any one of embodiments 40 to 54, wherein the complementary model is trained using training images obtained from a chest CT scan. (57) The system according to embodiment 56, wherein the training image obtained from a chest CT scan includes one or more lymph nodes. (58) The system according to any one of embodiments 40 to 57, wherein the complementary model is trained using training images from the National Lung Screening Trial (NLST). (59) The system according to any one of embodiments 40 to 57, wherein the complementary model is trained using training images from a custom dataset. (60) The system according to any one of embodiments 56 to 59, wherein the complementary model is trained by generating at least one, two or three or more radiomics panels from the training images.

[0153] (61) The system according to embodiment 60, wherein the two or more radiomics panels include the Laplacian of the Gaussian transform of the training images and the wavelet transform of the training images. (62) The system according to embodiment 60 or 61, wherein the complementary model is further trained by extracting features from each of the two or more radiomics panels and training the complementary model using at least the extracted features. (63) The complementary model is The process involves defining a region of interest (ROI) within the radiomics panel, wherein the defined region of interest includes lymph nodes. Extracting features from the ROI within the radiomics panel, The system according to embodiment 60 or 61, further trained by training the complementary model using at least the extracted features. (64) The system according to Embodiment 62 or 63, wherein the extracted features include one or more feature categories from among primary features, shape features, gray-level co-occurrence matrix 1 (GLCM) features, gray-level run-length matrix (GLRLM) features, gray-level size zone matrix (GLSZM) features, and neighboring gray-tone difference matrix (NGTDM) features. (65) The system according to any one of embodiments 56 to 64, wherein the complementary model is further trained using a reference ground truth value indicating the presence or absence of transfer in the training images.

[0154] (66) The system according to embodiment 65, wherein the reference ground truth value is derived from an N-stage covariate value indicating the stage of cancer, and a cancer stage of "0" or "1" indicates the absence of metastasis, and a cancer stage of "2" or "3" indicates the presence of metastasis. (67) The system according to Embodiment 66, wherein the reference ground truth value indicates whether an individual lymph node is cancerous or non-cancerous. (68) The system according to Embodiment 67, wherein the reference ground truth value indicating whether an individual lymph node is cancerous or non-cancerous is calculated backward from the N-stage covariate value based on the characteristics of the tumor or lymph node. (69) The system according to any one of embodiments 40 to 68, wherein the complementary model is a random forest classifier or a LASSO classifier. (70) The system according to any one of embodiments 40 to 69, wherein the risk model is a random forest classifier or a LASSO classifier.

[0155] (71) The system according to embodiment 70, wherein the risk model is trained using supervised learning techniques. (72) The system according to any one of embodiments 40 to 71, wherein the plurality of lymph nodes include 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, or 21 lymph nodes. (73) The system according to any one of embodiments 40 to 72, wherein the plurality of lymph nodes include 8 to 20 lymph nodes, 10 to 19 lymph nodes, 12 to 18 lymph nodes, or 14 to 16 lymph nodes. (74) The system according to any one of embodiments 40 to 73, wherein the metastatic cancer is metastatic lung cancer. (75) The system according to any one of embodiments 40 to 74, wherein the risk model exhibits performance of at least AUC = 78%, sensitivity = 82%, or specificity = 60%.

[0156] (76) The risk model described above is Applying a complementary model to generate complementary LN level risk for multiple lymph nodes in the training images, The selection of a set of lymph nodes, wherein the lymph nodes within the set have a risk level of LN higher than the median risk value. A system according to any one of embodiments 40 to 75, wherein the risk model is trained by using the risk of the LN level of the lymph nodes in the set of lymph nodes as a reference ground truth for training the risk model, so that the risk model can predict a risk of the LN level having a wider probability range than the complemented LN level risk predicted by the complemented model. (77) The system according to embodiment 76, wherein the risk model is further trained by extracting features from one or more radiomics panels and training the risk model using at least the extracted features. (78) Extracting features from two or more radiomics panels Defining a region of interest (ROI) within one or more radiomic panels, wherein the defined region of interest includes lymph nodes. The system according to embodiment 77, comprising extracting features from the ROI within one or more radiomic panels. (79) The system according to Embodiment 77 or 78, wherein the extracted features include one or more feature categories from among primary features, shape features, gray-level co-occurrence matrix 1 (GLCM) features, gray-level run-length matrix (GLRLM) features, gray-level size zone matrix (GLSZM) features, and neighboring gray-tone difference matrix (NGTDM) features. (80) A non-temporary computer-readable medium containing instructions, wherein when the instructions are executed by the processor, the processor has access to: Acquiring one or more images captured from the subject, including multiple lymph nodes of the subject, The process involves predicting the target level risk of metastatic cancer by applying a risk model to the extracted features of one or more images obtained, wherein the risk model is trained to predict the lymph node (LN) level risk of metastatic cancer for lymph nodes in the images. A non-transient, computer-readable medium wherein the risk model is trained using labels derived from the risk of metastatic cancer at a complemented LN level for at least one pair of lymph nodes of a reference individual, and the risk of the complemented LN level for the pair of lymph nodes is determined using at least a complementation model that distinguishes between cancerous and non-cancerous lymph nodes in training images.

[0157] (81) When the processor executes the instruction that causes the processor to predict the target level risk of the metastatic cancer, the processor: Based on the risk of metastatic cancer at the LN level predicted by the aforementioned risk model, one or more of the aforementioned lymph nodes are selected. A non-temporary computer-readable medium according to Embodiment 80, further comprising instructions to determine the target level risk of the metastatic cancer using the predicted LN level risk for one or more lymph nodes. (82) A non-temporary computer-readable medium according to Embodiment 81, wherein when the instruction causing the processor to select one or more of the plurality of lymph nodes is executed by the processor, the instruction causing the processor to identify the lymph node having the highest probability LN level risk, and when the instruction causing the processor to determine the target level risk of the metastatic cancer is executed by the processor, the instruction causing the processor to assign the highest probability LN level risk as the target level risk of the metastatic cancer. (83) A non-transient computer-readable medium according to any of embodiments 80 to 82, wherein the risk model predicts the risk of LN levels of metastatic cancer having a wider probability range than the risk of the complemented LN levels predicted by the complementation model. (84) The set of lymph nodes of the reference individual, Determining the median risk value, A non-transient computer-readable medium according to any of embodiments 80 to 83, selected by comparing the risk of the supplemented LN level of the lymph nodes of the reference individual with the median risk value. (85) A non-transient computer-readable medium according to Embodiment 84, wherein the set of lymph nodes of a reference individual is further selected by including one or more lymph nodes having a risk of an LN level higher than the median risk value of the set of lymph nodes.

[0158] (86) A non-transient computer-readable medium according to embodiment 84 or 85, wherein one or more lymph nodes having a risk at an LN level lower than the median risk value of the set of lymph nodes are excluded from the set of lymph nodes. (87) A non-temporary computer-readable medium according to any one of embodiments 84 to 86, wherein the median risk value is the median of the LN level risk within the reference individual. (88) A non-temporary computer-readable medium according to any one of embodiments 80 to 87, wherein the one or more images include computed tomography (CT) images. (89) A non-temporary computer-readable medium according to any of embodiments 80 to 88, wherein one or more images are obtained from a chest CT scan. (90) A non-transient computer-readable medium according to any one of embodiments 80 to 89, further comprising instructions that, when executed by the processor, cause the processor to determine the stage of the cancer in accordance with the risk level of the predicted metastatic cancer.

[0159] (91) A non-transient computer-readable medium according to any one of embodiments 80 to 90, further comprising instructions, when executed by the processor, causing the processor to select a diagnosis or treatment for the subject in accordance with the predicted level of risk of metastatic cancer for the subject. (92) A non-transient computer-readable medium according to Embodiment 91, wherein when the instruction causing the processor to select a diagnosis or treatment for the subject is executed by the processor, the instruction further includes causing the processor to select either surgical tumor resection or a combination of diagnosis and treatment by bronchoscopy or endoscopy when the risk level of the predicted metastatic cancer subject is lower than a threshold risk value. (93) A non-temporary computer-readable medium according to Embodiment 91, wherein when the instruction causing the processor to select a diagnosis or treatment for the subject is executed by the processor, the instruction further includes causing the processor to select an additional diagnostic test, either or both of the following: performing a PET or PET-CT scan, or performing a lymph node biopsy, if the risk level of the predicted metastatic cancer for the subject is higher than a threshold risk value. (94) A non-transient computer-readable medium according to Embodiment 91, wherein when the instruction causing the processor to select a diagnosis or treatment for the subject is executed by the processor, the instruction causing the processor to select an additional diagnostic test of PET or PET-CT scan when the risk level of the predicted metastatic cancer for the subject is lower than a threshold risk value. (95) A non-temporary computer-readable medium according to any of embodiments 80 to 94, wherein the complementary model is trained using training images obtained from a chest CT scan.

[0160] (96) A non-temporary computer-readable medium according to Embodiment 95, wherein the training image obtained from a chest CT scan includes one or more lymph nodes. (97) A non-temporary computer-readable medium according to any of embodiments 80 to 96, wherein the complementary model is trained using training images from the National Lung Screening Trial (NLST). (98) A non-temporary computer-readable medium according to any of embodiments 80 to 96, wherein the complementary model is trained using training images from a custom dataset. (99) A non-temporary computer-readable medium according to any of embodiments 95 to 98, wherein the complementary model is trained by generating at least one, two or three or more radiomics panels from the training images. (100) A non-temporary computer-readable medium according to Embodiment 99, wherein the two or more radiomics panels include the Laplacian of the Gaussian transform of the training images and the wavelet transform of the training images.

[0161] (101) A non-temporary computer-readable medium according to Embodiment 99 or 100, wherein the complementary model is further trained by extracting features from each of the two or more radiomics panels and training the complementary model using at least the extracted features. (102) The aforementioned complementary model, The process involves defining a region of interest (ROI) within the radiomics panel, wherein the defined region of interest includes lymph nodes. Extracting features from the ROI within the radiomics panel, A non-temporary computer-readable medium according to Embodiment 99 or 100, further trained by training the complementary model using at least the extracted features. (103) A non-temporary computer-readable medium according to Embodiment 101 or 102, wherein the extracted features include one or more feature categories from among primary features, shape features, gray-level co-occurrence matrix 1 (GLCM) features, gray-level run-length matrix (GLRLM) features, gray-level size zone matrix (GLSZM) features, and neighboring gray-tone difference matrix (NGTDM) features. (104) A non-transient computer-readable medium according to any of embodiments 95 to 103, wherein the complementary model is further trained using a reference ground truth value indicating the presence or absence of transfer in the training images. (105) A non-temporary computer-readable medium according to Embodiment 104, wherein the reference ground truth value is derived from an N-stage covariate value indicating the stage of cancer, where a cancer stage of "0" or "1" indicates no metastasis, and a cancer stage of "2" or "3" indicates the presence of metastasis.

[0162] (106) The reference ground truth value is a non-transient computer-readable medium according to Embodiment 105, indicating whether an individual lymph node is cancerous or non-cancerous. (107) A non-temporary computer-readable medium according to Embodiment 106, wherein the reference ground truth value indicating whether an individual lymph node is cancerous or non-cancerous is calculated backward from the N-stage covariate value based on the characteristics of the tumor or lymph node. (108) A non-temporary computer-readable medium according to any of embodiments 80 to 107, wherein the complementary model is a random forest classifier or a LASSO classifier. (109) A non-temporary computer-readable medium according to any of embodiments 80 to 108, wherein the risk model is a random forest classifier or a LASSO classifier. (110) A non-transient computer-readable medium according to Embodiment 109, wherein the risk model is trained using supervised learning techniques.

[0163] (111) A non-temporary computer-readable medium according to any of embodiments 80 to 110, wherein the plurality of lymph nodes comprises 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, or 21 lymph nodes. (112) A non-temporary computer-readable medium according to any of embodiments 80 to 111, wherein the plurality of lymph nodes include 8 to 20 lymph nodes, 10 to 19 lymph nodes, 12 to 18 lymph nodes, or 14 to 16 lymph nodes. (113) A non-transient computer-readable medium according to any of embodiments 80 to 112, wherein the metastatic cancer is metastatic lung cancer. (114) A non-transient computer-readable medium according to any of embodiments 80 to 113, wherein the risk model exhibits performance of at least AUC=78%, sensitivity=82%, or specificity=60%. (115) The risk model described above is Applying a complementary model to generate complementary LN level risk for multiple lymph nodes in the training images, The selection of a set of lymph nodes, wherein the lymph nodes within the set have a risk level of LN higher than the median risk value. A non-transient computer-readable medium according to any one of embodiments 80 to 114, wherein the risk model is trained by using the risk of the LN level of the lymph nodes in the set of lymph nodes as a reference ground truth for training the risk model, so that the risk model can predict a risk of the LN level having a wider probability range than the complemented LN level risk predicted by the complemented model.

[0164] (116) A non-temporary computer-readable medium according to Embodiment 115, wherein the risk model is further trained by extracting features from one or more radiomics panels and training the risk model using at least the extracted features. (117) Extracting features from two or more radiomics panels Defining a region of interest (ROI) within one or more radiomic panels, wherein the defined region of interest includes lymph nodes. A non-temporary computer-readable medium according to Embodiment 116, comprising extracting features from the ROI within one or more radiomic panels. (118) A non-temporary computer-readable medium according to Embodiment 115 or 116, wherein the extracted features include one or more feature categories from among primary features, shape features, gray-level co-occurrence matrix 1 (GLCM) features, gray-level run-length matrix (GLRLM) features, gray-level size zone matrix (GLSZM) features, and neighboring gray-tone difference matrix (NGTDM) features.

Claims

1. 1. A system for determining a subject's level risk of metastatic cancer in a subject, the system comprising: an imaging device configured to capture one or more images of the object; a computing device, the computing device comprising: obtaining one or more images captured from the subject, the images including a plurality of lymph nodes of the subject; predicting the subject-level risk of metastatic cancer by applying a risk model to the extracted features of the one or more acquired images, the risk model being trained to predict lymph node (LN)-level risk of metastatic cancer for lymph nodes in an image; The system, wherein the risk model is trained using signatures derived from the imputed LN-level risk of metastatic cancer for at least a set of lymph nodes of a reference individual, and the imputed LN-level risk of the set of lymph nodes is determined using at least an imputation model that distinguishes between cancerous and non-cancerous lymph nodes in training images.

2. predicting the subject's level of risk of metastatic cancer, selecting one or more of the plurality of lymph nodes based on the LN-level risk of metastatic cancer predicted by the risk model; 2. The system of claim 1, further comprising: determining a subject-level risk of metastatic cancer using the LN-level risk of metastatic cancer predicted for the one or more lymph nodes.

3. 3. The system of claim 2, wherein selecting one or more of the plurality of lymph nodes comprises identifying the lymph node having a highest probable LN level risk, and determining the subject level risk of metastatic cancer comprises assigning the highest probable LN level risk as the subject level risk of metastatic cancer.

4. 2. The system of claim 1, wherein the risk model predicts the LN-level risk of metastatic cancer with a wider range of probability than the imputed LN-level risk predicted by the imputation model.

5. The set of lymph nodes of a reference individual is determining a median risk value; and comparing the imputed LN level risk of the lymph node of the reference individual to the median risk value.

6. The set of lymph nodes of a reference individual is 6. The system of claim 5, further comprising selecting one or more lymph nodes having a LN level risk higher than the median risk value for the set of lymph nodes.

7. 6. The system of claim 5, wherein one or more lymph nodes having an LN level risk lower than the median risk value for the set of lymph nodes are excluded from the set of lymph nodes.

8. The system of claim 5 , wherein the median risk value is the median risk of LN levels in the subjects of the reference individual.

9. The system of claim 1 , wherein the imaging device is one of a computed tomography (CT) scanner, a magnetic resonance imaging (MRI) scanner, a positron emission tomography (PET) scanner, an X-ray scanner, or an ultrasound imager.

10. 1. A non-transitory computer-readable medium containing instructions that, when executed by a processor, cause the processor to: obtaining one or more images captured from a subject, the image including a plurality of lymph nodes of the subject; predicting a subject-level risk of metastatic cancer by applying a risk model to the extracted features of the one or more acquired images, the risk model being trained to predict lymph node (LN)-level risk of metastatic cancer for lymph nodes in an image; The risk model is trained using signatures derived from the imputed LN-level risk of metastatic cancer for at least a set of lymph nodes of a reference individual, and the imputed LN-level risk of the set of lymph nodes is determined using at least an imputation model that distinguishes between cancerous and non-cancerous lymph nodes in training images.