Deep learning-based method for predicting lung cancer recurrence using time-series data, and analysis device

Abstract

This deep learning-based method for predicting lung cancer recurrence using time-series data, comprises the steps in which: an analysis device receives, from a patient, inputs of static data collected before surgery and time-series dynamic data collected after the surgery; the analysis device pre-processes the static data and basic dynamic data among the time-series dynamic data; the analysis device inputs, into a trained prediction model, the pre-processed static data, the pre-processed basic dynamic data, and a medical image analysis sentence included in the time-series dynamic data; and the analysis device predicts the possibility of lung cancer recurrence of the patient within a predetermined period on the basis of a value output by the prediction model.

Description

Deep learning-based lung cancer recurrence prediction method and analysis device using time series data

[0001] The technology described below relates to a technique for predicting the prognosis of lung cancer.

[0002] According to a 2022 statistical report, lung cancer is the leading cause of cancer death in Korea. Although survival rates for lung cancer are improving due to recent drug developments, it remains a tumor with a poor prognosis. Even in stage 1 cases, lung cancer exhibits a recurrence rate of approximately 30%. Therefore, predicting the likelihood of recurrence in lung cancer patients is crucial for patient management.

[0003] Conventional lung cancer recurrence prediction techniques relied on fixed information collected from patients. Consequently, conventional methods had difficulty reflecting patient follow-up data.

[0004] The technology described below aims to predict lung cancer recurrence by integratively considering static and time-series data collected from patients. Furthermore, the technology described below aims to predict progression-free survival after lung cancer surgery.

[0005] A deep learning-based method for predicting lung cancer recurrence using time series data includes the steps of: receiving static data collected from a patient prior to surgery and time series dynamic data collected after surgery; the analysis device preprocessing basic dynamic data among the static data and the time series dynamic data; the analysis device inputting medical image interpretation sentences included in the preprocessed static data, the preprocessed basic dynamic data, and the time series dynamic data into a learned prediction model; and the analysis device predicting the likelihood of lung cancer recurrence in the patient within a certain period based on the value output by the prediction model.

[0006] An analysis device for predicting lung cancer recurrence includes an interface device that receives static data collected from a patient prior to surgery and time-series dynamic data collected after surgery; a storage device that stores a prediction model for predicting recurrence in a subject who has undergone lung cancer surgery; and a computing device that preprocesses basic dynamic data among the static data and time-series dynamic data, inputs medical image interpretation sentences included in the preprocessed static data, the preprocessed basic dynamic data, and the time-series dynamic data into the prediction model, and predicts the lung cancer prognosis of the patient within a certain period based on the value output by the prediction model.

[0007] The technology described below utilizes both static and time-varying patient data to accurately predict the likelihood of lung cancer recurrence and / or progression-free survival for individual patients. Through this, the technology described below contributes to the establishment of effective treatment plans for lung cancer patients.

[0008] Figure 1 is an example of a system for predicting lung cancer recurrence.

[0009] Figure 2 is an example of the process of building a predictive model to predict lung cancer recurrence.

[0010] Figure 3 is an example of a predictive model for predicting lung cancer recurrence.

[0011] Figure 4 shows the results of comparing the progression-free survival period with the results predicted by the prediction model.

[0012] Figure 5 is an example of an analysis device that predicts lung cancer recurrence.

[0013] The technology described below is subject to various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the technology described below to specific embodiments, and it should be understood that it includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the technology described below.

[0014] Terms such as first, second, A, B, etc., may be used to describe various components, but such components are not limited by the said terms and are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of rights of the technology described below, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of multiple related described items or any of the multiple related described items.

[0015] In terms used in this specification, singular expressions should be understood to include plural expressions unless the context clearly indicates otherwise, and terms such as “includes” should be understood to mean that the described features, number, steps, actions, components, parts, or combinations thereof exist, and not to exclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0016] Before providing a detailed description of the drawings, it is to clarify that the classification of components in this specification is merely based on the primary function each component is responsible for. That is, two or more components described below may be combined into a single component, or a single component may be divided into two or more components based on more subdivided functions. Furthermore, each component described below may additionally perform some or all of the functions of other components in addition to its own primary function, and it goes without saying that some of the primary functions of each component may be exclusively performed by other components.

[0017] Furthermore, in performing the method or operation method, each process constituting the method may occur differently from the specified order unless a specific order is clearly indicated in the context. That is, each process may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

[0018] The technology described below is a technique for predicting the likelihood of recurrence in lung cancer patients. Furthermore, since the technology described below utilizes the patient's time-series data to predict the likelihood of recurrence based on data at a specific point in time, it can also predict the patient's progression-free survival period after surgery.

[0019] Lung cancer below refers to non-small-cell lung cancer.

[0020] Lung cancer recurrence is defined as recurrence occurring within a certain period from the time of analysis using patient data. The certain period may be any one of 1 year, 3 years, and 5 years.

[0021] The analysis device can predict lung cancer recurrence and / or progression-free survival using a learning model.

[0022] A learning model refers to a machine learning model. There are various types of machine learning models. For example, learning models include regression analysis, decision trees, random forests, XGBoost (eXtreme Gradient Boosting), LightGBM (Light Gradient Boosting Machine), CatBoost (Categorical Boosting), KNN (K-nearest neighbor), Naive Bayes, SVM (support vector machine), and artificial neural networks. A learning model that predicts the onset of lung cancer can be any one of these various types of models. The following explanation of learning models will focus on deep learning models capable of analyzing both static and dynamic data.

[0023] FIG. 1 is an example of a system (100) for predicting lung cancer recurrence. FIG. 1 illustrates an example in which the analysis device is a computer terminal (130) and a server (140).

[0024] Medical institutions can collect data on patients using various devices.

[0025] Medical imaging equipment (111) acquires medical images of a patient. The medical imaging equipment (111) may be at least one of the following: an x-ray machine, a CT (Computed Tomography) scanner, an MRI (Magnetic Resonance Imaging) scanner, a PET (Positron Emission Tomography) scanner, etc. The medical images may include information such as tumor size, surrounding invasion status, and tumor TNM (Tumor, Node, Metastasis) stage. Medical personnel can verify such information through the medical images.

[0026] The tissue examination device (112) can generate histopathological information about the patient by analyzing the patient's tissue sample. The histopathological information may be the result of an expert analyzing the tissue sample. The histopathological information may include tumor subtype, tumor stage, histological grade (degree of malignancy), etc.

[0027] The genome analysis device (113) analyzes a sample of a subject to generate genetic information. The genetic information includes the expression status and mutations of specific genes. At this time, genetic mutations may include various receptor mutations related to lung cancer. The genome analysis device (110) may be an NGS (Next Generation Sequencing) device.

[0028] The blood test equipment (114) can analyze a patient's sample (blood) to obtain quantitative information on markers, such as specific proteins, within the sample. The markers that can be produced as a result of the blood test are described below.

[0029] Other diagnostic equipment (115) can collect basic demographic information about the patient (gender, age, etc.) and diagnostic information from medical staff. The diagnostic information may include ECOG PS (Eastern Cooperative Oncology Group Performance status) scores for lung cancer patients.

[0030] The aforementioned medical equipment (111 to 115) can collect patient data from the subject before surgery or on the day of surgery. Furthermore, the medical equipment (111 to 115) can continuously collect patient data for a certain period of time after surgery. In particular, the medical imaging equipment (111) and the blood test equipment (114) can collect time-series data about the patient.

[0031] Patient static data refers to fixed patient data acquired at a specific point in time. Patient dynamic data refers to patient data continuously collected over time.

[0032] An EMR (Electronic Medical Record, 120) can store data collected for specific patients. The EMR (120) can store static data and dynamic data for patients. For example, dynamic data may consist of patient data measured at monthly intervals for one, two, or three years.

[0033] The learning device (50) constructs a prediction model that predicts lung cancer recurrence by analyzing patient information. The learning device (50) refers to a computer device capable of preprocessing patient data and constructing a learning model. The learning device (50) can construct a prediction model using certain learning data. The learning process and the model will be described later. The computer terminal (130) and the server (140) can acquire the constructed prediction model and use it for subject analysis.

[0034] The computer terminal (130) can receive static and dynamic data of a specific patient from the EMR (120). The computer terminal (130) can extract significant factors (variables) from the patient data. The computer terminal (130) can preprocess the values ​​of the significant factors in a consistent manner. The preprocessing process will be described later. The computer terminal (130) can predict the likelihood of lung cancer recurrence in the patient by inputting the values ​​of the (preprocessed) significant factors into a prediction model. In addition, the computer terminal (130) can also predict the patient's progression-free survival period by inputting the values ​​of the (preprocessed) significant factors into a prediction model. The computer terminal (130) can output (provide) the prediction results to user A.

[0035] The server (140) can extract static and dynamic data of a specific patient from the EMR (120). The server (140) can extract significant factors (variables) among the patient data. The server (140) can preprocess the values ​​of the significant factors in a consistent manner. The preprocessing process will be described later. The server (140) can predict the likelihood of lung cancer recurrence in the patient by inputting the values ​​of the (preprocessed) significant factors into a prediction model. In addition, the server (140) can also predict the patient's progression-free survival period by inputting the values ​​of the (preprocessed) significant factors into a prediction model. The server (140) can transmit the prediction results to the terminal of user A.

[0036] The computer terminal (130) and / or server (140) may also store the analysis results in the EMR (120).

[0037] FIG. 2 is an example of the process (200) of constructing a predictive model for predicting lung cancer recurrence. Along with the model construction process of FIG. 2, the model construction process actually performed by a researcher is explained.

[0038] It is explained that the construction of the predictive model is performed by a separate training device. The training device is a computer device capable of performing patient data preprocessing, extraction of significant factors, and construction of the predictive model.

[0039] The learning device builds training data for building a prediction model (210).

[0040] The researcher used data from 14,618 patients who underwent lung cancer surgery at Samsung Seoul Hospital from 2008 to 2022 as the initial dataset. From the initial dataset, data from patients with primary cancer, those who underwent R1 resection, and those without time-series data were excluded, resulting in the use of data from 14,177 patients. This dataset includes both the static and dynamic data described below. Of these, data from 11,341 patients were used for training, and data from 2,836 patients were used for validation.

[0041] A patient database (DB) can store patient data.

[0042] The learning device can select significant factors from patient data that predict lung cancer recurrence and / or progression-free survival. For example, the learning device can select significant factors among candidate factors using a normality test, an independent samples t-test, Wilcoxon's rank sum test, etc. Alternatively, the learning device can select significant factors by performing regression analysis on the candidate factors.

[0043] The significant factors selected by the researcher from the static and dynamic data are as shown in Table 1 and Table 2, respectively.

[0044] Table 1 below contains static data. Static data consists of clinical information, pathological information, and genetic information collected primarily from lung cancer patients.

[0045] Static Data General Clinical Information Pathology Information Genetic Information Age TNM Stage EGFR (Epidermal Growth Factor Receptor) Variant Gender Lesion Location PD-L1 Expression Status Height Pathological Type (AD, SQ) ALK Variant Weight Pattern (Lepidic, Acinar, etc.) Other Variants (L858R, DEL19, etc.) ECOG PS Score Differentiation Grade (MD, PD, WD) Smoking History Vascular Invasion FVC (forced vital capacity) Lymphatic Invasion FEV1 (forced expiratory volume in one second) Perineural Invasion Type of Surgery Pleural Invasion Resection Margin Metastasis Tumor Size

[0046] Static data is data confirmed through examinations prior to surgery or on the day of surgery. Static data is data that has fixed values. Static data is classified into continuous variables and categorical variables. Categorical variables are values ​​classified as "negative (0) or positive (1)" or one of multiple categories. Pathological types are classified into adenocarcinoma (AD) and squamous cell carcinoma (SQ).

[0047] Lesion patterns are classified into lepidic, acinar, papillary, micropapillary, and solid.

[0048] Differentiation is classified into WD (well-differentiated), MD (moderated differentiated), and PD (poor differentiated).

[0049] Table 2 below contains dynamic data. Dynamic data is time-series data collected over a certain period after surgery. Dynamic data is divided into blood test information and CT interpretation information.

[0050] Dynamic Data Blood Test Information CT Diagnostic Evaluation Albumin / Globulin Ratio (A / G ratio) Urea Nitrogen and Creatinine Ratio (BUN & Creatinine ratio) Hematocrit Nucleated RBCs CT Imaging Interpretation Data Absolute Lymphocyte Count (ALC) Atypical Lymphocytes Hemoglobin Serum Phosphorus (P, Serum) Alkaline Phosphatase (ALP) Abnormal Lymphoid Immature Cells Prothrombin Time (PT(INR)) Alanine Aminotransferase (AALT) CEA (Carcinoembryonic Antigen) Marker Serum Potassium (K, Serum) Plasma Cells Absolute Neutrophil Count Count, ANC) Total CEA in Serum (CEA, Total, Serum) Lactate Dehydrogenase (LD, Serum) Platelets (Platelet Count) Activated Partial Thromboplastin Time (aPTT) Cholesterol Lymphocyte Total Protein (Protein, Total) Aspartate Aminotransferase (AST) Chloride in Serum (Cl,Serum) Mean Red Blood Cell Hemoglobin (MCH) Red Blood Cells (RBC Count) Albumin Band Neutrophil Mean Red Blood Cell Hemoglobin Concentration (MCHC) Segmented Neutrophil Basophil Erythrocyte Sedimentation Rate (ESR) Mean Red Blood Cell Volume (MCV) Uric Acid Bilirubin Eosinophil Metamyelocyte White Blood Cells (WBC Count) Blast Estimated Glomerular Filtration Rate (Estimated GFR) Monocyte Creatinine Globulin Myelocyte Urea Nitrogen (BUN) Fasting Blood Glucose (Glucose, Fasting) Serum Sodium (Na, Serum),

[0051] Dynamic data refers to data extracted by performing examinations at regular intervals (e.g., 1 month, 3 months, 6 months, 1 year). All dynamic data consists of continuous variables. CT interpretation information refers to diagnostic information derived from an expert analyzing CT images. CT interpretation information can be stored as text data (sentences) separately from the CT images. Therefore, CT interpretation information is composed of specific sentences. Meanwhile, medical images do not necessarily have to be CT scans. Medical image interpretation information derived from other types of medical images may also be utilized. However, for the sake of convenience in the following explanation, the description will be based on CT interpretation information.

[0052] The learning device can extract static data for the items of Table 1 and dynamic data for the items of Table 2 from the patient data.

[0053] In addition, the learning device can consistently preprocess the extracted static and dynamic data. The learning device can convert the static and dynamic data into numerical values ​​that can be input into a prediction model.

[0054] The learning device can perform normalization or one-hot vector encoding depending on the data type. (1) The learning device can normalize continuous data, such as blood test results, to a range of [0,1]. (2) The learning device can perform one-hot vector encoding for categorical data. For example, hormone receptor data can be set to a value such as 0 or 1 depending on whether it is negative or positive. Pathological T-stage data can be set to any one of 0, 1, 2, 3, and 4 depending on the stage.

[0055] The preprocessing process for dynamic data is described. (1) The learning device can divide data collected over a certain period into fixed unit cycles and set numerical values ​​for each unit cycle. For example, the learning device can determine one value for 36 months of data in units of one month. If there are multiple numerical values ​​in a unit period (e.g., one month), the learning device can determine the value measured at the later point in time during that unit period as the representative value of that unit interval. If there is no value measured in a specific unit period (e.g., March) or if it is missing, the learning device can set the representative value of the previous unit period (e.g., February) as the representative value of the current unit period. The learning device can also set label values ​​for unit time periods over a certain period. (2) The learning device can set label values ​​for dynamic data based on the recurrence interval to be predicted (e.g., one year). For example, if the data is of a patient who has recurred lung cancer in April 24, the label value for the dynamic data from May 22 to April 23 may be non-recurrence (0), and the label value for the dynamic data from May 24 onwards may be recurrence (1).

[0056] Meanwhile, the learning device may also extract and use only specific sentences of interest from among the CT interpretation sentences stored in the EMR.

[0057] The learning device stores the values ​​of significant factors (normalized or preprocessed values) and label values ​​for the population as training data. The label values ​​include whether lung cancer has recurred. The training dataset contains pairs of significant factor values ​​and label values ​​for a number of patients. The training DB can store the constructed training dataset.

[0058] The learning device can build a predictive model for predicting lung cancer recurrence using a learning data set (220). The predictive model may be a model for predicting progression-free survival.

[0059] A prediction model can be any one of various types of models. A prediction model can be any one of machine learning models. There are various types of machine learning models. For example, machine learning models include decision trees, RF (random forest), KNN (K-nearest neighbor), Naive Bayes, SVM (support vector machine), ANN (artificial neural network), and regression models. The structure of prediction models will be discussed later.

[0060] The learning device selects static and dynamic data from a single set of training data and inputs them into the prediction model. The learning device updates the model's parameters by comparing the probability values ​​output by the prediction model with the label values. The computer device repeats the process of training the prediction model using multiple sets of training data.

[0061] Figure 3 is an example of a prediction model (300) for predicting lung cancer recurrence.

[0062] The prediction model (300) includes a first encoder (310) that receives static data and encodes it, a second encoder (320) that encodes CT interpretation sentences, a third encoder (330) that receives basic dynamic data and an embedding vector output by the second encoder and encodes them integrally, and a classifier (340) that receives features (vector values) output by the first encoder (310) and features (vector values) output by the third encoder (330) and performs inference.

[0063] The first encoder (310) may utilize any one of various model structures for embedding input values. For example, the first encoder (310) may include a dense layer composed of convolution blocks. The first encoder (310) may receive preprocessed static data.

[0064] The second encoder (320) receives a CT interpretation sentence as input and outputs an embedding vector. The second encoder (320) corresponds to the configuration of an encoder in a natural language processing model. Therefore, any one of various types of structures can be applied to the second encoder (320). For example, the second encoder (320) can use the encoder of BERT. The second encoder (320) embeds words extracted from the CT interpretation sentence on a word-by-word basis. The second encoder (320) can simply combine the vectors of all embedded words and transmit them to the third encoder (330).

[0065] The second encoder (320) embeds CT interpretation sentences collected over a certain period of time. It is assumed that the certain period of time is 36 months. At this time, the certain period of time can be divided into certain unit intervals. It is assumed that the unit interval is 1 month. Time series data was also divided according to these criteria in the actual model construction. The second encoder (320) can embed CT interpretation sentences collected every month for 36 months.

[0066] The third encoder (330) may utilize any one of various model structures capable of embedding time series data. For example, the third encoder (330) may use a transformer-based encoder used for text encoding. The second encoder (330) may receive preprocessed time series dynamic data. The second encoder (330) receives time series data collected over a certain time interval. For example, the second encoder (330) may receive dynamic data (t1 to t36) over a period of 36 months. At this time, the dynamic data for each unit interval (e.g., 1 month) may include the dynamic data items of Table 2.

[0067] The third encoder (330) embeds basic dynamic data (blood test information) of a specific unit interval. The third encoder (330) can perform positional encoding by concatenating the embedding vector of the basic dynamic data and the embedding vector output by the second encoder (320). When described based on t1, the third encoder (330) performs positional encoding by concatenating the embedding vector of the basic dynamic data of t1 and the vector containing the CT interpretation sentence of t1.

[0068] The third encoder (330) embeds dynamic data for each unit interval. The third encoder (330) performs position encoding for the embeddings of multiple unit intervals (36). That is, the third encoder (330) processes time-series data by encoding based on the position over time.

[0069] The classifier (340) receives the features (vector values) output by the first encoder (310) and the features (vector values) output by the third encoder (330) and outputs the possibility of lung cancer recurrence. The classifier (340) is a neural network that performs a certain inference based on the input features. For example, the classifier (340) can be implemented as a dense layer such as a fully connected layer.

[0070] The prediction model (300) must be trained using the aforementioned training data. During the training process, the training device performs preprocessing on static data (Table 1) and dynamic data (Table 2). The preprocessing process is as described above.

[0071] The learning device inputs static data and dynamic data for a single patient from the learning data into the first encoder (310), the second encoder (320), and the third encoder (330), respectively, to embedding features, and receives the output values ​​of the first encoder (310) and the third encoder (330) to compare the output values ​​output by the classifier (340) with the label values ​​to train the prediction model (300) to output the correct answer. The overall operation of the model is as described in FIG. 3.

[0072] The prediction model (300) is constructed as a model that predicts the likelihood of lung cancer recurrence within a specific period based on a specific point in time according to training data. At this time, the specific period may vary, such as 1 year, 2 years, or 3 years. The prediction model (300) may be constructed as a separate model depending on the period being predicted.

[0073] The analysis device can predict the likelihood of lung cancer recurrence for a specific patient using a learned prediction model (300). The analysis device inputs static data and dynamic data of the specific patient being analyzed into a first encoder (310), a second encoder (320), and a third encoder (330), respectively, to embed features, and receives the output values ​​of the first encoder (310) and the third encoder (330) to predict the likelihood of lung cancer recurrence based on the probability values ​​output by a classifier (340).

[0074] Furthermore, the analysis device predicts the likelihood of lung cancer recurrence at a future point in time based on past data (e.g., static data prior to surgery and time-series dynamic data after surgery). Therefore, by adjusting the time point of the input data, the analysis device can also predict the patient's progression-free survival period. For example, assume that the analysis device inputs patient data for one year to predict the likelihood of lung cancer recurrence within one year. In this case, the analysis device can predict the likelihood of lung cancer recurrence and the time point at which lung cancer recurrence is possible by adjusting the endpoint time point of the one-year period, and simultaneously predict the progression-free survival period.

[0075] Alternatively, as described above, the prediction model may be constructed separately according to the time interval for predicting the likelihood of lung cancer recurrence. The analysis device may predict the likelihood of lung cancer recurrence using a model that predicts recurrence within one year, predict the likelihood of lung cancer recurrence using a model that predicts recurrence within two years, and predict the likelihood of lung cancer recurrence using a model that predicts recurrence within three years, and derive an approximate progression-free survival period.

[0076] The researcher constructed a predictive model to predict lung cancer recurrence within one year and verified its performance. In the validation data, the lung cancer stages of patients (Stages 1–3 and overall) were distinguished, and the performance of the model was verified for each stage. The validation results are shown in Table 3 below.

[0077] StageAUROCThresholdSensitivitySpecificityF1-scoretotal0.8540.60571.3%86.0%27.8%Stage I0.8720.49777.8%71.3%22.0%Stage II0.7370.79965.5%16.6%27.3%Stage III0.7240.88554.4%27.8%42.7%

[0078] Performance metrics used included AUROC (Area Under ROC), Threshold (the threshold value for the probability of predicting the positive class), Sensitivity, Specificity, and F1 score. As shown in Table 3, the prediction model demonstrated a performance of 0.85 or higher across all stages. For patients with advanced lung cancer (Stages 2 and 3), performance dropped somewhat, but the prediction model still demonstrated significant significance.

[0079] Figure 4 shows the results of comparing the progression-free survival period with the results predicted by the prediction model. Figure 4 corresponds to the survival curve based on the Kaplan-Meier analysis. In Figure 4, blue represents the group predicted to have a low probability of lung cancer recurrence within one year (recurrence probability value ≤ 0.3), green represents the group predicted to have a moderate probability of lung cancer recurrence within one year (0.3 < recurrence probability value ≤ 0.6), and red represents the group predicted to have a high probability of lung cancer recurrence within one year (recurrence probability value ≥ 0.6). Figure 4(A) shows the progression-free survival period for all patients in the validation data. Figure 4(B) shows the progression-free survival period for patients with stage 1 lung cancer. Figure 4(C) shows the progression-free survival period for patients with stage 2 lung cancer. Figure 4(D) shows the progression-free survival period for patients with stage 3 lung cancer. Looking at Figure 4, there is a significant causal relationship between the results predicted by the prediction model and the survival period of the patient. Therefore, it can be seen that the likelihood of lung cancer recurrence predicted by the prediction model is significant.

[0080] FIG. 5 is an example of an analysis device (400) for predicting lung cancer recurrence. The analysis device (400) corresponds to the analysis device described above (130 and 140 in FIG. 1). The analysis device (400) can be physically implemented in various forms. For example, the analysis device (400) can take the form of a computer device such as a PC, a server of a network, a chipset dedicated to data processing, etc.

[0081] The analysis device (400) may include a storage device (410), a memory (420), a computation device (430), an interface device (440), a communication device (450), and an output device (460).

[0082] The storage device (410) can store the patient's static data and dynamic data. The static data is as described in Table 1. The dynamic data is as described in Table 2. The dynamic data is data collected at regular intervals over a certain time interval (e.g., 1 year, 2 years, and 3 years, etc.).

[0083] The storage device (410) can store the aforementioned prediction model. The prediction model is as described in FIG. 3.

[0084] The storage device (410) can store the predicted results of the likelihood of lung cancer recurrence and / or the predicted results of progression-free survival for the patient.

[0085] The memory (420) can store data and information generated during the process of the analysis device predicting lung cancer recurrence and / or progression-free survival.

[0086] The interface device (440) is a device that receives certain commands and data from the outside.

[0087] The interface device (240) may be configured to receive user commands. Alternatively, the interface device (240) may be configured to receive certain data from an external storage medium. Alternatively, the interface device (240) may be configured to receive data received through a communication device (250).

[0088] The interface device (440) can receive static and dynamic data of the patient from a physically connected input device or external storage device.

[0089] The interface device (440) may also transmit the patient's lung cancer recurrence prediction result and / or progression-free survival prediction result to an external object.

[0090] The communication device (450) means a configuration that receives and transmits certain information through a wired or wireless network.

[0091] The communication device (450) can receive static data and dynamic data of the patient from an external object.

[0092] Alternatively, the communication device (450) may transmit the patient's lung cancer recurrence prediction result and / or progression-free survival prediction result to an external object such as a user terminal.

[0093] The output device (460) is a device that outputs certain information.

[0094] The output device (460) can output interfaces required for the analysis process, results of predicting the likelihood of lung cancer recurrence, results of predicting progression-free survival, etc.

[0095] The computing device (430) preprocesses the static data of the patient to be analyzed. The preprocessing process is as described above.

[0096] Static data includes general clinical information, pathological information, and genetic information (variant information). Specifically, (1) general clinical information may include age, sex, height, weight, ECOG PS score, smoking history, FVC (forced vital capacity), FEV1 (forced expiratory volume in one second), and type of surgery. (2) Pathological information may include TNM stage, lesion location, pathological type, lesion pattern, degree of differentiation, vascular invasion, lymphatic invasion, perineal invasion, pleural invasion, resection margin, presence of metastasis, and tumor size. (3) Genetic information may include EGFR (Epidermal Growth Factor Receptor) mutations, PD-L1 expression status, ALK mutations, and other mutations (L858R, DEL19, etc.).

[0097] The computing device (430) preprocesses the basic dynamic data of the patient to be analyzed.

[0098] Basic dynamic data is all data quantified into a fixed amount or value. Basic dynamic data includes blood test information.

[0099] Blood test information includes albumin / globulin ratio, absolute lymphocyte count, alkaline phosphatase, alanine aminotransferase, absolute neutrophil count, activated partial thromboplastin C, aspartate aminotransferase, albumin, basophils, bilirubin, blast cells, creatinine, urea nitrogen, urea nitrogen to creatinine ratio, atypical lymphocytes, abnormal lymphatic system, CEA (Carcinoembryonic antigen) marker, total serum CEA, cholesterol, serum chloride, double-banded neutrophils, erythrocyte sedimentation rate, eosinophils, estimated glomerular filtration rate, globulin, fasting blood glucose, hematocrit, hemoglobin, immature cells, serum potassium, lactate dehydrogenase, lymphocytes, mean erythrocyte hemoglobin, mean erythrocyte hemoglobin concentration, mean erythrocyte volume, retromyelocytes, monocytes, myelocytes, serum sodium. It includes nucleated red blood cells, serum phosphorus, prothrombin time, plasma cells, platelets, total protein, red blood cells, neutrophils, uric acid, and white blood cells.

[0100] The computing device (430) can extract some sentences of interest from the CT interpretation sentences in advance. The sentences of interest may be sentences containing specific keywords.

[0101] The computing device (430) inputs the preprocessed static data, basic dynamic data, and CT interpretation sentences into the prediction model. The operation of the prediction model is as described in FIG. 3.

[0102] The computing device (430) can predict the likelihood of lung cancer recurrence based on the value output by the prediction model. The computing device (430) can predict the likelihood of lung cancer recurrence within a certain period (1 year, 2 years, or 3 years).

[0103] Alternatively, the computing device (430) can predict the progression-free survival period based on the value output by the prediction model.

[0104] The computing device (430) may be a device such as a processor, AP, or a chip with a program embedded in it that processes data and performs certain operations.

[0105] A deep learning-based method for predicting lung cancer recurrence using time series data includes the steps of: receiving static data collected from a patient prior to surgery and time series dynamic data collected after surgery; the analysis device preprocessing basic dynamic data among the static data and the time series dynamic data; the analysis device inputting medical image interpretation sentences included in the preprocessed static data, the preprocessed basic dynamic data, and the time series dynamic data into a learned prediction model; and the analysis device predicting the likelihood of lung cancer recurrence in the patient within a certain period based on the value output by the prediction model.

[0106] An analysis device for predicting lung cancer recurrence includes an interface device that receives static data collected from a patient prior to surgery and time-series dynamic data collected after surgery; a storage device that stores a prediction model for predicting recurrence in a subject who has undergone lung cancer surgery; and a computing device that preprocesses basic dynamic data among the static data and time-series dynamic data, inputs medical image interpretation sentences included in the preprocessed static data, the preprocessed basic dynamic data, and the time-series dynamic data into the prediction model, and predicts the lung cancer prognosis of the patient within a certain period based on the value output by the prediction model.

[0107] Static data is data obtained once from the patient and may include general clinical information, pathological information, and genetic information.

[0108] Static data may include (1) general clinical information including age, sex, height, weight, ECOG PS score, smoking history, FVC (forced vital capacity), FEV1 (forced expiratory volume in one second) and type of surgery, (2) pathological information including TNM stage, lesion location, pathological type, lesion pattern, degree of differentiation, vascular invasion, lymphatic invasion, perineal invasion, pleural invasion, resection margin, presence of metastasis and tumor size, and (3) genetic information including EGFR (Epidermal Growth Factor Receptor) mutation, PD-L1 expression status and ALK mutation.

[0109] Time series dynamic data is data collected at regular intervals over a certain time period, and may include blood test information and the medical image interpretation sentence.

[0110] The basic dynamic data include albumin / globulin ratio, absolute lymphocyte count, alkaline phosphatase, alanine aminotransferase, absolute neutrophil count, activated partial thromboplastin, aspartate aminotransferase, albumin, basophils, bilirubin, blast cells, creatinine, urea nitrogen, urea nitrogen to creatinine ratio, atypical lymphocytes, abnormal lymphatic system, CEA (Carcinoembryonic antigen) marker, total serum CEA, cholesterol, serum chloride, double-banded neutrophils, erythrocyte sedimentation rate, eosinophils, estimated glomerular filtration rate, globulin, fasting blood glucose, hematocrit, hemoglobin, immature cells, serum potassium, lactate dehydrogenase, lymphocytes, mean erythrocyte hemoglobin, mean erythrocyte hemoglobin concentration, mean erythrocyte volume, retromyelocytes, monocytes, myelocytes, serum sodium. It may include nucleated red blood cells, serum phosphorus, prothrombin time, plasma cells, platelets, total protein, red blood cells, neutrophils, uric acid, and white blood cells.

[0111] The prediction model includes a first encoder that receives the preprocessed static data; a second encoder, which is a natural language processing model that receives the medical image interpretation sentence and outputs a first embedding vector; a transformer-based third encoder that receives the preprocessed basic dynamic data to generate a second embedding vector and performs encoding by combining the first embedding vector and the second embedding vector according to a time interval; and a classifier that receives the features output by the first encoder and the features output by the third encoder together and outputs a lung cancer recurrence probability value.

[0112] The analysis device or computing device can predict the likelihood of lung cancer recurrence by changing the time interval of the time series dynamic data, and predict the progression-free survival period for the patient.

[0113] The methods according to the embodiments described in the specification of the present disclosure may be implemented in the form of hardware, software, or a combination of hardware and software.

[0114] When implemented in software, a computer-readable storage medium may be provided for storing one or more programs (software modules). One or more programs stored in the computer-readable storage medium are configured for execution by one or more processors within an electronic device. One or more programs include instructions that cause the electronic device to execute methods according to embodiments described in the specification of this disclosure.

[0115] In addition, the method for predicting lung cancer recurrence or the method for predicting progression-free survival after lung cancer surgery as described above may be implemented as a program (or application) comprising an executable algorithm that can be executed on a computer. The program may be provided by being stored on a transitory or non-transitory computer-readable medium.

[0116] A non-transient readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, the various applications or programs described above may be stored and provided on a non-transient readable medium such as a CD, DVD, hard disk, Blu-ray disc, USB, memory card, ROM (read-only memory), PROM (programmable read-only memory), EPROM (Erasable PROM, EPROM), EEPROM (Electrically EPROM), or flash memory.

[0117] Transient readable media refers to various types of RAM such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synclink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM).

[0118] Additionally, the program may be stored on an attachable storage device that can be accessed via a communication network such as the Internet, Intranet, LAN (local area network), WAN (wide area network), or SAN (storage area network), or a combination thereof. Such a storage device may be connected to a device performing an embodiment of the present disclosure through an external port. Additionally, a separate storage device on a communication network may be connected to a device performing an embodiment of the present disclosure.

[0119] The embodiments and drawings attached to this specification merely clearly illustrate a part of the technical ideas included in the aforementioned technology, and it is self-evident that all variations and specific embodiments that can be easily inferred by a person skilled in the art within the scope of the technical ideas included in the specification and drawings of the aforementioned technology are included within the scope of the rights of the aforementioned technology.

Claims

1. A step in which an analysis device receives static data collected from a patient prior to surgery and time-series dynamic data collected after surgery; The above analysis device preprocesses basic dynamic data among the static data and the time-series dynamic data; The step of the analysis device inputting medical image interpretation sentences included in the preprocessed static data, the preprocessed basic dynamic data, and the time series dynamic data into a learned prediction model; and A deep learning-based lung cancer recurrence prediction method using time series data, comprising the step of the analysis device predicting the likelihood of lung cancer recurrence in the patient within a certain period based on the value output by the prediction model.

2. In Paragraph 1, The above static data is data obtained once from the patient, and is a deep learning-based method for predicting lung cancer recurrence using time-series data including general clinical information, pathological information, and genetic information.

3. In Paragraph 1, The above time series dynamic data is data collected at regular intervals during a certain time interval, and a deep learning-based lung cancer recurrence prediction method using time series data including blood test information and medical image interpretation sentences.

4. In Paragraph 1, A deep learning-based method for predicting lung cancer recurrence using time-series data including the above static data, (1) general clinical information including age, sex, height, weight, ECOG PS score, smoking history, FVC (forced vital capacity), FEV1 (forced expiratory volume in one second) and type of surgery, (2) pathological information including TNM stage, lesion location, pathological type, lesion pattern, degree of differentiation, vascular invasion, lymphoid invasion, perineal invasion, pleural invasion, resection margin, presence of metastasis and tumor size, and (3) genetic information including EGFR (Epidermal Growth Factor Receptor) mutation, PD-L1 expression status and ALK mutation.

5. In Paragraph 1, The above basic dynamic data Albumin / globulin ratio, absolute lymphocyte count, alkaline phosphatase, alanine aminotransferase, absolute neutrophil count, activated partial thromboplastin, aspartate aminotransferase, albumin, basophil, bilirubin, blast cell, creatinine, urea nitrogen, urea nitrogen to creatinine ratio, atypical lymphocyte, abnormal lymphatic system, CEA (Carcinoembryonic antigen) marker, total CEA in serum, cholesterol, serum chloride, double-banded neutrophil, erythrocyte sedimentation rate, eosinophil, estimated glomerular filtration rate, globulin, fasting blood glucose, hematocrit, hemoglobin, immature cells, serum potassium, lactate dehydrogenase, lymphocyte, mean erythrocyte hemoglobin, mean erythrocyte hemoglobin concentration, mean erythrocyte volume, retromyelocyte, monocyte, myelocyte, serum sodium, nucleated red blood cells, serum A deep learning-based method for predicting lung cancer recurrence using time series data including phosphorus, prothrombin time, plasma cells, platelets, total protein, red blood cells, neutrophils, uric acid, and leukocytes.

6. In Paragraph 1, The above prediction model A first encoder that receives the above-mentioned preprocessed static data; A second encoder, which is a natural language processing model that receives the above medical image interpretation sentence as input and outputs a first embedding vector; A transformer-based third encoder that receives the above-mentioned preprocessed basic dynamic data as input, generates a second embedding vector, and performs encoding by combining the first embedding vector and the second embedding vector according to a time interval, and A classifier comprising a feature output by the first encoder and a feature output by the third encoder, which receives both as input and outputs a lung cancer recurrence probability value. Deep learning-based lung cancer recurrence prediction method using time series data.

7. In Paragraph 1, The above analysis device predicts the possibility of lung cancer recurrence while changing the time interval of the time series dynamic data, and is a deep learning-based lung cancer recurrence prediction method using time series data that predicts the progression-free survival period for the patient.

8. An interface device that receives static data collected from the patient prior to surgery and time-series dynamic data collected after the surgery; A storage device for storing a predictive model for predicting recurrence in subjects who have undergone lung cancer surgery; and An analysis device for predicting lung cancer recurrence comprising a computing device that preprocesses basic dynamic data among the static data and the time-series dynamic data, inputs medical image interpretation sentences included in the preprocessed static data, the preprocessed basic dynamic data, and the time-series dynamic data into the prediction model, and predicts the lung cancer prognosis of the patient within a certain period based on the value output by the prediction model.

9. In Paragraph 8, The above static data is data obtained once from the patient, and is an analysis device for predicting lung cancer recurrence that includes general clinical information, pathological information, and genetic information.

10. In Paragraph 8, The above time series dynamic data is data collected at regular intervals during a certain time interval, and an analysis device for predicting lung cancer recurrence including blood test information and the above medical image interpretation sentence.

11. In Paragraph 8, The above static data includes (1) general clinical information including age, sex, height, weight, ECOG PS score, smoking history, FVC (forced vital capacity), FEV1 (forced expiratory volume in one second) and type of surgery, (2) pathological information including TNM stage, lesion location, pathological type, lesion pattern, degree of differentiation, vascular invasion, lymphatic invasion, perineal invasion, pleural invasion, resection margin, presence of metastasis and tumor size, and (3) genetic information including EGFR (Epidermal Growth Factor Receptor) mutation, PD-L1 expression status and ALK mutation, an analysis device for predicting lung cancer recurrence.

12. In Paragraph 8, The above basic dynamic data Albumin / globulin ratio, absolute lymphocyte count, alkaline phosphatase, alanine aminotransferase, absolute neutrophil count, activated partial thromboplastin, aspartate aminotransferase, albumin, basophil, bilirubin, blast cell, creatinine, urea nitrogen, urea nitrogen to creatinine ratio, atypical lymphocyte, abnormal lymphatic system, CEA (Carcinoembryonic antigen) marker, total CEA in serum, cholesterol, serum chloride, double-banded neutrophil, erythrocyte sedimentation rate, eosinophil, estimated glomerular filtration rate, globulin, fasting blood glucose, hematocrit, hemoglobin, immature cells, serum potassium, lactate dehydrogenase, lymphocyte, mean erythrocyte hemoglobin, mean erythrocyte hemoglobin concentration, mean erythrocyte volume, retromyelocyte, monocyte, myelocyte, serum sodium, nucleated red blood cells, serum An analysis device for predicting lung cancer recurrence including phosphorus, prothrombin time, plasma cells, platelets, total protein, red blood cells, neutrophils, uric acid, and white blood cells.

13. In Paragraph 8, The above prediction model A first encoder that receives the above-mentioned preprocessed static data; A second encoder, which is a natural language processing model that receives the above medical image interpretation sentence as input and outputs a first embedding vector; A transformer-based third encoder that receives the above-mentioned preprocessed basic dynamic data as input, generates a second embedding vector, and performs encoding by combining the first embedding vector and the second embedding vector according to a time interval, and An analysis device for predicting lung cancer recurrence, comprising a classifier that receives together the features output by the first encoder and the features output by the third encoder and outputs a lung cancer recurrence probability value.

14. In Paragraph 8, The above computing device is an analysis device for predicting lung cancer recurrence that predicts the likelihood of lung cancer recurrence by changing the time interval of the above time series dynamic data and predicting the progression-free survival period for the patient.

Images