A method, system, device and storage medium for predicting the prognosis of multi-drug resistant tuberculosis

By combining CT scan data with handcrafted and deep learning features, a GRU network is used to dynamically monitor multidrug-resistant tuberculosis, solving the accuracy and real-time issues of existing methods and enabling more accurate prognostic prediction and early identification of high-risk patients.

CN120976114BActive Publication Date: 2026-04-10JIANGXI CHEST HOSPITAL (THIRD PEOPLES HOSPITAL OF JIANGXI PROVINCE)
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
JIANGXI CHEST HOSPITAL (THIRD PEOPLES HOSPITAL OF JIANGXI PROVINCE)
Filing Date
2025-07-23
Publication Date
2026-04-10

AI Technical Summary

Technical Problem

Existing methods for predicting the prognosis of multidrug-resistant tuberculosis rely on sputum culture transformation, which has low sensitivity, insufficient specificity, limited real-time performance and accuracy, and difficulty in effectively identifying treatment failure. Traditional imaging methods also have limitations in dynamic monitoring.

Method used

By combining handcrafted radiographic features and deep learning-derived imaging features, and integrating baseline, two-month, and six-month CT scan data through a gated relapse unit (GRU) network, a prognostic prediction model for multidrug-resistant tuberculosis is constructed to dynamically capture lesion changes.

Benefits of technology

It improves the accuracy and clinical applicability of predicting the treatment of multidrug-resistant tuberculosis, enables early identification of high-risk patients, supports personalized treatment decisions and optimized intervention strategies, and has higher sensitivity and specificity, making it suitable for high-burden countries with limited resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120976114B_ABST
    Figure CN120976114B_ABST
Patent Text Reader

Abstract

The present application belongs to the technical field of disease prognosis prediction, and aims at the problem that the existing prognosis monitoring method for multidrug-resistant tuberculosis patients has certain limitations. The present application provides a multidrug-resistant tuberculosis prognosis prediction method, which integrates the manually made radiology features and the deep learning derived imaging features extracted from the baseline, two-month and six-month continuous CT scans, adopts a gated recurrent unit (GRU) network to process the data, and combines a multidrug-resistant tuberculosis prognosis prediction system to predict the treatment results of multidrug-resistant tuberculosis patients. The prediction method provided by the present application can accurately identify high-risk patients in the early stage of the treatment process, and provides strong support for personalized treatment decisions, optimization of intervention timing and reasonable allocation of public health resources. Moreover, the present application has higher sensitivity and specificity, and improves the potential of the accuracy of multidrug-resistant tuberculosis treatment result prediction and risk stratification.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of disease prognosis prediction, and particularly relates to a multidrug-resistant tuberculosis prognosis prediction method, system, device and storage medium. BACKGROUND

[0002] Multidrug-resistant tuberculosis (MDR-TB) is a major public health challenge due to its complex treatment regimen, high treatment failure rate, severe drug adverse reactions and significant disease transmission risk. Timely identification and accurate prediction of treatment outcomes are crucial for optimizing treatment strategies and improving patient management. Current clinical monitoring mainly relies on sputum culture conversion, which has high sensitivity but relatively low specificity, is susceptible to contamination and has a long reporting time, thus limiting its practical application.

[0003] In current clinical practice, sputum culture conversion (SCC) is still an important indicator for monitoring the treatment response of multidrug-resistant tuberculosis patients. A number of studies have shown that SCC at six months is closely related to treatment success, with a sensitivity of about 91%. However, its specificity is still relatively low, ranging from 56% to 58%, thus limiting its ability to accurately identify treatment failures. In addition, SCC relies on continuous high-quality sample collection, involves a long detection period, is susceptible to contamination and other technical factors, further limiting its real-time applicability and practicality. Other clinical predictors, such as fluoroquinolone resistance and early sputum smear non-conversion, have been identified as independent risk factors for poor treatment outcomes. Alene et al. developed a risk score-based prediction model based on these factors, achieving an AUC of 0.67-0.69. However, their overall predictive performance is still limited and shows instability in addressing biological and clinical heterogeneity among patients.

[0004] Recent advances in computed tomography (CT) imaging suggest that by providing detailed visualizations of lung pathology and structural abnormalities, it can help to strengthen the monitoring of disease progression, potentially overcoming these limitations. Due to its high-resolution visualization of lung structures, CT imaging plays an increasingly important role in the diagnosis and management of tuberculosis, particularly in complex cases such as MDR-TB. CT imaging can assess disease burden, lesion morphology, and detect potential structural complications in detail. In clinical practice, particularly in high-burden countries for tuberculosis, imaging follow-up has been a routine component of treatment monitoring, along with monthly sputum tests, with a particular emphasis on the first six months as the critical window for dynamic assessment. Emerging evidence further suggests that CT-based markers, including reduced metabolic activity, changes in lesion volume, CT scoring, and specific lesion characteristics, can serve as important imaging biomarkers for predicting treatment response in multidrug-resistant tuberculosis. Specifically, changes in cavitary lesions and consolidation show significant correlations in assessing treatment response in patients with multidrug-resistant tuberculosis. For example, the persistence of cavitary lesions at 6 months or 9 months has a high predictive accuracy for treatment outcomes, with AUCs of 0.702 and 0.818, respectively. Additionally, radiological features extracted from baseline CT cavities have an AUC of 0.839 in predicting SCC at 6 months, significantly outperforming traditional clinical models (AUC range: 0.525-0.688). Incorporating maximum lesion cross-sectional area measurements at baseline, two months, and six months into clinical models further improves predictive performance, achieving an AUC range of 0.869 to 0.920. However, these models exhibit imbalanced predictive capabilities, with significantly higher sensitivity (0.933-1.000) in identifying culture-conversion patients compared to specificity (0.130-0.478) in detecting non-conversion. Overall, these findings highlight the tremendous potential of time-series CT imaging in capturing disease progression and treatment response in the management of multidrug-resistant tuberculosis.

[0005] Spatiotemporal feature fusion has been widely applied in multi-timepoint CT imaging analysis to address the limitations of single-timepoint modeling in capturing dynamic disease processes, showing great potential in predicting treatment response in multidrug-resistant tuberculosis delta radiomics features that quantify changes between time points, requiring only ROI extraction without strict temporal alignment, have been shown to have stronger predictive power for treatment response and prognosis compared to baseline features in various diseases. Additionally, incorporating time intervals as adjustment factors can improve the performance of radiological models. While the application of radiomics, particularly through machine learning and deep learning methods, has emerged as a promising approach for quantitatively analyzing imaging features and their dynamic changes over treatment. Moreover, radiomics provides good interpretability by quantifying explicit lesion changes, but may face limitations in modeling complex and non-linear dynamic evolution. SUMMARY

[0006] The application aims to overcome the deficiencies of the prior art and provide a multidrug-resistant tuberculosis prognosis prediction method, system, device and storage medium.

[0007] The application integrates manually made radiological features and deep learning derived imaging features extracted from baseline, two-month and six-month continuous CT scans, and combines a gated recurrent unit (GRU) network to capture dynamic lesion changes, thereby improving the accuracy and clinical practicability of result prediction.

[0008] To achieve the above object, the technical scheme adopted by the application is:

[0009] A multidrug-resistant tuberculosis prognosis prediction method comprises the following steps:

[0010] S1, longitudinal CT scan image clinical data of multiple groups of multidrug-resistant tuberculosis patients are obtained, and are divided into a training set, an internal validation set and an external validation set;

[0011] S2, the tuberculosis lesion area and the lung parenchyma area on each CT scan image in the training set, the internal validation set and the external validation set are manually labeled and automatically segmented respectively, to obtain the tuberculosis lesion area and the lung parenchyma area; radiological features constructed manually and radiological features based on deep learning are extracted respectively, to obtain manually made radiological features and deep learning imaging features;

[0012] S3, the manually made radiological features and the deep learning imaging features extracted are respectively subjected to pairwise subtraction between different time points, to obtain manually made radiological incremental features and deep learning imaging incremental features; the manually made radiological features and the deep learning imaging features obtained are processed using a two-layer GRU network, to obtain hidden state vectors of the training set, the internal validation set and the external validation set respectively;

[0013] S4, the hidden state vectors of the training set, the manually made radiological incremental features and the deep learning imaging incremental features are used to train a constructed multidrug-resistant tuberculosis prognosis prediction model, and the hidden state vectors of the internal validation set and the external validation set, the manually made radiological incremental features and the deep learning imaging incremental features are used for verification respectively, to obtain a trained multidrug-resistant tuberculosis prognosis prediction model;

[0014] S5, the trained multidrug-resistant tuberculosis prognosis prediction model is used to predict the prognosis of a target multidrug-resistant tuberculosis patient.

[0015] Preferably, in step S1, the CT scan image clinical data of each group of patients comprises CT scan image clinical data of each patient before receiving the drug-resistant tuberculosis treatment, 2 months after the treatment, and 6 months after the treatment.

[0016] Preferably, step S2 comprises the following steps:

[0017] S21, independently performing twice annotation on the tuberculosis lesion area on each CT scan image in the training set, the internal validation set and the external validation set respectively;

[0018] S22, calculating the Dice similarity coefficient between the two annotations obtained on each CT scan image, and determining the tuberculosis lesion area of each CT scan image according to the calculated Dice similarity coefficient;

[0019] S23, extracting the artificially constructed radiomics features from the obtained tuberculosis lesion area to obtain the manually made radiological features;

[0020] S24, automatically segmenting the lung parenchyma area on each CT scan image in the training set, the internal validation set and the external validation set by using the deep learning model, and merging the left lung and right lung areas obtained by automatic segmentation into a unified area to determine the lung parenchyma area of each CT scan image;

[0021] S25, respectively extracting the deep learning-based radiomics features from the obtained tuberculosis lesion area and the lung parenchyma area to obtain the deep learning imaging features.

[0022] Preferably, in step S22, when the calculated Dice similarity coefficient is greater than or equal to 0.95, the two annotations are averaged to generate a reference ground truth, and the tuberculosis lesion area of each CT scan image is determined;

[0023] When the calculated Dice similarity coefficient is less than 0.95, the tuberculosis lesion area on the CT scan image is re-annotated, and the tuberculosis lesion area of each CT scan image is determined.

[0024] Preferably, step S3 comprises the following steps:

[0025] S31, by performing pair-wise subtraction on the extracted manually made radiological features and deep learning imaging features between different time points, respectively, to obtain manually made radiological incremental features and deep learning imaging incremental features;

[0026] S32, organizing the manually made radiomics features and deep learning imaging features extracted at different time points into time series respectively, and processing them using a two-layer GRU network to obtain hidden state vectors of the training set, the internal validation set and the external validation set, respectively.

[0027] Preferably, in step S4, the constructed multidrug-resistant tuberculosis prognosis prediction model is composed of two prediction modules;

[0028] One prediction module is MTBTPOS-FU, which is used for binary classification of favorable and unfavorable treatment results;

[0029] Another prediction module is MTBTPOS-SCF, which is used for stratifying multi-class treatment results into treatment success, treatment completion and treatment failure categories.

[0030] The application also provides a multidrug-resistant tuberculosis prognosis prediction system, which is realized by using the above-mentioned prediction method, comprising:

[0031] A data acquisition module is used to acquire a plurality of sets of CT scan image clinical data and divide them into a training set, an internal validation set and an external validation set;

[0032] A feature extraction module is used to manually label and automatically segment the tuberculosis lesion area and lung parenchyma area on each CT scan image in the training set, internal validation set and external validation set, respectively, to obtain the tuberculosis lesion area and lung parenchyma area; and extract the manually constructed radiomics features and deep learning-based radiomics features, respectively, to obtain the manually made radiomics features and deep learning imaging features;

[0033] A data calculation processing module is used to perform pairwise subtraction on the extracted manually made radiomics features and deep learning imaging features between different time points to obtain manually made radiomics incremental features and deep learning imaging incremental features; and process the obtained manually made radiomics features and deep learning imaging features using a two-layer GRU network to obtain hidden state vectors of the training set, internal validation set and external validation set, respectively;

[0034] A training and validation module is used to train the constructed multidrug-resistant tuberculosis prognosis prediction model by using the hidden state vectors, manually made radiomics incremental features and deep learning imaging incremental features of the training set, and validate the model by using the hidden state vectors, manually made radiomics incremental features and deep learning imaging incremental features of the internal validation set and external validation set, respectively, to obtain the trained multidrug-resistant tuberculosis prognosis prediction model;

[0035] A prediction module is used to perform multidrug-resistant tuberculosis prognosis prediction on a target multidrug-resistant tuberculosis patient by using the trained multidrug-resistant tuberculosis prognosis prediction model.

[0036] Preferably, the feature extraction module comprises:

[0037] The feature annotation module is configured to independently perform twice annotation on the tuberculosis lesion area of each CT scan image in the training set, the internal validation set and the external validation set respectively.

[0038] The tuberculosis lesion area determination module is configured to calculate the Dice similarity coefficient between the twice annotations obtained on each CT scan image, and determine the tuberculosis lesion area of each CT scan image according to the calculated Dice similarity coefficient.

[0039] The manual feature extraction module is configured to extract the manually constructed radiomics features from the obtained tuberculosis lesion area to obtain the manually made radiological features.

[0040] The lung parenchyma area determination module is configured to automatically segment the lung parenchyma area on each CT scan image in the training set, the internal validation set and the external validation set by using the deep learning model, and combine the automatically segmented left lung and right lung areas into a unified area to determine the lung parenchyma area of each CT scan image.

[0041] The deep learning feature extraction module is configured to extract the deep learning-based radiomics features from the obtained tuberculosis lesion area and the lung parenchyma area respectively to obtain the deep learning imaging features.

[0042] The application further provides a computer readable storage medium, which comprises a stored computer program; wherein the computer program controls a device where the computer readable storage medium is located to execute the steps of the multi-drug resistant tuberculosis prognosis prediction method when running.

[0043] The application further provides a terminal device, which comprises a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, and the processor realizes the steps of the multi-drug resistant tuberculosis prognosis prediction method when executing the computer program.

[0044] Compared with the prior art, the application has the following beneficial effects:

[0045] (1) The application focuses on the characteristic changes of disease evolution in the treatment of multidrug-resistant tuberculosis by conducting time-series CT imaging analysis at three key time points (baseline (before treatment), second month of treatment and sixth month of treatment). The selection of these time points is based on the natural course of the disease and the key stages of treatment. By selecting these representative time points, the application systematically captures the evolution of the disease from the initial lesion burden to the early dynamic changes and the mid-term treatment effect. The research results of the application further show that the prognosis prediction model for multidrug-resistant tuberculosis provided by the application is significantly better than the single time point model in predicting treatment response and final outcome, emphasizing the key value of dynamic imaging changes in disease monitoring;

[0046] (2) The prognosis prediction method for multidrug-resistant tuberculosis proposed in the application can non-invasively and dynamically capture the evolution of lung lesions for continuous disease monitoring throughout the treatment process. By extracting continuous imaging features, the prediction method can identify high-risk patients in the early stages of treatment, support timely treatment adjustment and optimized intervention strategies. In addition, the binary classification based on favorable treatment outcomes (FTO) and unfavorable treatment outcomes (UTO) is very suitable for treatment decision support, while the three risk stratifications - success, completion and failure - provide more detailed guidance for long-term prognosis evaluation and public health management. With the increasing popularity of CT imaging and the advancement of artificial intelligence algorithms, the prediction method of the application is expected to be widely used in high-burden countries and resource-limited environments, helping to achieve more accurate multidrug-resistant tuberculosis treatment and improve clinical outcomes.

[0047] (3) Compared with traditional sputum-based monitoring methods, the prediction method proposed in the application can accurately identify high-risk patients in the early stages of treatment, providing strong support for personalized treatment decisions, optimizing intervention timing and reasonably allocating public health resources. Moreover, it has higher sensitivity and specificity, improving the potential for accuracy in predicting treatment outcomes and risk stratification for multidrug-resistant tuberculosis, and providing valuable insights for early identification of disease progression. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 Flowchart for patient enrollment in the application and internal training cohort and external validation test cohort;

[0049] Figure 2 Development and validation diagram of the MTBTOPS model provided by the application; A is the overall design framework diagram of the development of MTBTOPS; B is the design diagram of the internal training cohort and external validation test cohort; C is the performance evaluation result diagram of MTBTOPS;

[0050] Figure 3 Framework diagram of the construction of 64 models in the application;

[0051] Figure 4 Fig. 1 is a diagram of prediction performance evaluation results of MTBTPOS-FU in models M1 to M8; A is a classification performance radar chart; B is a ROC curve chart; C is a calibration curve chart; D is a decision curve chart;

[0052] Figure 5 Fig. 2 is a diagram of prediction performance evaluation results of MTBTOPS-SCF in models M1 to M8; A is a classification performance radar chart; B is a classification report index heat map; C is a ROC curve chart; D is a confusion matrix chart;

[0053] Figure 6 Fig. 3 is a diagram of prediction performance comparison of MTBTOPS models, clinical features and 6-month SCC in the external validation cohort; A is a ROC curve of different methods identifying UTO patients; B is a calibration curve chart of MTBTOPS-FU and clinical feature models identifying UTO patients; C is a net clinical benefit chart of MTBTOPS-FU and clinical feature models; D is a radar chart of classification performance indicators of different methods identifying UTO patients; E is a confusion matrix chart of MTBTOPS-FU; F is a confusion matrix chart of MTBTOPS-SCF. DETAILED DESCRIPTION

[0054] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings of the embodiments of the present application. Figures 1 to 6 It should be apparent that the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0055] Embodiment 1

[0056] The embodiments of the present application provide a multidrug-resistant tuberculosis prognosis prediction method, comprising the following steps:

[0057] S1, obtaining multiple sets of longitudinal CT scan image clinical data of multidrug-resistant tuberculosis patients, and dividing them into a training set, an internal validation set and an external validation set; each set of CT scan image clinical data comprises CT scan image clinical data of each patient before receiving treatment for drug-resistant tuberculosis, 2 months after treatment and 6 months after treatment.

[0058] S2, manually labeling and automatically segmenting the tuberculosis lesion area and lung parenchyma area on each CT scan image in the training set, internal validation set and external validation set, respectively, to obtain the tuberculosis lesion area and lung parenchyma area; extracting manually constructed radiomics features and deep learning-based radiomics features, respectively, to obtain manually fabricated radiomics features and deep learning imaging features, which specifically include the following steps:

[0059] S21, independently perform twice annotation on the tuberculosis lesion area of each CT scan image in the training set, internal validation set and external validation set respectively;

[0060] S22, calculate the Dice similarity coefficient between the two annotations obtained on each CT scan image, and determine the tuberculosis lesion area of each CT scan image according to the calculated Dice similarity coefficient;

[0061] Specifically,

[0062] When the calculated Dice similarity coefficient is greater than or equal to 0.95, the two annotations are averaged to generate a reference ground truth, and the tuberculosis lesion area of each CT scan image is determined;

[0063] When the calculated Dice similarity coefficient is less than 0.95, the tuberculosis lesion area on the CT scan image is re-annotated, and the tuberculosis lesion area of each CT scan image is determined.

[0064] S23, extract the radiomics features constructed by artificial from the obtained tuberculosis lesion area, to obtain the handmade radiomics features;

[0065] S24, automatically segment the lung parenchyma area on each CT scan image in the training set, internal validation set and external validation set by the deep learning model, and combine the left lung and right lung areas of the automatic segmentation into a unified area, to determine the lung parenchyma area of each CT scan image;

[0066] S25, respectively extract the deep learning-based radiomics features from the obtained tuberculosis lesion area and lung parenchyma area, to obtain the deep learning imaging features.

[0067] S3, by performing pair-wise subtraction between the extracted handmade radiomics features and deep learning imaging features at different time points, respectively, to obtain handmade radiomics incremental features and deep learning imaging incremental features, and by processing the obtained handmade radiomics features and deep learning imaging features using two-layer GRU network, to obtain the hidden state vectors of the training set, internal validation set and external validation, specifically including the following steps:

[0068] S31, by performing pair-wise subtraction between the extracted handmade radiomics features and deep learning imaging features at different time points, respectively, to obtain handmade radiomics incremental features and deep learning imaging incremental features;

[0069] S32, the manually made radiomics features and deep learning imaging features extracted at different time points are respectively organized into time series, and are processed using a two-layer GRU network to obtain hidden state vectors of the training set, the internal validation set and the external validation set.

[0070] S4, the constructed multi-drug resistant tuberculosis prognosis prediction model is trained through the hidden state vectors of the training set, the manually made radiomics features and the deep learning imaging features, and is verified through the hidden state vectors of the internal validation set and the external validation set, the manually made radiomics features and the deep learning imaging features, to obtain the trained multi-drug resistant tuberculosis prognosis prediction model; the multi-drug resistant tuberculosis prognosis prediction model constructed in the embodiment of the application is composed of two prediction modules;

[0071] One prediction module is MTBTPOS-FU, which is used for binary classification of favorable treatment results and unfavorable treatment results.

[0072] The other prediction module is MTBTPOS-SCF, which is used for stratifying multi-class treatment results into treatment success, treatment completion and treatment failure categories.

[0073] S5, the trained multi-drug resistant tuberculosis prognosis prediction model is used to predict the prognosis of the target multi-drug resistant tuberculosis patient.

[0074] Embodiment 2

[0075] The application provides a multi-drug resistant tuberculosis prognosis prediction system, which is realized by using the prediction method in embodiment 1, and comprises:

[0076] A data acquisition module is configured to acquire a plurality of sets of CT scan image clinical data and divide the CT scan image clinical data into a training set, an internal validation set and an external validation set.

[0077] A feature extraction module is configured to manually label and automatically segment a tuberculosis lesion area and a lung parenchyma area on each CT scan image in the training set, the internal validation set and the external validation set, to obtain the tuberculosis lesion area and the lung parenchyma area; and extract manually constructed radiomics features and deep learning-based radiomics features to obtain manually made radiomics features and deep learning imaging features.

[0078] A data calculation processing module is configured to perform pairwise subtraction on the extracted manually made radiomics features and deep learning imaging features between different time points to obtain manually made radiomics features and deep learning imaging features; and process the obtained manually made radiomics features and deep learning imaging features using a two-layer GRU network to obtain hidden state vectors of the training set, the internal validation set and the external validation set.

[0079] The training verification module is configured to train the constructed multidrug-resistant tuberculosis prognosis prediction model by using the hidden state vectors of the training set, the manually created radiomic features and the deep learning imaging features, and to verify the multidrug-resistant tuberculosis prognosis prediction model by using the hidden state vectors of the internal verification set and the external verification set, respectively, to obtain the trained multidrug-resistant tuberculosis prognosis prediction model.

[0080] The prediction module is configured to perform multidrug-resistant tuberculosis prognosis prediction on a target multidrug-resistant tuberculosis patient by using the trained multidrug-resistant tuberculosis prognosis prediction model.

[0081] In the embodiment of the present application, the feature extraction module comprises:

[0082] The feature annotation module is configured to independently perform twice annotation on the tuberculosis lesion area of each CT scan image in the training set, the internal verification set and the external verification set, respectively.

[0083] The tuberculosis lesion area determination module is configured to calculate the Dice similarity coefficient between the twice annotations obtained on each CT scan image, and determine the tuberculosis lesion area of each CT scan image according to the calculated Dice similarity coefficient.

[0084] The manual feature extraction module is configured to extract the manually created radiomic features from the obtained tuberculosis lesion area to obtain the manually created radiomic features.

[0085] The lung parenchyma area determination module is configured to automatically segment the lung parenchyma area on each CT scan image in the training set, the internal verification set and the external verification set by using the deep learning model, and combine the automatically segmented left lung and right lung areas into a unified area to determine the lung parenchyma area of each CT scan image.

[0086] The deep learning feature extraction module is configured to extract the deep learning-based radiomic features from the obtained tuberculosis lesion area and the lung parenchyma area, respectively, to obtain the deep learning imaging features.

[0087] Embodiment 3

[0088] The embodiment of the present application provides a computer readable storage medium, which comprises a stored computer program; wherein the computer program controls the device where the computer readable storage medium is located to execute the steps of the multidrug-resistant tuberculosis prognosis prediction method of embodiment 1 when running.

[0089] Embodiment 4

[0090] This invention provides a terminal device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the steps of the multidrug-resistant tuberculosis prognosis prediction method of Embodiment 1.

[0091] The following is a detailed description of the prognostic prediction method for multidrug-resistant tuberculosis provided in Embodiment 1 of the present invention.

[0092] I. Research Design

[0093] 1. Data Collection

[0094] like Figure 1 As shown, the data for this study were obtained from 265 eligible multidrug-resistant tuberculosis (MDR-TB) patients between January 1, 2021 and September 30, 2024. The inclusion criteria for these patients were as follows: (1) tuberculosis patients with resistance to isoniazid and rifampin confirmed by Mycobacterium tuberculosis culture and DST; (2) patients aged between 18 and 75 years; and (3) patients who underwent a CT scan before starting treatment for drug-resistant tuberculosis and then underwent two additional scans during treatment, one at month 2 and the other at month 6. To ensure data quality, exclusion criteria were used, including: (1) incomplete images, poor image quality, or non-chest CT scans; (2) patients lost to follow-up, thus making treatment outcome assessment impossible; (3) patients with malignant tumors or other serious lung diseases; and (4) patients with a history of second-line anti-tuberculosis drug treatment.

[0095] This invention uses 198 multidrug-resistant tuberculosis (MDR-TB) patients treated at Centers 1 and 2 between January 1, 2021 and September 30, 2022 as the internal model development cohort, and 67 MDR-TB patients collected at Center 3 between January 1, 2022 and December 31, 2022 as the external testing cohort. The model development cohorts are randomly assigned to the training and internal validation sets in a 4:1 ratio to ensure even distribution of data between the two hospitals in the two subsets. The MDR-TB patients included in the study primarily received individualized anti-tuberculosis treatment regimens, managed by qualified physicians according to relevant guidelines from the World Health Organization (WHO) and China, with a total treatment duration of 18 to 20 months. Sociodemographic information, clinical data, laboratory results, and imaging studies were collected from the electronic medical records of the three hospitals, including each patient's age, sex, place of residence, smoking habits, alcohol consumption habits, tuberculosis treatment history, comorbidities, sputum smears, sputum cultures, GeneXpert results, drug sensitivity test results, and CT scans.

[0096] 2. Evaluation of treatment outcomes

[0097] Treatment outcomes were assessed by two qualified physicians according to patient follow-up data according to World Health Organization guidelines, the assessment criteria for treatment outcomes are shown below:

[0098] (1) Treatment success: completion of prescribed treatment without evidence of failure and three consecutive negative cultures at least 30 days apart after the intensive phase;

[0099] (2) Treatment completion: completion of prescribed treatment without evidence of failure but no record of three consecutive negative cultures at least 30 days apart after the intensive phase;

[0100] (3) Treatment failure: lack of conversion at the end of the intensive phase, return to positive culture after initial conversion during treatment, additional acquired resistance to drugs in a multidrug-resistant tuberculosis regimen, or serious drug adverse reactions, or death for any reason during treatment.

[0101] According to the above assessment criteria, in the training cohort, 123 patients achieved treatment success, 48 patients completed treatment, and 27 patients experienced treatment failure. In the external validation test cohort, 31 patients achieved treatment success, 17 patients completed treatment, and 19 patients experienced treatment failure.

[0102] 3. CT image acquisition and image preprocessing

[0103] All patients received standardized CT scans before treatment for drug-resistant tuberculosis, 2 months after treatment, and 6 months after treatment. The 3 CT scans of each person formed a longitudinal imaging sequence. Two hospitals in the training cohort used GE BrightSpeed 16-row 32-slice spiral CT scans (tube voltage: 120 kV, tube current: 150 mA, pitch: 1.75, slice thickness: 5.0 mm, field of view: 450 mm) and Philips Ingenuity spiral CT scans (tube voltage: 120 kV, tube current: 216 mA, pitch: 1.38, slice thickness: 5.0 mm, field of view: 450 mm), respectively. The hospital in the external validation test cohort used GE Optima 62-slice 124-row spiral CT scans (tube voltage: 120 kV, tube current: 248 mA, pitch: 1.016, slice thickness: 5.0 mm, field of view: 450 mm). The scanning range was from the upper edge of the frontal sinus to the lower edge of the clavicular sternal end. In the CT scans, SR, 2D longitudinal chest X-rays, and 2 to 3 axial image sequences could be obtained, and only the axial images were used for training and validation of the prediction model established by the present application, as well as lung parenchyma segmentation and image interpretation. The number of images in the axial image sequence varied from 32 to 221 according to the slice interval, and a sequence with a slice thickness of 5 mm was selected for further processing in each scan. The eligible CT scan images were resampled to 1*1*1. In order to accurately capture the changes of lesions at different stages of treatment, the Elastix registration module in the 3D Slicer software was used for registration from the follow-up CT to the pre-treatment CT, and the B-spline interpolator was used for smoothing the deformed images. In order to improve the generalization ability of the model, standard image enhancement techniques were applied to the training cohort, including flipping, translation, rotation, and deformation.

[0104] 4. Radiological feature extraction and temporal modeling

[0105] In order to accurately label the lesion area of patients with multidrug-resistant tuberculosis (MDR-TB), two radiologists with more than ten years of experience in chest CT reading used the 3D Slicer software to independently label the tuberculosis lesion area in each layer of CT images without knowing the treatment outcome. The Dice similarity coefficient was calculated for the two labeling results, and if the Dice value was not less than 0.95, the average result of the two was taken as the final ground truth; if the Dice value was less than 0.95, a senior radiologist with more than fifteen years of experience re-labeled the images to obtain the final labeling result; through the above operation, the local lesion area (i.e., the tuberculosis lesion area) manually outlined was obtained.

[0106] In addition to the local lesion region obtained by manual delineation, the present application also introduces the automatic segmentation of the lung parenchyma region by a deep learning model to obtain the lung parenchyma segmentation region, so as to provide more extensive spatial information and structural features, thereby reflecting the overall changes of the lung tissue. The lung parenchyma segmentation is realized by using a deep learning model R231 (https: / / github.com / JoHof / lungmask) based on the U-Net structure, and the initial segmentation result is manually reviewed and corrected by an experienced radiologist as necessary, and the left and right lung regions are combined into a unified region for subsequent analysis.

[0107] The present application extracts two types of radiomics features from the accurately segmented region of interest (ROI), i.e., the local lesion region obtained by manual delineation and the lung parenchyma segmentation region, i.e., hand-crafted radiomics (HCR) extracted from the local lesion region obtained by manual delineation and DL-based radiomics (DLR) extracted from the local lesion region obtained by manual delineation and the lung parenchyma segmentation region. The two types of features are used to represent the abnormal structure of the lung and track its temporal evolution during the treatment of multi-drug resistant tuberculosis (MDR-TB).

[0108] The extraction of HCR features is based on the standardized radiomics biomarker extraction process, which is realized by using PyRadiomics software (version 3.1.0). A total of 1130 features are extracted in each scan, covering first-order statistical features, shape features, texture features, and high-order filter features (including wavelet transform and LoG filter features).

[0109] DLR features are obtained by transfer learning method, specifically by using the pre-trained 3DResNet-18 model in the MedicalNet project (https: / / github.com / Tencent / MedicalNet) for deep learning imaging feature extraction. In the automatically segmented lung parenchyma segmentation region, each layer of image is adjusted to 256x256 pixels and deep direction padding is performed to keep the input dimension consistent; for the local lesion region obtained by manual delineation, the image is adjusted to 128x128 pixels and then input into the ResNet model, the feature map of the second last fully connected layer is extracted, and it is flattened into a feature vector. All images are intensity normalized before input to ensure the consistency of feature extraction.

[0110] The HCR and DLR features are extracted from the manually outlined local lesion area and lung parenchyma segmentation area respectively by the above method, and finally the manually made radiomics features and deep learning imaging features are obtained. In order to represent the dynamic changes of the manually made radiomics features and deep learning imaging features with treatment time, the present application calculates the difference value by performing pair feature subtraction on the manually made radiomics features and deep learning imaging features between different time points respectively, and obtains the delta-features of the training set, internal validation set and external validation set, which are suitable for HCR and DLR features respectively.

[0111] In addition, in order to further capture the nonlinear time progression pattern, the extracted manually made radiomics features and deep learning imaging features at each time point are respectively organized as a time sequence and input into a two-layer Gated Recurrent Unit (GRU) network for modeling, and finally the hidden state vectors of the training set, internal validation set and external validation set are output by the GRU network; the network structure can learn the forward and backward dependencies in the time dimension, thereby realizing robust modeling of the complex disease evolution process.

[0112] 5. Construction of multidrug-resistant tuberculosis prognosis prediction model / system (MTBTOPS)

[0113] In order to construct a robust and clinically applicable multidrug-resistant tuberculosis (MDR-TB) treatment outcome prediction framework, the present application proposes the MTBTOPS system, which is based on deep learning method and uses longitudinal CT image data to predict the treatment prognosis of patients.

[0114] The constructed multidrug-resistant tuberculosis prognosis prediction model is trained by the hidden state vectors and delta-features of the training set, and is verified by the hidden state vectors and delta-features of the internal validation set and external validation set respectively, so as to obtain the trained multidrug-resistant tuberculosis prognosis prediction model (MTBTOPS);

[0115] The trained multidrug-resistant tuberculosis prognosis prediction model is used to predict the prognosis of target multidrug-resistant tuberculosis patients. Specifically, the CT scan image clinical data of the target multidrug-resistant tuberculosis patient is input into the trained multidrug-resistant tuberculosis prognosis prediction model, and finally the prognosis prediction result is output.

[0116] As shown in Figure 2 A, the newly diagnosed MDR-TB patients receive CT examination at three time points, which are baseline (t0), 2 months of treatment (t1) and 6 months (t2). At each time point, the system extracts radiomics features, and models through time sequence to capture the dynamic changes of disease progression.

[0117] The MTBTOPS system contains two prediction modules: one is the MTBTOPS-FU module, which is used for binary classification task to divide patients into favorable treatment outcome (FTO) and unfavorable treatment outcome (UTO); the other is the MTBTOPS-SCF module, which realizes multi-class treatment outcome stratification prediction to divide patients into three categories of "successful treatment", "completed treatment" and "treatment failure". This modular design not only supports outcome prediction, but also helps individualized risk assessment and serves clinical decision support.

[0118] The system was trained and validated on a multicenter retrospective cohort data (see Figure 2 B), and the performance was verified by comprehensive evaluation indicators, including classification performance indicators, ROC curve analysis, calibration curve and decision curve analysis (DCA) (see Figure 2 C).

[0119] As an important part of the system exploration of modeling strategies, the present application has constructed 64 different configuration models, as shown in Figure 3 Each model variant (such as SingleM1-12, DualM1-12, M1-M8) has different configurations in the following dimensions: CT time point number (single time point, double time point or three time points), region of interest (lesion region ROI and whole lung region), feature type (human radiomics HCR and deep learning radiomics DLR) and time modeling strategy (no modeling, incremental feature method or GRU time modeling).

[0120] The above 64 models are systematically evaluated in binary and multi-classification prediction tasks, each of which corresponds to 32 models, which are the basis for finally determining and optimizing the structure of the MTBTOPS classifier.

[0121] 6. Statistical analysis

[0122] The comparison of categorical variables uses chi-square test or Fisher's exact test (selected according to data distribution); the analysis of continuous variables uses Mann-Whitney U test or independent sample t test according to their distribution characteristics. P value less than 0.05 is considered statistically significant.

[0123] The evaluation of model prediction performance adopts the receiver operating characteristic curve (ROC curve), and the main indicators include the area under the ROC curve (AUC), accuracy, sensitivity, specificity, precision, and F1 score. The training of the MTBTOPS-FU model uses a binary cross-entropy loss function, while the MTBTOPS-SCF model uses a categorical cross-entropy loss function. For the evaluation of the MTBTOPS-SCF model, macro-averaged and micro-averaged AUC, as well as the accuracy, recall, and F1 score of each category are further introduced.

[0124] All statistical analyses were performed in the Python (version 3.9.13) environment, and the deep learning models used were based on the PyTorch deep learning framework to implement feature extraction and prediction model training and inference.

[0125] At the same time, a clinical random forest model was constructed as a comparison, and the input features of the model included age, gender, residence, smoking and drinking habits, and tuberculosis treatment history. The classification performance of this clinical model was compared with the MTBTOPS-FU and MTBTOPS-SCF models in binary and multi-classification tasks, respectively, and the grid search was used to optimize its hyperparameters to improve the classification accuracy and AUC.

[0126] In addition, the present application uses the 6th month sputum culture conversion (SCC) as a clinical indicator of treatment failure and compares it with the prediction results of the MTBTOPS-FU model. The statistical analysis and machine learning modeling process are implemented using the Scikit-learn (version 0.24.2) and SciPy libraries.

[0127] II. Research Results

[0128] 1. Patient characteristics

[0129] Table 1 summarizes the demographic and clinical characteristics of the 265 MDR-TB patients included in this study, of which 198 patients were from the derivation cohort and 67 patients were from the test cohort. In the derivation cohort, 27 patients (13.6%) had UTO, and the remaining 171 patients (86.4%) had FTO; in the test cohort, 19 patients (28.4%) had UTO, and the remaining 48 patients (71.6%) had FTO. Detailed information on treatment outcomes and the results of phenotypic and molecular drug susceptibility testing (DST) in the three centers is shown in Table 2.

[0130] Table 1 Demographic and clinical characteristics of MDR-TB patients in the internal and external cohorts

[0131]

[0132]

[0133] As shown in Table 1, there were no statistically significant differences in age, gender, residence, smoking, and drinking habits between the FTO and UTO groups. However, two clinical variables were significantly associated with treatment outcomes. In the derivation cohort, sputum culture conversion (SCC) at 6 months after treatment was significantly more frequent in the FTO group than in the UTO group (P < 0.05). In addition, in both cohorts, patients with a history of anti-TB treatment were significantly more common in the UTO group (P < 0.05).

[0134] These results suggest that delayed sputum culture conversion and a history of anti-TB treatment may be important predictors of poor outcomes in MDR-TB patients.

[0135] Table 2 Distribution of treatment outcomes and results of phenotypic and molecular drug susceptibility testing

[0136]

[0137]

[0138] 2. Performance of MTBTPOS-FU

[0139] The present application constructed a binary classification model (MTBTPOS-FU) for predicting UTO in MDR-TB patients under the MTBTPOS framework. As shown in Table 3, the model that fused image data from three time points performed better than the models that used only single-time-point or double-time-point data in terms of prediction performance, as shown in Table 3.

[0140]

[0141]

[0142] As Figure 4 shown, the models M1 to M8 were systematically compared in terms of prediction performance under various evaluation metrics, including classification performance metrics, ROC curve, calibration curve, and decision curve analysis (DCA).

[0143] Among them, the model M4 constructed based on the DLR features extracted from the manually delineated ROI region performed the best in all evaluation metrics. The radar chart showed that the model M4 achieved the highest values in multiple performance metrics such as F1 score, sensitivity, and accuracy. In the ROC curve analysis, the AUC value of the model M4 was 0.873, the calibration curve showed that it had good consistency between the predicted probability and the actual observation results, and the DCA results further verified that it had high clinical net benefit under different decision thresholds. Therefore, the model M4 can be considered as the optimal configuration scheme of the MTBTPOS-FU model. The above results show that the MTBTPOS-FU model has robust prediction ability, further highlighting the potential clinical application value of longitudinal CT image data in the management of multidrug-resistant tuberculosis.

[0144] 3. Performance of MTBTPOS-SCF

[0145] For the hierarchical prediction task of multiple categories of treatment outcomes, the MTBTPOS-SCF model was constructed to distinguish three outcome types of treatment success (Success), treatment completion (Completed), and treatment failure (Failure). The results in Table 4 show that the models M1 to M8 constructed based on image data at three time points (before treatment, 2 months, and 6 months) showed consistent and superior performance in prediction, further verifying the effectiveness of time series modeling in multi-classification tasks. The macro-average AUC of the model M4 was 0.894, the accuracy was 0.777, and the macro-average F1 score was 0.763. Similar to MTBTPOS-FU, the above results also show that the model M4 is the optimal configuration of MTBTPOS-SCF.

[0146] Table 4 Model strategy, prediction accuracy, and macro-average performance comparison of MTBTOPS-SCF

[0147]

[0148]

[0149] As Figure 5As shown, all models were systematically evaluated by class average performance metrics, class-specific ROC curves, and confusion matrices. Among them, model M4 based on deep radiomic features extracted from lesion region ROI and GRU time-series modeling achieved the best comprehensive performance. Radar plot shows that M4 model achieved the highest scores in macro-average F1 score, AUC and precision, etc. In class-specific evaluation, M4 model achieved the highest precision, recall and F1 score in the "treatment success" category, indicating that it is superior to other outcome categories in identifying patients expected to have successful treatment (see Figure 5 B). ROC analysis further verified the good discriminant ability of the model in each category, with AUC of 0.91 for the success category, 0.88 for the completion category, and 0.90 for the failure category. The confusion matrix results show that the prediction accuracy of model M4 for "success", "completion" and "failure" outcomes is 81.0%, 76.5% and 72.7% respectively.

[0150] The superior performance of the model is due to the integration of CT images at three time points, feature extraction based on lesion regions, deep learning radiomic methods, and time-series modeling structure constructed by GRU neural network, highlighting the important application value of time-series radiomic analysis in multi-class treatment outcome prediction of multidrug-resistant tuberculosis.

[0151] 4. External validation

[0152] As can be seen from Table 5, in the external validation cohort, the previously selected best configuration of MTBTOPS-FU and MTBTOPS-SCF (both corresponding to M4) maintained its strong performance and was significantly superior to the traditional model.

[0153] Table 5. Comparison of prediction performance of MTBTOPS-FU and clinical characteristics model, 6-month sputum culture conversion

[0154]

[0155] Note: The clinical characteristics model predicts all samples as FTO;

[0156] As shown in Figure 6 A-D, MTBTOPS-FU achieved the highest AUC (0.838) in UTO prediction, with better calibration ( Figure 6 B) and better net clinical benefit ( Figure 6 C) compared with the clinical characteristics model (AUC = 0.698) and 6-month SCC (AUC = 0.579). Radar plot ( Figure 6 D) highlights its balanced classification indicators. The confusion matrix ( Figure 6E) shows that MTBTOPS-FU correctly predicted 75.0% of UTO cases, while the clinical characteristics model only reached 26.3%.

[0157] Likewise, MTBTOPS-SCF performed well in predicting the final treatment outcome. The confusion matrix ( Figure 6 F) indicates that MTBTOPS-SCF correctly identified 74.2% of successful cases, 58.8% of completed cases, and 68.4% of failed cases, all of which were higher than the predictions of the clinical model. This advantage was supported by higher class-specific and macro-averaged performance metrics in the external evaluation, the results of which are shown in Table 6.

[0158] Table 6. Comparison of prediction accuracy and macro-averaged performance of MTBTOPS-SCF and clinical characteristics model in independent center external test

[0159]

[0160] These results collectively demonstrate that MTBTOPS, including MTBTOPS-FU and MTBTOPS-SCF, is more universal and superior in clinical utility than clinical characteristics and SCC-based predictions.

[0161] In summary, the multi-drug resistant tuberculosis prognosis prediction system (MTBTOPS) and the prediction method provided by the present application provide a non-invasive, dynamic and continuous method for monitoring the disease progression during the treatment of multi-drug resistant tuberculosis. Compared with the traditional monitoring method based on sputum culture, the present application can accurately identify high-risk patients at an early stage of the treatment process, thereby providing strong support for personalized treatment decisions, optimized intervention timing and reasonable allocation of public health resources.

[0162] Although the embodiments of the present application have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, replacements and variations can be made to these embodiments without departing from the principles and purposes of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A method for prognosis of multidrug-resistant tuberculosis, characterized by, The method comprises the following steps: S1, obtaining longitudinal CT scan image clinical data of multiple groups of multidrug-resistant tuberculosis patients, and dividing them into a training set, an internal validation set, and an external validation set; S2, manually labeling and automatically segmenting the tuberculosis lesion area and lung parenchyma area on each CT scan image in the training set, the internal validation set, and the external validation set, respectively, to obtain the tuberculosis lesion area and the lung parenchyma area; extracting the manually constructed radiomics features and the deep learning-based radiomics features to obtain the manually made radiomics features and the deep learning imaging features; The method comprises the following steps: S21, independently annotating the tuberculosis lesion area on each CT scan image in the training set, the internal validation set, and the external validation set twice; S22, calculating the Dice similarity coefficient between the two annotations obtained on each CT scan image, and determining the tuberculosis lesion area of each CT scan image according to the calculated Dice similarity coefficient; S23, extracting the manually constructed radiomics features from the obtained tuberculosis lesion area to obtain the manually made radiological features; S24, automatically segmenting the lung parenchyma area on each CT scan image in the training set, the internal validation set, and the external validation set through a deep learning model, and merging the automatically segmented left lung and right lung areas into a unified area to determine the lung parenchyma area of each CT scan image; S25, extracting the deep learning-based radiomics features from the obtained tuberculosis lesion area and lung parenchyma area, respectively, to obtain the deep learning imaging features; S3, performing pairwise subtraction between the manually made radiological features and the deep learning imaging features extracted at different time points to obtain the manually made radiological incremental features and the deep learning imaging incremental features, processing the obtained manually made radiomics features and deep learning imaging features using a two-layer GRU network to obtain the hidden state vectors of the training set, the internal validation set, and the external validation set, respectively; comprising the following steps: S31, performing pairwise subtraction between the manually made radiological features and the deep learning imaging features extracted at different time points to obtain the manually made radiological incremental features and the deep learning imaging incremental features; S32, organizing the manually made radiomics features and the deep learning imaging features extracted at different time points into time series, respectively, and processing them using a two-layer GRU network to obtain the hidden state vectors of the training set, the internal validation set, and the external validation set, respectively; S4, training the constructed multidrug-resistant tuberculosis prognosis prediction model through the hidden state vectors of the training set, the manually made radiological incremental features, and the deep learning imaging incremental features, and verifying it through the hidden state vectors of the internal validation set and the external validation set, the manually made radiological incremental features, and the deep learning imaging incremental features, respectively, to obtain the trained multidrug-resistant tuberculosis prognosis prediction model; S5, performing multidrug-resistant tuberculosis prognosis prediction on a target multidrug-resistant tuberculosis patient through the trained multidrug-resistant tuberculosis prognosis prediction model.

2. The method for predicting the prognosis of multidrug-resistant tuberculosis according to claim 1, characterized in that, In step S1, each set of CT scan image clinical data includes CT scan image clinical data of each patient before receiving drug-resistant tuberculosis treatment, 2 months after treatment, and 6 months after treatment.

3. The method for predicting the prognosis of multidrug-resistant tuberculosis according to claim 1, characterized in that, In step S22, when the calculated Dice similarity coefficient is greater than or equal to 0.95, the two annotations are averaged to generate a reference ground truth, and the tuberculosis lesion area of each CT scan image is determined; When the calculated Dice similarity coefficient is less than 0.95, the tuberculosis lesion area on the CT scan image is re-annotated, and the tuberculosis lesion area of each CT scan image is determined.

4. The method for predicting the prognosis of multidrug-resistant tuberculosis according to claim 1, characterized in that, In step S4, the prognosis prediction model of multidrug-resistant tuberculosis is composed of two prediction modules; One prediction module is MTBTPOS-FU, which is used for binary classification of favorable treatment results and unfavorable treatment results; Another prediction module is MTBTPOS-SCF, which is used for stratifying multi-class treatment results into treatment success, treatment completion and treatment failure categories.

5. A multidrug-resistant tuberculosis prognosis prediction system, characterized by, The prediction method of any one of claims 1-4 is realized, comprising: A data acquisition module for acquiring a plurality of sets of CT scan image clinical data and dividing them into a training set, an internal validation set and an external validation set; A feature extraction module for manually annotating and automatically segmenting the tuberculosis lesion area and lung parenchyma area on each CT scan image in the training set, internal validation set and external validation set, respectively, to obtain the tuberculosis lesion area and lung parenchyma area; respectively extracting the manually constructed radiomics features and the deep learning-based radiomics features to obtain the manually made radiomics features and the deep learning imaging features; The feature extraction module comprises: A feature annotation module for independently annotating the tuberculosis lesion area on each CT scan image in the training set, internal validation set and external validation set twice; A tuberculosis lesion area determination module for calculating the Dice similarity coefficient between the two annotations obtained on each CT scan image, and determining the tuberculosis lesion area of each CT scan image according to the calculated Dice similarity coefficient; A manual feature extraction module for extracting manually constructed radiomics features from the obtained tuberculosis lesion area to obtain manually made radiological features; A lung parenchyma area determination module for automatically segmenting the lung parenchyma area on each CT scan image in the training set, internal validation set and external validation set by a deep learning model, and merging the left and right lung areas of the automatic segmentation into a unified area to determine the lung parenchyma area of each CT scan image; A deep learning feature extraction module for extracting deep learning-based radiomics features from the obtained tuberculosis lesion area and lung parenchyma area, respectively, to obtain deep learning imaging features; The data calculation processing module is configured to obtain a hand-crafted radiomics increment feature and a deep learning imaging increment feature by performing pairwise subtraction between the extracted hand-crafted radiomics features and the deep learning imaging features at different time points, and process the obtained hand-crafted radiomics features and the deep learning imaging features by using a two-layer GRU network to obtain hidden state vectors of a training set, an internal validation set and an external validation set, respectively. The working method of the data calculation processing module is as follows: The hand-crafted radiomics features and the deep learning imaging features are extracted at different time points, and pairwise subtraction is performed between the extracted hand-crafted radiomics features and the deep learning imaging features at different time points to obtain a hand-crafted radiomics increment feature and a deep learning imaging increment feature; The hand-crafted radiomics features and the deep learning imaging features extracted at different time points are organized into time series, and a two-layer GRU network is used for processing to obtain hidden state vectors of a training set, an internal validation set and an external validation set, respectively; The training and verification module is configured to train a constructed multidrug-resistant tuberculosis prognosis prediction model by using the hidden state vectors of the training set, the hand-crafted radiomics increment feature and the deep learning imaging increment feature, and verify the multidrug-resistant tuberculosis prognosis prediction model by using the hidden state vectors of the internal validation set and the external validation set, the hand-crafted radiomics increment feature and the deep learning imaging increment feature, respectively, to obtain a trained multidrug-resistant tuberculosis prognosis prediction model; The prediction module is configured to perform multidrug-resistant tuberculosis prognosis prediction on a target multidrug-resistant tuberculosis patient by using the trained multidrug-resistant tuberculosis prognosis prediction model.

6. A computer readable storage medium characterized by, The computer readable storage medium comprises a stored computer program; wherein the computer program, when executed, controls a device in which the computer readable storage medium is located to perform the steps of the multidrug-resistant tuberculosis prognosis prediction method according to any one of claims 1-4.

7. A terminal device, characterized by comprising: The processor, the memory and the computer program stored in the memory and configured to be executed by the processor are included, and the processor, when executing the computer program, implements the steps of the multidrug-resistant tuberculosis prognosis prediction method according to any one of claims 1-4.

Citation Information

Patent Citations

  • Deep learning-based identity recognition and face comparison system and training method

    CN114445875A

  • Sensor transformation attention network (STAN) model

    EP3404578A1