A method for identifying the infectiousness of pulmonary tuberculosis based on two-dimensional projection of CT images and deep learning

Abstract

The present invention relates to the field related to medical intelligent analysis, discloses a method for distinguishing the infectiousness of pulmonary tuberculosis based on two-dimensional projection of CT images and deep learning, and provides a method for analyzing the infectiousness of pulmonary tuberculosis CT images based on two-dimensional projection. The method does not require the use of a three-dimensional model and can greatly increase the sample size, which greatly alleviates the overfitting problem of the deep learning model; a two-stream two-dimensional ResNet model is proposed, which adopts a dual-backbone network design and uses a residual module to extract features, and performs excellently in the task of distinguishing the infectiousness of pulmonary tuberculosis CT images in two-dimensional projection; projections in the three directions of Z, X, and Y are adopted, and the mean and standard deviation are used as the attributes of the projection. Experimental results show that this approach is effective; the mean of the predicted probabilities in the three directions is used in the test phase, which makes the final prediction result of the model more robust.

Description

Technical Field

[0001] The present invention relates to fields related to medical intelligent analysis, and specifically to a method for distinguishing the infectiousness of pulmonary tuberculosis based on two-dimensional projection of CT images and deep learning. Background Art

[0002] Tuberculosis is a chronic infectious disease caused primarily by Mycobacterium tuberculosis. The World Health Organization estimates that approximately 1.5 million people died from tuberculosis in 2018, a higher mortality rate than any other single infectious pathogen. Mycobacterium tuberculosis is mainly transmitted through the respiratory tract (including coughing, sneezing, talking, singing, and even deep breathing). Preventing its spread is a difficult and long-term task, especially when drug-resistant Mycobacterium tuberculosis emerges. The WHO recommends triage of patients with symptoms of tuberculosis or diagnosed with tuberculosis, and respiratory isolation of patients who are considered or proven to be infectious to reduce the spread of Mycobacterium tuberculosis among medical workers, caregivers in medical institutions, or other people in high-risk environments. Therefore, the development of new tuberculosis prevention and management tools is urgently needed. Current clinical methods for detecting Mycobacterium tuberculosis include sputum testing. Smear examination and sputum culture examination; sputum smear examination is faster, but has poor repeatability and low single-time detection rate, and usually requires repeated examinations; sputum culture examination takes a long time, usually 2-6 weeks, and is expensive; both sputum smear examination and sputum culture examination rely on the quality of the sputum sample; in addition, there are other methods for assessing the infectiousness of tuberculosis patients, such as mathematical models for assessing the transmission pattern of tuberculosis in the population; and special molecular biological methods for measuring the number of live Mycobacterium tuberculosis in the exhaled air of a single tuberculosis patient; however, these methods have high technical requirements, require long-term training, and are also very expensive.

[0003] In recent years, deep learning methods have been widely used in the diagnosis of various diseases. Its application in tuberculosis is mainly combined with lung images to differentiate between tuberculosis and pneumonia, differentiate between drug-resistant and non-drug-resistant tuberculosis, differentiate between Mycobacterium tuberculosis and non-tuberculosis Mycobacterium infection, or quickly screen patients with active tuberculosis. Few studies have focused on the rapid detection and assessment of the infectiousness of tuberculosis patients in order to classify them. Existing three-dimensional deep learning models usually have a large number of parameters, while the clinical data available for the diagnostic analysis of tuberculosis excretion ability in CT images is relatively small, which often leads to overfitting in model training. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for distinguishing the infectiousness of pulmonary tuberculosis based on two-dimensional projection of CT images and deep learning, so as to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The method for identifying the infectiousness of pulmonary tuberculosis based on two-dimensional projection of CT images and deep learning includes the following steps:

[0007] Preprocessing the training data, obtaining CT image data and tuberculosis sputum smear test results of historical pulmonary tuberculosis patients, and determining the infectiousness of the patients based on the test results for use in classifying the CT image data;

[0008] Lung image segmentation and projection: segmenting the CT image data using an open source model to obtain a 3D segmentation result. Based on the 3D segmentation result, projecting the left and right lungs along the spatial coordinate axis to generate a projection image. The 3D segmentation result represents a CT image that retains only the lungs.

[0009] Neural network processing of the sample, processing the projected image through a two-dimensional ResNet neural network to generate projected image features, performing a maximum operation fusion based on multiple feature vectors of the projected image features, and obtaining a predicted probability through a fully connected layer and a Sigmoid activation function;

[0010] Binary classification of the data set: The pre-divided test data set is subjected to binary classification through the sample neural network processing step. The ROC curve is drawn based on the probability value output by the test set, and the RO-AUC is calculated as the evaluation indicator of the binary classification model. The threshold at the rated specificity is selected based on the ROC curve as the classification threshold, which is used to determine the infectivity;

[0011] To determine the infectiousness of a patient, a CT image of the lungs of a patient to be diagnosed with pulmonary tuberculosis is obtained, multiple prediction probabilities are obtained through cutting and projecting the lung image and performing neural network processing of the sample, the multiple prediction probabilities are averaged to obtain a final prediction probability of the patient, and the prediction probability is determined based on a classification threshold in the binary classification of the data set to determine the infectiousness of the patient.

[0012] As a further solution of the present invention: the pre-processing step of the training data further includes:

[0013] The original CT image data of the patient is converted into a data format, and the DICOM format data is converted into NII format data, thereby achieving data desensitization. The CT image data is divided into a highly infectious group and a weakly infectious group.

[0014] As a further embodiment of the present invention, before the lung image cutting and projection step is performed, the method further includes the following steps:

[0015] The data is divided into a training data set and a test data set based on the category ratio of the highly infectious group and the weakly infectious group to ensure that the training data set and the test data set are independent of each other and do not interfere with each other. The training data set and the test training set account for 80% and 20% respectively.

[0016] As a further embodiment of the present invention, the step of performing lung segmentation processing on the CT image data using the open source model to obtain a three-dimensional segmentation result specifically includes:

[0017] The lungs were segmented using the open source model lungmask. The lung segmentation result was multiplied with the original CT image to obtain a CT image retaining only the lungs, i.e., the three-dimensional segmentation result. The CT values ​​were normalized, the CT value range was cut to -1000 to 400, and linearly interpolated to 0 to 1.

[0018] As a further solution of the present invention, the step of projecting the left and right lungs in the direction of the spatial coordinate axis based on the three-dimensional segmentation result further includes the following pre-steps:

[0019] The three-dimensional segmentation result is judged. If the left and right lungs in the segmentation result are not adhered, the two areas are separated by the connected domain, and the left and right lungs are judged by the center of mass position of the connected domain. If the left and right lungs are adhered, the left and right lungs are directly segmented along the midline of the cross section of the segmentation result. When the projection is generated, the mean and standard deviation of the pixels in the corresponding direction are projected.

[0020] As a further solution of the present invention: the two-dimensional ResNet model includes two backbone networks, which are used to extract the projection image features of the left lung and the right lung respectively, and are fused through a maximum operation. In the training of the two-stream two-dimensional ResNet model, the amount of training samples is increased and the generalization ability of the model is improved by online data augmentation technology. The data augmentation technology includes random flipping, random scaling, random translation, random rotation and random noise. The loss function of the model training is the cross entropy loss function:

[0021]

[0022] in, represents the number of samples, Representation sample The classification label, Represents the model prediction sample is the probability of strong infectiousness. This loss function encourages the model to predict highly infectious samples as 1 and weakly infectious samples as 0.

[0023] Compared with the existing technology, the beneficial effects of the present invention are: providing a method for analyzing the infectiousness of tuberculosis CT images based on two-dimensional projection, which does not require the use of a three-dimensional model and can greatly increase the sample size, which greatly alleviates the overfitting problem of the deep learning model; proposing a two-stream two-dimensional ResNet model, which adopts a dual-backbone network design and uses a residual module to extract features, and performs well in the task of distinguishing the infectiousness of tuberculosis CT images in two-dimensional projection; adopting projections in the three directions of Z, X, and Y, and using the mean and standard deviation as the attributes of the projection, and experimental results show that this approach is effective; in the test phase, the mean of the predicted probabilities in the three directions is used, which makes the final prediction result of the model more robust. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flowchart of the method for identifying the infectiousness of tuberculosis based on two-dimensional projection of CT images and deep learning.

[0025] Figure 2 This is a flowchart of the two-dimensional projection method and deep learning prediction in the method for distinguishing the infectiousness of pulmonary tuberculosis based on two-dimensional projection of CT images and deep learning.

[0026] Figure 3 Examples of CT images of weakly infectious and highly infectious patients in the method for distinguishing the infectiousness of pulmonary tuberculosis based on two-dimensional projection of CT images and deep learning.

[0027] Figure 4 Schematic diagram of the lung segmentation results of CT images in the method for identifying the infectiousness of tuberculosis based on two-dimensional projection of CT images and deep learning.

[0028] Figure 5 Schematic diagram of the two-dimensional projection image in the method for identifying the infectiousness of pulmonary tuberculosis based on two-dimensional projection of CT images and deep learning.

[0029] Figure 6 This is a two-stream two-dimensional ResNet network structure in the tuberculosis infectiousness identification method based on two-dimensional projection of CT images and deep learning. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0031] The specific implementation of the present invention is described in detail below with reference to specific embodiments.

[0032] like Figures 1 to 6As shown, a method for distinguishing the infectiousness of pulmonary tuberculosis based on two-dimensional projection of CT images and deep learning is provided in one embodiment of the present invention. The main purpose of the present invention is to analyze three-dimensional CT images based on a two-dimensional projection method to alleviate the overfitting phenomenon of the three-dimensional deep learning model in the infectiousness discrimination task, and to improve the diagnostic performance of the deep learning model; the projection method used is to take the mean and standard deviation of the pixel points of the left and right lungs of the three-dimensional CT image in the Z, X, and Y directions respectively to obtain the two-dimensional projections of the three surfaces, and then use a two-stream two-dimensional ResNet model to analyze the projection data to realize the diagnosis of the patient's bacteria excretion ability; the technical problems to be solved by the present invention include preprocessing of three-dimensional CT images, generation of two-dimensional projections, training of the two-stream two-dimensional ResNet model, and use of the trained ResNet model to diagnose pulmonary tuberculosis patients; the steps include:

[0033] S10, preprocessing of training data, obtaining CT image data and tuberculosis sputum smear test results of historical pulmonary tuberculosis patients, and judging the infectiousness of the patients based on the test results for dividing the CT image data.

[0034] S20, cutting and projecting the lung image, performing lung segmentation processing on the CT image data through an open source model to obtain a three-dimensional segmentation result, and projecting the left and right lungs in the direction of the spatial coordinate axis based on the three-dimensional segmentation result to generate a projection image. The three-dimensional segmentation result represents the CT image that only retains the lungs.

[0035] S30, neural network processing of the sample, processing the projected image through a two-dimensional ResNet neural network to generate projected image features, performing a maximum operation fusion based on multiple feature vectors of the projected image features, and obtaining a prediction probability through a fully connected layer and a Sigmoid activation function.

[0036] S40, binary classification of the data set, the pre-divided test data set is binary classified through the sample neural network processing step, the ROC curve is drawn according to the probability value output by the test set, and the RO-AUC is calculated as the evaluation index of the binary classification model, and the threshold value at the rated specificity is selected based on the ROC curve as the classification threshold, and the classification threshold is used to judge the infectivity.

[0037] S50, determining the infectiousness of the patient, obtaining a CT image of the lungs of the patient to be diagnosed with pulmonary tuberculosis, obtaining multiple prediction probabilities through cutting and projecting the lung image and performing neural network processing of the sample, averaging the multiple prediction probabilities to obtain a final prediction probability for the patient, and determining the prediction probability based on the classification threshold in the binary classification of the data set to determine the infectiousness of the patient.

[0038] Furthermore, the pre-processing step of the training data further includes:

[0039] The original CT image data of the patient is converted into a data format, and the DICOM format data is converted into NII format data, thereby achieving data desensitization. The CT image data is divided into a highly infectious group and a weakly infectious group.

[0040] Furthermore, before the lung image cutting and projection step is performed, the following steps are also included:

[0041] The data is divided into a training data set and a test data set based on the category ratio of the highly infectious group and the weakly infectious group to ensure that the training data set and the test data set are independent of each other and do not interfere with each other. The training data set and the test training set account for 80% and 20% respectively.

[0042] Furthermore, the step of performing lung segmentation processing on the CT image data using the open source model to obtain a three-dimensional segmentation result specifically includes:

[0043] The lungs were segmented using the open source model lungmask. The lung segmentation result was multiplied with the original CT image to obtain a CT image retaining only the lungs, i.e., the three-dimensional segmentation result. The CT values ​​were normalized, the CT value range was cut to -1000 to 400, and linearly interpolated to 0 to 1.

[0044] Furthermore, the step of projecting the left and right lungs in the direction of the spatial coordinate axis based on the three-dimensional segmentation result further includes the following pre-steps:

[0045] The three-dimensional segmentation result is judged. If the left and right lungs in the segmentation result are not adhered, the two areas are separated by the connected domain, and the left and right lungs are judged by the center of mass position of the connected domain. If the left and right lungs are adhered, the left and right lungs are directly segmented along the midline of the cross section of the segmentation result. When the projection is generated, the mean and standard deviation of the pixels in the corresponding direction are projected.

[0046] Furthermore, the two-dimensional ResNet model includes two backbone networks, which are used to extract the projection image features of the left lung and the right lung respectively, and are fused through a maximum operation. In the training of the two-stream two-dimensional ResNet model, the amount of training samples is increased and the generalization ability of the model is improved by online data augmentation technology. The data augmentation technology includes random flipping, random scaling, random translation, random rotation and random noise. The loss function of the model training is the cross entropy loss function:

[0047]

[0048] in, represents the number of samples, Representation sample The classification label, Represents the model prediction sample is the probability of strong infectiousness. This loss function encourages the model to predict samples with strong infectiousness as 1 and samples with weak infectiousness as 0.

[0049] In this example, CT image data of pulmonary tuberculosis patients who had undergone tuberculosis sputum smear examinations were collected. The original CT image data is usually in DICOM format. The DICOM format data is converted into NII format data to achieve data desensitization. Based on the tuberculosis sputum smear examination results of pulmonary tuberculosis patients, experts divide the CT images into a highly infectious group and a weakly infectious group. If the tuberculosis patient has three or more negative tuberculosis sputum smear examinations within a month, the patient is classified as weakly infectious; if the tuberculosis patient has only one positive tuberculosis sputum smear examination, the patient is classified as highly infectious. Figure 2 CT images of patients with weakly contagious and highly contagious pulmonary tuberculosis are shown; all data are divided into 80% for training and 20% for testing according to the category ratio, ensuring that the training set and the test set are independent and mutually exclusive, and there is no information leakage problem. Before analyzing pulmonary tuberculosis, it is very necessary to segment the lungs, which not only eliminates the interference of irrelevant tissues but also reduces the learning cost of the deep learning model. The open source model lungmask (https: / / github.com / JoHof / lungmask) is used to segment the lungs, and the lung segmentation result is multiplied with the original CT image to obtain a CT image that only retains the lungs, as shown in the figure below. Figure 3 As shown. In order to facilitate the training of deep learning models, the CT values ​​are normalized. First, the CT value range is cut to -1000~400, and then linearly interpolated to 0~1; before generating the projection, it is necessary to distinguish the left and right lungs. According to the three-dimensional segmentation result of the lungs obtained in step S3, if the left and right lungs of the segmentation result are not adhered, the two areas are separated by the connected domain, and then the left and right lungs are determined by the center of mass position of the connected domain; if the left and right lungs of the segmentation result are adhered, they are directly cut along the midline of the cross section of the segmentation result and divided into left and right lungs according to the orientation. After distinguishing the left and right lungs, the left and right lungs are projected along the Z, X, and Y directions respectively, and the mean and standard deviation of the pixels in the corresponding directions are taken. Finally, the obtained images are scaled to 192×192 size, as shown Figure 4 As shown; the deep learning network used in the present invention is a two-dimensional ResNet model with two streams, and its network structure is as follows Figure 5As shown in Figure 2, the two-stream 2D ​​ResNet model has two backbone networks, one for extracting features from the left and right lung projection images. The extracted feature vectors are fused through a maximum operation and ultimately pass through a fully connected layer and a sigmoid activation function to obtain predicted probabilities. Both backbone networks utilize the same 2D ResNet neural network structure and do not share weights. Taking the right lung backbone network as an example, its input is the right lung projection image (i.e., the mean and standard deviation of the channel-wise concatenated right lung projections; in any of the three directions: Z, X, and Y). The first convolutional layer with a kernel size of 7×7 and a stride of 2×2 and the second max pooling layer rapidly reduce the image dimensionality, minimizing memory requirements. The subsequent 16 residual modules extract deep features from the projection images. A residual module consists of two convolutional layers, two batch normalization layers, and two rectified linear units (ReLUs). It directly adds input features to the output features through a cross-layer connection structure, improving feature reuse and alleviating the vanishing gradient problem during training. The backbone network is finally connected to an adaptive maximum pooling layer (AdaptiveMaxPool) to compress the two-dimensional feature map into a 512-dimensional deep feature vector. Through the above process, the left and right lung projections can respectively extract a 512-dimensional deep feature, and the left and right lung features are fused into a 512-dimensional deep feature using the maximum operation. Finally, a fully connected layer (Fullconnection) is used as a classifier; in the dual-stream two-dimensional ResNet model training stage, online data augmentation technology is used to increase the training sample size and improve the generalization ability of the model. The data augmentation methods used include random flipping, random scaling, random translation, random rotation, and random noise. In addition, the projections of the three directions of the left and right lungs are randomly combined as model inputs, so that the original three-dimensional CT image sample can generate 9 groups of different training samples. The present invention directly expands the model training sample size by 9 times. The optimizer used is Adam, and the learning rate is set to 0.0001. The loss function of the model training is the cross entropy loss function, and its expression is as follows:

[0050]

[0051] in, represents the number of samples, Representation sample The classification label, Represents the model prediction sample is the probability of being highly contagious. This loss function encourages the model to predict highly contagious samples as 1 and weakly contagious samples as 0. The test dataset is input into the model obtained in S5 for binary classification. Based on the probability values ​​output by the test dataset, a receiver operating characteristic (ROC) curve is plotted, and the RO-AUC is calculated as the evaluation metric for the binary classification model. A threshold of 0.7 with a specificity on the ROC curve is selected as the classification threshold. Samples greater than or equal to this threshold are considered positive, indicating highly contagious; samples less than this threshold are considered negative, indicating weakly contagious. The CT image of the tuberculosis patient to be diagnosed is processed in steps S3 and S4 to obtain projections in the Z, X, and Y directions. Each projection is input into the two-stream 2D ​​ResNet model trained in S5 for prediction, resulting in three predicted probabilities. The average of the predicted probabilities in the three directions is the final prediction result for the tuberculosis patient. The patient is classified as highly contagious or weakly contagious based on the threshold determined in S6.

[0052] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program.

[0053] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the disclosure in the specification and examples. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered merely as exemplary, and the true scope and spirit of the present disclosure are indicated by the claims.

[0054] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for identifying the infectiousness of pulmonary tuberculosis based on two-dimensional projection of CT images and deep learning, characterized in that: The following steps are involved: Preprocessing the training data, obtaining CT image data and tuberculosis sputum smear test results of historical pulmonary tuberculosis patients, and determining the infectiousness of the patients based on the test results for use in classifying the CT image data; Lung image segmentation and projection: segmenting the CT image data using an open source model to obtain a 3D segmentation result. Based on the 3D segmentation result, projecting the left and right lungs along the spatial coordinate axis to generate a projection image. The 3D segmentation result represents a CT image that retains only the lungs. Neural network processing of the sample, processing the projected image through a two-dimensional ResNet neural network to generate projected image features, performing a maximum operation fusion based on multiple feature vectors of the projected image features, and obtaining a predicted probability through a fully connected layer and a Sigmoid activation function; Binary classification of the data set: The pre-divided test data set is subjected to binary classification through the sample neural network processing step. The ROC curve is drawn based on the probability value output by the test set, and the RO-AUC is calculated as the evaluation indicator of the binary classification model. The threshold at the rated specificity is selected based on the ROC curve as the classification threshold, which is used to determine the infectivity; Determining the infectiousness of a patient by obtaining a CT image of the lungs of a patient to be diagnosed with pulmonary tuberculosis, obtaining multiple predicted probabilities through a segmentation and projection of the lung image and a neural network processing step of the sample, averaging the multiple predicted probabilities to obtain a final predicted probability for the patient, and determining the predicted probabilities based on a classification threshold in a binary classification of the data set to determine the infectiousness of the patient; The step of projecting the left and right lungs in the direction of the spatial coordinate axis based on the three-dimensional segmentation result further includes the following pre-steps: The 3D segmentation result is judged. If the left and right lungs in the segmentation result are not adhered, the two regions are separated by the connected domain, and the left and right lungs are judged by the center of mass position of the connected domain. If the left and right lungs are adhered, the left and right lungs are directly segmented along the midline of the cross section of the segmentation result. When the projection is generated, the mean and standard deviation of the pixels in the corresponding direction are projected; The two-dimensional ResNet model includes two backbone networks, which are used to extract the projection image features of the left lung and the right lung respectively, and are fused through a maximum operation. In the training of the two-stream two-dimensional ResNet model, the amount of training samples is increased and the generalization ability of the model is improved by online data augmentation technology. The data augmentation technology includes random flipping, random scaling, random translation, random rotation and random noise. The loss function of the model training is the cross entropy loss function: Where N represents the number of samples, y i represents the classification label of sample i, p i Represents the probability that the model predicts that sample i is highly contagious; this loss function encourages the model to predict highly contagious samples as 1 and weakly contagious samples as 0.

2. The method for identifying the infectiousness of pulmonary tuberculosis based on two-dimensional projection of CT images and deep learning according to claim 1, characterized in that: The pre-processing step of the training data further includes: The original CT image data of the patient is converted into a data format, and the DICOM format data is converted into NII format data, thereby achieving data desensitization. The CT image data is divided into a highly infectious group and a weakly infectious group.

3. The method for distinguishing the infectiousness of pulmonary tuberculosis based on two-dimensional projection of CT images and deep learning according to claim 2, characterized in that: Before the lung image cutting and projection step is performed, the method further includes the following steps: The data is divided into a training data set and a test data set based on the category ratio of the highly infectious group and the weakly infectious group to ensure that the training data set and the test data set are independent of each other and do not interfere with each other. The training data set and the test training set account for 80% and 20% respectively.

4. The method for distinguishing the infectiousness of pulmonary tuberculosis based on two-dimensional projection of CT images and deep learning according to claim 3, characterized in that: The step of performing lung segmentation processing on the CT image data using the open source model to obtain a three-dimensional segmentation result specifically includes: The lungs were segmented using the open source model lungmask. The lung segmentation result was multiplied with the original CT image to obtain a CT image retaining only the lungs, i.e., the three-dimensional segmentation result. The CT values ​​were normalized, the CT value range was cut to -1000 to 400, and linearly interpolated to 0 to 1.

Images