A method for classifying non-tuberculosis mycobacterium lung disease and pulmonary tuberculosis CT images
By improving the ResNet encoder and local-global feature extraction module, and combining the self-attention mechanism and feature aggregation module, the problem of classifying nontuberculous mycobacterial lung diseases and pulmonary tuberculosis CT images was solved, achieving efficient and accurate classification results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TIANJIN UNIV
- Filing Date
- 2023-05-22
- Publication Date
- 2026-05-12
AI Technical Summary
Existing image classification algorithms struggle to effectively handle the complex features of nontuberculous mycobacterial lung diseases and pulmonary tuberculosis on CT images, especially in 3D CT images where accurate classification is difficult.
An improved ResNet encoder is adopted, which combines a local-global feature extraction module and a self-attention mechanism in a parallel structure. Through a feature aggregation module and an auxiliary prediction task, a small number of lesion annotations are used to assist model training. An encoder with a parallel structure of convolution and self-attention mechanism is designed, and a loss function is constructed to improve lesion recognition.
It achieves accurate classification of CT images of nontuberculous mycobacterial lung diseases and pulmonary tuberculosis, improving the model's recognition ability and classification performance, especially outperforming other methods in AUC scores.
Smart Images
Figure CN116580242B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and medical image analysis, with the application scenario of classifying CT images of nontuberculous mycobacterial lung diseases and pulmonary tuberculosis. Specifically, it is a method for classifying CT images of nontuberculous mycobacterial lung diseases and pulmonary tuberculosis. Background Technology
[0002] With the rapid improvement of computing power and the extensive research into deep learning, image classification technology is constantly evolving, making leaps in both speed and accuracy. However, these image classification methods are mainly applied to natural image classification tasks where the background and foreground are clearly distinguishable. Nontuberculous mycobacterial lung disease and pulmonary tuberculosis have highly similar CT images, and both contain various radiological features, such as cavitation, bronchiectasis, consolidation, and nodules. Some features are highly similar to normal human tissue (such as pleural thickening), while others are too small to be easily identified (such as solitary nodules and calcifications). The complex features of nontuberculous mycobacterial lung disease and pulmonary tuberculosis on CT images make it difficult for current mainstream image classification algorithms to achieve effective performance.
[0003] Accurate classification of CT images of nontuberculous mycobacterial lung diseases and pulmonary tuberculosis is of great value for many studies. Given the development of the multiple instance learning (MIM) framework, which treats the image category as determined by multiple instances within the image, it is suitable for image classification problems where there are many types of objects within the image, and each object jointly determines the image category. The MIM architecture breaks the traditional one-to-one correspondence between image categories and objects within the image, effectively establishing a one-to-many relationship between image categories and objects within the image, making it suitable for situations where nontuberculous mycobacterial lung diseases and pulmonary tuberculosis CT images present with numerous and complex features. However, current MIM methods are mainly applied to 2D pathological image classification tasks, and no algorithm has yet solved the task of classifying 3D CT images with complex features.
[0004] This invention addresses the problem of CT image classification for patients with nontuberculous mycobacterial lung disease and pulmonary tuberculosis, proposing a method for classifying CT images of nontuberculous mycobacterial lung disease and pulmonary tuberculosis, thereby achieving accurate classification of CT images of nontuberculous mycobacterial lung disease and pulmonary tuberculosis.
[0005] References
[0006] [1]Ratnatunga CN, Lutzky VP, Kupz A, et al. The rise of non-tuberculosis mycobacterial lung disease[J]. Frontiers in immunology, 2020,11:303.
[0007] [2]Nasiri MJ,Dabiri H,Darban-Sarokhalil D,et al.Prevalence of non-tuberculosis mycobacterial infections among tuberculosis suspects in Iran:systematic review and meta-analysis[J].PloS one,2015,10(6):e0129073.
[0008] [3]Chu HQ,Li B,Zhao L,et al.Chest imaging comparison between non-tuberculous and tuberculosis mycobacteria in sputum acid fast bacilli smear-positive patients[J].Eur Rev Med Pharmacol Sci,2015,19(13):2429-2439..
[0009] [4] Wu Xiaoguang, Gao Mengqiu, Ma Liping. Clinical analysis of 50 cases of nontuberculous mycobacterial disease [J]. Chinese Journal of Antituberculosis, 2009, 31(8):481.
[0010] [5] Lai Yanfen, Wu Dongling, Yang Lin. Analysis of 50 cases of misdiagnosis of nontuberculous mycobacterial lung disease [J]. Journal of Youjiang Medical College for Nationalities, 2014, 36(3):374–375. Summary of the Invention
[0011] To address the problem of classifying nontuberculous mycobacterial lung disease and pulmonary tuberculosis in existing CT images, this invention provides a method for classifying nontuberculous mycobacterial lung disease and pulmonary tuberculosis in CT images, thereby achieving a binary classification of nontuberculous mycobacterial lung disease and pulmonary tuberculosis.
[0012] The technical solution of this invention:
[0013] A method for classifying nontuberculous mycobacterial lung diseases and pulmonary tuberculosis using CT images includes the following steps:
[0014] Step S1: Feature extraction is performed on CT images of nontuberculous mycobacterial lung disease and pulmonary tuberculosis using a neural network. Then, each feature pixel of the extracted feature map along its length, width, and height is regarded as an example, and each example represents the feature information within a certain area of the image.
[0015] Step S2: Calculate the relevance weights of each example obtained in step S1 to the classification task, and aggregate them into a package feature representation by weighted relevance weights to represent the features of the CT image.
[0016] Step S3: The package feature representation obtained in step S2) is predicted as the category of the CT image by the classifier.
[0017] Step S4: Based on the example relevance weights obtained in step S2), select examples with high weight values and evaluate them through a classifier to represent the degree of relevance between the network's learned content and the expected content.
[0018] Step S5: Construct the loss function, which includes bag classification loss and correlation loss. Bag classification loss represents the difference between the CT predicted value and the true value, and correlation loss represents the difference between the model learned content and the expected content.
[0019] The specific steps of step S1 are as follows:
[0020] Step S11: The data in the training set flows through each convolutional module in the network in sequence, and the input image first enters the improved ResNet encoder to extract features;
[0021] Step S12: The features from step S11) are fed into the Local-Global Feature Extraction (LGE) module to extract the feature map.
[0022] Step S13: Divide the feature map obtained in step S12) into examples along each feature pixel of length, width and height.
[0023] The improved ResNet encoder removes the lower four stages from ResNet18 and adjusts the number of basic block stacking layers in the first three stages to 3, 3, 3.
[0024] Step S2) uses the attention-based feature aggregation module FA.
[0025] Step S4) The classifier used to evaluate the correlation between the network's learned content and the expected content is pre-trained with fixed parameters by lesion annotations.
[0026] Compared with the prior art, the present invention has the following advantages:
[0027] Compared with existing technologies, the CT image classification method for nontuberculous mycobacterial lung diseases and pulmonary tuberculosis proposed in this invention has the following advantages:
[0028] 1) An encoder with a parallel structure of convolution and self-attention mechanisms is used to effectively model information at both local and global scales within the image.
[0029] 2) An auxiliary prediction task and an auxiliary prediction loss function were designed to enable the model to be trained using a small number of lesion annotations, which effectively improved the model's ability to identify lesions.
[0030] 3) Experimental comparison results verified the effectiveness of the present invention. By comparing with currently popular classification methods, the effectiveness of the proposed method for predicting nontuberculous mycobacterial lung diseases and pulmonary tuberculosis was verified. Attached Figure Description
[0031] Figure 1 This is a structural diagram of a CT image classification method for nontuberculous mycobacterial lung diseases and pulmonary tuberculosis according to the present invention.
[0032] Figure 2 The structure diagram of the Local-Global Feature Extraction (LGE) module designed for this invention is shown below.
[0033] Figure 3 The feature aggregation module FA module structure diagram designed for this invention;
[0034] Figure 4 This is a structural diagram of the improved ResNet encoder of this invention. Detailed Implementation
[0035] like Figure 1 As shown, a CT image classification method for nontuberculous mycobacterial lung diseases and pulmonary tuberculosis includes the following steps:
[0036] (I) Example Feature Extraction
[0037] CT images are used for feature extraction via a neural network and then converted into example representations. The specific steps are as follows:
[0038] (1) The CT image X is input into the improved ResNet encoder to extract features X. conv The improved ResNet encoder architecture is visible. Figure 4 The specific improvement involves removing the fourth stage of the ResNet18 encoder and adjusting the stacking order of the first three stages to 3, 3, 3. The improved ResNet encoder increases the number of layers in the first three stages and reduces the number of downsampling operations during feature extraction, preventing smaller lesions from disappearing during feature extraction.
[0039] X conv =T conv (X), #(1)
[0040] Where T conv (.) indicates the improved Resnet encoder of this invention.
[0041] (2) Feature X conv Input to the LGE module modeling feature map X proposed in this inventionenc The LGE module structure is visible. Figure 2 The LGE module consists of one 1x1x1 grouped convolution, a convolutional branch composed of two 3x3x3 grouped convolutional modules, a self-attention branch consisting of one 3x3x3 convolution and two 1x1x1 grouped convolutions, and one 1x1x1 convolutional branch. The LGE module employs a parallel structure of convolution and self-attention mechanisms to extract global image features while preserving local features.
[0042] X enc =T conv+mhsa (X), #(2)
[0043] Where T conv+mhsa (.) indicates three LGE modules improved in this invention.
[0044] (3) The feature map X enc Each feature pixel along its length, width, and height is considered as an example feature encoding x. enc Each feature pixel corresponds to a 16*16*16 area in the original CT image, and each example feature code corresponds to the feature code of the 16*16*16 area in the original image.
[0045] X enc =T conv+mhsa (X).#(3)
[0046] (II) Example Feature Aggregation
[0047] All examples obtained from (I) still need to be processed by the feature aggregation module FA (the specific structure can be seen in [link]). Figure 3 The features are aggregated into packet features to represent the entire CT image information. The feature aggregation process is as follows:
[0048] (1) For all examples obtained from (a), calculate the Key and Value of each example using 3x3x3 convolution and 1x1x1 convolution respectively. Using 3x3x3 convolution allows for the addition of information from surrounding examples to the Key for each example during computation. The formula for the above process is as follows:
[0049] Key=X enc ·W k Value = X enc ·W v , #(4)
[0050] Among them, W k and W v The values represent learnable weights, corresponding to 3*3*3 convolutions and 1*1*1 convolutions, respectively.
[0051] (2) The key obtained from (1) is used to calculate the similarity weight of each example through an MLP (Multilayer Perceptron) structure. The MLP mentioned above consists of two linear layers and a hyperbolic tangent activation function. The above process can be expressed by the following formula:
[0052] Similarity = FC1(Tanh(FC2(X) enc ))), #(5)
[0053] In this context, FC1 and FC2 represent linear layers, and Tanh represents the hyperbolic tangent activation function.
[0054] The Key and Value of each example obtained in (1) are concatenated along the channel, and the Similarity feature obtained in (2) is weighted and aggregated to obtain the bag feature encoding X. bag The above can be expressed by the following formula:
[0055] X bag =Similarity·Concat(Key,Value),#(6)
[0056] Concat(.) indicates concatenation along the channel.
[0057] (III) Package Category Prediction
[0058] (1) The packet feature encoding obtained in (II) is used to predict the packet category Y by the linear layer. predict (The package category reflects the category of the entire CT image). This can be expressed by the following formula:
[0059] Y predict =FC bag (X bag ), #(7)
[0060] Among them, FC bag This indicates that it is used for linear layers to encode and predict bag features as CT image categories.
[0061] (iv) Evaluation of Model Learning Content
[0062] Based on the example relevance weights obtained in (II), examples with high weight values are selected and evaluated using a classifier to represent the degree of relevance between the network's learned content and the expected content. The specific steps are as follows:
[0063] (1) The example feature codes are filtered using the Similarity weights obtained in (II), and the top k important examples X with higher weights (usually 3) are selected. imp This is used to represent the learning content of the network. It can be represented by the following formula:
[0064] Ximp =TopK(Similarity, X) enc ), #(8)
[0065] Among them, TopK(Similarity, X) enc ) indicates from X enc Select the top k examples with the highest Similarity weights.
[0066] (2) Calculate the important example X obtained from (1) using the pre-trained classifier. imp This refers to the correlation between the input example and the lesion information. The classifier is pre-trained using lesion annotations and is mainly used to determine whether the input example is associated with the lesion. The above process can be represented as:
[0067] S instance =FC instace (X imp ), #(10)
[0068] Among them, FC instace This represents a pre-trained classifier. The specific training method for this classifier is as follows: First, each example encoding x obtained in step (a) is processed individually. enc(i) ∈X enc Subsequently, the classifier is used to predict whether each example encoding contains a lesion prediction value; finally, the loss between each example prediction value and the lesion label value is calculated using the cross-entropy loss function.
[0069] (V) Construction of Loss Function
[0070] The loss function includes a bag classification loss L. bag and correlation loss L c The process consists of two parts, and the specific steps are as follows:
[0071] (1) Construct the bag classification loss L using the cross-entropy loss function. bag This is used to measure the distance between the model's predicted value for the CT category and the actual value for the CT category.
[0072] (2) Construct the correlation loss to measure the degree to which the learned content of the model is related to the lesion. The formula for constructing the correlation loss is as follows:
[0073]
[0074] (3) The final loss function is as follows:
[0075] L = L bag +0.2×L c #(12)
[0076] The results of this invention and comparisons with other image classification methods are shown in Table 1. The comparison primarily focuses on classification performance, specifically accuracy, specificity, sensitivity, and AUC score. Accuracy represents the ratio of correctly classified samples to the total number of samples; specificity represents the classification accuracy for all tuberculosis samples; sensitivity represents the classification accuracy for all non-tuberculous mycobacterial lung disease samples; and AUC is the most important metric, representing the probability that the algorithm correctly classifies any randomly given sample. In the task of classifying CT images of non-tuberculous mycobacterial lung disease and tuberculosis, this invention achieved an accuracy of 0.83, a sensitivity of 0.76, and an AUC score of 0.84. The accuracy, sensitivity, and AUC score of this invention are superior to other methods, especially in the AUC score, the main performance indicator for classifiers, demonstrating the effectiveness of this invention.
[0077] Table 1 Comparison results of the present invention with other image classification methods
[0078] Model Year proposed Accuracy Specificity Sensitivity AUC CotNet 2021 0.75 0.83 0.56 0.74 BotNet 2021 0.76 0.83 0.60 0.77 Resnet18 2016 0.72 0.79 0.56 0.73 ViT 2020 0.74 0.94 0.26 0.74 proposed - 0.83 0.86 0.76 0.84
Claims
1. A method for classifying nontuberculous mycobacterial lung diseases and pulmonary tuberculosis using CT images, characterized in that, Includes the following steps: Step S1: Feature extraction is performed on CT images of nontuberculous mycobacterial lung disease and pulmonary tuberculosis using a neural network. Then, each feature pixel of the extracted feature map along its length, width, and height is regarded as an example, and each example represents the feature information of a specific region of the image. Step S2: Calculate the relevance weights of each example obtained in step S1 to the classification task, and aggregate them into a package feature representation by weighted relevance weights to represent the features of the CT image. Step S3: The package feature representation obtained in step S2) is predicted as the category of the CT image using a classifier; Step S4: Based on the example relevance weights obtained in step S2), select examples with high weight values and evaluate them through a classifier to represent the degree of relevance between the network's learned content and the expected content. Step S5: Construct the loss function, which includes bag classification loss and correlation loss. Bag classification loss represents the difference between the predicted CT value and the true value, and correlation loss represents the difference between the model's learned content and the expected content. The correlation loss formula is as follows: 。 2. The method for classifying nontuberculous mycobacterial lung diseases and pulmonary tuberculosis using CT images according to claim 1, characterized in that, The specific steps of step S1 are as follows: In step S11, the data in the training set flows through each convolutional module in the network in sequence. The input image first enters the improved ResNet encoder to extract features. The improved ResNet encoder is ResNet18 with the fourth stage removed, and the number of basic block stacking layers in the first three stages is adjusted to 3, 3, 3. Step S12: The features from step S11) are fed into the Local-Global Feature Extraction (LGE) module to extract feature maps. The LGE module consists of one 1 1 1. Grouped convolution, consisting of two 3 3 A convolutional branch consisting of 3 grouped convolutional modules, with a 3 3 3. Convolution and two 1s 1 1. A self-attention branch composed of grouped convolutions and a 1 1 1. Convolutional branch structure; Step S13: Divide the feature map obtained in step S12) into examples along each feature pixel of length, width and height.
3. The method for classifying nontuberculous mycobacterial lung diseases and pulmonary tuberculosis using CT images according to claim 1, characterized in that, Step S2) uses the attention-based feature aggregation module FA.
4. The method for classifying nontuberculous mycobacterial lung diseases and pulmonary tuberculosis using CT images according to claim 1, characterized in that, Step S4) The classifier used to evaluate the correlation between the network's learned content and the expected content is pre-trained with fixed parameters by lesion annotations.