Deep learning-based pulmonary tuberculosis detection method and system
The comprehensive analysis of the characteristics and similarities of CT sections through the MBMGNN model has solved the problem of excessive dependence on segmentation labels in the prior art, and improved the accuracy and efficiency of tuberculosis detection.
Patent Information
- Application Number
- CN202510467864.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-15
- Publication Date
- 2025-07-29
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing deep learning methods require fully annotated segmentation labels in tuberculosis detection, resulting in overfitting problems during training and failing to effectively utilize the association between CT sections, affecting the diagnostic accuracy.
The MBMGNN model is adopted, and through modules such as shared feature extraction, organizational information extraction, spatial information extraction and attention pooling, combined with graph neural network, the feature vectors and similarity relationships of CT slices are used to perform comprehensive analysis to reduce dependence on segmented labels and improve diagnostic accuracy.
It improves the accuracy and consistency of tuberculosis diagnosis, reduces the workload of labeling workers, makes full use of the correlation between CT sections, and improves the lesion recognition rate.
Smart Images
Figure CN120387995A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical image analysis, and particularly to a tuberculosis detection method and system based on deep learning. Background Art
[0002] Tuberculosis remains one of the leading causes of death globally and is also a major public health problem. Approximately 3 million people die from tuberculosis every year worldwide, making it the infectious disease with the highest number of deaths. Tuberculosis is a lung infectious disease mainly caused by Mycobacterium tuberculosis and has extremely strong infectivity. Although most patients infected with Mycobacterium tuberculosis do not have obvious symptoms, if not treated in time, they may develop into active tuberculosis with a mortality rate as high as 50%. However, if detected early and comprehensively treated, the mortality rate of patients will be reduced. Therefore, early diagnosis of infectious PTB patients and providing timely treatment are of great significance for disease control and interruption of transmission.
[0003] Mycobacterium tuberculosis culture is the gold standard for tuberculosis detection and drug susceptibility testing. However, the traditional Löwenstein-Jensen culture method takes a long time, about 4 - 6 weeks, and by the time the culture results come out, the patient's condition is very likely to have changed. In contrast, computed tomography (CT) plays a crucial role in the diagnosis and monitoring of PTB due to its advantages such as fast speed, rich detail and spatial information, and no need for culture time. However, due to differences in professional knowledge and experience, the diagnoses of different doctors may lack consistency. In addition, there are also problems of large workload and low efficiency. Therefore, in clinical practice, the differential diagnosis of tuberculosis remains a challenge for radiologists and clinicians.
[0004] In recent years, computer-aided detection methods have been widely used in disease analysis and detection in the medical field. In the field of tuberculosis analysis, significant progress has been made in automatically analyzing CT images using deep learning models. Using deep learning technology can automate the analysis of CT images by providing a consistent benchmark and ensure consistency among observers, which helps to overcome the limitations of the lack of qualified professionals.
[0005] However, the existing deep learning methods have the following deficiencies:
[0006] 1. Many existing methods adopt a process of segmentation first and then classification, that is, first segment the lesion features of tuberculosis (such as tree-in-bud sign, nodules, pleural thickening, etc.), and then perform classification prediction of whether it is tuberculosis. This method requires fully annotated segmentation labels during the training process, which usually requires a large amount of labor from domain experts or experienced clinicians. And in the case where it is difficult to obtain detailed labels of medical samples and the available data is limited, the deep learning model may have an overfitting problem during the training process, resulting in the need to improve the classification accuracy.
[0007] 2. There is a strong correlation among dozens or even hundreds of CT slices corresponding to each case in the CT package. However, existing methods mostly only consider partial lesion features within some CT images and do not consider the correlation among CT slice images, which is not conducive to the identification of pulmonary tuberculosis.
[0008] Therefore, there is an urgent need for a new method to solve these problems, so as to improve the lesion recognition rate of the pulmonary tuberculosis diagnosis system, and only provide classification labels (weak labels), while reducing the dependence on segmentation labels (strong labels). Summary of the Invention
[0009] (I) Technical problems to be solved
[0010] In view of the deficiencies of the prior art, the present invention provides a method and system for detecting pulmonary tuberculosis based on deep learning to solve the problems existing in the prior art.
[0011] (II) Technical solutions
[0012] To achieve the above objectives, the present invention is realized through the following technical solutions: The method provided by the present invention includes the following steps:
[0013] Step 1: Obtain pulmonary CT images, and the format of the image data is: each case corresponds to a CT package, and each CT package includes several CT slices;
[0014] Step 2: Preprocess the pulmonary CT image data;
[0015] Step 3: Input the preprocessed pulmonary CT image data into the MBMGNN model to extract the feature vectors corresponding to the CT package, specifically including:
[0016] Step 3.1: Extract the feature vectors of each CT slice through the shared feature extraction module in the MBMGNN model: x i = F net (X i ), i ∈ {1,..., I}, where, and respectively represent the input feature map and the output feature vector of the i-th CT slice in the CT package, I represents the number of CT slices in the CT package, F net represents the mapping function of the shared feature extraction network, C, H, and W respectively represent the number of channels, height, and width of the input feature map, and h1 represents the dimension of the output feature vector x i ;
[0017] Step 3.2: Input the feature vectors x i of the CT slices extracted in Step 3.1 into the first tissue information extraction module F of the MBMGNN modelorg1 , obtain the slice feature vector containing tissue information: o i = F org1 (x i ), i ∈ {1,..., I}, F org1 represents the mapping function of the first tissue information extraction module, and h1 represents the dimension of the output feature vector o i ;
[0018] Step 3.3: Input the feature vector x of the CT slice extracted in Step 3.1 i into the spatial information extraction module F of the MBMGNN model spa , obtain the slice feature vector containing spatial information: s i = F spa (x i ), i ∈ {1,..., I}, F spa represents the mapping function of the spatial information extraction module, and h1 represents the dimension of the output feature vector s i ;
[0019] Step 3.4: Input the s extracted in Step 3.3 i into the second tissue information extraction module F of the MBMGNN model org2 , obtain the slice feature vector containing fusion information: f i = F org2 (s i ), i ∈ {1,..., I}, F org2 represents the mapping function of the second tissue information extraction module, and h1 represents the dimension of the output feature vector f i ;
[0020] Step 3.5: Through the splicing module, splice the feature vectors o i , s i , f i to obtain the spliced vector Specifically, the feature vectors o i and s i are spliced to obtain the feature vectors o i and f i are spliced to obtain the feature vectors s i and f i are spliced to obtain
[0021] Step 3.6: Through the attention pooling module F att pool the spliced vector l = {1, 2, 3} are respectively mapped to the bag feature vector Among them, g represents the dimension of the bag feature vector b l , and F att represents the mapping function of the attention pooling module;
[0022] Step 3.7: Perform weighted summation on the bag feature vector b l to obtain the final bag feature vector b; Step 4: Through the classifier, obtain the binary classification result of whether the CT bag is or is not tuberculosis.
[0023] Preferably, in the step 3.2, the first tissue information extraction module F org1 obtains the slice feature vector o containing tissue information i by the following specific method:
[0024] o i = F org1 (xi) = β i x i , i ∈ {1,..., I},
[0025] where the first tissue information extraction weight β i is
[0026] wherein, and represent parameter matrices, and tanh() is an activation function.
[0027] Preferably, in the step 3.3, the tissue information extraction module F spa obtains the slice feature vector s containing spatial information i by the following specific method:
[0028] For the CT bag, construct an undirected graph G = {N, E}, where N represents the node set of the undirected graph, and each CT slice in the CT bag is regarded as a node, and E represents the edge set of the undirected graph, which indicates whether there is an edge connection between nodes;
[0029] Use the graph attention network to extract spatial information and update the feature vector of the node: s i = F graph (x i ), where s i represents the feature vector of the node after being updated by the graph attention network, and F graph represents the mapping function of the graph attention network;
[0030] Preferably, the acquisition method of the edge set E is:
[0031] Calculate the nodes with the highest correlation with node N i The top k nodes with the highest correlation Among them, f knn (N i , k) represents the mapping function that calculates the cosine similarity between node N i and other nodes and obtains the top k nodes with the highest similarity to the current node N i ;
[0032] Represent the edge set E with the adjacency matrix E ij : N i ∈f knn (N j , k) means that node N i is one of the k adjacent nodes of node N j , N j ∈f knn (N i , k) means that node N j is one of the k adjacent nodes of node N i , E ij =1 indicates that there is an edge connection between node N i and N j , E ij =0 indicates that there is no edge connection between node N i and N j .
[0033] Preferably, use the graph attention network to extract spatial information and update the feature vectors of the nodes, specifically:
[0034] For any node N in the graph G i , by aggregating the features of the k similar nodes i of the node N , update the feature vector of the node N i : Among them, s i is the feature vector of the updated node N i , σ represents the softmax activation function, W g represents the weight matrix, x j represents the feature vector of the node N j , represents the top k nodes with the highest similarity to the node N i , α ij represents the attention coefficient of the node N i to the node N j , and its definition is: Among them, W2 represents the weight matrix, || represents the concatenation operation of feature vectors, and a() represents mapping the concatenated vector to a real number.
[0035] Preferably, in step 3.4, the second tissue information extraction module F org2 The specific method for obtaining the slice feature vector f containing the fusion information is as follows: i of
[0036] f i = F org2 (s i ) = γ i s i , i ∈ {1,..., I},
[0037] Among them, the weight γ of the first tissue information extraction i is
[0038] Among them, and represent parameter matrices, and tanh() represents the tanh activation function.
[0039] Preferably, in step 3.6,
[0040] The mapping function of the attention pooling module F att is specifically:
[0041]
[0042] Among them, b l represents the l-th packet feature vector, represents the feature vector of the CT slice, represents the weight coefficient of the importance of the CT slice,
[0043] Among them, W l , V l and U l represent parameter matrices, sigm represents the sigmoid activation function, and ⊙ represents element-wise multiplication.
[0044] Preferably, step 4 is specifically:
[0045] Send the packet feature vector b of the CT packet into the fully connected layer for classification prediction to obtain the probability that the case indicated by the CT packet has tuberculosis Mark the CT packet with a probability greater than the preset threshold ζ as having tuberculosis, otherwise, mark it as not having tuberculosis: Among them, represents having tuberculosis, represents not having tuberculosis.
[0046] Preferably, the loss function used when training the model is as follows:
[0047]
[0048] where L represents the model loss, N represents the number of CT packs, and y i ∈ {0, 1}, and represents the true class and predicted probability of the i-th CT pack.
[0049] To achieve the above invention purpose, the present invention also provides a tuberculosis detection system based on deep learning. The system is used to implement the above-mentioned tuberculosis detection method based on deep learning, and the system includes:
[0050] Image acquisition module: used to acquire lung CT image data;
[0051] Image preprocessing module: used to perform preprocessing operations of lung parenchyma segmentation and redundant CT slice removal on the acquired CT image data;
[0052] Shared feature extraction module: used to extract feature vectors from the preprocessed CT image slices respectively, and send the extracted feature vectors into the first tissue information extraction module and the spatial information extraction module respectively;
[0053] First tissue information extraction module: used to extract tissue information and update the feature vectors of the CT slices;
[0054] Spatial information extraction module: used to obtain spatial information and update the feature vectors of the CT slices;
[0055] Second tissue information extraction module: used to obtain the feature vectors of the CT slices containing fusion information;
[0056] Stitching module: respectively stitch the feature vectors of the CT slices output by the first tissue information extraction module, the spatial information extraction module, and the second tissue information extraction module pairwise to obtain three groups of stitched feature vectors;
[0057] Attention pooling module: used to respectively perform weighted integration on the three groups of stitched feature vectors of the CT slices to obtain three groups of pack feature vectors of the CT packs where the CT slices are located;
[0058] Feature aggregation module: used to perform weighted summation on the three groups of pack feature vectors to obtain the final pack feature vector of the CT pack;
[0059] Classification prediction module: used to obtain a binary classification prediction result of whether the CT pack is or is not tuberculosis according to the final pack feature vector.
[0060] (III) Advantageous Effects
[0061] The present invention provides a processing device for semiconductor wafers. It has the following advantageous effects:
[0062] (I). The present invention introduces a multi-instance network, such that the model proposed in the present application does not require providing segmentation annotations for lesions in each CT slice during the training process, but only needs to provide a classification annotation for each CT package corresponding to each case, that is, whether the case is a tuberculosis patient or not. This reduces the requirement for the fineness of labels and reduces the workload of annotators.
[0063] (II). Through the multi-instance network, all CT slices in the CT package can be comprehensively analyzed, taking all CT slices into consideration, and information that can determine the pathological changes of tuberculosis can be obtained, and then a judgment is made on whether the CT package is tuberculosis, improving the diagnostic accuracy of tuberculosis.
[0064] (III). The present invention uses a graph neural network to model with CT slices as the vertices of the graph and determines the existence of edges based on the distance of the feature vectors of CT slices, and uses the message passing mechanism of the graph neural network to update the node representations, fully considering the correlation between CT slices located in a CT package, and improving the recognition rate of tuberculosis. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] Figure 1 It is a flowchart of a tuberculosis detection method based on deep learning provided by the present invention.
[0066] Figure 2 It is a schematic structural diagram of a tuberculosis detection system based on deep learning provided by the present invention.
[0067] Figure 3 It is a network architecture diagram of the MBMGNN model. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0068] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
[0069] Embodiment 1
[0070] This embodiment provides a tuberculosis detection method based on deep learning, as Figure 1 shown, including the following steps:
[0071] Step 1: Obtain lung CT images. The format of the CT image data is as follows: Each case corresponds to a CT package, and each CT package contains several CT slices. Here, the cases include tuberculosis patients and non-tuberculosis patients. Each CT slice is a two-dimensional image.
[0072] Step 2: Preprocess the lung CT image data. The preprocessing steps include but are not limited to the steps of segmenting the lung parenchyma and removing redundant CT slices. Among them, the purpose of segmenting the lung parenchyma is to extract the effective lung region from the CT slices and reduce the influence of irrelevant regions. The lung parenchyma can be segmented by the region growing method or the global threshold method.
[0073] Step 3: Input the preprocessed lung CT image data into the MBMGNN model to extract the feature vector corresponding to the CT package, specifically including:
[0074] Step 3.1: Extract the feature vector of each CT slice through the shared feature extraction module in the MBMGNN model: x i = F net (X i ), i ∈ {1,..., I}, where and respectively represent the input feature map and the output feature vector of the i-th CT slice in the CT package, I represents the number of CT slices in the CT package, F net represents the mapping function of the shared feature extraction network. Here, the shared feature extraction network adopts the Resnet50 network architecture, C, H, and W respectively represent the number of channels, height, and width of the input feature map, and h1 represents the dimension of the output feature vector x i .
[0075] Step 3.2: Input the feature vector x i of the CT slice extracted in Step 3.1 into the first tissue information extraction module F org1 of the MBMGNN model to obtain the slice feature vector containing tissue information: o i = F org1 (x i ), i ∈ {1,..., I}, F org1 represents the mapping function of the first tissue information extraction module, and h1 represents the dimension of the output feature vector o i . Among them,
[0076] The specific method for the first tissue information extraction module F org1 to obtain the slice feature vector o i containing tissue information is as follows:
[0077] o i = Forg1 (x i ) = β i x i , i ∈ {1,..., I},
[0078] where the weight β for extracting the first tissue information i is
[0079]
[0080] where and represent parameter matrices, and tanh() is an activation function.
[0081] Step 3.3: Input the feature vector x of the CT slice extracted in Step 3.1 i into the group space information extraction module F of the MBMGNN model spa to obtain a slice feature vector containing spatial information: s i = F spa (x i ), i ∈ {1,..., I}, F spa represents the mapping function of the spatial information extraction module, and h1 represents the dimension of the output feature vector s i . Wherein, the specific method for the tissue information extraction module F spa to obtain the slice feature vector s containing spatial information i is as follows:
[0082] Step 3.3.1: For the CT package, construct an undirected graph G = {N, E}, where N represents the node set of the undirected graph, and each CT slice in the CT package is regarded as a node, and E represents the edge set of the undirected graph, which indicates whether there is an edge connection between nodes.
[0083] The way to obtain the edge set E is:
[0084] First, calculate the cosine similarity between pairwise nodes where x i is the feature vector of node N i , x j is the feature vector of node N j , x i ·x j is the dot product of vector x i and vector x j , ||x i || and ||x j || are the norms of vector x i and vector x jThe Euclidean length (or modulus). The cosine similarity value ranges from [-1, 1], and the closer it is to 1, the higher the correlation.
[0085] Secondly, calculate the k nodes i with the highest correlation with node N where f knn (N i , k) represents the mapping function that calculates the cosine similarity between node N i and other nodes and obtains the top k nodes with the highest similarity to the current node N i ;
[0086] Finally, represent the edge set E with the adjacency matrix E ij : N i ∈ f knn (N j means that node N i is one of the k adjacent nodes of node N j , N j ∈ f knn (N i means that node N j is one of the k adjacent nodes of node N i , E ij = 1 indicates that there is an edge connection between node N i and N j , E ij = 0 indicates that there is no edge connection between node N i and N j .
[0087] Step 3.3.2: Use the graph attention network to extract spatial information and update the feature vectors of the nodes in the undirected graph constructed above: s i = F graph (x i ), where s i represents the feature vector of node N i after being updated by the graph attention network, and F graph represents the mapping function of the graph attention network.
[0088] Among them, the method of using the graph attention network to extract spatial information and update the feature vectors of the nodes is specifically:
[0089] For any node N i in the graph G, by aggregating the features of the k similar nodes i of the node N , update the feature vector of the node N i : where si is the updated node N i 's feature vector, σ represents the softmax activation function, W g represents the weight matrix, x j represents the feature vector of node N j ; represents the top k nodes with the highest similarity to the node N i , α ij represents the attention coefficient of node N i to node N j , which is defined as: where W2 represents the weight matrix, || represents the concatenation operation of feature vectors, a() represents mapping the concatenated vector to a real number, exp() represents the exponential function, and LeakyReLU() represents the LeakyReLU activation function.
[0090] Step 3.4: Input the s i extracted in Step 3.3 into the second tissue information extraction module F org2 of the MBMGNN model to obtain the slice feature vector containing the fused information: f i = F org2 (s i ), i ∈ {1,..., I}, ]>F org2 represents the mapping function of the second tissue information extraction module, and h1 represents the dimension of the output feature vector f i .
[0091] Among them, the specific method for the second tissue information extraction module F org2 to obtain the slice feature vector f i containing the fused information is as follows:
[0092] f i = F org2 (s i ) = γ i s i , i ∈ {1,..., I},
[0093] where the first tissue information extraction weight γ i ]>is
[0094] where and represent the parameter matrices, and tanh() represents the tanh activation function.
[0095] Step 3.5: Through the concatenation module, for the feature vectors o i , s i , f iPerform vector concatenation to obtain a concatenated vector Specifically, the feature vector o i and s i are concatenated to obtain The feature vector o i and f i are concatenated to obtain The feature vector s i and f i are concatenated to obtain
[0096] Step 3.6: Through the attention pooling module F att Map the concatenated vector l = {1, 2, 3} to the bag feature vector where g represents the dimension of the bag feature vector b l and F att represents the mapping function of the attention pooling module.
[0097] Among them, the mapping function of the attention pooling module F att is specifically as follows:
[0098]
[0099] where b l represents the l-th bag feature vector, represents the feature vector of the CT slice, represents the weight coefficient of the importance of the CT slice,
[0100] where W l 、V l and U l represent parameter matrices, sigm represents the sigmoid activation function, and ⊙ represents element-wise multiplication.
[0101] Step 3.7: Perform weighted summation on the bag feature vector b l to obtain the final bag feature vector of the CT bag: b = w1b1 + w2b2 + w3b3, where b is the bag feature vector of the CT bag, and w1, w2, w3 are weight coefficients.
[0102] Step 4: Obtain the binary classification result of whether the CT bag is or is not tuberculosis through a classifier. Specifically:[[]]
[0103] Send the bag feature vector b of the CT bag into the fully connected layer for classification prediction to obtain the probability that the case indicated by the CT bag has tuberculosis Mark the CT bag with a probability greater than the preset threshold ζ as having tuberculosis, otherwise, mark it as not having tuberculosis: Among them, indicates suffering from pulmonary tuberculosis, indicates not suffering from pulmonary tuberculosis.
[0104] In addition, for the pulmonary tuberculosis detection method based on deep learning described in the present invention, during the process of training the model, the loss function used by this method is:
[0105] Among them, L represents the model loss, N represents the number of CT packages, y i ∈ {0, 1} and represents the true category and predicted probability of the i-th CT package.
[0106] In order to verify the effectiveness of the method provided in this embodiment, experiments were conducted on a clinical dataset of CT scan packages of 543 tuberculosis patients and 222 non-tuberculosis patients, and comparisons were made with other existing methods, including the resnet50 model and the efficientnet_b0 model. The experimental results are shown in Table 1:
[0107] Table 1: Performance comparison of different methods on the test dataset
[0108] Model F1 Score AUC Sensitivity Specificity Accuracy PPV NPV resnet50 0.944 0.990 0.935 0.953 0.944 0.953 0.935 efficientnet_b0 0.958 0.986 0.944 0.972 0.958 0.971 0.945 The method in this paper 0.962 0.991 0.935 0.991 0.963 0.990 0.938
[0109] It can be seen that the method provided in this embodiment has achieved optimal performance in terms of F1 score, AUC, specificity, accuracy, and PPV, reaching 96.2%, 99.1%, 99.1%, 96.3%, and 99.0% respectively, which proves the reliability and superiority of this method.
[0110] In addition, ablation experiments were also conducted in this embodiment to verify the roles of various innovative mechanisms in the method. The results are shown in Table 2:
[0111] Table 2: Ablation experiment results
[0112] Model F1-Score AUC Sensitivity Specificity Accuracy PPV NPV No spatial information extraction module 0.929 0.979 0.963 0.887 0.925 0.897 0.959 No first tissue information extraction 0.911 0.960 0.898 0.925 0.911 0.924 0.899 No second tissue information extraction 0.913 0.977 0.870 0.962 0.916 0.959 0.879 The method in this paper 0.962 0.991 0.935 0.991 0.963 0.990 0.938
[0113] It can be seen that the spatial information extraction module, the first tissue information extraction module, and the second tissue information extraction module all make significant contributions to the improvement of the model performance. After removing these modules, the model performance has decreased to varying degrees, confirming the roles of these modules in the present invention.
[0114] Example 2
[0115] This embodiment provides a pulmonary tuberculosis detection system based on deep learning, as Figure 2As shown in the figure, it includes an image acquisition module, an image preprocessing module, a shared feature extraction module, a first tissue information extraction module, a spatial information extraction module, a second tissue information extraction module, a splicing module, an attention pooling module, a feature aggregation module, and a classification prediction module. Specifically,
[0116] Image acquisition module: used to acquire pulmonary CT image data;
[0117] Image preprocessing module: used to perform preprocessing operations of lung parenchyma segmentation and redundant CT slice removal on the acquired CT image data;
[0118] Shared feature extraction module: used to extract feature vectors from the preprocessed CT image slices respectively, and send the extracted feature vectors to the first tissue information extraction module and the spatial information extraction module respectively;
[0119] First tissue information extraction module: used to extract tissue information and update the feature vectors of the CT slices;
[0120] Spatial information extraction module: used to obtain spatial information and update the feature vectors of the CT slices;
[0121] Second tissue information extraction module: used to obtain the feature vectors of the CT slices containing fusion information;
[0122] Splicing module: respectively splice the feature vectors of the CT slices output by the first tissue information extraction module, the spatial information extraction module, and the second tissue information extraction module pairwise to obtain three groups of spliced feature vectors;
[0123] Attention pooling module: used to perform weighted integration on the three groups of spliced feature vectors of the CT slices respectively to obtain three groups of packet feature vectors of the CT packet where the CT slices are located;
[0124] Feature aggregation module: used to perform weighted summation on the three groups of packet feature vectors to obtain the final packet feature vector of the CT packet;
[0125] Classification prediction module: used to obtain a binary classification prediction result of whether the CT packet is or is not pulmonary tuberculosis based on the final packet feature vector.
Claims
1. A tuberculosis detection method based on deep learning, characterized in that, The method includes the following steps: Step 1: Obtain lung CT images, and the format of the image data is: each case corresponds to a CT package, and each CT package includes several CT slices; Step 2: Preprocess the lung CT image data; Step 3: Input the preprocessed lung CT image data into the MBMGNN model to extract the feature vectors corresponding to the CT packages, specifically including: Step 3.1: Extract the feature vector of each CT slice through the shared feature extraction module in the MBMGNN model: x i = F net (X i ), i ∈ {1,..., I}, where and represent the input feature map and the output feature vector of the i-th CT slice in the CT packet respectively, I represents the number of CT slices in the CT packet, and F net represents the mapping function of the shared feature extraction network, C, H, and W represent the number of channels, height, and width of the input feature map respectively, and h1 represents the dimension of the output feature vector x i ; Step 3.2: The feature vector x of the CT slice extracted in Step 3.1 i is input into the first tissue information extraction module F of the MBMGNN model org1 to obtain a slice feature vector containing tissue information: o i = F org1 (x i ), i ∈ {1,...I}, F org1 represents the mapping function of the first tissue information extraction module, and h1 represents the dimension of the output feature vector o i ; Step 3.3: Input the feature vector x of the CT slice extracted in Step 3.1 i into the spatial information extraction module F of the MBMGNN model spa , and obtain the slice feature vector containing spatial information: s i = F spa (x i ), i ∈ {1,..., I}, F spa represents the mapping function of the spatial information extraction module, and h1 represents the dimension of the output feature vector s i ; Step 3.4: Input the s extracted in the above Step 3.3 i into the second tissue information extraction module F of the MBMGNN model org2 to obtain the slice feature vector containing the fusion information: f i = F org2 (s i ), i ∈ {1,..., I}, F org2 represents the mapping function of the second tissue information extraction module, and h1 represents the dimension of the output feature vector f i . Step 3.5: Use the splicing module to splice the feature vectors o i , s i , f i to obtain a spliced vector Specifically, the feature vectors o i and s i are spliced to obtain The feature vectors o i and f i are spliced to obtain The feature vectors s i and f i are spliced to obtain Step 3.6: Through the attention pooling module F att Map the concatenated vector to the bag feature vectors respectively where g represents the dimension of the bag feature vector b l and F att represents the mapping function of the attention pooling module; Step 3.7: Perform weighted summation on the packet feature vector b l to obtain the final packet feature vector b; Step 4: Obtain the binary classification result of whether the CT package is or is not tuberculosis through a classifier.
2. The tuberculosis detection method based on deep learning according to claim 1, wherein In step 3.2, the first tissue information extraction module F org1 obtains the slice feature vector o containing tissue information i The specific method is as follows: o i = F org1 (x i ) = β i x i , i ∈ {1,...O} Among them, the first tissue information extraction weight β i is Among them, and represent the parameter matrix, and tanh() is the activation function.
3. The tuberculosis detection method based on deep learning according to claim 1, wherein, In step 3.3, the tissue information extraction module F spa obtains the slice feature vector s containing spatial information i The specific method is as follows: For the CT package, construct an undirected graph G = {N, E}, where N represents the node set of the undirected graph, and each CT slice in the CT package is regarded as a node, and E represents the edge set of the undirected graph, which indicates whether there is an edge connection between the nodes; Extract spatial information using a graph attention network and update the feature vector of the node: s i = F graph (x i ), where s i represents the feature vector of the node after being updated by the graph attention network, and F graph represents the mapping function of the graph attention network.
4. The tuberculosis detection method based on deep learning according to claim 3, characterized in that, The acquisition method of the edge set E is: Calculate the nodes with the highest correlation with node N i The top k nodes with the highest correlation Among them, f knn (N i , k) represents the mapping function that calculates the cosine similarity between node N i and other nodes and obtains the top k nodes with the highest similarity to the current node N i ; The edge set E is represented by an adjacency matrix E ij as follows: N i ∈ f knn (N j , k) means that node N i is one of the k adjacent nodes of node N j , N j ∈ f knn (N i , k) means that node N j is one of the k adjacent nodes of node N i . E ij = 1 means that there is an edge connection between node N i and N j . E ij = 0 means that there is no edge connection between node N i and N j .
5. The tuberculosis detection method based on deep learning according to claim 3, characterized in that Use the graph attention network to extract spatial information and update the feature vectors of the nodes, specifically: For any node N in the graph G i , by aggregating the features of k similar nodes i of the node N , update the feature vector of the node N i : where s i is the feature vector of the updated node N i , σ represents the softmax activation function, W g represents the weight matrix, x j represents the feature vector of the node N j , represents the top k nodes with the highest similarity to the node N i , α ij represents the attention coefficient of the node N i to the node N j , and its definition is: where W2 represents the weight matrix, || represents the concatenation operation of feature vectors, and a() represents mapping the concatenated vector to a real number.
6. The tuberculosis detection method based on deep learning according to claim 1, wherein In step 3.4, the second tissue information extraction module F org2 obtains the slice feature vector f containing the fusion information i The specific method is as follows: f i = F org2 (s i ) = γ i s i ,i ∈ {1,..., I} Among them, the first tissue information extraction weight γ i is Among them, and represent the parameter matrix, and tanh() represents the tanh activation function.
7. The tuberculosis detection method based on deep learning according to claim 1, characterized in that, In step 3.6, Attention pooling module F att The mapping function of Among them, b l represents the l-th packet feature vector, represents the feature vector of the CT slice, represents the weight coefficient of the importance of the CT slice, Among them, W l , V l and U l represent parameter matrices, sigm represents the sigmoid activation function, and ⊙ represents element-wise multiplication.
8. The tuberculosis detection method based on deep learning according to claim 1, characterized in that, Step 4 is specifically: Send the packet feature vector b of the CT packet into a fully connected layer for classification prediction to obtain the probability that the case indicated by the CT packet has pulmonary tuberculosis Mark the CT packets with probabilities greater than the preset threshold ζ as having pulmonary tuberculosis; otherwise, mark them as not having pulmonary tuberculosis: where indicates having pulmonary tuberculosis indicates not having pulmonary tuberculosis 9. The method according to any one of claims 1-8, characterized in that, The loss function used by the method when training the model is: where L represents the model loss, N represents the number of CT packages, y i ∈ {0, 1} and represent the true class and predicted probability of the i-th CT package respectively.
10. A tuberculosis detection system based on deep learning, characterized in that, The system is used to implement the method according to any one of claims 1-9, and the system includes: An image acquisition module: used to acquire lung CT image data; An image preprocessing module: used to perform preprocessing operations such as lung parenchyma segmentation and redundant CT slice removal on the acquired CT image data; A shared feature extraction module: used to extract feature vectors from the preprocessed CT image slices respectively, and send the extracted feature vectors to the first tissue information extraction module and the spatial information extraction module respectively; A first tissue information extraction module: used to extract tissue information and update the feature vectors of the CT slices; A spatial information extraction module: used to obtain spatial information and update the feature vectors of the CT slices; A second tissue information extraction module: used to obtain the feature vectors of the CT slices containing fusion information; A splicing module: splice the feature vectors of the CT slices output by the first tissue information extraction module, the spatial information extraction module, and the second tissue information extraction module pairwise to obtain three groups of spliced feature vectors; An attention pooling module: used to perform weighted integration on the three groups of spliced feature vectors of the CT slices respectively to obtain three groups of package feature vectors of the CT package where the CT slices are located; A feature aggregation module: used to perform weighted summation on the three groups of package feature vectors to obtain the final package feature vector of the CT package; A classification prediction module: used to obtain the binary classification prediction result of whether the CT package is or is not tuberculosis according to the final package feature vector.