Pulmonary tuberculosis detection system based on attention and feature pyramid

Abstract

The invention discloses a pulmonary tuberculosis detection system based on attention and a feature pyramid, and the system comprises an image feature obtaining module which is used for obtaining pulmonary tuberculosis CT chest image features; the data preprocessing module is used for carrying out data preprocessing on the obtained pulmonary tuberculosis CT chest image features; the detection model module is used for detecting and classifying the preprocessed pulmonary tuberculosis CT chest image features; wherein the detection model module adopts a Faster RCNN model for detection and classification, the feature extraction network module in the model adopts a ResNet50 network, a residual structure of the ResNet50 network is embedded into the SE attention module, and a multi-scale feature map is constructed through a feature pyramid module. According to the method, an SE attention module is added on the basis of a two-stage detection network Faster RCNN to enhance the detection accuracy of the model on a PTB small focus, and then a multi-scale feature map is generated through a feature pyramid structure to better capture different details and positions of a target, so that the positioning of target detection is more accurate.

Description

Technical Field

[0001] The invention belongs to the technical field of pulmonary tuberculosis classification detection devices and relates to a pulmonary tuberculosis detection system based on attention and feature pyramid. Background Art

[0002] Tuberculosis (TB) ranks second in global mortality, second only to COVID-19. The ongoing global COVID-19 pandemic has undermined years of progress in TB control and severely affected efforts to manage and treat TB. In clinical diagnosis, bacterial culture and microscopy are the gold standards for TB diagnosis, but they have certain limitations in terms of time, sensitivity, and technical requirements.

[0003] In recent years, deep learning has been widely used in medical imaging diagnosis, demonstrating its ability to automatically learn data and its efficiency in extracting valuable feature information. The paper "Wang Min, Zhou Gaoxi, Wang Xun, et al. Semi-supervised learning segmentation network for lung cancer based on dual attention [J]. Optoelectronics Laser, 2023, 34(02): 132-139" proposed a dual-attention semi-supervised learning network (SDA-Net) to solve the problem of pulmonary nodule segmentation in computed tomography (CT) images. It can use both labeled and unlabeled data for training, significantly improving the segmentation performance. The paper "Xie Y, Zhang J, Xia Y. Semi-supervised adversarial model for benign–malignant lung nodule classification on chest CT [J]. Medical image analysis, 2019, 57: 237-248" proposed a multi-view knowledge collaborative deep model (MV-KBC) to decompose lung nodules into nine fixed views to capture their features with an accuracy of 91.60%. The paper "Zheng S, Guo J, Cui X, et al. Automatic pulmonary nodule detection in CT scans using convolutional neural networks based on maximum intensity projection [J]. IEEE transactions on medical imaging, 2019, 39 (3): 797-805" proposed a pulmonary nodule detection model that integrates multi-layer maximum intensity projection (MIP) to enhance its spatial representation. The sensitivity of detecting false positives in single and double scans was 92.7% and 94.2%, respectively.The paper "Masood A, Sheng B, Yang P, et al. Automated decision support system for lung cancer detection and classification via enhanced RFCN with multilayer fusion RPN[J]. IEEE Transactions on Industrial Informatics, 2020, 16(12): 7791-7801" proposed an enhanced multidimensional region detection model (mRFCN) for automatic detection of lung nodules, with a classification accuracy and sensitivity of 97.91% and 98.1% respectively. The paper "Maidili Nigati, Mizhiguli Damaoula, Zhang Bin, et al. Construction and verification of active / inactive pulmonary tuberculosis classification model based on CT deep learning [J]. Journal of Jinan University (Natural Science and Medicine Edition), 2023, 44(01): 69-77+86" first attempted to use 3D ResNet-50 to distinguish active / inactive pulmonary tuberculosis and used class activation maps to visualize the most indicative areas of disease prediction on CT scans, but it was limited in determining the type of lesions.

[0004] At present, the development of artificial intelligence systems for the classification of pulmonary tuberculosis (PTB) is not yet mature. Although both pulmonary tuberculosis lesions and pulmonary nodules are lung lesions, both can be detected and diagnosed through imaging examinations such as X-rays and CT scans, but PTB lesions are irregular in shape, with large differences in lesion size, and diverse manifestations (such as frosted shadows, cavities, fibrosis, etc.), and lesion boundaries are blurred, making it difficult to accurately distinguish lesions from normal tissues, which increases the difficulty of classification model processing. In addition, the segmentation model is highly dependent on training data. When dealing with unseen lesion types and morphologies, the generalization ability may be insufficient and the segmentation accuracy is not high.

[0005] In summary, the complexity, diversity and heterogeneity of PTB lesions make it difficult for segmentation models to handle and effectively classify lesions. In addition, the three-dimensional annotation of PTB lesions requires not only rich medical imaging knowledge and experience, but also the accuracy and consistency of the three-dimensional annotation, which is time-consuming, labor-intensive and difficult. Summary of the invention

[0006] The technical solution adopted by the present invention is: a tuberculosis detection system based on attention and feature pyramid, which can realize accurate classification of lesions.

[0007] To solve this problem, the technical solution adopted by the present invention is:

[0008] A tuberculosis detection system based on attention and feature pyramid, including:

[0009] An image feature acquisition module is used to acquire features of pulmonary tuberculosis CT chest images;

[0010] A data preprocessing module, used for preprocessing the acquired features of the pulmonary tuberculosis CT chest image;

[0011] The detection model module is used to detect and classify the preprocessed tuberculosis CT chest image features; the detection model module uses the Faster RCNN model for detection and classification. The Faster RCNN model includes a feature extraction network module, an RPN module, a RoIPooling module, and a classification and regression module. The feature extraction network module uses the ResNet50 network. The residual structure of the ResNet50 network is embedded in the SE attention module, and a multi-scale feature map is constructed through the feature pyramid module.

[0012] Furthermore, the data preprocessing module uses the K-Means clustering segmentation algorithm to preprocess the extracted image features. The specific steps are as follows:

[0013] 1) Normalize the slice pixel value from [0, 255] to [0, 1];

[0014]

[0015] Where, X i Represents the feature vector of the i-th CT slice. Usually, these feature vectors represent the pixel values ​​of the lung parenchyma area in the CT image;

[0016] u k represents the centroid of the kth cluster. k The center of a specific cluster representing lung tissue, which is the average feature of different lung regions that the clustering algorithm tries to find;

[0017] n k It indicates the number of data points contained in the kth cluster, that is, the number of lung parenchyma pixels belonging to this cluster;

[0018] J represents the objective function. This objective function optimizes the clustering process by minimizing the distance from the pixel to its centroid, so that similar pixels are clustered into the same class;

[0019] ||X i -u k || 2 Represents a pixel X i To cluster center u k The square of the Euclidean distance;

[0020] J represents the objective function. This objective function optimizes the clustering process by minimizing the distance from a pixel to its centroid so that similar pixels are clustered into the same class.

[0021] 2) Separate the foreground (lung ROI) and background through K-Means clustering segmentation algorithm and binary threshold method;

[0022] 3) Perform morphological (erosion / dilation) operations on the separated images;

[0023] 4) Use different colors to mark the separated lung areas;

[0024] 5) Create a mask based on the color marker and expand the mask area again;

[0025] 6) Perform convex hull operation on the filled area to retain more complete lung area information;

[0026] 7) Segment the ROI by multiplying the image in step 1) with the mask in step 6).

[0027] Furthermore, the SE attention module includes a compression module and an excitation module. In the structure of the compression module, the feature map (f i ) is compressed into a 1×1×C feature vector through global pooling; in the bottleneck structure of the excitation module, global learning and capture of the correlation between various feature channels occurs, and then the learned attention weights are assigned to the corresponding feature channels to obtain an output feature map of H×W×C with attention annotations

[0028] (f i '):

[0029]

[0030] Excition: F ex (z,w)=σ(W2σ(W1z)) (4)

[0031] Scale: F scale (u c ,s c )=s c ·u c (5);

[0032] Where H represents the height of the input feature map;

[0033] W represents the width of the input feature map;

[0034] u c (i, j) represents the pixel value at position (i, j) on channel c;

[0035] F sq (u c ) The compressed result (1×1×C feature vector);

[0036] W1 and W2 represent the weight matrices of the fully connected layer, which are used to reduce and increase the channel dimension respectively;

[0037] z represents the feature vector after global pooling;

[0038] w represents the final channel attention weight vector obtained in the Excitation operation;

[0039] σ represents the activation function, where σ(W1z) is the ReLU activation and σ(W2σ(W1z)) is the Sigmoid activation, which is used to generate channel attention weights;

[0040] F ex (z,w) represents the output weight vector of the Excitation operation;

[0041] s c Represents the channel weight generated by the Excitation operation;

[0042] Fscale(u c ,s c ) represents the weighted output features.

[0043] Furthermore, the implementation method of the feature pyramid module is as follows: first, extract feature maps of different levels from CT slices (512×512×3); then, starting from the highest-level feature map (16×16×256), gradually transfer the semantic information of the high-level feature map to the low-level feature map through upsampling. After each upsampling, the feature map is laterally connected with the corresponding underlying feature map (bottom-up), either output or continue to be upsampled; finally, multi-scale feature maps P2, P3, P4, P5 and P6 are generated.

[0044] Furthermore, LabelImg is used to label the lesion information of the above-mentioned tuberculosis CT chest image feature data.

[0045] Furthermore, in Faster R-CNN, L(p,u,t u ,v) is the total loss function of the model, L cla (p,u) is the classification loss function, L reg (t u ,v) is the regression loss function:

[0046] L(p,u,t u ,v)=Lcla (p,u)+λL reg (t u ,v) (6)

[0047] L cla (p,u)=-logp u (7)

[0048]

[0049] Where P is the probability distribution predicted by the classifier using softmax; u is the actual category label of the target; t u is the regression parameter of the bounding box regressor predicting the corresponding category u v is the bounding box regression parameter corresponding to the true target β is the threshold variable; s is the difference in regression error, which represents the error between the model prediction and the actual bounding box.

[0050]

[0051] In the formula, IoU is the intersection over union ratio, which quantifies the degree of intersection between the bounding box predicted by the model and the true target bounding box, A represents the bounding box predicted by the model, and B represents the bounding box of the true annotation. A∩B represents the intersection of bounding boxes A and B, that is, the area of ​​the overlap between the predicted box and the true box, and A∪B represents the union of bounding boxes A and B, that is, the intersection part and the non-overlapping part between the two.

[0052] Beneficial effects of the present invention: Compared with the prior art, the present invention segments the lung parenchyma based on data preprocessing, effectively reducing the false detection rate. At the same time, the SE attention module is added to the two-stage detection network Faster RCNN to enhance the model's detection accuracy for small PTB lesions, and then a multi-scale feature map is generated through a feature pyramid (FPN) structure to better capture the different details and positions of the target, making the positioning of target detection more accurate. The results show that the detection effect of the improved model can meet the expected requirements, and can be used as an auxiliary tool for doctors to provide a second opinion and help doctors make a more accurate diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Schematic diagram of data preprocessing;

[0054] Figure 2 This is the Faster R-CNN framework diagram;

[0055] Figure 3 SE attention module diagram

[0056] Figure 4 It is the FPN structure;

[0057] Figure 5 Iterate the curve for the performance index;

[0058] Figure 6 To improve the detection of Faster RCNN, the external detection effect diagram includes (a) infiltrative type, (b) miliary type, (c) caseous type, (d) tuberculous type and (e) cavitary type;

[0059] Figure 7 This is a flow chart of the tuberculosis detection method. DETAILED DESCRIPTION

[0060] The present invention is further described below in conjunction with specific embodiments.

[0061] Example 1: Figure 1 As shown, a tuberculosis detection system based on attention and feature pyramid includes:

[0062] An image feature acquisition module is used to acquire features of pulmonary tuberculosis CT chest images;

[0063] A data preprocessing module, used for preprocessing the acquired features of the pulmonary tuberculosis CT chest image;

[0064] The detection model module is used to detect and classify the features of the preprocessed pulmonary tuberculosis CT chest image; the detection model module uses the Faster RCNN model for detection and classification. The Faster RCNN model includes a feature extraction network module, an RPN module, a RoIPooling module, and a classification and regression module. The model framework is as follows: Figure 2 As shown in the figure, the feature extraction network module adopts the ResNet50 network. The residual structure of the ResNet50 network is embedded in the SE attention module, and a multi-scale feature map is constructed through the feature pyramid module. The feature extraction network module adopts ResNet50. In order to better capture important feature information, the SE attention module is embedded in the residual structure of ResNet50 to enhance the feature representation ability of the network, and a multi-scale feature map is generated through the FPN structure to enhance the model's detection ability for lesions of different scales.

[0065] Furthermore, the data preprocessing module uses the K-Means clustering segmentation algorithm to preprocess the extracted image features. The K-Means clustering segmentation algorithm is an unsupervised machine learning algorithm whose goal is to minimize the objective function J so that each data point is assigned to the closest centroid and forms K clusters, where the distance between the data points in each cluster and the centroid is the smallest. In each iteration, the K-Means algorithm updates the centroid of each cluster until the centroid shows the smallest change.

[0066] The present invention performs lung parenchyma segmentation on CT slices based on the K-Means clustering segmentation algorithm. The specific steps are as follows:

[0067] 1) Normalize the slice pixel value from [0, 255] to [0, 1], such as Figure 1 (a);

[0068]

[0069]

[0070] X i Represents the feature vector of the i-th CT slice. Usually, these feature vectors represent the pixel values ​​of the lung parenchyma area in the CT image;

[0071] u k represents the centroid of the kth cluster. k The center of a specific cluster representing lung tissue, which is the average feature of different lung regions that the clustering algorithm tries to find;

[0072] n k It indicates the number of data points contained in the kth cluster, that is, the number of lung parenchyma pixels belonging to this cluster;

[0073] ||X i -u k || 2 Represents a pixel X i To cluster center u k The square of the Euclidean distance;

[0074] J represents the objective function. This objective function optimizes the clustering process by minimizing the distance from a pixel to its centroid so that similar pixels are clustered into the same class.

[0075] 2) The foreground (lung ROI) and background are separated by K-Means clustering segmentation algorithm and binary threshold method, such as Figure 1 (b);

[0076] 3) Perform morphological (erosion / dilation) operations on the separated images, such as Figure 1 (c);

[0077] 4) Use different colors to mark the separated lung regions, such as Figure 1 (d);

[0078] 5) Create a mask based on the color marker and dilate to fill the mask area again, such as Figure 1 (d);

[0079] 6) Perform convex hull operation on the filled area to retain more complete lung area information, such as Figure 1 (f);

[0080] 7) Multiply the image in step 1) and the mask in step 6) to segment the ROI, such as Figure 1 (g).

[0081] Among them, the SE attention module adaptively learns the global information of the channel, re-weights the input features, improves the network's perception of important features in the image, and thus prevents information loss. Figure 3 As shown in Figure 2, the SE attention module includes a compression module and an excitation module. In the structure of the compression module, the feature map (f i ) is compressed into a 1×1×C feature vector through global pooling; in the bottleneck structure of the excitation module, global learning and capture of the correlations between various feature channels occur, and then the learned attention weights are assigned to the corresponding feature channels to obtain an H×W×C output feature map with attention annotations (f i '):

[0082]

[0083] Excition: F ex (z,w)=σ(W2σ(W1z)) (4)

[0084] Scale: F scale (u c ,s c )=s c ·u c (5);

[0085] Where H represents the height of the input feature map;

[0086] W represents the width of the input feature map;

[0087] u c (i, j) represents the pixel value at position (i, j) on channel c;

[0088] F sq (u c ) The compressed result (1×1×C feature vector);

[0089] W1 and W2 represent the weight matrices of the fully connected layer, which are used to reduce and increase the channel dimension respectively;

[0090] z represents the feature vector after global pooling;

[0091] w represents the final channel attention weight vector obtained in the Excitation operation;

[0092] σ represents the activation function, where σ(W1z) is the ReLU activation and σ(W2σ(W1z)) is the Sigmoid activation, which is used to generate channel attention weights;

[0093] F ex (z,w) represents the output weight vector of the Excitation operation;

[0094] s c Represents the channel weight generated by the Excitation operation;

[0095] Fscale(u c ,s c ) represents the weighted output features.

[0096] The feature pyramid (FPN) module builds multi-scale feature maps to enhance the model's ability to detect objects of different scales. It uses top-down paths and lateral connections to fuse high-level semantic information and low-level detail information, thereby improving the ability to detect small PTB lesions. Figure 4 shown.

[0097] The implementation method of the feature pyramid module is as follows: first, the feature extraction network module extracts feature maps of different levels from CT slices (512×512×3); then, starting from the highest-level feature map (16×16×256), the semantic information of the high-level feature map is gradually transferred to the low-level feature map through upsampling. After each upsampling, the feature map is horizontally connected with the corresponding underlying feature map (bottom-up), either output or continued to be upsampled; finally, multi-scale feature maps P2, P3, P4, P5 and P6 are generated.

[0098] Furthermore, LabelImg is used to label the lesion information of the above-mentioned tuberculosis CT chest image feature data.

[0099] Furthermore, in Faster R-CNN, L(p,u,t u ,v) is the total loss function of the model, L cla (p,u) is the classification loss function, L reg (t u ,v) is the regression loss function:

[0100] L(p,u,t u ,v)=L cla (p,u)+λL reg (t u ,v) (6)

[0101] L cla (p,u)=-log p u (7)

[0102]

[0103] Where P is the probability distribution predicted by the classifier using softmax; u is the actual category label of the target; t u is the regression parameter of the bounding box regressor predicting the corresponding category u v is the bounding box regression parameter corresponding to the true target β is the threshold variable; s is the difference in regression error, which represents the error between the model prediction and the actual bounding box.

[0104]

[0105] In the formula, IoU is the intersection over union ratio, which quantifies the degree of intersection between the bounding box predicted by the model and the true target bounding box, which helps to select positive and negative samples. A represents the bounding box predicted by the model, and B represents the bounding box of the true annotation. A∩B represents the intersection of bounding boxes A and B, that is, the area of ​​the overlap between the predicted box and the true box, and A∪B represents the union of bounding boxes A and B, that is, the intersection part and the non-overlapping part between the two.

[0106] In order to illustrate the effect of the present invention, the following simulation verification is performed:

[0107] Dataset selection,

[0108] The present invention uses 1000 cases of PTB CT (3D) data provided by the Guiyang Public Health Treatment Center in Guizhou Province, with 60 slices (2D) per case, a slice spacing of 5 mm, a window width of 1400, and a window level of -500. According to the expert consensus on tuberculosis imaging and graded diagnosis formulated by the Infectious Disease Radiology Committee of the Chinese Medical Association Radiology Society, PTB is defined as five types: infiltrative, caseous, miliary, tuberculous, and cavitary.

[0109] The present invention annotates information on the slices, and selects 4 to 6 typical slices of multiple types, multiple forms and multiple lesions from each data set to meet the complexity, diversity and heterogeneity of PTB lesions, with a total of 5,000 slices; and divides them into training set and validation set in a ratio of 8:2.

[0110] Among them, data set annotation: The size and shape of PTB lesions vary greatly, and the boundaries are relatively fuzzy. Lesions in different locations need to be accurately annotated so that the algorithm can correctly identify and locate the lesions. In lung images, PTB lesions are also obstructed or interfered with by other tissue structures, such as blood vessels, bronchi, ribs, etc. This interference will increase the difficulty of annotation, and the annotator needs to have rich imaging knowledge and experience to distinguish lesions from other tissue structures. In addition, due to the particularity and complexity of PTB lesions, different annotators may have different annotation results for the same image. Therefore, it is necessary to ensure the consistency of annotations between annotators and perform quality control to ensure the accuracy and reliability of annotations.

[0111] To achieve this goal, the present invention uses LabelImg to annotate the lesion information. These annotations are completed by experienced radiologists from the Guiyang Public Health Treatment Center. Specifically, each PTB slice in the data set is confirmed by a doctor and annotated with the lesion type. Subsequently, another doctor will check it. If the lesion type and the box selection are correct, the annotation is successful; otherwise, if the lesion type does not match or the box selection is wrong, adjustments will be made. Finally, the data information labels required for the model are constructed.

[0112] Model performance evaluation uses precision, recall, F1_score and area under the curve (AUC) as evaluation indicators of the model.

[0113]

[0114] In the confusion matrix, "Positive" indicates a lesion, and "Negative" indicates a non-lesion. A true positive (TP) indicates that the actual situation is a lesion, and the prediction is also a lesion. A false positive (FP) indicates that the actual situation is not a lesion, but the prediction is a lesion. A true negative (TN) indicates that the actual situation is not a lesion, and the prediction is not a lesion. A false negative (FN) indicates that the actual situation is a lesion, but the prediction is not a lesion.

[0115] Experiment and result analysis: The present invention adopts transfer learning to accelerate model convergence and obtains better training effect with fewer iterations.

[0116] In order to verify the detection effect of the improved model, two experimental comparisons were set up to analyze the performance indicators of improved Faster RCNN, Faster RCNN, YOLOv5 and SSD in the preprocessed data set and the original data set, including precision, recall, F1 score and AUC. The results are shown in Table 1, and the corresponding iteration curves are shown in Table 2. Figure 5 As shown (bold: indicates the current value is the highest).

[0117] Table 1 Performance index parameters

[0118]

[0119] Comparative experiment 1: On the preprocessed data set,

[0120] The precision of the improved Faster RCNN is 0.04, 0.13, and 0.14 higher than that of Faster RCNN, YOLOv5, and SSD, respectively, the recall is 0.05, 0.19, and 0.21 higher, the F1 Score is 0.04, 0.16, and 0.18 higher, and the AUC is 0.03, 0.11, and 0.12 higher. Comparative Experiment 2: On the original dataset, the precision of the improved Faster RCNN is 0.04, 0.12, and 0.14 higher than that of Faster RCNN, YOLOv5, and SSD, respectively, the recall is 0.03, 0.19, and 0.20 higher, the F1 Score is 0.04, 0.17, and 0.18 higher, and the AUC is 0.03, 0.10, and 0.11 higher.

[0121] The performance of the improved Faster RCNN is significantly better than that of Faster RCNN, YOLOv5 and SSD, with an accuracy of 0.95, a recall of 0.86, an F1 Score of 0.90 and an AUC of 0.97. It has achieved good detection results and can be used as an auxiliary tool for clinical diagnosis by radiologists.

[0122] The improved Faster RCNN proposed in the present invention has achieved the best performance indicators in both the preprocessed data set and the original data set. From the results, the data preprocessing method can effectively reduce the interference of noise and reduce the occurrence of false positives. At the same time, the improved FasterRCNN based on the SE attention module and the feature pyramid structure (FPN) has a higher detection accuracy (precision), indicating that there are fewer false detections in the detected PTB lesions; but the recall rate (recall) is relatively general, indicating that there is still a possibility of missed detection.

[0123] Improve Faster RCNN's detection of external detection effects such as Figure 6 As shown in the figure (ae are infiltrative, miliary, caseous, tuberculous and cavitary lesions, respectively). It can be seen from the prediction graph that PTB lesions can be accurately identified and selected by the model, proving the feasibility of the improved model.

[0124] The present invention has fully studied and explored the medical knowledge related to PTB. PTB lesions are characterized by diversity, complexity and heterogeneity, and are usually manifested as abnormal lung density images, showing different forms such as spots, patches, nodules or consolidation, and with unclear boundaries. Some lesions may also be blocked or overlapped by structures such as lung tissue, blood vessels, and trachea, increasing the difficulty of detection.

[0125] Therefore, the present invention segments the lung parenchyma based on data preprocessing, effectively reducing the false detection rate. At the same time, the SE attention module is added to the two-stage detection network Faster RCNN to enhance the model's detection accuracy for small PTB lesions, and then a multi-scale feature map is generated through the feature pyramid (FPN) structure to better capture the different details and positions of the target, making the positioning of target detection more accurate. The results show that the detection effect of the improved model can meet the expected requirements, and can be used as an auxiliary tool for doctors to provide a second opinion and help doctors make a more accurate diagnosis.

Claims

1. A tuberculosis detection system based on attention and feature pyramid, characterized by: include: An image feature acquisition module is used to acquire features of pulmonary tuberculosis CT chest images; A data preprocessing module, used for preprocessing the acquired features of the pulmonary tuberculosis CT chest image; The detection model module is used to detect and classify the features of the preprocessed pulmonary tuberculosis CT chest image; the detection model module uses the Faster RCNN model for detection and classification. The RCNN model includes a feature extraction network module, an RPN module, a RoIPooling module, and a classification and regression module. The feature extraction network module uses the ResNet50 network. The residual structure of the ResNet50 network is embedded in the SE attention module, and a multi-scale feature map is constructed through the feature pyramid module.

2. A tuberculosis detection system based on attention and feature pyramid according to claim 1, characterized in that: The data preprocessing module uses the K-Means clustering segmentation algorithm to preprocess the extracted image features. The specific steps are as follows: 1) Normalize the slice pixel value from [0, 255] to [0, 1]; Where, X i The feature vector representing the i-th CT slice, usually, these feature vectors represent the pixel values ​​of the lung parenchyma area in the CT image; u k represents the centroid of the kth cluster, u k The center of a specific cluster representing lung tissue, which is the average feature of different lung regions that the clustering algorithm tries to find; n k It indicates the number of data points contained in the kth cluster, that is, the number of lung parenchyma pixels belonging to this cluster; ||X i -u k || 2 Represents a pixel X i To cluster center u k The square of the Euclidean distance; J represents the objective function, which optimizes the clustering process by minimizing the distance from the pixel to its centroid so that similar pixels are clustered into the same class; 2) Separate the foreground and background through K-Means clustering segmentation algorithm and binary threshold method; 3) Perform morphological operations on the separated images; 4) Use different colors to mark the separated lung areas; 5) Create a mask based on the color marker and expand the mask area again; 6) Perform convex hull operation on the filled area to retain more complete lung area information; 7) Segment the ROI by multiplying the image in step 1) with the mask in step 6).

3. The tuberculosis detection system based on attention and feature pyramid according to claim 2, characterized in that: The SE attention module includes a compression module and an excitation module. In the structure of the compression module, the H×W×C-dimensional feature map (f i ) is compressed into a 1×1×C feature vector through global pooling; in the bottleneck structure of the excitation module, global learning and capture of the correlations between various feature channels occur, and then the learned attention weights are assigned to the corresponding feature channels to obtain an H×W×C output feature map with attention annotations (f i '): Excitation: F ex (z,w)=σ(W2σ(W1z)) (4) Scale:F scale (u c ,s c )=s c ·u c (5) Where H represents the height of the input feature map; W represents the width of the input feature map; u c (i, j) represents the pixel value at position (i, j) on channel c; F sq (u c ) The compressed result (1×1×C feature vector); W1 and W2 represent the weight matrices of the fully connected layer, which are used to reduce and increase the channel dimension respectively; z represents the feature vector after global pooling; w represents the final channel attention weight vector obtained in the Excitation operation; σ represents the activation function, where σ(W1z) is the ReLU activation and σ(W2σ(W1z)) is the Sigmoid activation, which is used to generate channel attention weights; F ex (z,w) represents the output weight vector of the Excitation operation; s c Represents the channel weights generated by the Excitation operation; Fscale(u c ,s c ) represents the weighted output features.

4. The tuberculosis detection system based on attention and feature pyramid according to claim 2, characterized in that: The implementation method of the feature pyramid module is as follows: first, extract feature maps of different levels from CT slices (512×512×3); then, starting from the highest-level feature map (16×16×256), gradually transfer the semantic information of the high-level feature map to the low-level feature map through upsampling. After each upsampling, the feature map is horizontally connected with the corresponding underlying feature map, either output or continue to be upsampled; finally, multi-scale feature maps P2, P3, P4, P5 and P6 are generated.

5. The tuberculosis detection system based on attention and feature pyramid according to claim 3, characterized in that: The data of tuberculosis CT chest image features uses LabelImg to annotate the lesion information.

6. The tuberculosis detection system based on attention and feature pyramid according to claim 3, characterized in that: In Faster R-CNN, L(p,u,t u ,v) is the total loss function of the model, L cla (p,u) is the classification loss function, L reg (t u ,v) is the regression loss function: L(p,u,t u ,v)=L cla (p,u)+λL reg (t u ,v) (6) L cla (p,u)=-logp u (7) Where P is the probability distribution predicted by the classifier using softmax; u is the actual category label of the target; t u is the regression parameter of the bounding box regressor predicting the corresponding category u v is the bounding box regression parameter corresponding to the true target β is the threshold variable; s is the difference in regression error, which represents the error between the model prediction and the actual bounding box; Where IoU is the intersection over union ratio, which quantifies the degree of intersection between the bounding box predicted by the model and the true target bounding box. A represents the bounding box predicted by the model, and B is the bounding box of the real annotation. A∩B represents the intersection of bounding boxes A and B, that is, the area of ​​the overlapping area between the predicted box and the true box. A∪B represents the union of bounding boxes A and B, that is, the intersection part and the non-overlapping part between the two.

Images