CNN-ViT-Meta fusion model-based pulmonary tuberculosis intelligent identification method

Abstract

The invention discloses an intelligent pulmonary tuberculosis identification method based on a CNN-ViT-Meta fusion model. The method comprises the following steps: carrying out data preprocessing on a disclosed pulmonary tuberculosis chest X-ray CT image data set; constructing a CNN-based metadata inversion model, performing metadata inversion and supplementation on the chest X-ray CT image, deducing the age and gender of the patient, and generating a metadata label; constructing a CNN-ViT-Meta fusion model, extracting low-level local features of the lung image by using CNN, capturing global feature information by ViT, and generating an image feature sequence; and meanwhile, inputting the metadata into a full connection layer to generate a metadata feature sequence, integrating the image feature sequence output by CNN-ViT and the metadata feature sequence through a feature fusion module, and outputting a pulmonary tuberculosis recognition result. Performance indexes such as accuracy are evaluated through the prediction result of the test set and the real label, and the recognition performance and generalization ability of the model are verified.

Description

Technical Field

[0001] This invention relates to the fields of medical image analysis and artificial intelligence technology, and more specifically to a method for intelligent identification of pulmonary tuberculosis based on a CNN-ViT-Meta fusion model. Background Technology

[0002] With the rapid development of deep learning technology, the performance of image classification tasks has made significant progress in many fields, especially in medical image analysis, where models such as Convolutional Neural Networks (CNNs) and Transformers have shown excellent performance in the automatic lesion detection of X-ray and CT images. However, in the cross-domain scenario of tuberculosis diagnosis, there are significant distributional differences between the source domain (such as image data from different regions and populations) and the target domain (such as image data from primary healthcare institutions). This distributional shift poses a significant challenge to the model's generalization ability. In addition, the scarcity of labeled data also limits the model's performance.

[0003] The CNN-ViT-Meta fusion model, as the core architecture of this study, aims to improve the performance of the model in the pulmonary tuberculosis (TB) identification task by combining the advantages of Convolutional Neural Networks (CNNs) and Visual Transformers (ViTs). CNNs are powerful in extracting local features, while ViTs can capture global features and long-range dependencies. Meta-learning is used to optimize the model's generalization ability, enabling it to better adapt to different data distributions. This model significantly improves the accuracy of TB lesion identification and enhances its generalization ability under different data distributions by introducing multi-level feature fusion and meta-learning strategies. Specifically, the CNN-ViT-Meta fusion model achieves its advantages in the following ways: 1) Utilizing the local feature extraction capability of CNNs to accurately capture the local details of TB lesions; 2) Capturing global features through ViTs to enhance the recognition of the overall structure of the lesions; 3) The meta-learning strategy enables the model to quickly adapt to new data distributions, improving cross-domain generalization ability. Compared with existing methods, this method exhibits higher accuracy and stronger generalization performance in the TB identification task. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide an intelligent identification method for pulmonary tuberculosis based on a CNN-ViT-Meta fusion model. By integrating the local feature extraction capability of CNN, the global context modeling advantage of ViT, and the multimodal information of clinical metadata, the method can achieve accurate identification of pulmonary tuberculosis lesions and improve the generalization ability of the model under different populations and data distributions.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows.

[0006] A method for intelligent identification of pulmonary tuberculosis based on a CNN-ViT-Meta fusion model includes the following steps: S100. Perform data preprocessing on the publicly available chest X-ray CT image dataset of pulmonary tuberculosis, including accurate extraction of lung lobe regions, image size standardization, and data augmentation; S200. Construct a CNN-based metadata inversion model to perform metadata inversion and supplementation on chest X-ray CT images, infer the patient's age and gender, and generate metadata labels; S300. Construct a CNN-ViT-Meta fusion model. CNN is used to extract low-level local features of lung images, and ViT captures global feature information to generate image feature sequences. At the same time, metadata is input into the fully connected layer to generate metadata feature sequences. The image feature sequences and metadata feature sequences output by CNN-ViT are integrated through the feature fusion module to output the tuberculosis identification results.

[0007] To further optimize the technical solution, the data preprocessing method in step S100 includes the following steps: S110. Image transformation methods such as random rotation, cropping, and brightness adjustment are used to enhance CT images and corresponding lung lobe mask images. Brightness adjustment utilizes gamma variation, and the formula is as follows:

[0008] in, Input values ​​for the grayscale image. This is the grayscale scaling factor, usually set to 1. It is the gamma factor; S120. Apply bilinear interpolation to the enhanced CT images to achieve a uniform size of 1024×1024. The bilinear interpolation formula is as follows:

[0009] in, Represents the target image (uniform size). The coordinates of the pixels on the CT image after interpolation, i.e., the coordinates of the target point to be interpolated. This represents the pixel value of the four neighboring pixels in the original image that are related to the target point (x, y); It is used to traverse the original image to the target point The indices of the four neighboring pixels, i and j, are both 0 or 1; S130. Train the U-Net precise segmentation model using the processed dataset images; the U-Net model is used for lung lobe region segmentation, employing the Dice loss function:

[0010] in, and Let i represent the label value and the predicted value of pixel i, respectively, and n be the total number of pixels. S140. Use the trained U-Net to obtain preliminary masking data for the remaining two datasets; S150. Scale the mask data back to its original size and intersect it with the CT image to obtain mask data that preserves the original lesions. The final lung region special area can be represented as:

[0011] in, It is the original image. It is the output of U-Net, and It is the binarization threshold, which is usually set to 0.5.

[0012] To further optimize the technical solution, step S200 includes the following steps: S210. Train a CNN-based metadata inversion model using chest X-rays of pulmonary tuberculosis and the Montgomery pulmonary tuberculosis dataset; S220. Apply the trained CNN inversion model to the Qatar dataset to perform metadata inversion and supplementation; S230. Infer the patient's gender and age, and generate metadata tags.

[0013] To further optimize the technical solution, in step S210, the method for establishing the inversion model is as follows: First, the CT samples in the integrated dataset are classified and encoded, and corresponding age and gender classifications are designed; then, the original CT images are scaled and filled proportionally, and the input images are unified to 1024×1024 while ensuring that the proportion of lung lobes remains unchanged; then, the integrated dataset is shuffled and divided into training and test sets in an 8:2 ratio; for the data in the training set, it is enhanced by the data augmentation method in S100, specifically including random rotation, random cropping and rescaling, and brightness adjustment; for each original sample, an enhanced version is generated, thereby doubling the size of the training set; the inversion model consists of a feature extraction network and a dual-task classification head.

[0014] To further optimize the technical solution, the feature extraction network adopts a 5-level convolutional module, each level including a convolutional layer, batch normalization, ReLU activation and max pooling; through these 5-level convolutional modules, the input image is downsampled 5 times to obtain a 16×16×256 feature map.

[0015] To further optimize the technical solution, the dual-task classification head consists of a fully connected layer for each classification head, one for gender classification and the other for age classification.

[0016] To further optimize the technical solution, step S300 specifically includes the following steps: S310. Construct a CNN module to extract local feature information from CT images through four convolutional layers; S320. Divide the feature map output by the CNN module into image blocks and use the ViT module to achieve global context modeling; S330. Construct a Meta module to integrate clinical metadata. Metadata such as gender and age are encoded using one-hot encoding and then input into a three-layer fully connected network for processing. The calculation formula is as follows:

[0017] in, d is the input vector after encoding the original metadata, and d is the feature dimension of the ViT output. S340. By fusing the features of the ViT and Meta modules through layer normalization, Input layer normalization (LN) after addition with m:

[0018] in, Indicates passage The sequence generated by the module; The sequence generated for the metadata module; S350. Outputs binary classification results through a fully connected layer and a softmax function, and then merges the features. Classify the input fully connected layers:

[0019] in, These are the weight parameters of the fully connected layer. For bias terms; S360. Focal Loss is used to mitigate the class imbalance problem. The formula is as follows:

[0020] in, To predict probabilities, As a balance factor, For focusing parameters; Furthermore, it integrates classification loss, feature fusion loss, and metadata inference loss to construct a joint optimization objective; S370. Set the model training parameters, including the number of iterations, early stopping strategy, batch size, and learning rate; S380. After the model training is completed, evaluate the model classification performance through the test set. Apply the trained model to the test set data and calculate indicators such as precision, recall, and F1 score to verify the model's recognition accuracy and generalization ability. S390. The effectiveness of each module was verified through ablation experiments. The Meta module (w / o Meta), CNN module (w / o CNN), and ViT module (w / o ViT) were removed respectively. The model was also retrained after the number of parameters was halved (params / 2). The accuracy and loss value of the validation set under different configurations were compared.

[0021] To further optimize the technical solution, in step S310, the convolutional block consists of a convolutional layer, a batch normalization layer, a ReLU activation function, and a max pooling layer. The calculation process of layers 1 to 3 is as follows:

[0022] in, For convolution kernel parameters, Representing the l Input feature maps of convolutional blocks; The max pooling operation is omitted in the fourth layer, and the output feature map size is gradually reduced to 128×128.

[0023] To further optimize the technical solution, in step S320, the method of modeling using a CNN module is as follows: The feature map output by the CNN module is divided into... Image patches of a certain size, each image patch is used to generate a D-dimensional embedding vector through linear projection:

[0024] in, For learnable embedding matrices, It is a 1-D absolute position encoding; The embedded vectors are input into an L-layer Transformer encoder, each layer containing a multi-head self-attention mechanism (MSA) and a multilayer perceptron (MLP), with the following calculation formulas:

[0025]

[0026] in, Representing the Attention output calculated from each "head"; Learnable parameter matrix used for linear transformation of the spliced ​​multi-head attention results; It is the core input for multi-head self-attention computation; For MLP input Perform a linear transformation. It is a learnable weight matrix; It is the bias term of the linear transformation, a learnable parameter; It is a learnable weight matrix that performs a linear transformation on the result after GeLU activation; The final output category label is used as a global feature. :

[0027] Where Z represents the feature representation of each layer of the Transformer encoder; This refers to the number of layers in the Transformer encoder; 0 indicates the position index in the feature sequence. That is, after The features at index 0 in the sequence after processing by the Transformer encoder.

[0028] Due to the adoption of the above technical solutions, the technical progress achieved by this invention is as follows.

[0029] This invention provides an intelligent tuberculosis identification method based on a CNN-ViT-Meta fusion model. It integrates classification loss, feature fusion loss, and metadata inference loss to construct a joint optimization objective, enabling end-to-end model training. After training, the model's recognition performance and generalization ability are evaluated by comparing prediction results on the test set with the ground truth labels, using metrics such as accuracy. Finally, to demonstrate the effectiveness of each module, the Meta module, CNN module, and ViT module were removed, and the number of parameters was halved for ablation studies. Attached Figure Description

[0030] Figure 1 This is a flowchart of the present invention; Figure 2 This is a flowchart of step S100 in the present invention; Figure 3 This is a flowchart of step S200 in the present invention; Figure 4 This is a flowchart of step S300 in the present invention; Figure 5 A GradCAM heatmap was generated for three tuberculosis-positive samples in this invention. Detailed Implementation

[0031] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0032] A method for intelligent identification of pulmonary tuberculosis based on a CNN-ViT-Meta fusion model, combining Figures 1 to 4 As shown, it includes.

[0033] S100. Perform data preprocessing on the publicly available chest X-ray CT image dataset of pulmonary tuberculosis, including accurate extraction of lung lobe regions, image size standardization, and data augmentation.

[0034] S110. Image transformation methods such as random rotation, cropping, and brightness adjustment are used to enhance CT images and corresponding lung lobe mask images. Brightness adjustment utilizes gamma variation, and the formula is as follows:

[0035] in, Input values ​​for the grayscale image. This is the grayscale scaling factor, usually set to 1. It is the gamma factor; S120. Apply bilinear interpolation to the enhanced CT images to achieve a uniform size of 1024×1024. The bilinear interpolation formula is as follows:

[0036] in, Represents the target image (uniform size). The coordinates of the pixels on the CT image after interpolation, i.e., the coordinates of the target point to be interpolated. This represents the pixel value of the four neighboring pixels in the original image that are related to the target point (x, y); It is used to traverse the original image to the target point The indices of the four neighboring pixels, i and j, are both 0 or 1.

[0037] S130. Train the U-Net precise segmentation model using the processed dataset images; the U-Net model is used for lung lobe region segmentation, employing the Dice loss function:

[0038] in, and Let i represent the label value and the predicted value of pixel i, respectively, and n be the total number of pixels. S140. Use the trained U-Net to obtain preliminary masking data for the remaining two datasets; S150. Scale the mask data back to its original size and intersect it with the CT image to obtain mask data that preserves the original lesions. The final lung region special area can be represented as:

[0039] in, It is the original image. It is the output of U-Net, and It is the binarization threshold, which is usually set to 0.5.

[0040] S200. Construct a CNN-based metadata inversion model to perform metadata inversion and supplementation on chest X-ray CT images, infer the patient's age and gender, and generate metadata labels.

[0041] S210. Train a CNN-based metadata inversion model using pulmonary tuberculosis chest X-ray and Montgomery pulmonary tuberculosis datasets.

[0042] The method for establishing the inversion model is as follows: First, the CT samples in the integrated dataset are classified and coded, and corresponding age and gender classifications are designed. Next, the original CT images are scaled and filled proportionally, and the input images are unified to 1024×1024 while keeping the lung lobe proportion unchanged. Then, the integrated dataset is shuffled and divided into training and test sets in an 8:2 ratio. For the training set data, data augmentation methods in S100 are used to enhance it, specifically including random rotation, random cropping and rescaling, and brightness adjustment. For each original sample, an enhanced version is generated, thereby doubling the size of the training set. The inversion model consists of a feature extraction network and a dual-task classification head.

[0043] The feature extraction network employs a 5-level convolutional module, with each level containing a convolutional layer (extracting spatial features), batch normalization (accelerating training and improving stability), ReLU activation (capturing non-linear features), and max pooling (reducing spatial dimensionality and extracting salient features). Through these 5-level convolutional modules, the input image (1024×1024) is downsampled 5 times to obtain a 16×16×256 feature map.

[0044] The dual-task classification head consists of a fully connected layer for each classification head, one for gender classification (256-dimensional hidden layer outputting 2 classes) and the other for age classification (256-dimensional hidden layer outputting 3 classes).

[0045] S220. Apply the trained CNN inversion model to the Qatar dataset to perform metadata inversion and supplementation.

[0046] S230. Infer the patient's gender and age, and generate metadata tags. S300. Construct a CNN-ViT-Meta fusion model. CNN is used to extract low-level local features of lung images, and ViT captures global feature information to generate image feature sequences. At the same time, metadata is input into the fully connected layer to generate metadata feature sequences. The image feature sequences and metadata feature sequences output by CNN-ViT are integrated through the feature fusion module to output the tuberculosis identification results.

[0047] S310. Construct a CNN module to extract local feature information from CT images through four convolutional blocks.

[0048] A convolutional block consists of a convolutional layer, a batch normalization layer, a ReLU activation function, and a max pooling layer. The computation process of layers 1 to 3 is as follows:

[0049] in, For convolution kernel parameters, Representing the l Input feature map of a convolutional block.

[0050] The max pooling operation is omitted in the fourth layer, and the output feature map size is gradually reduced to 128×128.

[0051] S320. Divide the feature map output by the CNN module into image blocks and use the ViT module to achieve global context modeling.

[0052] The method for modeling using CNN modules is as follows: The feature map output by the CNN module is divided into... Image patches of a certain size, each image patch is used to generate a D-dimensional embedding vector through linear projection:

[0053] in, For learnable embedding matrices, It is a 1-D absolute position encoding; The embedded vectors are input into an L-layer Transformer encoder, each layer containing a multi-head self-attention mechanism (MSA) and a multilayer perceptron (MLP), with the following calculation formulas:

[0054]

[0055] in, Representing the Attention output calculated from each "head"; Learnable parameter matrix used for linear transformation of the spliced ​​multi-head attention results; It is the core input for multi-head self-attention computation; For MLP input Perform a linear transformation. It is a learnable weight matrix; It is the bias term of the linear transformation, a learnable parameter; It is a learnable weight matrix that performs a linear transformation on the result after GeLU activation.

[0056] The final output category label is used as a global feature. :

[0057] Where Z represents the feature representation of each layer of the Transformer encoder; This refers to the number of layers in the Transformer encoder; 0 indicates the position index in the feature sequence. That is, after The features at index 0 in the sequence after processing by the Transformer encoder.

[0058] S330. Construct a Meta module to integrate clinical metadata. Metadata such as gender and age are encoded using one-hot encoding and then input into a three-layer fully connected network for processing. The calculation formula is as follows:

[0059] in, d is the input vector after encoding the original metadata, and d is the feature dimension of the ViT output.

[0060] S340. By fusing the features of the ViT and Meta modules through layer normalization, Input layer normalization (LN) after addition with m:

[0061] in, Indicates passage The sequence generated by the module; A sequence generated for the metadata module.

[0062] S350. Outputs binary classification results through a fully connected layer and a softmax function, and then merges the features. Classify the input fully connected layers:

[0063] in, These are the weight parameters of the fully connected layer. This is a bias term.

[0064] S360. Focal Loss is used to mitigate the class imbalance problem. The formula is as follows:

[0065] in, To predict probabilities, As a balance factor, For focusing parameters; Furthermore, classification loss, feature fusion loss, and metadata inference loss are integrated to construct a joint optimization objective.

[0066] S370. Set the model training parameters, including the number of iterations, early stopping strategy, batch size, and learning rate.

[0067] Specifically, the training epochs are set to 40, employing an early stopping strategy with a patience value of 10 (stopping if the validation set loss does not decrease for 10 consecutive epochs). The batch size is set to 4, and the learning rate is set to... All fully connected layers use dropout (random dropout probability of 0.1) to prevent overfitting.

[0068] S380. After the model training is completed, evaluate the model classification performance through the test set. Apply the trained model to the test set data and calculate indicators such as precision, recall, and F1 score to verify the model's recognition accuracy and generalization ability.

[0069] Throughout the training process, the model achieved its best accuracy in the 36th training round, at which point the validation set loss reached 0.0931 and the accuracy reached 92.20%. This result demonstrates that the model has high accuracy in the tuberculosis identification task.

[0070] Six CT images were selected from the validation set, and the results are shown in Table 1: Table 1

[0071] The results in the table above show that the predicted probability for negative samples is generally below 10%, while the predicted probability for positive samples is approximately above 80%. This indicates that the model performs very well in predicting the samples.

[0072] Throughout the training process, the model achieved its best accuracy in the 36th training round, at which point the validation set loss reached 0.0931 and the accuracy reached 92.20%. This result demonstrates that the model has high accuracy in the tuberculosis identification task.

[0073] S390. The effectiveness of each module was verified through ablation experiments. The Meta module (w / o Meta), CNN module (w / o CNN), and ViT module (w / o ViT) were removed respectively. The model was also retrained after the number of parameters was halved (params / 2). The accuracy and loss value of the validation set under different configurations were compared.

[0074] The results are summarized in Table 2: Table 2

[0075] Ablation experiments show that each module contributes significantly to model performance: removing the Meta module leads to a significant performance decrease, confirming the diagnostic value of patient metadata; removing the CNN module weakens the ability to capture local details, reducing accuracy; removing the ViT module results in the loss of global feature understanding, affecting the identification of complex cases. The parameter halving experiment shows that the model parameter selection is efficient and avoids over-parameterization; the significant performance drop without training on the lung lobe segmentation dataset demonstrates the crucial role of this preprocessing step.

[0076] Three tuberculosis-positive samples were selected, and GradCAM heatmaps were generated for each model. The heatmaps are shown below. Figure 5 As shown, the heatmap results reveal that CViTM-Net can balance lung attention and accurately highlight lesions; without the ViT module, only local features are captured, and excessive gradients affect lesion identification; removing the CNN module results in attention being scattered across the contours, missing internal details; the lack of a Meta module leads to oversaturation of the heatmap, reducing detection accuracy; the semi-parametric model exhibits blurred hotspots, and the reduced spatial resolution due to fewer channels. These visualization results confirm that CViTM-Net improves lesion localization capabilities by integrating local and global features, providing a more reliable basis for clinical diagnosis.

Claims

1. A lung tuberculosis intelligent identification method based on a CNN-ViT-Meta fusion model, characterized in that: Includes the following steps: S100. Perform data preprocessing on the publicly available chest X-ray CT image dataset of pulmonary tuberculosis, including accurate extraction of lung lobe regions, image size standardization, and data augmentation; S200. Construct a CNN-based metadata inversion model to perform metadata inversion and supplementation on chest X-ray CT images, infer the patient's age and gender, and generate metadata labels; S300. Construct a CNN-ViT-Meta fusion model. CNN is used to extract low-level local features of lung images, and ViT captures global feature information to generate image feature sequences. At the same time, metadata is input into the fully connected layer to generate metadata feature sequences. The image feature sequences and metadata feature sequences output by CNN-ViT are integrated through the feature fusion module to output the tuberculosis identification results.

2. The tuberculosis intelligent recognition method based on the CNN-ViT-Meta fusion model according to claim 1, characterized in that: In step S100, the data preprocessing method includes the following steps: S110. Image transformation methods such as random rotation, cropping, and brightness adjustment are used to enhance CT images and corresponding lung lobe mask images. Brightness adjustment utilizes gamma variation, and the formula is as follows: wherein, is the input value for a grayscale image, is a grayscale scaling factor, typically 1, is a gamma factor; S120. Apply bilinear interpolation to the enhanced CT images to achieve a uniform size of 1024×1024. The bilinear interpolation formula is as follows: wherein, represents the coordinates of a pixel point on the target image (uniform size CT image after interpolation), i.e. the target point coordinates to be calculated; represents the pixel values of four neighboring pixel points in the original image related to the target point (x, y) ; is an index for traversing the four neighboring pixel points in the original image related to the target point (x, y), i and j are both 0 or 1. S130. Train the U-Net precise segmentation model using the processed dataset images; the U-Net model is used for lung lobe region segmentation, employing the Dice loss function: in, and Let i represent the label value and the predicted value of pixel i, respectively, and n be the total number of pixels. S140. Use the trained U-Net to obtain preliminary masking data for the remaining two datasets; S150. Scale the mask data back to its original size and intersect it with the CT image to obtain mask data that preserves the original lesions. The final lung region special area can be represented as: in, It is the original image. It is the output of U-Net, and It is the binarization threshold, which is usually set to 0.

5.

3. The intelligent identification method for pulmonary tuberculosis based on the CNN-ViT-Meta fusion model according to claim 1, characterized in that: Step S200 includes the following steps: S210. Train a CNN-based metadata inversion model using chest X-rays of pulmonary tuberculosis and the Montgomery pulmonary tuberculosis dataset; S220. Apply the trained CNN inversion model to the Qatar dataset to perform metadata inversion and supplementation; S230. Infer the patient's gender and age, and generate metadata tags.

4. The intelligent identification method for pulmonary tuberculosis based on the CNN-ViT-Meta fusion model according to claim 3, characterized in that: In step S210, the method for establishing the inversion model is as follows: First, the CT samples in the integrated dataset are classified and encoded, and corresponding age and gender classifications are designed; then, the original CT images are scaled and filled proportionally, and the input images are unified to 1024×1024 while ensuring that the proportion of lung lobes remains unchanged; then, the integrated dataset is shuffled and divided into training and test sets in an 8:2 ratio; the data in the training set is enhanced using the data augmentation method in S100, specifically including random rotation, random cropping and rescaling, and brightness adjustment; for each original sample, an enhanced version is generated, thereby doubling the size of the training set; the inversion model consists of a feature extraction network and a dual-task classification head.

5. The intelligent identification method for pulmonary tuberculosis based on the CNN-ViT-Meta fusion model according to claim 4, characterized in that: The feature extraction network employs a 5-level convolutional module, with each level containing a convolutional layer, batch normalization, ReLU activation, and max pooling. Through these 5-level convolutional modules, the input image is downsampled 5 times to obtain a 16×16×256 feature map.

6. The intelligent identification method for pulmonary tuberculosis based on the CNN-ViT-Meta fusion model according to claim 4, characterized in that: The dual-task classification head consists of a fully connected layer for each classification head, one for gender classification and the other for age classification.

7. The intelligent identification method for pulmonary tuberculosis based on the CNN-ViT-Meta fusion model according to claim 1, characterized in that: Step S300 specifically includes the following steps: S310. Construct a CNN module to extract local feature information from CT images through four convolutional layers; S320. Divide the feature map output by the CNN module into image blocks and use the ViT module to achieve global context modeling; S330. Construct a Meta module to integrate clinical metadata. Metadata such as gender and age are encoded using one-hot encoding and then input into a three-layer fully connected network for processing. The calculation formula is as follows: in, d is the input vector after encoding the original metadata, and d is the feature dimension of the ViT output. S340. By fusing the features of the ViT and Meta modules through layer normalization, Input layer normalization (LN) after addition with m: in, Indicates passage The sequence generated by the module; The sequence generated for the metadata module; S350. Outputs binary classification results through a fully connected layer and a softmax function, and then merges the features. Classify the input fully connected layers: in, These are the weight parameters of the fully connected layer. For bias terms; S360. Focal Loss is used to mitigate the class imbalance problem. The formula is as follows: in, To predict probabilities, As a balance factor, For focusing parameters; Furthermore, it integrates classification loss, feature fusion loss, and metadata inference loss to construct a joint optimization objective; S370. Set the model training parameters, including the number of iterations, early stopping strategy, batch size, and learning rate; S380. After the model training is completed, evaluate the model classification performance through the test set. Apply the trained model to the test set data and calculate indicators such as precision, recall, and F1 score to verify the model's recognition accuracy and generalization ability. S390. The effectiveness of each module was verified through ablation experiments. The Meta module (w / o Meta), CNN module (w / o CNN), and ViT module (w / o ViT) were removed respectively. The model was also retrained after the number of parameters was halved (params / 2). The accuracy and loss value of the validation set under different configurations were compared.

8. The intelligent identification method for pulmonary tuberculosis based on the CNN-ViT-Meta fusion model according to claim 7, characterized in that: In step S310, the convolutional block consists of a convolutional layer, a batch normalization layer, a ReLU activation function, and a max pooling layer. The calculation process for layers 1 to 3 is as follows: in, For convolution kernel parameters, Representing the l Input feature maps of convolutional blocks; The max pooling operation is omitted in the fourth layer, and the output feature map size is gradually reduced to 128×128.

9. The intelligent identification method for pulmonary tuberculosis based on the CNN-ViT-Meta fusion model according to claim 1, characterized in that: In step S320, the method for modeling using a CNN module is as follows: The feature map output by the CNN module is divided into... Image patches of a certain size, each image patch is used to generate a D-dimensional embedding vector through linear projection: in, For learnable embedding matrices, It is a 1-D absolute position encoding; The embedded vectors are input into an L-layer Transformer encoder, each layer containing a multi-head self-attention mechanism (MSA) and a multilayer perceptron (MLP), with the following calculation formulas: in, Representing the Attention output obtained by calculating each "head"; Learnable parameter matrix used for linear transformation of the spliced ​​multi-head attention results; It is the core input for multi-head self-attention computation; For MLP input Perform a linear transformation. It is a learnable weight matrix; It is the bias term of the linear transformation, a learnable parameter; It is a learnable weight matrix that performs a linear transformation on the result after GeLU activation; The final output category label is used as a global feature. : Where Z represents the feature representation of each layer of the Transformer encoder; This refers to the number of layers in the Transformer encoder; 0 indicates the position index in the feature sequence. That is, after The features at index 0 in the sequence after processing by the Transformer encoder.

Images