A precision auxiliary screening method and system for pulmonary tuberculosis

Abstract

This invention discloses a precise auxiliary screening method and system for pulmonary tuberculosis, belonging to the field of smart healthcare technology. The method includes: extracting relevant features of pulmonary tuberculosis from CT images based on a first disease classification network, generating a simulated X-ray chest radiograph using a pre-projection algorithm; selecting a second disease classification network based on the first disease classification network, distilling the relevant features into the second disease classification network, and enhancing the model's feature extraction capability through domain adaptation learning to construct a screening model for screening pulmonary tuberculosis based on real X-ray chest radiographs. This invention teaches the X-ray chest radiograph pulmonary tuberculosis classification network to extract features more sensitive to pulmonary tuberculosis classification through feature distillation of CT images; then, through domain adaptation learning, it enhances the model's robustness to input, thereby achieving the accuracy of CT screening through X-ray screening.

Description

Technical Field

[0001] This invention relates to the field of smart medical technology, and more specifically, to a method and system for precise auxiliary screening of pulmonary tuberculosis. Background Technology

[0002] Tuberculosis, a chronic infectious disease caused by Mycobacterium tuberculosis, is most commonly seen in pulmonary tuberculosis. It primarily affects the lungs, trachea, bronchi, and pleura, including parenchymal tuberculosis of the lungs, tracheobronchial tuberculosis, and tuberculous pleurisy, accounting for the vast majority of all organ tuberculosis cases. Patients with tuberculosis, especially those with detectable levels of Mycobacterium tuberculosis in their sputum, are the main source of infection. The disease spreads through droplets containing the bacteria, transmitted via coughing, talking, or sneezing. Studies have shown that early screening for pulmonary tuberculosis can significantly improve the cure rate.

[0003] Chest X-ray is the routine and preferred method for diagnosing pulmonary tuberculosis. Its main advantages include convenience, low cost, and low radiation dose. However, its sensitivity is insufficient, failing to detect subtle lesions in the chest and tracheobronchial lesions, and it cannot clearly show the characteristics and nature of pulmonary tuberculosis lesions, the relationship between lesions and bronchi, or the presence of mediastinal lymph node enlargement. CT scans can overcome these shortcomings of chest X-rays. However, CT scans are expensive, placing a significant financial burden on patients and involving high radiation levels, hindering large-scale disease screening. Therefore, designing a highly sensitive X-ray pulmonary tuberculosis screening technology would not only further promote the concept of early prevention and control of pulmonary tuberculosis but also improve the social benefits of tuberculosis prevention and control. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to provide a precise auxiliary screening technology for pulmonary tuberculosis, which aims to improve the accuracy and sensitivity of pulmonary nodule screening by using CT image features to assist in the extraction of X-ray chest radiograph features.

[0005] To achieve the above technical objectives, this application provides a precise auxiliary screening method for pulmonary tuberculosis, comprising the following steps:

[0006] Based on the first disease classification network, relevant features of CT images for pulmonary tuberculosis are extracted, and a simulated X-ray chest X-ray is generated through the front projection algorithm. The first disease classification network includes, but is not limited to, one of DenseNet, ResNet, and SwinTransformer.

[0007] Based on the first disease classification network, a second disease classification network is selected. Relevant features are distilled into the second disease classification network, and the feature extraction capability of the model is enhanced through domain adaptation learning to construct a screening model for screening pulmonary tuberculosis based on real X-ray chest films.

[0008] Preferably, in the process of generating a simulated X-ray chest radiograph, a simulated X-ray chest radiograph is generated based on CT images annotated with relevant features, using a CT pre-projection algorithm, including but not limited to the Siddon algorithm.

[0009] Preferably, in the process of obtaining the second disease classification network, a network model that is different from the first disease classification network is selected as the second disease classification network.

[0010] Preferably, in the process of constructing the screening model, the loss function of the screening model is expressed as:

[0011] L distill =CE{f CT f X-Ray}

[0012] L CE1 =CE{f CT gt CT}

[0013] L CE2 =CE{f X-Ray gt X-Ray}

[0014]

[0015] In the formula, f CT The logits generated by the first disease classification network, f X-Ray logits,gt generated for the second disease classification network CT gt is the gold standard for CT imaging. X-Ray The gold standard for real-world chest X-rays is represented by CE, where CE is the cross-entropy loss, and x ~ P. 仿真 Indicates that X is a simulated X-ray chest X-ray, x~P 真实 X represents a real X-ray chest radiograph, and D represents a domain discriminator.

[0016] Preferably, in the process of constructing the screening model, the following methods are used: and L D Alternating training E X-Ray The training continues until the domain discriminator can no longer determine whether the input effect belongs to the simulation effect or the real image and the loss function oscillates stably, at which point the accuracy can no longer increase and the training stops.

[0017] This invention discloses a precision auxiliary screening system for pulmonary tuberculosis, comprising:

[0018] The simulation module is used to extract relevant features of CT images for pulmonary tuberculosis based on the first disease classification network, and generate simulated X-ray chest films through the front projection algorithm. The first disease classification network includes, but is not limited to, one of DenseNet, ResNet, and SwinTransformer.

[0019] The screening module is used to select a second disease classification network based on a first disease classification network, distill relevant features into the second disease classification network, and enhance the feature extraction capability of the model through domain adaptation learning to build a screening model for screening pulmonary tuberculosis based on real X-ray chest films.

[0020] Preferably, the simulation module is also used to generate a simulated X-ray chest radiograph based on CT images annotated with relevant features, using a CT pre-projection algorithm, including but not limited to the Siddon algorithm.

[0021] Preferably, the simulation module is also used to select a network model that is different from the first disease classification network as the second disease classification network.

[0022] The present invention discloses the following technical effects:

[0023] This invention uses CT image guidance to enable the X-Ray chest radiograph tuberculosis classification network to extract features that are more sensitive to the classification of tuberculosis disease;

[0024] This invention enables a tuberculosis screening network based on simulated X-ray chest X-rays to achieve the same accuracy and sensitivity for real X-ray chest X-rays in the future through domain adaptation learning.

[0025] This invention integrates CT medical physics, distillation learning, and domain adaptation algorithms. By distilling features from CT images, it teaches the X-Ray chest radiograph tuberculosis classification network to extract features that are more sensitive to the classification of tuberculosis. Then, through domain adaptation learning, it enhances the model's robustness to the input, thereby achieving the accuracy of CT screening through X-Ray screening. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of the overall training process of the method described in this invention;

[0028] Figure 2This is a schematic diagram of the inference process after the model training is completed according to the present invention;

[0029] Figure 3 This is a schematic diagram of the method described in this invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0031] like Figure 1-3 As shown, the present invention provides a precise auxiliary screening method for pulmonary tuberculosis, comprising the following steps:

[0032] Based on the first disease classification network, relevant features of CT images for pulmonary tuberculosis are extracted, and a simulated X-ray chest X-ray is generated through the front projection algorithm. The first disease classification network includes, but is not limited to, one of DenseNet, ResNet, and SwinTransformer.

[0033] Based on the first disease classification network, a second disease classification network is selected. Relevant features are distilled into the second disease classification network, and the feature extraction capability of the model is enhanced through domain adaptation learning to construct a screening model for screening pulmonary tuberculosis based on real X-ray chest films.

[0034] More preferably, the precision assisted screening method proposed in this invention generates a simulated X-ray chest X-ray based on CT images annotated with relevant features, using a CT pre-projection algorithm, including but not limited to the Siddon algorithm.

[0035] More preferably, in the precise assisted screening method proposed in this invention, during the process of obtaining the second disease classification network, a network model that is different from the first disease classification network is selected as the second disease classification network.

[0036] More preferably, in the precise assisted screening method proposed in this invention, the loss function of the screening model during the construction process is expressed as:

[0037] L distill =CE{f CT f X-Ray}

[0038] L CE1 =CE{f CT gt CT}

[0039] L CE2 =CE{f X-Ray gt X-Ray}

[0040]

[0041] In the formula, f CT The logits generated by the first disease classification network, f X-Ray logits,gt generated for the second disease classification network CT gt is the gold standard for CT imaging. X-Ray The gold standard for real-world chest X-rays is represented by CE, where CE is the cross-entropy loss, and x ~ P. 仿真 Indicates that X is a simulated X-ray chest X-ray, x~P 真实 X represents a real X-ray chest radiograph, and D represents a domain discriminator.

[0042] More preferably, the precision-assisted screening method proposed in this invention utilizes, during the process of constructing the screening model, and L D Alternating training E X-Ray The training continues until the domain discriminator can no longer determine whether the input effect belongs to the simulation effect or the real image and the loss function oscillates stably, at which point the accuracy can no longer increase and the training stops.

[0043] This invention discloses a precision auxiliary screening system for pulmonary tuberculosis, comprising:

[0044] The simulation module is used to extract relevant features of CT images for pulmonary tuberculosis based on the first disease classification network, and generate simulated X-ray chest films through the front projection algorithm. The first disease classification network includes, but is not limited to, one of DenseNet, ResNet, and SwinTransformer.

[0045] The screening module is used to select a second disease classification network based on a first disease classification network, distill relevant features into the second disease classification network, and enhance the feature extraction capability of the model through domain adaptation learning to build a screening model for screening pulmonary tuberculosis based on real X-ray chest films.

[0046] More preferably, the simulation module of the precision assisted screening system disclosed in this invention is also used to generate a simulated X-ray chest X-ray based on CT images annotated with relevant features, using CT pre-projection algorithms, including but not limited to the Siddon algorithm.

[0047] More preferably, the simulation module of the precision assisted screening system disclosed in this invention is also used to select a network model that is different from the first disease classification network as the second disease classification network.

[0048] More preferably, the screening module of the precision assisted screening system disclosed in this invention is further used to obtain the loss function of the screening model as follows:

[0049] L distill =CE{f CT f X-Ray}

[0050] L CE1 =CE{f CT gt CT}

[0051] L CE2 =CE{f X-Ray gt X-Ray}

[0052]

[0053] In the formula, f CT The logits generated by the first disease classification network, f X-Ray logits,gt generated for the second disease classification network CT gt is the gold standard for CT imaging. X-Ray The gold standard for real-world chest X-rays is represented by CE, where CE is the cross-entropy loss, and x ~ P. 仿真 Indicates that X is a simulated X-ray chest X-ray, x~P 真实 X represents a real X-ray chest radiograph, and D represents a domain discriminator.

[0054] More preferably, the screening module of the precision assisted screening system disclosed in this invention is also used to utilize and L D Alternating training E X-Ray The training continues until the domain discriminator can no longer determine whether the input effect belongs to the simulation effect or the real image and the loss function oscillates stably, at which point the accuracy can no longer increase and the training stops.

[0055] Example: Studies have shown that CT images are more sensitive than chest X-rays for screening pulmonary tuberculosis, and can detect hidden small lesions in the chest and trachea and bronchi. They can also clearly show the characteristics and nature of pulmonary tuberculosis lesions, the relationship between lesions and bronchi, and whether there is enlargement of mediastinal lymph nodes. Therefore, this invention aims to use CT image features to assist in the extraction of features from chest X-rays, thereby improving the accuracy and sensitivity of chest X-ray screening for pulmonary nodules.

[0056] This invention patent's CT-assisted domain-adaptive X-ray accurate screening scheme for pulmonary tuberculosis comprises several parts: 1. Three-dimensional CT feature extraction and pulmonary tuberculosis classification network; 2. Chest radiograph simulation; 3. X-ray chest radiograph feature extraction and pulmonary tuberculosis classification network; 4. Feature distillation; 5. Real X-ray chest radiograph feature extraction based on domain adaptation. The basic principle of this invention is as follows (see...). Figure 1 ):

[0057] This invention designs a three-dimensional disease classification network (including but not limited to common DenseNet, ResNet, SwinTransformer, etc.) that can extract relevant features of CT images for pulmonary tuberculosis, and these features can reflect the diagnostic information of pulmonary tuberculosis in CT images.

[0058] Furthermore, based on CT physics knowledge, this invention obtains corresponding simulated X-ray chest films through front projection algorithms (including but not limited to common Siddon algorithms, etc.), and uses new disease classification networks (including but not limited to common DenseNet, ResNet, SwinTransformer, etc.) to extract and classify the features of the simulated chest films;

[0059] To ensure that the feature richness extracted from X-ray chest images matches the rich features contained in CT images, this invention utilizes a distillation learning method to distill the features extracted from the 3D CT feature extraction and tuberculosis classification network into the X-ray chest image feature extraction and tuberculosis classification network, thereby enhancing the ability of X-ray chest image feature extraction.

[0060] Considering the difference between simulated and real chest X-rays, this invention employs a domain-adaptive learning approach, using a domain discriminator to improve the X-ray chest X-ray feature extractor E. X-Ray The extraction capability is more robust, thus making the X-Ray chest radiograph feature extractor E X-Ray It can extract image features from both simulated and real X-ray chest radiographs.

[0061] The training process of this invention is as follows:

[0062] First, based on clinical information, we enrolled patients who underwent a large number of CT scans and chest X-rays, and collected their relevant CT images, chest X-rays, and corresponding diagnostic results.

[0063] All CT images were assigned to the training set, and chest X-rays were assigned to the training set (70%), validation set (10%), and internal test set (20%).

[0064] During each training session, the CT image of one patient and the X-ray image of another patient are input simultaneously.

[0065] The network framework of this invention is trained alternately according to the following total loss function:

[0066] L distill =CE{f CT f X-Ray};

[0067] L CE1 =CE{f CT gt CT};

[0068] L CE2 =CE{f X-Ray gt X-Ray};

[0069]

[0070] Among them, f CT The logits generated for 3D CT feature extraction and tuberculosis classification network, f X-Ray logits,gt generated for X-ray chest radiograph feature extraction and pulmonary tuberculosis classification network CT gt is the gold standard for CT imaging. X-Ray The gold standard for real-world chest X-rays is represented by CE, where CE is the cross-entropy loss, and x ~ P. 仿真 Indicates that X is a simulated X-ray chest X-ray, x~R 真实 X represents a real X-ray chest radiograph, and D represents the domain discriminator;

[0071] This invention utilizes the training paradigm of generative adversarial networks. and L D Alternating training E X-Ray The training continues until the domain discriminator can no longer determine whether the input effect belongs to the simulation effect or the real image and the loss function oscillates stably, at which point the accuracy can no longer increase and the training stops.

[0072] The testing process of this invention is relatively simple (see...). Figure 2 Once the model training is complete, this invention only retains the X-Ray chest image feature extraction and pulmonary tuberculosis classification network to predict pulmonary tuberculosis from the input X-Ray chest image.

[0073] This invention differs from previous X-ray-based tuberculosis screening models. Conventional models mainly collect a large number of chest X-ray images of tuberculosis patients and use the gold standard for end-to-end training. They do not incorporate CT image features, so they cannot overcome the drawbacks of X-ray imaging screening for tuberculosis: insufficient sensitivity, inability to detect hidden small lesions in the chest and tracheobronchial lesions, and inability to clearly show the characteristics and nature of tuberculosis lesions, the relationship between lesions and bronchi, and whether there is mediastinal lymph node enlargement.

[0074] The advantages of this invention are as follows: This invention integrates CT medical physics, distillation learning, and domain adaptation algorithms; through feature distillation of CT images, it teaches the X-Ray chest radiograph tuberculosis classification network to extract features that are more sensitive to the classification of tuberculosis; then, through domain adaptation learning, it enhances the robustness of the model to the input, thereby enabling the use of simple and inexpensive X-Ray screening in primary hospitals to achieve the accuracy of CT screening.

[0075] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0076] In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0077] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A precise auxiliary screening method for pulmonary tuberculosis, characterized in that, Includes the following steps: Based on the first disease classification network, relevant features of CT images for pulmonary tuberculosis are extracted, and a simulated X-ray chest X-ray is generated through the front projection algorithm. The first disease classification network includes, but is not limited to, one of DenseNet, ResNet, and SwinTransformer. Based on the first disease classification network, a second disease classification network is selected. The relevant features are distilled into the second disease classification network, and the feature extraction capability of the model is enhanced through domain adaptation learning to construct a screening model for screening pulmonary tuberculosis based on real X-ray chest films.

2. The method for precise auxiliary screening of pulmonary tuberculosis according to claim 1, characterized in that: In the process of generating a simulated X-ray chest radiograph, the simulated X-ray chest radiograph is generated based on the CT image labeled with the relevant features, using a CT pre-projection algorithm, including but not limited to the Siddon algorithm.

3. The method for precise auxiliary screening of pulmonary tuberculosis according to claim 2, characterized in that: In the process of obtaining the second disease classification network, a network model that is different from the first disease classification network is selected as the second disease classification network.

4. The method for precise auxiliary screening of pulmonary tuberculosis according to claim 3, characterized in that: In the process of constructing the screening model, the loss function of the screening model is expressed as: IT distill =CE{f CT ,f X-Ray } IT CE1 =CE{f CT ,gt CT } IT CE2 =CE{f X-Ray ,gt X-Ray } In the formula, f CT The logits generated by the first disease classification network, f X-Ray logits,gt generated for the second disease classification network CT gt is the gold standard for CT imaging. X-Ray The gold standard for real-world chest X-rays is represented by CE, where CE is the cross-entropy loss, and x ~ P. 仿真 Indicates that X is a simulated X-ray chest X-ray, x~P 真实 X represents a real X-ray chest radiograph, and D represents a domain discriminator.

5. The method for precise auxiliary screening of pulmonary tuberculosis according to claim 4, characterized in that: In the process of building the screening model, L was used EX-Ray =L distill +L CE1 +L CE2 and L D Alternating training E X-Ray The training continues until the domain discriminator can no longer determine whether the input effect belongs to the simulation effect or the real image and the loss function oscillates stably, at which point the accuracy can no longer increase and the training stops.

6. A precision auxiliary screening system for pulmonary tuberculosis, characterized in that, include: The simulation module is used to extract relevant features of CT images for pulmonary tuberculosis based on a first disease classification network, and generate a simulated X-ray chest X-ray through a front projection algorithm. The first disease classification network includes, but is not limited to, one of DenseNet, ResNet, and SwinTransformer. The screening module is used to select a second disease classification network based on the first disease classification network, distill the relevant features into the second disease classification network, and enhance the feature extraction capability of the model through domain adaptation learning to construct a screening model for screening pulmonary tuberculosis based on real X-ray chest films.

7. The precision auxiliary screening system for pulmonary tuberculosis according to claim 6, characterized in that: The simulation module is also used to generate the simulated X-ray chest radiograph based on the CT image labeled with the relevant features, using a CT pre-projection algorithm, including but not limited to the Siddon algorithm.

8. The precision auxiliary screening system for pulmonary tuberculosis according to claim 7, characterized in that: The simulation module is also used to select a network model that is different from the first disease classification network as the second disease classification network.

Images