Pneumonia x-ray chest radiography classification method, device and medium based on domain adaptation

Abstract

The present application relates to a kind of based on field self-adapting pneumonia X-ray chest radiograph classification method, equipment and medium, comprising the following steps: obtaining pneumonia X-ray chest radiograph, input trained based on field self-adapting pneumonia X-ray chest radiograph classification model, obtain the classification result of pneumonia X-ray chest radiograph;The based on field self-adapting pneumonia X-ray chest radiograph classification model includes feature extractor, two classifiers, one domain discriminator, two feature converters and its corresponding two discriminators, based on the total loss of introducing cyclic consistency loss based on the feature after being converted multiple times with original feature calculation Iterative training is carried out to the based on field self-adapting pneumonia X-ray chest radiograph classification model, and the cyclic consistency loss is based on the feature after being converted multiple times with original feature calculation.Compared with prior art, the present application can further improve the classification accuracy of pneumonia X-ray chest radiograph.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, and in particular relates to a domain-adaptive method, device and medium for classifying chest X-rays of pneumonia. Background Technology

[0002] Currently, chest X-ray is one of the best methods for diagnosing pneumonia, and using deep learning methods to process medical images to assist in diagnosis has become a mainstream trend. Deep learning models require a large amount of training data, and due to the special nature of medical images, large public datasets are difficult to obtain. Domain adaptation can transfer a deep learning model from a domain that is already labeled or has a large amount of data (the source domain) to another domain that lacks a large amount of labeled data or is related to the source domain but has certain differences (the target domain), thereby improving its performance in the target domain.

[0003] Chinese invention patent CN114359629B discloses a method for classifying and recognizing pneumonia chest X-rays based on deep transfer learning. It constructs a deep transfer learning model, using large chest X-ray datasets of other diseases as the source domain dataset to first train the model and obtain base weights. These weights are then transferred to the transfer learning model, freezing the low-level network parameters. The model is then trained on the target domain dataset to construct the optimal deep transfer learning model, improving the recognition rate of pneumonia chest X-rays. This effectively solves the problems of small datasets not meeting training requirements and being prone to overfitting in deep learning methods. However, the above method lacks effective feature representation, making it difficult to ensure significant similarity between the source and target domains. Features learned from the source domain may not be applicable to the target domain, resulting in negative transfer problems and severely impacting the model's performance in the target domain. Furthermore, additional feature transformation or mapping steps increase the model's complexity. Therefore, a novel domain-adaptive pneumonia chest X-ray classification method is needed to further improve the classification accuracy of pneumonia chest X-rays. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art by providing a domain-adaptive method, device, and medium for classifying pneumonia chest X-rays, thereby further improving the classification accuracy of pneumonia chest X-rays.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] This invention provides a domain-adaptive chest X-ray classification method for pneumonia, comprising the following steps:

[0007] Obtain a chest X-ray of pneumonia, input it into a trained domain-adaptive chest X-ray classification model of pneumonia, and obtain the classification result of the chest X-ray of pneumonia.

[0008] The domain-adaptive pneumonia chest X-ray classification model includes a feature extractor, a source domain classifier, a target domain classifier, a source domain discriminator, a target domain discriminator, a domain discriminator, a first feature converter, and a second feature converter. The training process of the domain-adaptive pneumonia chest X-ray classification model is as follows:

[0009] S1. Obtain source domain data and target domain data and perform preprocessing, wherein the source domain data is a multi-classification chest X-ray dataset and the target domain data is a binary classification chest X-ray dataset of pneumonia.

[0010] S2. Input the preprocessed source domain data and target domain data into the feature extractor for feature extraction, input the obtained source domain features and target domain features into the source domain classifier and the target domain classifier respectively for classification, and calculate the corresponding classification loss.

[0011] S3. Input the source domain features and target domain features extracted in step S2 into the domain discriminator for discrimination and calculate the discrimination loss.

[0012] S4. Input the source domain features and target domain features extracted in step S2 into the first feature converter and the second feature converter respectively, convert the source domain features into target domain features, and convert the target domain features into source domain features. Then, input the source domain features and the converted source domain features into the source domain discriminator for discrimination, and input the target domain features and the converted target domain features into the target domain discriminator for discrimination, and calculate the corresponding conversion loss.

[0013] S5. Input the transformed features from step S4 back into the first feature converter or the second feature converter, and calculate the cycle consistency loss after the two feature transformations.

[0014] S6. Repeat steps S1-S5 until the total loss or number of iterations reaches a set threshold. The total loss is constructed based on the classification loss, discrimination loss, transformation loss and cycle consistency loss.

[0015] Furthermore, in step S1, the preprocessing includes image normalization and random cropping.

[0016] Furthermore, in step S3, the discrimination loss is calculated based on the joint variable between the features extracted by the feature extractor and the classification result obtained in step S2.

[0017] Furthermore, the specific formula for calculating the discrimination loss is as follows:

[0018]

[0019] in, To determine the loss, hs =(f s ,p s f is the source domain feature f obtained by the feature extractor F. s With the corresponding classifier P s The obtained classification result p s The joint variable between them, h t =(f t ,p t ) represents the feature f of the target domain. t With the corresponding classification result p t Joint variables between D is a conditional strategy for a domain discriminator based on multilinear mappings. d This refers to the discrimination operation of the domain discriminator.

[0020] Furthermore, in step S4, the specific formula for calculating the transformation loss from source domain features to target domain features is as follows:

[0021]

[0022] in, D is the transformation loss that transforms source domain features into target domain features. t For the discrimination operation of the target domain discriminator, f t For the target domain features, f s For source domain features, T s2t This refers to the operation of transforming source domain features into target domain features.

[0023] Furthermore, in step S4, the specific formula for calculating the transformation loss from target domain features to source domain features is as follows:

[0024]

[0025] in, D is the transformation loss that transforms target domain features into source domain features. s For the discrimination operation of the source domain discriminator, f t For the target domain features, f s For source domain features, T t2s This refers to the operation of transforming features in the target domain into features in the source domain.

[0026] Furthermore, in step S5, the cycle consistency loss is calculated based on the L2 loss between the features after two transformations and the features before transformation, and the specific calculation formula is as follows:

[0027]

[0028] in, For the cycle consistency loss, f t For the target domain features, fs For source domain features, T t2s T is the operation that transforms features from the target domain to features from the source domain. s2t This refers to the operation of transforming source domain features into target domain features.

[0029] Furthermore, the expression for the total loss is as follows:

[0030]

[0031] in, For the total loss, For classifying losses, To determine the loss, The transformation loss is used to convert features from the source domain to features from the target domain. The transformation loss is used to convert features in the target domain into features in the source domain. The loss is the cycle consistency loss, where α, β, and γ are coefficients.

[0032] The present invention also provides an electronic device, including a memory, a processor, and a program stored in the memory, wherein the processor executes the program to implement the above-described method.

[0033] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. This invention achieves the classification of pneumonia chest X-rays through a domain-adaptive pneumonia chest X-ray classification model. The domain-adaptive pneumonia chest X-ray classification model includes a feature extractor, a source domain classifier, a target domain classifier, a source domain discriminator, a target domain discriminator, a domain discriminator, a first feature converter, and a second feature converter. The total loss of the domain-adaptive pneumonia chest X-ray classification model introduces a cycle consistency loss. The cycle consistency loss is calculated based on the features after two feature transformations and the features before transformation. This can constrain the two feature converters, preserve the original feature information as much as possible, improve the feature transformation capability of the feature converters, and further improve the application effect of the model in the target domain, that is, further improve the classification accuracy of pneumonia chest X-rays.

[0036] 2. This invention builds and trains a domain-adaptive pneumonia chest X-ray classification model based on adversarial learning. It uses two feature transformers to transform the source domain features and target domain features extracted by the feature extractor twice. Then, two discriminators are used to make judgments and calculate the corresponding losses. The model is then iteratively trained. This can align the features between the source domain and the target domain, explicitly restrict the feature extractor and feature transformer from acquiring domain-invariant features, improve the transfer performance of the model, and reduce domain bias. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the structure and training process of a domain-adaptive pneumonia chest X-ray classification model. Detailed Implementation

[0038] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0039] Example:

[0040] This embodiment provides a domain-adaptive chest X-ray classification method for pneumonia, including the following steps:

[0041] Obtain a chest X-ray of pneumonia, input it into a trained domain-adaptive chest X-ray classification model of pneumonia, and obtain the classification result of the chest X-ray of pneumonia.

[0042] The domain-adaptive pneumonia chest X-ray classification model includes a feature extractor, a source domain classifier, a target domain classifier, and a recurrent adversarial learning (ROL) module. The ROLSA-based ROLSA module comprises a source domain discriminator, a target domain discriminator, a domain discriminator, a first feature converter, and a second feature converter. The model training process is as follows: Figure 1 As shown, the details are as follows:

[0043] S1. Obtain source domain data and target domain data and perform preprocessing. The source domain data is a multi-classified chest X-ray dataset. In this embodiment, a chest X-ray dataset (Chest x-ray 14 dataset) including 14 common chest diseases is selected. The target domain data is a binary (pneumonia or non-pneumonia) chest X-ray dataset for pneumonia.

[0044] Preprocessing includes image standardization and random cropping. Specifically, a suitable window is selected based on the image sizes of the source and target domain datasets, and the images are uniformly scaled to a fixed size of 224*224. Random cropping is performed with a 50% probability, and all images are standardized.

[0045] S2. Input the preprocessed source domain data and target domain data into the feature extractor for feature extraction. Input the obtained source domain features and target domain features into the source domain classifier and the target domain classifier respectively for classification, and calculate the corresponding classification loss.

[0046] Preferably, the feature extractor can use models such as ResNet18 and ResNet50, which learn representative features in the image through a neural network (e.g., ResNet18 network) (black box learning), which may include, but are not limited to, lung contours, edges of thoracic structures, lesions, higher-level local shapes, etc.

[0047] S3. Input the source domain features and target domain features extracted in step S2 into the domain discriminator for discrimination (discriminator: target and feature or source domain features), and calculate the discrimination loss.

[0048] The discriminant loss is calculated based on the joint variable between the features extracted by the feature extractor and the classification result obtained in step S2. The specific calculation formula is as follows:

[0049]

[0050] in, To determine the loss, h s =(f s ,p s f is the source domain feature f obtained by the feature extractor F. s With the corresponding classifier P s The obtained classification result p s The joint variable between them, h t =(f t ,p t ) represents the feature f of the target domain. t With the corresponding classification result p t Joint variables between This is a conditional strategy for a domain discriminator based on multilinear mappings (see Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. Conditional adversarial domain adaptation. Advances in neural information processing systems, 31, 2018). d This refers to the discrimination operation of the domain discriminator.

[0051] S4. Input the source domain features and target domain features extracted in step S2 into the first feature converter and the second feature converter respectively, convert the source domain features into target domain features, and convert the target domain features into source domain features. Then, input the source domain features and the converted source domain features into the source domain discriminator for discrimination, input the target domain features and the converted target domain features into the target domain discriminator for discrimination, and calculate the corresponding transformation loss.

[0052] The specific formula for calculating the transformation loss from source domain features to target domain features is as follows:

[0053]

[0054] in, D is the transformation loss that transforms source domain features into target domain features. t For the discrimination operation of the target domain discriminator, f t For the target domain features, f s For source domain features, T s2t This refers to the operation of transforming source domain features into target domain features.

[0055] The specific formula for calculating the transformation loss from target domain features to source domain features is as follows:

[0056]

[0057] in, D is the transformation loss that transforms target domain features into source domain features. s For the discrimination operation of the source domain discriminator, f t For the target domain features, f s For source domain features, T t2s This refers to the operation of transforming features in the target domain into features in the source domain.

[0058] S5. Input the transformed features from step S4 back into the first feature converter or the second feature converter, and calculate the cycle consistency loss after the two feature transformations.

[0059] The cycle consistency loss is calculated based on the L2 loss between the features after two transformations and the features before the transformation. The specific calculation formula is as follows:

[0060]

[0061] in, For the cycle consistency loss, f t For the target domain features, f s For source domain features, T t2s T is the operation that transforms features from the target domain to features from the source domain. s2t This refers to the operation of transforming source domain features into target domain features.

[0062] S6. Repeat steps S1-S5 until the total loss or number of iterations reaches a set threshold. The total loss is constructed based on the classification loss, discriminant loss, transformation loss, and cycle consistency loss. The specific expression for the total loss is as follows:

[0063]

[0064] in, For the total loss, For classifying losses, To determine the loss, The transformation loss is used to convert features from the source domain to features from the target domain. The transformation loss is used to convert features in the target domain into features in the source domain. The loss is the cycle consistency loss, where α, β, and γ are coefficients.

[0065] To address the domain shift problem, this embodiment proposes a domain-adaptive pneumonia chest X-ray classification model that incorporates a cycle-consistent adversarial learning module to align features between the source and target domains. First, the model employs a domain discriminator (determining whether a sample originates from the source or target domain) to capture potential discriminative information and complex knowledge using the cross-covariance between features and predictions. Second, based on the assumption that features are interconvertible between different domains, the method trains two feature transformers (generators): one transforms features from the source domain to the target domain, and the other does the reverse. Building upon this, two discriminators are configured: one distinguishes between real target domain samples and fake target domain samples (samples transformed from the source domain to the target domain), and the other distinguishes between real source domain samples and fake source domain samples (samples transformed from the target domain to the source domain). Furthermore, a cycle-consistency loss is calculated to enforce constraints on the transformers. To address the challenge of heterogeneous problems (source domain is multi-class, target domain is binary), this model designs two sub-networks that share a feature extractor and a conditional adversarial module. One sub-network is used for the multi-class task in the source domain, and the other is used for the binary task in the target domain. This allows the model to handle cases where the source and target domain categories are inconsistent. By introducing a total loss with multiple losses such as cycle consistency, the features are explicitly restricted, thereby ensuring significant similarity between the source and target domains.

[0066] If the above methods are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0067] The above description of the embodiments is provided to enable those skilled in the art to understand and use the invention. It will be apparent to those skilled in the art that various modifications can be made to these embodiments, and the general principles described herein can be applied to other embodiments without inventive effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the invention should be within the protection scope of the present invention.

Claims

1. A domain-adaptive classification method for chest X-rays of pneumonia, characterized in that, Includes the following steps: Obtain a chest X-ray of pneumonia, input it into a trained domain-adaptive chest X-ray classification model of pneumonia, and obtain the classification result of the chest X-ray of pneumonia. The domain-adaptive pneumonia chest X-ray classification model includes a feature extractor, a source domain classifier, a target domain classifier, a source domain discriminator, a target domain discriminator, a domain discriminator, a first feature converter, and a second feature converter. The training process of the domain-adaptive pneumonia chest X-ray classification model is as follows: S1. Obtain source domain data and target domain data and perform preprocessing, wherein the source domain data is a multi-classification chest X-ray dataset and the target domain data is a binary classification chest X-ray dataset of pneumonia. S2. Input the preprocessed source domain data and target domain data into the feature extractor for feature extraction, input the obtained source domain features and target domain features into the source domain classifier and the target domain classifier respectively for classification, and calculate the corresponding classification loss. S3. Input the source domain features and target domain features extracted in step S2 into the domain discriminator for discrimination and calculate the discrimination loss. S4. Input the source domain features and target domain features extracted in step S2 into the first feature converter and the second feature converter respectively, convert the source domain features into target domain features, and convert the target domain features into source domain features. Then, input the source domain features and the converted source domain features into the source domain discriminator for discrimination, and input the target domain features and the converted target domain features into the target domain discriminator for discrimination, and calculate the corresponding transformation loss. S5. Input the transformed features from step S4 back into the first feature converter or the second feature converter, and calculate the cycle consistency loss after the two feature transformations. S6. Repeat steps S1-S5 until the total loss or number of iterations reaches a set threshold. The total loss is constructed based on the classification loss, discrimination loss, transformation loss and cycle consistency loss. In step S5, the cycle consistency loss is calculated based on the L2 loss between the features after two transformations and the features before transformation. The specific calculation formula is as follows: in, For cycle consistency loss, For target domain features, Features of the source domain This refers to the operation of transforming features from the target domain into features from the source domain. This refers to the operation of transforming source domain features into target domain features.

2. The domain-adaptive chest X-ray classification method for pneumonia according to claim 1, characterized in that, In step S1, the preprocessing includes image normalization and random cropping.

3. The domain-adaptive chest X-ray classification method for pneumonia according to claim 1, characterized in that, In step S3, the discrimination loss is calculated based on the joint variable between the features extracted by the feature extractor and the classification result obtained in step S2.

4. The domain-adaptive chest X-ray classification method for pneumonia according to claim 3, characterized in that, The specific formula for calculating the discrimination loss is as follows: in, To determine the loss, For feature extractor Source domain features obtained With corresponding classifier The obtained classification results Joint variables between For target domain features Corresponding classification results Joint variables between This is a conditional strategy for a domain discriminator based on multilinear mappings. This refers to the discrimination operation of the domain discriminator.

5. A domain-adaptive chest X-ray classification method for pneumonia according to claim 1, characterized in that, In step S4, the specific formula for calculating the transformation loss from source domain features to target domain features is as follows: in, The transformation loss is used to convert features from the source domain to features from the target domain. For the discrimination operation of the target domain discriminator, For target domain features, Features of the source domain This refers to the operation of transforming source domain features into target domain features.

6. The domain-adaptive chest X-ray classification method for pneumonia according to claim 1, characterized in that, In step S4, the specific formula for calculating the transformation loss from target domain features to source domain features is as follows: in, The transformation loss is used to convert features in the target domain into features in the source domain. For the discrimination operation of the source domain discriminator, For target domain features, Features of the source domain This refers to the operation of transforming features in the target domain into features in the source domain.

7. The domain-adaptive chest X-ray classification method for pneumonia according to claim 1, characterized in that, The expression for the total loss is as follows: in, For the total loss, For classifying losses, To determine the loss, The transformation loss is used to convert features from the source domain to features from the target domain. The transformation loss is used to convert features in the target domain into features in the source domain. For cycle consistency loss, , and is a coefficient.

8. An electronic device comprising a memory, a processor, and a program stored in the memory, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images