Joint wasserstein distance and difference measure for chest radiograph abnormality recognition domain adaptation method and system

Abstract

The present application belongs to the technical field of deep learning and medical image processing, and specifically relates to a chest radiograph abnormality recognition domain adaptive method and system combining Wasserstein distance and difference measurement. The method comprises the following steps: S1, chest radiograph data preparation and preprocessing; S2, multi-scale feature extraction based on Swin Transformer; S3, loss minimization based on Wasserstein distance and contrast domain difference; S4, model verification for chest radiograph prediction. The present application has the characteristics that not only the source domain sample closest to the target domain sample can be selected, but also the distance between the same categories of samples in the target domain and the source domain can be shortened in the feature space, and the distance between different categories can be widened, and at the same time, the extracted multi-scale features are used for chest radiograph classification task, which can effectively improve the receptive field and capture more information beneficial to the chest radiograph classification task.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of deep learning and medical image processing, and particularly relates to a chest radiograph abnormality recognition domain adaptive method and system combining Wasserstein distance and difference measurement. BACKGROUND

[0002] At present, machine learning has been widely applied to various fields, but it is very difficult to collect large-scale data with complete annotation as a training set, and it is also necessary to consider whether the collected training set and test set have the same distribution. When there is a certain difference in data distribution or features between the training set and the test set, the model trained on the training set will lead to a significant decline in performance when applied to the test set, that is, the model has poor generalization ability and does not learn useful knowledge from the training set to apply to the test set.

[0003] As for X-ray chest radiographs in the field of medical images, it is usually expensive and time-consuming to label medical images, especially for multiple center point studies. Sample data may come from different machine equipment of multiple hospitals, in addition, scanning protocols, photographing parameters and shooting angles, and subject groups will also lead to the migration of image distribution.

[0004] The literature

Pan S J, Qiang Y. A Survey on Transfer Learning [J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359

[0005] Then, the idea of confrontation is introduced into domain adaptation, and the training process is a game process of feature extractor and domain discriminator. Among them, the typical methods are domain adversarial neural network DANN and adversarial discriminative domain adaptation ADDA, but the adversarial training process will make the feature extraction network of the target domain pay more attention to the background area, so as to cause incorrect positioning. Only by confusing the domain discriminator can the features be aligned, which will cause large extraction feature error, and then an improved conditional domain adversarial network CDAN is appeared, which expands the conditional adversarial mechanism, defines the domain discriminator on the feature, and the performance is improved, but at the same time, it also brings huge parameter quantity.

[0006] Therefore, it is very important to design a joint Wasserstein distance and difference measure chest X-ray abnormality recognition domain adaptation method and system which can learn more useful knowledge from the training set (source domain) to the test set (target domain), has smaller parameter quantity, and improves the X-ray chest X-ray image classification evaluation index.

[0007] The Wasserstein distance is also called bulldozer distance.

[0008] For example, the X-ray chest X-ray bone suppression processing method based on wavelet decomposition and convolutional neural network described in the Chinese patent document with patent application number CN201710246611.8 adopts a convolutional neural network structure, takes the wavelet coefficient of the chest X-ray image as input, and predicts the wavelet coefficient image of the corresponding bone image or soft tissue image. Although the input chest X-ray image is processed in the wavelet domain, simple amplitude normalization processing can be used; the wavelet decomposition image of the chest X-ray image can be directly taken as input, the convolution kernel can be automatically learned and optimized through training of the convolutional neural network, and the feature extraction process of the image can be completed without the need to design an image feature extraction method, but the above method still has limitations in the classification accuracy and generalization ability of the X-ray chest X-ray. SUMMARY

[0009] The present application is to overcome the problems of incorrect positioning and large feature extraction error in conventional domain adaptation methods, and to provide a joint Wasserstein distance and difference measure chest X-ray abnormality recognition domain adaptation method and system which not only selects the source domain sample closest to the target domain sample, but also shortens the distance between the same categories in the target domain and the source domain samples in the feature space, and simultaneously uses the extracted multi-scale features for chest X-ray classification tasks, effectively improves the receptive field, and captures more information beneficial to the chest X-ray classification task.

[0010] In order to achieve the above-mentioned application purposes, the present application adopts the following technical solutions:

[0011] The joint Wasserstein distance and difference measure chest X-ray abnormality recognition domain adaptation method comprises the following steps:

[0012] S1, chest X-ray data preparation and preprocessing:

[0013] Obtain the CT image of the chest X-ray film, and perform data preprocessing on the CT image;

[0014] S2, multi-scale feature extraction based on Swin Transformer:

[0015] The CT image is input into a Patch partition module for partitioning, 4*4=16 adjacent pixels are set as one Patch, and flattening is performed in the channel direction, linear transformation is performed on the channel data of each pixel through a linear embedding layer, and then four stages are sequentially passed to construct feature maps of different sizes;

[0016] S3, loss minimization based on Wasserstein distance and contrast domain difference:

[0017] The source domain sample closest to the target domain sample is selected, and the Wasserstein distance is calculated; the target domain sample is processed by cross-domain similar-to-similar and dissimilar-to-dissimilar, to obtain the contrast domain difference; the total objective function is constructed through the obtained Wasserstein distance and contrast domain difference; the Swin Transformer network is optimized and the parameters are updated according to the total objective function;

[0018] S4, model verification for chest radiograph prediction:

[0019] The optimized and parameter-updated Swin Transformer network is verified, and the classification prediction task of the chest radiograph is performed.

[0020] Preferably, the data preprocessing in step S1 includes standardization and data enhancement; the standardization is normalization using the mean and standard deviation of the image; and the data enhancement is data enhancement of the image in the manner of random rotation and horizontal flip.

[0021] Preferably, the selecting the source domain sample closest to the target domain sample and calculating the Wasserstein distance in step S3 include the following steps:

[0022] S31, defining the source domain dataset as D S , the source domain data sample obeys the probability distribution D S ~X S ; the target domain dataset is D T , the target domain data sample obeys the probability distribution D T ~X T , and the distribution difference between the source domain and the target domain is measured by calculating the Wasserstein distance;

[0023] The Wasserstein distance is defined as shown in formula (1):

[0024]

[0025] Where, ∏(X S ,X T ) represents the source domain distribution X S and the target domain distribution X TThe set of all possible joint distributions combined, γ ~ Π(X S , T ) represents the cost of transforming X S to X T so that both are subject to the same distribution; for each possible joint distribution γ, one can obtain a sample p and q from sampling (p, q) ~ γ, and compute the distance between the samples ||p-q||, obtaining the expected value of the sample pair distance under the corresponding joint distribution γ, E (p,q)～γ [‖p-q‖], the smaller the expected value, the smaller the cost of transforming X S to X T .

[0026] Preferably, the cross-domain similar-to-same and dissimilar-to-different processing of the target domain samples in step S3 to obtain the contrast domain difference comprises the following steps.

[0027] S32, estimating the label assumption of the target domain samples by clustering, introducing the contrast domain difference in the clustering process, which is to calculate the difference between the specified class c and c'; minimizing the difference between the same classes to compress the feature representation of the intra-class samples, and maximizing the difference between the different classes to further push the feature representation of the inter-class samples away from the decision boundary;

[0028] The contrast domain difference is defined as formula (2):

[0029]

[0030] Where M is 5 classes, n t is the size of the target domain dataset, represents the set of target domain class labels, D cc′ is the average embedding estimation of class c and class c' in the reproducing kernel Hilbert space; minimizing the contrast domain difference D cdd so that the first half of the formula is as small as possible and the second half is as large as possible, that is, for the five types of lesions of the chest film, the sample distribution of the same class c in the target domain and the source domain is to be pulled closer, and the sample distribution of a certain class c in the target domain and all different classes c' in the source domain is to be pulled apart; φ is the network parameter.

[0031] Preferably, the total target function is constructed by the obtained Wasserstein distance and the contrast domain difference in step S3 comprises the following steps.

[0032] S33, introducing the Wasserstein distance obtained in step S31 and the contrast domain difference D cdd obtained in step S32 as a regularization term of the loss function into the classification network, obtaining the total target function as formula (3):

[0033]

[0034] wherein, l c is the cross-entropy loss obtained by source domain training, and lambda and beta are weight parameters for balancing the two parts of loss of Wasserstein distance and contrast domain difference, respectively, and the loss calculated through the loss formula is used to adjust the model parameters until the model converges.

[0035] Preferably, the optimization and parameter updating of the Swin Transformer network according to the total objective function in step S3 include the following steps:

[0036] S34, in the iterative process of training, first fix the network parameters phi, and update the sample clustering of the target domain, and after the clustering is completed, each target domain sample is assigned a label corresponding to the cluster to which each target domain sample belongs, and then the contrast domain difference is calculated through the updated target domain label, the data samples participating in the calculation of the contrast domain difference are determined by calculating the Wasserstein distance between the two domain samples, and finally the network parameters are updated by minimizing the contrast domain difference to complete the iteration.

[0037] Preferably, step S4 includes the following steps:

[0038] S41, in the verification mode, only standardization processing is performed on the data to be verified;

[0039] S42, the AUC, accuracy, sensitivity, specificity, positive predictive rate and negative predictive rate of each category of chest radiograph are calculated respectively, which are used for further verification of the model.

[0040] Preferably, the step S2 further includes the following steps:

[0041] The picture input size is set to 224x224, so that the output feature matrix size after 32 times downsampling in the four stages of Swin Transformer is 7x7, which is just the window size of 7x7 in the fourth stage;

[0042] The Swin-T model in the Swin Transformer architecture is selected, the number of layers of the Swin-T model is {2, 2, 6, 2}, and the channel depth of the hidden layer in the first stage is 96.

[0043] The application also provides a chest radiograph abnormality recognition domain adaptive system combined with Wasserstein distance and difference measurement, comprising:

[0044] A data preparation and preprocessing module is used to obtain CT images of chest X-ray films and perform data preprocessing on the CT images.

[0045] The feature extraction module is used for inputting the CT image into a Patch partition module for blocking, setting every 4*4=16 adjacent pixels as a Patch, and flattening in the channel direction, performing linear transformation on the channel data of each pixel through a linear embedding layer, and then sequentially constructing feature maps of different sizes through four stages;

[0046] The loss minimization module is used for selecting the source domain sample closest to the target domain sample and calculating the Wasserstein distance; performing cross-domain similar-to-similar and dissimilar-to-dissimilar processing on the target domain sample to obtain a contrast domain difference; constructing a total target function through the obtained Wasserstein distance and the contrast domain difference; and optimizing and updating the parameters of the Swin Transformer network according to the total target function;

[0047] The model verification module is used for verifying the Swin Transformer network after optimization and parameter updating, and performing a chest radiograph classification prediction task.

[0048] Compared with the prior art, the present application has the following advantages: (1) the present application uses Swin Transformer to extract deep feature representations of data samples; on the basis of selecting the source domain sample closest to the target domain sample, cross-domain similar-to-similar and dissimilar-to-dissimilar processing is realized, the classification accuracy of chest radiographs is improved, and better generalization ability is achieved; (2) the present application combines the respective advantages of the contrast domain difference and the Wasserstein distance, can not only select the source domain sample closest to the target domain sample, but also can shorten the distance between samples of the same class and lengthen the distance between samples of different classes in the feature space; (3) the present application uses Swin Transformer as a feature extractor, uses the multi-scale features extracted therefrom for chest radiograph classification tasks, and the hierarchical architecture of Swin Transformer can flexibly model different scales, effectively improve the receptive field, and capture more information conducive to chest radiograph classification tasks. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 A flow chart of the chest radiograph abnormality recognition domain self-adaptive method combining the Wasserstein distance and the difference metric provided by the embodiment of the present application;

[0050] Figure 2 A thermal contrast schematic diagram of the chest radiograph generated by the convolutional neural network and the Swin Transformer provided by the embodiment of the present application;

[0051] Figure 3 A contrast schematic diagram of the chest radiograph abnormality recognition model based on ResNet50 and the trained model of the present application on the evaluation AUC curve. Detailed Implementation

[0052] To more clearly illustrate the embodiments of the present invention, specific implementation methods will be described below with reference to the accompanying drawings. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings and other implementation methods can be obtained based on these drawings without any creative effort.

[0053] Example:

[0054] like Figure 1 As shown, this invention provides an adaptive method for chest radiograph anomaly recognition based on a combination of Wasserstein distance and difference metric, comprising the following steps:

[0055] S1, Chest X-ray data preparation and preprocessing:

[0056] Acquire CT images from chest X-rays and perform data preprocessing on the CT images;

[0057] S2, Multi-scale feature extraction based on Swin Transformer:

[0058] The CT image is input into the Patch partitioning module for block division. Each Patch consists of 4×4=16 adjacent pixels and is flattened in the channel direction. The channel data of each pixel is linearly transformed through a linear embedding layer, and feature maps of different sizes are constructed in four stages.

[0059] S3, loss minimization based on Wasserstein distance and contrast domain difference:

[0060] Select the source domain sample that is closest to the target domain sample and calculate the Wasserstein distance; perform cross-domain processing on the target domain samples to obtain the comparison domain difference; construct the overall objective function based on the obtained Wasserstein distance and comparison domain difference; optimize and update the parameters of the Swing Transformer network according to the overall objective function.

[0061] S4, after model validation, is used for chest X-ray prediction:

[0062] The optimized and parameter-updated Swin Transformer network was validated, and a chest X-ray classification prediction task was performed.

[0063] The specific implementation of step S1 is as follows:

[0064] Step S1: Chest X-ray data preparation and preprocessing

[0065] The chest radiograph dataset used in this embodiment is as follows: the source domain is the CheXpert dataset, which contains 224316 X-ray chest radiographs of 65240 patients taken from the front, back and side, among which front and back pictures are the majority. The dataset has uncertain medical labels and radiologist annotated reference standard evaluation set, which can be used to predict the probability of 14 different observations from multi-view chest radiographs; the target domain Chest X-Ray14 dataset contains 112120 front X-ray chest radiographs of 30805 patients, and the radiology report contains 14 common diseases. The dataset is randomly divided into training set, test set and validation set in the ratio of 8:1:1 as shown in Table 1, and the batchsize is set to 32.

[0066] Table 1 Dataset division table

[0067] Dataset Training set Test set Validation set CheXpert 179452 22432 22432 Chest X-Ray14 89696 11212 11212

[0068] In the method of the application, the same five types of diseases in the CheXpert dataset and the Chest X-Ray14 dataset are selected, which are Atelectasis, Cardiomegaly, Effusion, Consolidation and Edema.

[0069] The data preprocessing part includes two parts of standardization and data enhancement: the image is normalized by using the mean and standard deviation, so that the data distribution meets the standard normal distribution, and finally the picture is scaled to (32, 3, 224, 224); the data enhancement part includes horizontal flip and random rotation of-25 degrees to +25 degrees.

[0070] The specific implementation of step S2 is as follows:

[0071] Step S2: multi-scale feature extraction based on Swin Transformer

[0072] Swin Transformer is selected as the multi-scale feature extractor, and a set of heat maps generated by convolutional neural network (such as ResNet50) and Swin Transformer are compared as shown in the accompanying Figure 2 The darker the color in the heat map, the more attention the model pays to this area, and the greater the impact of the area on the final classification task. As can be seen from the figure, compared with the convolutional neural network, Swin Transformer pays more attention to the RoIs (region of interest) area where the chest radiograph has lesions, so the model can pay more attention to the feature information that is beneficial to the classification task.

[0073] The input chest film is a three-channel picture with an image height H and width W of 224*224. The picture is input into a Patch partition module for partitioning, each Patch having 16 pixels, each pixel having R, G, B three values. After flattening in the channel direction, the shape of the image changes from (32, 3, 224, 224) to (32, 48, 56, 56).

[0074] Then, different size feature maps are constructed through four stages, and the Swin Transformer Block is repeatedly stacked. For the Swin-T model, the number of stacked Blocks in the four stages is (2, 2, 6, 2) respectively, and the window size used in each stage is 7*7. Specifically, the first stage first passes through a linear embedding layer to linearly transform the channel data of each pixel into 96 dimensions, and the image shape changes from (32, 48, 56, 56) to (32, 96, 56, 56). The second, third and fourth stages are all based on the output of the previous stage and downsampled by two times. The image shape changes as follows: (32, 96, 56, 56)→(32, 192, 28, 28)→(32, 384, 14, 14)→(32, 768, 7, 7). In this way, multi-level feature maps are generated. The structure configuration information of the four stages of Swin-T is shown in Table 2.

[0075] Table 2 Structure configuration information table of four stages of Swin-T model

[0076]

[0077] The stacked Swin Transformer Block in the above includes two structures. The first one is a W-MSA module (Windows Multi-head Self-Attention). The introduction of this module can reduce the calculation amount of the self-attention operation. The calculation amount of the feature map is shown in formula (1):

[0078] A=4HWC 2 +2M 2 HWC (1)

[0079] Wherein, A represents the calculation amount, H is the height of the feature map, W is the width of the feature map, C is the depth of the feature map, and M is the width of the window.

[0080] The second one is a SW-MSA module (Shifted Windows Multi-Head Self-Attention). This module uses an offset window to enable information exchange between different windows. The two self-attention structures are connected in series to form a Block.

[0081] For the chest radiograph classification task, after the Swin Transformer network architecture, a Layer Norm layer, a global pooling layer and a fully connected layer are connected to obtain the final output.

[0082] The specific implementation of step S3 is as follows:

[0083] Step S3: Loss minimization based on Wasserstein distance and contrastive domain difference

[0084] The feature vectors of the source domain and the target domain are obtained from the fully connected layer output, and the Wasserstein distance between them is calculated. One advantage of the Wasserstein distance compared to the KL divergence and the JS divergence is that even if the sample distribution of the source domain and the target domain overlaps very little, it can reflect the distance between the two domains. When there is no overlap or the overlap part is negligible between the two distributions, the KL divergence may be meaningless, and the JS divergence is a constant. This means that the gradient is 0 for the gradient descent method, while the Wasserstein distance is smooth, which can provide more stable gradients.

[0085] Since the lower bound is not easy to directly obtain the result, based on the Kantorovich-Rubenstein binary, the Wasserstein distance can be written in the form of a dual, as shown in equation (2):

[0086]

[0087] Where sup represents the supremum, and respectively represent the edge probability distribution X S and X T The lower source domain sample D S and the target domain sample D T The expected value of the function g(x) is ‖g‖ L Lipschitz function, its definition is shown in equation (3):

[0088] ||g|| L =sup|g(x1)-g(x2)| / |x1-x2| (3)

[0089] By minimizing the Wasserstein distance loss, the source domain sample closest to the target domain sample is selected, and then the contrastive domain difference is used for domain adaptation to perform class-aware alignment. The intra-class and inter-class differences are jointly optimized to improve the adaptive performance.

[0090] Since the contrastive domain discrepancy is the maximum mean discrepancy after adding the class condition, and the maximum mean discrepancy is determined by the average embedding of the reproducing kernel Hilbert space distribution, when the data volume is large, sufficient statistical calculation is less likely to be affected by label noise.

[0091] In the training process, in addition to minimizing the cross-entropy loss on the labeled source domain samples, the domain adaptive network also estimates the underlying label assumption of the target domain samples by clustering. After clustering, when estimating the contrastive domain discrepancy, the ambiguous target domain data far from the cluster center and the ambiguous class containing very few target samples around the cluster center are set to zero. As the training proceeds, more and more classes are included, and the model becomes more and more accurate. During the iteration process, class-aware sampling is used for the source domain and the target domain, that is, data is sampled from the two domains for each class in the randomly sampled class subset, so as to improve the training efficiency.

[0092] Wasserstein distance and contrastive domain discrepancy D cdd The regularization term as the loss function is introduced into the classification network, and the total target function is obtained as shown in equation (4):

[0093]

[0094] Wherein, l c is the cross-entropy loss obtained by training the source domain, and λ and β are weight parameters for balancing the two parts of the loss of the Wasserstein distance and the contrastive domain discrepancy.

[0095] The feature representation is adjusted by back propagation to perform the network optimization process, the epoch is set to 50, the learning rate learningrate is set to 0.0001, the momentum momentum is set to 0.9, and the decay rate decay is set to 0.0001. The AUC value of the model on the validation set is calculated for each epoch, and whether the current training model is the best model is determined according to the performance of the validation set.

[0096] The specific implementation of step 4 is as follows:

[0097] Step 4: Model verification for chest radiograph prediction

[0098] In the model verification mode, the best model parameters obtained by the above step training are loaded, and the X-ray chest radiograph image to be tested is standardized and then subjected to classification prediction task. The predicted label is used to calculate the confusion matrix TP / TN / FP / FN and AUC, so as to evaluate the classification effect of the model. The AUC is the area under the ROC curve, and the horizontal coordinate of the ROC curve is FPR (False Positive Rate), and its calculation formula is shown in equation (5):

[0099]

[0100] The vertical coordinate of the ROC curve is TPR (True Positive Rate), and the calculation formula is shown as formula (6):

[0101]

[0102] The closer the AUC value is to 1, the better the classification performance of the model. After verification, the average AUC value of the five types of lesions of the traditional domain adaptation method using the convolutional neural network (ResNet50) as the chest X-ray abnormality recognition model is 0.7492, and the average AUC value obtained by the domain adaptation model trained by the method of the application for prediction on the target domain is 0.8188, which is 8.5% higher than the former. The model verification result of the method of the application is shown in the accompanying Figure 3 It can be seen that compared with only using ResNet50 as the chest X-ray abnormality recognition model, the prediction performance and generalization performance of the model trained by the application are more outstanding.

[0103] The application also provides a chest X-ray abnormality recognition domain adaptation system combining Wasserstein distance and difference measurement, comprising:

[0104] A data preparation and preprocessing module is configured to obtain a CT image of a chest X-ray film and perform data preprocessing on the CT image.

[0105] A feature extraction module is configured to input the CT image into a Patch partition module for partitioning, set every 4x4=16 adjacent pixels as a Patch, flatten in the channel direction, perform linear transformation on the channel data of each pixel through a linear embedding layer, and then sequentially pass through four stages to construct feature maps of different sizes.

[0106] A loss minimization module is configured to select source domain samples closest to target domain samples and calculate Wasserstein distance, perform cross-domain similar-to-similar and dissimilar-to-dissimilar processing on the target domain samples to obtain a contrast domain difference, construct a total objective function through the obtained Wasserstein distance and the contrast domain difference, and optimize and update parameters of a Swin Transformer network according to the total objective function.

[0107] A model verification module is configured to verify the Swin Transformer network after optimization and parameter update, and perform a chest X-ray classification prediction task.

[0108] The application extracts deep feature representation of data samples by using Swin Transformer; on the basis of selecting source domain samples closest to target domain samples, cross-domain similar and dissimilar samples are realized, the classification precision of chest radiographs is improved, and better generalization ability is obtained; the application combines the respective advantages of contrast domain difference and Wasserstein distance, can not only select source domain samples closest to target domain samples, but also can shorten the distance of the same categories in the feature space between target domain and source domain samples, and can widen the distance of different categories; the method uses Swin Transformer as a feature extractor, uses the multi-scale features extracted therefrom for chest radiograph classification task, and the hierarchical architecture of Swin Transformer can flexibly model different scales, effectively improve the receptive field, and capture more information conducive to the chest radiograph classification task.

[0109] The above only describes the preferred embodiments and principles of the application in detail, and for ordinary skilled persons in the art, the specific implementation manner can be changed according to the idea provided by the application, and these changes should be regarded as the protection scope of the application.

Claims

1. A chest radiograph abnormality recognition domain adaptation method combining Wasserstein distance and difference measure, characterized in that, It comprises the following steps: S1, chest radiograph data preparation and preprocessing: Obtain the CT image of the chest X-ray film, and perform data preprocessing on the CT image; S2, multi-scale feature extraction based on Swin Transformer: Input the CT image into the Patch partition module for blocking, set every 4x4=16 adjacent pixels as a Patch, and flatten in the channel direction, linearly transform the channel data of each pixel through a linear embedding layer, and then sequentially pass through four stages to construct feature maps of different sizes; S3, loss minimization based on Wasserstein distance and contrastive domain difference: Select the source domain sample closest to the target domain sample and calculate the Wasserstein distance; perform cross-domain similar-to-similar and dissimilar-to-dissimilar processing on the target domain sample to obtain the contrastive domain difference; construct the total objective function through the obtained Wasserstein distance and contrastive domain difference; optimize and update the Swin Transformer network according to the total objective function, comprising the following steps: In the iterative process of training, first fix the network parameters φ, and update the sample clustering of the target domain. After clustering is completed, each target domain sample is assigned a label corresponding to the cluster to which each target domain sample belongs. Then the updated target domain label is used to calculate the contrastive domain difference. The Wasserstein distance between the two domain samples is calculated to determine the data samples participating in the calculation of the contrastive domain difference. Finally, the network parameters are updated by minimizing the contrastive domain difference to complete the iteration. S4, model verification for chest radiograph prediction: Verify the Swin Transformer network after optimization and parameter update, and perform classification prediction task for chest radiograph.

2. The chest radiograph abnormality recognition domain adaptation method combining Wasserstein distance and difference metrics according to claim 1, characterized in that, The data preprocessing in step S1 includes standardization and data enhancement; the standardization is to normalize using the mean and standard deviation of the image; and the data enhancement is to perform data enhancement on the image by random rotation and horizontal flipping.

3. The chest radiograph abnormality recognition domain adaptation method combining Wasserstein distance and difference metrics of claim 1, wherein, The selection of the source domain sample closest to the target domain sample and the calculation of the Wasserstein distance in step S3 comprise the following steps: S31, define the source domain dataset as D S , the source domain data samples obey the probability distribution D S ~ X S ; the target domain dataset is D T , the target domain data samples obey the probability distribution D T ~ X T , the distribution difference between the source domain and the target domain is measured by calculating the Wasserstein distance; The Wasserstein distance is defined as shown in formula (1): where ∏(X S ,X T ) denotes the set of all possible joint distributions of the source domain distribution X S and the target domain distribution X T combined, γ ~ ∏(X S ,X T ) denotes the cost of transforming X S to X T so that both obey the same distribution; for each possible joint distribution γ, one can obtain a sample p and q from sampling (p, q) ~ γ, and compute the distance between the sample p and q, ||p - q||, to obtain the expected value of the sample pair distance under the corresponding joint distribution γ, E (p,q)～γ [‖p - q‖], and the smaller the expected value, the smaller the cost of transforming X S to X T .

4. The chest radiograph abnormality recognition domain adaptation method of claim 3, wherein, The cross-domain similar-to-similar and dissimilar-to-dissimilar processing of the target domain sample in step S3 to obtain the contrastive domain difference comprises the following steps: S32, estimate the label hypothesis of the target domain sample by clustering. In the clustering process, the contrastive domain difference is introduced, which is to calculate the difference between the specified classes c and c'. Minimize the difference between the same classes to compress the feature representation of the intra-class samples, and maximize the difference between the different classes to further push the feature representation of the inter-class samples away from the decision boundary; The contrastive domain difference is defined as shown in formula (2): where M is the 5th class, n t is the target domain dataset size, is the set of target domain class labels, D cc′ is the average embedding estimate of class c and class c' in the reproducing kernel Hilbert space; minimizing the contrastive domain discrepancy D cdd so that the former part of the formula is as small as possible, and the latter part is as large as possible, that is, for the five types of lesions of the chest film, the sample distribution of the same class c in the target domain and the source domain is pulled close, and the sample distribution of a certain class c in the target domain and all different classes c' in the source domain is pulled apart; φ is the network parameter.

5. The chest radiograph abnormality recognition domain adaptation method of claim 4, wherein, The construction of the total objective function through the obtained Wasserstein distance and contrastive domain difference in step S3 comprises the following steps: S33, obtaining the Wasserstein distance in step S31 and the contrast field difference D in step S32 cdd The regular term as the loss function is introduced into the classification network, and the total objective function is obtained as shown in equation (3): wherein, l c is the cross-entropy loss obtained by training the source domain, and λ and β are weight parameters for balancing the Wasserstein distance and the contrast domain difference, respectively. The loss obtained by the loss formula is used to adjust the model parameters until the model converges.

6. The chest radiograph abnormality recognition domain adaptation method of joint Wasserstein distance and discrepancy measure according to claim 1, characterized in that, Step S4 comprises the following steps: S41, in the verification mode, only standardization processing is performed on the data to be verified; S42, respectively calculate the AUC, accuracy, sensitivity, specificity, positive predictive rate and negative predictive rate of each category of chest radiograph, for further validation of the model.

7. The chest radiograph abnormality recognition domain adaptation method of joint Wasserstein distance and discrepancy measure according to claim 1, characterized in that, The step S2 further comprises the following steps: The picture input size is set to 224x224, ensuring that the output feature matrix size after four-stage down-sampling of 32 times by Swin Transformer is 7x7, which corresponds to the window size of 7x7 in the fourth stage; The Swin-T model in the Swin Transformer architecture is selected, the number of layers of the Swin-T model is {2, 2, 6, 2}, and the channel depth of the hidden layer in the first stage is 96.

8. A system for domain adaptation of chest radiograph abnormality recognition using joint Wasserstein distance and discrepancy measure, for implementing the method of domain adaptation of chest radiograph abnormality recognition using joint Wasserstein distance and discrepancy measure according to any one of claims 1 to 7, characterized in that, It comprises: A data preparation and preprocessing module for obtaining CT images of chest X-ray films and preprocessing the CT images; A feature extraction module for inputting the CT images into a Patch partition module for partitioning, setting every 4x4=16 adjacent pixels as a Patch, flattening in the channel direction, performing linear transformation on the channel data of each pixel through a linear embedding layer, and then constructing feature maps of different sizes through four stages; A loss minimization module for selecting source domain samples closest to target domain samples and calculating the Wasserstein distance; performing cross-domain similar-to-similar and different-to-different processing on the target domain samples to obtain the contrast domain difference; constructing a total objective function through the obtained Wasserstein distance and the contrast domain difference; and optimizing and updating the parameters of the Swin Transformer network according to the total objective function; A model verification module for verifying the Swin Transformer network after optimization and parameter update, and performing chest radiograph classification prediction task.

Images