Cross-domain few-sample learning method for chest X-ray image recognition

Abstract

The invention discloses a cross-domain few-sample learning method for chest X-ray image recognition, and the method comprises the following key technologies: X-ray field enhancement (XDA): converting a sample in a source field into an enhanced sample similar to an X-ray image field through a series of image processing technologies; image edges are highlighted by applying Canny edge detection; a Laplacian filter is used to enhance the marginal definition; the distinction degree of different tissues and structures in the image is improved through a contrast enhancement technology; and adding a fuzzy effect to simulate artifacts in X-ray imaging. And CDAT (Cross-Domain Alternating Training): alternately training between a basic domain sample and an enhanced domain sample, so that the model can adapt to two different image fields. According to the invention, the accuracy of chest X-ray image recognition is effectively improved. When a rare disease cannot be diagnosed, a doctor is helped to quickly and accurately judge the disease from a limited sample. And an innovative solution is provided for cross-domain challenges in medical image analysis.

Description

Technical Field

[0001] The invention relates to the field of computer vision and multimedia analysis, and in particular to a small sample classification method for processing medical X-ray chest images. Background Art

[0002] With the rapid development of deep learning technology in the field of medical image analysis, it has shown great potential in disease diagnosis, treatment planning, and prognosis assessment. However, the performance of deep learning models is often limited by the availability of high-quality annotated images. In the field of medical images, it is challenging to obtain a large number of accurately annotated samples because different imaging technologies produce diverse image data, and professional annotation is costly, time-consuming, and resource-intensive.

[0003] Traditional few-shot learning (FSL) methods aim to enable models to learn from limited samples and quickly adapt to new tasks, which is particularly important for medical image analysis because the emergence of new diseases and imaging techniques requires models to adapt quickly. However, most existing FSL models operate within a single data domain, while medical imaging involves multiple different domains with significant variations, which is a major challenge facing FSL models.

[0004] In addition, medical image datasets not only have small sample sizes and limited distribution, but also have an imbalanced distribution of positive and negative samples, which limits the generalization ability of the model. These challenges make it difficult to develop efficient and stable models for a variety of medical image analysis tasks, and the scarcity and cost of professionally annotated data further exacerbate this challenge.

[0005] In order to overcome the above obstacles, the present invention introduces the Cross-Domain Few-shot Medical Image Classification (CD-FSMIC) task. The main challenge of CD-FSMIC is how to effectively migrate the model trained in the source domain to the target domain with different distributions for the few-shot classification task. In view of the scarcity and high cost of labeled medical imaging data in clinical reality, as well as the large amount of data domain differences introduced by different imaging technologies, CD-FSMIC is of great significance in the field of medical image analysis. Summary of the invention

[0006] Based on the above practical needs and key issues, the purpose of the present invention is to solve the problem of limited performance of deep learning models due to insufficient annotated images in the field of medical image analysis, especially in the cross-domain few-sample learning scenario of chest X-ray image recognition. In order to overcome the field differences caused by different imaging technologies, the present invention proposes an innovative cross-domain few-sample learning method, which specifically includes the following key technologies:

[0007] Step S100, performing random data enhancement on samples using a data enhancement method to improve the diversity of samples;

[0008] Step S200, using the data enhancement method XDA to perform data enhancement on the enhanced sample again, so as to simulate the X-ray imaging field and generate a large number of pictures similar to the X-ray image domain;

[0009] Step S300, performing basic training on the training set of the base domain to train the initial performance of the model;

[0010] Step S400, performing enhancement training on the training set of the enhancement domain to further improve the adaptability of the model to the X-ray image domain;

[0011] Step S500 uses cross-domain alternating training (CDAT) to alternately train the model in steps S300 and S400. This is used to improve the adaptability and generalization ability of the model to the field of X-ray images while reducing the impact of overfitting;

[0012] Step S600, after training on the training set, small sample cross-domain classification is performed on the X-ray image domain to finally obtain the classification result;

[0013] As an improvement, for the data enhancement method, the samples in the base domain are converted into enhanced samples similar to those in the X-ray image field through image processing technology, including the following steps:

[0014] Step S201, for each sample x from the base domain training set, first grayscale, the formula is as follows:

[0015] g=Gray(x)=0.2989×x R +0.5870×x G +0.1140×x B ,

[0016] Among them, x R ,x G ,x B Represent the red, green and blue channels of the image respectively.

[0017] Step S202, applying Canny edge detection to the grayscale image g to obtain an edge detection image e, the formula is as follows:

[0018] e=Edge(g),

[0019] Where Edge(·) represents Canny edge detection. Specific Canny edge detection includes Gaussian filtering, gradient calculation, non-maximum suppression and double threshold detection. Gaussian filtering is used to smooth the image to reduce noise; gradient calculation is used to determine the edge strength and direction of each pixel in the image; non-maximum suppression is used to retain the local maximum, that is, the actual edge; double threshold detection is used to determine strong edges and weak edges for subsequent edge tracking.

[0020] Step S203, perform Laplacian filtering on the edge detection image e to further enhance the edge information in the image, and obtain a Laplacian filtered image l. The formula is as follows:

[0021] l=Laplacian(e),

[0022] Where Laplacian(·) represents the Laplacian filter, which is a second-order derivative operator used to enhance the edges and details in an image. It highlights the rapidly changing areas in the image by calculating the second-order derivative of the image brightness, thereby enhancing the edge information.

[0023] Step S204, performing histogram equalization on the grayscale image g to obtain a high-contrast image c with high-contrast features, the formula is as follows:

[0024] c=Contrast(g),

[0025] Contrast(·) represents the histogram equalization technology, which enhances the global contrast by adjusting the histogram distribution of the image, making the brightness distribution of the image more uniform and enhancing the details and edges in the image.

[0026] Step S205: weighted superposition of the high contrast image c and the Laplacian filtered image l to obtain a superimposed image with high contrast features and enhanced edge features. The formula is as follows:

[0027]

[0028] Step S206, when superimposing the image Based on , dynamic blur processing is performed to obtain the blurred image b. The formula is as follows:

[0029]

[0030] Blur(·) represents the operation of applying a blur filter. Motion blur is achieved by converting the enhanced three-channel image to PIL format and processing it using the ImageFilter.BLUR filter in the python library. This operation simulates the artifact effect in X-ray imaging, which can reduce the clarity of image details, thereby producing a blur effect and enhancing the overall visual characteristics of the image.

[0031] Step S207: Overlay the image Superimpose with blurred image b to get the final enhanced image The formula is as follows:

[0032]

[0033] Final enhanced image The image features include high contrast, prominent edges and overall motion blur, thus approaching the X-ray image domain.

[0034] As an improvement, for the cross-domain alternating training (CDAT) method, the method includes the following steps:

[0035] Step S300, in the base domain D base Basic training is performed on the CNN to train the initial performance of the model and ensure that the model performance decreases; specifically:

[0036] Step S301, first in the base domain D base Sufficient rounds of pre-training are performed on the dataset to ensure that the model has good performance in the first place.

[0037] Step S302, in each CDAT training round, only fine-tune the model without training a large number of rounds. This step is only for a single round in the base domain D base Conduct basic training on the

[0038] Step S400: the model is in the enhanced domain D of the data after XDA processing. Aug base A single round of enhanced training is performed on the CNN to further improve the model's adaptability to the X-ray image domain;

[0039] In step S500, the training of step S302 and step S400 are performed alternately, so that the model can adapt to the X-ray image domain without causing the performance of the model to degrade too much, and the influence of overfitting is reduced.

[0040] As an improvement, the following steps are also included:

[0041] Step S601: From the target domain D target The support set S is sampled from the target domain D, which contains N categories and K samples for each category.target The sampled query set Q contains N categories, which are the same as the query set S, and each category has 15 samples.

[0042] Step S602, using the model trained in step S500 to classify the query set Q. Specifically, after the query set Q and the support set S are processed by the model, the model calculates a prototype for each category on the support set S, which is used to classify the sample features in the query set Q for cosine similarity measurement classification.

[0043] Beneficial effects of the present invention:

[0044] 1. X-ray Domain Enhancement (XDA): This paper proposes a data enhancement strategy that converts samples in the source domain into enhanced samples similar to the X-ray image domain through a series of image processing techniques. This process includes converting color images into grayscale images, applying Canny edge detection to highlight image edges, using Laplacian filters to enhance edge clarity, using contrast enhancement techniques to improve the distinction between different tissues and structures in the image, and adding blur effects to simulate artifacts in X-ray imaging;

[0045] 2. Cross-domain alternating training (CDAT): This paper proposes an alternating training method, which alternates between base domain and enhanced domain samples, so that the model can adapt to two different image domains. This training strategy includes two stages: base training and enhanced training. In the base training stage, the model is trained on the original base domain data; in the enhanced training stage, the model is trained on the data processed by XDA. Through this alternating training, the model can learn to extract features from different image domains while reducing the impact of overfitting.

[0046] 3. Experimental verification: The experimental results show that the accuracy of chest X-ray image recognition is effectively improved and an innovative solution is provided for cross-domain challenges in medical image analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flow chart of Embodiment 1 of the cross-domain small sample chest X-ray classification method of the present invention;

[0048] Figure 2 This is a flow chart of Embodiment 2 of the cross-domain small sample chest X-ray classification method of the present invention; DETAILED DESCRIPTION

[0049] See also Figure 1 The flow chart of the first embodiment of the cross-domain small sample X-ray chest film classification method of the present invention. The model training process of the present invention generally includes the following steps:

[0050] Step S100, processing the original samples to diversify the samples and reduce model overfitting, specifically includes the following steps:

[0051] Step S101: AutoContrast: automatically adjust the contrast of the image.

[0052] Step S102: Brightness: adjust the brightness of the image.

[0053] Step S103: Color: adjust the color intensity of the image.

[0054] Step S104: Contrast: adjust the contrast of the image.

[0055] Step S105: Cutout: Randomly select an area on the image and replace it with gray.

[0056] Step S106: CutoutAbs: Similar to Cutout, but allows the size of the area to be cut to be specified.

[0057] Step S107: Equalize: Apply histogram equalization to enhance the details of low-contrast areas in the image.

[0058] Step S108: Identity: Do not change the image, use it as a baseline.

[0059] Step S109: Invert: invert the color of the image.

[0060] Step S110: Posterize: Reduce the tones of the image to a specified amount.

[0061] Step S111: Rotate: randomly rotate the image.

[0062] Step S112: Sharpness: adjust the sharpness of the image.

[0063] Step S113: ShearX: shear the image along the X axis.

[0064] Step S114: ShearY: shear the image along the Y axis.

[0065] Step S115: Solarize: flip the pixel values ​​of the image to produce an overexposed effect.

[0066] Step S116: SolarizeAdd: similar to Solarize, but with an additional threshold parameter.

[0067] Step S117: TranslateX: translate the image along the X axis.

[0068] Step S118: TranslateY: translate the image along the Y axis.

[0069] For each selected step, a value v (between 1 and self.m) is randomly selected, and then the operation is applied with a probability of 50%. Finally, regardless of whether the previous operation is applied, step S106: CutoutAbs operation is applied. Each enhancement operation randomly selects multiple operations for combination. This random combination can increase the diversity of data and help improve the generalization ability of the model.

[0070] Step S200: Perform XDA enhancement on samples in the base domain so that the enhanced domain is close to the X-ray image domain.

[0071] Step S500: Using the cross-domain alternating training technology, the model is trained alternately in the base domain and the enhanced domain. Figure 1 As shown, in each round, the model is first fine-tuned on the base domain and then fine-tuned on the enhanced domain. After a small number of rounds of training, the model can be fine-tuned.

[0072] Specifically:

[0073] Step S301: In the basic domain D base Perform enough rounds of basic pre-training on the training model to train the initial performance of the model and ensure that the model performance decreases;

[0074] Step S302, in each CDAT training round, only fine-tune the model without training a large number of rounds. This step is only for a single round in the base domain D base Conduct basic training on the

[0075] Step S400: the model is in the enhanced domain D of the data after XDA processing. Aug base Enhanced training is performed on the X-ray image domain to further improve the model's adaptability to the X-ray image domain;

[0076] In step S500, the training of step S302 and step S400 are performed alternately, so that the model can adapt to the X-ray image domain without causing the performance of the model to degrade too much, and the influence of overfitting is reduced.

[0077] Step S600: performing cross-domain small sample classification in the target domain.

[0078] Specifically:

[0079] Step S601: From the target domain D target The support set S is sampled from the target domain D, which contains N categories and K samples for each category. targetThe sampled query set Q contains N categories, which are the same as the query set S, and each category has M samples.

[0080] Step S602, using the model trained in step S200 to classify the query set Q. Specifically, after the query set Q and the support set S are processed by the model, the model calculates a prototype for each category on the support set S, which is used to classify the sample features in the query set Q and classify them using the cosine similarity metric. The final classification result is obtained based on the cosine similarity.

[0081] See also Figure 2 This is a flowchart of Example 2 of the cross-domain small sample X-ray chest film classification method of the present invention.

[0082] Specifically, the samples in the source domain are converted into enhanced samples similar to those in the X-ray image domain through image processing technology, including the following steps:

[0083] Step S101, for each sample x from the base domain training set, first grayscale, the formula is as follows:

[0084] g=Gray(x)=0.2989×x R +0.5870×x G +0.1140×x B ,

[0085] Among them, x R ,x G ,x B Represent the red, green and blue channels of the image respectively.

[0086] Step S102, applying Canny edge detection to the grayscale image g to obtain an edge detection image e, the formula is as follows:

[0087]

[0088] Where Edge(·) represents Canny edge detection. Specific Canny edge detection includes Gaussian filtering, gradient calculation, non-maximum suppression and double threshold detection. Gaussian filtering is used to smooth the image to reduce noise; gradient calculation is used to determine the edge strength and direction of each pixel in the image; non-maximum suppression is used to retain the local maximum, that is, the actual edge; double threshold detection is used to determine strong edges and weak edges for subsequent edge tracking.

[0089] Step S103, perform Laplacian filtering on the edge detection image e to further enhance the edge information in the image and obtain a Laplacian filtered image l. The formula is as follows:

[0090] l=Laplacian(e),

[0091] Where Laplacian(·) represents the Laplacian filter, which is a second-order derivative operator used to enhance the edges and details in an image. It highlights the rapidly changing areas in the image by calculating the second-order derivative of the image brightness, thereby enhancing the edge information.

[0092] Step S104, performing histogram equalization on the grayscale image g to obtain a high-contrast image c with high-contrast features, the formula is as follows:

[0093] c=Contrast(g),

[0094] Contrast(·) represents the histogram equalization technology, which enhances the global contrast by adjusting the histogram distribution of the image, making the brightness distribution of the image more uniform and enhancing the details and edges in the image.

[0095] Step S105: weighted superposition of the high contrast image c and the Laplacian filtered image l to obtain a superimposed image with high contrast features and enhanced edge features. The formula is as follows:

[0096]

[0097] Step S106, superimposing the image Based on , dynamic blur processing is performed to obtain the blurred image b. The formula is as follows:

[0098]

[0099] Blur(·) represents the operation of applying a blur filter. Motion blur is achieved by converting the enhanced three-channel image to PIL format and processing it using the ImageFilter.BLUR filter in the python library. This operation simulates the artifact effect in X-ray imaging, which can reduce the clarity of image details, thereby producing a blur effect and enhancing the overall visual characteristics of the image.

[0100] Step S107: Overlay the image Superimpose with blurred image b to get the final enhanced image The formula is as follows:

[0101]

[0102] Final enhanced image The image features include high contrast, prominent edges and overall motion blur, thus approaching the X-ray image domain.

[0103] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A cross-domain small sample learning method for chest X-ray image recognition, the method comprising the following steps: Step S100, performing random data enhancement on samples using a data enhancement method to improve the diversity of samples; Step S200, using the data enhancement method XDA to perform data enhancement on the enhanced sample again, so as to simulate the X-ray imaging field and generate a large number of pictures similar to the X-ray image domain; Step S300, performing basic training on the training set of the base domain to train the initial performance of the model; Step S400, performing enhancement training on the training set of the enhancement domain to further improve the adaptability of the model to the X-ray image domain; Step S500 uses cross-domain alternating training (CDAT) to alternately train the model in steps S300 and S400. This is used to improve the adaptability and generalization ability of the model to the field of X-ray images while reducing the impact of overfitting; Step S600, after training on the training set, small sample cross-domain classification is performed in the X-ray image domain to finally obtain the classification result.

2. A data enhancement method according to claim 1, wherein the base domain samples are converted into enhanced samples similar to those in the X-ray image domain by image processing technology, comprising the following steps: Step S201, for each sample x from the base domain training set, first grayscale, the formula is as follows: g=Gray(x)=0.2989×x R +0.5870×x G +0.1140×x B , Among them, x R ,x G ,x B Represent the red, green and blue channels of the image respectively. Step S202, applying Canny edge detection to the grayscale image g to obtain an edge detection image e, the formula is as follows: e=Edge(g), Where Edge(·) represents Canny edge detection. Specific Canny edge detection includes Gaussian filtering, gradient calculation, non-maximum suppression and double threshold detection. Gaussian filtering is used to smooth the image to reduce noise; gradient calculation is used to determine the edge strength and direction of each pixel in the image; non-maximum suppression is used to retain the local maximum, that is, the actual edge; double threshold detection is used to determine strong edges and weak edges for subsequent edge tracking. Step S203, perform Laplacian filtering on the edge detection image e to further enhance the edge information in the image, and obtain a Laplacian filtered image l. The formula is as follows: l=Laplacian(e), Where Laplacian(·) represents the Laplacian filter, which is a second-order derivative operator used to enhance the edges and details in an image. It highlights the rapidly changing areas in the image by calculating the second-order derivative of the image brightness, thereby enhancing the edge information. Step S204, performing histogram equalization on the grayscale image g to obtain a high-contrast image c with high-contrast features, the formula is as follows: c=Contrast(g), Contrast(·) represents the histogram equalization technology, which enhances the global contrast by adjusting the histogram distribution of the image, making the brightness distribution of the image more uniform and enhancing the details and edges in the image. Step S205: weighted superposition of the high contrast image c and the Laplacian filtered image l to obtain a superimposed image with high contrast features and enhanced edge features. The formula is as follows: Step S206, superimposing the image Based on , dynamic blur processing is performed to obtain the blurred image b. The formula is as follows: Blur(·) represents the operation of applying a blur filter. Motion blur is achieved by converting the enhanced three-channel image to PIL format and processing it using the ImageFilter.BLUR filter in the python library. This operation simulates the artifact effect in X-ray imaging, which can reduce the clarity of image details, thereby producing a blur effect and enhancing the overall visual characteristics of the image. Step S207: Overlay the image Superimpose with blurred image b to get the final enhanced image The formula is as follows: Final enhanced image The image features include high contrast, prominent edges and overall motion blur, thus approaching the X-ray image domain.

3. The cross-domain alternating training (CDAT) method according to claim 1, comprising the following steps: Step S300, in the base domain D base Basic training is performed on the CNN to train the initial performance of the model and ensure that the model performance decreases; specifically: Step S301, first in the base domain D base Perform enough rounds of pre-training on the dataset to make the model have good performance in the first place. Step S302, in each CDAT training round, only fine-tuning the model is performed without a large number of rounds of training. This step is only for a single round in the base domain D base Conduct basic training on the Step S400: the model is in the enhanced domain D of the data after XDA processing. Augbase A single round of enhanced training is performed on the CNN to further improve the model's adaptability to the X-ray image domain; In step S500, the training of step S302 and step S400 are performed alternately, so that the model can adapt to the X-ray image domain without causing the performance of the model to degrade too much, and the influence of overfitting is reduced.

4. The cross-domain few-sample learning method for chest X-ray image recognition according to claim 1 further comprises the following steps: Step S601: From the target domain D target The support set S is sampled from the target domain D, which contains N categories and K samples for each category. target The sampled query set Q contains N categories, which are the same as the query set S, and each category has 15 samples. Step S602, using the model trained in step S500 to classify the query set Q. Specifically, after the query set Q and the support set S are processed by the model, the model calculates a prototype for each category on the support set S, which is used to classify the sample features in the query set Q for cosine similarity measurement classification.

Images