A method for auxiliary judgment of abnormal shadows in chest X-ray

Abstract

The present invention proposes a method for assisting in judging abnormal shadows in chest X-rays, and relates to the technical field of image enhancement. The present invention proposes an assisting process for judging abnormal shadows in chest X-rays, including making a chest X-ray data set, constructing a spatial noise reduction module SD, constructing a grayscale generation module GGB, constructing a coarse image generation module CCB, constructing a noise reduction processing module DHB, constructing a second-order refinement module TRF, constructing a chest X-ray image enhancement module, constructing a chest X-ray abnormal shadow detection model, and training the chest X-ray abnormal shadow detection model and detecting; at the same time, a chest X-ray image enhancement module is proposed, wherein the spatial noise reduction module SD can suppress noise and predict a noise-free grayscale image, the noise reduction processing module DHB can eliminate noise, and the second-order refinement module TRF further refines the chest X-ray image features, and finally generates an enhanced chest X-ray image with rich colors and details.

Description

Technical Field

[0001] The invention belongs to the technical field of image enhancement, and in particular relates to a method for auxiliary judgment of abnormal shadows in chest X-rays. Background Art

[0002] Visual technology can assist in determining abnormal shadows on chest X-rays, and can quickly and accurately identify and analyze subtle abnormalities in X-ray images, avoiding fatigue and errors that may occur in manual interpretation. Through advanced image processing and machine learning algorithms, visual technology can automatically mark suspected lesion areas and give preliminary diagnostic suggestions, greatly improving diagnostic efficiency and accuracy, and providing patients with faster and more reliable medical services.

[0003] Image enhancement technology can significantly improve the quality and visual effects of images, making the targets in the image clearer and easier to identify by increasing contrast, adding details, and reducing noise. In visual inspection, image enhancement technology can help machines better capture and understand image information, thereby more accurately detecting and judging targets and reducing false detection and missed detection rates. The application of this technology not only improves the efficiency and accuracy of visual inspection, but also provides strong support for the development of automated and intelligent inspection systems. Summary of the invention

[0004] The present invention provides a chest X-ray abnormal shadow auxiliary judgment method, aiming to propose a chest X-ray image enhancement module, wherein the spatial noise reduction module SD uses a low-rank representation of low-light features to suppress noise and predict a noise-free grayscale image, while the noise reduction processing module DHB converts the YCbCr color space and replaces the noisy coarse image Y channel with the predicted noise-free grayscale image to eliminate noise, and the second-order refinement module TRF further refines the chest X-ray image features, and finally generates an enhanced chest X-ray image with rich colors and details.

[0005] The present invention aims to propose a chest X-ray image enhancement module and provide a chest X-ray abnormal shadow auxiliary judgment method, comprising the following steps:

[0006] S1. Chest X-ray dataset preparation: collect multiple chest X-ray images containing multiple abnormal shadows, annotate the multiple chest X-ray images, and annotate the targets as multiple abnormal shadows. After the annotation is completed, a chest X-ray dataset is obtained;

[0007] S2, construct a spatial noise reduction module SD, which includes 1×1 convolution, 3×3 convolution, transposition, dimension transformation, matrix multiplication and element-by-element addition;

[0008] S3, build the grayscale generation module GGB, which includes 1×1 convolution, 3×3 convolution, spatial attention, element-by-element addition and spatial noise reduction module SD;

[0009] S4, build a coarse image generation module CCB, including 3×3 convolution, channel attention, element-by-element addition and noisy coarse image calculation;

[0010] S5, constructing a noise reduction processing module DHB, including color space transformation;

[0011] S6, construct the second-order refinement module TRF, which includes 3×3 convolution and element-wise addition;

[0012] S7, constructing a chest X-ray image enhancement module, including a grayscale generation module GGB, a coarse image generation module CCB, a noise reduction processing module DHB and a second-order refinement module TRF;

[0013] S8, constructing a chest X-ray abnormal shadow detection model, which includes an input, a chest X-ray image enhancement module, a backbone network, a detection head, and an output in sequence;

[0014] S9. Train a chest X-ray abnormal shadow detection model and perform detection. Use a chest X-ray data set to train the chest X-ray abnormal shadow detection model. After the training is completed, input a new chest X-ray image to be detected into the chest X-ray abnormal shadow detection model to obtain an abnormal shadow detection result.

[0015] Preferably, in step S1, the multiple abnormal shadows include point shadows, spot shadows, miliary shadows, nodular shadows, block shadows, patchy shadows, stripe shadows, band shadows, ring shadows and diffuse shadows.

[0016] Preferably, in step S2, for the spatial noise reduction module SD, the input feature , , H, W and C are characteristics The height, width and channels of Input to 3×3 convolution to get , ,Will Input to the first 1×1 convolution to get , , Not equal to C, Input to the second 1×1 convolution to get , ,Will Transform the dimension to get , , ,Will Input to the third 1×1 convolution to get , ,Will Input to the fourth 1×1 convolution to get , ,Will Transform and transpose to get , ,Will and Perform matrix multiplication to get , ,Will Transpose to get , ,Will and Perform matrix multiplication to get , ,Will Transform and transpose to get , ,Will and Add element by element to get , , is the output of the spatial noise reduction module SD, and The dimensions are consistent.

[0017] Preferably, in step S3, for the grayscale generation module GGB, the input image , , H, W and 3 are the input images The height, width and channels of Input to the first 1×1 convolution to get , ,Will Input to the first spatial attention to get , ,Will Input to the first spatial noise reduction module SD to get , ,Will Input to the first 3×3 convolution to get , ,Will Input to the second spatial attention to get , ,Will Input to the second spatial noise reduction module SD to get , ,Will Input to the second 3×3 convolution to get , ,Will Input to the third space attention to get , ,Will Input to the third spatial noise reduction module SD to get , ,Will Input to the third 3×3 convolution to get , ,Will Input to the fourth space attention to get , ,Will Input to the fourth spatial noise reduction module SD to get , ,Will Input to the fourth 3×3 convolution to get , ,Will and Add element by element to get , ,Will Input to the fifth 3×3 convolution to get , ,Will and Add element by element to get , ,Will Input to the sixth 3×3 convolution to get , ,Will and Add element by element to get the grayscale image , , It is the output of the grayscale generation module GGB.

[0018] Preferably, in step S4, for the coarse image generation module CCB, the input image , , H, W and 3 are the input images The height, width and channels of Input to the first 3×3 convolution to get , , ,Will Input to the first channel attention to get , ,Will Input to the second 3×3 convolution to get , , ,Will Input to the second channel attention to get , ,Will Input to the third 3×3 convolution to get , ,Will and Add element by element to get , ,Will Input to the fourth 3×3 convolution to get , ,Will and Add element by element to get , Then the output of the coarse image generation module CCB is obtained by calculating the noisy coarse image The calculation of the noisy coarse image is as follows: First, calculate the dark channel prior , , yes The three primary color channels, So A local block centered on yes Pixels in represents the minimum value calculation, and then calculates the bright channel prior , , Represents the maximum value calculation, and then through the filter The first 10% of pixels are used as atmospheric light , and then calculate the transmission map , , yes The three primary color channels are calculated and Get a noisy rough image , , Representatives in and 0.14, take the maximum value. .

[0019] Preferably, in step S5, for the noise reduction processing module DHB, the noisy rough image is input and grayscale , , ,Will Perform color space transformation to obtain , that is, convert from RGB to YCbCr, and use replace The Y channel in the image is used to obtain the denoised rough image. ,Will Perform color space conversion, that is, convert from YCbCr to RGB, to obtain the output of the noise reduction processing module DHB , It is a rough image with noise reduction. .

[0020] Preferably, in step S6, for the second-order refinement module TRF, the denoised coarse image is input , , H, W and 3 are the rough images of noise reduction The height, width and channels of Input to the first 3×3 convolution to get , , ,Will Input to the second 3×3 convolution to get , , ,Will Input to the third 3×3 convolution to get , , ,Will Input to the fourth 3×3 convolution to get , ,Will and Add element by element to get , ,Will Input to the fifth 3×3 convolution to get , ,Will and Add element by element to get , ,Will Input to the sixth 3×3 convolution to get , ,Will and Add element by element to get , , is the output of the second-order refinement module TRF.

[0021] Preferably, in step S7, for the chest X-ray image enhancement module, the chest X-ray image is input into the grayscale generation module GGB to obtain a grayscale image, the chest X-ray image is input into the coarse image generation module CCB to obtain a noisy coarse image, the grayscale image and the noisy coarse image are input into the noise reduction processing module DHB to obtain a denoised coarse image, and the denoised coarse image is input into the second-order refinement module TRF to obtain an enhanced chest X-ray image.

[0022] Compared with the prior art, the present invention has the following technical effects:

[0023] The technical solution provided by the present invention proposes a chest X-ray image enhancement module, wherein the spatial denoising module SD uses a low-rank representation of low-light features to suppress noise and predicts a noise-free grayscale image, while the denoising processing module DHB eliminates noise by converting the YCbCr color space and replacing the noisy coarse image Y channel with the predicted noise-free grayscale image, and the second-order refinement module TRF further refines the chest X-ray image features, and finally generates an enhanced chest X-ray image with rich colors and details. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The present invention provides a flow chart for detecting abnormal shadows in chest X-rays.

[0025] Figure 2 It is a structural diagram of the spatial noise reduction module SD provided by the present invention.

[0026] Figure 3 It is a structural diagram of the grayscale generation module GGB provided by the present invention.

[0027] Figure 4 It is a structural diagram of the coarse graph generation module CCB provided by the present invention.

[0028] Figure 5 It is a structural diagram of the noise reduction processing module DHB provided by the present invention.

[0029] Figure 6 It is a TRF structure diagram of the second-order refinement module provided by the present invention.

[0030] Figure 7 It is a structural diagram of the chest X-ray image enhancement module provided by the present invention. DETAILED DESCRIPTION

[0031] The present invention aims to propose a method for auxiliary judgment of abnormal shadows in chest X-rays, and proposes a chest X-ray image enhancement module, wherein the spatial denoising module SD uses a low-rank representation of low-light features to suppress noise and predicts a noise-free grayscale image, while the denoising processing module DHB eliminates noise by converting the YCbCr color space and replacing the noisy coarse image Y channel with the predicted noise-free grayscale image, and the second-order refinement module TRF further refines the chest X-ray image features, and finally generates an enhanced chest X-ray image with rich colors and details.

[0032] See also Figure 1 As shown, a chest X-ray abnormal shadow auxiliary judgment method in an embodiment of the present application:

[0033] S1. Chest X-ray dataset preparation: collect 200 chest X-ray images containing various abnormal shadows, annotate the 200 chest X-ray images, and annotate the targets as various abnormal shadows. After the annotation is completed, a chest X-ray dataset is obtained;

[0034] S2, build a spatial noise reduction module SD, which includes 4 1×1 convolutions, 1 3×3 convolution, 3 transpositions, 3 dimension transformations, 2 matrix multiplications and 1 element-by-element addition;

[0035] S3, build the grayscale generation module GGB, which includes 1 1×1 convolution, 6 3×3 convolutions, 4 spatial attentions, 3 element-by-element additions and 4 spatial noise reduction modules SD;

[0036] S4, build the coarse image generation module CCB, which includes 4 3×3 convolutions, 2 channel attentions, 2 element-by-element additions and 1 noisy coarse image calculation;

[0037] S5, constructing a noise reduction processing module DHB, including two color space transformations;

[0038] S6, construct the second-order refinement module TRF, which includes 5 3×3 convolutions and 3 element-wise additions;

[0039] S7, constructing a chest X-ray image enhancement module, including a grayscale generation module GGB, a coarse image generation module CCB, a noise reduction processing module DHB and a second-order refinement module TRF;

[0040] S8, building a chest X-ray abnormal shadow detection model, which includes input, chest X-ray image enhancement module, RetinaNet backbone network, RetinaNet detection head and output in sequence;

[0041] S9. Train a chest X-ray abnormal shadow detection model and perform detection. Use a chest X-ray dataset to train the chest X-ray abnormal shadow detection model. Divide the chest X-ray dataset into a training set and a validation set at a ratio of 7 to 3. The number of training rounds is 200. After the training is completed, input a new chest X-ray image to be detected into the chest X-ray abnormal shadow detection model to obtain the abnormal shadow detection result.

[0042] Further, in step S1, the various abnormal shadows include point shadows, spot shadows, miliary shadows, nodular shadows, block shadows, patchy shadows, stripe shadows, band shadows, ring shadows and diffuse shadows.

[0043] Further, in step S2, for the spatial noise reduction module SD, the input feature , , H, W and C are characteristics The height, width and channels of Input to 3×3 convolution to get , ,Will Input to the first 1×1 convolution to get , , Not equal to C, Input to the second 1×1 convolution to get , ,Will Transform the dimension to get , , ,Will Input to the third 1×1 convolution to get , ,Will Input to the fourth 1×1 convolution to get , ,Will Transform and transpose to get , ,Will and Perform matrix multiplication to get , ,Will Transpose to get , ,Will and Perform matrix multiplication to get , ,Will Transform and transpose to get , ,Will and Add element by element to get , , is the output of the spatial noise reduction module SD, and The dimensions are consistent.

[0044] Further, in step S3, for the grayscale generation module GGB, the input image , , 1280, 1280 and 3 are the input images The height, width and channels of Input to the first 1×1 convolution to get , ,Will Input to the first spatial attention to get , ,Will Input to the first spatial noise reduction module SD to get , ,Will Input to the first 3×3 convolution to get , ,Will Input to the second spatial attention to get , ,Will Input to the second spatial noise reduction module SD to get , ,Will Input to the second 3×3 convolution to get , ,Will Input to the third space attention to get , ,Will Input to the third spatial noise reduction module SD to get , ,Will Input to the third 3×3 convolution to get , ,Will Input to the fourth space attention to get , ,Will Input to the fourth spatial noise reduction module SD to get , ,Will Input to the fourth 3×3 convolution to get , ,Will and Add element by element to get , ,Will Input to the fifth 3×3 convolution to get , ,Will and Add element by element to get , ,Will Input to the sixth 3×3 convolution to get , ,Will and Add element by element to get the grayscale image , , It is the output of the grayscale generation module GGB.

[0045] Further, in step S4, for the coarse image generation module CCB, the input image , , 1280, 1280 and 3 are the input images The height, width and channels of Input to the first 3×3 convolution to get , ,Will Input to the first channel attention to get , ,Will Input to the second 3×3 convolution to get , ,Will Input to the second channel attention to get , ,Will Input to the third 3×3 convolution to get , ,Will and Add element by element to get , ,Will Input to the fourth 3×3 convolution to get , ,Will and Add element by element to get , Then the output of the coarse image generation module CCB is obtained by calculating the noisy coarse image The calculation of the noisy coarse image is as follows: First, calculate the dark channel prior , , yes The three primary color channels, So A local block centered on yes Pixels in represents the minimum value calculation, and then calculates the bright channel prior , , Represents the maximum value calculation, and then through the filter The first 10% of pixels are used as atmospheric light , and then calculate the transmission map , , yes The three primary color channels are calculated and Get a noisy rough image , , Representatives in and 0.14, take the maximum value. .

[0046] Further, in step S5, for the noise reduction processing module DHB, the noisy rough image is input and grayscale , , ,Will Perform color space transformation to obtain , that is, convert from RGB to YCbCr, and use replace The Y channel in the image is used to obtain the denoised rough image. ,Will Perform color space conversion, that is, convert from YCbCr to RGB, to obtain the output of the noise reduction processing module DHB , It is a rough image with noise reduction. .

[0047] Further, in step S6, for the second-order refinement module TRF, the denoised coarse image is input , , H, W and 3 are the rough images of noise reduction The height, width and channels of Input to the first 3×3 convolution to get , ,Will Input to the second 3×3 convolution to get , ,Will Input to the third 3×3 convolution to get , ,Will Input to the fourth 3×3 convolution to get , ,Will and Add element by element to get , ,Will Input to the fifth 3×3 convolution to get , ,Will and Add element by element to get , ,Will Input to the sixth 3×3 convolution to get , ,Will and Add element by element to get , , is the output of the second-order refinement module TRF.

[0048] Further, in step S7, for the chest X-ray image enhancement module, the chest X-ray image is input Go to the grayscale generation module GGB to get the grayscale image , , , input chest X-ray image Go to the coarse image generation module CCB to get the noisy coarse image , , input grayscale image and noisy coarse image Go to the noise reduction processing module DHB to get the noise reduction rough image , the denoised coarse image Input to the second-order refinement module TRF to obtain enhanced chest X-ray image , .

[0049] The above are only preferred embodiments of the present invention. It should be pointed out that a person skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A method for assisting in determining abnormal shadows in chest X-rays, characterized in that: The following steps are involved: S1. Chest X-ray dataset preparation: collect multiple chest X-ray images containing multiple abnormal shadows, annotate the multiple chest X-ray images, and annotate the targets as multiple abnormal shadows. After the annotation is completed, a chest X-ray dataset is obtained; S2, construct a spatial noise reduction module SD, including 1×1 convolution, 3×3 convolution, transposition, dimension transformation, matrix multiplication and element-by-element addition; the spatial noise reduction module SD, input feature X1, X1∈R H×W×C , H, W and C are the height, width and channel of feature X1, and X1 is input into the 3×3 convolution to obtain X 13 , X 13 ∈R H×W×C , X 13 Input the first 1×1 convolution to get X 13C1 , C1 is not equal to C, change X 13C1 Input to the second 1×1 convolution to get X 13C2 , X 13C2 Transform the dimension to get X2, S = H × W, X 13 Input to the third 1×1 convolution to get X 13C3 , X 13C3 ∈R H×W×C , X 13C3 Input to the fourth 1×1 convolution to get X 13C4 , X 13C4 ∈R H ×W×C , X 13C4 Transform and transpose the dimension to obtain X3, X3∈R C×S , perform matrix multiplication of X3 and X2 to obtain X4, Transpose X2 to get X5, Perform matrix multiplication of X4 and X5 to obtain X6, X6∈R C ×S , transform and transpose X6 to obtain X7, X7∈R H×W×C , add X7 and X1 element by element to get X 7+ , X 7+ ∈R H×W×C , X 7+ is the output of the spatial noise reduction module SD, X 7+ The same dimension as X1; S3, construct a grayscale generation module GGB, including 1×1 convolution, 3×3 convolution, spatial attention, element-by-element addition and spatial noise reduction module SD; the grayscale generation module GGB, input image I, I∈R H×W×3 , H, W and 3 are the height, width and channel of the input image I, and I is input to the first 1×1 convolution to obtain X GGB1 , X GGB1 ∈R H×W×1 , X GGB1 Input to the first spatial attention to get X GGB1S , X GGB1S ∈R H×W×1 , X GGB1S Input to the first spatial noise reduction module SD to get X GGB1SD , X GGB1SD ∈R H×W×1 , X GGB1SD Input to the first 3×3 convolution to get X GGB2 , X GGB2 Input to the second spatial attention to get X GGB2S , X GGB2S Input to the second spatial noise reduction module SD to get X GGB2SD , X GGB2SD Input to the second 3×3 convolution to get X GGB3 , X GGB3 Input to the third spatial attention to get X GGB3S , X GGB3S Input to the third spatial noise reduction module SD to get X GGB3SD , X GGB3BD Input to the third 3×3 convolution to get X GGB4 , X GGB4 Input to the fourth spatial attention to get X GGB4S , X GGB4S Input to the fourth spatial noise reduction module SD to get X GGB4SD , X GGB4SD Input to the fourth 3×3 convolution to get X GGB5 , X GGB5 and X GGB3 Add element by element to get X GGB5+ , X GGB5+ Input to the fifth 3×3 convolution to get X GGB6 , X GGB6 and X GGB2 Add element by element to get X GGB6+ , X GGB6+ Input to the sixth 3×3 convolution to get X GGB7 , X GGB7 ∈R H×W×1 , X GGB7 and X GGB1 Add element by element to get the grayscale image X GGBOut , X GGBOut ∈R H×W×1 , X GGBOut It is the output of the grayscale generation module GGB; S4, construct a coarse image generation module CCB, including 3×3 convolution, channel attention, element-by-element addition and noisy coarse image calculation; the coarse image generation module CCB, input image I, I∈R H×W×3 , H, W and 3 are the height, width and channel of the input image I, and I is input to the first 3×3 convolution to obtain X CCB1 , C1>3, X CCB1 Input to the first channel attention to get X CCB1C , X CCB1C Input to the second 3×3 convolution to get X CCB2 , C2>C1, X CCB2 Input to the second channel attention to get X CCB2C , X CCB2C Input to the third 3×3 convolution to get X CCB3 , X CCB3 and X CCB1 Add element by element to get X CCB3+ , X CCB3+ Input to the fourth 3×3 convolution to get X CCB4 , X CCB4 ∈R H×W×3 , X CCB4 Add I element by element to get I + , I + ∈R H×W×3 Then, the output I of the coarse image generation module CCB is obtained by calculating the noisy coarse image NC The calculation of the noisy coarse image is as follows: First, calculate the dark channel prior I +d (x), isI + The three primary color channels, Ω(x) is the local block centered on x, y is the pixel in Ω(x), min represents the minimum value calculation, and then calculate the bright channel prior I +b (x), max represents the maximum value calculation, and then through the screening I +b The first 10% of pixels in (x) are taken as atmospheric light A, and then the transmission map t(x) is calculated. A C are the three primary color channels of A. Finally, t(x) and A are calculated to obtain the noisy rough image I NC , max(t(x), 0.14) means taking the maximum value between t(x) and 0.

14. NC ∈R H×W×3 ; S5, construct a noise reduction processing module DHB, including color space transformation; the noise reduction processing module DHB inputs the noisy rough image I DHBNC and grayscale image I DHBGS , I DHBNC ∈R H×W×3 , I DHBGS ∈R H×W×1 , will I DHBNC Perform color space transformation to obtain I DHBY , that is, convert from RGB to YCbCr, and use I DHBGS Replace I DHBY The Y channel in the image is obtained by DHBY+ , will I DHBY+ Perform color space conversion, that is, convert from YCbCr to RGB, so as to obtain the output I of the noise reduction processing module DHB DHBOut , I DHBOut is the denoised rough image, I DHBOut ∈R H×W×3 ; S6, construct a second-order refinement module TRF, including 3×3 convolution and element-by-element addition; the second-order refinement module TRF inputs the denoised coarse image I TRFDC , I TRFDC ∈R H×W×3 , H, W and 3 are the noise reduction coarse image I TRFDC The height, width and channel of I TRFDC Input to the first 3×3 convolution to get I TRF1 , C1>3, I TRF1 Input to the second 3×3 convolution to get I TRF2 , C2>C1, I TRF2 Input to the third 3×3 convolution to get I TRF3 , C3>C2, I TRF3 Input to the fourth 3×3 convolution to get I TRF4 , Will I TRF4 and I TRF2 Add element by element to get I TRF4+ , Will I TRF4+ Input to the fifth 3×3 convolution to get I TRF5 , Will I TRF5 and I TRF1 Add element by element to get I TRF5+ , Will I TRF5+ Input to the sixth 3×3 convolution to get IT RF6 , I TRF6 ∈R H×W×3 , will I TRF6 and I TRFDC Add element by element to get I TRFOut , I TRFOut ∈R H×W×3 , I TRFOut is the output of the second-order refinement module TRF; S7, constructing a chest X-ray image enhancement module, including a grayscale generation module GGB, a coarse image generation module CCB, a noise reduction processing module DHB and a second-order refinement module TRF; the chest X-ray image enhancement module inputs the chest X-ray image to the grayscale generation module GGB to obtain a grayscale image, inputs the chest X-ray image to the coarse image generation module CCB to obtain a noisy coarse image, inputs the grayscale image and the noisy coarse image to the noise reduction processing module DHB to obtain a denoised coarse image, and inputs the denoised coarse image to the second-order refinement module TRF to obtain an enhanced chest X-ray image; S8, constructing a chest X-ray abnormal shadow detection model, which includes an input, a chest X-ray image enhancement module, a backbone network, a detection head, and an output; S9. Train a chest X-ray abnormal shadow detection model and perform detection. Use a chest X-ray data set to train the chest X-ray abnormal shadow detection model. After the training is completed, input a new chest X-ray image to be detected into the chest X-ray abnormal shadow detection model to obtain an abnormal shadow detection result.

2. The method for assisting in determining abnormal chest X-ray shadows according to claim 1, characterized in that: In step S1, the various abnormal shadows include point shadows, spot shadows, miliary shadows, nodular shadows, block shadows, patchy shadows, stripe shadows, band shadows, ring shadows and diffuse shadows.

Images