Image quality enhancement method for X-ray multi-contrast imaging based on feature prior learning

Abstract

The present invention relates to an image quality enhancement method for X-ray multi-contrast imaging based on feature prior learning, and belongs to the field of deep learning and X-ray multi-contrast imaging technology. The method comprises: using an X-ray grating multi-contrast imaging system to obtain low-quality absorption, phase, and dark-field contrast computed tomography images and corresponding three types of normal-quality contrast computed tomography images, and constructing a training data set; constructing a conditional generative adversarial network image quality enhancement model based on feature prior learning; using the training data set to train the conditional generative adversarial network image quality enhancement model; and using the trained conditional generative adversarial network image quality enhancement model to enhance the acquired low-quality absorption, phase, and dark-field contrast computed tomography images. The present invention can restore and enhance low-quality multi-contrast images acquired under different conditions, and promote the application of X-ray multi-contrast imaging technology in biomedical clinics and cutting-edge materials science.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning and X-ray multi-contrast imaging, and in particular relates to an image quality enhancement method for X-ray multi-contrast imaging based on feature prior learning. Background Art

[0002] Among X-ray imaging methods, multi-contrast X-ray imaging can provide more information. Absorption contrast imaging, based on the absorption of X-rays by a substance, can provide better contrast for high-density materials. Phase contrast imaging, based on the phase shift of X-rays by a substance, can provide better contrast for low-atomic-number materials such as soft tissue. Dark-field contrast imaging, on the other hand, can provide better imaging of fine structures and their edges.

[0003] The most prominent X-ray multi-contrast imaging method currently is grating imaging based on the Talbot-Lau interferometer. However, this method currently suffers from issues such as high imaging dose, long imaging time, and high mechanical motion precision requirements, which severely hinder its application in biomedical clinical settings and cutting-edge materials science. Therefore, methods such as reducing tube current, exposure time, sampling angle, and single-exposure imaging analysis have been proposed to address these issues. However, while these methods address the high imaging dose, long imaging time, and high mechanical motion precision requirements, they can significantly degrade the image quality of the three contrast types. For example, reducing tube current and exposure time introduces quantum noise; reducing the sampling angle produces image artifacts; and single-exposure imaging leads to degradation in spatial resolution. However, multiple studies have demonstrated that under these conditions, the three contrast types in X-ray multi-contrast imaging are not uniformly affected. Of the three contrast types, absorption contrast suffers the least degradation, while dark-field contrast suffers the most.

[0004] Previous image restoration methods have used convolutional neural networks to achieve three complementary contrasts. However, these methods simultaneously fuse the three contrasts and fail to specifically exploit the different characteristics of different contrasts. Furthermore, convolutional neural network-based models fail to effectively handle the conflicting effects of overlapping redundant information in the three contrast images during feature fusion. This can cause distortion in all three contrast images or render one contrast excessively similar to another.

[0005] At present, no image quality enhancement method for X-ray multi-contrast imaging based on feature prior learning has been found. Summary of the Invention

[0006] To address the above-mentioned technical problems, the present invention provides an image quality enhancement method for X-ray multi-contrast imaging based on feature prior learning. The method incorporates a convolutional neural network into the generator of a generative adversarial network to extract features of different contrasts and utilizes the feature maps of contrast images with less impact as priors to guide feature enhancement of contrast images with greater impact. Simultaneously, the discriminator in the generative adversarial network competes with the generator to discriminate the enhanced image, thereby eliminating redundant feature information that conflicts between contrasts after passing through the network. This method effectively preserves the original contrast information while restoring structural features as much as possible. This method can obtain low-quality projection images by reducing tube current, reducing the number of sampled projections, or using incomplete projection angles, and then reconstructs the computed tomography image. The resulting low-quality reconstructed image is then input into the network, ultimately completing image quality enhancement of low-quality X-ray multi-contrast imaging without requiring modifications to the imaging device.

[0007] To achieve the above object, the technical solution adopted by the present invention is:

[0008] The image quality enhancement method of X-ray multi-contrast imaging based on feature prior learning includes:

[0009] Step S1, using an X-ray grating multi-contrast imaging system, acquiring low-quality absorption, phase, and dark-field contrast computed tomography images and corresponding normal-quality absorption, phase, and dark-field contrast computed tomography images to construct a training data set;

[0010] Step S2: constructing a conditional generative adversarial network image quality enhancement model based on feature prior learning;

[0011] Step S3: using the training data set to train the constructed conditional generative adversarial network image quality enhancement model based on feature prior learning;

[0012] Step S4: Use the trained conditional generative adversarial network image quality enhancement model based on feature prior learning to enhance the acquired low-quality absorption, phase, and dark field contrast computed tomography images.

[0013] The beneficial effects of the present invention are:

[0014] Compared with existing X-ray multi-contrast imaging methods, the present invention specifically considers the specific extent to which different contrasts are affected under low-quality imaging conditions. It can better enhance the contrast that is more affected by using the less affected contrast as a priori while preserving the original characteristics of the image contrast. It solves the problem of low image quality that exists in methods such as reducing the tube current of the X-ray tube during imaging, reducing the imaging exposure time, reducing the imaging sampling angle, and single-exposure imaging, and promotes the engineering and clinical applications of X-ray multi-contrast imaging methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of the image quality enhancement method for X-ray multi-contrast imaging based on feature prior learning of the present invention;

[0016] Figure 2 Schematic diagram of an X-ray grating multi-contrast imaging system provided by an embodiment of the present invention;

[0017] Figure 3 This is a schematic diagram of the network structure of the conditional generative adversarial network image quality enhancement model based on feature prior learning of the present invention;

[0018] Figure 4 A set of normal-quality and low-quality absorption, phase, and dark-field contrast CT images acquired using an X-ray grating multi-contrast imaging system; wherein, (a) is a normal-quality absorption contrast CT image; (b) is a normal-quality phase contrast CT image; (c) is a normal-quality dark-field contrast CT image; (d) is a low-quality absorption contrast CT image; (e) is a low-quality phase contrast CT image; and (f) is a low-quality dark-field contrast CT image.

[0019] Figure 5 The results obtained after restoring three low-quality contrast CT images based on the method of the present invention; among them, (a) is the enhanced absorption contrast CT image; (b) is the enhanced phase contrast CT image; and (c) is the enhanced dark field contrast CT image.

[0020] Reference numerals:

[0021] 20. X-ray source, 21. Source grating, 22. Test object, 23. Phase grating, 24. Absorption grating, 25. Detector. DETAILED DESCRIPTION

[0022] The present invention is further described below with reference to the accompanying drawings and specific embodiments.

[0023] like Figure 1 The figure shows a flow chart of the image quality enhancement method for X-ray multi-contrast imaging based on feature prior learning provided by the present invention. Based on the fact that different contrast images in X-ray multi-contrast imaging are affected differently by low-quality imaging conditions, the present invention utilizes the less affected image features as priors to guide the enhancement of the more affected image features, ultimately completing the image quality enhancement of the three X-ray contrasts without requiring any modifications to the imaging device. The specific steps of the method are as follows:

[0024] Step S1: Using an X-ray grating multi-contrast imaging system, low-quality absorption, phase, and dark field contrast computed tomography images and corresponding normal-quality absorption, phase, and dark field contrast computed tomography images are acquired to construct a training data set;

[0025] The method for obtaining a low-quality image in step S1 includes reducing the tube current of the X-ray tube during imaging, reducing the exposure time of imaging, using single-exposure imaging methods such as tilted grating and staggered grating, and other X-ray multi-contrast imaging methods.

[0026] Figure 2 Schematic diagram of the X-ray grating multi-contrast imaging system in step S1 provided in an embodiment of the present invention; Figure 2 As shown, the system is mainly based on the Talbot-Lau effect, including an X-ray source 20, a source grating 21, a test object 22, a phase grating 23, an absorption grating 24, and a detector 25 arranged in sequence along the optical path; wherein the source grating 21 is used to generate the Lau effect so that the X-rays produce local quasi-coherence, the phase grating 23 is used to generate the Talbot interference effect, the absorption grating 24 is used to modulate the interference fringes, and the detector 25 is used to convert the received light signal into an electrical signal to generate an image on a computer; in order to meet the conditions of the Talbot interference effect, the parameters of the imaging system should satisfy the following formulas (1)-(4):

[0027] (1)

[0028] (2)

[0029] (3)

[0030] (4)

[0031] in, represents the distance between the phase grating 23 and the absorption grating 24; is the magnification ratio, is the distance between the source grating 21 and the phase grating 23; Indicates the Fractional Talbot distance; is the period of the phase grating 23, is the wavelength of X-rays, is the period of the absorption grating 24, is the period of the source grating 21, is the width of the source grating 21 that allows X-rays to pass through in each period;

[0032] Step S2: constructing a conditional generative adversarial network image quality enhancement model based on feature prior learning;

[0033] Figure 3 This is a structural diagram of the conditional generative adversarial network image quality enhancement model based on feature prior learning described in step S2. It includes a generator and a discriminator. The generator uses low-quality absorption, phase, and dark field contrast CT images as conditional inputs and utilizes a feature fusion enhancement module to perform preliminary enhancement on the structural features of the low-quality absorption, phase, and dark field contrast CT images. Furthermore, based on the existing information that the quality of images with different contrasts is affected to varying degrees, a feature prior-based image restoration module is constructed for the most affected contrast images to restore high-quality absorption, phase, and dark field contrast CT images. The discriminator is used to judge the authenticity and quality of the high-quality absorption, phase, and dark field contrast CT images generated by the generator at different scales based on normal-quality absorption, phase, and dark field contrast CT images, effectively eliminating the overlapping redundant information between contrasts generated during feature fusion enhancement.

[0034] like Figure 3 As shown, the feature fusion enhancement module of the generator of the conditional generative adversarial network image quality enhancement model based on feature prior learning in step S2 includes an N-layer Unet network. The input low-quality absorption, phase, and dark field contrast computed tomography images are fed into a three-channel N-layer Unet network for deep feature fusion. The fused deep features are then separated and enhanced using three single-channel N-layer Unet networks. N is the number of downsampling steps. For a 512x512 input image, the maximum value of N is 8.

[0035] like Figure 3 As shown, the feature prior-based image restoration module of the generator of the conditional generative adversarial network image quality enhancement model based on feature prior learning in step S2 is composed of three branches, corresponding to absorption, phase and dark field contrast respectively. Each branch is sequentially connected to two convolutional layers with a size of 3x3, a step size of 1 and a number of channels of 32, five residual blocks, and a convolutional layer with a size of 3x3, a step size of 1 and a number of channels of 1. The residual block includes a convolutional layer with a size of 3x3, a step size of 1 and a number of channels of 128 and a convolutional layer with a size of 3x3, a step size of 1 and a number of channels of 32, which are sequentially connected. The residual block enhances the obtained feature map and directly adds it to the input to obtain the feature map finally output by the residual block to avoid the gradient disappearance during the training process.

[0036] like Figure 3As shown, the discriminator of the conditional generative adversarial network image quality enhancement model based on feature prior learning described in step S2 is a multi-scale discriminator, including a 2x2 maximum pooling layer for downsampling and four convolutional layers of size 3x3 connected in sequence, wherein the step size of the first two convolutional layers is 2, and the first two convolutional layers are followed by a LeakyRelu activation layer and a BatchNorm normalization layer, and the step size of the last two convolutional layers is 1; the feature map obtained by inputting the absorption, phase, and dark field contrast of the normal image quality into the discriminator and the output result of the generator into the feature map obtained by inputting the discriminator to perform cross entropy loss, and the probability that the output result of the generator is true is calculated, and the generated loss is used as the back propagation of the training process.

[0037] Step S3: using the training data set to train the constructed conditional generative adversarial network image quality enhancement model based on feature prior learning;

[0038] The training process in step S3 includes forward propagation, loss calculation, and back propagation. Forward propagation is performed based on the network described in step S2. The losses include generation loss and L1 loss. The generation loss is calculated from the discriminator results in step S2. The L1 loss is calculated by calculating the sum of the pixel value differences between the low-quality image and the high-quality image after enhancement by the image quality enhancement model:

[0039] (5)

[0040] in, Represent the pixel positions in the horizontal and vertical directions of the image respectively; Represents the size of the image in the horizontal and vertical directions respectively; Represent the absorption, phase, and dark field contrast images input to the network respectively; Enhanced absorption, phase, and dark-field contrast images representing the output of the network generator.

[0041] Step S4: Use the trained conditional generative adversarial network image quality enhancement model based on feature prior learning to enhance the acquired low-quality three-contrast computed tomography images.

[0042] Example

[0043] According to step S1, multiple sets of normal-quality and low-quality X-ray grating differential phase contrast imaging absorption, phase, and dark-field contrast images can be acquired as a training dataset. The training dataset includes 200 sets of images. This example utilizes a low-tube current method, but the present invention is not limited to this method and is effective for degraded images obtained using various methods, including low tube current, low exposure time, sparse sampling angles, incomplete sampling angles, and single-exposure imaging.

[0044] According to step S3, the training data set is input into the conditional generative adversarial network image quality enhancement model constructed in step S2 for training. The generator iterates 200 generations and the discriminator iterates 40 generations, that is, the generator iterates 5 times and the discriminator iterates once; the learning rate is set to 0.002.

[0045] Furthermore, in order to verify the effectiveness of the embodiment of the present invention, in addition to the training data set, 100 sets of low-quality absorption, phase, and dark field contrast images were obtained as test data sets. Figure 4 is one of the test datasets. (a) is a normal-quality absorption contrast image, (b) is a normal-quality phase contrast image, (c) is a normal-quality dark-field contrast image, (d) is a low-quality absorption contrast image, (e) is a low-quality phase contrast image, and (f) is a low-quality dark-field contrast image.

[0046] like Figure 5 As shown in the figure, the conditional generation adversarial network image quality enhancement model trained in step S3 is used to perform image enhancement on the 100 test data sets, and one of the results (corresponding to Figure 4 Figure 1 shows the data shown in Figure 2, where (a) is an enhanced absorption contrast CT image; (b) is an enhanced phase contrast CT image; and (c) is an enhanced dark-field contrast CT image. It can be clearly seen that the quality of all three contrast images has been significantly improved, approaching normal quality. In particular, the dark-field contrast image and the phase contrast image, which were the most affected, have seen significant improvements in image quality.

[0047] The above specific embodiments further illustrate the purpose, technical solutions, and beneficial effects of the present invention. Those skilled in the art will clearly understand that for the sake of convenience and brevity, only the division of the above-mentioned network modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different network modules as needed, that is, the network structure can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the network described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0048] It should be understood that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some or all of the deep learning network structures and parameters therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for image quality enhancement of X-ray multi-contrast imaging based on feature prior learning, characterized in that: The steps include: Step S1, using an X-ray grating multi-contrast imaging system, acquiring low-quality absorption, phase, and dark-field contrast computed tomography images and corresponding normal-quality absorption, phase, and dark-field contrast computed tomography images to construct a training data set; Step S2: constructing a conditional generative adversarial network image quality enhancement model based on feature prior learning, wherein the model includes a generator, and the generator uses a feature recovery module to preliminarily enhance the structural features of the input low-quality absorption, phase, and dark field contrast computed tomography images to obtain preliminary absorption, phase, and dark field contrast enhancement results, and uses the preliminary absorption contrast enhancement result as the absorption contrast enhancement result, and uses the preliminary absorption contrast enhancement result as the prior for the preliminary phase contrast enhancement result to obtain the phase contrast enhancement result, and uses the preliminary absorption and phase contrast enhancement results as the prior for the preliminary dark field contrast enhancement result to obtain the dark field contrast enhancement result; Step S3: using the training data set to train the constructed conditional generative adversarial network image quality enhancement model based on feature prior learning; Step S4: Use the trained conditional generative adversarial network image quality enhancement model based on feature prior learning to enhance the acquired low-quality absorption, phase, and dark field contrast computed tomography images.

2. The image quality enhancement method for X-ray multi-contrast imaging based on feature prior learning according to claim 1, characterized in that: The X-ray grating multi-contrast imaging system in step S1 includes an X-ray source, a source grating, a test object, a phase grating, an absorption grating, and a detector arranged in sequence along the optical path. The parameters of the imaging system satisfy: （1） （2） （3） （4） in, represents the distance between the phase grating and the absorption grating; is the magnification ratio, is the distance between the source grating and the phase grating; Indicates the Fractional Talbot distance; is the period of the phase grating, is the wavelength of X-rays, is the period of the absorption grating, is the period of the source grating, is the width of the source grating that allows X-rays to pass through in each period.

3. The image quality enhancement method for X-ray multi-contrast imaging based on feature prior learning according to claim 1, characterized in that: The method for obtaining low-quality absorption, phase, and dark-field contrast computed tomography images in step S1 includes reducing the tube current of the X-ray tube during imaging, reducing the exposure time of imaging, using tilted gratings, staggered gratings, and single-exposure imaging methods.

4. The method for image quality enhancement of X-ray multi-contrast imaging based on feature prior learning according to claim 1, characterized in that: In step S2, the generator also includes a feature prior enhancement module, which restores high-quality absorption, phase, and dark field contrast computed tomography images based on the absorption, phase, and dark field contrast enhancement results; the conditional generative adversarial network image quality enhancement model also includes a discriminator for judging the authenticity and quality of the high-quality absorption, phase, and dark field contrast computed tomography images restored by the generator at different scales based on normal-quality absorption, phase, and dark field contrast computed tomography images.

5. The method for image quality enhancement of X-ray multi-contrast imaging based on feature prior learning according to claim 4, characterized in that: The feature recovery module of the generator includes an N-layer Unet network, in which the input low-quality absorption, phase, and dark field contrast computed tomography images are subjected to deep feature fusion through a three-channel N-layer Unet network, and then three single-channel N-layer Unet networks are used to separate the fused deep features and enhance the separated deep features respectively, where N is the number of downsampling times.

6. The method for image quality enhancement of X-ray multi-contrast imaging based on feature prior learning according to claim 4, characterized in that: The feature prior enhancement module includes three branches, corresponding to absorption, phase, and dark field contrast, respectively. Each branch sequentially connects two convolutional layers with a size of 3x3, a step size of 1, and a channel number of 32, five residual blocks, and a convolutional layer with a size of 3x3, a step size of 1, and a channel number of 1; wherein the residual block includes a convolutional layer with a size of 3x3, a step size of 1, and a channel number of 128, and a convolutional layer with a size of 3x3, a step size of 1, and a channel number of 32, which are sequentially connected. The residual block enhances the obtained feature map and directly adds it to the input to obtain the feature map finally output by the residual block.

7. The method for image quality enhancement of X-ray multi-contrast imaging based on feature prior learning according to claim 4, characterized in that: The discriminator is a multi-scale discriminator, including a 2x2 maximum pooling layer for downsampling and four convolutional layers of size 3x3, wherein the stride of the first two convolutional layers is 2, and the first two convolutional layers are followed by a LeakyRelu activation layer and a BatchNorm normalization layer, and the stride of the last two convolutional layers is 1; the feature map obtained by inputting the absorption, phase, and dark field contrast of the computed tomography image of normal image quality into the discriminator and the output result of the generator into the feature map obtained by inputting the discriminator to perform cross entropy loss, and the probability that the output result of the generator is true is calculated, and the generated loss is used as the back propagation of the training process.

8. The method for image quality enhancement of X-ray multi-contrast imaging based on feature prior learning according to claim 1, characterized in that: The training process in step S3 includes calculating the loss, which includes the generation loss and the L1 loss. The generation loss is calculated based on the result of the discriminator in step S2, and the L1 loss is calculated by calculating the sum of the pixel value differences between the low-quality image and the high-quality image after being enhanced by the conditional generative adversarial network image quality enhancement model based on feature prior learning: （5） in, Represent the pixel positions in the horizontal and vertical directions of the image respectively; Represents the size of the image in the horizontal and vertical directions respectively; They represent the absorption, phase, and dark field contrast images input to the conditional generative adversarial network image quality enhancement model based on feature prior learning; Enhanced absorption, phase, and dark-field contrast images representing the generator output.

Images