X-ray multi-contrast imaging image quality enhancement method based on feature prior learning

Abstract

The invention relates to an X-ray multi-contrast imaging image quality enhancement method based on feature prior learning, and belongs to the technical field of deep learning and X-ray multi-contrast imaging. The method comprises the following steps: acquiring low-image-quality absorption, phase and dark field contrast computed tomography images and corresponding normal-image-quality three-contrast computed tomography images by using an X-ray grating multi-contrast imaging system, and constructing a training data set; constructing a conditional generative adversarial network image quality enhancement model based on feature prior learning; using the training data set to train a conditional generative adversarial network image quality enhancement model; and using the trained conditional generative adversarial network image quality enhancement model to enhance the obtained low-image-quality absorption, phase and dark field contrast computed tomography image. According to the invention, the low-quality multi-contrast images obtained under different conditions can be recovered and enhanced, and the application of the X-ray multi-contrast imaging technology in biomedical clinic and frontier material science is promoted.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning and X-ray multi-contrast imaging, and in particular to an image quality enhancement method for X-ray multi-contrast imaging based on feature prior learning. Background Art

[0002] Among X-ray imaging methods, X-ray multi-contrast imaging methods can provide more information. Among them, absorption contrast images are based on the absorption effect of X-rays by materials, and can provide better contrast for high-density materials; phase contrast images are based on the phase shift of X-rays by materials, and can provide better contrast for materials with low atomic numbers such as soft tissues; and dark field contrast images can provide better imaging for fine structures and the edges of structures.

[0003] The most important X-ray multi-contrast imaging method at present is the grating imaging method based on the Talbot-Lau interferometer. However, this method currently has problems such as high imaging dose, long imaging time, and high requirements for mechanical motion accuracy, which seriously hinders its application in biomedical clinics and cutting-edge material science. Therefore, methods such as reducing tube current, reducing exposure time, reducing sampling angle, and single exposure imaging analysis technology have been proposed to solve the above problems. However, although the above methods can solve the problems of high imaging dose, long imaging time, and high requirements for mechanical motion accuracy, they will significantly affect the image quality of the three contrast images. For example, reducing tube current and reducing exposure time will introduce quantum noise; reducing sampling angle will produce image artifacts; and single exposure imaging technology will lead to problems such as spatial resolution degradation. However, many studies have shown that under the above conditions, the three contrasts of X-ray multi-contrast imaging are not affected in the same way. Among the three contrast images, the absorption contrast image quality degrades the least, while the dark field contrast image quality degrades the most.

[0004] Previously, there has been an image restoration method that uses convolutional neural networks to achieve three contrast complementarities, but this method simultaneously performs complementary fusion of the three contrasts and does not specifically utilize the different characteristics of different contrasts that are affected. In addition, the convolutional neural network-based model fails to properly handle the situation where the three contrast images overlap and conflict when performing feature fusion, which can cause distortion of the three contrast images or cause one contrast to be too similar to another.

[0005] At present, no image quality enhancement method for X-ray multi-contrast imaging based on feature prior learning has been found. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides an image quality enhancement method for X-ray multi-contrast imaging based on feature prior learning. The method adds a convolutional neural network to the generator in the generative adversarial network to extract features of different contrasts and use the feature map of the contrast image with less image quality affected as a priori to guide the feature enhancement of the contrast image with greater image quality affected. At the same time, the discriminator in the generative adversarial network confronts the generator and discriminates the enhanced image, thereby eliminating the redundant feature information that conflicts between the contrasts after passing through the network, while restoring the structural features as much as possible, and better preserving the original information of the contrast. The method can obtain low-quality projection images by reducing tube current, reducing the number of sampled projections, using incomplete projection angles, etc., and then reconstruct the computer tomography image, and input the obtained low-quality reconstructed image into the network, and finally complete the image quality enhancement of the low-quality X-ray multi-contrast imaging image without improving the imaging device.

[0007] To achieve the above object, the technical solution adopted by the present invention is:

[0008] The image quality enhancement method of X-ray multi-contrast imaging based on feature prior learning includes:

[0009] Step S1, using an X-ray grating multi-contrast imaging system to obtain low-quality absorption, phase, and dark-field contrast computed tomography images and corresponding normal-quality absorption, phase, and dark-field contrast computed tomography images to construct a training data set;

[0010] Step S2, constructing a conditional generative adversarial network image quality enhancement model based on feature prior learning;

[0011] Step S3, using the training data set to train the constructed conditional generative adversarial network image quality enhancement model based on feature prior learning;

[0012] Step S4: Use the trained conditional generative adversarial network image quality enhancement model based on feature prior learning to enhance the acquired low-quality absorption, phase, and dark field contrast computed tomography images.

[0013] The beneficial effects of the present invention are:

[0014] Compared with the existing X-ray multi-contrast imaging method, the present invention specifically considers the specific extent to which different contrasts are affected under low-quality imaging conditions, and can better enhance the contrast that is greatly affected by using the contrast that is less affected as a priori while maintaining the original characteristics of the image contrast. It solves the problem of low image quality existing in methods such as reducing the tube current of the X-ray tube during imaging, reducing the imaging exposure time, reducing the imaging sampling angle, and single exposure imaging, and promotes the engineering and clinical applications of the X-ray multi-contrast imaging method. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of the image quality enhancement method of X-ray multi-contrast imaging based on feature prior learning of the present invention;

[0016] Figure 2 A schematic diagram of an X-ray grating multi-contrast imaging system provided by an embodiment of the present invention;

[0017] Figure 3 This is a schematic diagram of the network structure of the conditional generative adversarial network image quality enhancement model based on feature prior learning of the present invention;

[0018] Figure 4 A set of normal quality and low quality absorption, phase and dark field contrast CT images acquired using an X-ray grating multi-contrast imaging system; wherein, (a) is an absorption contrast CT image of normal quality; (b) is a phase contrast CT image of normal quality; (c) is a dark field contrast CT image of normal quality; (d) is an absorption contrast CT image of low quality; (e) is a phase contrast CT image of low quality; (f) is a dark field contrast CT image of low quality;

[0019] Figure 5 The results are obtained after restoring three low-quality contrast CT images based on the method of the present invention; among them, (a) is the enhanced absorption contrast CT image; (b) is the enhanced phase contrast CT image; (c) is the enhanced dark field contrast CT image.

[0020] Reference numerals:

[0021] 20. X-ray source, 21. Source grating, 22. Test object, 23. Phase grating, 24. Absorption grating, 25. Detector. DETAILED DESCRIPTION

[0022] The present invention is further described below with reference to the accompanying drawings and specific embodiments.

[0023] like Figure 1 The figure shows a flow chart of the image quality enhancement method for X-ray multi-contrast imaging based on feature prior learning provided by the present invention; the present invention can use the less affected image features as priors to guide the enhancement of the image features that are greatly affected based on the different properties of different contrast images in X-ray multi-contrast imaging affected by low-quality imaging conditions, and finally complete the image quality enhancement of three X-ray contrasts without improving the imaging device. The specific steps of the method are as follows:

[0024] Step S1, using an X-ray grating multi-contrast imaging system, obtaining low-quality absorption, phase, and dark field contrast computed tomography images and corresponding normal-quality absorption, phase, and dark field contrast computed tomography images to construct a training data set;

[0025] Among them, the method for obtaining low-quality images in step S1 includes reducing the tube current of the X-ray tube during imaging, reducing the exposure time of imaging, using single exposure imaging methods such as tilted grating and offset grating, and other X-ray multi-contrast imaging methods.

[0026] Figure 2 Schematic diagram of the X-ray grating multi-contrast imaging system in step S1 provided in an embodiment of the present invention; Figure 2 As shown, the system is mainly based on the Talbot-Lau effect, including an X-ray source 20, a source grating 21, a test object 22, a phase grating 23, an absorption grating 24, and a detector 25 arranged in sequence along the optical path; wherein the source grating 21 is used to generate the Lau effect so that the X-rays produce local quasi-coherence, the phase grating 23 is used to generate the Talbot interference effect, the absorption grating 24 is used to modulate the interference fringes, and the detector 25 is used to convert the received optical signal into an electrical signal to generate an image on a computer; in order to meet the conditions of the Talbot interference effect, the parameters of the imaging system should satisfy the following formulas (1)-(4):

[0027] (1)

[0028] (2)

[0029] (3)

[0030] (4)

[0031] in, represents the distance between the phase grating 23 and the absorption grating 24; is the magnification ratio, is the distance between the source grating 21 and the phase grating 23; Indicates Fractional Talbot distance; is the period of the phase grating 23, is the wavelength of the X-rays, is the period of the absorption grating 24, is the period of the source grating 21, is the width of the source grating 21 that allows X-rays to pass through in each period;

[0032] Step S2, constructing a conditional generative adversarial network image quality enhancement model based on feature prior learning;

[0033] Figure 3 The structural diagram of the conditional generation of the adversarial network image quality enhancement model based on feature prior learning described in step S2. It includes a generator and a discriminator. The generator uses low-quality absorption, phase, and dark field contrast computer tomography images as conditional inputs, and uses a feature fusion enhancement module to preliminarily enhance the structural features of low-quality absorption, phase, and dark field contrast computer tomography images. Furthermore, based on the information that the quality of images with different contrasts is affected to different degrees, a feature prior-based image restoration module is constructed for the contrast images that are greatly affected to restore high-quality absorption, phase, and dark field contrast computer tomography images; the discriminator is used to judge the authenticity and quality of the high-quality absorption, phase, and dark field contrast computer tomography images generated by the generator at different scales based on the normal-quality absorption, phase, and dark field contrast computer tomography images, which can effectively eliminate the overlapping redundant information between contrasts generated when performing feature fusion enhancement.

[0034] like Figure 3 As shown, the feature fusion enhancement module of the generator of the conditional generative adversarial network image quality enhancement model based on feature prior learning in step S2 includes an N-layer Unet network. The input low-quality absorption, phase, and dark field contrast computed tomography images are subjected to deep feature fusion through a three-channel N-layer Unet network, and then three single-channel N-layer Unet networks are used to separate the fused deep features and enhance the separated deep features respectively. N is the number of downsampling times. For an input image of size 512x512, the maximum value of N is 8.

[0035] like Figure 3 As shown, the image restoration module based on feature prior of the generator of the conditional generative adversarial network image quality enhancement model based on feature prior learning in step S2 is composed of three branches, corresponding to absorption, phase and dark field contrast respectively. Each branch sequentially connects two convolutional layers with a size of 3x3, a step size of 1 and a number of channels of 32, five residual blocks, and a convolutional layer with a size of 3x3, a step size of 1 and a number of channels of 1. The residual block includes a convolutional layer with a size of 3x3, a step size of 1 and a number of channels of 128 and a convolutional layer with a size of 3x3, a step size of 1 and a number of channels of 32, which are sequentially connected. The residual block enhances the obtained feature map and directly adds it to the input to obtain the feature map finally output by the residual block to avoid the disappearance of the gradient during the training process.

[0036] like Figure 3As shown, the discriminator of the conditional generative adversarial network image quality enhancement model based on feature prior learning described in step S2 is a multi-scale discriminator, including a 2x2 maximum pooling layer for downsampling and four convolutional layers of size 3x3 connected in sequence, wherein the step size of the first two convolutional layers is 2, and the first two convolutional layers are followed by a LeakyRelu activation layer and a BatchNorm normalization layer, and the step size of the last two convolutional layers is 1; the feature map obtained by inputting the absorption, phase, and dark field contrast computed tomography images of normal image quality into the discriminator and the output result of the generator into the feature map obtained by inputting the discriminator to perform cross entropy loss, and the probability that the output result of the generator is true is calculated, and the back propagation of the training process is participated in as the generation loss.

[0037] Step S3, using the training data set to train the constructed conditional generative adversarial network image quality enhancement model based on feature prior learning;

[0038] The training process in step S3 includes forward propagation, loss calculation, and back propagation. The forward propagation is completed based on the network in step S2. The loss includes generation loss and L1 loss, wherein the generation loss is calculated by the result of the discriminator in step S2; the L1 loss is the sum of the pixel value differences between the low-quality image and the high-quality image after being enhanced by the image quality enhancement model:

[0039] (5)

[0040] in, Represent the pixel positions in the horizontal and vertical directions of the image respectively; Represent the size of the image in the horizontal and vertical directions respectively; They represent the absorption, phase, and dark-field contrast images input to the network, respectively; Enhanced absorption, phase, and dark-field contrast images representing the output of the network generator.

[0041] Step S4: Use the trained conditional generative adversarial network image quality enhancement model based on feature prior learning to enhance the acquired low-quality three-contrast computed tomography images.

[0042] Example

[0043] According to step S1, multiple groups of absorption, phase, and dark field contrast images of normal quality and low quality X-ray grating differential phase contrast imaging can be obtained as training data sets. The training data set includes 200 groups of images. This embodiment of the scheme uses a low tube current method, but the embodiment of the present invention is not limited to the low tube current method, but is effective for degraded images obtained under various methods such as low tube current, low exposure time, sparse sampling angle, incomplete sampling angle, and single exposure imaging.

[0044] According to step S3, the training data set is input into the conditional generative adversarial network image quality enhancement model constructed in step S2 for training, the generator iterates 200 generations, and the discriminator iterates 40 generations, that is, the generator iterates 5 times and the discriminator iterates once; the learning rate is set to 0.002.

[0045] In addition, in order to verify the effectiveness of the embodiment of the present invention, in addition to the training data set, 100 sets of low-quality absorption, phase, and dark field contrast images were acquired as test data sets. Figure 4 is one of the test data sets. Among them, (a) is a normal quality absorption contrast image, (b) is a normal quality phase contrast image, (c) is a normal quality dark field contrast image, (d) is a low quality absorption contrast image, (e) is a low quality phase contrast image, and (f) is a low quality dark field contrast image.

[0046] like Figure 5 As shown in the figure, one of the results (corresponding to Figure 4 ), where (a) is the enhanced absorption contrast CT image; (b) is the enhanced phase contrast CT image; (c) is the enhanced dark field contrast CT image. It can be clearly seen that the quality of the three contrast images has been significantly improved, closer to the normal quality image; especially the dark field contrast image and the phase contrast image, which are the most affected, have a more obvious improvement in image quality.

[0047] The above specific implementation methods further describe the purpose, technical solutions and beneficial effects of the present invention in detail. Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned network modules is used as an example. In practical applications, the above-mentioned functions can be assigned to different network modules as needed, that is, the network structure is divided into different functional modules to complete all or part of the functions described above. The specific working process of the network described above can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0048] It should be understood that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the deep learning network structures and parameters therein. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for image quality enhancement of X-ray multi-contrast imaging based on feature prior learning, characterized in that: The steps include: Step S1, using an X-ray grating multi-contrast imaging system to obtain low-quality absorption, phase, and dark-field contrast computed tomography images and corresponding normal-quality absorption, phase, and dark-field contrast computed tomography images to construct a training data set; Step S2, constructing a conditional generative adversarial network image quality enhancement model based on feature prior learning; Step S3, using the training data set to train the constructed conditional generative adversarial network image quality enhancement model based on feature prior learning; Step S4: Use the trained conditional generative adversarial network image quality enhancement model based on feature prior learning to enhance the acquired low-quality absorption, phase, and dark field contrast computed tomography images.

2. The method for image quality enhancement of X-ray multi-contrast imaging based on feature prior learning according to claim 1, characterized in that: The X-ray grating multi-contrast imaging system in step S1 includes an X-ray source, a source grating, a test object, a phase grating, an absorption grating, and a detector arranged in sequence along the optical path. The parameters of the imaging system satisfy: （1） （2） （3） （4） in, represents the distance between the phase grating and the absorption grating; is the magnification ratio, is the distance between the source grating and the phase grating; Indicates Fractional Talbot distance; is the period of the phase grating, is the wavelength of the X-rays, is the period of the absorption grating, is the period of the source grating, is the width of the source grating that allows X-rays to pass through in each period.

3. The image quality enhancement method for X-ray multi-contrast imaging based on feature prior learning according to claim 1, characterized in that: The method for obtaining low-quality absorption, phase, and dark field contrast computed tomography images in step S1 includes reducing the tube current of the X-ray tube during imaging, reducing the exposure time of imaging, using tilted gratings, offset gratings, and single exposure imaging methods.

4. The method for image quality enhancement of X-ray multi-contrast imaging based on feature prior learning according to claim 1, characterized in that: The conditional generative adversarial network image quality enhancement model based on feature prior learning in step S2 includes a generator and a discriminator. The generator uses low-quality absorption, phase, and dark field contrast computer tomography images as conditional inputs, uses a feature fusion enhancement module to preliminarily enhance the structural features of the low-quality absorption, phase, and dark field contrast computer tomography images, and uses an image restoration module based on feature priors to restore high-quality absorption, phase, and dark field contrast computer tomography images; the discriminator is used to judge the authenticity and quality of the high-quality absorption, phase, and dark field contrast computer tomography images generated by the generator at different scales based on normal-quality absorption, phase, and dark field contrast computer tomography images.

5. The method for image quality enhancement of X-ray multi-contrast imaging based on feature prior learning according to claim 4, characterized in that: The feature fusion enhancement module of the generator includes an N-layer Unet network, wherein the input low-quality absorption, phase, and dark field contrast computed tomography images are subjected to deep feature fusion through a three-channel N-layer Unet network, and then three single-channel N-layer Unet networks are used to separate the fused deep features and enhance the separated deep features respectively, where N is the number of downsampling times.

6. The method for image quality enhancement of X-ray multi-contrast imaging based on feature prior learning according to claim 4, characterized in that: The feature prior-based image restoration module includes three branches, corresponding to absorption, phase, and dark field contrast, respectively. Each branch sequentially connects two convolutional layers with a size of 3x3, a step size of 1, and a channel number of 32, five residual blocks, and a convolutional layer with a size of 3x3, a step size of 1, and a channel number of 1; wherein the residual block includes a convolutional layer with a size of 3x3, a step size of 1, and a channel number of 128, and a convolutional layer with a size of 3x3, a step size of 1, and a channel number of 32, which are sequentially connected. The residual block enhances the obtained feature map and directly adds it to the input to obtain the feature map finally output by the residual block.

7. The method for image quality enhancement of X-ray multi-contrast imaging based on feature prior learning according to claim 4, characterized in that: The discriminator is a multi-scale discriminator, including a 2x2 maximum pooling layer for downsampling and four convolutional layers of size 3x3 connected in sequence, wherein the step size of the first two convolutional layers is 2, and the first two convolutional layers are connected to a LeakyRelu activation layer and a BatchNorm normalization layer, and the step size of the last two convolutional layers is 1; the feature map obtained by inputting the absorption, phase, and dark field contrast of the computed tomography image of normal image quality into the discriminator and the output result of the generator into the feature map obtained by inputting the discriminator to perform cross entropy loss, and the probability that the output result of the generator is true is calculated, and the generated loss is used as the back propagation of the training process.

8. The method for image quality enhancement of X-ray multi-contrast imaging based on feature prior learning according to claim 1, characterized in that: The training process in step S3 includes calculating the loss, and the loss includes the generation loss and the L1 loss, wherein the generation loss is calculated by the result of the discriminator in step S2, and the L1 loss is the sum of the pixel value differences between the low-quality image and the high-quality image enhanced by the conditional generative adversarial network image quality enhancement model based on feature prior learning: （5） in, Represent the pixel positions in the horizontal and vertical directions of the image respectively; Represent the size of the image in the horizontal and vertical directions respectively; They respectively represent the absorption, phase, and dark field contrast images input into the conditional generative adversarial network image quality enhancement model based on feature prior learning; Enhanced absorption, phase, and dark-field contrast images representing the output of the generator.

Images