X-ray image inversion enhancement method based on deep neural network

Abstract

The invention discloses an X-ray image inversion enhancement method based on a deep neural network. The method comprises the following steps: S1, constructing a light tube focal spot-image effect model; s2, constructing a noise-image effect model; s3, carrying out the degradation of the X-ray image set based on the constructed light tube focal spot-image effect model and noise-image effect model, and obtaining a degraded X-ray image set; s4, constructing an X-ray imaging inversion model based on a convolutional neural network, and training the X-ray imaging inversion model by using the X-ray image before degradation and the X-ray image after degradation as training data of the network; and S5, based on the trained X-ray imaging inversion model, performing inversion enhancement on the to-be-processed X-ray image. According to the method, the X-ray imaging inversion enhancement model is established, noise reduction processing can be effectively carried out on the image, the image is sharpened, and good robustness and universality are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of X-ray image processing, and in particular to an X-ray image inversion enhancement method based on a deep neural network. Background Art

[0002] In X-ray images, the clarity of the image edge is affected not only by the absorption of X-ray energy by the object itself, but also by the focal spot and noise of the light tube. The smaller the focal spot of the light tube, the smaller the transition area of ​​the edge and the greater the edge gradient. Conversely, the larger the transition area, the smaller the edge gradient, but the smaller the focal spot of the light tube, the higher the price. At the same time, noise affects the signal-to-noise ratio of the image. As the noise increases, the signal-to-noise ratio of the image decreases, and the image quality becomes worse, which is not conducive to subsequent human eye observation and image processing.

[0003] like Figure 1 The figure shows a frame of X-ray image of a battery cell. Figure 2 The figure shows the X-ray image of the battery cell after taking eight frames and superimposing them. Figure 3 The following is an X-ray image of a battery cell taken with a large focal spot of a light tube after superimposing eight frames. Figure 1 and Figure 2 , we can clearly see the impact of X-ray noise on image quality. Figure 1 Obviously has a lower signal-to-noise ratio, while the contrast Figure 2 and Figure 3 , it can be seen that the focal spot of the light tube affects the clarity of the image edge.

[0004] At present, X-ray images need to be enhanced through image preprocessing in order to achieve better image effects. For X-ray images, the image processing process is noise reduction and sharpening. For noise reduction, commonly used image processing methods include Gaussian filtering, median filtering, mean filtering and other traditional image processing methods. For image sharpening, commonly used image processing methods include unsharp masking, image convolution sharpening methods based on various convolution kernels, etc. However, these algorithms often need to adjust relevant parameters according to the actual image quality, which has certain limitations.

[0005] In addition, patent document CN111507918A discloses a medical X-ray image enhancement method based on deep learning technology. This method uses VGG16 network and DenseUnet network to build an image enhancement network, uses VGG16 network to extract features from input images, then uses DenseUnet network for multi-scale feature fusion, and finally outputs the enhanced result image. However, this method requires a large number of image pairs for model training, and high-definition images of X-rays are often difficult to obtain. At the same time, the decode-encode network structure used in the method may lose image details when restoring image resolution. Summary of the invention

[0006] The purpose of the present invention is to provide an X-ray image inversion enhancement method based on deep neural network. By establishing an X-ray imaging inversion enhancement model, the image can be effectively denoised and sharpened, and it has good robustness and versatility.

[0007] In order to achieve the above purpose, the following technical solutions are adopted:

[0008] A method for inversion and enhancement of X-ray images based on deep neural network comprises the following steps:

[0009] S1: Constructing the light pipe focal spot-image effect model;

[0010] S2: Construct a noise-image effect model;

[0011] S3: Degrade the X-ray image set based on the constructed light tube focus-image effect model and noise-image effect model to obtain a degraded X-ray image set;

[0012] S4: Construct an X-ray imaging inversion model based on a convolutional neural network, and use the X-ray images before and after degradation as training data of the network to train the X-ray imaging inversion model;

[0013] S5: Based on the trained X-ray imaging inversion model, the X-ray image to be processed is inverted and enhanced.

[0014] Furthermore, the S1 specifically includes the following steps:

[0015] S11: Generate a random Gaussian kernel, where the size of the Gaussian kernel is (2k+1)×(2k+1). For each element of the Gaussian kernel, the calculation formula is as follows:

[0016] Where i and j represent the index of the kernel matrix, and δ is the standard deviation;

[0017] S12: Normalized Gaussian kernel, where the formula for normalizing the Gaussian kernel is:

[0018]

[0019] S13: Gaussian blur is performed on the X-ray image based on the Gaussian kernel to simulate the influence of different light tube focal spots on the clarity of the X-ray image.

[0020] Furthermore, in S13, the calculation formula for Gaussian blurring of the X-ray image is as follows:

[0021] Among them, I is the original image, I bluris the image after Gaussian blur, x and y are the coordinate values ​​of the image, i and j represent the index of the kernel matrix; at the same time, the light tube focus-image effect model can be described by the following formula:

[0022] I blur =f spot (I,δ,k), where δ is the standard deviation and k is the kernel radius.

[0023] Furthermore, the S2 specifically includes the following steps:

[0024] S21: Use Poisson noise to simulate quantum noise, where the probability mass function of the Poisson distribution is as follows:

[0025] Where x is the observed photon count and λ is the expected value of the photon count;

[0026] S22: Superimpose Poisson noise onto the X-ray image.

[0027] Furthermore, in S22, it is also necessary to limit the pixel grayscale value of the image after superimposing the Poisson noise, and the specific formula is:

[0028] Among them, I d is the image after superimposing Poisson noise. At the same time, the noise-image effect model can be described by the following formula: d =f noise (I,λ).

[0029] Furthermore, the S3 specifically includes the following steps:

[0030] S31: Based on the light tube focus spot-image effect model, the standard deviation σ of the appropriate Gaussian function and the kernel radius k of the Gaussian blur are randomly selected to simulate the light tube focus spot size to blur and degrade the X-ray image;

[0031] S32: For the blurred and degraded image, a noise-image effect model is used to add quantum noise to the image.

[0032] Furthermore, in S4, when constructing an X-ray imaging inversion model based on a convolutional neural network, it is necessary to replace the generative network with an RDN network based on the ESRGAN super-resolution network to make the network suitable for the current scenario.

[0033] Furthermore, in S4, the construction of training data for the X-ray imaging inversion model specifically includes the following steps:

[0034] Image dataset D using DIV2K and Flickr2K public datasets as models rgb ;

[0035] The image dataset D rgb Convert to grayscale image dataset D s ;

[0036] For the grayscale image dataset D s Perform imaging degradation to obtain the degraded image dataset D d , which can be described by the following formula:

[0037] D d (i) = f noise (f spot (D s (i),δ′,k′),λ′), where, D s (i) is the i-th image in the grayscale image dataset; D d (i) is the i-th image in the degraded image dataset; δ′ is the standard deviation of the randomly assigned Gaussian kernel, ranging from [0.2, 3]; k′ is the radius of the randomly assigned Gaussian kernel, ranging from [5, 25]; λ′ is the randomly assigned Poisson noise parameter, ranging from [0.1, 3].

[0038] Furthermore, when training the X-ray imaging inversion model, the degraded image dataset D d As the input of the imaging inversion model, the grayscale image dataset D s As the target output of the imaging inversion model; at the same time, the stochastic gradient descent optimizer was used, the learning rate was set to 0.001, the momentum parameter was set to 0.9, and the epoch was set to 2000.

[0039] By adopting the above scheme, the beneficial effects of the present invention are:

[0040] This method starts from the principle of X-ray imaging, constructs an X-ray image degradation model, and establishes an X-ray imaging inversion model based on a convolutional neural network to enhance the image. Compared with the traditional image enhancement algorithm, this method has better robustness for X-ray images, and the enhancement effect is significantly better than the traditional image enhancement algorithm. In addition, in addition to having a good enhancement effect on 2D X-ray images, this method also has a good enhancement effect on CT slice images, and has strong versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A frame of X-ray image of a battery cell;

[0042] Figure 2 This is the X-ray image of the battery cell after superimposing eight frames;

[0043] Figure 3 This is an X-ray image of a battery cell taken with a large focal spot of a light tube after stacking eight frames;

[0044] Figure 4 X-ray image of chip solder bubbles;

[0045] Figure 5 To use the X-ray imaging inversion model of the present invention Figure 4 Enhanced result diagram of

[0046] Figure 6 This is an X-ray image of the overhang of the battery cell;

[0047] Figure 7 To use the X-ray imaging inversion model of the present invention Figure 6 Enhanced result diagram of

[0048] Figure 8 This is a CT slice image of a laminated battery cell;

[0049] Fig. 9 To use the X-ray imaging inversion model of the present invention Figure 8 Enhanced result diagram of

[0050] Fig.10 It is a flow chart of the present invention;

[0051] Fig.11 This is a schematic diagram of the modified RDN network structure. DETAILED DESCRIPTION

[0052] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0053] Reference Figures 1 to 11 As shown, the present invention provides an X-ray image inversion and enhancement method based on a deep neural network, comprising the following steps:

[0054] S1: Constructing a light pipe focal spot-image effect model (constructing a light pipe focal spot and image effect model by analyzing the influence of the light pipe focal spot on the image).

[0055] The focal spot size of the X-ray tube affects the clarity of the image. A smaller focal spot can make the image have clearer edges. When the edge of the object being measured is imaged by the imaging system, it will be blurred by the focal spot. The degree of blur depends on the size of the focal spot. A larger focal spot will lead to an increase in the degree of edge blurring, thereby reducing the clarity of the image. A smaller focal spot will reduce the degree of edge blurring, making the image clearer.

[0056] Usually, the blur of the edge can be described by Gaussian blur. The standard deviation of Gaussian blur is related to the scale of the focal spot, as shown in the following formula:

[0057] σ g =γσ s , where σ s is the size of the light tube focal spot, σg is the standard deviation of Gaussian blur, and γ is the conversion coefficient, which is related to the ratio of the pixel size of the flat-panel detector to the distance from the light tube to the detection object and from the detection object to the flat-panel detector.

[0058] Therefore, different Gaussian blur coefficients can be used to perform Gaussian blur on the image to simulate the effect of different light tube focal spots on image clarity. The specific steps are as follows:

[0059] S11: Generate a random Gaussian kernel (a Gaussian kernel is a matrix generated based on a normal distribution), where the size of the Gaussian kernel is (2k+1)×(2k+1). For each element of the Gaussian kernel, the calculation formula is as follows:

[0060] Where i and j represent the index of the kernel matrix, and δ is the standard deviation;

[0061] S12: Normalized Gaussian kernel. In order to ensure that the pixel values ​​after filtering are within a reasonable range, the Gaussian kernel G(i,j) needs to be normalized so that the sum of all its elements is 1. The normalization formula is as follows:

[0062] Where G′(i,j) is the normalized Gaussian filter kernel.

[0063] S13: Gaussian blur is performed on the X-ray image based on the Gaussian kernel to simulate the influence of different light tube focal spots on the clarity of the X-ray image.

[0064] In S13, the calculation formula for Gaussian blurring of the X-ray image is as follows:

[0065] Among them, I is the original image, I blur is the image after Gaussian blur, x and y are the coordinate values ​​of the image, i and j represent the index of the kernel matrix; at the same time, the light tube focus-image effect model can be described by the following formula: blur =f spot (I,δ,k), where δ is the standard deviation and k is the kernel radius. For image I, by randomly selecting values ​​for δ and k, the effect of different light tube focal spots on image clarity can be simulated.

[0066] S2: Construct a noise-image effect model (mainly by analyzing the noise sources in the X-ray imaging process, determining the main noise of the X-ray image, and thus establishing a noise and image effect model).

[0067] In X-ray imaging systems, the main component of random noise in X-ray images is quantum noise, which originates from the discrete nature of X-rays and the counting of photons by flat-panel detectors. During X-ray imaging, X-rays are used to illuminate and interact with a target object. When X-rays interact with atoms inside the object, they are absorbed or scattered, generating one or more photons. These photons are captured by the detector and converted into electrical signals. Since the number of X-rays is discrete and the generation of photons is a random process, the photon counts observed in the image are subject to statistical fluctuations.

[0068] The S2 specifically includes the following steps:

[0069] S21: Using Poisson noise to simulate quantum noise;

[0070] In the X-ray imaging process, quantum noise can be modeled using Poisson distribution, where the probability mass function of Poisson distribution is as follows:

[0071] Among them, x is the observed photon count and λ is the expected value of the photon count. For X-ray imaging, λ represents the expected photon count at each pixel position, which is usually related to factors such as the intensity of X-rays and the absorption characteristics of the object.

[0072] S22: Superimpose Poisson noise onto the X-ray image.

[0073] For the scenario of adding Poisson distribution noise to the image grayscale, λ can be set to a suitable value, and the noise image I that obeys the Poisson distribution can be generated based on the above formula n , and then superimpose it onto image I, i.e.

[0074] I d =I+I n , where I d This is the result image after superimposing Poisson noise. In order to prevent the pixel grayscale value of the superimposed result image from overflowing, the pixel grayscale value needs to be limited, as shown in the following formula:

[0075] Among them, I d is the image after superimposing Poisson noise. At the same time, the noise-image effect model can be described by the following formula: d =f noise (I,λ), different image signal-to-noise ratios can be simulated by using different parameters.

[0076] S3: Degrade the X-ray image set based on the constructed light tube focus-image effect model and noise-image effect model to obtain a degraded X-ray image set.

[0077] Based on the two relationship models constructed above, the light tube focus-image effect model can be used to simulate the blur degradation of the original image caused by light tube focus spots of different sizes, and then the noise-image effect model can be used to add quantum noise to the blurred and degraded image, including:

[0078] S31: Based on the light tube focus spot-image effect model, the standard deviation σ of the appropriate Gaussian function and the kernel radius k of the Gaussian blur are randomly selected to simulate the light tube focus spot size to blur the X-ray image. The specific description can be:

[0079] I blur =f spot (I,δ,k), where I is the high-definition original image, I blur is a blurred degraded image.

[0080] S32: For the blurred and degraded image, a noise-image effect model is used to add quantum noise to the image, which can be specifically described as:

[0081] I d =f noise (I blur ,λ), where I d For I blur The resulting image after adding quantum noise.

[0082] S4: Construct an X-ray imaging inversion model based on a convolutional neural network, and use the X-ray images before and after degradation as training data of the network to train the X-ray imaging inversion model;

[0083] S5: Based on the trained X-ray imaging inversion model, the X-ray image to be processed is inverted and enhanced.

[0084] The ESRGAN network is a generative adversarial network for image super-resolution, which maps low-resolution images to high-resolution images, thereby improving the visual quality and details of the image. Compared with traditional super-resolution algorithms, ESRGAN introduces perceptual loss, which is a loss function that measures the perceptual quality of images based on deep learning models, and can effectively reduce the perceptual feature differences between the generated image and the target image. However, when using the ESRGAN network for image enhancement, in order to be compatible with the super-resolution function of the network, the Pixel-Unshuffle method is required to reduce the size of the input image by 0.25 times the original size, and then use the upsampling layer in the back of the network to enlarge the image size by 4 times. The Pixel-Unshuffle operation rearranges the image data so that spatially adjacent pixels are no longer adjacent, but are scattered into different channels. This reorganization may cause confusion in the network when processing these scattered information, especially in the reconstruction process, and the original spatial relationships and details may not be effectively captured. Therefore, the generative network of ESRGAN needs to be modified to be suitable for the inversion enhancement model, which is implemented as follows:

[0085] 1) Construction of Generative Model

[0086] Since the ESRGAN model needs to use the Pixel-Unshuffle method and upsampling when performing image enhancement tasks, this method may cause some deficiencies in detail recovery, so the generation network needs to be improved. Considering that RDN (Residual Dense Network) has good feature learning and expression capabilities and a lightweight design, the RDN generation network is used, and the upsampling layer in the RDN network is removed to ensure that the size of the input image is consistent with the size of the output image. The modified RDN network is shown in the figure below. Fig.11 shown.

[0087] 2) About the loss function

[0088] The loss function consists of three parts, namely perceptual loss, adversarial loss and content loss. Perceptual loss is calculated by calculating the feature distance between the inversion enhancement result image and the real image. Usually, these features are extracted from the image through a pre-trained deep learning model (such as VGG, etc.); adversarial loss measures the difference between the generated image and the real high-resolution image by training an auxiliary discriminator network, and encourages the distribution of the generated image to be as close as possible to the distribution of the real image; content loss measures the similarity between the generated image and the real high-resolution image by comparing the pixel-level difference between them; content loss can help the network retain the global structure and details of the image, thereby generating a more realistic and detailed image. Here, the present invention selects the VGG network to generate the perceptual loss function, and selects the root mean square error as the content loss function.

[0089] 3) Training data preparation

[0090] Considering that model training requires more training data and high-resolution X-ray images are difficult to obtain, the public datasets of DIV2K and Flickr2K are used as the image datasets of the model. rgb ; Since X-ray images are grayscale images, the image dataset is converted into a grayscale image D s ; For the transformed data set D s , simulate the degradation of X-ray imaging and obtain the degraded data set D d , which can be described by the following formula:

[0091] D d (i) = f noise (f spot (D s (i),δ′,k′),λ′), where, D s (i) is the i-th image in the grayscale image dataset; D d (i) is the i-th image in the degraded image dataset; δ′ is the standard deviation of the randomly assigned Gaussian kernel, ranging from [0.2, 3]; k′ is the radius of the randomly assigned Gaussian kernel, ranging from [5, 25]; λ′ is the randomly assigned Poisson noise parameter, ranging from [0.1, 3].

[0092] 4) Model training

[0093] When training the X-ray imaging inversion model, the degraded image dataset D d As the input of the imaging inversion model, the grayscale image dataset D s As the target output of the imaging inversion model; at the same time, the stochastic gradient descent optimizer was used, the learning rate was set to 0.001, the momentum parameter was set to 0.9, and the epoch was set to 2000.

[0094] In each training iteration, a small batch of images is randomly selected from the dataset, cropped to 256*256 image size, the loss is calculated through forward propagation, and the model parameters are updated through back propagation. Each epoch contains a traversal of the entire training dataset. When the number of epochs reaches 2000, the training process automatically terminates.

[0095] As mentioned above, finally, based on the trained X-ray imaging inversion model, the X-ray image to be processed is inverted and enhanced. Figure 4 and Figure 5 ,in, Figure 4 X-ray image of chip solder bubbles. Figure 5 This is the inversion enhancement result diagram of chip welding bubbles using the X-ray imaging inversion model of the present invention. It can be seen that Figure 5 The noise level is significantly reduced and the bubble edges are more distinct.

[0096] Reference Figure 6 and Figure 7 ,in, Figure 6 This is an X-ray image of the overhang of the battery cell. Figure 7 This is a diagram showing the enhancement results of the cell overhang using the X-ray imaging inversion model of the present invention. By comparison, it can be seen that: Figure 7 The contrast between the electrode and the background is greatly improved, and the noise reduction effect is obvious.

[0097] Reference Figure 8 and Fig. 9 ,in Figure 8 This is a CT slice image of a laminated battery cell. Fig. 9 The enhanced result diagram of the CT slice of the laminated battery core using the X-ray imaging inversion model of the present invention shows that: Fig. 9 The image clarity is greatly improved, and the anode-to-node contrast is obvious.

[0098] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for inversion and enhancement of X-ray images based on deep neural network, characterized in that: The steps include: S1: Constructing the light pipe focal spot-image effect model; S2: Construct a noise-image effect model; S3: Degrading the X-ray image set based on the constructed light tube focus-image effect model and noise-image effect model to obtain a degraded X-ray image set; S4: Construct an X-ray imaging inversion model based on a convolutional neural network, and use the X-ray images before and after degradation as training data of the network to train the X-ray imaging inversion model; S5: Based on the trained X-ray imaging inversion model, the X-ray image to be processed is inverted and enhanced.

2. The X-ray image inversion and enhancement method based on deep neural network according to claim 1, characterized in that: The S1 specifically includes the following steps: S11: Generate a random Gaussian kernel, where the size of the Gaussian kernel is (2k+1)×(2k+1). For each element of the Gaussian kernel, the calculation formula is as follows: Where i and j represent the index of the kernel matrix, and δ is the standard deviation; S12: Normalized Gaussian kernel, where the formula for normalizing the Gaussian kernel is: S13: Gaussian blur is performed on the X-ray image based on the Gaussian kernel to simulate the influence of different light tube focal spots on the clarity of the X-ray image.

3. The X-ray image inversion and enhancement method based on deep neural network according to claim 2, characterized in that: In S13, the calculation formula for Gaussian blurring of the X-ray image is as follows: Among them, I is the original image, I blur is the image after Gaussian blur, x and y are the coordinate values ​​of the image, i and j represent the index of the kernel matrix; at the same time, the light tube focus-image effect model can be described by the following formula: I blur =f spot (I,δ,k), where δ is the standard deviation and k is the kernel radius.

4. The X-ray image inversion and enhancement method based on deep neural network according to claim 1, characterized in that: The S2 specifically includes the following steps: S21: Use Poisson noise to simulate quantum noise, where the probability mass function of the Poisson distribution is as follows: Where x is the observed photon count and λ is the expected value of the photon count; S22: Superimpose Poisson noise onto the X-ray image.

5. The X-ray image inversion and enhancement method based on deep neural network according to claim 4 is characterized in that: In S22, it is also necessary to limit the pixel grayscale value of the image after superimposing the Poisson noise. The specific formula is: Among them, I d is the image after superimposing Poisson noise. At the same time, the noise-image effect model can be described by the following formula: d =f noise (I,λ).

6. The X-ray image inversion and enhancement method based on deep neural network according to claim 1, characterized in that: The S3 specifically includes the following steps: S31: Based on the light tube focus spot-image effect model, the standard deviation σ of the appropriate Gaussian function and the kernel radius k of the Gaussian blur are randomly selected to simulate the light tube focus spot size to blur and degrade the X-ray image; S32: For the blurred and degraded image, a noise-image effect model is used to add quantum noise to the image.

7. The X-ray image inversion and enhancement method based on deep neural network according to claim 1, characterized in that: In S4, when constructing an X-ray imaging inversion model based on a convolutional neural network, it is necessary to replace the generation network therein with an RDN network based on the ESRGAN super-resolution network to make the network suitable for the current scenario.

8. The X-ray image inversion and enhancement method based on deep neural network according to claim 7, characterized in that: In S4, the construction of the training data of the X-ray imaging inversion model specifically includes the following steps: Image dataset D using DIV2K and Flickr2K public datasets as models rgb ; The image dataset D rgb Convert to grayscale image dataset D s ; For the grayscale image dataset D s Perform imaging degradation to obtain the degraded image dataset D d , which can be described by the following formula: D d (i) = f noise (f spot (D s (i),δ′,k′),λ′), where, D s (i) is the i-th image in the grayscale image dataset; D d (i) is the i-th image in the degraded image dataset; δ′ is the standard deviation of the randomly assigned Gaussian kernel, ranging from [0.2, 3]; k′ is the radius of the randomly assigned Gaussian kernel, ranging from [5, 25]; λ′ is the randomly assigned Poisson noise parameter, ranging from [0.1, 3].

9. The X-ray image inversion and enhancement method based on deep neural network according to claim 8, characterized in that: When training the X-ray imaging inversion model, the degraded image dataset D d As the input of the imaging inversion model, the grayscale image dataset D s As the target output of the imaging inversion model; at the same time, the stochastic gradient descent optimizer was used, the learning rate was set to 0.001, the momentum parameter was set to 0.9, and the epoch was set to 2000.

Images