A X-ray image inversion enhancement method based on deep neural network

Abstract

The present invention discloses a method for X-ray image inversion and enhancement based on a deep neural network, comprising the following steps: S1: constructing a light tube focal spot-image effect model; S2: constructing a noise-image effect model; S3: degrading an X-ray image set based on the constructed light tube focal spot-image effect model and the constructed noise-image effect model to obtain a degraded X-ray image set; S4: constructing an X-ray imaging inversion model based on a convolutional neural network, and using the pre-degraded X-ray images and the degraded X-ray images as training data for the network to train the X-ray imaging inversion model; S5: performing inversion enhancement on the processed X-ray images based on the trained X-ray imaging inversion model. By establishing the X-ray imaging inversion enhancement model, the present invention can effectively reduce noise and sharpen images, and has good robustness and versatility.

Description

Technical Field

[0001] The present invention relates to the technical field of X-ray image processing, and in particular to an X-ray image inversion and enhancement method based on a deep neural network. Background Art

[0002] In X-ray images, edge clarity is affected not only by the X-ray energy absorption of the object itself but also by the focal spot and noise of the tube. A smaller focal spot reduces the edge transition area and increases the edge gradient. Conversely, a larger transition area reduces the edge gradient, but the smaller the focal spot, the higher the price. Noise also affects the image's signal-to-noise ratio. As noise increases, the signal-to-noise ratio decreases, resulting in poor image quality, hindering subsequent human observation and image processing.

[0003] like Figure 1 The figure shows a frame of X-ray image of a battery cell. Figure 2 The figure shows the X-ray image of the battery cell after taking eight frames and superimposing them. Figure 3 The following is an X-ray image of a battery cell taken with a large focal spot of a light tube, after stacking eight frames. Figure 1 and Figure 2 , we can clearly see the impact of X-ray noise on image quality. Figure 1 Obviously has a lower signal-to-noise ratio, and the contrast Figure 2 and Figure 3 , it can be seen that the focal spot of the light tube affects the clarity of the image edge.

[0004] Currently, X-ray images require image preprocessing and enhancement to achieve optimal image quality. For X-ray images, the image processing process involves noise reduction and sharpening. For noise reduction, commonly used image processing methods include Gaussian filtering, median filtering, mean filtering, and other traditional image processing methods. For image sharpening, commonly used image processing methods include unsharp masking and image convolution sharpening methods based on various convolution kernels. However, these algorithms often require parameter adjustments based on actual image quality, which leads to certain limitations.

[0005] In addition, patent document CN111507918A discloses a medical X-ray image enhancement method based on deep learning technology. This method uses the VGG16 network and the DenseUnet network to build an image enhancement network. The VGG16 network is used to extract features from the input image, and the DenseUnet network is then used for multi-scale feature fusion. Finally, the enhanced result image is output. However, this method requires a large number of image pairs for model training, and high-definition X-ray images are often difficult to obtain. At the same time, the decoder-encoder network structure used in this method may lose image details when restoring image resolution. Summary of the Invention

[0006] The purpose of the present invention is to provide an X-ray image inversion and enhancement method based on deep neural network. By establishing an X-ray imaging inversion enhancement model, the image can be effectively denoised and sharpened, and it has good robustness and versatility.

[0007] In order to achieve the above objectives, the following technical solutions are adopted:

[0008] A method for inversion and enhancement of X-ray images based on a deep neural network comprises the following steps:

[0009] S1: Constructing a light pipe focal spot-image effect model;

[0010] S2: Build a noise-image effect model;

[0011] S3: Degrade the X-ray image set based on the constructed light tube focus-image effect model and noise-image effect model to obtain a degraded X-ray image set;

[0012] S4: Construct an X-ray imaging inversion model based on a convolutional neural network, and use the X-ray images before and after degradation as training data to train the X-ray imaging inversion model;

[0013] S5: Based on the trained X-ray imaging inversion model, the X-ray image to be processed is inverted and enhanced.

[0014] Furthermore, the S1 specifically includes the following steps:

[0015] S11: Generate a random Gaussian kernel, where the size of the Gaussian kernel is (2k+1)×(2k+1). For each element of the Gaussian kernel, the calculation formula is as follows:

[0016] Where i and j represent the index of the kernel matrix, and δ is the standard deviation;

[0017] S12: Normalized Gaussian kernel, where the formula for normalizing the Gaussian kernel is:

[0018]

[0019] S13: Gaussian blur is performed on the X-ray image based on the Gaussian kernel to simulate the effect of different light tube focal spots on the clarity of the X-ray image.

[0020] Furthermore, in S13, the calculation formula for Gaussian blurring of the X-ray image is as follows:

[0021] Among them, I is the original image, I bluris the image after Gaussian blurring, x and y are the coordinate values ​​of the image, and i and j represent the indexes of the kernel matrix. Meanwhile, the light tube focal spot-image effect model can be described by the following formula:

[0022] I blur =f spot (I,δ,k), where δ is the standard deviation and k is the kernel radius.

[0023] Furthermore, the S2 specifically includes the following steps:

[0024] S21: Use Poisson noise to simulate quantum noise, where the probability mass function of the Poisson distribution is as follows:

[0025] Where x is the observed photon count and λ is the expected value of the photon count;

[0026] S22: Superimpose Poisson noise onto X-ray images.

[0027] Furthermore, in S22, it is also necessary to limit the pixel grayscale value of the image after superimposing the Poisson noise. The specific formula is:

[0028] Among them, I d is the image after superimposing Poisson noise. At the same time, the noise-image effect model can be described by the following formula: d =f noise (I,λ).

[0029] Furthermore, the S3 specifically includes the following steps:

[0030] S31: Based on the light tube focal spot-image effect model, the appropriate standard deviation σ of the Gaussian function and the kernel radius k of the Gaussian blur are randomly selected to simulate the light tube focal spot size to blur the X-ray image;

[0031] S32: For the blurred and degraded image, quantum noise is added to the image using a noise-image effect model.

[0032] Furthermore, in S4, when constructing an X-ray imaging inversion model based on a convolutional neural network, it is necessary to replace the generative network with an RDN network based on the ESRGAN super-resolution network to make the network suitable for the current scenario.

[0033] Furthermore, in S4, the construction of training data for the X-ray imaging inversion model specifically includes the following steps:

[0034] Image dataset D using DIV2K and Flickr2K public datasets as models rgb ;

[0035] The image dataset D rgb Convert to grayscale image dataset D s ;

[0036] For the grayscale image dataset D s Perform imaging degradation to obtain the degraded image dataset D d , which can be described by the following formula:

[0037] D d (i) = f noise (f spot (D s (i),δ′,k′),λ′), where, D s (i) is the i-th image in the grayscale image dataset; D d (i) is the i-th image in the degraded image dataset; δ′ is the standard deviation of the randomly assigned Gaussian kernel, ranging from [0.2, 3]; k′ is the radius of the randomly assigned Gaussian kernel, ranging from [5, 25]; λ′ is the randomly assigned Poisson noise parameter, ranging from [0.1, 3].

[0038] Furthermore, when training the X-ray imaging inversion model, the degraded image dataset D d As the input of the imaging inversion model, the grayscale image dataset D s As the target output of the imaging inversion model; at the same time, the stochastic gradient descent optimizer is used, the learning rate is set to 0.001, the momentum parameter is set to 0.9, and the epoch is set to 2000.

[0039] By adopting the above scheme, the beneficial effects of the present invention are:

[0040] This method starts from the principle of X-ray imaging, constructs an X-ray image degradation model, and establishes an X-ray imaging inversion model based on convolutional neural network to enhance the image. Compared with traditional image enhancement algorithms, this method has better robustness for X-ray images and the enhancement effect is significantly better than traditional image enhancement algorithms. In addition, in addition to having a good enhancement effect on 2D X-ray images, this method also has a good enhancement effect on CT slice images, and has strong versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A frame of X-ray image of a battery cell;

[0042] Figure 2 This is the X-ray image of the battery cell after superimposing eight frames;

[0043] Figure 3 This is an X-ray image of a battery cell taken with a large focal spot of a light tube, after stacking eight frames;

[0044] Figure 4 X-ray image of chip solder bubbles;

[0045] Figure 5 To use the X-ray imaging inversion model of the present invention Figure 4 Enhanced result graph of ;

[0046] Figure 6 This is an X-ray image of the battery cell's overhang;

[0047] Figure 7 To use the X-ray imaging inversion model of the present invention Figure 6 Enhanced result graph of ;

[0048] Figure 8 This is a CT slice image of the laminated battery cell;

[0049] Figure 9 To use the X-ray imaging inversion model of the present invention Figure 8 Enhanced result graph of ;

[0050] Figure 10 It is a flow chart of the present invention;

[0051] Figure 11 This is a schematic diagram of the modified RDN network structure. DETAILED DESCRIPTION

[0052] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0053] Reference Figures 1 to 11 As shown, the present invention provides an X-ray image inversion and enhancement method based on a deep neural network, comprising the following steps:

[0054] S1: Constructing a light tube focal spot-image effect model (constructing a light tube focal spot and image effect model by analyzing the influence of the light tube focal spot on the image).

[0055] The size of the focal spot of the X-ray tube affects the clarity of the image. A smaller focal spot can make the image have clearer edges. When the edge of the object being measured passes through the imaging system for imaging, it will be affected by the focal spot and blurred. The degree of blur depends on the size of the focal spot. A larger focal spot will lead to increased edge blurring, thereby reducing image clarity. A smaller focal spot will reduce the degree of edge blurring, making the image clearer.

[0056] Typically, edge blur can be described using Gaussian blur. The standard deviation of the Gaussian blur is related to the scale of the focal spot, as shown in the following formula:

[0057] σ g =γσ s , where σ s is the size of the focal spot of the light tube, σg is the standard deviation of Gaussian blur, and γ is the conversion coefficient, which is related to the ratio of the pixel size of the flat-panel detector to the distance from the light tube to the detection object and the distance from the detection object to the flat-panel detector.

[0058] Therefore, different Gaussian blur coefficients can be used to perform Gaussian blur on the image to simulate the effect of different light tube focal spots on image clarity. The specific steps are as follows:

[0059] S11: Generate a random Gaussian kernel (a Gaussian kernel is a matrix generated based on a normal distribution). The size of the Gaussian kernel is (2k+1)×(2k+1). For each element of the Gaussian kernel, the calculation formula is as follows:

[0060] Where i and j represent the index of the kernel matrix, and δ is the standard deviation;

[0061] S12: Normalize the Gaussian kernel. To ensure that the pixel values ​​after filtering are within a reasonable range, the Gaussian kernel G(i,j) needs to be normalized so that the sum of all its elements is 1. The normalization formula is as follows:

[0062] Where G′(i,j) is the normalized Gaussian filter kernel.

[0063] S13: Gaussian blur is performed on the X-ray image based on the Gaussian kernel to simulate the effect of different light tube focal spots on the clarity of the X-ray image.

[0064] In S13, the calculation formula for Gaussian blurring of the X-ray image is as follows:

[0065] Among them, I is the original image, I blur is the image after Gaussian blur, x and y are the coordinate values ​​of the image, i and j represent the index of the kernel matrix; at the same time, the light tube focus-image effect model can be described by the following formula: blur =f spot (I, δ, k), where δ is the standard deviation and k is the kernel radius. For image I, randomly selecting values ​​for δ and k can simulate the effect of different light tube focal spots on image clarity.

[0066] S2: Construct a noise-image effect model (mainly by analyzing the noise sources in the X-ray imaging process, determining the main noise of the X-ray image, and thus establishing the noise and image effect model).

[0067] In X-ray imaging systems, the primary component of random noise in X-ray images is quantum noise, which originates from the discrete nature of X-rays and the counting of photons by flat-panel detectors. During X-ray imaging, X-rays are used to illuminate and interact with a target object. When the X-rays interact with atoms within the object, they are absorbed or scattered, generating one or more photons. These photons are captured by the detector and converted into an electrical signal. Because the number of X-rays is discrete and photon generation is a random process, the photon count observed in the image is subject to statistical fluctuations.

[0068] The S2 specifically includes the following steps:

[0069] S21: Using Poisson noise to simulate quantum noise;

[0070] In the X-ray imaging process, quantum noise can be modeled using the Poisson distribution, where the probability mass function of the Poisson distribution is as follows:

[0071] Here, x is the observed photon count and λ is the expected value of the photon count. For X-ray imaging, λ represents the expected photon count at each pixel position, which is usually related to factors such as the intensity of the X-ray and the absorption characteristics of the object.

[0072] S22: Superimpose Poisson noise onto X-ray images.

[0073] For the scenario of adding Poisson distribution noise to the image grayscale, λ can be set to a suitable value, and the noise image I that obeys the Poisson distribution can be generated based on the above formula. n , and then superimpose it into image I, i.e.

[0074] I d =I+I n , where I d This is the result image after superimposing Poisson noise. In order to prevent the pixel grayscale value of the superimposed result image from overflowing, the pixel grayscale value needs to be limited, as shown in the following formula:

[0075] Among them, I d is the image after superimposing Poisson noise. At the same time, the noise-image effect model can be described by the following formula: d =f noise (I,λ), different image signal-to-noise ratios can be simulated by using different parameters.

[0076] S3: Degrade the X-ray image set based on the constructed light tube focus-image effect model and noise-image effect model to obtain a degraded X-ray image set.

[0077] Based on the two relationship models constructed above, we can first use the light tube focus-image effect model to simulate the blur degradation of the original image caused by light tube focus spots of different sizes. Then, we use the noise-image effect model to add quantum noise to the blurred and degraded image. Specifically, we can:

[0078] S31: Based on the light tube focal spot-image effect model, we randomly select the appropriate standard deviation σ of the Gaussian function and the kernel radius k of the Gaussian blur to simulate the light tube focal spot size and blur the X-ray image. The specific description can be:

[0079] I blur =f spot (I,δ,k), where I is the high-definition original image, I blur Blurred degraded image.

[0080] S32: For the blurred and degraded image, quantum noise is added to the image using the noise-image effect model, which can be described as follows:

[0081] I d =f noise (I blur ,λ), where I d For I blur The resulting image after adding quantum noise.

[0082] S4: Construct an X-ray imaging inversion model based on a convolutional neural network, and use the X-ray images before and after degradation as training data to train the X-ray imaging inversion model;

[0083] S5: Based on the trained X-ray imaging inversion model, the X-ray image to be processed is inverted and enhanced.

[0084] The ESRGAN network is a generative adversarial network for image super-resolution. It maps low-resolution images to high-resolution images, thereby improving visual quality and detail. Compared to traditional super-resolution algorithms, ESRGAN introduces a perceptual loss, a loss function that measures the perceptual quality of images based on a deep learning model. This loss effectively reduces the difference in perceptual features between the generated and target images. However, when using the ESRGAN network for image enhancement, to ensure compatibility with the network's super-resolution capabilities, a pixel-unshuffle method is required. This method first downscales the input image by 0.25 times its original size and then uses an upsampling layer at the end of the network to upscale it by a factor of 4. The pixel-unshuffle operation rearranges the image data, distributing spatially adjacent pixels across different channels. This reshuffling can cause confusion when the network processes this scattered information, particularly during reconstruction, where the original spatial relationships and details may not be captured effectively. Therefore, the ESRGAN generative network needs to be modified to be suitable for inversion enhancement models. The specific implementation is as follows:

[0085] 1) Construction of generative model

[0086] Since the ESRGAN model needs to use the Pixel-Unshuffle method and upsampling when performing image enhancement tasks, this method may lead to some deficiencies in detail recovery, so the generative network needs to be improved. Considering that RDN (Residual Dense Network) has good feature learning and expression capabilities and a lightweight design, the RDN generative network is adopted, and the upsampling layer in the RDN network is removed to ensure that the size of the input image is consistent with the size of the output image. The modified RDN network is shown in the figure below. Figure 11 shown.

[0087] 2) About loss function

[0088] The loss function consists of three parts: perceptual loss, adversarial loss and content loss. Perceptual loss is calculated by calculating the feature distance between the inversion enhancement result image and the real image. Usually, these features are extracted from the image through a pre-trained deep learning model (such as VGG, etc.); adversarial loss measures the difference between the generated image and the real high-resolution image by training an auxiliary discriminator network, and encourages the distribution of the generated image to be as close as possible to the distribution of the real image; content loss measures the similarity between the generated image and the real high-resolution image by comparing the pixel-level differences between them; content loss can help the network retain the global structure and details of the image, thereby generating more realistic and detailed images. Here, the present invention selects the VGG network to generate the perceptual loss function, and selects the root mean square error as the content loss function.

[0089] 3) Training data preparation

[0090] Considering that model training requires more training data and high-resolution X-ray images are difficult to obtain, the public datasets of DIV2K and Flickr2K are used as the image datasets of the model. rgb Since X-ray images are grayscale images, the image dataset is converted into grayscale images D s ; For the transformed dataset D s , simulate X-ray imaging degradation and obtain the degraded data set D d , which can be described by the following formula:

[0091] D d (i) = f noise (f spot (D s (i),δ′,k′),λ′), where, D s (i) is the i-th image in the grayscale image dataset; D d (i) is the i-th image in the degraded image dataset; δ′ is the standard deviation of the randomly assigned Gaussian kernel, ranging from [0.2, 3]; k′ is the radius of the randomly assigned Gaussian kernel, ranging from [5, 25]; λ′ is the randomly assigned Poisson noise parameter, ranging from [0.1, 3].

[0092] 4) Model training

[0093] When training the X-ray imaging inversion model, the degraded image dataset D d As the input of the imaging inversion model, the grayscale image dataset D s As the target output of the imaging inversion model; at the same time, the stochastic gradient descent optimizer is used, the learning rate is set to 0.001, the momentum parameter is set to 0.9, and the epoch is set to 2000.

[0094] During each training iteration, a mini-batch of images is randomly sampled from the dataset and cropped to 256x256 pixels. The loss is calculated using forward propagation, and the model parameters are updated using backpropagation. Each epoch consists of a single pass through the entire training dataset. Training terminates automatically when the number of epochs reaches 2000.

[0095] As above, finally, based on the trained X-ray imaging inversion model, the processed X-ray image is inverted and enhanced. Figure 4 and Figure 5 ,in, Figure 4 This is an X-ray image of chip solder bubbles. Figure 5 The inversion enhancement result diagram of chip welding bubbles using the X-ray imaging inversion model of the present invention can be seen. Figure 5 The noise level is significantly reduced and the bubble edges are more distinct.

[0096] Reference Figure 6 and Figure 7 ,in, Figure 6 This is an X-ray image of the battery cell's overhang. Figure 7 This is the enhanced result of the cell overhang using the X-ray imaging inversion model of the present invention. By comparison, it can be seen that Figure 7 The contrast between the electrodes and the background is greatly improved, and the noise reduction effect is obvious.

[0097] Reference Figure 8 and Figure 9 ,in Figure 8 This is a CT slice of a laminated battery cell. Figure 9 The figure shows the enhancement result of CT slices of laminated cores using the X-ray imaging inversion model of the present invention. It can be seen that Figure 9 The image clarity is greatly improved, and the anode-to-node contrast is obvious.

[0098] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for X-ray image inversion and enhancement based on deep neural network, characterized in that: The steps include: S1: Constructing a light pipe focal spot-image effect model; S2: Build a noise-image effect model; S3: Degrade the X-ray image set based on the constructed light tube focus-image effect model and noise-image effect model to obtain a degraded X-ray image set; S4: Construct an X-ray imaging inversion model based on a convolutional neural network, and use the X-ray images before and after degradation as training data to train the X-ray imaging inversion model; S5: Based on the trained X-ray imaging inversion model, the processed X-ray image is inverted and enhanced; The S2 specifically includes the following steps: S21: Use Poisson noise to simulate quantum noise, where the probability mass function of the Poisson distribution is as follows: ,in, is the observed photon count, is the expected value of photon counts; S22: superimpose Poisson noise onto the X-ray image; In S22, it is also necessary to limit the pixel grayscale value of the image after superimposing the Poisson noise. The specific formula is: ,in, This is the image after superimposing Poisson noise. At the same time, the noise-image effect model is described by the following formula: ; The S3 specifically includes the following steps: S31: Based on the light tube focus-image effect model, randomly select the standard deviation of the appropriate Gaussian function and the kernel radius of the Gaussian blur , the X-ray image is blurred and degraded by simulating the focal spot size of the light tube; S32: For the blurred and degraded image, quantum noise is added to the image using the noise-image effect model; In S4, when constructing an X-ray imaging inversion model based on a convolutional neural network, it is necessary to replace the generative network with an RDN network based on the ESRGAN super-resolution network to make the network suitable for the current scenario.

2. The X-ray image inversion and enhancement method based on deep neural network according to claim 1, characterized in that: The S1 specifically includes the following steps: S11: Generate a random Gaussian kernel, where the size of the Gaussian kernel is , for each element of the Gaussian kernel, the calculation formula is as follows: ,in, and represents the index of the kernel matrix, is the standard deviation; S12: Normalized Gaussian kernel, where the formula for normalizing the Gaussian kernel is: ； S13: Gaussian blur is performed on the X-ray image based on the Gaussian kernel to simulate the effect of different light tube focal spots on the clarity of the X-ray image.

3. The X-ray image inversion and enhancement method based on deep neural network according to claim 2, characterized in that: In S13, the calculation formula for Gaussian blurring of the X-ray image is as follows: ,in, For the original picture, is the image after Gaussian blur, and is the coordinate value of the image, and represents the index of the kernel matrix; meanwhile, the light pipe focal spot-image effect model is described by the following formula: ,in, is the standard deviation, is the nuclear radius.

4. The X-ray image inversion and enhancement method based on deep neural network according to claim 1, characterized in that: In S4, the construction of training data for the X-ray imaging inversion model specifically includes the following steps: Image datasets using DIV2K and Flickr2K public datasets as models ; The image dataset Convert to grayscale image dataset ; Grayscale image dataset Perform imaging degradation to obtain the degraded image dataset , which is described by the following formula: ,in, is the grayscale image dataset images; is the first image in the degraded image dataset images; is the standard deviation of the randomly assigned Gaussian kernel, ranging from [0.2, 3]; is the randomly assigned Gaussian kernel radius, ranging from [5,25]; is a randomly assigned Poisson noise parameter in the range of [0.1,3].

5. The X-ray image inversion and enhancement method based on deep neural network according to claim 4, characterized in that: When training the X-ray imaging inversion model, the degraded image dataset As the input of the imaging inversion model, the grayscale image dataset As the target output of the imaging inversion model; at the same time, the stochastic gradient descent optimizer is used, the learning rate is set to 0.001, the momentum parameter is set to 0.9, and the epoch is set to 2000.

Images