X-Ray image enhancement method and system based on deep learning

Abstract

The invention discloses an X-Ray image enhancement method and system based on deep learning, and belongs to the technical field of medical image processing, and the method comprises the steps: S1, constructing a training data set which comprises an X-Ray image and a CT image corresponding to the X-Ray image, and converting the X-Ray image and the CT image into DRR images; s2, constructing a first neural network, and training the first neural network by using the CT image and a DRR image converted from the CT image of the instrument appearing in the X-Ray image; s3, constructing a second neural network, and training the second neural network by using a DDR image obtained by segmenting and converting a skeleton structure in the CT image; and S4, inputting the X-Ray image of the patient into the trained first neural network, outputting the image, inputting the output image into the trained second neural network, and outputting a final image. According to the invention, the problems of large difference of X-Ray images shot by different devices and different doses, poor image quality, noise, instrument shielding and the like can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and particularly to an X-Ray image enhancement method and system based on deep learning. Background Art

[0002] Medical X-Ray images are important in modern medical diagnosis, with wide applications and many advantages. In terms of applications, in the skeletal system, it is a key tool for diagnosing orthopedic fractures. When a fracture is suspected due to trauma, doctors can use X-Ray to determine the details of the fracture and formulate treatment plans. It can also assist in diagnosing diseases such as skeletal deformities. In the respiratory system, chest X-Ray is used for physical examinations and diagnosing respiratory diseases. It can screen for pneumonia, pneumothorax, etc., and judge the disease conditions based on the image features and monitor the disease progression. In the digestive system, although there is endoscopic technology, in specific cases such as barium meal examination of the digestive tract, X-Ray can help doctors observe the contour, peristalsis, and space-occupying lesions of the digestive tract. It has the advantages of simple operation, low cost, fast imaging, etc.

[0003] However, in the current field of medical imaging, the existing methods for enhancing X-Ray images do face many intractable problems.

[0004] From the perspective of image quality, there are relatively significant shortcomings. Traditional X-Ray image enhancement means, such as some basic filtering methods, are often unable to cope when dealing with images of complex structures. Taking chest X-Ray as an example, due to the overlapping of tissues such as the heart and lungs, the clarity and distinguishability of the original image are affected to a certain extent, and the existing enhancement technologies may not be able to accurately highlight the key details, making it still difficult for the image quality to meet the requirements of accurate diagnosis after processing. For example, mean filtering can remove noise to a certain extent, but at the same time, it will blur important features such as lung textures and rib edges, resulting in a significant reduction in the overall image quality.

[0005] The noise problem cannot be underestimated either. During the X-Ray imaging process, due to various factors such as equipment performance and radiation scattering, the image is often mixed with various types of noise. Common Gaussian noise will make the image show a granular feeling, affecting doctors' observation of subtle lesions. Traditional denoising methods, such as simple median filtering, are difficult to effectively remove noise with continuous changes and a spectrum similar to the image signal, resulting in frequent noise residue problems and interfering with the diagnostic accuracy.

[0006] Instrument occlusion poses even greater challenges to X-Ray image enhancement. During actual shooting, if there are medical devices such as orthopedic implants and cardiac pacemakers in the patient's body, these devices will form obvious occlusion areas on the X-Ray image, generating artifacts. Existing enhancement techniques are difficult to perfectly remove these artifacts and restore the true image of the occluded tissue. For example, in the X-Ray images of orthopedic postoperative follow-up, the bone details around the metal fixation are often masked by artifacts, making it difficult for doctors to accurately judge the bone healing situation and bringing great difficulties to the formulation of subsequent treatment plans. These problems urgently require the help of more advanced and intelligent technologies, such as deep learning-based methods, to achieve effective breakthroughs and improve the quality and diagnostic value of X-Ray images.

[0007] The information disclosed in this background section is only intended to enhance the overall understanding of the present invention and should not be regarded as an admission or any form of suggestion that this information constitutes prior art known to those of ordinary skill in the art. Summary of the Invention

[0008] The object of the present invention is to provide a deep learning-based X-Ray image enhancement method and system, which can solve the problems that X-Ray images taken by different devices and at different doses vary greatly, and there are problems such as poor image quality, noise, and instrument occlusion.

[0009] To achieve the above object, the present invention provides a deep learning-based X-Ray image enhancement method, including the following steps:

[0010] S1: Construct a training data set, which includes high-resolution X-Ray images and their corresponding CT images at different shooting angles, different shooting devices, and different doses, and convert the X-Ray images and CT images into DRR images respectively;

[0011] S2: Construct a first neural network, and use the CT images and the DRR images converted from the CT images of the instruments appearing in the X-Ray images to train the first neural network; the first neural network is used to perform denoising and gray value distribution change operations on the input X-Ray images;

[0012] S3: Construct a second neural network, and use the DDR images obtained by segmenting and converting the bone structures in the CT images to train the second neural network; the second neural network is used to remove instrument occlusion and enhance the pixel values of the bone structures;

[0013] S4: Input the patient's X-Ray image into the trained first neural network and output the image, and then input the output image into the trained second neural network to output the final image.

[0014] In an embodiment of the present invention, in step S1, the following steps are specifically included:

[0015] S101: Obtain high-resolution X-Ray images at different shooting angles, with different shooting devices, and at different doses, and the corresponding CT images;

[0016] S102: For the CT images of each patient and the CT images of the instruments appearing in the X-Ray images, use the DRR generation algorithm to project them onto a two-dimensional plane to obtain DRR images; among them, multiple internal parameters and external parameters similar to those during the shooting of the X-Ray images are used during the projection;

[0017] S103: Use the trained deep semantic segmentation network to segment the bone structure in the CT image and output the segmentation result of the bone structure;

[0018] S104: Use the same internal parameters and external parameters in step S102 for the bone structure segmentation result in step S103 to obtain the corresponding mask DRR image, highlighting the bone structure therein, as the training data for the second neural network.

[0019] In an embodiment of the present invention, in step S104, the highlighting of the bone structure therein includes: appropriately increasing the values in the DRR image of the pixels with values greater than 0 in the mask DRR image.

[0020] In an embodiment of the present invention, the first neural network includes a style network, a generation network, and a discriminant network;

[0021] The input of the style network of the first neural network is an X-ray image or a DRR image, and the output is a one-dimensional feature vector representing the image; among them, the one-dimensional feature vector after output can act on the convolutional layer weights of the generation network, and the calculation method is as follows:

[0022] w i,j,k ′ = s i *w i,j,k

[0023] where w i,j,k is the original weight of the convolutional layer of the generation network, w i,j,k ′ is the stylized convolutional layer weight, s i is the one-dimensional feature vector output by the style network, i is the number of input channels, j is the number of output channels, and k is the convolution kernel size;

[0024] The generation network of the first neural network is a U-shaped structure, including convolutional layers that normalize the weights, and the operation of normalizing the weights is as shown below:

[0025]

[0026] where w_norm i,j,k is the weight of the convolutional layer after normalization;

[0027] The input of the discriminative network of the first neural network is the output image of the generative network or the real DRR image, and the output is 0 or 1.

[0028] In an embodiment of the present invention, the first neural network includes a generative network; the generative network in the second neural network is a U-shaped structure, including a convolutional layer for normalizing weights.

[0029] In an embodiment of the present invention, in step S2, the specific steps of training the first neural network are as follows:

[0030] S201: Set the input of the style network and the output of the generative network of the first neural network to the same DRR image obtained in step S103, and train the generative network to reconstruct the DRR image, the style network to adapt to the image input, and the discriminative network to judge the DRR image;

[0031] S202: Set the style network and the generative network input to the X-ray image of the vertically photographed human body, so that the generative network outputs a DRR image, and the discriminative network judges the DRR image;

[0032] S203: Set the style network and the generative network input to the X-ray images of all shooting angles, so that the generative network outputs a DRR image, and the discriminative network judges the DRR image.

[0033] 7. The deep learning-based X-Ray image enhancement method according to claim 5, wherein in step S2, the specific steps of training the second neural network are as follows: Train the second neural network, with the input being the DRR image obtained in step S103 and the output being the DRR image obtained in step S104.

[0034] In an embodiment of the present invention, in step S201, by calculating the L1 loss between the input and the output, the backpropagation algorithm is used to adjust the parameters of the first neural network; where the L1 loss = |predicted value - actual value|, || is to take the absolute value, the predicted value is the output of the first neural network, and the actual value is the input of the first neural network.

[0035] In an embodiment of the present invention, in steps S202 and S203, the following loss function is used to optimize the model: when the discriminative network determines that the output of the generative network is true, the output of the generative network is used as its input again, and the L1 loss is calculated for the image obtained by the secondary output and the X-Ray image; wherein, the style network outputs the same feature vector for X-ray images taken at the same shooting angle, with the same shooting device, and at the same dose.

[0036] The present invention also provides an image enhancement system based on the above-mentioned deep learning-based X-Ray image enhancement method, including:

[0037] A data acquisition module, configured to acquire high-resolution X-Ray images and their corresponding CT images taken at different shooting angles, with different shooting devices, and at different doses, and convert the X-Ray images and CT images into DRR images respectively;

[0038] A first neural network module, configured to construct a first neural network, and use the CT images and the DRR images converted from the CT images of the instruments appearing in the X-Ray images to train the first neural network; the first neural network is used to perform denoising and gray value distribution change operations on the input X-Ray images;

[0039] A second neural network module, configured to construct a second neural network, and use the DDR images obtained by segmenting and converting the bone structures in the CT images to train the second neural network; the second neural network is used to remove instrument occlusion and enhance the pixel values of the bone structures;

[0040] An inference module, configured to input the X-Ray image of the patient into the trained first neural network and output an image, and input the output image into the trained second neural network to output the final image.

[0041] Compared with the prior art, a deep learning-based X-Ray image enhancement method and system according to the present invention have the following advantages and beneficial effects:

[0042] (1) The style network is used, enabling it to be applicable to the processing of high-resolution X-Ray images taken at multiple different shooting angles, with different shooting devices, and at different doses.

[0043] (2) Through two-stage neural network inference, an image with more stable and clearer image quality and no instrument occlusion problems can be obtained after processing the X-Ray image.

[0044] (3) A convolutional layer that normalizes the weights is used in the generative network, which can accelerate the convergence speed during training and improve the quality of the output image.

[0045] (4) Through two-stage recognition, it can simulate the operation of a doctor accurately delineating each organ after magnifying the perspective during delineation, and make the network more robust in recognizing small organs.

[0046] (5) The multi-stage training process and complex loss function design can accelerate the convergence speed and improve the training stability.

[0047] (6) The present invention can improve various problems of X-Ray images and is attractive in medical scenarios such as navigation surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flowchart of a method for enhancing X-Ray images based on deep learning according to the present invention;

[0049] Figure 2 is a data relationship diagram of a method for enhancing X-Ray images based on deep learning according to the present invention;

[0050] Figure 3 is a training flowchart of the first neural network of a method for enhancing X-Ray images based on deep learning according to the present invention.

[0051] Figure 4 is a training flowchart of the second neural network of a method for enhancing X-Ray images based on deep learning according to the present invention.

[0052] Figure 5 is a framework structure diagram of the first neural network of a method for enhancing X-Ray images based on deep learning according to the present invention.

[0053] Figure 6 is a framework structure diagram of the second neural network of a method for enhancing X-Ray images based on deep learning according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0054] The following combines the drawings to describe in detail the specific embodiments of the present invention, but it should be understood that the protection scope of the present invention is not limited by the specific embodiments.

[0055] Unless otherwise clearly stated, throughout the specification and claims, the term "comprising" or its variations such as "including" or "having" etc. will be understood to include the stated elements or components, without excluding other elements or other components.

[0056] As Figures 1 to 5 shown, a method for enhancing X-Ray images based on deep learning according to a preferred embodiment of the present invention includes the following steps:

[0057] S1: Construct a training dataset, where the training dataset includes X-Ray images and their corresponding CT images under various different shooting angles, different shooting devices, and different doses, and convert the X-Ray images and CT images into DRR (Digitally Reconstructed Radiograph) images respectively.

[0058] Among them, the X-Ray images are preferably high-resolution X-Ray images, that is, X-Ray images with 1024*1024 pixels.

[0059] S2: Construct a first neural network, and use the CT images and the DRR images converted from the CT images of the instruments appearing in the X-Ray images to train the first neural network. This first neural network is used to perform denoising and gray value distribution change operations on the input X-Ray images to make their gray value distributions closer to those of the DRR images.

[0060] S3: Construct a second neural network, and use the DDR images obtained by segmenting and converting the bone structures in the CT images to train the second neural network. This second neural network is used to remove instrument occlusion and enhance the pixel values of the bone structures.

[0061] S4: Inference stage: Input the X-Ray image into the trained first neural network and output the image to achieve denoising of the X-Ray image and changing the gray value distribution to make it closer to the gray value distribution of the DRR image; then input the above output image into the trained second neural network to output the final image, achieving removal of instrument occlusion and enhancement of the pixel values of the bone structures in the image.

[0062] In this step S1, it specifically includes:

[0063] S101: Obtain high-resolution X-Ray images and their corresponding CT images under various different shooting angles, different shooting devices, and different doses as training data. The purpose of this step is to provide basic training data for the entire training process.

[0064] S102: For the CT images of each patient and the CT images of the instruments appearing in the X-Ray images, project them onto a two-dimensional plane using the DRR generation algorithm, and use multiple internal and external parameters similar to those during the X-Ray image shooting during the projection, and finally obtain the DRR images as the training data for the first neural network.

[0065] The DRR (Digital Reconstructed Radiograph) generation algorithm mainly simulates the X-ray imaging process based on CT image data. Its basic principle is the Ray-Casting technology. Specifically, starting from the virtual X-ray source position, rays are emitted along a specific direction. These rays pass through the three-dimensional volume data represented by the CT image. During the process of the rays passing through the volume data, according to information such as the attenuation coefficient of each voxel (the smallest unit in three-dimensional space), the attenuation of the rays during propagation is calculated, and finally the pixel values on the projection plane are obtained. The DRR generation algorithm can choose the Ray-Tracing method, the Distance-Driven method, the Rasterization method, etc., and different algorithms can be specifically selected according to specific situations such as image quality requirements, computing resources, and time limitations.

[0066] Similar internal parameters can generally be set as follows: the differences in focal length, principal point coordinates, and / or distortion coefficients are within the range of 5% - 10%. Similar external parameters can generally be set as follows: the differences in translation vectors and / or rotation matrices are within the range of 5% - 10%.

[0067] S103: Use the trained deep semantic segmentation network to segment the bone structure in the CT image and output the segmentation result of the bone structure. The segmentation result mask is a three-dimensional matrix, and the pixels with a value of 1 in the matrix indicate that the pixel is a bone structure, and the pixels with a value of 0 indicate that the pixel is a non-bone structure.

[0068] Among them, the trained deep semantic segmentation network is a model that can perform pixel-level segmentation of different semantic categories in an image after being trained with a large amount of data based on deep learning technology. The trained deep semantic segmentation network can choose networks such as FCN (Fully Convolutional Networks), U-Net network, SegNet network, etc., and different networks can be specifically selected according to the specific situation of the image and dataset to be segmented.

[0069] S104: Use the same internal and external parameters in step S102 for the bone structure segmentation result in step S103 to obtain the corresponding mask DRR image, highlighting the bone structure therein as the training data for the second neural network.

[0070] Among them, the pixels with a value greater than 0 in the mask DRR image indicate that they are bone structures. Appropriately increasing the value of this pixel in the DRR image, preferably increasing it by 100, can highlight the bone structure in the DRR image as the training data for the second neural network.

[0071] The training data of the present invention covers a variety of shooting conditions, endowing the model with strong generalization ability and enabling it to adapt to X-Ray images from different sources. The end-to-end learning method of deep learning reduces manual intervention. Combining with the weight-normalized convolutional layer, it speeds up the training convergence speed and improves the training efficiency and image generation quality.

[0072] In step S2, the first neural network includes a style network, a generation network, and a discriminant network.

[0073] The input of the style network (style encoder) in the first neural network is an X-ray image or a DRR image, and the output is a one-dimensional feature vector representing the image, so that the generation network (generator) can adapt to input images under different shooting angles, different shooting devices, and different doses.

[0074] The output one-dimensional feature vector acts on the convolutional layer weights of the generation network, and the calculation method is as follows:

[0075] w i,j,k ′ = s i * w i,j,k

[0076] Among them, w i,j,k is the original weight of the convolutional layer of the generation network, w i,j,k ′ is the stylized convolutional layer weight, s i is the one-dimensional feature vector output by the style network. After expanding its dimension to be consistent with w i,j,k , a *(dot product) operation is performed. i is the number of input channels, j is the number of output channels, and k is the convolutional kernel size.

[0077] The generation network (generator) in the first neural network is a U-shaped structure, including a convolutional layer that normalizes the weights. Among them, the operation of normalizing the weights is as follows:

[0078]

[0079] Among them, w_norm i,j,k is the normalized convolutional layer weight.

[0080] In the generation network of the first neural network, replacing the normalization layer in the traditional neural network with a convolutional layer that normalizes the weights can speed up the convergence speed during training and improve the quality of the output image.

[0081] The input of the discriminant network (discriminator) in the first neural network is the output image of the generation network or a real DRR image, and the output is 0 or 1. Among them, 1 means the image is real, and 0 means the image is generated.

[0082] In step S3, the second neural network includes a generation network. The generation network in the second neural network has a U-shaped structure and includes a convolutional layer that normalizes weights. The generation network in the second neural network is similar in structure to the generation network in the first neural network.

[0083] In the generation network of the second neural network, replacing the normalization layer in the traditional neural network with a convolutional layer that normalizes weights can accelerate the convergence speed during training and improve the quality of the output image.

[0084] The present invention significantly improves the quality of X-Ray images by constructing a unique dual neural network. The first neural network effectively removes noise and optimizes the gray value distribution, enhancing the image contrast; the second neural network accurately removes instrument occlusion and strengthens the bone structure, providing doctors with clearer and more accurate imaging information to assist in precise diagnosis.

[0085] In step S2, the specific steps for training the first neural network are as follows:

[0086] S201: Pre-train the first neural network: Set the input of the style network and the output of the generation network of the first neural network as the same DRR image obtained in step S103, train the generation network to reconstruct the DRR image, adapt the style network to the image input, and the discriminative network judges the DRR image.

[0087] S202: Continue to train the first neural network based on step S201. Set the style network and the generation network input as the X-ray image of the human body taken vertically, so that the generation network outputs a DRR image, and the discriminative network judges the DRR image.

[0088] S203: Continue to train the first neural network based on step S202. Set the style network and the generation network input as the X-ray images at all shooting angles, so that the generation network outputs a DRR image, and the discriminative network judges the DRR image.

[0089] In step S3, the specific steps for training the second neural network are as follows:

[0090] Train the second neural network. The input is the DRR image obtained in step S103, and the output is the DRR image obtained in step S104. There are no instruments in the DRR image obtained in this step S104, and the pixel values of the bone structure are strengthened.

[0091] In step S201 of the present invention, by calculating the L1 loss between the input and the output, the backpropagation algorithm is used to adjust the parameters of the first neural network. Where the L1 loss = |predicted value - actual value|, || represents taking the absolute value, the predicted value is the output of the first neural network in this process, and the actual value is the input of the first neural network in this process. The absolute value operation here intuitively reflects the difference between the predicted value and the actual value.

[0092] In steps S202 and S203 of the present invention, the following loss function is adopted to optimize the model: when the discriminative network determines that the output of the generative network is true, the output of the generative network is used as its input again, and the L1 loss is calculated for the image of the secondary output and the X-Ray image. Where the L1 loss = |predicted value - actual value|, || represents taking the absolute value, the predicted value is the output of the first neural network in this process, and the actual value is the input of the first neural network in this process.

[0093] Among them, it is required that the style network outputs the same feature vector for X-ray images under the same shooting angle, the same shooting device, and the same dose.

[0094] In step S3 of the present invention, the L1 loss between the input and the output is adopted as the loss function for optimizing the model. Its core formula is that the L1 loss = |predicted value - actual value|, || represents taking the absolute value, the predicted value is the output of the second neural network, and the actual value is the input of the second neural network.

[0095] The present invention constructs the loss function for multi-stage training. By setting different objectives and measurement criteria at different stages, the performance of the neural network is gradually optimized, and the L1 loss is continuously minimized, enabling it to better complete tasks such as the conversion from X-Ray images to DRR images. The loss functions at different stages have different focuses, from relatively basic difference measurement to more complex considerations such as discrimination and feature consistency.

[0096] The present invention also provides a deep learning-based X-Ray image enhancement system, including:

[0097] A data acquisition module 1, configured to acquire high-resolution X-Ray images and their corresponding CT images under different shooting angles, different shooting devices, and different doses, and convert the X-Ray images and CT images into DRR images respectively;

[0098] A first neural network module 2, configured to construct a first neural network, and use the CT images and the DRR images converted from the CT images of the instruments appearing in the X-Ray images to train the first neural network; the first neural network is used to perform denoising and gray value distribution change operations on the input X-Ray images.

[0099] The second neural network module 3 is used to construct a second neural network and train the second neural network using the DDR images obtained by segmenting and converting the bone structures in the CT images; the second neural network is used to remove instrument occlusion and enhance the pixel values of the bone structures;

[0100] The inference module 4 is used to input the X-Ray image of the patient into the trained first neural network and output an image, and then input the output image into the trained second neural network to output the final image.

[0101] The foregoing description of specific exemplary embodiments of the present invention is for purposes of illustration and exemplification. These descriptions are not intended to limit the invention to the precise forms disclosed, and it is apparent that many modifications and variations are possible in light of the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the invention and its practical applications, so that those skilled in the art can implement and utilize various different exemplary embodiments of the invention, as well as various different selections and modifications. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. A deep learning-based X-Ray image enhancement method, characterized in that: The following steps are involved: S1: construct a training data set, wherein the training data set includes high-resolution X-Ray images and their corresponding CT images at different shooting angles, different shooting devices, and different doses, and convert the X-Ray images and CT images into DRR images respectively; S2: constructing a first neural network, and using the CT image and the DRR image converted from the CT image of the device appearing in the X-Ray image to train the first neural network; the first neural network is used to perform denoising and gray value distribution operations on the input X-Ray image; S3: construct a second neural network, and use the DDR image that segments and converts the bone structure in the CT image to train the second neural network; the second neural network is used to remove instrument occlusion and enhance the pixel value of the bone structure; S4: Input the patient's X-Ray image into the trained first neural network and output the image, and input the output image into the trained second neural network to output the final image.

2. The X-Ray image enhancement method based on deep learning according to claim 1, characterized in that: Step S1 specifically includes the following steps: S101: Acquire high-resolution X-Ray images at different shooting angles, different shooting devices, and different doses, and the corresponding CT images; S102: projecting the CT image of each patient and the CT image of the device appearing in the X-Ray image into two dimensions using a DRR generation algorithm to obtain a DRR image; wherein, during the projection, a plurality of internal and external references similar to those when the X-Ray image was taken are used; S103: using the trained deep semantic segmentation network to segment the bone structure in the CT image, and outputting the segmentation result of the bone structure; S104: The bone structure segmentation result in step S103 is used with the same internal and external parameters as in step S102 to obtain a corresponding mask DRR image, highlighting the bone structure therein, as training data for the second neural network.

3. The X-Ray image enhancement method based on deep learning as claimed in claim 2, characterized in that: In step S104, highlighting the bone structure includes: appropriately increasing the values ​​of pixels in the mask DRR image whose values ​​are greater than 0 in the DRR image.

4. The X-Ray image enhancement method based on deep learning according to claim 2, characterized in that: The first neural network includes a style network, a generation network and a discrimination network; The input of the style network of the first neural network is an X-ray image or a DRR image, and the output is a one-dimensional feature vector representing the image; wherein the output one-dimensional feature vector can act on the convolutional layer weight of the generation network, and the calculation method is as follows: w i,j,k ′=s i *w i,j,k Among them, w i,j,k is the original weight of the convolutional layer of the generated network, w i,j,k ′ is the weight of the convolutional layer after stylization, s i is the one-dimensional feature vector output by the style network, i is the number of input channels, j is the number of output channels, and k is the convolution kernel size; The generating network of the first neural network is a U-shaped structure, including a convolution layer for normalizing weights, wherein the operation of normalizing weights is as follows: Among them, w_norm i,j,k is the normalized convolutional layer weight; The input of the discriminant network of the first neural network is the output image of the generating network or the real DRR image, and the output is 0 or 1.

5. The X-Ray image enhancement method based on deep learning according to claim 2, characterized in that: The first neural network includes a generating network; the generating network in the second neural network is a U-shaped structure, including a convolution layer for normalizing weights.

6. The X-Ray image enhancement method based on deep learning according to claim 4, characterized in that: In step S2, the specific steps of training the first neural network are as follows: S201: setting the input of the style network and the output of the generative network of the first neural network to the same DRR image obtained in step S103, training the generative network to reconstruct the DRR image, the style network to adapt to the image input, and the discriminative network to judge the DRR image; S202: setting the input of the style network and the generation network to an X-ray image of a human body taken vertically, so that the generation network outputs a DRR image, and the discriminant network judges the DRR image; S203: The style network and the generation network input are set to X-ray images of all shooting angles, so that the generation network outputs a DRR image, and the discrimination network judges the DRR image.

7. The X-Ray image enhancement method based on deep learning according to claim 5, characterized in that: In step S2, the specific steps of training the second neural network are as follows: the second neural network is trained, the input is the DRR image obtained in step S103, and the output is the DRR image obtained in step S104.

8. The X-Ray image enhancement method based on deep learning according to claim 6, characterized in that: In step S201, the parameters of the first neural network are adjusted by calculating the L1 loss of the input and output using the back propagation algorithm; wherein, L1 loss = |predicted value - actual value|, || is the absolute value, the predicted value is the output of the first neural network, and the actual value is the input of the first neural network.

9. The X-Ray image enhancement method based on deep learning according to claim 6, characterized in that: In steps S202 and S203, the following loss function is used to optimize the model: when the discriminant network judges that the output of the generative network is true, the output of the generative network is used as its input again, and the L1 loss is calculated for the secondary output image and the X-Ray image; wherein the style network outputs the same feature vector for X-ray images with the same shooting angle, the same shooting device, and the same dose.

10. An image enhancement system based on the deep learning based X-Ray image enhancement method according to any one of claims 1 to 9, characterized in that: include: A data acquisition module is used to acquire high-resolution X-Ray images and their corresponding CT images at different shooting angles, different shooting devices, and different doses, and convert the X-Ray images and CT images into DRR images respectively; A first neural network module is used to construct a first neural network and use a CT image and a DRR image converted from the CT image of the device appearing in the X-Ray image to train the first neural network; the first neural network is used to perform denoising and gray value distribution operations on the input X-Ray image; A second neural network module is used to construct a second neural network and use a DDR image that segments and converts the bone structure in the CT image to train the second neural network; the second neural network is used to remove instrument occlusion and enhance the pixel value of the bone structure; The inference module is used to input the patient's X-Ray image into the trained first neural network and output the image, and input the output image into the trained second neural network to output the final image.

Images