A Cycle-EEM-GAN Network for CBCT Image Enhancement

Abstract

This invention relates to a Cycle-EEM-GAN network for CBCT image enhancement, comprising a CBCT-CT generator. The CBCT-CT generator transforms CBCT images into CBCT-enhanced images with CT effects. Its parameters are optimized through the following training process: the CBCT-CT generator transforms a preprocessed CBCT image into a first CT-effect image; a CT discriminator calculates a first loss function based on the style difference between the first CT-effect image and the preprocessed CT image; the value of the first loss function is backpropagated to optimize the parameters of the CT discriminator and the CBCT-CT generator; the CT-CBCT generator transforms the first CT-effect image into a first CBCT-effect image; the difference between the first CBCT-effect image and the preprocessed CBCT image is compared, and a second loss function is calculated; the value of the second loss function is backpropagated to optimize the parameters of the CT-CBCT generator and the CBCT-CT generator. The Cycle-EEM-GAN network of this invention can enhance the quality of CBCT images to the level of CT images and enhances the network's ability to process image details by introducing an EEM module.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, and in particular to a Cycle-EEM-GAN network for CBCT image enhancement. Background Technology

[0002] Cone-beam computed tomography (CBCT) has been widely used in medical diagnostics. The difference between CBCT and CT imaging lies in the beam pattern: CBCT uses cone-beam scanning, while CT uses fan-beam scanning. Compared to CT, CBCT can obtain volumetric information of human organs or tissues more quickly with lower exposure. However, compared to CT images, CBCT images have relatively lower quality and resolution, making them difficult to use directly in clinical diagnosis. Therefore, CBCT images need to be enhanced to achieve the same image quality as CT images.

[0003] Existing methods for improving the quality of CBCT images mainly include:

[0004] 1. Filtering-based methods: These include traditional Gaussian and median filtering. Filtering-based methods can reduce noise in images, but they cannot significantly improve image quality.

[0005] 2. Reconstruction algorithm-based methods: such as filtering and backprojection. Reconstruction algorithm-based methods can improve the image reconstruction process, resulting in some improvement in image quality. However, they can also affect image resolution and detail to some extent. The aforementioned filtering and reconstruction algorithm-based methods are simple to implement and have fast computation speeds; however, for CBCT images, which contain excessive noise and interference, the effectiveness of these methods in improving image quality is not ideal.

[0006] 3. Deep learning-based methods: including Convolutional Neural Networks (CNNs). Deep learning-based methods can train a deep learning model that transforms low-quality images into high-quality ones by learning from a series of low-quality and high-quality images. CBCT images can then be input into this model to improve image quality. However, existing deep learning-based methods cannot effectively reproduce more image details, especially edge information, which is easily lost due to convolution. Summary of the Invention

[0007] Based on this, the purpose of the present invention is to provide a Cycle-EEM-GAN network for CBCT image enhancement with more complete image enhancement effect.

[0008] A Cycle-EEM-GAN network for CBCT image enhancement includes: a CBCT-CT generator, a CT discriminator, a first parameter optimization unit, a CT-CBCT generator, a second loss calculation unit, and a second parameter optimization unit; the CBCT-CT generator transforms CBCT images into CBCT-enhanced images with CT effects; wherein, the parameters of the CBCT-CT generator are optimized through the following training process:

[0009] The CBCT-CT generator converts a batch of preprocessed CBCT images into a first CT effect image; the preprocessed CBCT images are multiple CBCT images obtained through standardization processing.

[0010] The CT discriminator determines the probability that the first CT effect image is a real CT image based on the style difference between the first CT effect image and the preprocessed CT image, and calculates a first loss function; the preprocessed CT image is multiple CT images obtained through standardization processing.

[0011] The first parameter optimization unit backpropagates the value of the first loss function to optimize the parameters of the CT discriminator and the CBCT-CT generator;

[0012] The CT-CBCT generator converts the first CT effect image into a first CBCT effect image;

[0013] The second loss calculation unit compares the difference between the first CBCT effect image and the preprocessed CBCT image and calculates the second loss function;

[0014] The second parameter optimization unit backpropagates the value of the second loss function to optimize the parameters of the CT-CBCT generator and the CBCT-CT generator;

[0015] The parameters of the CBCT-CT generator, CT discriminator, and CT-CBCT generator are updated once for each batch of preprocessed CBCT images and preprocessed CT images, until the parameter update count reaches the set number of training iterations.

[0016] The Cycle-EEM-GAN network for CBCT image enhancement of the present invention can highlight the structural information of CBCT images and improve the quality of CBCT images to the level of CT images.

[0017] Furthermore, the Cycle-EEM-GAN network for CBCT image enhancement of the present invention also includes a CBCT discriminator, a third parameter optimization unit, a fourth parameter optimization unit, and a fourth loss calculation unit; the parameter training process of the CBCT-CT generator, on the other hand, includes the following steps:

[0018] The CT-CBCT generator converts a batch of preprocessed CT images into a second CBCT effect image.

[0019] The CBCT discriminator determines the probability that the second CBCT effect image is a real CBCT image based on the style difference between the second CBCT effect image and the preprocessed CBCT image, and calculates a third loss function.

[0020] The third parameter optimization unit backpropagates the value of the third loss function to optimize the parameters of the CBCT discriminator and the CT-CBCT generator.

[0021] The CBCT-CT generator converts the second CBCT effect image into a second CT effect image;

[0022] The fourth loss calculation unit compares the difference between the second CT effect image and the preprocessed CT image and calculates the fourth loss function.

[0023] The fourth parameter optimization unit backpropagates the value of the fourth loss function to optimize the parameters of the CT-CBCT generator and the CBCT-CT generator.

[0024] The CT-CBCT generator converts a batch of preprocessed CT images into a second CBCT effect image.

[0025] The CBCT discriminator determines the probability that the second CBCT effect image is a real CBCT image based on the style difference between the second CBCT effect image and the preprocessed CBCT image, and calculates a third loss function.

[0026] The third parameter optimization unit backpropagates the value of the third loss function to optimize the parameters of the CBCT discriminator and the CT-CBCT generator.

[0027] The CBCT-CT generator converts the second CBCT effect image into a second CT effect image;

[0028] The fourth loss function calculation unit compares the difference between the second CT effect image and the preprocessed CT image to calculate the fourth loss function;

[0029] The fourth parameter optimization unit backpropagates the value of the fourth loss function to optimize the parameters of the CT-CBCT generator and the CBCT-CT generator.

[0030] The parameters of the CBCT-CT generator, CT discriminator, CT-CBCT generator, and CBCT discriminator are updated once for each batch of preprocessed CBCT images and preprocessed CT images, until the number of parameter updates reaches the set number of training iterations.

[0031] Furthermore, the structure of the CBCT-CT generator is as follows: it includes a first encoder and a first decoder; the first encoder is an EEM network and the first decoder is a convolutional network; the EEM network includes several EEM layers, and each EEM layer includes at least 6 convolutional layers, stacking layers and activation layers.

[0032] Furthermore, the structure of the CT-CBCT generator is as follows: it includes a second encoder and a second decoder; the second encoder is an EEM network and the second decoder is a convolutional network; the EEM network includes several EEM layers, and each EEM layer includes at least 6 convolutional layers, stacking layers and activation layers.

[0033] Furthermore, the step of the CBCT-CT generator converting the CBCT image into a CBCT-enhanced image with CT effect includes: inputting the CBCT image into each EEM layer of the first encoder of the CBCT-CT generator; extracting features from the CBCT image through the six convolutional layers of each EEM layer to obtain six feature images; superimposing the six feature images and the input CBCT image into a superimposed image; multiplying the superimposed image by an activation function to obtain an enhanced image; using the enhanced image obtained from the previous EEM layer as the input CBCT image for the next EEM layer, and repeating the above steps; using the enhanced image output from the last EEM layer as the final enhanced image, and inputting the final enhanced image into the convolutional network of the first decoder for decoding to obtain a CBCT-enhanced image with CT effect.

[0034] This invention also provides a CBCT image enhancement model for converting CBCT images into CBCT enhanced images with CT effects; its parameters are trained through the following steps:

[0035] ST311, through the CBCT image enhancement model, transforms a batch of preprocessed CBCT images into a first CT effect image; the preprocessed CBCT images are multiple CBCT images obtained through standardization processing, which are divided into several batches and input into the CBCT image enhancement model.

[0036] ST312a uses a CT discriminator to determine the probability that the first CT effect image is a real CT image based on the style difference between the first CT effect image and the preprocessed CT image, and calculates a first loss function; the preprocessed CT image is a number of CT images obtained through standardization, which are divided into several batches and the number of images in each batch is equal to the number of images in each batch of the preprocessed CBCT image.

[0037] ST312b backpropagates the value of the first loss function to optimize the parameters of the CT discriminator and the CBCT image enhancement model;

[0038] ST313a converts the first CT effect image into a first CBCT effect image using a CT-CBCT image conversion model.

[0039] ST313b: Compare the difference between the first CBCT result image and the preprocessed CBCT image, and calculate the second loss function;

[0040] ST313c, backpropagating the value of the second loss function to optimize the parameters of the CT-CBCT image conversion model and the CBCT image enhancement model;

[0041] For each batch of preprocessed CBCT images and preprocessed CT images, the parameters of the CBCT image enhancement model, CT discriminator, and CT-CBCT image conversion model are updated once, until the number of parameter updates reaches the set number of training iterations.

[0042] Furthermore, the parameter training process of the CBCT image enhancement model includes the following steps on the other side:

[0043] ST321, through the CT-CBCT image conversion model, converts a batch of preprocessed CT images into second CBCT effect images;

[0044] ST322a uses a CBCT discriminator to determine the probability that the second CBCT effect image is a real CBCT image based on the style difference between the second CBCT effect image and the preprocessed CBCT image, and calculates a third loss function.

[0045] ST322b backpropagates the value of the third loss function to optimize the parameters of the CBCT discriminator and the CT-CBCT image conversion model;

[0046] ST323a, through the CBCT image enhancement model, transforms the second CBCT effect image into a second CT effect image;

[0047] ST323b, compare the difference between the second CT effect image and the preprocessed CT image, and calculate the fourth loss function;

[0048] ST323c, backpropagates the value of the fourth loss function to optimize the parameters of the CT-CBCT image conversion model and the CBCT image enhancement model;

[0049] For each batch of preprocessed CBCT images and preprocessed CT images, the parameters of the CBCT image enhancement model, CT discriminator, CT-CBCT image conversion model, and CBCT discriminator are updated once, until the number of parameter updates reaches the set number of training iterations.

[0050] Furthermore, the structure of the CBCT image enhancement model is as follows: it includes a first encoder and a first decoder; the first encoder is an EEM network and the first decoder is a convolutional network; the EEM network includes several EEM layers, and each EEM layer includes at least 6 convolutional layers, stacking layers and activation layers.

[0051] Furthermore, the structure of the CT-CBCT image conversion model is as follows: it includes a second encoder and a second decoder; the second encoder is an EEM network and the second decoder is a convolutional network; the EEM network includes several EEM layers, and each EEM layer includes at least 6 convolutional layers, stacking layers and activation layers.

[0052] Furthermore, the steps of the CBCT image enhancement model to transform a CBCT image into a CBCT enhanced image with CT effect include: inputting the CBCT image into each EEM layer of the first encoder of the CBCT-CT generator; extracting features from the CBCT image using the six convolutional layers of each EEM layer to obtain six feature images; superimposing the six feature images and the input CBCT image into a superimposed image; multiplying the superimposed image by an activation function to obtain an enhanced image; using the enhanced image obtained from the previous EEM layer as the input CBCT image for the next EEM layer, and repeating the above steps; using the enhanced image output from the last EEM layer as the final enhanced image, and inputting the final enhanced image into the convolutional network of the first decoder for decoding to obtain a CBCT enhanced image with CT effect.

[0053] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of the Cycle-EEM-GAN network module for CBCT image enhancement according to the present invention;

[0055] Figure 2 This is a schematic diagram of the data flow on the first side of the training method for the Cycle-EEM-GAN network of the present invention;

[0056] Figure 3 This is a schematic diagram of the data flow on the second side of the training method for the Cycle-EEM-GAN network of the present invention;

[0057] Figure 4 This is a schematic diagram of the computation process of the CBCT-CT generator of the Cycle-EEM-GAN network of the present invention;

[0058] Figure 5 This example shows a CBCT image and a CBCT-enhanced image of the CT effect obtained after image enhancement.

[0059] Figure 6 This is another CBCT image in this embodiment and the CBCT enhanced image of the CT effect obtained after image enhancement. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0061] Compared to traditional convolutional neural networks, generative adversarial networks (GANs) can learn from a series of low-quality images and their corresponding high-quality images to train an image enhancement model with good generalization ability, capable of generating high-quality images from low-quality images. Since CBCT images have lower quality than CT images, CBCT and CT images of the same body part can be used as training data and input into a GAN network to train an image enhancement model that converts CBCT images into CT-like images. This image enhancement model can improve the quality of CBCT images to the level of CT images. Cycle-GAN networks can enhance CBCT images to some extent, but due to the characteristics of convolution, this method loses the quality and information of image edges, causing some distortion. Therefore, this invention adds an EEM module to the Cycle-GAN network to form a Cycle-EEM-GAN network to improve the image enhancement model's ability to process edge information.

[0062] The following detailed description, with reference to the accompanying drawings, illustrates specific embodiments of the Cycle-EEM-GAN network for CBCT image enhancement according to the present invention. Please refer to the accompanying drawings. Figure 1 , Figure 1This is a schematic diagram of the Cycle-EEM-GAN network module for CBCT image enhancement according to the present invention. The Cycle-EEM-GAN network for CBCT image enhancement of the present invention includes a CBCT image enhancement model, a CT discriminator, a first parameter optimization unit, a CT-CBCT image conversion model, a second loss calculation unit, a second parameter optimization unit, a CBCT discriminator, a third parameter optimization unit, a fourth loss calculation unit, and a fourth parameter optimization unit. The CBCT image enhancement model is used to convert the input CBCT image into a CBCT enhanced image with CT effects; the other components—the CT discriminator, the first parameter optimization unit, the CT-CBCT image conversion model, the second loss calculation unit, the second parameter optimization unit, the CBCT discriminator, the third parameter optimization unit, the fourth loss calculation unit, and the fourth parameter optimization unit—are all used to train the parameters of the CBCT image enhancement model.

[0063] Furthermore, the CBCT image enhancement model is a CBCT-CT generator, and the CT-CBCT image conversion model is a CT-CBCT generator. The parameters of the CBCT-CT generator are trained using the following method:

[0064] ST1: Acquire the original CBCT image B and the original CT image T. The original CBCT image B and the original CT image T are the same number of images of the same human body part, captured by both CBCT and CT methods respectively. In this embodiment, the original CBCT image B and the original CT image T are images of the pelvic region. Since paired CBCT and CT images are difficult to obtain, the original CBCT image B and the original CT image T are not required to be images of the same patient in a one-to-one correspondence, as long as they meet the condition of the same human body part and the same number.

[0065] Each image is stored in the computer as a digital matrix. Let the original CBCT image B be B = {b1, b2, ..., b...}. N}, where N is the number of CBCT images in the original CBCT image B, ranging from 1000 to 10000; where b i =(b i (1) ,b i (2) ,…,b i (n) ), i = 1, 2, ..., N, b i Let n represent the digital matrix of the i-th CBCT image, where n is the dimension, the first dimension represents the number of channels, and the second and third dimensions represent the number of pixels in the image, i.e., n = 3, b i (n)Let represent the value of the nth dimension of the i-th CBCT image. Let the original CT image be T = {t1, t2, ..., t...} N}, where N is the number of CT images in the original CT image T; where t i =(t i (1) ,t i (2) ,…,t i (n) ), i = 1, 2, ..., N, t i Let t represent the digital matrix of the i-th CT image, where n is the dimension, the first dimension represents the number of channels, and the second and third dimensions represent the number of pixels in the image, i.e., n = 3. i (n) This represents the value of the nth dimension of the i-th CT image. In this embodiment, the number of CBCT images contained in the original CBCT image B and the number of CT images N contained in the original CT image T are both 3380.

[0066] ST2, preprocessing the original CBCT image B and the original CT image T to obtain a preprocessed CBCT image B' and a preprocessed CT image T'. The preprocessing includes standardization. In this embodiment, the standardization is max-min normalization, which normalizes the values ​​of the digital matrix of all images to the range of -1 to 1, where the maximum value in the digital matrix corresponds to 1 and the minimum value corresponds to -1.

[0067] ST3, the preprocessed CBCT image B' and preprocessed CT image T' are used as training data and input into the Cycle-EEM-GAN network for CBCT image enhancement of the present invention in batches to train the network parameters. In this embodiment, the number of images in a batch is 128, that is, each batch inputs 128 CBCT images from preprocessed CBCT image B' and 128 CT images from preprocessed CT image T' into the Cycle-EEM-GAN network.

[0068] Please see Figure 2-3 , Figure 2 This is a schematic diagram of the data flow on the first side of the training method for the Cycle-EEM-GAN network according to the present invention. Figure 3 This is a schematic diagram of the data flow on the second side of the training method for the Cycle-EEM-GAN network according to the present invention. When the preprocessed CBCT image B' and the preprocessed CT image T' are input into the Cycle-EEM-GAN network, each module of the Cycle-EEM-GAN network simultaneously performs the following steps on both sides:

[0069] First side:

[0070] The CBCT-CT generator executes step ST311: converting a batch of preprocessed CBCT images B' into a first CT effect image T_gen'.

[0071] Specifically, the CBCT-CT generator includes an encoder and a decoder; the encoder is an EEM network, and the decoder is a convolutional network. The EEM network is a convolutional neural network with an added EEM module. The EEM module, short for Embedded Excitation and Convolution module, aims to perform global context awareness, feature importance filtering, and inter-channel interactions on feature maps, thereby enhancing the model's visual expressiveness and feature extraction capabilities. Furthermore, the EEM module simultaneously employs multiple dilated convolutions of different sizes, expanding the receptive field and enhancing the ability to grasp the spatial relationships of feature maps.

[0072] The EEM network comprises z EEM layers, where z ranges from 3 to 10. Each layer includes at least six convolutional layers, stacking layers, and activation layers. Let conv be the m-th convolutional layer of the k-th EEM layer. m (k) m = 1, 2, ..., 6, k = 1, 2, ..., z. The parameters of the 6 convolutional layers are as follows: conv1 (k) The kernel size has two dimensions, one with a range of 4 to 16, and the other with a range of 4 to 16. The stride, padding, and dilation are both 1. (k) The kernel size has two dimensions, one with a range of 4 to 16, and the other with a range of 4 to 16. The stride is 1, the padding is 2, and the dilation is 2. (conv3) (k) The kernel size has two dimensions, one with a range of 4 to 16, and the other with a range of 4 to 16. The stride is 1, the padding is 4, and the dilation is 4. (conv4) (k) The kernel size has two dimensions, one with a range of 4 to 16, and the other with a range of 4 to 16. The stride is 1, the padding is 8, and the dilation is 8. (conv5) (k) The kernel size has two dimensions, one with a value between 4 and 16, and the other with a value between 4 and 16. The stride is 1, the padding is 16, and the dilation is 16. (conv6) (k)The kernel size has a first dimension ranging from 4 to 16, a second dimension ranging from 4 to 16, a stride of 1, padding of 1, and dilation of 1. The number of input channels (in_channels) and the number of output channels (out_channels) of the six convolutional layers all range from 1 to 1024.

[0073] In this embodiment, z = 5; conv1 (k) The convolution kernel size has a first dimension of 4 and a second dimension of 4; conv2 (k) The convolution kernel size has a first dimension of 4 and a second dimension of 4; conv3 (k) The convolution kernel size has a first dimension of 4 and a second dimension of 4; conv4 (k) The convolution kernel size has a first dimension of 4 and a second dimension of 4; conv5 (k) The convolution kernel size has a first dimension of 4 and a second dimension of 4; conv6 (k) The convolution kernel size has a first dimension of 4 and a second dimension of 4; where k = 1, 2, ..., z. m (1) The number of input channels (in_channels) is 3, and the number of output channels (out_channels) is 64; conv m (2) The number of input channels (in_channels) is 64, and the number of output channels (out_channels) is 128; conv m (3) The number of input channels (in_channels) is 128, and the number of output channels (out_channels) is 256; conv m (4) The number of input channels (in_channels) is 256, and the number of output channels (out_channels) is 512; conv m (5) The number of input channels in_channels is 512, and the number of output channels out_channels is 1024; where m = 1, 2, ..., 6.

[0074] The convolutional network includes j convolutional layers, where j ranges from 3 to 10. The first dimension of the convolutional kernel of each convolutional layer ranges from 3 to 16, the second dimension ranges from 3 to 16, the number of input channels ranges from 64 to 1024, the number of output channels ranges from 3 to 512, the padding ranges from 1 to 4, and the stride ranges from 1 to 2. In this embodiment, j = 5; the kernel size of the first convolutional layer of the convolutional network is 3×3, the number of input channels is 1024, the number of output channels is 512, the padding is 1, and the stride is 2; the kernel size of the second convolutional layer of the convolutional network is 3×3, the number of input channels is 512, the number of output channels is 256, the padding is 1, and the stride is 2; the kernel size of the third convolutional layer of the convolutional network is 3×3, the number of input channels is 256, the number of output channels is 128, the padding is 1, and the stride is 2; the kernel size of the fourth convolutional layer of the convolutional network is 3×3, the number of input channels is 128, the number of output channels is 64, the padding is 1, and the stride is 2; the kernel size of the fifth convolutional layer of the convolutional network is 3×3, the number of input channels is 64, the number of output channels is 3, the padding is 3, and the stride is 1.

[0075] Please see Figure 4 , Figure 4 This is a schematic diagram of the computation flow of the CBCT-CT generator in the Cycle-EEM-GAN network of the present invention. When a CBCT image is input to each EEM layer of the CBCT-CT generator, the six convolutional layers extract features from the CBCT image to obtain six feature images; the stacking layer stacks the six feature images and the input CBCT image to obtain a stacked image; the activation layer multiplies the stacked image by an activation function to obtain an enhanced image. The enhanced image obtained from the previous EEM layer is used as the input CBCT image for the next EEM layer, and the above steps are repeated. The enhanced image output from the z-th EEM layer is used as the final enhanced image, and the final enhanced image is input into the convolutional network to obtain the CT effect image.

[0076] The CT discriminator executes step ST312a: based on the style difference between the first CT effect image T_gen' and the preprocessed CT image T', it determines the probability that the first CT effect image T_gen' is a real CT image, and calculates a first loss function. The parameter optimization unit executes step ST312b: backpropagating the value of the first loss function to optimize the parameters of the CT discriminator and the CBCT-CT generator.

[0077] Specifically, the CT discriminator includes at least c convolutional layers, where c ranges from 4 to 8; the first dimension of the convolutional kernel of each convolutional layer ranges from 4 to 16, the second dimension ranges from 4 to 16, the number of input channels ranges from 3 to 512, the number of output channels ranges from 1 to 512, the padding ranges from 1 to 4, and the stride ranges from 1 to 2. In this embodiment, c = 5; the first convolutional layer of the CT discriminator has 3 input channels, 64 output channels, a kernel size of 4×4, padding of 1, and stride of 2; the second convolutional layer of the CT discriminator has 64 input channels, 128 output channels, a kernel size of 4×4, padding of 1, and stride of 2; the third convolutional layer of the CT discriminator has 128 input channels, 256 output channels, a kernel size of 4×4, padding of 1, and stride of 2; the fourth convolutional layer of the CT discriminator has 256 input channels, 512 output channels, a kernel size of 4×4, padding of 1, and stride of 1; the fifth convolutional layer of the CT discriminator has 512 input channels, 1 output channel, a kernel size of 4×4, padding of 1, and stride of 1.

[0078] The CT-CBCT generator executes step ST313a: converting the first CT effect image T_gen' into a first CBCT effect image B_cyc'. The second loss function calculation unit executes step ST313b: comparing the difference between the first CBCT effect image B_cyc' and the preprocessed CBCT image B', and calculating the second loss function. The parameter optimization unit executes step ST313c: backpropagating the value of the second loss function to optimize the parameters of the CT-CBCT generator and the CBCT-CT generator. Specifically, the structure of the CT-CBCT generator is the same as that of the CBCT-CT generator, also including an encoder and a decoder; the encoder is an EEM network, and the decoder is a convolutional network; the EEM network includes z EEM layers, where z ranges from 3 to 10, and each EEM layer includes at least 6 convolutional layers, stacking layers, and activation layers.

[0079] Second side:

[0080] The CT-CBCT generator executes step ST321: converting a batch of preprocessed CT images T' into a second CBCT effect image B_gen'.

[0081] The CBCT discriminator executes step ST322a: based on the style difference between the second CBCT effect image B_gen' and the preprocessed CBCT image B', it determines the probability that the second CBCT effect image B_gen' is a real CBCT image, and calculates a third loss function. The parameter optimization unit executes step ST322b: backpropagating the value of the third loss function to optimize the parameters of the CBCT discriminator and the CT-CBCT generator. Specifically, the structure of the CBCT discriminator is the same as that of the CT discriminator.

[0082] The CBCT-CT generator executes step ST323a: converting the second CBCT effect image B_gen' into a second CT effect image T_cyc'. The fourth loss function calculation unit executes step ST323b: comparing the difference between the second CT effect image T_cyc' and the preprocessed CT image T', and calculating the fourth loss function. The parameter optimization unit executes step ST323c: backpropagating the value of the fourth loss function to optimize the parameters of the CT-CBCT generator and the CBCT-CT generator.

[0083] For each batch of preprocessed CBCT images B' and preprocessed CT images T', the parameters of the CT discriminator and CBCT-CT generator are updated according to the first loss function, the CT-CBCT generator and CBCT-CT generator according to the second loss function, the CBCT discriminator and CT-CBCT generator according to the third loss function, and the CT-CBCT generator and CBCT-CT generator according to the fourth loss function. Then the training iteration is incremented by 1, and the next batch of preprocessed CBCT images B' and preprocessed CT images T' is processed. This process is repeated to update the parameters of the Cycle-EEM-GAN network.

[0084] ST4: Training ends when the number of training iterations reaches a set threshold; the CBCT-CT generator from the trained Cycle-EEM-GAN network is then used as the trained CBCT image enhancement model.

[0085] Furthermore, the hyperparameters controlling the training process include training batch size, training iteration threshold, optimizer, initial weights, and hidden layer activation function. Optimal hyperparameters are selected using grid search and 10-fold cross-validation. In this embodiment, the optimal hyperparameters selected by grid search and 10-fold cross-validation are: training batch size of 128; training iteration threshold of 3000; adaptive moment estimation as the optimizer; initial weights initialized using a uniform distribution; and ReLU activation function as the hidden layer activation function.

[0086] Please see Figure 5 and Figure 6 , Figure 5 This example shows a CBCT image and a CBCT-enhanced image of the CT effect obtained after image enhancement. Figure 6 This embodiment shows another CBCT image and its CBCT-enhanced image (the CT-like effect obtained after image enhancement). It is evident that the CBCT image enhancement model of this invention can effectively enhance the important structural information of CBCT images, achieving an enhancement effect close to that of CT images. The Cycle-EEM-GAN network for CBCT image enhancement of this invention can highlight the structural information of CBCT images, improving the quality of CBCT images to the level of CT images; furthermore, by introducing an EEM module, it enhances the network's ability to process image details, achieving both good structural information enhancement and better preservation of image edge information, resulting in a more complete image enhancement effect.

[0087] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A Cycle-EEM-GAN network system for CBCT image enhancement, comprising: CBCT-CT generator, CT discriminator, first parameter optimization unit, CT-CBCT generator, second loss calculation unit, and second parameter optimization unit; The CBCT-CT generator transforms CBCT images into CBCT-enhanced images with CT effects; wherein, the parameters of the CBCT-CT generator are optimized through the following training process: The CBCT-CT generator converts a batch of preprocessed CBCT images into a first CT effect image; the preprocessed CBCT images are multiple CBCT images obtained through standardization. The CBCT-CT generator includes a first encoder and a first decoder; the first encoder is an EEM network and the first decoder is a convolutional network; the EEM network includes several EEM layers, and each EEM layer includes at least 6 convolutional layers, stacking layers and activation layers. The CT discriminator determines the probability that the first CT effect image is a real CT image based on the style difference between the first CT effect image and the preprocessed CT image, and calculates a first loss function; the preprocessed CT image is multiple CT images obtained through standardization processing. The first parameter optimization unit backpropagates the value of the first loss function to optimize the parameters of the CT discriminator and the CBCT-CT generator; The CT-CBCT generator converts a first CT effect image into a first CBCT effect image; wherein, the CT-CBCT generator includes a second encoder and a second decoder; the second encoder is an EEM network, and the second decoder is a convolutional network; the EEM network includes several EEM layers, and each EEM layer includes at least 6 convolutional layers, stacking layers and activation layers; The second loss calculation unit compares the difference between the first CBCT result image and the preprocessed CBCT image, and calculates the second loss function; The second parameter optimization unit backpropagates the value of the second loss function to optimize the parameters of the CT-CBCT generator and the CBCT-CT generator; The parameters of the CBCT-CT generator, CT discriminator, and CT-CBCT generator are updated once for each batch of preprocessed CBCT images and preprocessed CT images, until the parameter update count reaches the set number of training iterations.

2. The Cycle-EEM-GAN network system for CBCT image enhancement according to claim 1, characterized in that: It also includes a CBCT discriminator, a third parameter optimization unit, a fourth parameter optimization unit, and a fourth loss calculation unit; the parameter training process of the CBCT-CT generator further includes the following steps: The CT-CBCT generator converts a batch of preprocessed CT images into a second CBCT effect image; The CBCT discriminator determines the probability that the second CBCT effect image is a real CBCT image based on the style difference between the second CBCT effect image and the preprocessed CBCT image, and calculates the third loss function; The third parameter optimization unit backpropagates the value of the third loss function to optimize the parameters of the CBCT discriminator and the CT-CBCT generator. The CBCT-CT generator converts the second CBCT effect image into a second CT effect image; The fourth loss calculation unit compares the difference between the second CT effect image and the preprocessed CT image and calculates the fourth loss function. The fourth parameter optimization unit backpropagates the value of the fourth loss function to optimize the parameters of the CT-CBCT generator and the CBCT-CT generator. The CT-CBCT generator converts a batch of preprocessed CT images into a second CBCT effect image; The CBCT discriminator determines the probability that the second CBCT effect image is a real CBCT image based on the style difference between the second CBCT effect image and the preprocessed CBCT image, and calculates the third loss function; The third parameter optimization unit backpropagates the value of the third loss function to optimize the parameters of the CBCT discriminator and the CT-CBCT generator. The CBCT-CT generator converts the second CBCT effect image into a second CT effect image; The fourth loss function calculation unit compares the difference between the second CT effect image and the preprocessed CT image and calculates the fourth loss function. The fourth parameter optimization unit backpropagates the value of the fourth loss function to optimize the parameters of the CT-CBCT generator and the CBCT-CT generator. The parameters of the CBCT-CT generator, CT discriminator, CT-CBCT generator, and CBCT discriminator are updated once for each batch of preprocessed CBCT images and preprocessed CT images, until the parameter update count reaches the set training count.

3. The Cycle-EEM-GAN network system for CBCT image enhancement according to claim 2, characterized in that: The steps of the CBCT-CT generator to convert a CBCT image into a CBCT-enhanced image with CT effect include: inputting the CBCT image into each EEM layer of the first encoder of the CBCT-CT generator; extracting features from the CBCT image using six convolutional layers of each EEM layer to obtain six feature images; stacking the six feature images and the input CBCT image to obtain a stacked image; multiplying the stacked image by an activation function to obtain an enhanced image; using the enhanced image obtained from the previous EEM layer as the input CBCT image for the next EEM layer, and repeating the above steps; using the enhanced image output from the last EEM layer as the final enhanced image, and inputting the final enhanced image into the convolutional network of the first decoder for decoding to obtain a CBCT-enhanced image with CT effect.

4. A method for using a CBCT image enhancement model to convert CBCT images into CBCT enhanced images with CT effects, characterized in that: Its parameters are trained through the following steps: ST311 uses the CBCT image enhancement model to transform a batch of preprocessed CBCT images into a first CT effect image. The preprocessed CBCT images are multiple CBCT images obtained through standardization, which are divided into several batches and input into the CBCT image enhancement model. The CBCT image enhancement model includes a first encoder and a first decoder. The first encoder is an EEM network, and the first decoder is a convolutional network. The EEM network includes several EEM layers, and each EEM layer includes at least 6 convolutional layers, stacking layers, and activation layers. ST312a uses a CT discriminator to determine the probability that the first CT effect image is a real CT image based on the style difference between the first CT effect image and the preprocessed CT image, and calculates a first loss function; the preprocessed CT image is a number of CT images obtained through standardization, which are divided into several batches and the number of images in each batch is equal to the number of images in each batch of the preprocessed CBCT image. ST312b backpropagates the value of the first loss function to optimize the parameters of the CT discriminator and the CBCT image enhancement model; ST313a converts a first CT effect image into a first CBCT effect image using a CT-CBCT image conversion model; wherein, the CT-CBCT image conversion model includes a second encoder and a second decoder; the second encoder is an EEM network, and the second decoder is a convolutional network; the EEM network includes several EEM layers, and each EEM layer includes at least 6 convolutional layers, stacking layers, and activation layers; ST313b: Compare the difference between the first CBCT result image and the preprocessed CBCT image, and calculate the second loss function; ST313c, backpropagating the value of the second loss function to optimize the parameters of the CT-CBCT image conversion model and the CBCT image enhancement model; For each batch of preprocessed CBCT images and preprocessed CT images, the parameters of the CBCT image enhancement model, CT discriminator, and CT-CBCT image conversion model are updated once, until the number of parameter updates reaches the set number of training iterations.

5. The method of using the CBCT image enhancement model according to claim 4, characterized in that: The parameter training process of the CBCT image enhancement model also includes the following steps: ST321, through the CT-CBCT image conversion model, converts a batch of preprocessed CT images into second CBCT effect images; ST322a uses a CBCT discriminator to determine the probability that the second CBCT effect image is a real CBCT image based on the style difference between the second CBCT effect image and the preprocessed CBCT image, and calculates a third loss function. ST322b backpropagates the value of the third loss function to optimize the parameters of the CBCT discriminator and the CT-CBCT image conversion model; ST323a, uses the CBCT image enhancement model to transform the second CBCT effect image into a second CT effect image; ST323b, calculates the fourth loss function by comparing the difference between the second CT effect image and the preprocessed CT image; ST323c, backpropagates the value of the fourth loss function to optimize the parameters of the CT-CBCT image conversion model and the CBCT image enhancement model; For each batch of preprocessed CBCT images and preprocessed CT images, the parameters of the CBCT image enhancement model, CT discriminator, CT-CBCT image conversion model, and CBCT discriminator are updated once, until the number of parameter updates reaches the set number of training iterations.

6. The method of using the CBCT image enhancement model according to claim 5, characterized in that: The steps of the CBCT image enhancement model to transform a CBCT image into a CBCT enhanced image with CT effect include: inputting the CBCT image into each EEM layer of the first encoder of the CBCT-CT generator; extracting features from the CBCT image using the six convolutional layers of each EEM layer to obtain six feature images; stacking the six feature images and the input CBCT image to obtain a stacked image; multiplying the stacked image by an activation function to obtain an enhanced image; using the enhanced image obtained from the previous EEM layer as the input CBCT image for the next EEM layer, and repeating the above steps; using the enhanced image output from the last EEM layer as the final enhanced image, and inputting the final enhanced image into the convolutional network of the first decoder for decoding to obtain a CBCT enhanced image with CT effect.

Images