CBCT super-resolution method based on generative adversarial network

Abstract

The invention discloses a CBCT (Cone Beam Computed Tomography) super-resolution method based on a generative adversarial network, which relates to the technical field of medical image processing and comprises the following steps: S1, establishing a data set, acquiring CBCT image and micro-ct image data of an isolated tooth, and registering to acquire a paired image data set; s2, designing an edge loss function for the fine structure of the extracted tooth; s3, carrying out reconstruction and redesign based on an ESRGAN network architecture, and obtaining an edge generative adversarial network (Edge-SRGAN) suitable for the method; s4, training a neural network to realize CBCT super-resolution reconstruction; s5, using super-resolution CBCT to reconstruct a tooth body and a root canal system for auxiliary treatment. According to the method, the ESRGAN super-resolution network added with the edge loss function is trained to enhance the CBCT, so that the root canal edge and the fine structure are clear and sharp, the accuracy is high through verification, and the method has a good clinical application prospect.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and more particularly to a CBCT super-resolution method based on a generative adversarial network. Background Art

[0002] Cone-beam computed tomography (CBCT) is a commonly used three-dimensional image in oral clinical practice. Before a clinician performs root canal treatment on teeth with complex anatomical structures, it is usually necessary to review the patient's CBCT to understand the root canal anatomical morphology of the affected tooth so as to formulate a reasonable treatment plan. However, due to factors such as low resolution, short scanning time, and motion artifacts, CBCT has problems such as blurred pulp and crown edges, and it is difficult to accurately display the anatomical morphology of the tooth body and root canal structure. Micro-computed tomography (Micro-CT) is the gold standard for tooth morphology research. It has advantages such as high definition and high contrast and can clearly display the tooth body and root canal structure. However, Micro-CT has a high radiation dose, a long scanning time, a limited scanning space, and a high cost, and is currently only used for experimental research. Finding a method that enables CBCT to obtain the diagnostic information of Micro-CT is of great significance.

[0003] Deep Learning (DL) is a cutting-edge research direction in the field of Artificial Intelligence (AI). By constructing a neural network to combine low-level features to form more abstract high-level features, it has currently been widely used in the field of medical image processing, such as detecting disease regions, classifying diseases, segmenting, and predicting disease prognosis. The applications of deep learning in oral imaging include the detection and segmentation of dental caries and periapical lesions; the segmentation of tooth bodies and dental pulp, etc., and all have achieved high precision. At the same time, deep learning also provides a new perspective for Super-Resolution (SR) technology.

[0004] Currently, most of the research on super-resolution technology for medical images focuses on magnetic resonance imaging (MRI) and computed tomography (CT). In view of this, developing a deep learning method for in vitro tooth CBCT super-resolution and a deep learning model for segmenting and extracting tooth images in CBCT will be beneficial for clinicians to obtain more accurate tooth anatomical structures, so as to formulate a more reasonable treatment strategy for complex root canal treatment cases and improve the success rate of root canal treatment. Summary of the Invention

[0005] The object of the present invention is to provide a CBCT super-resolution method based on a generative adversarial network to solve the technical problems of low resolution and blurred images in existing CBCTs. Through this method, dentists can obtain diagnostic information on the dental structure of partial Micro-CT images from CBCT images, providing a reference for the formulation of clinical root canal treatment plans.

[0006] To achieve the above object, the present invention specifically adopts the following technical solutions:

[0007] The present invention provides a CBCT super-resolution method based on a generative adversarial network, including the following steps:

[0008] S1. Collection of extracted teeth and establishment of a dataset: Collect multiple extracted teeth, and respectively take the original CBCT images and original Micro-CT images of each extracted tooth. Register the CBCT image and the Micro-CT image of the same extracted tooth to obtain a paired image dataset, which is divided into a training set and a validation set;

[0009] S2. Design an edge loss function for the fine structure of the extracted tooth: Calculate the mean square error between the edge of the predicted image generated by the neural network and the edge of the micro-CT target image, and use this as the edge loss function, which integrates Gaussian blur technology, median blur technology, and Sobel edge detection technology;

[0010] S3. Modify and redesign based on the ESRGAN network: Simplify the number of DenseResidual Blocks in the original ESRGAN network to five layers, and adjust it in combination with the edge loss function in step S2 to obtain an edge generative adversarial network model (Edge-SRGAN);

[0011] S4. Train the neural network: Use the data in the training set to train the edge generative adversarial network model (Edge-SRGAN) obtained in step S3, adopt a multi-scale loss function for training, and use the data in the validation set to predict the edge generative adversarial network model (Edge-SRGAN) to evaluate the performance of the network, and save the trained edge generative adversarial network model (Edge-SRGAN);

[0012] S5. Use the super-resolution CBCT image to three-dimensionally reconstruct the dental and root canal structures: Input the interpolated CBCT image into the Edge-SRGAN software to obtain a picture sequence of the super-resolution CBCT image; Import the picture sequence into the VGS software for three-dimensional reconstruction, obtain the surface mask of the dental and root canal structures through threshold segmentation, and export it as an STL file;

[0013] S6. Use the MeVisLab software to overlap the Micro-CT images with the super-resolution CBCT root canal structure obtained in step S5 and visualize the differences between the two.

[0014] Specifically, to adapt to the super-resolution task of CBCT images, the original ESRGAN (Enhanced Super-Resolution Generative Adversarial Networks) network was modified and redesigned. Since the network structure of the original ESRGAN was complex and had too many layers, this led to low processing efficiency for CBCT images and was prone to overfitting problems. The Dense Residual Block layer in the original ESRGAN network was streamlined, and this adjustment effectively reduced the complexity of the model, thereby improving the performance of the model in the super-resolution task of CBCT images and reducing the risk of overfitting.

[0015] In one embodiment, in step S1, the CBCT images and Micro-CT images of the same extracted tooth are imported into the ITK-SNAP software for registration; use the rigid registration function of the ITK-SNAP software to register the CBCT images onto the Micro-CT image plane; then use linear interpolation in the rescaling step to automatically register the acquired CBCT images onto the volume of the Micro-CT images; in addition to geometric alignment, after the registration process, the CBCT images and Micro-CT images of the same extracted tooth have the same voxel size.

[0016] Specifically, rigid registration means that during the registration process, the deformation of the image or dataset is restricted within rigid body transformations, that is, it only includes rotation and translation transformations and does not include deformations such as scaling and shearing. This registration method is commonly used in medical image processing to make the two images completely consistent in spatial position and anatomical structure. The mathematical model of rigid registration usually includes rotation transformation and translation transformation. Specifically in terms of algorithms, local optimization algorithms (such as the new structure secant method) and global optimization algorithms (such as evolutionary strategies) can be used to optimize the registration function. The new structure secant method approximates the second-order information term through the secant line, avoiding the problem of matrix non-invertibility in the Gauss-Newton method, thereby improving the accuracy and efficiency of registration.

[0017] Linear interpolation scaling refers to a method of scaling an image through a linear interpolation algorithm in image processing. Linear interpolation is an interpolation method based on distance. It calculates the pixel value of the target point by weighted averaging according to the pixel values around the target point according to the distance.

[0018] In one embodiment, in step S2, the edge loss function for the fine structure design of the extracted tooth is as follows:

[0019] S21. First, use Gaussian blur technology to smooth the predicted image and the target image, and adopt median blur technology to reduce noise and highlight the main features of the image;

[0020] S22. Extract the image edges through Sobel edge detection technology;

[0021] S23. Calculate the mean square error between the edges of the predicted image and the target image, and use this as the edge loss function to guide the network optimization process.

[0022] Specifically, in CBCT images, edge features are crucial for accurate image analysis. However, micro-computed tomography (Micro-CT) images are often disturbed by machine-generated noise, which lacks semantic value. Using only the L1 loss function to constrain network training is not sufficient to effectively guide the network to focus on edge information extraction. Therefore, this solution proposes a novel image super-resolution loss function. This method can not only effectively retain the detailed information of the image but also highlight the edge features, thereby improving the performance of the network in the image super-resolution task.

[0023] In one embodiment, in step S21, the role of the Gaussian filter kernel in the Gaussian blur technology is to eliminate high-frequency details in the image, such as noise and small irrelevant details, through low-pass filtering, thereby smoothing the image and retaining larger structural features; the formula for Gaussian filtering is as follows

[0024]

[0025] In the formula, G(x, y) respectively represent the horizontal and vertical coordinates of a certain point in the filter relative to the center. σ is the standard deviation of the Gaussian function, which determines the width of the Gaussian distribution, that is, the smoothness of the filter. The larger the standard deviation, the more dispersed the distribution, and the smoother the filtering effect;

[0026] The Gaussian function has rotational symmetry, which means that in a two-dimensional space, it is symmetric about the center point, and the farther away from the center point, the smaller the weight; therefore, when the Gaussian filter processes an image, it gives higher weights to the pixels near the center and smaller weights to the pixels far from the center;

[0027] When performing Gaussian filtering on an image, a two-dimensional Gaussian kernel (also known as a Gaussian mask or convolution kernel) is usually created. It is a discretized version of the G(x, y) function on a square region. Then, this kernel is convolved with the image to smooth the image and reduce noise.

[0028] In one embodiment, in step S22, the median blur technique utilizes the non-linear processing ability of the median filter to enable it to effectively process isolated pixels with abnormal brightness. The median filter replaces the values of isolated pixel clusters with an area less than half of the filtering area with the median of the pixel grayscales in the neighborhood, thereby achieving image smoothing and noise reduction.

[0029] Specifically, due to its unique filtering effect, the median filter is widely regarded as one of the most useful statistical sorting filters in the field of image processing, and is particularly suitable for Micro-CT image processing to improve image quality and reduce noise interference.

[0030] In one embodiment, in step S23, after eliminating the noise in the image, the Sobel edge detection technique is used to extract the edge features of the image, specifically as follows:

[0031] The formula for the Sobel operator in the horizontal direction is as follows:

[0032]

[0033] Written in matrix form as follows:

[0034]

[0035] The formula for the Sobel operator in the vertical direction is as follows:

[0036]

[0037] Written in matrix form as follows:

[0038]

[0039] Each coefficient in the operator coefficient filter kernel in the horizontal and vertical directions represents the weight for the corresponding pixel in the image. In the convolution operation, the coefficients in the filter kernel are multiplied by the pixel values at the corresponding positions in the image and then summed to obtain a pixel value in the output image. The magnitude of the coefficient determines the influence degree of the original pixel value when calculating the new pixel value. Normalization processing is performed to ensure that the sum of the operator coefficients in the horizontal and vertical directions is 0, so as to obtain a generated image with more accurate edges by constraining the edge similarity between the image generated by the neural network and the micro-CT target image.

[0040] In one embodiment, in step S4, the specific steps for training the neural network to achieve CBCT super-resolution reconstruction are as follows;

[0041] In step S4, the specific steps for training the neural network to achieve CBCT super-resolution reconstruction are as follows:

[0042] S41. During the training process of the edge generative adversarial network model, a training set of high-resolution and low-resolution CBCT images was loaded. The CBCT image training set was obtained by slicing the original three-dimensional images into axial images, and the data of the axial images was normalized.

[0043] S42. The network structures of the generator and discriminator were defined, and a multi-scale loss function was used for training. The ratios of these loss functions were adjusted by hyperparameters, and the data in the validation set was used to predict the edge generative adversarial network model to evaluate the performance of the network. The trained edge generative adversarial network model was saved to achieve CBCT super-resolution reconstruction.

[0044] In one embodiment, in step S42, the multi-scale loss function includes a pixel loss function, a perceptual loss function, an adversarial loss function, and an edge loss function.

[0045] In one embodiment, in step S5, the volume size of the interpolated CBCT image is calculated by voxel conversion between the target Micro-CT image and the CBCT image. The preprocessing of the CBCT image is completed through three-dimensional linear interpolation and setting the voxel interval. The axial sequence of the interpolated CBC image is input into Edge-SRGAN for derivation to obtain a sequence of images of the super-resolution CBCT image. The sequence of images is imported into the VGS software for three-dimensional reconstruction. The surface mask of the tooth and root canal structure is obtained through threshold segmentation and exported as an STL file.

[0046] The beneficial effects of the present invention are as follows:

[0047] 1. The present invention enhances CBCT by training the ESRGAN super-resolution network with an added edge loss function, making the three-dimensional root canal edges and fine structures of ex vivo teeth clear and sharp. After verification, it has a high accuracy rate and has good clinical application prospects.

[0048] 2. The present invention obtains the CBCT images and Micro-CT images of ex vivo teeth and performs registration to obtain an image dataset. A novel edge loss is designed to improve the blurred root canal structure of CBCT images, and the structure of ESRGAN is streamlined for this task. The present invention adds the edge loss to the modified ESRGAN and names it Edge-SRGAN, and uses the established image dataset to train the model. To verify the three-dimensional reconstruction efficiency of the root canal structure of the model, the present invention combines techniques such as Otsu Threshold segmentation, root canal surface extraction, and surface distance algorithm to visualize the three-dimensional differences in the root canal system between the super-resolution CBCT image and the original Micro-CT image. Description of the Drawings

[0049] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the embodiments. It should be understood that the following drawings only show certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0050] Figure 1 This is the flowchart for the present invention to achieve CBCT super-resolution imaging.

[0051] Figure 2 These are the effect diagrams of CBCT and Micro-CT before and after registration; Figure 2 A is the effect diagram before registration; Figure 2 B is the effect diagram after registration.

[0052] Figure 3 This is the edge image obtained from the Micro-CT image using the loss function.

[0053] Figure 4 This is the flowchart for the design and use of the loss function.

[0054] Figure 5 These are the change diagrams of each loss function during the training process; Figure 5 A is the change diagram of the overall loss; Figure 5 B is the change diagram of the pixel loss; Figure 5 C is the change diagram of the perceptual loss; Figure 5 D is the change diagram of the generator loss; Figure 5 E is the change diagram of the discriminator loss; Figure 5 F is the change diagram of the edge loss.

[0055] Figure 6 These are the change diagrams of PSNR and SSIM of Edge-ESRGAN in the test set.

[0056] Figure 7 These are the comparison tables of Edge-ESRGAN with ESRGAN and bicubic interpolation in the test set.

[0057] Figure 8 This is the flowchart for network derivation.

[0058] Figure 9 These are the effect diagrams of CBCT super-resolution; Figure 9 A is the original CBCT image; Figure 9 B is the super-resolution CBCT image generated by ESRGAN; Figure 9 C is the super-resolution CBCT image generated by Edge-SRGAN; Figure 9 D is the Micro-CT image.

[0059] Figure 10 Flowchart for evaluating the three-dimensional reconstruction effect of the super-resolution CBCT root canal system.

[0060] Figure 11 Three-dimensional reconstruction effect diagram of the super-resolution CBCT root canal system; Figure 11 A is the reconstructed image of the tooth body and root canal structure of Micro-CT; Figure 11 B is the overlapping image of Edge-SRGAN and Micro-CT root canals; Figure 11 C is the overlapping image of ESRGAN and Micro-CT root canals; Figure 11 D is the overlapping image of CBCT and Micro-CT root canals. Detailed implementation manners

[0061] To make the technical problems, technical solutions, and technical effects of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Usually, the components of the embodiments of the present invention described and shown here can be arranged and designed in various different configurations.

[0062] Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed present invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0063] Embodiment 1

[0064] This embodiment provides a CBCT super-resolution method based on a generative adversarial network, including the following steps:

[0065] S1. Data acquisition and preprocessing:

[0066] S11. Collect 45 extracted teeth from the Department of Stomatology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, and take CBCT images and Micro-CT images for each extracted tooth respectively;

[0067] S12. Import the CBCT image and Micro-CT image of the same extracted tooth into the ITK-SNAP software for registration; use the rigid registration function of ITK-SNAP to register the CBCT image to the Micro-CT image layer; in the resizing step, use linear interpolation to automatically register the collected CBCT image to the Micro-CT image volume. In addition to geometric alignment, after the registration process, the two groups of images have a common voxel size. The effects before and after registration are asFigure 2 as shown

[0068] S2. Design and derive the edge loss function:

[0069] In Micro-CT images, edge features are crucial for accurate image analysis. However, Micro-CT images are often disturbed by machine-generated noise, which lacks semantic value. Using only the L1 loss function to constrain network training is not sufficient to effectively guide the network to focus on edge information extraction. Therefore, a novel image super-resolution loss function is proposed, which combines Gaussian blur technology, median blur technology, and Sobel edge detection technology;

[0070] This method first smooths the predicted image and the target image to reduce noise and highlight the main features of the image; subsequently, the image edges are extracted through Sobel edge detection technology. Finally, the mean square error between the predicted image edges and the target image edges is calculated as the loss function to guide the network optimization process; this method can not only effectively retain the detail information of the image but also highlight the edge features, thus improving the performance of the network in the image super-resolution task; specifically as follows:

[0071] S21. The role of the Gaussian filter kernel in Gaussian blur technology is to eliminate high-frequency details in the image, such as noise and small irrelevant details, through low-pass filtering, thereby smoothing the image and retaining larger structural features; by selecting appropriate size and standard deviation parameters, a more selective Gaussian kernel shape can be generated to achieve the desired filtering effect. The formula for Gaussian filtering is as follows:

[0072]

[0073] S22. In Micro-CT image processing, impulse noise is a common and intractable problem. It is generated by the shooting device, often lacks semantic information, and even after Gaussian filtering, these noises are still difficult to completely eliminate; for this reason, a median filter needs to be used to further process the image. As a statistical sorting filter, the core principle of the median filter is to sort the pixel values within the filter coverage area and replace the value of the central pixel with the median of the sorting result. This method shows significant noise reduction effects for random noise, especially impulse noise (salt-and-pepper noise), and at the same time, compared with linear smoothing filters, it can more effectively retain the edges and details of the image.

[0074] The non-linear processing ability of the median filter enables it to effectively handle those isolated pixels with abnormal brightness. For example, in a 3×3 neighborhood, the median is the fifth value, while in a 5×5 neighborhood, it is the thirteenth value. When there are multiple identical values in the neighborhood, these values are considered together. The median ξ of a set of values is the value that satisfies the following condition: half of the values in this set are less than or equal to ξ, and half are greater than or equal to ξ. To perform median filtering at a certain point in the image, it is necessary to sort the pixel values in the neighborhood, determine their median, and assign this median to the pixel corresponding to the neighborhood center in the filtered image. For example, in a 3×3 neighborhood, the median is the 5th largest value, and in a 5×5 neighborhood, the median is the 13th largest value, and so on. When there are several identical values in a neighborhood, the identical values are grouped together. For example, assume the values in a 3×3 neighborhood are (10, 20, 20, 20, 15, 20, 20, 25, 100). After sorting these values, they become (10, 15, 20, 20, 20, 20, 20, 25, 100), so the median is 20. Therefore, the main function of the median filter is to force each point to be more like its neighboring points. The mxm median filter forces the values of isolated pixel clusters (with an area less than half of the filtering area m 2 / 2) to be the median of the gray levels of these pixels in the neighborhood. The main role of the median filter is to make the values of each pixel closer to their neighboring points, thereby eliminating isolated noise points without causing significant blurring of the image. The median filter can effectively replace the values of isolated pixel clusters with an area less than half of the filtering area with the median of the gray levels of the pixels in the neighborhood, thereby achieving image smoothing and noise reduction. Due to its unique filtering effect, the median filter is widely regarded as one of the most useful statistical sorting filters in the field of image processing, especially suitable for Micro-CT image processing to improve image quality and reduce noise interference.

[0075] S23. After eliminating the noise in the image, use the sobel operator to extract the edge features of the image. The sobel operator formula in the horizontal direction is as follows:

[0076]

[0077] Written in matrix form, the formula is as follows:

[0078]

[0079] The sobel operator formula in the vertical direction is as follows:

[0080]

[0081] Written in matrix form, the formula is as follows:

[0082]

[0083] The operator coefficients in all directions are normalized to ensure that the sum of their responses in all directions is 1, without changing the brightness information of the original image.

[0084] Figure 3 The edge image obtained from the target Micro-CT image using the loss function is given. The flowchart of the loss function design and use is as Figure 4 shown. By constraining the edge similarity between the generated image and the target image, a generated image with more accurate edges can be obtained.

[0085] S3. Modification and redesign based on ESRGAN:

[0086] To adapt to the super-resolution task of CBCT images, the original ESRGAN (Enhanced Super-Resolution Generative Adversarial Networks) network was modified and redesigned. Since the network structure of the original ESRGAN is complex and has too many layers, this results in low processing efficiency for CBCT images and is prone to overfitting problems. Therefore, the DenseResidualBlock layer in ESRGAN was streamlined, and its number of layers was reduced to 5 layers. This adjustment effectively reduces the complexity of the model, thereby improving the performance of the model in the CBCT image super-resolution task and reducing the risk of overfitting. Combining with the edge loss function adjustment in step S2, an edge generative adversarial network model (Edge-SRGAN) is obtained.

[0087] S4. Train the neural network:

[0088] Use the data in the training set to train the edge generative adversarial network model (Edge-SRGAN) obtained in step S3. A multi-scale loss function is used for training, and the data in the validation set is used to predict the edge generative adversarial network model (Edge-SRGAN) to evaluate the performance of the network. Save the trained edge generative adversarial network model (Edge-SRGAN), as follows:

[0089] During the training process of the neural network, first, the high-resolution (HR) and low-resolution (LR) CBCT image datasets are loaded. The dataset is sliced from the original three-dimensional image into axial plane images. Among them, 39 extracted teeth are used in the training set, and 6 extracted teeth are used in the validation set to ensure the diversity of the validation set. Then, the data is normalized. Next, the network structures of the generator and discriminator are defined, and a multi-scale loss function is used for training. The multi-scale loss function includes pixel loss function, perceptual loss function, adversarial loss function, and edge loss function. The ratios of these loss functions are adjusted through hyperparameters.

[0090] The AdamW optimizer was used for network training, and the learning rate scheduling strategy was set to StepLR. Specifically, the initial learning rate was 0.0001, and the decay coefficients were 0.9 and 0.999. To ensure the generalization of the model, the weight decay was set to 0.0001, the step size was set to 8, and the decay rate was 0.1. After multiple iterative trainings, tensorboard was used to record the metrics and the changes in the loss function on the training set and the validation set, and it was observed that the performance of the network gradually improved. The overall comprehensive loss is composed of the pixel loss function, the perceptual loss function, the generator loss function, and the edge loss function added together with certain weights. The dynamic changes of each loss function during the training process are as Figure 5 shown. By observing the downward trend of the overall comprehensive loss, it can be clearly seen that as the training process progresses, the loss value decreases significantly, indicating that the network gradually masters the internal laws and patterns of the images during training, thereby effectively improving the quality and realism of the generated images. In the research and application of generative adversarial networks, a key challenge is that the network may not be able to spontaneously distinguish meaningful structures in the images from meaningless noise and artifacts. These meaningless elements may become "short cuts" for the network to learn, thus affecting the correct generation of the image structure. When using the VGG19 network as the feature extractor, the obtained feature maps are extremely vulnerable to interference from artifacts and noise. This results in either difficulty in capturing important semantic information in the feature maps or being filled with noise, making them insufficient as effective constraints during the network training process. To overcome this problem, in addition to the traditional pixel loss, adversarial loss, and perceptual loss, this scheme introduces a new edge loss function, aiming to strengthen the constraint on the semantic information of the image structure to guide the network to focus more on the key structural features in the generation task. By analyzing the line chart of the loss function and the SSIM evaluation metric, it can be observed that as the training process progresses, the edge loss function gradually decreases, indicating that the network pays more and more attention to the accuracy of the edge features. This optimization strategy effectively prompts the network to focus on the structural information of the image rather than being misled by irrelevant details such as noise and artifacts. It can be seen from the training process that the relationship between the edge loss and the SSIM is very close. The decrease in the edge loss means an increase in the SSIM value, indicating that using this optimization strategy of the edge loss helps to improve the overall SSIM score because it ensures the structural consistency of the image, making the generated image visually closer to the real image.

[0091] Finally, the trained network model was saved, and this model was used to predict the test data to evaluate the performance of the network. Through the analysis of the generated images, the accuracy of the edges of the images generated using the edge loss function is better than that without adding the edge loss. Figure 6 ,Figure 7 The PSNR and SSIM on the validation set respectively.

[0092] S5. Network derivation and three-dimensional reconstruction of the root canal structure from super-resolution CBCT:

[0093] The volume size of the interpolated CBCT image was calculated by voxel conversion between the target Micro-CT image and the CBCT image. The preprocessing of the CBCT image was completed through three-dimensional linear interpolation and setting the voxel interval. The axial plane sequence of the interpolated CBCT image was input into Edge-SRGAN for derivation to obtain the image sequence of the super-resolution CBCT image. Figure 8 The flowchart of the network derivation is given. The effect diagram of the super-resolution CBCT is shown in Figure 9 . As Figure 10 As shown in the given flowchart, the image sequence was imported into the VGS software for three-dimensional reconstruction. The surface mask of the hard tissue was obtained through Otsu Threshold segmentation and exported as an STL file. Subsequently, the STL file of the super-resolution CBCT image was imported into the Geomagic software to extract the surface mask of the root canal system; the same operations as above were performed on the interpolated CBCT image and the original Micro-CT image. Thus, the surface masks of the root canal systems of the CBCT image, the Micro-CT image, and the super-resolution CBCT image were obtained.

[0094] S6. Using the customized MeVisLab framework, the root canal structures of the Micro-CT image, the original CBCT image, and the super-resolution CBCT image were respectively overlapped. The "Surface / Surface Distance" function of the framework would calculate the absolute value of the deviation between the two superimposed surfaces. The color section was set according to the distance deviation of the root canal surface to visualize the differences in the root canal systems. The surface differences of the three root canal systems are shown in Figure 11 .

Claims

1. A CBCT super-resolution method based on generative adversarial network, characterized in that: The steps include: S1. Collection of ex vivo teeth and establishment of data set: Collect multiple ex vivo teeth, and take original CBCT images and original Micro-CT images of each ex vivo tooth respectively. Register the CBCT image and Micro-CT image of the same ex vivo tooth to obtain a paired image data set, which is divided into a training set and a validation set. S2. Design edge loss function for the fine structure of ex vivo tooth: Calculate the mean square error between the edge of the predicted image generated by the neural network and the edge of the micro-CT target image, and use it as the edge loss function. This edge loss function combines Gaussian blur technology, median blur technology and Sobel edge detection technology; S3, transformation and redesign based on ESRGAN network: the number of Dense ResidualBlock layers in the original ESRGAN network was reduced to five layers, and the edge generative adversarial network model was obtained by combining the edge loss function adjustment in step S2; S4, training neural network: using the data in the training set to train the edge generation adversarial network model obtained in step S3, adopting a multi-scale loss function for training, and using the data in the validation set to predict the edge generation adversarial network model to evaluate the performance of the network, and saving the trained edge generation adversarial network model; S5. Reconstruct the tooth and root canal structure using super-resolution CBCT images: Input the interpolated CBCT images into Edge-SRGAN software to obtain a sequence of super-resolution CBCT images; import the sequence of images into VGS software for three-dimensional reconstruction, obtain the surface mask of the tooth and root canal structure by threshold segmentation and export it as an STL file; S6. Using MeVisLab software, overlap the Micro-CT image with the super-resolution CBCT root canal structure obtained in step S5 to visualize the difference between the two.

2. The CBCT super-resolution method based on generative adversarial network according to claim 1, characterized in that: In step S1, the CBCT image and the Micro-CT image of the same ex vivo tooth are imported into the ITK-SNAP software for registration; the CBCT image is registered to the Micro-CT image layer using the rigid registration function of the ITK-SNAP software; then the acquired CBCT image is automatically registered to the volume of the Micro-CT image using linear interpolation in the rescaling step; in addition to geometric alignment, after the registration process, the CBCT image and the Micro-CT image of the same ex vivo tooth have a common voxel size.

3. The CBCT super-resolution method based on generative adversarial network according to claim 1, characterized in that: In step S2, the fine structure of the ex vivo tooth is designed with an edge loss function, and the specific steps are as follows: S21, firstly, the predicted image and the target image are smoothed by using Gaussian blur technology, and the median blur technology is used to reduce noise and highlight the main features of the image; S22, extracting image edges by using Sobel edge detection technology; S23. Calculate the mean square error between the edge of the predicted image and the edge of the target image, and use this as the edge loss function to guide the network optimization process.

4. The CBCT super-resolution method based on generative adversarial network according to claim 3, characterized in that: In step S21, the function of the Gaussian filter kernel in the Gaussian blur technique is to eliminate high-frequency details in the image through low-pass filtering, thereby smoothing the image and retaining larger structural features; the formula for Gaussian filtering is as follows Where G(x, y) is the pixel value at the center of the filter, x and y represent the horizontal and vertical coordinates of a point in the filter relative to the center, respectively, and σ is the standard deviation of the Gaussian function, which determines the width of the Gaussian distribution, that is, the smoothness of the filter. The larger the standard deviation, the more dispersed the distribution, and the smoother the filtering effect. The Gaussian function has rotational symmetry, which means that in two-dimensional space, it is symmetric about the center point, and the farther away from the center point, the smaller the weight; therefore, when processing an image, the Gaussian filter will give higher weights to pixels near the center, and smaller weights to pixels far from the center; When performing Gaussian filtering on an image, a two-dimensional Gaussian kernel (also called a Gaussian mask or convolution kernel) is usually created, which is a discretized version of the G(x,y) function over a square area. This kernel is then convolved with the image to smooth the image and reduce noise.

5. The CBCT super-resolution method based on generative adversarial network according to claim 3, characterized in that: In step S22, the median blur technique uses the nonlinear processing capability of the median filter to effectively process isolated pixels with abnormal brightness. The median filter replaces the values ​​of isolated pixel clusters whose area is less than half of the filter area with the median grayscale of the pixels in the neighborhood, thereby achieving image smoothing and noise reduction.

6. The CBCT super-resolution method based on generative adversarial network according to claim 3, characterized in that: In step S23, after eliminating the noise in the image, the edge features of the image are extracted using the Sobel edge detection technology, as follows: The formula of the Sobel operator in the horizontal direction is as follows: Written in matrix form as follows: The formula for the Sobel operator in the vertical direction is as follows: Written in matrix form as follows: Z1-Z9 represent adjacent 3x3 pixels, g x , g y are the operator gradient magnitudes in the horizontal and vertical directions at pixel Z5, respectively. Each coefficient in the filter kernel of the operator coefficients in the horizontal and vertical directions represents the weight of the corresponding pixel in the image. In the convolution operation, the coefficients in the filter kernel are multiplied by the pixel values ​​at the corresponding positions in the image, and then summed to obtain a pixel value in the output image. The size of the coefficient determines the degree of influence of the original pixel value when calculating the new pixel value. Normalization is performed to ensure that the sum of the operator coefficients in the horizontal and vertical directions is 0. In this way, the edge similarity between the image generated by the constrained neural network and the micro-CT target image can be obtained to obtain a generated image with more accurate edges.

7. The CBCT super-resolution method based on generative adversarial network according to claim 1, characterized in that: In step S4, the specific steps of training the neural network to achieve CBCT super-resolution reconstruction are as follows: S41. During the training process of the edge generative adversarial network model, high-resolution and low-resolution CBCT image training sets were loaded. The CBCT image training sets were segmented into axial views from the original three-dimensional images, and the axial view data were normalized. S42. The network structure of the generator and discriminator is defined, and multi-scale loss functions are used for training; the proportions of these loss functions are adjusted through hyperparameters, and the edge generative adversarial network model is predicted using the data in the validation set to evaluate the performance of the network, save the trained edge generative adversarial network model, and achieve CBCT super-resolution reconstruction.

8. The CBCT super-resolution method based on generative adversarial network according to claim 7, characterized in that: In step S42, the multi-scale loss function includes a pixel loss function, a perceptual loss function, an adversarial loss function and an edge loss function.

9. The CBCT super-resolution method based on generative adversarial network according to claim 1, characterized in that: In step S5, the volume size of the interpolated CBCT image is calculated by converting the voxels of the target Micro-CT image and the CBCT image, and the preprocessing of the CBCT image is completed by three-dimensional linear interpolation and setting the voxel interval. The axial sequence of the interpolated CBCT image is input into Edge-SRGAN for derivation to obtain the image sequence of the super-resolution CBCT image. The image sequence is imported into the VGS software for three-dimensional reconstruction, and the surface mask of the tooth and root canal structure is obtained by threshold segmentation and exported as an STL file.

Images