Tooth CBCT image super-resolution reconstruction method based on potential diffusion model

Abstract

The invention discloses a tooth CBCT image super-resolution reconstruction method based on a potential diffusion model, and relates to an image processing and generating technology in the field of computer vision. The in-vivo medical image is trained based on hidden space coding and probability diffusion, so that the network has a better super-resolution effect on the real in-vivo tooth CBCT image.

Description

Technical Field

[0001] This invention pertains to image processing and generation techniques in the field of computer vision, specifically relating to a method for super-resolution reconstruction of dental CBCT images based on a latent diffusion model. Background Technology

[0002] Medical image super-resolution is a method that improves the resolution of medical images through sophisticated image processing and machine learning techniques. This technology is of great value in improving the quality of medical images, enhancing diagnostic accuracy, and increasing the efficiency of physicians' work.

[0003] While existing technologies such as CNNs have made significant progress in image super-resolution, several challenges remain when applying them to medical image processing. For example, training deep learning models requires a large amount of high-quality training data, but obtaining such a large volume of high-quality medical images is difficult in the medical field. Secondly, super-resolution of medical images requires handling various noises and artifacts. Furthermore, the significant differences between in-situ and extracted teeth present a considerable challenge in training models to learn the mapping between distributions. Summary of the Invention

[0004] To address the challenge of directly applying existing technologies to super-resolution reconstruction of CBCT images of in vivo teeth, this invention proposes a method for super-resolution reconstruction of CBCT images of teeth based on a latent diffusion model. This invention is achieved through the following technical solution: A method for super-resolution reconstruction of dental CBCT images based on a latent diffusion model includes the following steps: S1. Constructing the training dataset: Collect low-resolution in vivo CBCT images and out vivo CBCT images, as well as high-resolution micro-CT images of the same object, and perform dataset registration. The specific process of dataset registration is as follows: S1-1. Image Preprocessing and Export: Acquire in vivo CBCT images, out vivo CBCT images, high-resolution micro-CT images, and corresponding in vivo CBCT tooth mask images, out vivo CBCT tooth mask images, and micro-CT tooth mask images obtained through manual segmentation; export the acquired micro-CT images, in vivo CBCT images, and out vivo CBCT images into digital image formats suitable for 3D reconstruction and subsequent processing; using image processing techniques, map the in vivo CBCT tooth mask images, out vivo CBCT tooth mask images, and micro-CT tooth mask images onto the corresponding original images, and extract the tooth regions through threshold segmentation to generate point clouds containing grayscale information. This is used for subsequent image transformation; S1-2. 3D Reconstruction: Image processing software was used to process the in vivo CBCT tooth mask images, out vivo CBCT tooth mask images, and micro-CT tooth mask images obtained through manual segmentation to obtain the micro-CT mask point cloud. CBCT mask point cloud ; S1-3. Image Registration: Point cloud registration is performed using the Iterative Closest Point (ICP) algorithm. and Calculate the transformation matrix The optimal transformation matrix is ​​obtained through multiple iterations of calculation. ,use For the aforementioned point cloud containing grayscale information Transform to obtain This allows it to be spatially matched with the CBCT point cloud; finally, for Parallel projection is used to obtain micro-CT images registered with CBCT images, thus completing the construction of the training dataset; S2. Constructing the Super-Resolution Network: The super-resolution network consists of a variational autoencoder module and a diffusion network; the specific steps are as follows: S2-1. Constructing the variational autoencoder module: Define the encoder input as preprocessed and registered in vivo CBCT images, ex vivo CBCT images, and high-resolution micro-CT images; the encoder extracts features through convolutional layers, and then uses reparameterization techniques to generate latent variables. ; S2-2. Constructing the variational autoencoder decoder module: Define the decoder input as latent variables. Deconvolution is used to decode it into a reconstructed image. ; S2-3. Constructing the forward diffusion process of the latent diffusion network: Set the total number of time steps T in the diffusion process, and the model gradually diffuses towards the original image within T time steps. Noise is added; the denoising neural network predicts the noise using U-net to gradually recover the latent encoding; finally, the original image... The image is converted into a reconstructed image by a variational autoencoder decoder, thus completing the image generation process. S3: Constructing the training process and loss function: The training process consists of two parts: a variational autoencoder and a diffusion model. During training, the variational autoencoder is pre-trained first, followed by the diffusion model. The parameter update method in one iteration is as follows: S3-1. Variational Autoencoder Module Training: The training dataset is input into the variational autoencoder. The variational autoencoder receives the original image x, extracts features through multiple convolutional layers to generate a feature map, and then maps it to latent variables z in the latent space through fully connected layers. The variational autoencoder decoder converts z into a reconstructed image through deconvolution. The loss functions used for training include: reconstruction loss, used to measure pixel differences; KL divergence loss, used to constrain the distribution of latent variables; perceptual loss, used to focus on semantic structural similarity; and the total loss of the variational autoencoder. It is a weighted sum of reconstruction loss, KL divergence loss, and perception loss, according to Update the variational autoencoder parameters to complete pre-training; S3-2: Diffusion Model Training: Fix the parameters of the variational autoencoder module and train its latent variables from the encoder output. Input diffusion model, forward diffusion is transformed into Gaussian noise Inverse denoising relies on a denoising neural network based on the U-Net architecture. U-Net is used to predict noise in order to recover the latent encoding. The diffusion loss is calculated by the mean square error between the predicted noise and the real noise. The parameters of the denoising network are updated and the training is repeated until the loss converges or the maximum number of rounds is reached to complete the model training. S4. Set the optimizer and hyperparameters: Use the dataset constructed in step S1 and the training process and loss function constructed in step S3 to train the super-resolution network constructed in step S2; use the Adam optimizer, set the initial learning rate of the variational autoencoder module and the diffusion network parameters, the learning rate decreases linearly with the number of iterations, and during training, randomly crop the images in the dataset to the same size resolution. S5. Generate high-resolution images: based on the network parameters obtained from steps S3 and S4. The super-resolution network constructed in step S2 is used to perform super-resolution processing on the input image; the CBCT image to be processed is stacked in groups of three adjacent layers as the input of the super-resolution network; in the three-channel image output by the super-resolution network, the middle channel is selected as the output of the corresponding input middle channel to generate the final super-resolution CBCT image; for large-size CBCT images, a sliding window strategy is adopted to reduce stitching artifacts.

[0005] Furthermore, in the step S2-1 of constructing the variational autoencoder encoder module: a 3×3×C size convolutional kernel is used, where C is the number of feature map channels, and padding is set to 1; 5 convolutional layers are set, and the number of output channels is... Increasing; potential space dimension emd_dim = 64; In the steps of constructing the variational autoencoder decoder module in S2-2: the deconvolution kernel size is set to 3×3×D, where D is the number of feature map channels, and padding is set to 1; 5 deconvolution layers are set, and the number of output channels is adjusted accordingly. Decreasing; Sigmoid is chosen as the activation function to map the output to the [0, 1] interval; In step S2-3 of constructing the reverse denoising module of the potential diffusion network: Forward diffusion: Set the time step T = 1000, and adopt a linear variance scheduling strategy, where the scheduling parameters are... Increasing from 0.0001 to 0.02; U-Net denoising neural network: The downsampling path contains 4 modules, each consisting of 2 3×3 convolutional layers with channel numbers of 64, 128, 256, and 512 respectively; The upsampling path is symmetrical to the downsampling path, and the same-sized feature maps are fused through skip connections to achieve the integration of high-level semantic features and detailed information.

[0006] Furthermore, in step S3-1, the total loss of the variational autoencoder... The weight parameters are set to ,in To rebuild the losses, For KL divergence loss, To perceive loss; In step S5, multi-layer image stacking is used as the input for feature extraction to enhance the ability to acquire information between layers; In step S5, a sliding window weighted fusion algorithm is used to reduce generated image artifacts. The sliding window weighted fusion formula is as follows: ,in It is the first The pixel value of each sub-window at coordinates (x, y). Representing the The fusion weight of each sub-window For the current pixel to the th The distance between the window boundaries.

[0007] The beneficial effects of this invention are: by enhancing low-resolution dental CBCT images, and utilizing a learning method that combines unsupervised and supervised learning, a mapping between ex vivo dental medical images and in vivo dental CBCT images is effectively constructed, achieving super-resolution reconstruction of in vivo dental CBCT images, improving the quality and resolution of in vivo dental CBCT images, solving the application limitations of image super-resolution processing in practice, and improving the accuracy of medical diagnosis. Attached Figure Description

[0008] Figure 1 This is the overall flowchart of the present invention; Figure 2This is a schematic diagram of an in vivo CBCT tooth image before processing using the technical solution of this invention; Figure 3 This is a schematic diagram of an in vivo CBCT tooth image processed using the technical solution of this invention. Figure 4 These are super-resolution CBCT images reconstructed using the method of this invention. Detailed Implementation

[0009] like Figures 1-4 As shown, a method for super-resolution reconstruction of dental CBCT images based on a latent diffusion model includes the following steps: S1. Constructing the training dataset: Collect low-resolution in vivo CBCT images and out vivo CBCT images, as well as high-resolution micro-CT images of the same object, and perform dataset registration. The specific process of dataset registration is as follows: S1-1. Image Preprocessing and Export: Acquire in vivo CBCT images, out vivo CBCT images, high-resolution micro-CT images, and corresponding in vivo CBCT tooth mask images, out vivo CBCT tooth mask images, and micro-CT tooth mask images obtained through manual segmentation; export the acquired micro-CT images, in vivo CBCT images, and out vivo CBCT images into digital image formats suitable for 3D reconstruction and subsequent processing; using image processing techniques, map the in vivo CBCT tooth mask images, out vivo CBCT tooth mask images, and micro-CT tooth mask images onto the corresponding original images, and extract the tooth regions through threshold segmentation to generate point clouds containing grayscale information. This is used for subsequent image transformation; S1-2. 3D Reconstruction: Image processing software was used to process the in vivo CBCT tooth mask images, out vivo CBCT tooth mask images, and micro-CT tooth mask images obtained through manual segmentation to obtain the micro-CT mask point cloud. CBCT mask point cloud ; S1-3. Image Registration: Point cloud registration is performed using the Iterative Closest Point (ICP) algorithm. and Calculate the transformation matrix The optimal transformation matrix is ​​obtained through multiple iterations of calculation. ,use For the aforementioned point cloud containing grayscale information Transform to obtain This allows it to be spatially matched with the CBCT point cloud; finally, for Parallel projection is used to obtain micro-CT images registered with CBCT images, thus completing the construction of the training dataset; S2. Constructing the Super-Resolution Network: The super-resolution network consists of a variational autoencoder module and a diffusion network; the specific steps are as follows: S2-1. Constructing the variational autoencoder module: Define the encoder input as preprocessed and registered in vivo CBCT images, ex vivo CBCT images, and high-resolution micro-CT images; the encoder extracts features through convolutional layers, and then uses reparameterization techniques to generate latent variables. ; S2-2. Constructing the variational autoencoder decoder module: Define the decoder input as latent variables. Deconvolution is used to decode it into a reconstructed image. ; S2-3. Constructing the forward diffusion process of the latent diffusion network: Set the total number of time steps T in the diffusion process, and the model gradually diffuses towards the original image within T time steps. Noise is added; the denoising neural network predicts the noise using U-net to gradually recover the latent encoding; finally, the original image... The image is converted into a reconstructed image by a variational autoencoder decoder, thus completing the image generation process. S3: Constructing the training process and loss function: The training process consists of two parts: a variational autoencoder and a diffusion model. During training, the variational autoencoder is pre-trained first, followed by the diffusion model. The parameter update method in one iteration is as follows: S3-1. Variational Autoencoder Module Training: The training dataset is input into the variational autoencoder. The variational autoencoder receives the original image x, extracts features through multiple convolutional layers to generate a feature map, and then maps it to latent variables z in the latent space through fully connected layers. The variational autoencoder decoder converts z into a reconstructed image through deconvolution. The loss functions used for training include: reconstruction loss, used to measure pixel differences; KL divergence loss, used to constrain the distribution of latent variables; perceptual loss, used to focus on semantic structural similarity; and the total loss of the variational autoencoder. It is a weighted sum of reconstruction loss, KL divergence loss, and perception loss, according to Update the variational autoencoder parameters to complete pre-training; S3-2: Diffusion Model Training: Fix the parameters of the variational autoencoder module and train its latent variables from the encoder output. Input diffusion model, forward diffusion is transformed into Gaussian noise Inverse denoising relies on a denoising neural network based on the U-Net architecture. U-Net is used to predict noise in order to recover the latent encoding. The diffusion loss is calculated by the mean square error between the predicted noise and the real noise. The parameters of the denoising network are updated and the training is repeated until the loss converges or the maximum number of rounds is reached to complete the model training. S4. Set the optimizer and hyperparameters: Use the dataset constructed in step S1 and the training process and loss function constructed in step S3 to train the super-resolution network constructed in step S2; use the Adam optimizer, set the initial learning rate of the variational autoencoder module and the diffusion network parameters, the learning rate decreases linearly with the number of iterations, and during training, randomly crop the images in the dataset to the same size resolution. S5. Generate high-resolution images: based on the network parameters obtained from steps S3 and S4. The super-resolution network constructed in step S2 is used to perform super-resolution processing on the input image; the CBCT image to be processed is stacked in groups of three adjacent layers as the input of the super-resolution network; in the three-channel image output by the super-resolution network, the middle channel is selected as the output of the corresponding input middle channel to generate the final super-resolution CBCT image; for large-size CBCT images, a sliding window strategy is adopted to reduce stitching artifacts.

[0010] Furthermore, in the step S2-1 of constructing the variational autoencoder encoder module: a 3×3×C size convolutional kernel is used, where C is the number of feature map channels, and padding is set to 1; 5 convolutional layers are set, and the number of output channels is... Increasing; potential space dimension emd_dim = 64; In the steps of constructing the variational autoencoder decoder module in S2-2: the deconvolution kernel size is set to 3×3×D, where D is the number of feature map channels, and padding is set to 1; 5 deconvolution layers are set, and the number of output channels is adjusted accordingly. Decreasing; Sigmoid is chosen as the activation function to map the output to the [0, 1] interval; In step S2-3 of constructing the reverse denoising module of the potential diffusion network: Forward diffusion: Set the time step T = 1000, and adopt a linear variance scheduling strategy, where the scheduling parameters are... Increasing from 0.0001 to 0.02; U-Net denoising neural network: The downsampling path contains 4 modules, each consisting of 2 3×3 convolutional layers with channel numbers of 64, 128, 256, and 512 respectively; The upsampling path is symmetrical to the downsampling path, and the same-sized feature maps are fused through skip connections to achieve the integration of high-level semantic features and detailed information.

[0011] Furthermore, in step S3-1, the total loss of the variational autoencoder... The weight parameters are set to ,in To rebuild the losses, For KL divergence loss, To perceive loss; In step S5, multi-layer image stacking is used as the input for feature extraction to enhance the ability to acquire information between layers; In step S5, a sliding window weighted fusion algorithm is used to reduce generated image artifacts. The sliding window weighted fusion formula is as follows: ,in It is the first The pixel value of each sub-window at coordinates (x, y). Representing the The fusion weight of each sub-window For the current pixel to the th The distance between the window boundaries.

[0012] This invention enhances low-resolution dental CBCT images by employing a combination of unsupervised and supervised learning methods to effectively construct a mapping between ex vivo dental medical images and in vivo dental CBCT images. This achieves super-resolution reconstruction of in vivo dental CBCT images, improving their quality and resolution. It overcomes the limitations of image super-resolution processing in practical applications and enhances the accuracy of medical diagnosis.

[0013] Example 1: The training dataset is constructed according to step S1. The data consists of in vivo CBCT scan images, ex vivo CBCT scan images, and Micro-CT scan images of the same batch of teeth. In image preprocessing S1-1, a threshold T=128 is set for binarization; after 3D reconstruction in S1-2, image registration is completed through 10 iterations in S1-3 to obtain the registered dataset.

[0014] The super-resolution network is constructed according to step S2. In S2-1, C=3 is set, and after 5 convolutional layers, the final number of output channels is 192, and the latent space dimension emb_dim = 64. In S2-2, the decoder uses the Sigmoid activation function. In S2-3, a diffusion network is constructed with time step T = 1000, β increases from 0.0001 to 0.02, and the U-Net downsampling and upsampling paths are constructed according to the predetermined structure.

[0015] Construct the training process according to step S3. During the variational autoencoder pre-training, set... The training of the diffusion network uses cosine annealing for learning rate scheduling, with the minimum learning rate set to [value missing]. .

[0016] Training: Follow step S4 for training. Use the Adam optimizer with an initial learning rate of [missing information]. The image is randomly cropped into The resolution and batch size were set to 32, and the super-resolution network parameters were obtained after 100,000 iterations.

[0017] Test: Follow step S5 for testing. Input the CBCT image to be processed in three adjacent stacked layers. For large images, use... A sliding window strategy with a 32-pixel overlap area is used to output a super-resolution image through weighted fusion.

[0018] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for dental CBCT image super-resolution reconstruction based on a latent diffusion model, characterized in that, Comprising the following steps: S1. Constructing training dataset: Collecting low-resolution in-vivo CBCT images and ex-vivo CBCT images and high-resolution micro-CT images from the same object, performing dataset registration, and the process of dataset registration is as follows: S1-1. Image preprocessing and derivation: acquire in-vivo CBCT images, ex-vivo CBCT images, high-resolution micro-CT images, and corresponding in-vivo CBCT tooth mask images, ex-vivo CBCT tooth mask images and micro-CT tooth mask images obtained by manual segmentation; export the acquired micro-CT images, in-vivo CBCT images, ex-vivo CBCT images into digital image formats that can be used for three-dimensional reconstruction and subsequent processing; use image processing techniques to map the in-vivo CBCT tooth mask images, ex-vivo CBCT tooth mask images and micro-CT tooth mask images onto the corresponding original images, and extract the tooth region by threshold segmentation and generate point clouds containing gray scale information for subsequent image transformation; , for subsequent image transformation; S1-2. Three-dimensional reconstruction: using image processing software to process the in-vivo CBCT tooth mask image, the ex-vivo CBCT tooth mask image and the micro-CT tooth mask image obtained by manual segmentation to obtain a micro-CT mask point cloud , a CBCT mask point cloud ; S1-3. Image registration: registering point clouds by iterative closest point algorithm ICP With , calculate the transformation matrix , calculate the transformation matrix through multiple iterations to obtain the optimal transformation matrix , using the aforementioned point cloud containing grayscale information is transformed to , so that it matches the CBCT point cloud in spatial position; finally, parallel projection is adopted, so as to obtain the micro-CT image registered with the CBCT image, and the construction of the training data set is completed; S2. Constructing super-resolution network: The super-resolution network is composed of a variational autoencoder module and a diffusion network; the specific steps are as follows: S2-1. Constructing the variational autoencoder encoder module: define the encoder input as the pre-processed registered in-vivo CBCT image, ex-vivo CBCT image, high-resolution micro-CT image; the encoder extracts features through convolutional layers, then generates latent variables using the reparameterization trick ; S2-2. Constructing the variational autoencoder decoder module: define the decoder input as the latent variable , decode it into a reconstructed image using a deconvolutional operation ; S2-3. Constructing the forward diffusion process of the potential diffusion network: Set the total number of time steps T of the diffusion process, and the model gradually diffuses the original image in T time steps Add noise; The denoising neural network predicts noise through a U-net to realize step-by-step recovery of latent encoding; finally, the original image is converted into a reconstructed image through a variational autoencoder decoder to complete the image generation process. S3: Constructing training process and loss function: The training process includes two parts of variational autoencoder and diffusion model, during training, first pre-train the variational autoencoder, then train the diffusion model; in one iteration, the parameter updating method is as follows: S3-1. Variational autoencoder module training: input the training dataset into the variational autoencoder, the variational autoencoder encoder receives the original image x, extracts features through multiple convolutional layers, generates feature maps, and then maps them to the latent variables z in the latent space through the fully connected layer; The variational autoencoder decoder converts z into a reconstructed image by deconvolution; the loss function for training includes: a reconstruction loss to measure pixel difference; a KL divergence loss to constrain the latent variable distribution; a perceptual loss to focus on semantic structure similarity; a total loss of the variational autoencoder is a weighted sum of the reconstruction loss, the KL divergence loss and the perceptual loss, according to updating the variational autoencoder parameters to complete pre-training; S3-2: Diffusion model training: fix the variational autoencoder module parameters, the latent variable output by the encoder of which Input diffusion model, forward diffusion transform into Gaussian noise ; reverse denoising depends on the denoising neural network of U-Net architecture , U-Net is used to predict noise to recover latent encoding, the mean square error between the predicted noise and the real noise is calculated to calculate the diffusion loss, the denoising network parameters are updated, and the training is repeated until the loss converges or the maximum number of rounds is reached, and the model training is completed; S4. Setting optimizer and hyperparameters: using the dataset constructed in step S1 and the training process and loss function constructed in step S3, training the super-resolution network constructed in step S2; using Adam optimizer, setting the initial learning rate of the parameters of the variational autoencoder module and the diffusion network, the learning rate is linearly decayed with the number of iterations, and during training, the images in the dataset are randomly cropped to the same size resolution; S5. Generate high-resolution images: based on the network parameters obtained from steps S3 and S4. The super-resolution network constructed in step S2 is used to perform super-resolution processing on the input image; the CBCT image to be processed is stacked in groups of three adjacent layers as the input of the super-resolution network; in the three-channel image output by the super-resolution network, the middle channel is selected as the output of the corresponding input middle channel to generate the final super-resolution CBCT image; for large-size CBCT images, a sliding window strategy is adopted to reduce stitching artifacts.

2. The tooth CBCT image super-resolution reconstruction method based on the latent diffusion model according to claim 1, characterized in that: In the step of constructing the variational autoencoder encoder module in S2-1: a 3x3xC size convolution kernel is used, where C is the number of feature map channels, and padding is set to 1; 5 convolution layers are set, and the output channel number is incremented by the latent space dimension emd_dim = 64. In the step of constructing the variational autoencoder decoder module in S2-2: set the deconvolution kernel size to 3x3xD, where D is the number of feature map channels, and set the padding to 1; set 5 deconvolution layers, and the output channel numbers decrease by Sigmoid is selected as the activation function to map the output to the interval [0, 1]. In the step of constructing the potential diffusion network reverse denoising module: forward diffusion: set the time step T = 1000, adopt the linear variance scheduling strategy, wherein the scheduling parameter from 0.0001 to 0.02; U-Net denoising neural network: the down-sampling path contains 4 modules, each module is composed of 2 3x3 convolution layers, and the number of channels is 64, 128, 256, 512 in turn; the up-sampling path is symmetrical with the down-sampling path, and the same size feature maps are fused through the jump connection, so as to realize the integration of high-level semantic features and detail information.

3. The tooth CBCT image super-resolution reconstruction method based on the latent diffusion model according to claim 1, characterized in that: In step S3-1, the total loss of the variational autoencoder is set as wherein is the reconstruction loss, is the KL divergence loss, is the perceptual loss;​ In step S5, multi-layer image stacking is used as the input for feature extraction to enhance the ability to obtain inter-layer information; In step S5, a sliding window weighted fusion algorithm is used to reduce generated image artifacts. The sliding window weighted fusion formula is as follows: ,in It is the first The pixel value of each sub-window at coordinates (x, y). Representing the The fusion weight of each sub-window For the current pixel to the th The distance between the window boundaries.

Images