Semantic perception-based high-quality decayed tooth image synthesis method

Abstract

The invention relates to the field of computer medical image processing, and provides a semantic perception-based high-quality decayed tooth image synthesis method, which comprises the following steps of: 1, preparing a data set; and step 3, performing high-quality decayed tooth image synthesis on a to-be-predicted CBCT image by using the semantic perception-based synthesis network framework which is trained and optimized in the step 2, and adopting the data set in the step 1 to perform training optimization on the semantic perception-based synthesis network framework. The method is easy to implement, focuses on key tissue boundaries of dental caries diagnosis, and enhances details of a focus area and a dental pulp area. According to the invention, the decayed tooth image quality can be effectively improved, and the risk of decayed tooth misdiagnosis caused by image blurring and artifacts in medical auxiliary diagnosis is reduced. By applying the method, the problem of low quality of the decayed tooth image in oral medicine auxiliary diagnosis can be relieved, and an accurate diagnosis basis is provided for diagnosis of doctors.

Description

Technical Field

[0001] The present invention relates to the field of computer medical image processing. Background Art

[0002] Caries is one of the most common oral diseases in humans. Cone beam computed tomography (CBCT) is often used to capture dental caries images in clinical practice. However, due to limited resolution and insufficient sensitivity to details, it is usually unable to show tiny caries lesions and specific caries progression. Micro-computed tomography (Micro-CT) is a high-quality imaging technology with high resolution and clear contrast. However, Micro-CT images require single caries scanning in vitro and cannot be directly applied in clinical practice.

[0003] In recent years, cross-modal medical image synthesis methods based on deep learning have shown their advantages in terms of effect, computational performance, and scalability. However, most of these methods consider the overall data mapping relationship between modalities, such as modeling the overall noise distribution of low-quality images. These methods ignore the information of the edges of different tissues in caries images, resulting in inaccurate boundaries of caries lesions and pulp areas in the synthesized images. Errors in the boundaries of critical tissues can lead to doctors' misjudgments and pose extremely high risks. Recently, some synthesis methods based on edge information enhancement have been proposed. However, these methods only enhance the edge contrast and cannot specifically simulate the impact of caries lesion progression, and their robustness in practical applications cannot be guaranteed. Summary of the invention

[0004] In order to overcome the problems of lack of caries progress and blurring of tiny pulp and carious tissue in the current high-quality caries image synthesis process, the present invention proposes a high-quality caries image synthesis method based on semantic perception.

[0005] Technical solution of the present invention:

[0006] A high-quality caries image synthesis method based on semantic perception includes the following steps:

[0007] Step 1: Dataset preparation:

[0008] Step 2 builds a synthetic network framework based on semantic perception, and uses the data set in step 1 to train and optimize the synthetic network framework based on semantic perception:

[0009] Step 3 uses the semantic-awareness-based synthesis network framework trained and optimized in step 2 to synthesize high-quality caries images for the predicted CBCT images.

[0010] Furthermore, in step 1, clinical CBCT images are collected from the hospital, and ex vivo Micro-CT images are taken, and a synthetic caries dataset is constructed after image registration and tissue semantic segmentation map annotation.

[0011] Furthermore, the step 2:

[0012] Design of a synthetic network framework based on semantic perception:

[0013] The semantic-aware synthetic network framework includes a shared encoder, a bottleneck layer, a semantic segmentation branch, an image synthesis branch, and a feature enhancement module, among which:

[0014] Shared encoder: The shared encoder takes the original CBCT image as input and is used to extract low-level general features to input the semantic segmentation branch and the image synthesis branch. The shared encoder consists of a convolution layer and four encoding convolution modules, generating four general features of different scales. The encoding convolution module consists of a double-layer convolution module and a downsampling layer. The double-layer convolution module is the basic module for feature learning. The double-layer convolution module consists of a two-dimensional convolution layer, a batch normalization layer, and an activation layer stacked twice; the downsampling layer is used to reduce the spatial dimension of the feature to help the network learn more abstract features.

[0015] Bottleneck layer: The bottleneck layer is connected to the shared encoder, the semantic segmentation branch, and the image synthesis branch. The bottleneck layer takes the minimum-scale common features from the shared encoder as input to generate the initial input features of the semantic segmentation branch and the initial input features of the image synthesis branch. The bottleneck layer consists of a two-layer convolution module.

[0016] Semantic segmentation branch: The semantic segmentation branch is connected to the shared encoder, bottleneck layer, and feature enhancement module, and is used to predict the organizational semantic segmentation map of the image and the organizational semantic features of different scales. Among them, the organizational semantic features are used as the input of the feature enhancement module. The semantic segmentation branch consists of four semantic segmentation modules and a convolutional layer. Among them, the semantic segmentation module consists of a double-layer convolutional module and an upsampling layer. The first semantic segmentation module of the semantic segmentation branch accepts the output from the bottleneck layer. Each subsequent semantic segmentation module will accept the output of the previous semantic segmentation module and the common features of the corresponding scale from the shared encoder to learn and generate new features. After the features learned by the four semantic segmentation modules, they are passed through the convolutional layer to generate a predicted organizational semantic segmentation map. The upsampling layer is used to increase the spatial dimension of the features and restore the resolution.

[0017] Image synthesis branch: The image synthesis branch is connected to the shared encoder, bottleneck layer, and feature enhancement module. The image synthesis branch takes the general features and enhanced features output by the feature enhancement module as input to generate high-quality caries images. The image synthesis branch consists of four image synthesis modules and a convolution layer. Among them, the image synthesis module consists of a double-layer convolution module and an upsampling layer. The first image synthesis module of the image synthesis branch accepts the output from the bottleneck layer. Thereafter, each image synthesis module of the image synthesis branch will accept the general features of the corresponding scale from the shared encoder and the enhanced features from the feature enhancement module as input, and generate new synthesized features, which will be output to the feature enhancement module. The features learned by the four image synthesis modules and the feature enhancement module are then passed through the convolution layer to finally generate a high-quality caries image.

[0018] Feature enhancement module: The feature enhancement module is connected to the semantic segmentation branch and the image synthesis branch. There are four feature enhancement modules, each of which receives the organizational semantic features generated by the semantic segmentation branch and the synthetic features generated by the image synthesis branch, and outputs enhanced features. The feature enhancement module consists of a cross attention module.

[0019] The double-layer convolution module of the bottleneck layer, the double-layer convolution module of the semantic segmentation branch, and the double-layer convolution module of the image synthesis branch adopt the same structure as the double-layer convolution module of the shared encoder.

[0020] Furthermore, the step 2:

[0021] Training of semantically-aware synthetic network framework:

[0022] 2.1: Use the shared encoder to extract common features from the CBCT image obtained in step 1.

[0023] 2.2: Take the smallest common feature as the input of the bottleneck layer to generate the initial semantic segmentation branch input feature and the initial image synthesis branch input feature.

[0024] 2.3: Obtain the predicted organization semantic segmentation map through the semantic segmentation branch. The process includes: the first semantic segmentation module of the semantic segmentation branch accepts the initial semantic segmentation branch input features generated by 2.2 to generate organization semantic features. Each subsequent semantic segmentation module takes the output of the previous semantic segmentation module and the corresponding scale universal features obtained in 2.1 as input to generate new organization semantic features. In this way, the features learned by the four semantic segmentation modules are passed through the convolution layer to generate the predicted organization semantic segmentation map, and finally the predicted organization semantic segmentation map is obtained.

[0025] 2.4: Obtain high-quality caries images through image synthesis branches and feature enhancement modules. The process includes: the first image synthesis module of the image synthesis branch receives the initial image synthesis branch input features generated by 2.2 to generate synthetic features; the synthetic features and the tissue semantic features obtained in 2.3 are input into the feature enhancement module to obtain enhanced features. Thereafter, other image synthesis modules of the image synthesis branch use the enhanced features output by the feature enhancement module and the general features of the corresponding scale in 2.1 as input to obtain new synthetic features, and then input the new synthetic features and tissue semantic features into the enhancement module to obtain new enhanced features. In this way, the features learned by the four image synthesis modules and the feature enhancement module are passed through the convolution layer to finally obtain a synthesized high-quality caries image.

[0026] 2.5: The predicted tissue semantic segmentation map obtained in 2.3 and the tissue semantic segmentation map annotation obtained in step 1 are learned, and the semantic segmentation loss is used for loss calculation; the synthesized high-quality caries image obtained in 2.4 is combined with the Micro-CT image in the dataset of step 1 for learning, and the image synthesis loss is used for loss calculation. By training and optimizing the overall network composed of the above-mentioned shared encoder, bottleneck layer, semantic segmentation branch, feature enhancement module, and image synthesis branch, a stable semantic-aware synthetic network is obtained.

[0027] 2.6: Repeat the above steps 2.1 to 2.5 until the semantic-aware synthetic network training is stable, and save the trained network parameters.

[0028] The beneficial effects of the present invention are:

[0029] The semantically-aware high-quality caries image synthesis method proposed in the present invention enhances clinical oral CBCT images and improves the clarity and contrast of the images. The feature enhancement module of the semantically-aware synthetic network framework constructed in the present invention can effectively improve the ability to reconstruct tissue edges and small area details in caries image synthesis and effectively improve the synthetic network's recognition of fuzzy areas. As a medical image auxiliary enhancement tool, it provides a basis for doctors' subsequent diagnosis and treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a general flow chart of the present invention for synthesizing high-quality caries images from CBCT caries images.

[0031] Figure 2 It is a flow chart of implementing training optimization of the present invention.

[0032] Figure 3 It is a flow chart of the present invention for implementing semantic prediction of caries tissue and synthesis of high-quality caries images.

[0033] Figure 4It is a schematic diagram of the training optimization process of the synthetic network framework based on semantic perception of the present invention.

[0034] Figure 5 Schematic diagram of generating high-quality caries images based on CBCT caries image data according to an embodiment of the present invention. Figure 5 (a) shows the training phase; Figure 5 (b) shows the testing phase. DETAILED DESCRIPTION

[0035] The technical solution provided by the present application will be further described below in conjunction with specific embodiments and accompanying drawings. The advantages and features of the present application will become more apparent with the following description.

[0036] Figure 1 The overall flow chart of the present invention is given:

[0037] Step 1: Dataset preparation. The specific implementation process is as follows:

[0038] We collect CBCT image data and corresponding Micro-CT images and semantic segmentation labels from hospitals to create a dataset, denoted as D = {X, T, M}, where X represents the CBCT image set, T represents the Micro-CT dataset, and M represents the labeled tissue semantic segmentation map dataset.

[0039] Step 2: Build a synthetic network framework based on semantic perception and train and optimize ( Figure 4 The specific implementation process is as follows:

[0040] 2.1: Randomly take a sample (x, t, m) ∈ D from the dataset, input the CBCT image x into the shared encoder to extract features, and obtain universal features f = {f1, f2, f3, f4} of different scales. Among them, f1 is the feature with the largest scale, f2 is the feature with the second largest scale, and so on.

[0041] 2.2: Input the minimum scale universal feature f4 into the bottleneck layer to obtain the initial semantic segmentation branch input feature s init And the image synthesis branch input feature g init .

[0042] 2.3: Obtaining the predicted tissue semantic segmentation map The process includes:

[0043] Input the initial semantic segmentation branch obtained in 2.2 into feature s init Input the first semantic segmentation module of the semantic segmentation branch to obtain the organizational semantic feature s3;

[0044] Organize the semantic features i and the common feature f of the corresponding scale iInput the next semantic segmentation module to obtain the next scale organizational semantic feature s i-1 , where i = 3, 2, 1. Thus, after three semantic segmentation module operations, the organizational semantic feature s0 is obtained;

[0045] Input s0 into the convolutional layer to obtain the predicted tissue semantic segmentation map

[0046] 2.4: Obtaining high-quality caries images The process includes:

[0047] Input the image synthesis branch obtained in 2.2 into feature g init Input the first image synthesis module of the image synthesis branch to obtain the synthesized feature g3; input the synthesized feature g3 and the corresponding scale s3 into the feature enhancement module to obtain the enhanced feature g3′;

[0048] The enhanced feature g i ′ and the common feature f of the corresponding scale i Splice to get the new synthetic feature g i ″, the new synthetic feature g i ″Input the next image synthesis module of the image synthesis branch to obtain the synthesis feature g i-1 ; The synthetic feature g i-1 and the corresponding scale s i-1 Input feature enhancement module to obtain enhanced feature g i-1 ′, where i=3, 2, 1. After three operations of the image synthesis module and the feature enhancement module, the enhanced feature g0′ is obtained;

[0049] Input g0′ into the convolutional layer to obtain a high-quality caries image

[0050] 2.5: The predicted organization semantic segmentation map obtained in step 2.3 Calculate the semantic segmentation loss with the semantic segmentation label m in the dataset in step 1, and convert the high-quality caries image obtained in step 2.4 Calculate the image synthesis loss with the Micro-CT image t in the dataset in step 1 and train the semantically-aware synthesis network;

[0051] The semantic segmentation loss is:

[0052]

[0053] Where H and W represent the length and width of the image respectively, and C represents the total number of semantic categories; ij It represents the true category of the pixel at the image position (i, j), which is expressed by the value m of the image position (i, j) of the semantic segmentation label m.ij One-hot encoding is performed; Represents the probability that the model predicts that the position (i, j) belongs to category c, by organizing the semantic segmentation map for the prediction The value at image position (i, j) is Obtained by one-hot encoding;

[0054] The image synthesis loss consists of pixel loss MAE and structural similarity loss SSIM.

[0055] The MAE loss is calculated as follows:

[0056]

[0057] Where H and W represent the length and width of the image respectively; t ij It represents the value of the corresponding Micro-CT image t at the image position (i, j); Represents a high-quality caries image The value at image position (i, j);

[0058] The SSIM loss formula is calculated as:

[0059]

[0060] in and μ t Represent the average values ​​of the predicted image and the real image respectively; and σ t Represent the variance of the predicted image and the real image respectively; represents the covariance between the two; c1 and c2 are constants used to avoid the denominator being 0.

[0061] 2.6: Repeat the above steps 2.1 to 2.5 until the semantic-aware synthetic network training is stable, and save the trained network parameters.

[0062] Step 3: predict the tissue semantic segmentation map and synthesize the high-quality caries image. The specific implementation process is as follows:

[0063] 3.1 Input the CBCT image to be predicted into step 2.6 to train a stable semantic-aware synthetic network to obtain a predicted tissue semantic segmentation map and a high-quality caries image.

[0064] Example

[0065] A set of implementation examples are given in accordance with the illustrated process and model structure.

[0066] During training, Figure 5As shown in (a), the CBCT image data in the collected dataset needs to be input into the semantic-aware image synthesis network to model its feature generation process. The semantic-aware synthesis network consists of a shared encoder, a bottleneck layer, a semantic segmentation branch, an image synthesis branch, and a feature enhancement module.

[0067] First, the image data is input into the shared encoder for feature extraction, and its corresponding universal features can be obtained. This universal feature can reflect the universal feature information of the image such as grayscale and texture to a certain extent, providing effective support for image synthesis and semantic segmentation. Subsequently, the universal features of the minimum scale are input into the bottleneck layer to obtain the initial input features of the tissue semantic segmentation branch and the initial input features of the image synthesis branch. The initial semantic segmentation branch input features and universal features are then input into the semantic segmentation branch to generate tissue semantic features and predict tissue semantic segmentation maps. The initial image synthesis branch input features and universal features are input into the image synthesis branch, and combined with the tissue semantic features generated by the semantic segmentation branch and the feature enhancement module, high-quality caries images are synthesized. Finally, the reconstructed image synthesis loss and semantic segmentation loss are calculated respectively in combination with the real Micro-CT image and semantic segmentation label, and back propagation is performed to update the network parameters. Repeat the above operations several times until the network training is completed.

[0068] Testing phase, such as Figure 5 (b), the test CBCT image is input into the network to obtain the predicted tissue semantic segmentation map and high-quality caries image.

[0069] The above description is only a description of the preferred embodiments of the present application, and is not intended to limit the scope of the present application. Any changes or modifications made by any person skilled in the art based on the above disclosed technical contents shall be deemed as equivalent effective embodiments and shall fall within the scope of protection of the technical solution of the present application.

Claims

1. A high-quality caries image synthesis method based on semantic perception, characterized in that: Includes steps: Step 1: Dataset preparation: Step 2 builds a synthetic network framework based on semantic perception, and uses the data set in step 1 to train and optimize the synthetic network framework based on semantic perception: Step 3 uses the semantic-awareness-based synthesis network framework trained and optimized in step 2 to synthesize high-quality caries images for the predicted CBCT images.

2. The semantically-aware high-quality caries image synthesis method according to claim 1, characterized in that: Step 1: Clinical CBCT images were collected from the hospital, and in vitro Micro-CT images were taken. After image registration and tissue semantic segmentation annotation, a synthetic caries dataset was constructed.

3. The semantically-aware high-quality caries image synthesis method according to claim 1, characterized in that: Step 2: The synthetic network framework structure based on semantic perception includes a shared encoder, a bottleneck layer, a semantic segmentation branch, an image synthesis branch, and a feature enhancement module, wherein: Shared encoder: The shared encoder takes the original CBCT image as input and is used to extract low-level general features to input the semantic segmentation branch and the image synthesis branch; the shared encoder consists of a convolution layer and four encoding convolution modules, generating four general features of different scales; wherein the encoding convolution module consists of a double-layer convolution module and a downsampling layer, the double-layer convolution module is the basic module for feature learning, and the double-layer convolution module consists of a two-dimensional convolution layer, a batch normalization layer, and an activation layer stacked twice; the downsampling layer is used to reduce the spatial dimension of the feature to help the network learn more abstract features; Bottleneck layer: The bottleneck layer is connected to the shared encoder, the semantic segmentation branch, and the image synthesis branch. The bottleneck layer takes the minimum-scale common features from the shared encoder as input to generate the initial input features of the semantic segmentation branch and the initial input features of the image synthesis branch. The bottleneck layer consists of a two-layer convolution module. Semantic segmentation branch: The semantic segmentation branch is connected to the shared encoder, the bottleneck layer, and the feature enhancement module. The semantic segmentation branch is used to predict the organizational semantic segmentation map of the image and the organizational semantic features of different scales; wherein the organizational semantic features are used as the input of the feature enhancement module; the semantic segmentation branch is composed of four semantic segmentation modules and a convolutional layer; wherein the semantic segmentation module is composed of a double-layer convolutional module and an upsampling layer; the first semantic segmentation module of the semantic segmentation branch receives the output from the bottleneck layer, and each subsequent semantic segmentation module receives the output of the previous semantic segmentation module and the common features of the corresponding scale from the shared encoder to learn and generate new features. The features learned by the four semantic segmentation modules are then passed through the convolutional layer to generate a predicted organizational semantic segmentation map; the upsampling layer is used to increase the spatial dimension of the features and restore the resolution; Image synthesis branch: The image synthesis branch is connected to the shared encoder, the bottleneck layer, and the feature enhancement module; the image synthesis branch uses the general features and the enhanced features output by the feature enhancement module as input to generate a high-quality caries image; the image synthesis branch consists of four image synthesis modules and a convolution layer; wherein the image synthesis module consists of a double-layer convolution module and an upsampling layer; the first image synthesis module of the image synthesis branch receives the output from the bottleneck layer, and thereafter each image synthesis module of the image synthesis branch receives the general features of the corresponding scale from the shared encoder and the enhanced features from the feature enhancement module as input, and generates new synthesized features, which are output to the feature enhancement module; the features learned by the four image synthesis modules and the feature enhancement module are then passed through the convolution layer to finally generate a high-quality caries image; Feature enhancement module: The feature enhancement module is connected to the semantic segmentation branch and the image synthesis branch. There are four feature enhancement modules, each of which receives the organizational semantic features generated by the semantic segmentation branch and the synthetic features generated by the image synthesis branch, and outputs enhanced features. The feature enhancement module consists of a cross attention module. The double-layer convolution module of the bottleneck layer, the double-layer convolution module of the semantic segmentation branch, and the double-layer convolution module of the image synthesis branch adopt the same structure as the double-layer convolution module of the shared encoder.

4. The semantically-aware high-quality caries image synthesis method according to claim 3, characterized in that: Step 2: Training of a semantically aware synthetic network framework, including: 2.1: Use the shared encoder to extract common features from the CBCT image obtained in step 1; 2.2: Take the smallest common feature as the input of the bottleneck layer to generate the initial semantic segmentation branch input feature and the initial image synthesis branch input feature; 2.3: Obtain the predicted organization semantic segmentation map through the semantic segmentation branch. The process includes: The first semantic segmentation module of the semantic segmentation branch receives the initial semantic segmentation branch input features generated in 2.2 to generate organizational semantic features; each subsequent semantic segmentation module takes the output of the previous semantic segmentation module and the corresponding scale universal features obtained in 2.1 as input to generate new organizational semantic features; in this way, the features learned by the four semantic segmentation modules are passed through the convolutional layer to generate a predicted organizational semantic segmentation map, and finally a predicted organizational semantic segmentation map is obtained; 2.4: Obtain high-quality caries images through the image synthesis branch and feature enhancement module. The process includes: The first image synthesis module of the image synthesis branch receives the initial image synthesis branch input features generated in 2.2 to generate synthesized features; the synthesized features and the tissue semantic features obtained in 2.3 are input into the feature enhancement module to obtain enhanced features; thereafter, the other image synthesis modules of the image synthesis branch use the enhanced features output by the feature enhancement module and the general features of the corresponding scale in 2.1 as input to obtain new synthesized features, and then the new synthesized features and the tissue semantic features are input into the enhancement module to obtain new enhanced features; in this way, the features learned by the four image synthesis modules and the feature enhancement module are passed through the convolution layer to finally obtain a synthesized high-quality caries image; 2.5: The predicted tissue semantic segmentation map obtained in 2.3 and the tissue semantic segmentation map annotation obtained in step 1 are learned, and the semantic segmentation loss is used for loss calculation; the synthesized high-quality caries image obtained in 2.4 is combined with the Micro-CT image in the data set of step 1 for learning, and the image synthesis loss is used for loss calculation; by training and optimizing the overall network composed of the above-mentioned shared encoder, bottleneck layer, semantic segmentation branch, feature enhancement module, and image synthesis branch, a stable semantic-aware synthetic network is obtained; 2.6: Repeat the above steps 2.1 to 2.5 until the semantic-aware synthetic network training is stable, and save the trained network parameters.

5. The semantically-aware high-quality caries image synthesis method according to claim 4, characterized in that: CBCT image data and corresponding Micro-CT images and semantic segmentation labels are collected from the hospital to create a dataset, denoted as D = {X, T, M}, where: X represents the CBCT image set, T represents the Micro-CT dataset, and M represents the labeled tissue semantic segmentation map dataset; Then, the specific implementation process of training optimization based on semantic-aware synthetic network framework is as follows: 2.1: Randomly take a sample (x, t, m) ∈ D from the data set, input the CBCT image x into the shared encoder to extract features, and obtain universal features f = {f1, f2, f3, f4} of different scales; where, f1 is the feature with the largest scale, f2 is the feature with the second largest scale, and so on; 2.2: Input the minimum scale universal feature f4 into the bottleneck layer to obtain the initial semantic segmentation branch input feature s init And the image synthesis branch input feature g init ; 2.3: Obtaining the predicted tissue semantic segmentation map The process includes: Input the initial semantic segmentation branch obtained in 2.2 into feature s init Input the first semantic segmentation module of the semantic segmentation branch to obtain the organizational semantic feature s3; Organize the semantic features i and the common feature f of the corresponding scale i Input the next semantic segmentation module to obtain the next scale organizational semantic feature s i-1 , where i = 3, 2, 1. Thus, after three semantic segmentation module operations, the organizational semantic feature s0 is obtained; Input s0 into the convolutional layer to obtain the predicted tissue semantic segmentation map 2.4: Obtaining high-quality caries images The process includes: Input the image synthesis branch obtained in 2.2 into feature g init Input the first image synthesis module of the image synthesis branch to obtain the synthesized feature g3; input the synthesized feature g3 and the corresponding scale s3 into the feature enhancement module to obtain the enhanced feature g3′; The enhanced feature g i ′ and the common feature f of the corresponding scale i Splice to get the new synthetic feature g i ″, the new synthetic feature g i ″Input the next image synthesis module of the image synthesis branch to obtain the synthesis feature g i-1 ; The synthetic feature g i-1 and the corresponding scale s i-1 Input feature enhancement module to obtain enhanced feature g i-1 ′, where i = 3, 2, 1. After three operations of the image synthesis module and the feature enhancement module, the enhanced feature g0′ is obtained; Input g0′ into the convolutional layer to obtain a high-quality caries image 2.5: The predicted organization semantic segmentation map obtained in step 2.3 Calculate the semantic segmentation loss with the semantic segmentation label m in the dataset in step 1, and convert the high-quality caries image obtained in step 2.4 Calculate the image synthesis loss with the Micro-CT image t in the dataset in step 1 and train the semantically-aware synthesis network; The semantic segmentation loss is: Where H and W represent the length and width of the image respectively, and C represents the total number of semantic categories; ij It represents the true category of the pixel at the image position (i, j), through the semantic segmentation label, the value m of the image position (i, j) ij One-hot encoding is performed; Represents the probability that the model predicts that the position (i, j) belongs to category c, by organizing the semantic segmentation map for the prediction The value at image position (i,j) is Obtained by one-hot encoding; The image synthesis loss consists of pixel loss MAE and structural similarity loss SSIM. The MAE loss is calculated as follows: Where H and W represent the length and width of the image respectively; t ij It represents the value of the corresponding Micro-CT image t at the image position (i, j); Represents a high-quality caries image The value at image position (i, j); The SSIM loss formula is calculated as: in and μ h Represent the average values ​​of the predicted image and the real image respectively; and t Represent the variance of the predicted image and the real image respectively; represents the covariance between the two; c1 and c2 are constants used to avoid the denominator being 0; 2.6: Repeat the above steps 2.1 to 2.5 until the semantic-aware synthetic network training is stable, and save the trained network parameters.

Images