Prosthesis Image Generation Method, Device, Equipment and Storage Medium

Abstract

The present disclosure relates to a restoration image generation method, apparatus, device, and storage medium. The method comprises: acquiring an image of a tooth to be restored; extracting a first a priori feature and a first decoding feature from the image of the tooth to be restored based on a trained tooth restoration model; and generating a restoration image based on the first a priori feature and the first decoding feature. The tooth restoration model is obtained by training a preset network based on images of missing teeth and sample restoration images. Through the above-described technical solution, the output restoration image can be made to conform to the actual tooth shape, thereby improving the accuracy of the restoration image and reducing the design cost of the restoration.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a method, apparatus, device and storage medium for generating a restoration image. Background Art

[0002] In the field of modern dentistry, dentists often use restorations, commonly known as dentures, to protect damaged tooth tissue or restore missing teeth when patients have missing or damaged teeth.

[0003] Currently, restoration design involves dentists using an intraoral scanner to digitally reconstruct the tooth and its surroundings in 3D, obtaining initial 3D tooth data. This data is then manually restored to create a complete crown or inlay. However, current restoration design relies on manually adjusting the rules and parameters in a computer-aided design (CAD) system to obtain the 3D tooth data. This is then followed by manual restoration to create the restoration. Consequently, traditional restoration design is limited by the technician's experience, and the entire restoration process is complex and time-consuming. Summary of the Invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a restoration image generation method, device, equipment and storage medium.

[0005] In a first aspect, the present disclosure provides a method for generating a restoration image, the method comprising:

[0006] Acquire an image of the tooth to be restored;

[0007] Based on the trained tooth restoration model, the first prior feature and the first decoding feature of the tooth image to be restored are extracted, and the restoration image is obtained based on the first prior feature and the first decoding feature, wherein the tooth restoration model is obtained by training a preset network based on the tooth missing image and the sample restoration image.

[0008] In a second aspect, the present disclosure provides a restoration image generation device, the device comprising:

[0009] An image acquisition module, used to acquire an image of the tooth to be repaired;

[0010] An image generation module is used to extract the first prior feature and the first decoding feature of the image of the tooth to be restored based on the trained tooth restoration model, and obtain the restoration image based on the first prior feature and the first decoding feature, wherein the tooth restoration model is obtained by training a preset network based on the tooth missing image and the sample restoration image.

[0011] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the device comprising:

[0012] one or more processors;

[0013] a storage device for storing one or more programs,

[0014] When the one or more programs are executed by the one or more processors, the one or more processors implement the restoration image generation method provided in the first aspect.

[0015] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the restoration image generation method provided in the first aspect.

[0016] The technical solution provided by the embodiments of the present disclosure has the following advantages over the prior art:

[0017] A restoration image generation method, apparatus, device and storage medium of the disclosed embodiments can, after acquiring the image of the tooth to be restored, extract the first priori feature and the first decoding feature of the image of the tooth to be restored based on the trained tooth restoration model, and obtain the restoration image based on the first priori feature and the first decoding feature, wherein the tooth restoration model is obtained by training a preset network based on the image of the missing tooth and the sample restoration image. Since the tooth restoration model can extract the first priori feature and the first decoding feature of the image of the tooth to be restored, and use the first priori feature of the image of the tooth to be restored as supplementary structural information, the problem of lack of collective structural information of the tooth can be improved, and the first decoding feature is used as the overall feature of the image of the tooth to be restored, so that the output restoration image is consistent with the actual tooth shape, thereby improving the accuracy of the restoration image and reducing the design cost of the restoration. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0019] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A schematic flow chart of a restoration image generation method provided in an embodiment of the present disclosure;

[0021] Figure 2 A schematic diagram of a process for training a tooth restoration model according to an embodiment of the present disclosure;

[0022] Figure 3 A schematic diagram of an occlusal plane provided in an embodiment of the present disclosure;

[0023] Figure 4 A schematic structural diagram of a tooth restoration model provided in an embodiment of the present disclosure;

[0024] Figure 5 A schematic diagram of the principle of a tooth restoration model provided in an embodiment of the present disclosure;

[0025] Figure 6 A schematic diagram of the principle of a restoration image prediction subnetwork provided by an embodiment of the present disclosure;

[0026] Figure 7a A depth image corresponding to a tooth-missing image provided by an embodiment of the present disclosure is shown;

[0027] Figure 7b shows a depth image corresponding to a sample restoration image provided by an embodiment of the present disclosure;

[0028] Figure 7c A depth image corresponding to a restoration image obtained based on a traditional generative adversarial network provided by an embodiment of the present disclosure is shown;

[0029] Figure 7d A depth image corresponding to a restoration image obtained based on an edge-guided method provided by an embodiment of the present disclosure is shown;

[0030] Figure 7e A depth image corresponding to a restoration image obtained based on a tooth restoration model provided by an embodiment of the present disclosure is shown;

[0031] Figure 8a An image of a sample restoration provided by an embodiment of the present disclosure is shown;

[0032] Figure 8b An image of a restoration obtained based on a traditional generative adversarial network provided by an embodiment of the present disclosure is shown;

[0033] Figure 8c The figure shows a restoration image obtained based on a generative adversarial network with a priori feature extraction subnetwork provided by an embodiment of the present disclosure;

[0034] Figure 8d An image of a restoration obtained based on a generative adversarial network with a priori feature extraction subnetwork and a self-evaluation mechanism, provided by an embodiment of the present disclosure, is shown;

[0035] Figure 8eThe figure shows a restoration image obtained by a generative adversarial network with a prior feature extraction subnetwork, a self-evaluation mechanism, and an introduction of a structural consistency loss function, provided by an embodiment of the present disclosure;

[0036] Figure 9a A tooth missing image provided by an embodiment of the present disclosure is shown;

[0037] Figure 9b An image of a sample restoration provided by an embodiment of the present disclosure is shown;

[0038] Figure 9c An image of a restoration obtained based on a traditional generative adversarial network provided by an embodiment of the present disclosure is shown;

[0039] Figure 9d An image of a restoration obtained based on a deep convolutional neural network provided by an embodiment of the present disclosure is shown;

[0040] Figure 9e A restoration image obtained based on a tooth restoration model provided by an embodiment of the present disclosure is shown;

[0041] Figure 10 A schematic structural diagram of a restoration image generating device provided by an embodiment of the present disclosure;

[0042] Figure 11 A schematic structural diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0043] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0044] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0045] In the field of modern dentistry, dentists often use restorations, commonly known as dentures, to protect damaged tooth tissue or restore missing teeth when patients have missing or damaged teeth.

[0046] Currently, restoration design involves dentists using an intraoral scanner to digitally reconstruct the tooth and its surroundings in 3D, obtaining initial 3D tooth data. This data is then manually restored to create a complete crown or inlay. However, current restoration design relies on manually adjusting the rules and parameters in a computer-aided design (CAD) system to obtain the 3D tooth data. This is then followed by manual restoration to create the restoration. Consequently, traditional restoration design is limited by the technician's experience, and the entire restoration process is complex and time-consuming.

[0047] In order to solve the problem of relying on technicians' experience to generate restorations and reduce the design efficiency and cost of restorations, there is currently a restoration design method based on deep neural networks, which uses a large number of training samples to learn deep neural networks without formulating fixed rules. The image of the tooth to be restored is input into the trained deep neural network, and the trained deep neural network is used to extract the feature information of the upper jaw image, lower jaw image and occlusal groove image in the image of the tooth to be restored to generate the restoration image.

[0048] However, the above-mentioned restoration design method based on deep neural network has the following disadvantages:

[0049] Point 1: Deep neural networks lack prior information and will output restoration images that do not match the actual tooth shape.

[0050] Point 2: The deep neural network is unable to identify and correct incorrect predictions, resulting in the output of incorrect restoration images.

[0051] Point 3: The deep neural network does not consider the structural consistency information of the intermediate layer and lacks the ability to detect the intermediate layer of the restored tooth image, resulting in the output of incorrect restoration images.

[0052] In summary, the accuracy of the repaired image designed based on the existing restoration design method is poor, and the design cost is high, which cannot meet the dentist's needs for tooth restoration.

[0053] In order to solve the above problems, the embodiments of the present disclosure propose a restoration image generation method, device, equipment and storage medium, which can use a trained tooth restoration model to generate a restoration image of the tooth image to be restored. Since the tooth restoration model can extract the first prior feature and the first decoding feature of the tooth image to be restored, and use the first prior feature of the tooth image to be restored as supplementary structural information, the problem of lack of collective structural information of the teeth can be improved, and the first decoding feature is used as the overall feature of the tooth image to be restored, so that the output restoration image is consistent with the actual tooth shape, thereby improving the accuracy of the restoration image and reducing the design cost of the restoration.

[0054] First, combine Figures 1 to 5The restoration image generation method provided by the embodiment of the present disclosure is described.

[0055] Figure 1 A flow chart of a restoration image generation method provided by an embodiment of the present disclosure is shown.

[0056] In some embodiments of the present disclosure, Figure 1 The restoration image generation method shown can be executed by an electronic device. The electronic device may include a mobile phone, tablet computer, desktop computer, laptop computer, vehicle-mounted terminal, wearable device, all-in-one device, smart home device, and other devices with communication functions, and may also include devices simulated by a virtual machine or simulator.

[0057] like Figure 1 As shown, the restoration image generation method may include the following steps.

[0058] S110: Acquire an image of the tooth to be repaired.

[0059] In the embodiment of the present disclosure, the image of the tooth to be repaired may be a rendered image corresponding to a three-dimensional scan image including defective teeth.

[0060] The image of the teeth to be repaired may include an upper jaw image, a lower jaw image, and an occlusal groove image. The upper jaw teeth in the upper jaw image are positioned relative to the lower jaw teeth in the lower jaw image.

[0061] Specifically, a three-dimensional scanning device is used to scan the oral cavity to obtain a three-dimensional scanning image, which can be a depth image. The three-dimensional scanning image is sent to an electronic device, which renders the three-dimensional scanning image to obtain an image of the tooth to be repaired, so that the image of the tooth to be repaired includes depth information.

[0062] S120 , extracting first a priori features and first decoding features of the image of the tooth to be restored based on the trained tooth restoration model, and obtaining a restoration image based on the first a priori features and the first decoding features.

[0063] In the disclosed embodiment, the tooth restoration model is obtained by training a preset network based on images of missing teeth and sample restorations.

[0064] In the disclosed embodiments, the dental restoration model can be a deep learning model. For example, the dental restoration model can be a Generative Adversarial Network (GAN), a network model consisting of a generator and a discriminator. The dental restoration model, trained based on images of missing teeth and sample restorations, can extract first prior features and first decoding features from the image of the tooth to be restored, and then generate a restoration image based on the first prior features and first decoding features.

[0065] The first priori features may include semantic features and depth features of the image of the tooth to be repaired at each layer, and the semantic features and depth features may be used to characterize the structural features of the tooth.

[0066] The first decoding feature can be used to characterize the overall feature of the tooth.

[0067] Specifically, when training a tooth restoration model, the electronic device can input a tooth missing image and a sample restoration image into a preset network, extract the prior features and decoding features of the tooth missing image based on the preset network, and perform feature extraction on the prior features and decoding features of the tooth missing image to obtain a predicted restoration image, and adjust the network parameters of the preset network based on the predicted restoration image and the sample restoration image until the output predicted restoration image is the sample restoration image, so that the network parameters of the preset network become stable, and a trained tooth restoration model is obtained.

[0068] Furthermore, after obtaining the trained tooth restoration model, the electronic device obtains the image of the tooth to be restored, inputs the image of the tooth to be restored into the trained tooth restoration model, extracts the first prior feature and the first decoding feature of the image of the tooth to be restored based on the trained tooth restoration model, and continues to perform feature extraction on the first prior feature and the first decoding feature to obtain the restoration image.

[0069] In the embodiment of the present disclosure, after obtaining the image of the tooth to be repaired, the first prior feature and the first decoding feature of the image of the tooth to be repaired can be extracted based on the trained tooth restoration model, and the restoration image can be obtained based on the first prior feature and the first decoding feature, wherein the tooth restoration model is obtained by training a preset network based on the image of the missing tooth and the sample restoration image. Since the tooth restoration model can extract the first prior feature and the first decoding feature of the image of the tooth to be repaired, and use the first prior feature of the image of the tooth to be repaired as supplementary structural information, the problem of lack of collective structural information of the tooth can be improved, and the first decoding feature is used as the overall feature of the image of the tooth to be repaired, so that the output restoration image is consistent with the actual tooth shape, thereby improving the accuracy of the restoration image and reducing the design cost of the restoration.

[0070] In another embodiment of the present disclosure, in order to ensure that the trained tooth restoration model can be used to generate a restoration image of the tooth to be restored, the electronic device may further perform a model training step of the tooth restoration model before executing S110.

[0071] Figure 2 A flow chart of a tooth restoration model training method provided by an embodiment of the present disclosure is shown.

[0072] like Figure 2 As shown, the tooth restoration model training method may further include the following steps before obtaining the image of the tooth to be restored.

[0073] S210: Acquire a tooth missing image and a sample restoration image.

[0074] In the embodiment of the present disclosure, the tooth-missing image may be a sample rendered image including defective teeth.

[0075] In the embodiment of the present disclosure, the sample restoration image may be a restoration image obtained after repairing a defective tooth, and may be a real restoration image.

[0076] Specifically, after acquiring a 3D scanned image from a 3D scanning device, the electronic device responds to a user's tooth cutting and data annotation operations and renders the cut and annotated 3D scanned image. The rendered image then performs data augmentation processing to produce a missing tooth image. Simultaneously, the electronic device renders a 3D scanned image of intact teeth corresponding to the missing tooth image to produce a sample restoration image. Furthermore, the electronic device divides the sample data, consisting of the missing tooth image and the sample restoration image, into a training set, a validation set, and a test set according to a preset construction ratio.

[0077] The data enhancement processing of the rendered image to obtain the tooth missing image may include: marking the occlusal feature points on the teeth of the rendered image, that is, marking the upper jaw feature points and the lower jaw feature points as points of different categories, and using the linear classifier algorithm to optimize the distance from the feature points to the interface to obtain the interface, that is, obtaining Figure 3 The occlusal plane is shown. The teeth in the upper and lower jaws are projected perpendicular to the occlusal plane at the appropriate positions and proportions. Each pixel value represents the normalized distance from the occlusal plane to the corresponding tooth surface, where 0 indicates the surface is attached to the occlusal position and 1 indicates the distance from the upper jaw to the lower jaw. The grayscale value of each pixel is expanded 256 times.

[0078] The preset construction ratio may be a ratio pre-set as needed and is not limited here. For example, the preset ratio may be 15:1:1, that is, the ratio of the training set, the validation set, and the test set may be 15:1:1.

[0079] Optionally, the data augmentation process may include at least one of rotation, scaling, and translation. The rotation angle range in the XY space for the rotation process may be [-5°, 5°]; the scaling range in the scaling process may be [0.8, 1.2]; and the pixel translation range in the XY space for the translation process may be [-8, 8].

[0080] S220: Training a preset network based on the tooth missing image and the sample restoration image to obtain a tooth restoration model that generates a restoration image based on the image of the tooth to be restored.

[0081] Specifically, the electronic device can input the training set in the above-mentioned sample data into a preset network, that is, train the preset network based on the tooth missing image and the sample restoration image, set the hyperparameters of the preset network, and detect the training degree of the preset network based on the preset evaluation criteria to obtain a tooth restoration model that generates a restoration image based on the image of the tooth to be restored.

[0082] Among them, hyperparameters can be external parameters used to train the preset network. Optionally, hyperparameters can include image size of missing teeth images and sample restoration images, number of layers, momentum of convolution kernel weights, weight decay value, batch size of missing teeth images and sample restoration images, learning rate, etc. Weight, image size can be 256×256, number of layers can be 5, momentum can be 0.9, weight decay value can be 0.002, batch size can be 4, and learning rate of the first 60k computing units can be , the learning rate of the last 20k computing units can be .

[0083] The preset evaluation criteria may be parameters pre-set as needed for evaluating the degree of network training. Optionally, the preset evaluation criteria may include at least one of mean absolute error (Mae), mean square error (Mse), root mean square error (Rmse), and peak signal-to-noise ratio (Psnr).

[0084] Optionally, the electronic device may train the dental restoration model based on an end-to-end training approach, and the training process of the dental restoration model may be executed in a graphics processing unit (GPU).

[0085] Furthermore, after the tooth restoration model is obtained using the training samples, the tooth restoration model can be verified and tested using the validation set and the test set to improve the robustness and accuracy of the tooth restoration model.

[0086] In an embodiment of the present disclosure, the dental restoration model may include a priori feature extraction subnetwork and a restoration image prediction subnetwork.

[0087] Accordingly, S220 may include:

[0088] S2201. Extract the second decoding features of the current layer and the second prior features of the current layer in the tooth missing image based on the prior feature extraction subnetwork.

[0089] S2202: Input the second decoding feature and the second prior feature into the restoration image prediction subnetwork to obtain a predicted restoration image.

[0090] S2203. Iteratively train a preset network based on the predicted restoration image and the sample restoration image to obtain a tooth restoration model.

[0091] Among them, the prior feature extraction subnetwork and the restoration image prediction subnetwork can be composed of multiple convolutional layers respectively.

[0092] The second prior features may include semantic features and depth features of the tooth missing image at the current layer, and the semantic features and depth features may be used to characterize the structural features of the teeth.

[0093] The second decoding feature may be a decoding vector of the image of the tooth to be repaired at the current layer, which may be used to characterize the overall features of the tooth.

[0094] The predicted restoration image may be a target context feature when the next layer of the current layer is the last layer of the restoration image prediction subnetwork. The predicted restoration image may be obtained by further feature extraction of the second decoding feature and the second prior feature based on the restoration image prediction subnetwork.

[0095] Specifically, after the electronic device obtains the tooth missing image and the sample restoration image, it can input the tooth missing image into the tooth restoration model, extract the second decoding feature of the current layer and the second prior feature of the current layer in the tooth missing image based on the prior feature extraction subnetwork, and then input the second decoding feature and the second prior feature into the restoration image prediction subnetwork, and continue to perform feature analysis on the second decoding feature and the second prior feature based on the restoration image prediction subnetwork to obtain a predicted restoration image, and then iteratively train the preset network based on the predicted restoration image and the sample restoration image to obtain the tooth restoration model.

[0096] Therefore, in the embodiment of the present disclosure, when training a tooth restoration model, the second decoding feature and the second prior feature of the current layer can be extracted based on the prior feature extraction subnetwork to mine the geometric features and semantic features of the tooth image to be restored as the second prior feature, and then the second decoding feature and the second prior feature are input into the restoration image prediction subnetwork to obtain a predicted restoration image, thereby improving the robustness and accuracy of the tooth restoration model, so that when the trained tooth restoration model is used to generate a restoration image corresponding to the tooth image to be restored, the accuracy of the restoration image is improved and the design cost of the restoration is reduced.

[0097] In another embodiment of the present disclosure, in order to specifically explain the training steps of the tooth restoration model, based on the above embodiment, the tooth restoration model can be further refined, and the refined sub-network can be used to perform feature extraction to obtain a trained tooth restoration model.

[0098] In the disclosed embodiment, the tooth restoration model further includes: an encoding subnetwork and a self-attention subnetwork, and the prior feature extraction subnetwork includes: a semantic segmentation subnetwork and a depth prediction subnetwork.

[0099] Before S2201, the model training method may further include the following steps:

[0100] The first encoding feature of the current layer in the tooth missing image is extracted based on the encoding sub-network.

[0101] The first encoding feature is input into the self-attention sub-network to obtain the second decoding feature.

[0102] Accordingly, S2201 may specifically include the following steps:

[0103] The semantic features of the second decoding features are extracted based on the semantic segmentation subnetwork, and the depth features of the second decoding features are extracted based on the depth prediction subnetwork, and the prior features are composed of the semantic features and the depth features.

[0104] Among them, the encoding subnetwork can be understood as an encoder.

[0105] Among them, the self-attention sub-network can be understood as a neural network layer with an added attention mechanism.

[0106] Among them, the semantic segmentation subnetwork and the depth prediction subnetwork can both be located in the decoder. The semantic segmentation subnetwork can extract semantic features from the input second decoding features to obtain semantic features, and the depth prediction subnetwork can extract depth information from the second decoding features to obtain depth features.

[0107] For example, Figure 4 A structural schematic diagram of a tooth restoration model provided by an embodiment of the present disclosure is shown. Figure 5 A schematic diagram of the principles of a tooth restoration model provided by an embodiment of the present disclosure is shown.

[0108] like Figure 4 As shown, the dental restoration model includes a prior feature extraction subnetwork 10, a restoration image prediction subnetwork 20, an encoding subnetwork 30, and a self-attention subnetwork 40.

[0109] The prior feature extraction subnetwork 10 includes a semantic segmentation subnetwork 11 and a depth prediction subnetwork 12.

[0110] See also Figure 4 and Figure 5 The above S11~S13 are explained in detail.

[0111] Step 1: Get the tooth-missing image, which includes the current jaw image X and the opposite jaw image Y.

[0112] The current jaw image X and the opposite jaw image Y may be an upper jaw image and a lower jaw image, respectively.

[0113] Step 2: Input the tooth loss image into the tooth restoration model and extract the first encoding feature of the current layer based on the encoding sub-network 30 ,in, is the number of layers, i.e. the current layer, , It can be 5.

[0114] It should be noted that when extracting the first encoding feature of the current layer Before, you can also extract the second encoding features of the next layer Encoded features to the first layer .

[0115] It should be noted that since the number of layers of the encoder is symmetrical with the number of layers of the decoder, the current layer in the encoder is , the previous layer is , then in the decoder, the current layer is , the next layer is .

[0116] Step 3: Electronics will first encode the features Input to the self-attention sub-network 40 to obtain the second decoding feature , the second decoding feature It can be a context feature map.

[0117] Step 4: The electronic device extracts the second decoding feature based on the semantic segmentation sub-network 11 Semantic features of , and extract the depth feature of the second decoding feature based on the depth prediction sub-network 12 , and by semantic features and deep features Constitute the prior features.

[0118] in, , .

[0119] Among them, the function and function Both can include 5 residual learning modules.

[0120] Furthermore, in the embodiment of the present disclosure, before S2202, S220 may further include:

[0121] The second encoding feature of the next layer in the tooth missing image is extracted based on the encoding sub-network.

[0122] Specifically, the electronic device can extract the second encoding features of the next layer in the tooth missing image based on the encoding sub-network.

[0123] Accordingly, S2202 may specifically include the following steps:

[0124] S21. Input the second decoding feature, the second encoding feature, and the second prior feature into the restoration image prediction subnetwork to obtain a predicted restoration image.

[0125] The restoration image prediction subnetwork may be a neural network layer located in the decoder for predicting the restoration image.

[0126] In the disclosed embodiment, the restoration image prediction subnetwork includes: a context feature extraction subnetwork, a mask mapping subnetwork, and a fusion subnetwork.

[0127] Accordingly, S21 may specifically include the following steps:

[0128] S211: Input the second decoding feature and the second encoding feature into the context feature extraction subnetwork to obtain the intermediate context feature of the next layer.

[0129] S212: Input the second prior feature into the mask mapping subnetwork to obtain the mask mapping feature of the next layer.

[0130] S213. For each layer, the intermediate context features of the next layer and the mask mapping features of the next layer are input into the fusion sub-network to obtain the target context features of the next layer.

[0131] S214. According to the forward propagation method, the current layer is reduced until the next layer is 0, and the target context features of the next layer are used as the predicted restoration image.

[0132] Specifically, first, the electronic device extracts the second decoding feature of the current layer, the second prior feature of the current layer, and the second encoding feature of the next layer through the prior feature extraction subnetwork in the dental restoration model, and then inputs the second decoding feature and the second encoding feature into the context feature extraction subnetwork to obtain the intermediate context feature of the next layer; secondly, the second prior feature is input into the mask mapping subnetwork to obtain the mask mapping feature of the next layer; then, for each layer, the intermediate context feature of the next layer and the mask mapping feature of the next layer are fused using the fusion subnetwork to obtain the target context feature of the next layer; finally, S211~S213 are iteratively executed, that is, according to the forward propagation method, the current layer is reduced until the next layer is 0, and the target context feature of the next layer is used as the predicted restoration image.

[0133] Continue to see Figure 4 , the restoration image prediction subnetwork 11 includes: context feature extraction subnetwork 21, mask mapping subnetwork 22 and fusion subnetwork 23.

[0134] See also Figure 4 The above S211~S213 are explained in detail.

[0135] Step 1: Decode the second feature and the second coded feature Input to the context feature extraction sub-network 21 to obtain the intermediate context features of the next layer .

[0136] Step 2: Input the second prior feature into the mask mapping subnetwork 22, that is, input the semantic feature obtained by the semantic segmentation subnetwork 13 and the depth feature obtained by the depth prediction subnetwork 14 into the mask mapping subnetwork 22 to obtain the mask mapping feature of the next layer .

[0137] Step 3: For each layer, the intermediate context features of the next layer are and the mask mapping features of the next layer Input to the fusion sub-network 23 to obtain the target context features of the next layer , repeat steps 1 to 3.

[0138] in, , is the feature forward propagation process.

[0139] Step 4: Follow the forward propagation method to lower the current layer until the next layer 0, next layer Target context features As a predicted restoration image .

[0140] It should be noted that the context feature extraction subnetwork, mask mapping subnetwork and fusion subnetwork can also be composed of multiple neural network layers with different functions.

[0141] S211 may specifically include the following steps:

[0142] S2111. Deconvolve the second decoding feature, concatenate the deconvolved decoding feature with the second encoding feature to generate a fusion feature of the next layer, and determine the mean and variance of the fusion feature of the next layer in a specific channel.

[0143] S2112. Deconvolve the semantic features of the current layer to obtain the deconvoluted semantic features of the next layer, and pass the deconvoluted semantic features of the next layer through a convolutional network to obtain a radial transformation parameter tensor with the same number of layers as the fusion features of the next layer.

[0144] S2113. The radiation transformation parameter tensor, the fusion features of the next layer, the mean of the fusion features of the next layer in a specific channel, and the variance of the fusion features of the next layer in the specific channel are fused again to obtain the intermediate context features of the next layer.

[0145] Figure 6 A schematic diagram of the principle of a restoration image prediction subnetwork provided by an embodiment of the present disclosure is shown.

[0146] See also Figure 6 Exemplary explanation S2111~S2113.

[0147] Step 1: Decode the second feature Deconvolution is performed to expand the resolution and the deconvolution decoding features are combined with the second encoding features Cascade to generate the next layer Fusion features , and determine the fusion features exist Channel mean and variance .

[0148] Step 2: Move the current layer Semantic features of Perform deconvolution to get the next layer Deconvolution semantic features , and the next layer Deconvolution semantic features Through convolutional networks , get the next layer Fusion features Radial transformation parameter tensors with the same number of layers 、 .

[0149] Among them, convolutional network The number of convolutional layers can be 2.

[0150] Step 3: Transform the parameter tensor 、 , next level Fusion features , next level Fusion features The mean value in a specific channel , next level Fusion features Variance in a specific channel Fusion again to get the next layer Intermediate context features .

[0151] S212 may specifically include the following steps:

[0152] S2121. Extract the maximum value of each pixel position of the deconvolution semantic feature of the next layer at the channel level, and binarize it through a fixed threshold to generate the reliability mask of the next layer.

[0153] S2122. Use a transformation function to map the reliability mask to the feature space to obtain the mask mapping features of the next layer.

[0154] Continue to see Figure 6 Exemplary explanation S2121~S2122.

[0155] Step 1: Extract the next layer of deconvolution semantic features Each pixel position The maximum value at the channel level and through a fixed threshold Binarization is performed to correct unreliable pixels and generate the next layer Reliability mask .

[0156] Among them, the pixel position The corresponding elements are .

[0157] Among them, the pixel position Corresponding The formula is:

[0158]

[0159] Step 2: Using the Transformation Function , and the reliability mask is transformed based on the convolutional network Map to the feature space to obtain the mask mapping features of the next layer .

[0160] Among them, convolutional network The number of convolutional layers can be 4.

[0161] S213 may specifically include the following steps:

[0162] S2131. For each layer, concatenate the mask mapping features of the next layer and the fusion features of the next layer to obtain a cascade feature.

[0163] S2132. Input the fused features of the next layer into the first context processing function, and input the cascade features into the second context processing function, fuse the context features output by the first context processing function and the second context processing function, and obtain the target context features of the next layer.

[0164] Continue to see Figure 6 Exemplary explanation S2131~S2132.

[0165] Step 1: For each layer, move the next layer Mask mapping features and the next layer Fusion features Perform cascade to obtain cascade features.

[0166] The pixel-by-pixel update method is as follows:

[0167] , e and Element-wise multiplication and subtraction.

[0168] Step 2: Fusion features of the next layer Input to the first context handler function , and input the cascade features into the second context processing function , fuse the context features output by the first context processing function and the second context processing function to obtain the next layer Target context features .

[0169] in, .

[0170] Among them, Cat is a cascade operation.

[0171] Among them, the function and function Both can include 5 residual learning modules.

[0172] Furthermore, for S214, according to the forward propagation method, the current layer is reduced until the next layer is 0, and the target context features of the next layer are used as the predicted restoration image.

[0173] Among them, the predicted restoration image is .

[0174] in, For the generator.

[0175] Therefore, in the disclosed embodiment, the dental restoration model can include a priori feature extraction subnetwork and a restoration image prediction subnetwork, and the priori feature extraction subnetwork and the restoration image prediction subnetwork can be further subdivided into multiple subnetworks. Feature extraction, information cascading, and information fusion are performed based on each subnetwork to improve the robustness and accuracy of the dental restoration model. In addition, by extracting the maximum value of each pixel position at the channel level of the deconvolution semantic features of the next layer and binarizing it with a fixed threshold, a reliability mask for the next layer is generated to guide the subsequent feature processing process, so that the dental restoration model has a self-assessment mechanism to identify and correct erroneous predictions.

[0176] In another embodiment of the present disclosure, in order to improve the accuracy of restoration information extraction, an intermediate feature layer loss function and a structural consistency loss function may be introduced to improve the detection capability of the dental restoration model for the intermediate layer.

[0177] In the embodiment of the present disclosure, S2203 may specifically include the following steps:

[0178] S31. Calculate the loss function of the preset network based on the predicted restoration image and the sample restoration image.

[0179] S32. Iteratively train the preset network based on the loss function until the loss function is less than the preset loss value, thereby obtaining a tooth restoration model.

[0180] In an embodiment of the present disclosure, optionally, the loss function is obtained by weighted averaging a reconstruction loss function, an adversarial loss function, a first intermediate feature layer loss function, a second intermediate feature layer loss function, and a structural consistency loss function.

[0181] Among them, the reconstruction loss function You can use paradigm, so that the generated predicted restoration images At the image level, as close as possible to the sample restoration image .

[0182] Optionally, the calculation formula of the reconstruction loss function can be:

[0183]

[0184] Among them, the adversarial loss function Can be used to evaluate predicted restoration images and sample restoration images The potential distribution between the two can be based on the predicted restoration image and sample restoration images Depth information calculation.

[0185] Optionally, the adversarial loss function can be calculated as:

[0186]

[0187] in, is the expectation function. The adversarial loss function can evaluate the predicted restoration image based on the depth information of the current jaw of the defective tooth and the opposite jaw. and sample restoration images The potential distribution between the depth information.

[0188] Among them, the first intermediate feature layer loss function and the second intermediate feature layer loss function can be used to narrow the gap between the second prior feature and the predicted restoration image at different layers.

[0189] Optionally, the first intermediate feature layer loss function can be a loss function between the semantic features in the second prior features and the semantic features of the sample restoration image; the second intermediate feature layer loss function can be a loss function between the depth features in the second prior features and the depth features of the sample restoration image.

[0190] Optionally, the calculation formula of the first intermediate feature layer loss function can be:

[0191]

[0192] Optionally, the calculation formula of the loss function of the second intermediate feature layer can be:

[0193]

[0194] in, The upsampling operation is continued to meet the resolution requirements.

[0195] Among them, the structural consistency loss function can be used to enhance the structural consistency between the second prior feature and any layer or predicted restoration image.

[0196] Among them, the structural consistency loss function can be calculated based on the depth features and semantic features in the second prior features and the depth features and semantic features in the sample restoration image.

[0197] Optionally, the calculation formula of the structural consistency loss function can be:

[0198] in, Can be a predicted restoration image .

[0199] Among them, after obtaining the second prior feature, the semantic discriminator It can be used to determine whether the input depth feature is the depth feature of the predicted restoration image or the depth feature in the second prior feature.

[0200] Therefore, the loss function can be expressed as:

[0201]

[0202] in, is the loss function, 、 、 and are weights respectively, and + + + =1.

[0203] Figure 7a shows the depth image corresponding to the tooth missing image, Figure 7b shows the depth image corresponding to the sample restoration image, Figure 7c The depth image corresponding to the restoration image obtained based on the traditional generative adversarial network is shown. Figure 7d The depth image corresponding to the restoration image obtained based on the edge guidance method is shown. Figure 7e The figure shows the depth image corresponding to the restoration image obtained based on the tooth restoration model.

[0204] Will Figure 7c~Figure 7e Respectively Figure 7a and Figure 7b By comparison, it can be seen that the depth image corresponding to the restoration image obtained based on the tooth restoration model is closer to the depth image corresponding to the sample restoration image.

[0205] Figure 8a A sample restoration image is shown, Figure 8b The restoration image obtained based on the traditional generative adversarial network is shown. Figure 8c shows the restoration image obtained based on the generative adversarial network with a priori feature extraction subnetwork, Figure 8d The restoration image is shown based on the generative adversarial network with a prior feature extraction subnetwork and a self-evaluation mechanism. Figure 8eThe restoration image is shown based on a generative adversarial network with a prior feature extraction subnetwork, a self-evaluation mechanism, and an introduced structural consistency loss function.

[0206] Will Figure 8b~Figure 8e The feature information in the dotted box is respectively Figure 8a From the comparison of the feature information in the dotted box, it can be seen that the depth image corresponding to the restoration image obtained based on the tooth restoration model is closer to the depth image corresponding to the sample restoration image.

[0207] Figure 9a An image showing missing teeth is shown. Figure 9b A sample restoration image is shown, Figure 9c The restoration image obtained based on the traditional generative adversarial network is shown. Figure 9d shows the restoration image obtained based on deep convolutional neural network, Figure 9e The figure shows the restoration image obtained based on the tooth restoration model.

[0208] Will Figure 9c~Figure 9e The feature information in the dotted box is respectively Figure 9a and Figure 9b From the comparison of the feature information in the dotted box, it can be seen that the restoration image obtained based on the tooth restoration model is closer to the sample restoration image.

[0209] Therefore, in the embodiment of the present disclosure, the intermediate feature layer loss function and the structural consistency loss function are introduced to improve the detection capability of the dental restoration model for the intermediate layer, thereby improving the accuracy of the restoration information extraction.

[0210] The present disclosure also provides a device for generating the above-mentioned restoration image. Figure 10 In the disclosed embodiment, the restoration image generation device may be an electronic device. The electronic device may include a mobile terminal, a tablet computer, an in-vehicle terminal, a wearable electronic device, a virtual reality (VR) all-in-one device, a smart home device, or other device with communication capabilities.

[0211] Figure 10 A schematic structural diagram of a restoration image generation device provided by an embodiment of the present disclosure is shown.

[0212] like Figure 10 As shown, the restoration image generating device 1000 may include: an image acquisition module 1110 and an image generation module 1120 .

[0213] The image acquisition module 1110 is used to acquire an image of the tooth to be repaired.

[0214] The image generation module 1120 is used to extract the first prior feature and the first decoding feature of the image of the tooth to be restored based on the trained tooth restoration model, and obtain the restoration image based on the first prior feature and the first decoding feature, wherein the tooth restoration model is obtained by training a preset network based on the tooth missing image and the sample restoration image.

[0215] In the embodiment of the present disclosure, after obtaining the image of the tooth to be repaired, the first prior feature and the first coding feature of the image of the tooth to be repaired can be extracted based on the trained tooth restoration model, and the restoration image can be obtained based on the first prior feature and the first coding feature, wherein the tooth restoration model is obtained by training a preset network based on the image of the missing tooth and the sample restoration image. Since the tooth restoration model can extract the first prior feature and the first decoding feature of the image of the tooth to be repaired, and use the first prior feature of the image of the tooth to be repaired as supplementary structural information, the problem of lack of collective structural information of the tooth can be improved, and the first decoding feature is used as the overall feature of the image of the tooth to be repaired, so that the output restoration image is consistent with the actual tooth shape, thereby improving the accuracy of the restoration image and reducing the design cost of the restoration.

[0216] Optionally, the device may further include: a tooth restoration model training module;

[0217] The tooth restoration model training module may include: a sample image acquisition unit and a training unit.

[0218] The sample image acquisition unit is used to acquire the image of missing teeth and the image of the sample restoration;

[0219] The training unit is used to train a preset network based on the tooth missing image and the sample restoration image to obtain a tooth restoration model that generates a restoration image based on the image of the tooth to be restored.

[0220] Optionally, the tooth restoration model includes a prior feature extraction subnetwork and a restoration image prediction subnetwork;

[0221] Accordingly, the training unit may include: a feature extraction subunit, a prediction subunit, and an iterative training subunit.

[0222] The feature extraction subunit is used to extract the second decoding feature of the current layer and the second prior feature of the current layer in the tooth missing image based on the prior feature extraction subnetwork;

[0223] A prediction subunit, configured to input the second decoding feature and the second prior feature into a restoration image prediction subnetwork to obtain a predicted restoration image;

[0224] The iterative training subunit is used to iteratively train the preset network based on the predicted restoration image and the sample restoration image to obtain a tooth restoration model.

[0225] Optionally, the tooth restoration model further includes: an encoding subnetwork and a self-attention subnetwork; the prior feature extraction subnetwork includes: a semantic segmentation subnetwork and a depth prediction subnetwork;

[0226] Accordingly, the device further comprises: a first encoding feature extraction unit and a second decoding feature extraction unit;

[0227] A first coding feature extraction unit is used to extract a first coding feature of a current layer in the tooth missing image based on the coding sub-network;

[0228] A second decoding feature extraction unit is used to input the first encoding feature into the self-attention sub-network to obtain a second decoding feature;

[0229] Correspondingly, the feature extraction subunit can also be used to extract semantic features of the second decoding features based on the semantic segmentation subnetwork, and extract depth features of the second decoding features based on the depth prediction subnetwork, so that the semantic features and depth features constitute the prior features.

[0230] Optionally, the feature extraction subunit may also be used to extract a second encoding feature of the next layer in the tooth loss image based on the prior feature extraction subnetwork;

[0231] Correspondingly, the prediction subunit can also be used to input the second decoding feature, the second encoding feature and the second prior feature into the restoration image prediction subnetwork to obtain a predicted restoration image.

[0232] Optionally, the restoration image prediction subnetwork includes: a context feature extraction subnetwork, a mask mapping subnetwork, and a fusion subnetwork;

[0233] The prediction subunit can also be used to input the second decoding feature and the second encoding feature into the context feature extraction subnetwork to obtain the intermediate context feature of the next layer;

[0234] Input the second prior feature into the mask mapping subnetwork to obtain the mask mapping feature of the next layer;

[0235] For each layer, the intermediate context features of the next layer and the mask mapping features of the next layer are input into the fusion sub-network to obtain the target context features of the next layer;

[0236] According to the forward propagation method, the current layer is reduced until the next layer is 0, and the target context features of the next layer are used as the predicted restoration image.

[0237] Optionally, the iterative training subunit may also be used to calculate a loss function of a preset network based on the predicted restoration image and the sample restoration image;

[0238] The preset network is iteratively trained based on the loss function until the loss function is less than the preset loss value to obtain a tooth restoration model.

[0239] Optionally, the loss function is obtained by weighted averaging a reconstruction loss function, an adversarial loss function, a first intermediate feature layer loss function, a second intermediate feature layer loss function, and a structural consistency loss function.

[0240] It should be noted that Figure 10 The restoration image generating apparatus 1000 shown can execute Figures 1 to 9e The various steps in the method embodiment shown are implemented Figures 1 to 9e The various processes and effects in the illustrated method embodiment are not described in detail here.

[0241] Figure 11 A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown.

[0242] like Figure 11 As shown, the electronic device 1100 may include a processor 1101 and a memory 1102 storing computer program instructions.

[0243] Specifically, the processor 1101 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0244] Memory 1102 may include a large-capacity memory for information or instructions. By way of example and not limitation, memory 1102 may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 1102 may include removable or non-removable (or fixed) media. Where appropriate, memory 1102 may be internal or external to the integrated gateway device. In a specific embodiment, memory 1102 is a non-volatile solid-state memory. In a specific embodiment, memory 1102 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0245] The processor 1101 reads and executes the computer program instructions stored in the memory 1102 to perform the steps of the data acquisition method provided in the embodiment of the present disclosure.

[0246] In one example, the electronic device may further include a transceiver 1103 and a bus 1104. Figure 11 As shown, the processor 1101 , the memory 1102 and the transceiver 1103 are connected via a bus 1104 and communicate with each other.

[0247] The bus 1104 may include hardware, software, or both. By way of example, and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, the bus 1104 may include one or more buses. Although embodiments herein describe and illustrate a particular bus, this application contemplates any suitable bus or interconnect.

[0248] The following is an embodiment of a computer-readable storage medium provided in an embodiment of the present disclosure. The computer-readable storage medium and the restoration image generation method of each of the above-mentioned embodiments belong to the same inventive concept. For details not fully described in the embodiment of the computer-readable storage medium, please refer to the embodiment of the above-mentioned restoration image generation method.

[0249] This embodiment provides a storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are used to perform a method for generating a restoration image. The method includes:

[0250] Acquire an image of the tooth to be restored;

[0251] Based on the trained tooth restoration model, the first prior feature and the first decoding feature of the tooth image to be restored are extracted, and the restoration image is obtained based on the first prior feature and the first decoding feature, wherein the tooth restoration model is obtained by training a preset network based on the tooth missing image and the sample restoration image.

[0252] Of course, the computer executable instructions of the storage medium provided by the embodiment of the present disclosure are not limited to the above method operations, but can also execute the relevant operations in the restoration image generation method provided by any embodiment of the present disclosure.

[0253] Through the above description of the embodiments, those skilled in the art can clearly understand that the present disclosure can be implemented with the help of software and necessary general-purpose hardware. Of course, it can also be implemented with hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the existing technology, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., and includes a number of instructions for enabling a computer cloud platform (which can be a personal computer, server, or network cloud platform, etc.) to execute the restoration image generation method provided by various embodiments of the present disclosure.

[0254] Note that the above are only preferred embodiments of the present disclosure and the technical principles employed. Those skilled in the art will understand that the present disclosure is not limited to the specific embodiments herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present disclosure. Therefore, although the present disclosure has been described in more detail through the above embodiments, the present disclosure is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present disclosure, and the scope of the present disclosure is determined by the scope of the appended claims.

Claims

1. A method for generating a restoration image, characterized in that: include: Acquire an image of the tooth to be restored; Extracting the prior features, encoding features, and decoding features of the image of the tooth to be restored based on the trained tooth restoration model, and obtaining a restoration image based on the prior features, encoding features, and decoding features, wherein the tooth restoration model is obtained by training a preset network based on the image of the missing tooth and the sample restoration image; the tooth restoration model includes an encoding subnetwork, a self-attention subnetwork, a prior feature extraction subnetwork, and a restoration image prediction subnetwork; The training method of the tooth restoration model comprises: Acquiring the tooth missing image and the sample restoration image; extracting a first encoding feature of a current layer in the tooth missing image based on the encoding subnetwork; Inputting the first encoding feature into the self-attention sub-network to obtain a second decoding feature; Extracting a second priori feature of the current layer corresponding to the second decoded feature of the current layer in the tooth missing image based on the priori feature extraction subnetwork; extracting a second encoding feature of a next layer in the tooth missing image based on the encoding subnetwork; Inputting the second decoding feature, the second encoding feature and the second prior feature into the restoration image prediction subnetwork to obtain a predicted restoration image; The preset network is iteratively trained based on the predicted restoration image and the sample restoration image to obtain the tooth restoration model.

2. The method according to claim 1, characterized in that The prior feature extraction subnetwork includes: a semantic segmentation subnetwork and a depth prediction subnetwork; The step of extracting the second decoding feature of the current layer and the second priori feature of the current layer in the tooth missing image based on the priori feature extraction subnetwork includes: The semantic features of the second decoding features are extracted based on the semantic segmentation subnetwork, and the depth features of the second decoding features are extracted based on the depth prediction subnetwork, and the prior features are formed by the semantic features and the depth features.

3. The method according to claim 1, characterized in that Before inputting the second decoding feature and the second prior feature into the restoration image prediction subnetwork to obtain a predicted restoration image, the method further includes: Extracting a second encoding feature of the next layer in the tooth missing image based on the prior feature extraction subnetwork; The step of inputting the second decoding feature and the second prior feature into the restoration image prediction subnetwork to obtain a predicted restoration image includes: The second decoding feature, the second encoding feature and the second prior feature are input into the restoration image prediction subnetwork to obtain the predicted restoration image.

4. The method according to claim 3, characterized in that The restoration image prediction subnetwork includes: a context feature extraction subnetwork, a mask mapping subnetwork and a fusion subnetwork; The step of inputting the second decoding feature, the second encoding feature, and the second prior feature into the restoration image prediction subnetwork to obtain the predicted restoration image includes: Inputting the second decoding feature and the second encoding feature into the context feature extraction subnetwork to obtain the intermediate context feature of the next layer; Inputting the second priori feature into the mask mapping subnetwork to obtain the mask mapping feature of the next layer; For each layer, the intermediate context features of the next layer and the mask mapping features of the next layer are input into the fusion sub-network to obtain the target context features of the next layer; According to the forward propagation method, the current layer is reduced until the next layer is 0, and the target context feature of the next layer is used as the predicted restoration image.

5. The method according to claim 1, wherein The iterative training of the preset network based on the predicted restoration image and the sample restoration image to obtain the tooth restoration model includes: Calculating a loss function of the preset network based on the predicted restoration image and the sample restoration image; The preset network is iteratively trained based on the loss function until the loss function is less than a preset loss value, thereby obtaining the tooth restoration model.

6. The method according to claim 5, characterized in that The loss function is obtained by weighted average calculation of the reconstruction loss function, the adversarial loss function, the first intermediate feature layer loss function, the second intermediate feature layer loss function and the structural consistency loss function.

7. A restoration image generating device, characterized in that: include: An image acquisition module, used to acquire an image of the tooth to be repaired; An image generation module is configured to extract prior features, encoding features, and decoding features of the image of the tooth to be restored based on a trained tooth restoration model, and obtain a restoration image based on the prior features, encoding features, and decoding features, wherein the tooth restoration model is obtained by training a preset network based on images of missing teeth and sample restoration images; the tooth restoration model includes an encoding subnetwork, a self-attention subnetwork, a prior feature extraction subnetwork, and a restoration image prediction subnetwork; The training of the tooth restoration model includes: Acquiring the tooth missing image and the sample restoration image; extracting a first encoding feature of a current layer in the tooth missing image based on the encoding subnetwork; Inputting the first encoding feature into the self-attention sub-network to obtain a second decoding feature; Extracting a second priori feature of the current layer corresponding to the second decoded feature of the current layer in the tooth missing image based on the priori feature extraction subnetwork; extracting a second encoding feature of a next layer in the tooth missing image based on the encoding subnetwork; Inputting the second decoding feature, the second encoding feature and the second prior feature into the restoration image prediction subnetwork to obtain a predicted restoration image; The preset network is iteratively trained based on the predicted restoration image and the sample restoration image to obtain the tooth restoration model.

8. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the restoration image generation method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for generating a restoration image according to any one of claims 1 to 6 is implemented.

Images