Method and device for generating three-dimensional dental model

Abstract

The invention provides a method and a device for generating a three-dimensional dental model. The method comprises the following steps: preprocessing point cloud data of an orthodontic three-dimensional tooth jaw model in an existing data set, and extracting geometric features from the preprocessed point cloud data; obtaining a to-be-trained vector quantization variational auto-encoder model, and training in the following manner: encoding the extracted geometric features into a hidden space based on an encoder of the vector quantization variational auto-encoder model, and further reconstructing the point cloud data of the orthodontic three-dimensional dental model from the encoding information in the hidden space through a decoder; a diffusion model is further trained in the hidden space of the vector quantization variational auto-encoder model, the diffusion model is used for carrying out diffusion processing on encoding information input into the hidden space to generate new encoding information, and therefore the trained decoder generates new point cloud data of the orthodontic three-dimensional tooth jaw model according to the new encoding information.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of computer technology, and more particularly to a method and device for generating a three-dimensional dental model. Background Art

[0002] Currently, malocclusion has become one of the three major oral diseases in the world. Malocclusion can lead to many oral-related diseases, and orthodontic treatment is the main means to solve malocclusion. Orthodontic treatment refers to correcting irregular or deformed tooth arrangement to a good arrangement.

[0003] Traditional orthodontic treatment usually requires experienced dentists or dental technicians to manually design treatment plans. With the continuous development of machine learning technology, automatic tooth arrangement technology based on neural networks (such as TANet, PSTN, TADPM, Tooth Motion Diffusion, etc.) has emerged in recent years to predict ideal tooth arrangement, thereby assisting dentists in formulating orthodontic treatment plans.

[0004] In the related technology, automatic tooth arrangement technology requires a pair of orthodontic three-dimensional jaw models (i.e., the three-dimensional jaw model before orthodontics and the three-dimensional jaw model after orthodontics) as training data. However, since the data collection and organization of the three-dimensional jaw models before and after orthodontics is a difficult and time-consuming task, the existing automatic tooth arrangement technology faces the problem of insufficient training data, especially for complex cases (such as crossbite, overjet, etc.), the three-dimensional jaw model data is very scarce; as a result, the existing automatic tooth arrangement prediction results are not ideal.

[0005] In summary, there is a need to provide a simple and low-cost solution for generating three-dimensional dental models. Summary of the invention

[0006] The embodiments of this specification provide a method and device for generating a three-dimensional dental model.

[0007] According to a first aspect of an embodiment of the present specification, a method for generating a three-dimensional dental jaw model is provided, the method comprising:

[0008] Preprocessing the point cloud data of the orthodontic three-dimensional dental jaw model in the existing data set, and extracting the geometric features of the orthodontic three-dimensional dental jaw model from the preprocessed point cloud data of the orthodontic three-dimensional dental jaw model;

[0009] Obtain a vector quantized variational autoencoder model to be trained, and train the vector quantized variational autoencoder model in the following manner:

[0010] The encoder based on the vector quantized variational autoencoder model encodes the extracted geometric features of the orthodontic three-dimensional dental model into a latent space, and further reconstructs the point cloud data of the orthodontic three-dimensional dental model through the decoder of the vector quantized variational autoencoder model from the encoded information in the latent space;

[0011] In which, a diffusion model is also trained in the latent space of the vector quantized variational autoencoder model, and the diffusion model is used to diffuse the encoded information input into the latent space to generate new encoded information, so that the trained decoder generates point cloud data of a new post-orthodontic three-dimensional dental model according to the new encoded information.

[0012] Optionally, the step of preprocessing the point cloud data of the orthodontic three-dimensional dental jaw model in the existing data set and extracting geometric features of the orthodontic three-dimensional dental jaw model from the preprocessed point cloud data of the orthodontic three-dimensional dental jaw model includes:

[0013] The point cloud data of the orthodontic three-dimensional dental model in the existing data set is centralized and normalized to obtain a tooth sequence consisting of 32 teeth corresponding to the orthodontic three-dimensional dental model; wherein the tooth sequence is a 2×2×8 spatial structure;

[0014] Each tooth is divided into an r×r×r voxel structure, and the PointNet network is used to extract features from the voxels of each tooth. The features extracted from all teeth are aggregated to obtain the geometric features of the orthodontic 3D dental model with a shape of [2r, 2r, 8r], where r is the resolution of the tooth.

[0015] Optionally, the vector quantization variational autoencoder model includes a VQ-VAE model using a 3D U-Net network architecture;

[0016] The encoder based on the vector quantized variational autoencoder model encodes the extracted geometric features of the orthodontic three-dimensional dental model into a latent space, and further reconstructs the point cloud data of the orthodontic three-dimensional dental model through the decoder of the vector quantized variational autoencoder model from the encoded information in the latent space, including:

[0017] Using the encoder of the VQ-VAE model, a multi-layer convolution operation is performed on the extracted geometric features of the three-dimensional dental model after orthodontics, so as to encode the geometric features into a latent space, and obtain a latent vector corresponding to the three-dimensional dental model after orthodontics;

[0018] Further using the VQ-VAE decoder to perform a multi-layer deconvolution operation on the latent vectors in the latent space to obtain point cloud data of the orthodontic three-dimensional dental model reconstructed based on the latent vectors;

[0019] The reconstruction loss function of the decoder is calculated using the point cloud data of the reconstructed three-dimensional dental jaw model after orthodontic treatment and the point cloud data of the three-dimensional dental jaw model in the existing data set.

[0020] Optionally, the calculation of the reconstruction loss function of the decoder using the point cloud data of the reconstructed three-dimensional dental jaw model after orthodontic treatment and the point cloud data of the three-dimensional dental jaw model in the existing data set includes:

[0021] respectively calculating the chamfer distance of each tooth between the point cloud data of the reconstructed orthodontic three-dimensional dental jaw model and the point cloud data of the orthodontic three-dimensional dental jaw model in the existing data set;

[0022] The average value of the chamfer distance of each pair of teeth is calculated to obtain the reconstruction loss function.

[0023] Optionally, the diffusion model is used to perform diffusion processing on the coded information input into the latent space to generate new coded information, so that the trained decoder generates point cloud data of a new post-orthodontic three-dimensional dental model according to the new coded information, including:

[0024] The diffusion model performs noise processing on the latent vector of the latent space in which the encoder encodes the features of the real three-dimensional dental model after orthodontics, and uses the denoiser of the diffusion model to perform denoising processing to obtain a new latent vector; further, the trained decoder is used to generate point cloud data of a new three-dimensional dental model after orthodontics according to the new latent vector.

[0025] Optionally, also include:

[0026] Obtain a pre-orthodontic three-dimensional dental model of a specified tooth arrangement style from an existing data set;

[0027] Extracting the style features of the tooth arrangement in the three-dimensional dental model before orthodontics with the specified tooth arrangement style, and extracting the shape features of the point cloud data of the new three-dimensional dental model after orthodontics generated by the trained decoder;

[0028] The style feature and the shape feature are used as dual conditions of the diffusion model to guide the diffusion model to generate a rigid transformation parameter of each tooth in the three-dimensional dental jaw model after orthodontics that meets the requirements;

[0029] The generated rigid transformation parameters are applied to each tooth in the three-dimensional jaw model after orthodontics to obtain point cloud data of the three-dimensional jaw model before orthodontics corresponding to the three-dimensional jaw model after orthodontics, and the point cloud data of the three-dimensional jaw model before orthodontics has the specified tooth arrangement style.

[0030] Optionally, the step of extracting the style features of the tooth arrangement in the three-dimensional pre-orthodontic dental model of the specified tooth arrangement style includes:

[0031] Centralizing and normalizing the point cloud data of the pre-orthodontic three-dimensional dental model of the specified tooth arrangement style to obtain a tooth sequence composed of 32 teeth; wherein the tooth sequence is a 2×2×8 spatial structure;

[0032] Each tooth is divided into an r×r×r voxel structure, and the PointNet network is used to extract features from the voxels of each tooth. The features extracted from all teeth are aggregated to obtain geometric features with a shape of [2r, 2r, 8r]; where r is the resolution of the tooth;

[0033] Mean pooling and variance pooling are used inside the voxel, and after linear transformation, the style feature with a shape of [2r, 2r, 8r] is obtained.

[0034] Optionally, extracting shape features of point cloud data of a new orthodontic three-dimensional dental jaw model generated by the trained decoder includes:

[0035] The point cloud data of the new orthodontic three-dimensional dental model generated by the trained decoder is centralized and normalized to obtain a tooth sequence consisting of 32 teeth; wherein the tooth sequence is a 2×2×8 spatial structure;

[0036] Each tooth is divided into an r×r×r voxel structure, and the PointNet network is used to extract features from the voxels of each tooth. The features extracted from all teeth are aggregated to obtain geometric features with a shape of [2r, 2r, 8r]; where r is the resolution of the tooth;

[0037] Max pooling is used inside each tooth to aggregate the features of all voxels and obtain shape features.

[0038] Optionally, the diffusion model includes a diffusion model based on a 3D U-Net network architecture;

[0039] The method of using the style feature and the shape feature as dual conditions of the diffusion model to guide the diffusion model to generate a rigid transformation parameter of each tooth in the three-dimensional jaw model after orthodontics that meets the requirements includes:

[0040] The style features and shape features are input into each layer of the 3D U-Net network through a multi-layer perceptron, guiding the scattered model to generate rigid body transformation parameters of the teeth, and outputting the rigid transformation parameters for each tooth in the three-dimensional dental model after orthodontics.

[0041] Optionally, the step of applying the generated rigid transformation parameters to each tooth in the three-dimensional jaw model after orthodontics to obtain a three-dimensional jaw model before orthodontics corresponding to the three-dimensional jaw model after orthodontics, and the three-dimensional jaw model before orthodontics has the specified tooth arrangement style, comprises:

[0042] Each tooth in the post-orthodontic three-dimensional dental model is translated and rotated according to the translation parameters and rotation parameters in the generated rigid transformation parameters to obtain point cloud data of the pre-orthodontic three-dimensional dental model corresponding to the post-orthodontic three-dimensional dental model, and the point cloud data of the pre-orthodontic three-dimensional dental model has the specified tooth arrangement style.

[0043] According to a second aspect of an embodiment of this specification, a device for generating a three-dimensional dental and jaw model is provided, the device comprising:

[0044] A feature extraction unit preprocesses the point cloud data of the orthodontic three-dimensional dental jaw model in the existing data set, and extracts geometric features of the orthodontic three-dimensional dental jaw model from the preprocessed point cloud data of the orthodontic three-dimensional dental jaw model;

[0045] A model training unit, which obtains a vector quantized variational autoencoder model to be trained, and trains the vector quantized variational autoencoder model in the following manner: an encoder based on the vector quantized variational autoencoder model encodes the extracted geometric features of the orthodontic three-dimensional dental jaw model into a latent space, and further reconstructs the point cloud data of the orthodontic three-dimensional dental jaw model from the encoded information in the latent space through a decoder of the vector quantized variational autoencoder model;

[0046] The data generation unit also trains a diffusion model in the latent space of the vector quantized variational autoencoder model, and the diffusion model is used to diffuse the encoded information input into the latent space to generate new encoded information, so that the trained decoder generates point cloud data of a new post-orthodontic three-dimensional dental model according to the new encoded information.

[0047] The embodiment of this specification provides a scheme for generating a three-dimensional dental model. The geometric features of the three-dimensional dental model after orthodontics are extracted through the inherent structure of the point cloud data of the three-dimensional dental model after orthodontics, and then the geometric features are encoded into the latent space by the encoder of the vector quantized variational autoencoder model, and the decoder reconstructs the encoded information in the latent space back to the point cloud data of the three-dimensional dental model after orthodontics to perform model training, so as to learn the relationship between the encoded information of the geometric features in the latent space and the point cloud data of the three-dimensional dental model after orthodontics; in this way, a diffusion model can be applied in the latent space to generate new encoding information for the existing encoding information, and then the trained decoder is used to reconstruct the point cloud data of the new three-dimensional dental model after orthodontics according to the new encoding information. The above-mentioned diffusion processing can increase the number of three-dimensional dental models after orthodontics in the data set.

[0048] On the other hand, for the point cloud data of the new post-orthodontic 3D jaw model generated by the diffusion model and the decoder, a rigid transformation is applied to each tooth in the new post-orthodontic 3D jaw model to change the tooth arrangement and obtain the point cloud data of the pre-orthodontic 3D jaw model. This ensures that the size and shape of the teeth in the pre-orthodontic 3D jaw model are consistent. In this way, the corresponding pre-orthodontic 3D jaw model can be added to the newly added post-orthodontic 3D jaw model, thereby increasing the volume of the existing data set.

[0049] On the other hand, the style features of the orthodontic pre-3D dental jaw model with a specified tooth arrangement style in the existing data set are used to control the generated orthodontic pre-3D dental jaw model to also have the specified tooth arrangement style. In this way, orthodontic pre-3D dental jaw models with a variety of different tooth arrangement styles can be generated, thereby increasing the diversity of tooth arrangement styles of the orthodontic pre-3D dental jaw models in the data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A diagram of the system architecture for generating a three-dimensional dental model according to an embodiment of this specification;

[0051] Figure 2 is a flow chart of a method for generating a three-dimensional dental model provided in an embodiment of this specification;

[0052] Figure 3 A schematic diagram of a three-dimensional jaw model and tooth sequence after orthodontic treatment provided in an embodiment of this specification;

[0053] Figure 4 A schematic diagram of a style extractor and a shape extractor provided in an embodiment of this specification;

[0054] Figure 5 A schematic diagram of a three-dimensional dental model with different tooth arrangement styles provided in an embodiment of this specification;

[0055] Figure 6 A comparison diagram of a three-dimensional dental model provided in an embodiment of this specification;

[0056] Figure 7 It is a hardware structure diagram of a device for generating a three-dimensional dental model provided in an embodiment of this specification;

[0057] Figure 8 It is a module of a device for generating a three-dimensional dental model provided in an embodiment of this specification. DETAILED DESCRIPTION

[0058] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0059] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0060] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0061] In the related technology, automatic tooth arrangement technology requires a pair of orthodontic three-dimensional jaw models (i.e., the three-dimensional jaw model before orthodontics and the three-dimensional jaw model after orthodontics) as training data. However, since the data collection and organization of the three-dimensional jaw models before and after orthodontics is a difficult and time-consuming task, the existing automatic tooth arrangement technology faces the problem of insufficient training data, especially for complex cases (such as crossbite, overjet and other tooth arrangement styles). The three-dimensional jaw model data is very scarce; as a result, the prediction results of the existing automatic tooth arrangement network are not ideal.

[0062] The idea to solve this problem is to use AI technology to automatically generate paired point cloud data for orthodontic three-dimensional dental models. At present, there are some common data generation technologies, such as PointFlow technology, which can use the normalizing flow method to hierarchically generate three-dimensional point cloud data, first generate the distribution of shapes, and then generate the distribution of sampling points after the given shape; another example is DPM (Denoising Diffusion Probabilistic Models) technology, which can apply the diffusion model to three-dimensional data and iteratively generate three-dimensional point cloud data; another example is LION technology, which can first use VAE to encode the original data into the latent space, and then apply the diffusion model in the latent space to generate new encoding based on the encoding results of the original data, so as to achieve the purpose of generating new data.

[0063] However, when these general data generation techniques are applied to generate point cloud data of paired orthodontic three-dimensional dental models, the tooth shapes in the generated three-dimensional dental models are not ideal and cannot reflect the precise shapes of the teeth due to some differences between the three-dimensional dental model generation task and the traditional three-dimensional data generation task.

[0064] Specifically, when applied to generating paired orthodontic 3D jaw model point cloud data, the task objectives can be:

[0065] 1. All individual teeth in the generated 3D dental model must be separated so that the features of each tooth can be easily extracted and transformed. Otherwise, each tooth needs to be segmented again.

[0066] 2. The number of teeth in the generated three-dimensional dental model is not fixed, and the size and shape of different teeth in the entire three-dimensional dental model must be coordinated. Therefore, it is not possible to generate each tooth separately and then splice them together.

[0067] 3. The generated three-dimensional dental models need to be in pairs, representing the three-dimensional dental models before and after orthodontics respectively, and the tooth shape and size of the same tooth before and after orthodontics need to be consistent.

[0068] The data generated by general point cloud generation methods such as PointFlow and LION technology are all unsegmented as a whole, and require additional segmentation processing when applied to downstream tasks. In addition, general point cloud generation methods do not utilize the unique prior knowledge of the jaw, resulting in the generated three-dimensional jaw model shape is not ideal and cannot reflect the precise shape of the teeth.

[0069] In view of this, the present invention proposes a simple and low-cost three-dimensional dental model generation scheme. The geometric features of the orthodontic three-dimensional dental model point cloud data are extracted through the inherent structure of the data, and then the geometric features are encoded into the latent space by the encoder of the vector quantized variational autoencoder model, and the decoder reconstructs the encoded information in the latent space back to the point cloud data of the orthodontic three-dimensional dental model to perform model training, so as to learn the relationship between the encoded information of the geometric features in the latent space and the point cloud data of the orthodontic three-dimensional dental model; in this way, the diffusion model can be applied in the latent space to generate new encoded information for the existing encoded information, and then the trained decoder is used to reconstruct new point cloud data of the orthodontic three-dimensional dental model according to the new encoded information. The above-mentioned diffusion processing can increase the number of orthodontic three-dimensional dental models in the data set.

[0070] On the other hand, for the new post-orthodontic 3D jaw model point cloud data generated by the diffusion model and the decoder, a rigid transformation is applied to each tooth in the new post-orthodontic 3D jaw model to change the tooth arrangement and obtain the new pre-orthodontic 3D jaw model point cloud data. This ensures that the size and shape of the teeth in the pre-orthodontic 3D jaw model are consistent. In this way, the corresponding pre-orthodontic 3D jaw model can be added to the newly added post-orthodontic 3D jaw model, thereby increasing the volume of the existing data set.

[0071] On the other hand, the style features of the orthodontic pre-3D dental jaw model with a specified tooth arrangement style in the existing data set are used to control the generated orthodontic pre-3D dental jaw model to also have the specified tooth arrangement style. In this way, orthodontic pre-3D dental jaw models with a variety of different tooth arrangement styles can be generated, thereby increasing the diversity of tooth arrangement styles of the orthodontic pre-3D dental jaw models in the data set.

[0072] Please refer to the following Figure 1 The system architecture diagram of the three-dimensional dental model generation system shown in the figure can include a two-stage architecture. First, relatively regularly arranged orthodontic three-dimensional dental model point cloud data is generated ( Figure 1 The first stage shown in FIG), and then generate the three-dimensional dental jaw model point cloud data before orthodontics corresponding to the three-dimensional dental jaw model point cloud data after orthodontics reconstructed in the first stage according to the specified tooth arrangement style ( Figure 1 The second stage is shown in Figure 2).

[0073] like Figure 1 As shown, in the first stage, a vector quantized variational autoencoder model (such as Figure 1 The VQ-VAE shown in FIG) generates more different orthodontic three-dimensional jaw model point cloud data based on the input orthodontic three-dimensional jaw model point cloud data. The generation process of the vector quantized variational autoencoder model will be introduced in subsequent embodiments.

[0074] In the second stage, the point cloud data of the three-dimensional jaw model after orthodontics generated in the first stage is combined with the tooth arrangement style of other specified three-dimensional jaw models before orthodontics to generate the point cloud data of the three-dimensional jaw model before orthodontics with the specified tooth arrangement style corresponding to the point cloud data of the three-dimensional jaw model after orthodontics. Among them, since what is generated in the second stage is actually the rigid body transformation parameters of each tooth, the tooth shape and size of the three-dimensional jaw model before and after orthodontics can be consistent.

[0075] Please refer to the following Figure 2 The schematic diagram of the process flow of the method for generating a three-dimensional dental jaw model is shown, and the method may include the following steps:

[0076] Step 110, preprocessing the point cloud data of the orthodontic three-dimensional jaw model in the existing data set, and extracting the geometric features of the orthodontic three-dimensional jaw model from the preprocessed point cloud data of the orthodontic three-dimensional jaw model.

[0077] Since the orthodontic three-dimensional dental models collected in the data set may come from different data sources, and the data formats or data types in these different data sources are different, in order to facilitate subsequent processing, the orthodontic three-dimensional dental models in the data set can be preprocessed and converted into a unified three-dimensional dental model.

[0078] In an exemplary embodiment, the preprocessing may include centralization and normalization processing;

[0079] Among them, centering refers to the operation of adjusting the center point of the three-dimensional dental model to the coordinate origin (0, 0, 0). The purpose of removing data offset is achieved by subtracting the vector between the existing center point and the coordinate origin from the coordinates of each vertex. Centering can eliminate data offset and make the data distribution centered on zero; it is convenient for subsequent operations such as rotation and translation of the model, avoiding additional calculations and adjustments.

[0080] Normalization refers to scaling the vertex coordinate values ​​of the three-dimensional dental model to a specific range ([-1,1]). By dividing the coordinates of each vertex by the modulus from the vertex coordinate to the origin, all three-dimensional dental models can be unified to the same order of magnitude so that the neural network can process data in different ranges more efficiently. Normalization can map data to a fixed range to avoid adverse effects on model training caused by certain eigenvalues ​​being too large or too small. In addition, it is particularly important for some algorithms that need to calculate distance or similarity (such as K-means clustering, neural networks, etc.) to ensure that each feature contributes equally to the model.

[0081] When the centering and normalization processing is used, the above step 210 may include:

[0082] Step 211, performing centralization and normalization processing on the point cloud data of the orthodontic three-dimensional dental jaw model in the existing data set to obtain a tooth sequence consisting of 32 teeth corresponding to the orthodontic three-dimensional dental jaw model; wherein the tooth sequence is a 2×2×8 spatial structure;

[0083] In this step, the tooth sequence is as follows Figure 3 As shown in (b), there may be a sequence of 32 teeth; where the missing teeth are located, 0 (or other preset characters or numbers) may be used to fill the missing teeth in the tooth sequence. The tooth sequence arranged in this way allows for good interaction of information between adjacent tooth pairs, symmetrical tooth pairs in the same jaw (upper or lower jaw), and symmetrical tooth pairs in the upper and lower jaws.

[0084] Step 213, each tooth is divided into an r×r×r voxel structure, the voxel of each tooth is extracted using the PointNet network, and the features extracted from all teeth are aggregated to obtain the geometric features of the orthodontic three-dimensional dental model with a shape of [2r, 2r, 8r]; where r is the resolution of the tooth.

[0085] It should be noted that, in this specification, whether it is a three-dimensional dental model before or after orthodontics, it can usually refer to a large number of three-dimensional spatial coordinate points in three-dimensional space, and these three-dimensional spatial coordinate points can constitute the shape of the three-dimensional dental model. Generally, the three-dimensional spatial coordinate points can be called point cloud data, that is, point cloud data of the three-dimensional dental model.

[0086] The voxel structure of teeth refers to the discretized representation of teeth in three-dimensional space. Voxel technology is used to divide the point cloud of each tooth into many small cubic units in the form of a grid to describe its details and composition. Each voxel represents a small area in space.

[0087] PointNet is a deep learning network architecture specifically designed for processing point cloud data. It can effectively process irregular and disordered point cloud data without converting point cloud data into grid or voxel representation as in traditional methods. For each tooth voxel, the PointNet network is used to extract features and aggregate them to obtain the geometric features of a three-dimensional dental model with a shape of [2r, 2r, 8r].

[0088] Step 120, obtaining a vector quantization variational autoencoder model to be trained, and training the vector quantization variational autoencoder model in the following manner:

[0089] The encoder based on the vector quantization variational autoencoder model encodes the extracted geometric features of the three-dimensional dental model after orthodontics into a latent space, and further reconstructs the point cloud data of the three-dimensional dental model after orthodontics through the decoder of the vector quantization variational autoencoder model using the encoded information in the latent space.

[0090] In this step, after the geometric features of the three-dimensional dental model with a shape of [2r, 2r, 8r] are obtained in step 210, they can be used to train a vector quantized variational autoencoder model.

[0091] The vector quantized variational autoencoder model may include an encoder and a decoder. The encoder is used to encode the geometric features of the input three-dimensional dental model into the latent space; while the decoder is used to decode the encoded information encoded by the encoder in the latent space to reconstruct the point cloud data of the corresponding three-dimensional dental model after orthodontic treatment.

[0092] In an exemplary embodiment, the vector quantized variational autoencoder model may include a VQ-VAE model using a 3D U-Net network architecture.

[0093] Among them, VQ-VAE (Vector Quantized Variational Autoencoder) is a variant of the variational autoencoder (VAE). VQ-VAE discretizes the latent space by introducing vector quantization, so that the generated three-dimensional dental model has higher quality.

[0094] In addition, since the dental data involved in this embodiment is three-dimensional data, in order to enable VQ-VAE to generate point cloud data of the three-dimensional dental model, a VQ-VAE model based on the 3D U-Net network architecture can be used.

[0095] The 3D U-Net network here is a deep learning model that is widely used in the processing of three-dimensional image data (such as CT, MRI and other medical images) and three-dimensional voxel models.

[0096] By using the 3D U-Net network as the basic network of the VQ-VAE model, the VQ-VAE model can be used to generate high-quality three-dimensional dental models.

[0097] When the VQ-VAE model of the 3D U-Net network architecture is adopted, the encoder based on the vector quantization variational autoencoder model in the above step 220 encodes the extracted geometric features of the three-dimensional dental model after orthodontics into a latent space, and further reconstructs the point cloud data of the three-dimensional dental model after orthodontics through the decoder of the vector quantization variational autoencoder model from the encoded information in the latent space, which may further include:

[0098] Step 221, using the encoder of the VQ-VAE model to perform multi-layer convolution operations on the extracted geometric features of the three-dimensional dental model after orthodontics, so as to encode the geometric features into a latent space and obtain a latent vector corresponding to the three-dimensional dental model after orthodontics.

[0099] After obtaining the geometric features of the three-dimensional dental model with a shape of [2r, 2r, 8r], the VQ-VAE encoder based on the 3D U-Net network is used to encode the geometric features of the entire three-dimensional dental model into a latent space with a smaller dimension through multi-layer convolution operations to obtain the latent vector.

[0100] Step 223, further using the VQ-VAE decoder to perform multi-layer deconvolution operations on the latent vectors in the latent space to obtain point cloud data of the orthodontic three-dimensional dental model reconstructed based on the latent vectors.

[0101] The decoder of VQ-VAE decodes the latent vector into the point cloud data of the original orthodontic 3D dental model through multi-layer deconvolution operations. In addition, a mask can be output at the same time. The point cloud data of the reconstructed 3D dental model can contain 32 teeth, and the mask can be used to indicate which of the 32 teeth should appear in the final output.

[0102] Step 225 , using the point cloud data of the reconstructed three-dimensional dental jaw model after orthodontics and the point cloud data of the three-dimensional dental jaw model in the existing data set, calculate the reconstruction loss function of the decoder.

[0103] Since the default or initial model parameters of the model are usually not optimal, it is necessary to perform several rounds of iterative training with the help of the data set to achieve model convergence and thus determine the optimal model parameters.

[0104] Taking the above-mentioned VQ-VAE model using the 3D U-Net network architecture as an example, the point cloud data of the orthodontic three-dimensional dental model in the existing data set can be regarded as the real result; and the point cloud data of the orthodontic three-dimensional dental model reconstructed after the decoder can be regarded as the predicted result; usually there is an error between the predicted result and the real result, and the reconstruction loss function is a function used to measure the error between the predicted result and the real result. The reconstruction loss function can be used to adjust the model parameters of the encoder according to the measured error.

[0105] Exemplarily, calculating the reconstruction loss function of the decoder may include:

[0106] The chamfer distance between the point cloud data of the reconstructed orthodontic three-dimensional dental model and the point cloud data of the orthodontic three-dimensional dental model in the existing data set is calculated for each tooth respectively; then, the average value of the chamfer distance of each pair of teeth is calculated to obtain the reconstruction loss function.

[0107] In this embodiment, the reconstruction loss calculation method during model training is to respectively calculate the chamfer distance (Chamfer Di stance) between the input real result and the output predicted result corresponding to a single tooth, and the calculation formula is as follows:

[0108]

[0109] Among them, S1 represents the predicted result (i.e., the point cloud data of the reconstructed three-dimensional dental model after orthodontics), S2 represents the real result (i.e., the point cloud data of the three-dimensional dental model after orthodontics in the data set), Represents the sum of the minimum distances from any point x of the prediction result S1 to S2; The sum of the minimum distances from any point y to S1 representing the true result S2.

[0110] If the value of the chamfer distance Chamfer (S1, S2) is larger, it means that the difference between the actual result and the predicted result is larger; conversely, if the value of the chamfer distance is smaller, it means that the difference between the actual result and the predicted result is smaller, that is, the reconstruction effect of the point cloud data of the three-dimensional dental model after orthodontics is better. Finally, the average value of the chamfer distance of each pair of teeth is calculated to obtain the reconstruction loss function.

[0111] By calculating the reconstruction loss function, the iterative optimization of the model parameters is completed, and the model training is performed again based on the adjusted model parameters. In this way, after multiple iterations, the model parameters can be gradually optimized to minimize the error between the predicted results and the actual results until the model converges or reaches the convergence condition.

[0112] In step 230, a diffusion model is also trained in the latent space of the vector quantized variational autoencoder model, and the diffusion model is used to diffuse the coded information input into the latent space to generate new coded information, so that the trained decoder generates point cloud data of a new post-orthodontic three-dimensional dental model according to the new coded information.

[0113] The vector quantized variational autoencoder model in this embodiment also has a diffusion model in the latent space; in order to adapt to three-dimensional data such as three-dimensional dental model point cloud data, the diffusion model here can also adopt a diffusion model based on the 3D U-Net network architecture. The mechanism of the diffusion model is to gradually add noise to the input data, and then restore the original data from the noise through the denoiser, and the denoiser is trained by repeating this process.

[0114] In an exemplary embodiment, after the vector quantized variational autoencoder model is trained, the geometric features of the orthodontic three-dimensional dental model in the existing data set or the geometric features of other real three-dimensional dental models after orthodontics can be input into the trained vector quantized variational autoencoder model, and the encoder encodes the geometric features into the latent space; at this time, the diffusion model in the latent space can perform noise processing on the latent vector encoded into the latent space, and use the denoiser of the diffusion model to perform denoising processing to obtain a new latent vector; then, the trained decoder is further used to generate new point cloud data of the orthodontic three-dimensional dental model according to the new latent vector.

[0115] The above-mentioned noise adding process may add noise to the latent vector in the latent space, and the noise may include first sampling random noise from an isotropic Gaussian distribution.

[0116] Through the above embodiment, the inherent structure of the point cloud data of the three-dimensional dental model after orthodontics is used to extract geometric features, and then the encoder of the vector quantized variational autoencoder model encodes the geometric features of the three-dimensional dental model into the latent space, and the decoder reconstructs the encoded information in the latent space back to the point cloud data of the three-dimensional dental model after orthodontics to perform model training, so as to learn the relationship between the encoded information of the geometric features of the three-dimensional dental model in the latent space and the point cloud data of the three-dimensional dental model after orthodontics; in this way, the diffusion model can be applied in the latent space to generate new encoding information for the existing encoding information, and then the trained decoder is used to reconstruct the new point cloud data of the three-dimensional dental model after orthodontics according to the new encoding information. The above diffusion processing can increase the number of three-dimensional dental models after orthodontics in the data set.

[0117] In practical applications, as mentioned above, automatic tooth arrangement technology requires paired orthodontic three-dimensional jaw model point cloud data (i.e., three-dimensional jaw model point cloud data before orthodontics and three-dimensional jaw model point cloud data after orthodontics) as training data. Although more three-dimensional jaw model point cloud data after orthodontics can be generated through the above embodiment, paired three-dimensional jaw model point cloud data before orthodontics is also required to be generated for it; and in order to improve the diversity of the data set, these three-dimensional jaw model point cloud data before orthodontics are also required to have a variety of tooth arrangement styles (such as tooth arrangement styles of various complex cases).

[0118] In view of this, this manual is Figure 2 Based on the method embodiment shown in the figure, the following steps may also be included (corresponding to Figure 1 The second stage in

[0119] Step 240, obtaining a pre-orthodontic three-dimensional dental model of a specified tooth arrangement style in an existing data set;

[0120] Step 250 , extracting the style features of the tooth arrangement in the three-dimensional jaw model before orthodontics with the specified tooth arrangement style, and extracting the shape features of the point cloud data of the new three-dimensional jaw model after orthodontics generated by the trained decoder.

[0121] For example, the style feature and shape feature can be respectively Figure 4 The style feature extractor and shape feature extractor shown in are used for extraction.

[0122] In an exemplary embodiment, the step of extracting the style features of the tooth arrangement in the three-dimensional pre-orthodontic dental jaw model of the specified tooth arrangement style includes:

[0123] Centralizing and normalizing the point cloud data of the pre-orthodontic three-dimensional dental model of the specified tooth arrangement style to obtain a tooth sequence composed of 32 teeth; wherein the tooth sequence is a 2×2×8 spatial structure;

[0124] Each tooth is divided into an r×r×r voxel structure, and the PointNet network is used to extract features from the voxels of each tooth. The features extracted from all teeth are aggregated to obtain geometric features with a shape of [2r, 2r, 8r]; where r is the resolution of the tooth;

[0125] Mean pooling and variance pooling are used inside the voxel, and after linear transformation, the style feature with a shape of [2r, 2r, 8r] is obtained.

[0126] In this embodiment, the centering and normalization processing, voxel structure, PointNet network feature extraction, etc. are the same as the aforementioned step 210 and will not be repeated here.

[0127] The difference from the above step 210 is that after extracting the jaw features, mean pooling and variance pooling are used inside the voxel (such as Figure 4 MP and SP shown in the style feature extractor of , after linear transformation, the style feature with the same shape of [2r,2r,8r] is obtained. This process can be referred to the following formula:

[0128]

[0129] in, represents the style feature extracted from the i-th voxel.

[0130] In an exemplary embodiment, extracting shape features of point cloud data of a new orthodontic three-dimensional dental jaw model generated by the trained decoder includes:

[0131] The point cloud data of the new orthodontic three-dimensional dental model generated by the trained decoder is centralized and normalized to obtain a tooth sequence consisting of 32 teeth; wherein the tooth sequence is a 2×2×8 spatial structure;

[0132] Each tooth is divided into an r×r×r voxel structure, and the PointNet network is used to extract features from the voxels of each tooth. The features extracted from all teeth are aggregated to obtain geometric features with a shape of [2r, 2r, 8r]; where r is the resolution of the tooth;

[0133] Max pooling is used inside each tooth to aggregate the features of all voxels and obtain shape features.

[0134] In this embodiment, the centering and normalization processing, voxel structure, and extraction of dental features by the PointNet network are the same as the aforementioned step 210 and will not be described in detail here.

[0135] The difference from the above step 210 is that after extracting the geometric features, the maximum pooling (such as Figure 4 MaxP) shown in the shape feature extractor is used to aggregate the features of all voxels to obtain shape features.

[0136] Step 160, taking the style feature and the shape feature as dual conditions of the diffusion model, and guiding the diffusion model to generate a rigid transformation parameter of each tooth in the three-dimensional dental model after orthodontics that meets the requirements.

[0137] As mentioned above, the diffusion model may include a diffusion model based on a 3D U-Net network architecture for processing three-dimensional spatial data such as a three-dimensional dental model.

[0138] In order to conditionally input the style and shape features into the denoiser of the diffusion model, a multilayer perceptron (MLP) can be used to fuse the two features and pass them as input to the 3D U-Net network.

[0139] The output of each layer of the 3D U-Net network carries the progressively extracted spatial information. The conditional vector generated by the MLP is combined with the 3D U-Net features of each layer so that the generation process can be adjusted in each layer by combining style and shape features. To ensure that the features of each layer are effectively influenced by style and shape features, the output of the MLP (as a conditional vector) can be weighted merged or concatenated with the output of the layer. This approach ensures that each layer can perform a denoising process guided by the features and generate more accurate rigid body transformation parameters.

[0140] The goal of the diffusion model can be set as the rigid body transformation parameters of the teeth. The diffusion model gradually recovers the rigid body transformation parameters of each tooth from the random noise through a multi-step denoising process. Therefore, each round of denoising in the diffusion model needs to be affected by the output of the previous round, and the generation of rigid body transformation is optimized through conditional features.

[0141] Since the points in the paired three-dimensional dental models in the data set are one-to-one corresponding, that is, in the paired three-dimensional dental models before orthodontics and the three-dimensional dental models after orthodontics, the points that constitute the same tooth are one-to-one corresponding, but the three-dimensional coordinate positions of the points in the three-dimensional dental models before orthodontics and the three-dimensional dental models after orthodontics are different; therefore, the rigid body transformation parameters corresponding to the transformation of each tooth from after orthodontics to before orthodontics can be calculated based on the correspondence between the points in the paired three-dimensional dental models.

[0142] The rigid body transformation parameters may be used to describe how the tooth is transformed from the position before orthodontic treatment to the position after orthodontic treatment through transformations such as translation and rotation.

[0143] Step 270, applying the generated rigid transformation parameters to each tooth in the three-dimensional dental model after orthodontics to obtain point cloud data of the three-dimensional dental model before orthodontics corresponding to the three-dimensional dental model after orthodontics, and the point cloud data of the three-dimensional dental model before orthodontics has the specified tooth arrangement style.

[0144] In an exemplary embodiment, the rigid body transformation parameters include 9-dimensional rigid body transformation parameters of each of the 32 teeth, including 3-dimensional translation parameters in three-dimensional space (i.e., translation parameters of the X-axis, Y-axis and Z-axis), and 6-dimensional rotation parameters in three-dimensional space (such as rotation parameters along the X-axis, Y-axis, Z-axis, X and Y-axis rotation parameters, X and Z-axis rotation parameters, and Y and Z-axis rotation parameters).

[0145] For any tooth i, x i =(m i ,r i )∈R 9 It can represent the 9-dimensional rigid body transformation parameters of the i-th tooth, where i is an integer in [1,32], mi ∈R 3 is the 3D translation parameter of the i-th tooth, and r i ∈R 6 is the 6-dimensional rotation parameter of the i-th tooth.

[0146] When the 6D rotation parameter r is used i To represent the rotation in three-dimensional space, the 6D rotation parameter r can be i ∈R 6 Transformed into a 3×3 rotation matrix R i ; Then, translate the i-th tooth to the origin of the coordinate system and apply the rotation matrix R i Apply a rotation transformation; and translate the i-th tooth to its original position, and then according to the translation parameter m i to pan.

[0147] By using rigid body transformation parameters to translate and rotate each tooth of the three-dimensional dental model after orthodontics in the above-mentioned manner, a three-dimensional dental model before orthodontics corresponding to the three-dimensional dental model after orthodontics can be generated, and the three-dimensional dental model before orthodontics has a specified tooth arrangement style (i.e., the tooth arrangement style of the three-dimensional dental model before orthodontics specified in step 250).

[0148] Through the above embodiments, on the one hand, a new post-orthodontic three-dimensional dental model is generated by using the diffusion model and the decoder, and then a rigid transformation is applied to each tooth in the new post-orthodontic three-dimensional dental model to change the tooth arrangement, thereby obtaining a pre-orthodontic three-dimensional dental model. This ensures that the size and shape of the teeth in the three-dimensional dental model before and after orthodontics are consistent. In this way, the corresponding pre-orthodontic three-dimensional dental model can be added to the post-orthodontic three-dimensional dental model added by the diffusion process, thereby increasing the volume of the existing data set.

[0149] On the other hand, the tooth arrangement style of the orthodontic pre-3D jaw model in the existing data set is used to control the tooth arrangement style type of the generated orthodontic pre-3D jaw model. In this way, a variety of orthodontic pre-3D jaw models with different tooth arrangement styles can be generated, thereby increasing the diversity of tooth arrangement styles of the orthodontic pre-3D jaw model in the data set. Figure 5 Schematic diagram of three-dimensional dental models before orthodontics showing various tooth arrangement styles.

[0150] In practical applications, the data quality of the generated orthodontic three-dimensional dental model can also be evaluated. A schematic diagram of comparing the three-dimensional dental model generated by the embodiment provided in this specification with the existing general data generation technology for the method of generating the three-dimensional dental model provided in this specification;

[0151] The existing common data generation technologies include PointFlow technology, DPM technology, LION technology, PVD technology and DiT-3D technology. Following the idea of ​​LION technology, the CD and CMD indicators of 1-NNA are used to evaluate the quality of the generated results. At the same time, in order to evaluate the uniqueness of the generated data, it is possible to define As an indicator, S is defined as:

[0152] S={fi:Chamfer(fi,fj)>r,j <i,i,j∈{1…N}

[0153] Among them, f i represents the i-th sample in the dataset, and r is a predefined distance threshold (set to 1 cm in subsequent experiments). In this way, |S| can represent the number of unique samples in the dataset, and U CD Can represent the proportion of unique samples.

[0154] The final comparison experimental results are shown in the following table (bold represents good data effect):

[0155] CD(%,↓) EMD(%,↓) <![CDATA[U CD (%,↑)]]> PointFlow 97.62 83.88 62.22 DPM 89.25 74.50 75.69 PVD 84.87 78.12 49.58 LION 90.41 77.93 52.78 DiT-3D 95.75 82.01 34.03 Methods of this manual 69.50 71.88 96.25

[0156] It can be seen that the data quality of the three-dimensional dental model generated by the embodiment provided in this specification is significantly higher than that of the existing general data generation technology. The exemplary generation results can be seen in Figure 6 shown.

[0157] like Figure 1 As shown, since the three-dimensional dental model generation system provided in this specification can generate a pair of orthodontic three-dimensional dental models (i.e., a pair of pre-orthodontic three-dimensional dental models and post-orthodontic three-dimensional dental models) through two stages, it is impossible to compare the pre-orthodontic three-dimensional dental models with the general data generation technology alone, and it is also impossible to evaluate the generation effect of the pre-orthodontic three-dimensional dental models.

[0158] In order to further evaluate the quality of the generated paired orthodontic 3D dental models, the existing point cloud-based automatic tooth arrangement tools TANet and PSTN are trained on the real dataset and the generated dataset respectively (the two training sets have the same size and both contain 720 pairs of data). They are also tested on the real dataset. The experimental results are shown in the following table:

[0159]

[0160] It can be seen from the table that after training on the data set generated by the three-dimensional dental model generation method provided in this specification, the performance of the automatic tooth arrangement tools TANet and PSTN is only slightly inferior to the results of training on the real data set. It can be seen that the data set generated by the embodiment of this specification is indeed of high quality.

[0161] Corresponding to the aforementioned embodiment of the method for generating a three-dimensional dental jaw model, this specification also provides an embodiment of a device for generating a three-dimensional dental jaw model. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of the device in which it is located reading the corresponding computer program in the non-volatile memory into the memory and running it. From the hardware level, if Figure 7 The figure is a hardware structure diagram of the device where the three-dimensional dental model generation device in this specification is located. Figure 7 In addition to the processor, network interface, memory and non-volatile memory shown, the device in which the apparatus in the embodiment is located may also include other hardware according to the actual communication function, which will not be described in detail.

[0162] See also Figure 8 , is a block diagram of a device for generating a three-dimensional dental model according to an embodiment of the present specification, and the device corresponds to Figure 1 In the embodiment shown, the device comprises:

[0163] The feature extraction unit 810 pre-processes the point cloud data of the orthodontic three-dimensional dental jaw model in the existing data set, and extracts the geometric features of the orthodontic three-dimensional dental jaw model from the pre-processed point cloud data of the orthodontic three-dimensional dental jaw model;

[0164] The model training unit 820 obtains a vector quantized variational autoencoder model to be trained, and trains the vector quantized variational autoencoder model in the following manner: an encoder based on the vector quantized variational autoencoder model encodes the extracted geometric features of the orthodontic three-dimensional dental jaw model into a latent space, and further reconstructs the point cloud data of the orthodontic three-dimensional dental jaw model from the encoded information in the latent space through a decoder of the vector quantized variational autoencoder model;

[0165] The data generation unit 830 also trains a diffusion model in the latent space of the vector quantized variational autoencoder model, and the diffusion model is used to diffuse the encoded information input into the latent space to generate new encoded information, so that the trained decoder generates point cloud data of a new post-orthodontic three-dimensional dental model according to the new encoded information.

[0166] Optionally, the feature extraction unit 810 includes:

[0167] The preprocessing subunit performs centralization and normalization processing on the point cloud data of the orthodontic three-dimensional dental jaw model in the existing data set to obtain a tooth sequence consisting of 32 teeth corresponding to the orthodontic three-dimensional dental jaw model; wherein the tooth sequence is a 2×2×8 spatial structure;

[0168] The feature extraction subunit divides each tooth into an r×r×r voxel structure, uses the PointNet network to extract features from the voxels of each tooth, and aggregates the features extracted from all teeth to obtain the geometric features of the post-orthodontic 3D dental model with a shape of [2r, 2r, 8r]; where r is the resolution of the tooth.

[0169] Optionally, the vector quantization variational autoencoder model includes a VQ-VAE model using a 3D U-Net network architecture;

[0170] The model training unit 820 includes:

[0171] An encoding subunit, using the encoder of the VQ-VAE model to perform a multi-layer convolution operation on the extracted geometric features of the three-dimensional dental jaw model after orthodontics, so as to encode the geometric features into a latent space, and obtain a latent vector corresponding to the three-dimensional dental jaw model after orthodontics;

[0172] A decoding subunit, using the VQ-VAE decoder to perform a multi-layer deconvolution operation on the latent vector in the latent space, to obtain point cloud data of the orthodontic three-dimensional dental model reconstructed based on the latent vector;

[0173] The calculation subunit calculates the reconstruction loss function of the decoder by using the point cloud data of the reconstructed three-dimensional dental jaw model after orthodontic treatment and the point cloud data of the three-dimensional dental jaw model in the existing data set.

[0174] Optionally, the computing subunit is further used to calculate the chamfer distance between the point cloud data of the reconstructed orthodontic three-dimensional dental model and the point cloud data of the orthodontic three-dimensional dental model in the existing data set for each tooth; calculate the average value of the chamfer distance of each pair of teeth to obtain the reconstruction loss function.

[0175] Optionally, the data generating unit 830 is further used to perform noise processing on the latent vector of the latent space in which the encoder encodes the features of the real three-dimensional dental model after orthodontics using a diffusion model, and perform denoising processing using a denoiser of the diffusion model to obtain a new latent vector; and further use the trained decoder to generate new point cloud data of the three-dimensional dental model after orthodontics according to the new latent vector.

[0176] Optionally, the device further comprises:

[0177] An acquisition unit, which acquires a pre-orthodontic three-dimensional dental jaw model of a specified tooth arrangement style from an existing data set;

[0178] A style and shape extraction unit, which extracts the style features of the tooth arrangement in the three-dimensional jaw model before orthodontics with the specified tooth arrangement style, and extracts the shape features of the point cloud data of the new three-dimensional jaw model after orthodontics generated by the trained decoder;

[0179] A parameter generation unit, which uses the style feature and the shape feature as dual conditions of the diffusion model, and guides the diffusion model to generate a rigid transformation parameter of each tooth in the three-dimensional dental jaw model after orthodontics that meets the requirements;

[0180] The second data generating unit applies the generated rigid transformation parameters to each tooth in the three-dimensional dental model after orthodontics to obtain point cloud data of the three-dimensional dental model before orthodontics corresponding to the three-dimensional dental model after orthodontics, and the point cloud data of the three-dimensional dental model before orthodontics has the specified tooth arrangement style.

[0181] Optionally, when extracting the style features of the tooth arrangement in the pre-orthodontic three-dimensional dental model of the specified tooth arrangement style, the style and shape extraction unit is further used to:

[0182] Centralizing and normalizing the point cloud data of the pre-orthodontic three-dimensional dental model of the specified tooth arrangement style to obtain a tooth sequence composed of 32 teeth; wherein the tooth sequence is a 2×2×8 spatial structure;

[0183] Each tooth is divided into an r×r×r voxel structure, and the PointNet network is used to extract features from the voxels of each tooth. The features extracted from all teeth are aggregated to obtain geometric features with a shape of [2r, 2r, 8r]; where r is the resolution of the tooth;

[0184] Mean pooling and variance pooling are used inside the voxel, and after linear transformation, the style feature with a shape of [2r, 2r, 8r] is obtained.

[0185] Optionally, when extracting the shape features of the point cloud data of the new orthodontic three-dimensional dental jaw model generated by the trained decoder, the style and shape extraction unit is further used to:

[0186] The point cloud data of the new orthodontic three-dimensional dental model generated by the trained decoder is centralized and normalized to obtain a tooth sequence consisting of 32 teeth; wherein the tooth sequence is a 2×2×8 spatial structure;

[0187] Each tooth is divided into an r×r×r voxel structure, and the PointNet network is used to extract features from the voxels of each tooth. The features extracted from all teeth are aggregated to obtain geometric features with a shape of [2r, 2r, 8r]; where r is the resolution of the tooth;

[0188] Max pooling is used inside each tooth to aggregate the features of all voxels and obtain shape features.

[0189] Optionally, the diffusion model includes a diffusion model based on a 3D U-Net network architecture;

[0190] The parameter generation unit is further used to input the style features and shape features into each layer of the 3D U-Net network through a multi-layer perceptron, guide the scattered model to generate rigid transformation parameters of the teeth, and output the rigid transformation parameters for each tooth in the three-dimensional dental model after orthodontic treatment.

[0191] Optionally, the second data generating unit is further used to translate and rotate each tooth in the three-dimensional dental model after orthodontics according to the translation parameters and rotation parameters in the generated rigid transformation parameters to obtain point cloud data of the three-dimensional dental model before orthodontics corresponding to the three-dimensional dental model after orthodontics, and the point cloud data of the three-dimensional dental model before orthodontics has the specified tooth arrangement style.

[0192] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0193] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0194] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. Ordinary technicians in this field can understand and implement it without paying creative work.

[0195] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0196] Those skilled in the art will readily appreciate other embodiments of the specification after considering the specification and practicing the invention disclosed herein. The specification is intended to cover any variations, uses, or adaptations of the specification that follow the general principles of the specification and include common knowledge or customary techniques in the art that are not disclosed in the specification. The specification and examples are to be considered exemplary only, and the true scope and spirit of the specification are indicated by the following claims.

[0197] It should be understood that the present description is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

Claims

1. A method for generating a three-dimensional dental model, characterized in that: The method comprises: Preprocessing the point cloud data of the orthodontic three-dimensional dental jaw model in the existing data set, and extracting the geometric features of the orthodontic three-dimensional dental jaw model from the preprocessed point cloud data of the orthodontic three-dimensional dental jaw model; Obtain a vector quantized variational autoencoder model to be trained, and train the vector quantized variational autoencoder model in the following manner: The encoder based on the vector quantized variational autoencoder model encodes the extracted geometric features of the orthodontic three-dimensional dental model into a latent space, and further reconstructs the point cloud data of the orthodontic three-dimensional dental model through the decoder of the vector quantized variational autoencoder model from the encoded information in the latent space; In which, a diffusion model is also trained in the latent space of the vector quantized variational autoencoder model, and the diffusion model is used to diffuse the encoded information input into the latent space to generate new encoded information, so that the trained decoder generates point cloud data of a new post-orthodontic three-dimensional dental model according to the new encoded information.

2. The method according to claim 1, characterized in that The method of preprocessing the point cloud data of the orthodontic three-dimensional dental jaw model in the existing data set and extracting the geometric features of the orthodontic three-dimensional dental jaw model from the preprocessed point cloud data of the orthodontic three-dimensional dental jaw model comprises: The point cloud data of the orthodontic three-dimensional dental model in the existing data set is centralized and normalized to obtain a tooth sequence consisting of 32 teeth corresponding to the orthodontic three-dimensional dental model; wherein the tooth sequence is a 2×2×8 spatial structure; Each tooth is divided into an r×r×r voxel structure, and the PointNet network is used to extract features from the voxels of each tooth. The features extracted from all teeth are aggregated to obtain the geometric features of the orthodontic 3D dental model with a shape of [2r, 2r, 8r], where r is the resolution of the tooth.

3. The method according to claim 2, characterized in that The vector quantization variational autoencoder model includes a VQ-VAE model using a 3D U-Net network architecture; The encoder based on the vector quantized variational autoencoder model encodes the extracted geometric features of the orthodontic three-dimensional dental model into a latent space, and further reconstructs the point cloud data of the orthodontic three-dimensional dental model through the decoder of the vector quantized variational autoencoder model from the encoded information in the latent space, including: Using the encoder of the VQ-VAE model, a multi-layer convolution operation is performed on the extracted geometric features of the three-dimensional dental model after orthodontics, so as to encode the geometric features into a latent space, and obtain a latent vector corresponding to the three-dimensional dental model after orthodontics; Further using the VQ-VAE decoder to perform a multi-layer deconvolution operation on the latent vectors in the latent space to obtain point cloud data of the orthodontic three-dimensional dental model reconstructed based on the latent vectors; The reconstruction loss function of the decoder is calculated using the point cloud data of the reconstructed three-dimensional dental jaw model after orthodontic treatment and the point cloud data of the three-dimensional dental jaw model in the existing data set.

4. The method according to claim 3, characterized in that The step of calculating the reconstruction loss function of the decoder by using the point cloud data of the reconstructed three-dimensional dental jaw model after orthodontic treatment and the point cloud data of the three-dimensional dental jaw model in the existing data set comprises: respectively calculating the chamfer distance of each tooth between the point cloud data of the reconstructed orthodontic three-dimensional dental jaw model and the point cloud data of the orthodontic three-dimensional dental jaw model in the existing data set; The average value of the chamfer distance of each pair of teeth is calculated to obtain the reconstruction loss function.

5. The method according to claim 1, characterized in that: The diffusion model is used to perform diffusion processing on the coded information input into the latent space to generate new coded information, so that the trained decoder generates point cloud data of a new orthodontic three-dimensional dental model according to the new coded information, including: The diffusion model performs noise processing on the latent vector of the latent space in which the encoder encodes the features of the real three-dimensional dental model after orthodontics, and uses the denoiser of the diffusion model to perform denoising processing to obtain a new latent vector; further, the trained decoder is used to generate point cloud data of a new three-dimensional dental model after orthodontics according to the new latent vector.

6. The method according to claim 1, characterized in that Also includes: Obtain a pre-orthodontic three-dimensional dental model of a specified tooth arrangement style from an existing data set; Extracting the style features of the tooth arrangement in the three-dimensional dental model before orthodontics with the specified tooth arrangement style, and extracting the shape features of the point cloud data of the new three-dimensional dental model after orthodontics generated by the trained decoder; The style feature and the shape feature are used as dual conditions of the diffusion model to guide the diffusion model to generate a rigid transformation parameter of each tooth in the three-dimensional dental jaw model after orthodontics that meets the requirements; The generated rigid transformation parameters are applied to each tooth in the three-dimensional jaw model after orthodontics to obtain point cloud data of the three-dimensional jaw model before orthodontics corresponding to the three-dimensional jaw model after orthodontics, and the point cloud data of the three-dimensional jaw model before orthodontics has the specified tooth arrangement style.

7. The method according to claim 6, characterized in that The step of extracting the style features of the tooth arrangement in the three-dimensional pre-orthodontic dental jaw model of the specified tooth arrangement style comprises: Centralizing and normalizing the point cloud data of the pre-orthodontic three-dimensional dental model of the specified tooth arrangement style to obtain a tooth sequence composed of 32 teeth; wherein the tooth sequence is a 2×2×8 spatial structure; Each tooth is divided into an r×r×r voxel structure, and the PointNet network is used to extract features from the voxels of each tooth. The features extracted from all teeth are aggregated to obtain geometric features with a shape of [2r, 2r, 8r]; where r is the resolution of the tooth; Mean pooling and variance pooling are used inside the voxel, and after linear transformation, the style feature with a shape of [2r, 2r, 8r] is obtained.

8. The method according to claim 6, characterized in that The shape features of the point cloud data of the new orthodontic three-dimensional dental jaw model generated by the trained decoder are extracted, including: The point cloud data of the new orthodontic three-dimensional dental model generated by the trained decoder is centralized and normalized to obtain a tooth sequence consisting of 32 teeth; wherein the tooth sequence is a 2×2×8 spatial structure; Each tooth is divided into an r×r×r voxel structure, and the PointNet network is used to extract features from the voxels of each tooth. The features extracted from all teeth are aggregated to obtain geometric features with a shape of [2r, 2r, 8r]; where r is the resolution of the tooth; Max pooling is used inside each tooth to aggregate the features of all voxels and obtain shape features.

9. The method according to claim 6, characterized in that The diffusion model includes a diffusion model based on a 3D U-Net network architecture; The method of using the style feature and the shape feature as dual conditions of the diffusion model to guide the diffusion model to generate a rigid transformation parameter of each tooth in the three-dimensional jaw model after orthodontics that meets the requirements includes: The style features and shape features are input into each layer of the 3D U-Net network through a multi-layer perceptron, guiding the scattered model to generate rigid body transformation parameters of the teeth, and outputting the rigid transformation parameters for each tooth in the three-dimensional dental model after orthodontics.

10. The method according to claim 9, characterized in that The method of applying the generated rigid transformation parameters to each tooth in the three-dimensional jaw model after orthodontics to obtain a three-dimensional jaw model before orthodontics corresponding to the three-dimensional jaw model after orthodontics, and the three-dimensional jaw model before orthodontics has the specified tooth arrangement style, comprises: Each tooth in the post-orthodontic three-dimensional dental model is translated and rotated according to the translation parameters and rotation parameters in the generated rigid transformation parameters to obtain point cloud data of the pre-orthodontic three-dimensional dental model corresponding to the post-orthodontic three-dimensional dental model, and the point cloud data of the pre-orthodontic three-dimensional dental model has the specified tooth arrangement style.

11. A device for generating a three-dimensional dental model, characterized in that: The device comprises: A feature extraction unit preprocesses the point cloud data of the orthodontic three-dimensional dental jaw model in the existing data set, and extracts geometric features of the orthodontic three-dimensional dental jaw model from the preprocessed point cloud data of the orthodontic three-dimensional dental jaw model; A model training unit, which obtains a vector quantized variational autoencoder model to be trained, and trains the vector quantized variational autoencoder model in the following manner: an encoder based on the vector quantized variational autoencoder model encodes the extracted geometric features of the orthodontic three-dimensional dental jaw model into a latent space, and further reconstructs the point cloud data of the orthodontic three-dimensional dental jaw model from the encoded information in the latent space through a decoder of the vector quantized variational autoencoder model; The data generation unit also trains a diffusion model in the latent space of the vector quantized variational autoencoder model, and the diffusion model is used to diffuse the encoded information input into the latent space to generate new encoded information, so that the trained decoder generates point cloud data of a new post-orthodontic three-dimensional dental model according to the new encoded information.

12. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method according to any one of claims 1 to 10.

Images