Method and apparatus for obtaining tooth model, storage medium, and electronic device

Abstract

Disclosed in the present application are a method and apparatus for obtaining a tooth model, a storage medium, and an electronic device. The method comprises: acquiring first tooth model data of a first tooth of a patient, wherein the first tooth comprises at least one accessory; predicting, on the basis of a neural network model, the first tooth model data to obtain predicted point cloud data, wherein the predicted point cloud data is tooth model data which is predicted on the basis of the neural network model and does not comprise data of a first accessory, and the first accessory is an accessory on the first tooth; determining, according to the predicted point cloud data, a first area of the first tooth model data, wherein the first area comprises an area corresponding to the first accessory; and generating, according to the first area, the first tooth model data, and the predicted point cloud data, second tooth model data, wherein the second tooth model data is configured for indicating the first tooth with the first accessory removed.

Description

Method, device, storage medium and electronic device for obtaining tooth model

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on December 29, 2023, with application number 202311869829.0 and application name "A method, device, storage medium and electronic device for obtaining a tooth model", the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of medical technology, and in particular to a method, device, storage medium and electronic device for obtaining a tooth model. Background Art

[0004] Nowadays, dental attachments are often used to assist with orthodontic treatment. Dental attachments are physical structures bonded to the teeth, typically enhancing appliance retention and assisting tooth movement. Upon completion of the orthodontic treatment, the attachments must be removed.

[0005] Currently, most cases involve the removal of tooth attachments by tooth grinding. Typically, before performing tooth grinding, a tooth model is created to simulate the state of the tooth before and after the surgery, thereby predicting the postoperative condition of the tooth.

[0006] Existing methods typically obtain a tooth model with attachments, manually remove the attachments, and then use a pre-trained model to predict the post-operative state of the tooth. However, the resulting tooth model often differs significantly from the actual tooth model, resulting in inaccurate results.

[0007] Therefore, how to obtain a true and reliable tooth model before grinding away the tooth attachments is an urgent problem to be solved. Summary of the Invention

[0008] The present application provides a method, device, storage medium and electronic device for obtaining a tooth model, so as to at least partially solve the above-mentioned problems existing in the prior art.

[0009] This application adopts the following technical solutions:

[0010] The present application provides a method for obtaining a tooth model, comprising:

[0011] Acquire first tooth model data of a first tooth of a patient; the first tooth includes at least one attachment;

[0012] Based on the neural network model, the first tooth model data is predicted to obtain predicted point cloud data; the predicted point cloud data is the tooth model data predicted by the neural network model excluding the first attachment, the first attachment being an attachment on the first tooth;

[0013] Determining a first region of the first tooth model data based on the predicted point cloud data; the first region includes a region corresponding to the first attachment;

[0014] Second tooth model data is generated based on the first area, the first tooth model data and the predicted point cloud data; the second tooth model data is used to indicate the first tooth without the first attachment.

[0015] Optionally, determining the first region of the first tooth model data according to the predicted point cloud data specifically includes:

[0016] Obtain a first surface patch of the first tooth model data; the first surface patch is at least one surface patch in the first tooth model data;

[0017] Determine whether the first patch belongs to the first area based on distance information between the first patch and each patch corresponding to the predicted point cloud data, wherein each patch is composed of three adjacent data points in the point cloud data.

[0018] Optionally, determining whether the first surface patch belongs to the first area according to distance information between the first surface patch and each surface patch of the predicted point cloud data specifically includes:

[0019] Determining position information of the first surface patch;

[0020] Determining the shortest distance between the first surface patch and the predicted point cloud data based on the position information of the first surface patch and the position information of each surface patch corresponding to the predicted point cloud data;

[0021] When the shortest distance information meets a first threshold condition, it is determined that the first surface patch does not belong to the first area.

[0022] Optionally, when the shortest distance information satisfies a first threshold condition, determining that the first facet does not belong to the first area specifically includes:

[0023] Determining a second surface in the predicted point cloud data based on the position information of the first surface and the position information of each surface corresponding to the predicted point cloud data, wherein a distance between the second surface and the first surface satisfies a first condition;

[0024] When the distance between the first surface patch and the second surface patch meets a second threshold condition, it is determined that the first surface patch does not belong to the first area.

[0025] Optionally, the position information includes coordinate information of the center point of the patch.

[0026] Optionally, generating second tooth model data according to the first region, the first tooth model data, and the predicted point cloud data specifically includes:

[0027] Determining first point cloud data according to the first area, where the first point cloud data is other point cloud data in the first tooth model data except for the first area;

[0028] The first point cloud data and the predicted point cloud data are fused to generate second tooth model data.

[0029] Optionally, predicting the first tooth model data based on a neural network model to obtain predicted point cloud data includes:

[0030] obtaining second point cloud data and third point cloud data based on the first tooth model data; wherein the second point cloud data and the third point cloud data have different data volumes;

[0031] performing feature extraction on the second point cloud data to obtain second point cloud feature data;

[0032] performing feature extraction on the third point cloud data to obtain third point cloud feature data;

[0033] The second point cloud feature data and the third point cloud feature data are input into the prediction subnet of the neural network model to obtain predicted point cloud data.

[0034] Optionally, obtaining second point cloud data according to the first tooth model data includes:

[0035] Determine first candidate point cloud data corresponding to candidate vertices of the first tooth model data;

[0036] determining second candidate point cloud data in the first tooth model data according to distance information between the first candidate point cloud data and the first tooth model data;

[0037] Second point cloud data is determined based on the first candidate point cloud data and the second candidate point cloud data.

[0038] Optionally, the method further includes:

[0039] In response to receiving the display instruction, the first tooth model data and / or the second tooth model data are displayed according to the content of the display instruction.

[0040] Optionally, before displaying the second tooth model data, the method further includes:

[0041] In response to receiving the removal instruction, the first attachment in the first tooth model data is removed.

[0042] Optionally, the method further includes:

[0043] In response to receiving an update instruction, updating a first region of the first tooth model data;

[0044] The second tooth model data is updated according to the updated first area, the first tooth model data and the predicted point cloud data; the second tooth model data is used to indicate the first tooth with the first attachment removed.

[0045] The present invention provides a method for displaying a tooth model, comprising:

[0046] The second tooth model data is displayed, where the second tooth model data is obtained by the method for obtaining a tooth model according to any one of the above embodiments.

[0047] Optionally, the method further includes:

[0048] The first tooth model data is displayed.

[0049] Optionally, the method further includes:

[0050] In response to a first display instruction input by a user, displaying first tooth model data;

[0051] Alternatively, in response to a second display instruction input by the user, the second tooth model data is displayed.

[0052] The present application provides a device for obtaining a tooth model, the device comprising:

[0053] An acquisition module, configured to acquire first tooth model data of a first tooth of a patient; the first tooth includes at least one attachment;

[0054] a prediction module, configured to predict the first tooth model data based on a neural network model to obtain predicted point cloud data; the predicted point cloud data is the tooth model data predicted by the neural network model excluding a first attachment, where the first attachment is an attachment on the first tooth;

[0055] a determination module, configured to determine a first region of the first tooth model data based on the predicted point cloud data; the first region includes a region corresponding to the first attachment;

[0056] A generating module is used to generate second tooth model data based on the first area, the first tooth model data and the predicted point cloud data; the second tooth model data is used to indicate the first tooth with the first attachment removed.

[0057] The present application provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method for obtaining a tooth model is implemented.

[0058] The present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for obtaining a tooth model when executing the program.

[0059] At least one of the above technical solutions adopted in this application can achieve the following beneficial effects:

[0060] In the method for obtaining a tooth model provided in the present application, first tooth model data of a patient's first tooth is obtained; the first tooth includes at least one accessory; based on a neural network model, the first tooth model data is predicted to obtain predicted point cloud data; the predicted point cloud data is the tooth model data predicted by the neural network model excluding the first accessory, and the first accessory is the accessory on the first tooth; based on the predicted point cloud data, a first area of ​​the first tooth model data is determined; the first area includes the area corresponding to the first accessory; based on the first area, the first tooth model data and the predicted point cloud data, second tooth model data is generated; the second tooth model data is used to indicate the first tooth without the first accessory.

[0061] When using the method for obtaining a tooth model provided in the present application to predict the second tooth model data after the attachments are ground off the first tooth model data, the predicted point cloud data of the attachment area after the attachments are ground off can be obtained based on the first tooth model data through a neural network model, and the attachment part in the first tooth model data can be removed by comparing it with the predicted point cloud data. Finally, the first tooth model data after the attachments are removed and the predicted point cloud data are fused to obtain the second tooth model data. Compared with traditional methods, this method does not require any manual operation during implementation, reducing the inaccuracy caused by errors in manual operation. At the same time, the method of obtaining the second tooth model data by fusing the predicted point cloud data with the first tooth model data can not only more accurately restore the shape of the tooth model after the attachments are removed, but also retain the original information of the tooth model to the greatest extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0063] FIG1 is a schematic diagram of a process for obtaining a tooth model in the present application;

[0064] FIG2 is a schematic diagram of the working process of the neural network model in this application;

[0065] FIG3 is a schematic diagram showing the positional relationship between the output of the neural network model and the first tooth model data in the present application;

[0066] FIG4 is a schematic diagram of the fusion process of the first tooth model data and the predicted point cloud data in this application;

[0067] FIG5 is a schematic diagram of a device for obtaining a tooth model provided by the present application;

[0068] FIG6 is a schematic diagram of an electronic device corresponding to FIG1 provided in this application. DETAILED DESCRIPTION

[0069] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0070] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0071] FIG1 is a flow chart of a method for obtaining a tooth model provided in this application, which specifically includes the following steps:

[0072] S100: Acquire first tooth model data of a first tooth of a patient; the first tooth includes at least one attachment.

[0073] All steps in the method for obtaining a tooth model provided in the present application can be implemented by any electronic device with computing capabilities, such as a terminal, a server, and the like.

[0074] The method for obtaining a tooth model provided in this application is used to simulate, in software, the state of a tooth with attachments after the attachments have been removed. Therefore, in this step, first tooth model data of the patient's first tooth can be obtained. The patient's first tooth is a tooth with at least one attachment; the first tooth model data is constructed in three-dimensional space, with each three adjacent vertices forming a facet. Vertices can be obtained in a variety of ways, such as oral scanning, and this application does not impose specific limitations on this.

[0075] S102: Based on the neural network model, the first tooth model data is predicted to obtain predicted point cloud data; the predicted point cloud data is the tooth model data predicted by the neural network model excluding the first accessory, and the first accessory is the accessory on the first tooth.

[0076] In this step, the first tooth model data obtained in step S100 can be input into a pre-trained neural network model. It can be imagined that when any tooth model is input into a neural network, it can be considered as inputting the position information of several vertices and the connection relationship between the vertices. In the method for obtaining a tooth model provided in this application, all tooth models are constructed using three adjacent vertices to form a patch. Therefore, when the connection method between the points is fixed, it can be considered as inputting a point cloud set into the neural network model.

[0077] In the method for obtaining a tooth model provided in the present application, the function of the neural network model is to predict the state of the accessory area in the tooth model after the accessories are removed based on the input point cloud set, and output the predicted point cloud data of the tooth model without the accessories.

[0078] Additionally, in practical applications, it's important to consider that the number of vertices in the first tooth model data obtained during oral scanning varies for different teeth. However, for neural network models, a fixed number of vertices in the input point cloud facilitates model parameter adjustment and allows for training a neural network model with better prediction performance. Therefore, after obtaining the first tooth model data, a fixed-size point cloud can be extracted and used as input for the neural network model.

[0079] Specifically, second point cloud data and third point cloud data can be obtained based on the first tooth model data; the data volume of the second point cloud data and the third point cloud data is different; feature extraction is performed on the second point cloud data to obtain second point cloud feature data; feature extraction is performed on the third point cloud data to obtain third point cloud feature data; the second point cloud feature data and the third point cloud feature data are input into the prediction subnet of the neural network model to obtain predicted point cloud data.

[0080] Taking the acquisition of the second point cloud data as an example, when obtaining point cloud data of other dimensions, specifically, determine the first candidate point cloud data corresponding to the candidate vertices of the first tooth model data; determine the second candidate point cloud data in the first tooth model data based on the distance information between the first candidate point cloud data and the first tooth model data; determine the second point cloud data based on the first candidate point cloud data and the second candidate point cloud data.

[0081] In practical applications, one point that needs to be taken into consideration is that when the model has multi-scale input, the model can extract a variety of features with different focuses. Therefore, each time the original point cloud data is constructed based on the first tooth model data, a plurality of original point cloud data of different sizes can be constructed, that is, a plurality of original point cloud data containing different numbers of vertices, namely the second point cloud data and the third point cloud data. Each original point cloud data is used as input and input into the neural network model in parallel so that the neural network model can extract point cloud features of multiple different scales and obtain better output results. Correspondingly, the neural network model can be a model that can receive multi-scale input, such as a neural network model such as Point Fractal Network (PF-Net), and this application does not impose specific restrictions on this. Among them, the number of original point cloud data determined and the number of vertices contained in each original point cloud data can be preset according to specific needs.

[0082] The original point cloud data can be constructed in a variety of ways, and this application provides an embodiment for reference. Specifically, for each piece of original point cloud data, candidate vertices can be selected from the vertices of the first tooth model data to create the original point cloud data; for each vertex in the first tooth model data that is not in the original point cloud data, the minimum distance between the vertex and the original point cloud data is determined; and the vertex with the largest minimum distance from the vertex that is not in the original point cloud data is added to the original point cloud data until the number of vertices contained in the original point cloud data is no less than the number of vertices preset for the original point cloud data.

[0083] A possible implementation method is that the difference between several original point cloud data determined based on a first tooth model data is that the number of vertices contained is different. Therefore, each original point cloud data can be constructed in the same way. When constructing an original point cloud data, initially, the original point cloud data is empty and does not contain any vertices. First, a vertex can be randomly selected from all the vertices of the first tooth model data and added to the original point cloud data. Subsequently, among all the remaining vertices, the vertex with the largest minimum distance to the original point cloud data is selected and added to the original point cloud data, and this process is repeated until the number of vertices contained in the original point cloud data is not less than the preset number of vertices of the original point cloud data. In this way, the original point cloud data representing the first tooth model data can be obtained.

[0084] The minimum distance from a vertex not in the original point cloud data to the original point cloud data is the minimum distance between the vertex and each vertex in the original point cloud data. Specifically, when determining the minimum distance from a vertex to the original point cloud data, the distance between the vertex and each vertex in the original point cloud data can be determined; the minimum distance among the determined distances is selected as the minimum distance from the vertex to the original point cloud data.

[0085] For example, when there is only one vertex A in the original point cloud data, the minimum distance from a vertex to the original point cloud data is the distance between the vertex and vertex A in the original point cloud data. Therefore, the vertex B with the largest distance from vertex A in the original point cloud data among the other vertices in the first tooth model data can be added to the original point cloud data. At this time, there are two vertices in the original point cloud data, namely vertex A and vertex B. Then the minimum distance from a vertex that is not in the original point cloud data to the original point cloud data is the smaller value between the distance from the vertex to vertex A and the distance from the vertex to vertex B. Among all the vertices that are not in the original point cloud data, select vertex C with the largest minimum distance to the original point cloud data and add it to the original point cloud data. Repeat the above steps until the number of vertices in the original point cloud data meets the preset number of vertices.

[0086] Optionally, after constructing the raw point cloud data, the raw point cloud data can be normalized to simplify the workload of the neural network model when calculating the data. Specifically, the center point of the raw point cloud data can be determined based on the positions of each vertex in the raw point cloud data; the raw point cloud data is then normalized based on the center point.

[0087] Among them, the center point of the original point cloud data can be the average of the position coordinates of each vertex in the original point cloud data. After determining the center point of the original point cloud data, all points in the original point cloud data can be translated so that the center point of the original point cloud data is located at the origin of the coordinate system. Subsequently, the vertex with the largest distance from the center point in the original point cloud data can be determined, and all vertices can be normalized using the distance between the vertex and the center point as the normalization coefficient. In other words, the distance between the vertex and the center point is used as a divisor, and the distance between all vertices and the center point is divided by the divisor to obtain the normalized distance between each vertex and the center point, and the position coordinates of each vertex are adjusted accordingly.

[0088] S104: Determine a first region of the first tooth model data according to the predicted point cloud data; the first region includes a region corresponding to the first attachment.

[0089] According to the predicted point cloud data determined in step S102, a first region in the first tooth model data can be further determined in this step, wherein the first region is a region in the first tooth model data including the first attachment.

[0090] When determining the first area, specifically, a first patch of the first tooth model data can be obtained; the first patch is at least one patch in the first tooth model data; based on the distance information between the first patch and each patch corresponding to the predicted point cloud data, it is determined whether the first patch belongs to the first area, wherein each patch is composed of three adjacent data points in the point cloud data.

[0091] Among them, when judging whether the first surface patch belongs to the first area, specifically, the position information of the first surface patch can be determined; based on the position information of the first surface patch and the position information of each surface patch corresponding to the predicted point cloud data, the shortest distance information between the first surface patch and the predicted point cloud data is determined; when the shortest distance information meets the first threshold condition, it is determined that the first surface patch does not belong to the first area.

[0092] In this method, although the first tooth model data and the predicted point cloud data are two different point cloud sets, they are both point cloud sets related to the first tooth. They reside in the same space and are measured using the same coordinate system. In other words, their relative positions are true relative positions. Therefore, based on the distance information between the patches, it can be determined whether the first patch belongs to the first region.

[0093] In this method, the first region is the area where the attachment is located in the first tooth model data. That is, the meshes in the first region represent the meshes of the attachment in the first tooth model data. As explained in the above steps, the first tooth model data is a tooth model that includes the attachment, and the predicted point cloud data is a tooth model that does not include the attachment. Based on this, it can be understood that the difference between the first tooth model data and the predicted point cloud data should be the point cloud data of the first region where the attachment is located.

[0094] Therefore, whether the first patch in the first tooth model data belongs to the first region can be determined based on the distance information. Specifically, based on the position information of the first patch and the position information of each patch corresponding to the predicted point cloud data, a second patch in the predicted point cloud data can be determined, and the distance between the second patch and the first patch satisfies a first condition. When the distance between the first patch and the second patch satisfies a second threshold condition, the first patch is determined not to belong to the first region. The position information of a patch can be the coordinate information of the patch's center point.

[0095] Among them, the distance between the second facet and the first facet satisfies the first condition, which can be that the second facet is the facet with the shortest distance to the first facet among all the facets of the predicted point cloud data; the distance between the first facet and the second facet satisfies the second threshold condition, which can be that the distance between the first facet and the second facet is not greater than the specified distance.

[0096] By adopting the above method, it is possible to find the face with the shortest distance to the first face in the predicted point cloud data, that is, the face with the highest similarity. When the distance between the two is short enough, that is, not greater than the specified distance, it can be considered that the positions of the two in physical space are repeated. At this time, the first face and the second face can be regarded as the same face in different point cloud sets. Since there is no attachment in the predicted point cloud set, for any first face in the first tooth model data, as long as there is a face with a distance from it that is not greater than the specified distance in the predicted point cloud set, it can be considered that the first face is not a face belonging to the attachment area, that is, it does not belong to the first area. Conversely, for the first face, when there is no second face with a distance from it that is not greater than the specified distance in the predicted point cloud set, it can be considered that the first face is a face representing the attachment and belongs to the first area.

[0097] In this way, all the facets belonging to the first region in the first tooth model data can be determined, and then the first region can be determined.

[0098] S106: Generate second tooth model data according to the first area, the first tooth model data and the predicted point cloud data; the second tooth model data is used to indicate the first tooth with the first attachment removed.

[0099] In this step, second tooth model data after the attachments have been removed can be constructed based on the first tooth model data obtained in step S100, the predicted point cloud data obtained in step S102, and the first region obtained in step S104. Specifically, first point cloud data can be determined based on the first region, where the first point cloud data is the point cloud data of the first tooth model data excluding the first region. The first point cloud data and the predicted point cloud data are fused to generate the second tooth model data.

[0100] When constructing the complete second tooth model data after grinding away the attachments, it is necessary to first remove the attachment part in the first tooth model data, and then integrate the predicted point cloud data output by the neural network model, that is, the tooth model without attachments, into the first tooth model data.

[0101] More advantageously, because in actual applications, the data points in the first dental model data are directly acquired from real teeth through methods such as 3D dental scans, while the predicted point cloud data is predicted using a neural network model, the accuracy of the positions of the points in the first dental model data is generally higher than that of the predicted point cloud data. Therefore, during the fusion process, areas of the first dental model data not related to the attachments can be retained without using the content in the predicted point cloud data, further improving the accuracy of the reconstructed second dental model data.

[0102] Based on the above idea, the output of the predicted point cloud data can be changed so that the predicted point cloud data only outputs the tooth surface area of ​​the portion of the first tooth with the attachment after the attachment is removed.

[0103] As shown in Figures 2 and 3, Figure 2 is a schematic diagram of the predicted point cloud data output by the neural network model based on a first tooth model data set; Figure 3 is a schematic diagram of the neural network model output after restoring the first tooth model data set. For ease of understanding, Figures 2 and 3 only show simplified views of the point cloud set; individual vertices are not drawn in either Figure.

[0104] In Figure 2, the arc-shaped protrusion on the right side of the first tooth model data is the attachment, and the part circled by the dotted ellipse in the figure is the attachment area, that is, the attachment and the area around the attachment. After the first tooth model data is input into the neural network model, the neural network model will output a point cloud set of the tooth surface area after the attachment in the attachment area is ground off. In Figure 2, since the observation angle is from the side of the tooth surface, it is observed as a straight line; when observed from the front of the tooth surface, a plane can actually be observed. Figure 3 shows a schematic diagram of restoring the point cloud set of the attachment area output by the neural network model to the tooth model after the attachment is ground off. It should be noted that Figure 3 is only a schematic diagram given in this application to facilitate understanding of the model output. In actual applications, the restoration step shown in Figure 3 does not exist.

[0105] At this time, the method of removing the attachment part in the first tooth model data is to isolate the attachment in the first tooth model data by comparing the first tooth model data with the predicted point cloud data, and to achieve the purpose of removing the attachment by deleting the free surface patches in the first tooth model data.

[0106] Specifically, as mentioned in step S100 of this application, in the method for obtaining a dental model provided herein, the connection relationship between each vertex in all dental models and point cloud sets is that three adjacent vertices form a facet. Therefore, the attachment removal operation in the first dental model data can be performed using the facet as the smallest unit. The neural network model does not change the size or coordinate system of the dental model during output, so the first dental model data can be compared with the output predicted point cloud data in the same coordinate system.

[0107] For each facet within the attachment area in the first dental model data, the minimum distance between the facet and the predicted point cloud data can be determined. When the minimum distance is no greater than the specified distance, the facet can be deleted from the first dental model data. This allows the attachment portion of the first dental model data to be isolated, making the facets of the attachment portion free faces. Subsequently, the free facets in the first dental model data can be deleted, that is, the attachment portion of the first dental model data can be deleted. Finally, the first dental model data after the attachment is removed is fused with the predicted point cloud data to obtain the second dental model data after the attachment is removed.

[0108] The minimum distance from a patch in the first dental model data to the predicted point cloud data may be the minimum value of the distances between the patch and all patches in the predicted point cloud data. Specifically, when determining the minimum distance from a patch in the first dental model data to the predicted point cloud data, the center point of each patch in the first dental model data may be determined based on the vertices constituting each patch in the first dental model data, and the center point of each patch in the predicted point cloud data may be determined based on the vertices constituting each patch in the predicted point cloud data; for each patch in the attachment area of ​​the first dental model data, the distance between the center point of the patch and the center point of each patch in the predicted point cloud data may be determined; and the minimum distance among the determined distances may be selected as the minimum distance from the patch to the predicted point cloud data.

[0109] For any patch, the average of the positional coordinates of the three vertices that make up the patch can be used as the patch's center point. Thus, the distance between any two patches can be determined as the distance between their centers. The minimum distance from a patch in the attachment area of ​​the first tooth model data to the predicted point cloud data can be the minimum of the distances between the patch's center point and the center point of each patch in the predicted point cloud data.

[0110] FIG4 illustrates the process of constructing the second tooth model data after removing the attachments based on the first tooth model data and the predicted point cloud data. As shown in FIG4 , the first tooth model data and the predicted point cloud data are first placed in the same coordinate system. Subsequently, in process A, each facet in the attachment area of ​​the first tooth model data whose minimum distance from the predicted point cloud data is no greater than a specified distance is deleted to obtain the first tooth model data with the attachments isolated. Subsequently, in process B, the free facets in the first tooth model data, that is, the attachment portion, can be deleted to obtain the first tooth model data after removing the attachments. Subsequently, in process C, the first tooth model data and the predicted point cloud data can be fused to obtain the second tooth model data after grinding away the attachments.

[0111] More preferably, after obtaining the second tooth model data, the area near the suture between the first tooth model data and the predicted point cloud data in the second tooth model data may be smoothed to obtain more natural second tooth model data.

[0112] When using the method for obtaining a tooth model provided in the present application to predict the second tooth model data after the attachments are ground off the first tooth model data, the predicted point cloud data of the attachment area after the attachments are ground off can be obtained based on the first tooth model data through a neural network model, and the attachment part in the first tooth model data can be removed by comparing it with the predicted point cloud data. Finally, the first tooth model data after the attachments are removed and the predicted point cloud data are fused to obtain the second tooth model data. Compared with traditional methods, this method does not require any manual operation during implementation, reducing the inaccuracy caused by errors in manual operation. At the same time, the method of obtaining the second tooth model data by fusing the predicted point cloud data with the first tooth model data can not only more accurately restore the shape of the tooth model after the attachments are removed, but also retain the original information of the tooth model to the greatest extent.

[0113] Additionally, the first and second dental model data constructed in the present method can be displayed to the user as needed, in conjunction with actual medical operations by the user. Specifically, in response to receiving a display instruction, the first and / or second dental model data can be displayed according to the content of the display instruction.

[0114] Furthermore, after observing the first tooth model data, the user can, as needed, use the function of removing the attachment to cause the execution subject to further display the second tooth model data to the user. Specifically, before displaying the second tooth model data, the first attachment in the first tooth model data can be removed in response to receiving a removal instruction. Similarly, in the present method, the execution of determining the first area can also be performed according to user needs. Specifically, the first area of ​​the first tooth model data can be updated in response to receiving an update instruction; the second tooth model data can be updated based on the updated first area, the first tooth model data, and the predicted point cloud data; the second tooth model data is used to indicate the first tooth with the first attachment removed.

[0115] Additionally, the neural network model used in the method for obtaining a tooth model provided in this application can be pre-trained. Specifically, during an accessory removal surgery, a sample tooth model can be obtained based on the pre-operative state of the tooth, and an annotation can be obtained based on the post-operative state of the tooth, wherein the annotation is a point cloud set of the area on the tooth with the accessory removed. The sample tooth model is input into the neural network model to be trained, and the predicted point cloud data output by the neural network model is determined. The neural network model is trained with the optimization goal of minimizing the difference between the predicted point cloud data and the annotation.

[0116] The sample tooth model is a 3D mesh model of a tooth requiring accessory grinding surgery before the procedure; the tooth marked as requiring accessory grinding after the procedure is a point cloud collection corresponding to the original accessory area. It is worth noting that, for the sample tooth model, several point cloud collections can also be constructed using the method described in this application and used as input to the neural network model during training, replacing the sample tooth model. This application will not elaborate on this further.

[0117] A possible implementation is also provided below.

[0118] 1. Point cloud processing of triangular mesh model

[0119] The initial data for this application is a triangulated mesh of a single tooth, and we need to perform some pre-processing to obtain data that can be used as input to the network.

[0120] Here are the steps:

[0121] A. Obtain all vertex coordinates (X, Y, Z) from the 3D tooth model M and denote the sampling point set as P.

[0122] B. Farthest Point Sampling (FPS): Set the maximum number of sampled point clouds to N. Randomly select a vertex pt1 from all vertices in the 3D tooth model M and add it to the sampling set P. Calculate the distance D1 from all vertices to pt1. Select pt2, the vertex farthest from pt1, and add it to the set P. Calculate the distance D2 from all points not in P to pt2. Combining D1 and D2, select the minimum distance from each non-P point to a point in P, denoted as the non-P to P distance D.

[0123] C. During iteration B: select the point with the farthest distance from non-P to P, add it to P, and update D. This continues until the size of point set P is N.

[0124] D. Perform distance normalization on the points in P: The center point of the P(X, Y, Z) point set can be the average of the position coordinates of each point in the P(X, Y, Z) point set, that is, Mean(P(X, Y, Z)). All points in the P(X, Y, Z) point set are translated by P(X, Y, Z)_=P(X, Y, Z)-Mean(P(X, Y, Z)) so that the center point of the P(X, Y, Z) point set is moved to the origin of the coordinate system (0, 0, 0), where P(X, Y, Z)_ is the point set obtained after the P(X, Y, Z) point set is translated; calculate the distances of all points in P(X, Y, Z)_ to (0, 0, 0) to obtain the maximum distance D MAX ; then D MAX is the normalization coefficient, through P(X, Y, Z)_processed=P(X, Y, Z)_ / D MAX , to achieve normalization of the P(X, Y, Z)_point set.

[0125] E. After ABCD processing, the data set P before inputting the network is obtained input Since the network has a multi-scale input and output structure, in actual operation, P will be adjusted according to the multi-scale requirements. input By setting different N and performing steps B and C multiple times, we can obtain P at multiple scales. input (N1, N2, N3) set.

[0126] 2. A special training method for neural networks to solve the problem of accessory removal

[0127] A common problem addressed by neural networks is predicting the shape of the missing portion of an object based on the remaining shape. Specifically, the input data is a point cloud of the incomplete shape, and the output data is a point cloud of the missing portion. In this application, the input data is a point cloud of a complete tooth with an attachment attached, and the output data is a point cloud of the tooth's attachment area after the attachment has been removed. A common approach is to predict the point cloud of the attachment and then remove the predicted point cloud to form an input-output pattern suitable for the neural network. However, in practice, the resulting missing portion point cloud can deviate significantly from the actual point cloud, especially when the data size is small. The advantages of this application include: 1. The original shape of the attachment is fixed, but the shape of the missing portion can vary depending on the predicted attachment point cloud and the missing portion created. In this case, allowing the neural network to select the region itself is superior to using auxiliary selection. 2. In most cases, there is no need to specifically predict the attachment area; the attachment area can be inferred by calculating the overlap between the generated missing point cloud and the input point cloud in a certain direction, with the effective overlap exceeding 90%. 3. This approach makes better use of data information and expands the application scope of neural network models.

[0128] 3. Fusion of predicted point cloud and residual point cloud and reconstruction of triangular mesh

[0129] This stage mainly removes the attachments on the 3D mesh and fuses the predicted point cloud with the 3D mesh to reconstruct the shape after the attachments are removed. The specific steps are as follows:

[0130] A. Calculate the center coordinates Center(X, Y, Z) of the facets in the 3D mesh model (i.e., tooth model), and calculate the shortest distance D from Center(X, Y, Z) to the predicted point cloud set Output(X, Y, Z) generated by the neural network. nearest .

[0131] B. When D nearest If the value is less than a certain threshold Thr, the corresponding facets on the tooth model are removed, so that the facets of the attachment part on the tooth model become free facets. At the same time, the free facets in the tooth model are deleted to obtain the tooth model M without the attachment part. crop .

[0132] C. Triangulated mesh output point cloud and M crop Suture to form the final tooth model M after grinding new .

[0133] D.To M new Smoothing is performed on 2-3 neighborhoods around the stitching area.

[0134] After A, B, C, and D, the final tooth triangular mesh model M is obtained. new ,Here, in order to obtain a more robust and accurate attachment area, ,the B operation can also be integrated into the method of predicting the ,attachment area to comprehensively evaluate the final target area.

[0135] 4. Neural network structure and parameter setting

[0136] The neural network training method can refer to existing training methods. The point cloud required for prediction is generated in a multi-scale manner, and prediction accuracy is optimized using adversarial loss and distance loss. Possible training parameters and optimizer parameters include: input data scale: 2048×3, 1024×3, 512×3; output scale: 256×3, 128×3, 64×3; batch size: 24; Adam optimizer; learning rate = 0.0001.

[0137] Accordingly, the present application also provides a method for displaying a tooth model, comprising:

[0138] Displaying second dental model data, where the second dental model data is obtained using the dental model obtaining method provided herein. Additionally, the method may further display the first dental model data. Specifically, the first dental model data may be displayed in response to a first display instruction input by the user; or the second dental model data may be displayed in response to a second display instruction input by the user.

[0139] The first display instruction and the second display instruction may be generated based on a user operation. For example, the user may activate corresponding software to instruct the display of the first tooth model, or may activate a corresponding function in a corresponding patient case. The functions may be determined based on actual needs and are not limited here. The display instruction method may also refer to the display instruction or update instruction in the above embodiment.

[0140] The above is the method for obtaining a tooth model provided by the present application. Based on the same idea, the present application also provides a corresponding device for obtaining a tooth model, as shown in FIG5 .

[0141] FIG5 is a schematic diagram of a device for obtaining a tooth model provided by the present application, which specifically includes:

[0142] An acquisition module 200 is configured to acquire first tooth model data of a first tooth of a patient; the first tooth includes at least one attachment;

[0143] A prediction module 202 is configured to predict the first tooth model data based on a neural network model to obtain predicted point cloud data; the predicted point cloud data is the tooth model data predicted by the neural network model excluding the first attachment, where the first attachment is an attachment on the first tooth;

[0144] A determination module 204 is configured to determine a first region of the first tooth model data based on the predicted point cloud data; the first region includes a region corresponding to the first attachment;

[0145] The generating module 206 is configured to generate second tooth model data according to the first region, the first tooth model data and the predicted point cloud data; the second tooth model data is used to indicate the first tooth with the first attachment removed.

[0146] Optionally, the determination module 204 is specifically used to obtain a first facet of the first tooth model data; the first facet is at least one facet in the first tooth model data; and determine whether the first facet belongs to the first area based on the distance information between the first facet and each facet corresponding to the predicted point cloud data, wherein each facet is composed of three adjacent data points in the point cloud data.

[0147] Optionally, the determination module 204 is specifically used to determine the position information of the first facet; determine the shortest distance information between the first facet and the predicted point cloud data based on the position information of the first facet and the position information of each facet corresponding to the predicted point cloud data; when the shortest distance information meets the first threshold condition, determine that the first facet does not belong to the first area.

[0148] Optionally, the determination module 204 is specifically used to determine the second facet in the predicted point cloud data based on the position information of the first facet and the position information of each facet corresponding to the predicted point cloud data, and the distance between the second facet and the first facet meets the first condition; when the distance between the first facet and the second facet meets the second threshold condition, determine that the first facet does not belong to the first area.

[0149] Optionally, the position information includes coordinate information of the center point of the patch.

[0150] Optionally, the generation module 206 is specifically used to determine first point cloud data based on the first area, where the first point cloud data is other point cloud data in the first tooth model data except the first area; and fuse the first point cloud data and the predicted point cloud data to generate second tooth model data.

[0151] Optionally, the prediction module 202 is specifically used to obtain second point cloud data and third point cloud data based on the first tooth model data; the second point cloud data and the third point cloud data have different data volumes; perform feature extraction on the second point cloud data to obtain second point cloud feature data; perform feature extraction on the third point cloud data to obtain third point cloud feature data; input the second point cloud feature data and the third point cloud feature data into the prediction subnet of the neural network model to obtain predicted point cloud data.

[0152] Optionally, the prediction module 202 is specifically used to determine the first candidate point cloud data corresponding to the candidate vertices of the first tooth model data; determine the second candidate point cloud data in the first tooth model data based on the distance information between the first candidate point cloud data and the first tooth model data; and determine the second point cloud data based on the first candidate point cloud data and the second candidate point cloud data.

[0153] Optionally, the apparatus further comprises a first response module 208, which is specifically configured to, in response to receiving a display instruction, display the first tooth model data and / or the second tooth model data according to the content of the display instruction.

[0154] Optionally, the device further includes a second response module 210, specifically configured to remove the first attachment in the first tooth model data in response to receiving a removal instruction.

[0155] Optionally, the device also includes a third response module 212, which is specifically used to update the first area of ​​the first tooth model data in response to receiving an update instruction; update the second tooth model data based on the updated first area, the first tooth model data and the predicted point cloud data; the second tooth model data is used to indicate the removal of the first tooth of the first attachment.

[0156] The present application also provides a computer-readable storage medium, which stores a computer program. The computer program can be used to execute the method for obtaining a tooth model provided in FIG. 1 .

[0157] The present application also provides a schematic structural diagram of an electronic device as shown in FIG6 . As shown in FIG6 , at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the method for obtaining a tooth model and the display method described in FIG1 above. Of course, in addition to software implementation, the present application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0158] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0159] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0160] The memory may also include a program tool (or utility) having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0161] The processor executes various functional applications and data processing by running the computer programs stored in the memory, such as the method in the above embodiment.

[0162] The electronic device may also communicate with one or more external devices. Such communication may be performed via an input / output (I / O) interface. Furthermore, the model-generated electronic device may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter. The network adapter communicates with other modules of the electronic device via a bus. It should be understood that, although not shown in the figures, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (RAID) systems, tape drives, and data backup storage systems.

[0163] The input and / or output device may include a scanning device, a camera interface, an input device (e.g., a mouse, a keyboard, etc.), a display device (e.g., a monitor), a printer, and / or one or more other input devices. The input / output interface may receive executable instructions and / or data, which may be stored in a data storage device (e.g., a memory). For example, it may be used to receive a generated face centerline and store the generated face centerline.

[0164] In some embodiments, the scanning device can be configured to scan one or more physical dental models of a patient's dentition. In one or more embodiments, the scanning device can be configured to directly scan a patient's dentition and / or dental appliances. The scanning device can be configured to input data into a computing device.

[0165] In some embodiments, the camera interface can receive input from an imaging device (e.g., a 2D or 3D imaging device), such as a digital camera, a print photo scanner, and / or other suitable imaging device. For example, the input from the imaging device can be stored in a memory.

[0166] The processor can execute instructions to provide a display of a dental model, dental treatment positions, treatment plans, etc. on a display. For example, the computing device can be configured to allow a treating professional or other user to input treatment goals. The received input can be sent to the processor as data and / or can be stored in a memory.

[0167] Such connectivity may allow for the input and / or output of data and / or instructions and other types of information.Some embodiments may be distributed among various computing devices within one or more networks and used to collect, calculate, and / or analyze any of the methods described herein.

[0168] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, depending on the embodiment of the present application, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0169] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method in the above embodiment is implemented.

[0170] The readable storage medium may include, but is not limited to, a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0171] In a possible implementation manner, the present application may also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps in the above embodiments.

[0172] The program code for executing the present application may be written in any combination of one or more programming languages, and the program code may be executed entirely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or entirely on the remote device.

[0173] The various embodiments in this application are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiment is generally similar to the method embodiment, so the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment.

[0174] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for obtaining a dental model, characterized in that, Including: Obtaining first tooth model data of a patient's first tooth; The first tooth includes at least one attachment; Based on a neural network model, predicting the first tooth model data to obtain predicted point cloud data; the predicted point cloud data is tooth model data predicted by the neural network model that does not include a first attachment, and the first attachment is an attachment on the first tooth; Determining a first region of the first tooth model data according to the predicted point cloud data; The first region includes a region corresponding to the first attachment; Generating second tooth model data according to the first region, the first tooth model data, and the predicted point cloud data; The second tooth model data is used to indicate the first tooth with the first attachment removed.

2. The method according to claim 1, characterized in that, Determining a first region of the first tooth model data according to the predicted point cloud data specifically includes: Obtaining a first patch of the first tooth model data; the first patch is at least one patch in the first tooth model data; Determining whether the first patch belongs to the first region according to distance information between the first patch and corresponding patches of the predicted point cloud data, where each patch is composed of three adjacent data points in the point cloud data.

3. The method according to claim 2, wherein Determining whether the first patch belongs to the first region according to distance information between the first patch and the patches of the predicted point cloud data specifically includes: Determining position information of the first patch; Determining shortest distance information between the first patch and the predicted point cloud data according to the position information of the first patch and the position information of the corresponding patches of the predicted point cloud data; When the shortest distance information meets a first threshold condition, determining that the first patch does not belong to the first region.

4. The method according to claim 3, wherein When the shortest distance information meets a first threshold condition, determining that the first patch does not belong to the first region specifically includes: Determining a second patch in the predicted point cloud data according to the position information of the first patch and the position information of the corresponding patches of the predicted point cloud data, and the distance between the second patch and the first patch meets a first condition; When the distance between the first patch and the second patch meets a second threshold condition, determining that the first patch does not belong to the first region.

5. The method according to any one of claims 3-4, characterized in that The position information includes central point coordinate information of the patch.

6. The method according to claim 1, characterized in that, Generating second tooth model data according to the first region, the first tooth model data, and the predicted point cloud data specifically includes: Determining first point cloud data according to the first region, where the first point cloud data is other point cloud data in the first tooth model data except the first region; Fusing the first point cloud data and the predicted point cloud data to generate second tooth model data.

7. The method according to claim 1, wherein Based on a neural network model, predicting the first tooth model data to obtain predicted point cloud data, including: Obtaining second point cloud data and third point cloud data according to the first tooth model data; the data amounts of the second point cloud data and the third point cloud data are different; Performing feature extraction on the second point cloud data to obtain second point cloud feature data; Extract features from the third point cloud data to obtain third point cloud feature data; Input the second point cloud feature data and the third point cloud feature data into the prediction subnet of the neural network model to obtain predicted point cloud data.

8. The method according to claim 7, wherein Obtain second point cloud data according to the first tooth model data, including: Determine first candidate point cloud data corresponding to candidate vertices of the first tooth model data; Determine second candidate point cloud data in the first tooth model data according to the distance information between the first candidate point cloud data and the first tooth model data; Determine the second point cloud data according to the first candidate point cloud data and the second candidate point cloud data.

9. The method according to claim 1, wherein The method further includes: In response to receiving a display instruction, display the first tooth model data and / or the second tooth model data according to the content of the display instruction.

10. The method according to claim 9, wherein Before displaying the second tooth model data, the method further includes: In response to receiving a removal instruction, remove the first attachment in the first tooth model data.

11. The method according to claim 1, wherein The method further includes: In response to receiving an update instruction, update a first region of the first tooth model data; Update the second tooth model data according to the updated first region, the first tooth model data, and the predicted point cloud data; the second tooth model data is used to indicate the first tooth with the first attachment removed.

12. A display method for a dental model, characterized in that, Includes: Display the second tooth model data, which is obtained by the method for obtaining a tooth model according to any one of claims 1-11.

13. The method according to claim 12, wherein The method further includes: Display the first tooth model data.

14. The method according to claim 12 or 13, characterized in that, The method further includes: In response to a first display instruction input by a user, display the first tooth model data; Or, in response to a second display instruction input by a user, display the second tooth model data.

15. An apparatus for obtaining a dental model, characterized in that, Includes: An acquisition module for acquiring first tooth model data of a patient's first tooth; The first tooth includes at least one attachment; A prediction module for predicting the first tooth model data based on a neural network model to obtain predicted point cloud data; the predicted point cloud data is tooth model data predicted by the neural network model without the first attachment, and the first attachment is an attachment on the first tooth; A determination module for determining a first region of the first tooth model data according to the predicted point cloud data; The first region includes the region corresponding to the first attachment; A generation module for generating second tooth model data according to the first region, the first tooth model data, and the predicted point cloud data; The second tooth model data is used to indicate the first tooth with the first attachment removed.

16. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1-14 above is implemented.

17. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, the method according to any one of claims 1-14 above is implemented.

Images