Method and device for obtaining tooth model, storage medium and electronic equipment

Abstract

The invention discloses a method and device for obtaining a tooth model, a storage medium and electronic equipment. In the method for obtaining a tooth model provided in the present specification, first tooth model data of a first tooth of a patient is obtained; the first tooth comprises at least one accessory; based on a neural network model, predicting the first tooth model data to obtain predicted point cloud data; the predicted point cloud data is tooth model data which is predicted by the neural network model and does not comprise a first attachment, and the first attachment is an attachment on the first tooth; determining a first area of the first tooth model data according to the predicted point cloud data; the first area comprises an area corresponding to the first accessory; generating second tooth model data according to the first region, the first tooth model data and the predicted point cloud data; the second tooth model data is used for indicating removal of the first tooth of the first attachment.

Description

Technical Field

[0001] This specification relates to the field of medical technology, and particularly to a method, device, storage medium, and electronic device for obtaining a tooth model. Background Art

[0002] Nowadays, during the process of orthodontics, dental attachments are often used to assist in orthodontic treatment. Dental attachments are solid structures adhered to teeth and usually can play roles such as increasing the retention of orthodontic appliances and assisting tooth movement. Correspondingly, when the orthodontic process is completed, the dental attachments need to be removed from the teeth.

[0003] Currently, in most cases, the method of attachment grinding is used to remove the attachments on teeth. Usually, before performing the attachment grinding surgery, the state of the teeth before and after the surgery is simulated by establishing a tooth model to predict the situation of the teeth after the surgery.

[0004] Existing methods usually, after obtaining a tooth model with attachments, manually remove the attachment part in the tooth model and then use a pre-trained model to predict the state of the teeth after the surgery. However, the tooth model obtained in this way usually has a large difference from the real tooth model, and the result is not accurate enough.

[0005] Therefore, how to obtain a real and reliable tooth model before grinding dental attachments is an urgent problem to be solved. Summary of the Invention

[0006] This specification provides a method, device, storage medium, and electronic device for obtaining a tooth model to at least partially solve the above problems existing in the prior art.

[0007] This specification adopts the following technical solutions:

[0008] This specification provides a method for obtaining a tooth model, including:

[0009] Obtaining first tooth model data of a patient's first tooth; the first tooth includes at least one attachment;

[0010] Based on a neural network model, predicting the first tooth model data to obtain predicted point cloud data; the predicted point cloud data is tooth model data predicted by the neural network model without a first attachment, and the first attachment is an attachment on the first tooth;

[0011] Determining a first region of the first tooth model data according to the predicted point cloud data; the first region includes the region corresponding to the first attachment;

[0012] Generate second dental model data based on the first region, the first dental model data, and the predicted point cloud data; the second dental model data is used to indicate the teeth with the first attachments removed.

[0013] Optionally, determining the first region of the first dental model data according to the predicted point cloud data specifically includes:

[0014] Obtain the first patch of the first dental model data; the first patch is at least one patch in the first dental model data;

[0015] Determine whether the first patch belongs to the first region according to the distance information between the first patch and the corresponding patches of the predicted point cloud data, where each patch is composed of three adjacent data points in the point cloud data.

[0016] Optionally, determining whether the first patch belongs to the first region according to the distance information between the first patch and the patches of the predicted point cloud data specifically includes:

[0017] Determine the position information of the first patch;

[0018] Determine the shortest distance information of the first patch according to the position information of the first patch and the position information of the corresponding patches of the predicted point cloud data;

[0019] When the shortest distance information meets the first threshold condition, determine that the first patch does not belong to the first region.

[0020] Optionally, when the shortest distance information meets the first threshold condition, determining that the first patch does not belong to the first region specifically includes:

[0021] Determine the second patch in the predicted point cloud data according to the position information of the first patch and the position information of the corresponding patches of the predicted point cloud data, and the distance between the second patch and the first patch meets the first condition;

[0022] When the distance between the first patch and the second patch meets the second threshold condition, determine that the first patch does not belong to the first region.

[0023] Optionally, the position information includes the central point coordinate information of the patch.

[0024] Optionally, generating the second dental model data according to the first region, the first dental model data, and the predicted point cloud data specifically includes:

[0025] Determine the first point cloud data according to the first region, where the first point cloud data is the other point cloud data in the first dental model data except the first region;

[0026] Fuse the first point cloud data and the predicted point cloud data to generate second dental model data.

[0027] Optionally, predicting the first dental model data based on a neural network model includes:

[0028] Obtain second point cloud data and third point cloud data according to the first dental model data; the data volumes of the second point cloud data and the third point cloud data are different;

[0029] Extract features from the second point cloud data to obtain second point cloud feature data;

[0030] Extract features from the third point cloud data to obtain third point cloud feature data;

[0031] Input the second point cloud feature data and the third point cloud feature data into the prediction subnet of the neural network model to obtain predicted point cloud data.

[0032] Optionally, obtaining second point cloud data according to the first dental model data includes:

[0033] Determine first candidate point cloud data corresponding to candidate vertices of the first dental model data;

[0034] Determine second candidate point cloud data in the first dental model data according to the distance information between the first candidate point cloud data and the first dental model data;

[0035] Determine second point cloud data according to the first candidate point cloud data and the second candidate point cloud data.

[0036] Optionally, the method further includes:

[0037] In response to receiving a display instruction, display the first dental model data and / or the second dental model data according to the content of the display instruction.

[0038] Optionally, before displaying the second dental model data, the method further includes:

[0039] In response to receiving a removal instruction, remove a first attachment in the first dental model data.

[0040] Optionally, the method further includes:

[0041] In response to receiving an update instruction, update a first region of the first dental model data;

[0042] Update the second dental model data according to the updated first region, the first dental model data, and the predicted point cloud data; the second dental model data is used to indicate the first tooth with the first attachment removed.

[0043] An embodiment of the present application provides a method for displaying a dental model, including:

[0044] Display the second dental model data, where the second dental model data is obtained by the method for obtaining a dental model according to any one of the above embodiments.

[0045] Optionally, the method further includes:

[0046] Display the first dental model data.

[0047] Optionally, the method further includes:

[0048] In response to a first display instruction input by the user, display the first dental model data;

[0049] Or, in response to a second display instruction input by the user, display the second dental model data.

[0050] An apparatus for obtaining a dental model provided in this specification, the apparatus includes:

[0051] An acquisition module, configured to acquire first dental model data of a first tooth of a patient; the first tooth includes at least one attachment;

[0052] A prediction module, configured to perform prediction on the first dental model data based on a neural network model to obtain predicted point cloud data; the predicted point cloud data is dental model data predicted by the neural network model without the first attachment, and the first attachment is an attachment on the first tooth;

[0053] A determination module, configured to determine a first region of the first dental model data according to the predicted point cloud data; the first region includes the region corresponding to the first attachment;

[0054] A generation module, configured to generate second dental model data according to the first region, the first dental model data, and the predicted point cloud data; the second dental model data is used to indicate the first tooth with the first attachment removed.

[0055] This specification provides a computer-readable storage medium, where the storage medium stores a computer program, and when the computer program is executed by a processor, the method for obtaining a dental model described above is implemented.

[0056] This specification provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method for obtaining a tooth model as described above is implemented.

[0057] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0058] In the method for obtaining a tooth model provided in this specification, first tooth model data of a patient's first tooth is obtained; the first tooth includes at least one attachment; based on a neural network model, the first tooth model data is predicted to obtain predicted point cloud data; the predicted point cloud data is tooth model data predicted by the neural network model that does not include the first attachment, and the first attachment is an attachment on the first tooth; according to the predicted point cloud data, a first region of the first tooth model data is determined; the first region includes the region corresponding to the first attachment; according to the first region, the first tooth model data, and the predicted point cloud data, second tooth model data is generated; the second tooth model data is used to indicate the first tooth after removing the first attachment.

[0059] When using the method for obtaining a tooth model provided in this specification to predict the second tooth model data after grinding the attachment from the first tooth model data, the neural network model can obtain the predicted point cloud data of the attachment region after grinding the attachment according to the first tooth model data, and by comparing with the predicted point cloud data, the attachment part in the first tooth model data is removed. Finally, the first tooth model data after removing the attachment and the predicted point cloud data are fused to obtain the second tooth model data. Compared with the traditional method, this method does not require any manual operation during the implementation process, reducing the inaccuracy caused by the error of manual operation. At the same time, by fusing the predicted point cloud data with the first tooth model data to obtain the second tooth model data, not only can the shape of the tooth model after removing the attachment be restored more accurately, but also the original information of the tooth model can be retained to the greatest extent. Description of the Drawings

[0060] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The schematic embodiments of this specification and their descriptions are used to explain this specification and do not constitute an improper limitation to this specification. In the drawings:

[0061] Figure 1 It is a schematic flowchart of a method for obtaining a tooth model in this specification;

[0062] Figure 2 It is a schematic diagram of the working process of the neural network model in this specification;

[0063] Figure 3 Schematic diagram of the positional relationship between the output of the neural network model and the first tooth model data in this specification;

[0064] Figure 4 Schematic diagram of the fusion process of the first tooth model data and the predicted point cloud data in this specification;

[0065] Figure 5 Schematic diagram of a device for obtaining a tooth model provided in this specification;

[0066] Figure 6 Corresponding to what is provided in this specification Figure 1 Schematic diagram of the electronic device Specific embodiments

[0067] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, rather than all the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

[0068] The following will detail the technical solutions provided in each embodiment of this specification in conjunction with the drawings.

[0069] Figure 1 Schematic diagram of the flowchart of a method for obtaining a tooth model in this specification, specifically including the following steps:

[0070] S100: Obtain the first tooth model data of the patient's first tooth; the first tooth includes at least one attachment.

[0071] All steps in the method for obtaining a tooth model provided in this specification can be implemented by any electronic device with computing functions, such as devices like terminals and servers.

[0072] The method for obtaining a tooth model provided in this specification is used to simulate the state of a tooth with attachments after the attachments are ground off in software. Therefore, in this step, the first tooth model data of the patient's first tooth can be obtained first. Among them, the patient's first tooth is a tooth with at least one attachment; the first tooth model data is the first tooth model data constructed in a three-dimensional space by taking every three adjacent vertices to form a patch. The vertices can be obtained through various methods, such as oral scanning and other methods, and this specification does not make specific limitations on this.

[0073] S102: Based on the neural network model, predict the first dental model data to obtain predicted point cloud data; the predicted point cloud data is the dental model data predicted by the neural network model without the first attachment, and the first attachment is the attachment on the first tooth.

[0074] In this step, the first dental model data obtained in step S100 can be input into a pre-trained neural network model. It can be imagined that when any model is input into a neural network, it can be regarded as inputting the position information of several vertices and the connection relationships between the vertices. In the method for obtaining a dental model provided in this specification, all models are constructed by using three adjacent vertices to form a patch. Therefore, when the connection method between points is fixed, it can be regarded as inputting a point cloud set into the neural network model.

[0075] In the method for obtaining a dental model provided in this specification, the role of the neural network model is to predict the state of the dental model after grinding off the attachment area in the dental model according to the input point cloud set, and the output is the predicted point cloud data of the dental model without the attachment.

[0076] Additionally, in practical applications, one point that needs to be considered is that for different teeth, the number of vertices in the first dental model data obtained during oral scanning is different. However, for the neural network model, when the number of vertices in the input point cloud set is fixed, it is more convenient to adjust the parameters of the model, and a neural network model with better prediction effect can also be trained. Therefore, after obtaining the first dental model data, a point cloud set of a fixed size can be extracted first, and the extracted point cloud set can be used as the input of the neural network model.

[0077] Specifically, second point cloud data and third point cloud data can be obtained according to the first dental model data; the data volumes of the second point cloud data and the third point cloud data are different; feature extraction is performed on the second point cloud data to obtain second point cloud feature data; feature extraction is performed on the third point cloud data to obtain third point cloud feature data; the second point cloud feature data and the third point cloud feature data are input into the prediction subnet of the neural network model to obtain predicted point cloud data.

[0078] Taking the acquisition of the second point cloud data as an example, when obtaining point cloud data in other dimensions, specifically, the first candidate point cloud data corresponding to the candidate vertices of the first dental model data can be determined; according to the distance information between the first candidate point cloud data and the first dental model data, second candidate point cloud data is determined in the first dental model data; according to the first candidate point cloud data and the second candidate point cloud data, the second point cloud data is determined.

[0079] In practical applications, one point that needs to be considered is that when the model has multi-scale inputs, the model can extract features with different focuses. Therefore, when constructing the original point cloud data based on the first dental model data each time, multiple original point cloud data with different sizes can be constructed, that is, multiple original point cloud data with different numbers of vertices, namely the second point cloud data and the third point cloud data. The original point cloud data are used as inputs together and are input into the neural network model in parallel, so that the neural network model can extract point cloud features at multiple different scales and obtain better output results. Correspondingly, the neural network model can be a model that can receive multi-scale inputs, such as neural network models like Point Fractal Network (PF-Net), etc., and this specification does not make specific limitations on this. Among them, the number of the determined original point cloud data and the number of vertices included in each original point cloud data can be preset according to specific requirements.

[0080] The original point cloud data can be constructed in various ways, and this specification provides an embodiment here for reference. Specifically, for each original point cloud data, candidate vertices can be selected from the vertices of the first dental model data to create this original point cloud data; for each vertex in the first dental model data that is not in this original point cloud data, determine the minimum distance from this vertex to this original point cloud data; among the vertices not in this original point cloud data, add the vertex with the largest minimum distance to this original point cloud data until the number of vertices included in this original point cloud data is not less than the preset number of vertices of this original point cloud data.

[0081] A possible implementation method is that the difference between several original point cloud data determined from one first dental model data lies in the different numbers of vertices included. Therefore, the same method can be used to construct each original point cloud data. When constructing an original point cloud data, initially, this original point cloud data is empty and does not contain any vertices. First, a vertex can be randomly selected from all the vertices of the first dental model data and added to this original point cloud data. Subsequently, among the remaining all vertices, select the vertex with the largest minimum distance to this original point cloud data and add it to this original point cloud data, and repeat this process until the number of vertices included in this original point cloud data is not less than the preset number of vertices of this original point cloud data. Thus, the original point cloud data representing the first dental model data can be obtained.

[0082] Among them, the minimum distance from a vertex not in the original point cloud data to the original point cloud data is the minimum value among the distances between this vertex and each vertex in the original point cloud data. Specifically, when determining the minimum distance from a vertex to the original point cloud data, the distances between this vertex and each vertex in the original point cloud data can be determined; then, the minimum distance among the determined distances is selected as the minimum distance from this vertex to the original point cloud data.

[0083] For example, when there is only one vertex A in the original point cloud data, the minimum distance from another vertex to the original point cloud data is the distance between this vertex and vertex A in the original point cloud data. Thus, vertex B, which has the largest distance among other vertices in the first tooth model data from the vertices in the original point cloud data, can be added to the original point cloud data. At this time, there are two vertices in the original point cloud data, namely vertex A and vertex B. Then, the minimum distance from a vertex not in the original point cloud data to the original point cloud data is the smaller value between the distance from this vertex to vertex A and the distance from this vertex to vertex B. Among all the vertices not in the original point cloud data, vertex C, which has the largest minimum distance to the original point cloud data, is selected and added to the original point cloud data. Repeat the above steps until the number of vertices in the original point cloud data meets the preset number of vertices.

[0084] Optionally, after the construction of the original point cloud data is completed, in order to simplify the workload of the neural network model when calculating data, the original point cloud data can be normalized. Specifically, the center point of the original point cloud data can be determined according to the positions of the vertices in the original point cloud data; and then, the original point cloud data is normalized according to this center point.

[0085] Among them, the center point of the original point cloud data can be the average of the position coordinates of the vertices in the original point cloud data. After determining the center point of the original point cloud data, all points in the original point cloud data can be translated so that the center point of the original point cloud data is located at the origin of the coordinate system. Subsequently, the vertex with the largest distance from the center point in the original point cloud data can be determined, and this distance between this vertex and the center point is used as the normalization coefficient to normalize all vertices. In other words, this distance between this vertex and the center point is used as the divisor, and the distances between all vertices and the center point are divided by this divisor to obtain the distances between the normalized vertices and the center point, and the position coordinates of each vertex are adjusted accordingly.

[0086] S104: Determine a first region of the first tooth model data according to the predicted point cloud data; the first region includes the region corresponding to the first attachment.

[0087] Based on the predicted point cloud data determined in step S102, the first region in the first tooth model data can be further determined in this step. The first region is the region in the first tooth model data that includes the first attachment.

[0088] When determining the first region, specifically, the first patch of the first tooth model data can be obtained; the first patch is at least one patch in the first tooth model data; according to the distance information between the first patch and the corresponding patches of the predicted point cloud data, it is determined whether the first patch belongs to the first region, where each patch is composed of three adjacent data points in the point cloud data.

[0089] Among them, when judging whether the first patch belongs to the first region, specifically, the position information of the first patch can be determined; according to the position information of the first patch and the position information of the corresponding patches of the predicted point cloud data, the shortest distance information of the first patch is determined; when the shortest distance information meets the first threshold condition, it is determined that the first patch does not belong to the first region.

[0090] In this method, although the first tooth model data and the predicted point cloud data are two different point cloud sets, both are point cloud sets related to the first tooth, they are in the same space, and are measured using the same coordinate system. In other words, the relative position between them is the real relative position. Therefore, it is possible to judge whether the first patch belongs to the first region according to the distance information between the patches.

[0091] In this method, the first region is the region where the attachment contained in the first tooth model data is located, that is, the patches in the first region represent the patches of the attachment in the first tooth model data. As described in the above steps, the first tooth model data is a tooth model containing an attachment, and the predicted point cloud data is a tooth model without an attachment. Based on this, it can be understood that the difference set between the first tooth model data and the predicted point cloud data should be the point cloud data of the first region where the attachment is located.

[0092] Therefore, it is possible to judge whether the first patch in the first tooth model data belongs to the first region according to the distance information. Specifically, the second patch in the predicted point cloud data can be determined according to the position information of the first patch and the position information of the corresponding patches of the predicted point cloud data, and the distance between the second patch and the first patch meets the first condition; when the distance between the first patch and the second patch meets the second threshold condition, it is determined that the first patch does not belong to the first region. Among them, the position information of a patch can be the center point coordinate information of the patch.

[0093] Among them, the distance between the second patch and the first patch satisfies the first condition. It can be that the second patch is the patch with the shortest distance from the first patch among all the patches of the predicted point cloud data; the distance between the first patch and the second patch satisfies the second threshold condition, which can be that the distance between the first patch and the second patch is not greater than the specified distance.

[0094] By using the above method, the patch with the shortest distance, that is, the highest similarity, to the first patch can be found in the predicted point cloud data for the first patch. When the distance between the two is short enough, that is, not greater than the specified distance, it can be considered that their positions in physical space overlap. At this time, the first patch and the second patch can be regarded as the same patch in different point cloud sets. Since there are no attachments in the predicted point cloud set, for any first patch in the first tooth model data, as long as there is a patch in the predicted point cloud set whose distance from it is not greater than the specified distance, it can be considered that the first patch does not belong to the attachment area patch, that is, does not belong to the first area. Conversely, for the first patch, when there is no second patch in the predicted point cloud set whose distance from it is not greater than the specified distance, it can be considered that the first patch is a patch representing an attachment and belongs to the first area.

[0095] Thus, all the patches in the first tooth model data that belong to the first area can be determined, and then the first area can be determined.

[0096] S106: Generate second tooth model data according to the first area, the first tooth model data, and the predicted point cloud data; the second tooth model data is used to indicate the first tooth with the first attachment removed.

[0097] In this step, the second tooth model data after grinding the attachment can be constructed according to the first tooth model data obtained in step S100, the predicted point cloud data obtained in step S102, and the first area obtained in step S104. Specifically, the first point cloud data can be determined according to the first area, and the first point cloud data is the other point cloud data in the first tooth model data except the first area; the first point cloud data and the predicted point cloud data are fused to generate the second tooth model data.

[0098] When constructing the complete second tooth model data after grinding the attachment, it is necessary to first remove the attachment part in the first tooth model data, and then fuse the predicted point cloud data output by the neural network model, that is, the tooth model without the attachment, into the first tooth model data.

[0099] Preferably, in practical applications, each data point in the first tooth model data is directly collected from real teeth through methods such as three-dimensional oral scanning, etc., while the predicted point cloud data is obtained by predicting through a neural network model. Generally, the accuracy of the positions of each point in the first tooth model data must be higher than that of the predicted point cloud data. Therefore, during the fusion process, for the area in the first tooth model data that has nothing to do with the attachment, this part of the content can be directly retained without using the content in the predicted point cloud data, further improving the accuracy of the reconstructed second tooth model data.

[0100] Based on the above idea, the output of the predicted point cloud data can be changed so that the predicted point cloud data only outputs the tooth surface area after removing the attachment in the part of the first tooth with the attachment.

[0101] As Figure 2 and Figure 3 shown, Figure 2 is a schematic diagram of the predicted point cloud data output by the neural network model according to a first tooth model data; Figure 3 is a schematic diagram after restoring the output of the neural network model to the first tooth model data. For ease of understanding, Figure 2 and Figure 3 only show a simplified view of the point cloud set. The vertices are not drawn in Figure 2 and Figure 3 both.

[0102] In Figure 2 , the right arc protrusion of the first tooth model data is the attachment. The part circled by the dashed ellipse in the figure is the attachment area, that is, the area around the attachment. After inputting the first tooth model data into the neural network model, the neural network model will output the point cloud set of the tooth surface area after the attachment in the attachment area is ground off. In Figure 2 , since the observation angle is the side of this part of the tooth surface, a straight line is observed; when observing from the front of this tooth surface, a plane can actually be observed. Figure 3 shows a schematic diagram of restoring the point cloud set of the attachment area output by the neural network model to the tooth model after grinding off the attachment. It should be noted that Figure 3 is only a schematic diagram given in this specification for ease of understanding the model output. In practical applications, there is no Figure 3 shown restoration step.

[0103] At this time, the method of removing the attachment part in the first tooth model data is to isolate the attachment in the first tooth model data by comparing the first tooth model data with the predicted point cloud data, and achieve the purpose of removing the attachment by deleting the free patches in the first tooth model data.

[0104] Specifically, as mentioned in step S100 of this specification, in the method for obtaining a dental model provided in this specification, in all dental models and point cloud sets, the connection relationship of each vertex is that three adjacent vertices form a patch. Therefore, the patch can be used as the smallest unit to perform the operation of removing attachments in the first dental model data. During the output process of the neural network model, the size and coordinate system of the dental model will not be changed. Therefore, the first dental model data and the predicted point cloud data output can be compared in the same coordinate system.

[0105] For each patch in the attachment area of the first dental model data, the minimum distance from the patch to the predicted point cloud data can be determined. When the minimum distance is not greater than the specified distance, the patch can be deleted from the first dental model data. Thus, the attachment part in the first dental model data can be isolated, and the patches of the attachment part become free patches. Subsequently, the free patches in the first dental model data can be deleted, that is, the attachment part in the first dental model data can be deleted. Finally, by fusing the first dental model data after removing the attachments with the predicted point cloud data, the second dental model data after grinding the attachments can be obtained.

[0106] Among them, the minimum distance from a patch in the first dental model data to the predicted point cloud data can be the minimum value among the distances between the patch and all patches in the predicted point cloud data. Specifically, when determining the minimum distance from a patch in the first dental model data to the predicted point cloud data, the center points of each patch in the first dental model data can be determined according to the vertices forming each patch in the first dental model data, and the center points of each patch in the predicted point cloud data can be determined according to the vertices forming each patch in the predicted point cloud data; for each patch in the attachment area of the first dental model data, determine the distance between the center point of the patch and the center point of each patch in the predicted point cloud data; select the minimum distance among the determined distances as the minimum distance from the patch to the predicted point cloud data.

[0107] For any patch, the average of the position coordinates of the three vertices forming the patch can be used as the center point of the patch. Thus, the distance between any two patches can be determined as the distance between the center points of the two patches. And the minimum distance from a patch in the attachment area of the first dental model data to the predicted point cloud data can be the minimum value among the distances between the center point of the patch and the center points of each patch in the predicted point cloud data.

[0108] Figure 4 Shows the process of constructing the second dental model data after removing attachments based on the first dental model data and the predicted point cloud data. As Figure 4As shown, first, the first tooth model data and the predicted point cloud data are placed in the same coordinate system. Subsequently, in process A, in the attachment area of the first tooth model data, each patch with a minimum distance from the predicted point cloud data not greater than the specified distance is deleted, resulting in the first tooth model data with the attachment isolated; subsequently, in process B, the floating patches in the first tooth model data can be deleted, that is, the attachment part is deleted, resulting in the first tooth model data after removing the attachment; subsequently, in process C, the first tooth model data and the predicted point cloud data can be fused to obtain the second tooth model data after grinding the attachment.

[0109] More preferably, after obtaining the second tooth model data, the area near the suture of the first tooth model data and the predicted point cloud data in the second tooth model data can be smoothed to obtain a more natural second tooth model data.

[0110] When predicting the second tooth model data after grinding the attachment of the first tooth model data by using the method for obtaining a tooth model provided in this specification, the predicted point cloud data of the attachment area after grinding the attachment can be obtained from the first tooth model data through a neural network model, and the attachment part in the first tooth model data can be removed by comparing it with the predicted point cloud data. Finally, the first tooth model data after removing the attachment and the predicted point cloud data are fused to obtain the second tooth model data. Compared with the traditional method, this method does not require any manual operations during implementation, reducing the inaccuracy caused by errors in manual operations. At the same time, by fusing the predicted point cloud data with the first tooth model data to obtain the second tooth model data, not only can the shape of the tooth model after removing the attachment be restored more accurately, but also the original information of the tooth model can be retained to the greatest extent.

[0111] Additionally, in combination with the actual operation of the user during medical use, the first tooth model data and the second tooth model data constructed in this method can be displayed according to requirements. Specifically, in response to receiving a display instruction, the first tooth model data and / or the second tooth model data can be displayed according to the content of the display instruction.

[0112] Further, after observing the first dental model data, the user can, according to the needs, make the execution subject further display the second dental model data by using the function of removing attachments. Specifically, before displaying the second dental model data, the first attachment in the first dental model data can be removed in response to receiving a removal instruction. Similarly, in this method, the determination of the first area can also be performed according to the user's needs. Specifically, the first area of the first dental model data can be updated in response to receiving an update instruction; the second dental model data can be updated according to the updated first area, the first dental model data, and the predicted point cloud data; the second dental model data is used to indicate the first tooth with the first attachment removed.

[0113] Additionally, the neural network model adopted in the method for obtaining a dental model provided in this specification can be pre-trained. Specifically, during the attachment grinding surgery, according to the state of the tooth before the surgery, a sample dental model can be obtained, and according to the state of the tooth after the surgery, a label can be obtained, where the label is the point cloud set of the area with the attachment on the tooth after the attachment is ground off; the sample dental model is input into the neural network model to be trained, and the predicted point cloud data output by the neural network model is determined; taking the minimum difference between the predicted point cloud data and the label as the optimization goal, the neural network model is trained.

[0114] Among them, the sample dental model is the three-dimensional meshed model of the tooth that needs to undergo attachment grinding surgery before the surgery; the label is the point cloud set corresponding to the original attachment area of the tooth that needs to undergo attachment grinding surgery after the surgery. It is worth mentioning that for the sample dental model, several point cloud sets can also be constructed according to the method given in this specification to replace the sample dental model as the input of the neural network model during training, which will not be elaborated in this specification.

[0115] A possible implementation manner is also provided below.

[0116] 1. Point cloud processing of the triangular mesh model

[0117] The initial data targeted by this application is a single triangular-meshed tooth, and we need to perform some pre-processing to obtain data that can be used as the input to the network.

[0118] The steps are as follows:

[0119] A. Obtain all vertex coordinates (X, Y, Z) from the three-dimensional dental model M, and record the sampling point set as P.

[0120] B. Farthest Point Sampling (FPS): Set the maximum number of sampled point clouds as N. Randomly select a vertex pt1 and add it to the sampling set P. Calculate the distances D1 from all vertices to pt1. Select the vertex farthest from pt1 as pt2 and add it to the set P. Calculate the distances D2 from all points not in P to pt2. Combine D1 and D2, and select the minimum distance from each non-P point to the points in P, denoted as the distance D from non-P to P.

[0121] C. During the iterative process of B: Select the point with the farthest distance D from non-P, update D, and add it to P. Continue this process until the size of the point set P reaches N.

[0122] D. Perform distance normalization on the points in P: P(X, Y, Z)_ = P(X, Y, Z) -

[0123] Mean(P(X, Y, Z)), move the center of the point set P(X, Y, Z) to (0, 0, 0);

[0124] Calculate the distances from all points in P(X, Y, Z)_ to (0, 0, 0), and obtain the maximum distance D MAX ;

[0125] P(X, Y, Z)_processed = P(X, Y, Z)_ / D MAX .

[0126] After processing steps A, B, C, and D, the dataset P before inputting into the network is obtained input . Since the network has a multi-scale input-output structure, in actual operation, according to the multi-scale requirements, input By setting different Ns, perform steps B and C multiple times to obtain P input (N1, N2, N3) sets.

[0127] 2. Special training method of the neural network for the problem of attachment grinding

[0128] A common problem addressed in a possible neural network is to predict the shape of the missing part of an object from the remaining part of its shape, that is, the input data is the incomplete shape point cloud, and the output data is the missing part point cloud. In this application, the input data is the complete tooth point cloud with attachments, and the output data is the tooth point cloud in the attachment area after grinding off the attachments. An inertial solution is to predict the point cloud of the attachment part and then remove the predicted point cloud to form the input-output pattern applicable in the neural network. However, the actual result is that the missing part point cloud generated in this way will have a large deviation from the actual point cloud, especially when the amount of data is small. The advantages of this application are as follows: 1. The original shape pattern of the attachment is fixed, but when predicting the attachment point cloud and creating the missing part, the shape of the missing part is various. In this case, it is better for the neural network to select the area by itself than to assist in selecting the area; 2. In most cases, there is no need to specifically predict the attachment area. As long as the overlapping part of the calculated generated missing point cloud and the input point cloud in a certain direction is calculated, the attachment area can be inferred, and the effective area overlap is higher than 90%; 3. It makes better use of the information of the data and expands the application field of the neural network model.

[0129] 3. Fusion and reconstruction of triangular mesh for predicted point cloud and defective point cloud

[0130] This stage mainly removes the attachment part on the three-dimensional mesh and fuses and reconstructs the predicted point cloud with the three-dimensional mesh to generate the shape after grinding off the attachment. The specific steps are as follows:

[0131] A. Calculate the central point coordinates Center(X, Y, Z) of the triangular patches of the three-dimensional mesh model, and calculate

[0132] the shortest distance from Center(X, Y, Z) to the point cloud set Output(X, Y, Z) generated by the network

[0133] D nearest 。

[0134] B. When D nearest is less than a certain threshold Thr, remove the corresponding triangular patches on the tooth model and delete the free triangular patches in the tooth model. Obtain the tooth model M after removing the attachment crop 。

[0135] C. Triangulate the Output point cloud and suture it with M crop to form the final tooth model after grinding

[0136] M new 。

[0137] D. Smooth the 2-3 neighborhood near the suture area of M new

[0138] ​After the above steps A, B, C, and D, the final triangular mesh model M of the abraded tooth is obtained. new , where to obtain a more robust and accurate attachment area, operation B can also incorporate a method for predicting the attachment area and comprehensively evaluate the final target area.

[0139] 4. Neural network structure and parameter settings

[0140] The training method of the neural network can refer to existing training methods. The required predicted point cloud is finely generated in a multi-scale manner, and the prediction accuracy is optimized through adversarial loss and distance loss. A possible set of training parameters and optimizers, etc., can be: input data scales: 2048×3, 1024×3, 512×3; output scales: 256×3, 128×3, 64×3; batch size: 24; Adam optimizer; learning rate = 0.0001, etc.

[0141] Correspondingly, this specification also provides a method for displaying a tooth model, including:

[0142] Displaying second tooth model data, where the second tooth model data is obtained by the method for obtaining a tooth model provided in this specification. Additionally, this method can also display first tooth model data. Specifically, in response to a first display instruction input by the user, the first tooth model data can be displayed; or, in response to a second display instruction input by the user, the second tooth model data can be displayed.

[0143] The first display instruction and the second display instruction can be generated based on the user's operations. For example, when the user opens the corresponding software to indicate the display of the first tooth model, or when the user opens the corresponding function in the corresponding patient case, and the function can be determined based on actual needs and is not limited here. The manner of the display instruction can also refer to the display instruction or update instruction in the above embodiments.

[0144] The above is the method for obtaining a tooth model provided in this specification. Based on the same idea, this specification also provides a corresponding device for obtaining a tooth model, as Figure 5 shown.

[0145] Figure 5 A schematic diagram of a device for obtaining a tooth model provided in this specification specifically includes:

[0146] An acquisition module 200, configured to acquire first tooth model data of a patient's first tooth; the first tooth includes at least one attachment;

[0147] The prediction module 202 is configured to predict the first dental model data based on a neural network model to obtain predicted point cloud data; the predicted point cloud data is dental model data predicted by the neural network model without the first attachment, and the first attachment is an attachment on the first tooth.

[0148] The determination module 204 is configured to determine a first region of the first dental model data according to the predicted point cloud data; the first region includes the region corresponding to the first attachment.

[0149] The generation module 206 is configured to generate second dental model data according to the first region, the first dental model data, and the predicted point cloud data; the second dental model data is used to indicate the first tooth with the first attachment removed.

[0150] Optionally, the determination module 204 is specifically configured to obtain a first patch of the first dental model data; the first patch is at least one patch in the first dental model data; determine whether the first patch belongs to the first region according to the distance information between the first patch and the corresponding patches of the predicted point cloud data, where each patch is composed of three adjacent data points in the point cloud data.

[0151] Optionally, the determination module 204 is specifically configured to determine the position information of the first patch; determine the shortest distance information of the first patch according to the position information of the first patch and the position information of the corresponding patches of the predicted point cloud data; when the shortest distance information meets the first threshold condition, determine that the first patch does not belong to the first region.

[0152] Optionally, the determination module 204 is specifically configured to determine a second patch in the predicted point cloud data according to the position information of the first patch and the position information of the corresponding patches of the predicted point cloud data, and the distance between the second patch and the first patch meets the first condition; when the distance between the first patch and the second patch meets the second threshold condition, determine that the first patch does not belong to the first region.

[0153] Optionally, the position information includes the center point coordinate information of the patch.

[0154] Optionally, the generation module 206 is specifically configured to determine first point cloud data according to the first region, where the first point cloud data is other point cloud data in the first dental model data except the first region; fuse the first point cloud data and the predicted point cloud data to generate second dental model data.

[0155] Optionally, the prediction module 202 is specifically configured to obtain second point cloud data and third point cloud data according to the first tooth model data; the data volumes of the second point cloud data and the third point cloud data are different; extract features from the second point cloud data to obtain second point cloud feature data; extract features from the third point cloud data to obtain third point cloud feature data; and input the second point cloud feature data and the third point cloud feature data into a prediction subnet of a neural network model to obtain predicted point cloud data.

[0156] Optionally, the prediction module 202 is specifically configured to determine first candidate point cloud data corresponding to candidate vertices of the first tooth model data; determine second candidate point cloud data in the first tooth model data according to distance information between the first candidate point cloud data and the first tooth model data; and determine second point cloud data according to the first candidate point cloud data and the second candidate point cloud data.

[0157] Optionally, the apparatus further includes a first response module 208, which is specifically configured to, in response to receiving a display instruction, display the first tooth model data and / or the second tooth model data according to the content of the display instruction.

[0158] Optionally, the apparatus further includes a second response module 210, which is specifically configured to, in response to receiving a removal instruction, remove a first attachment from the first tooth model data.

[0159] Optionally, the apparatus further includes a third response module 212, which is specifically configured to, in response to receiving an update instruction, update a first area of the first tooth model data; update the second tooth model data according to the updated first area, the first tooth model data, and the predicted point cloud data; and the second tooth model data is used to indicate the first tooth with the first attachment removed.

[0160] This specification also provides a computer-readable storage medium, which stores a computer program that can be used to execute the method for obtaining a tooth model described above. Figure 1 provided.

[0161] This specification also provides Figure 6 a schematic structural diagram of the electronic device shown. As Figure 6 described, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, other hardware required for other services may also be included. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above Figure 1The method for obtaining a dental model and the display method described above. Of course, in addition to the software implementation, this specification does not exclude other implementation manners, such as a logic device or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, and may also be hardware or a logic device.

[0162] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0163] The memory may include non-permanent memory in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0164] The memory may also include program tools (or utilities) having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment.

[0165] The processor executes various functional applications and data processing by running computer programs stored in the memory, such as the methods in the above embodiments.

[0166] The electronic device may also communicate with one or more external devices. Such communication may be performed through an input / output (I / O) interface. Moreover, the electronic device for model generation may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter. The network adapter communicates with other modules of the electronic device through a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in combination with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0167] The input and / or output device may include a scanning device, a camera interface, input devices (such as a mouse, a keyboard, etc.), a display device (such as a monitor), a printer, and / or one or more other input devices. The input / output interface may receive executable instructions and / or data, which may be stored in a data storage device (such as a memory). For example, it may be used to receive the generated midfacial line and store the generated midfacial line.

[0168] In some embodiments, the scanning device may be configured to scan one or more physical dental models of a patient's dentition. In one or more embodiments, the scanning device may be configured to directly scan a patient's dentition and / or dental appliance. The scanning device may be configured to input data into a computing device.

[0169] In some embodiments, the camera interface may receive input from an imaging device (e.g., a 2D or 3D imaging device), such as a digital camera, a printed photo scanner, and / or other suitable imaging devices. For example, the input from the imaging device may be stored in a memory.

[0170] The processor may execute instructions to provide a display of a dental model, a dental orthodontic position, a treatment plan, etc. on a display. For example, the computing device may be configured to allow a treatment professional or other user to input treatment goals. The received input may be sent as data to the processor and / or may be stored in a memory.

[0171] Such connectivity may allow for the input and / or output of data and / or instructions and other types of information. Some embodiments may be distributed among various computing devices within one or more networks and are used for collecting, computing, and / or analyzing any of the methods mentioned in the present application.

[0172] It should be noted that although several units / modules or sub-units / modules of an electronic device are mentioned in the above detailed description, such a division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more of the above-mentioned units / modules may be embodied in one unit / modules. Conversely, the features and functions of one unit / modules described above may be further divided and embodied by multiple units / modules.

[0173] This embodiment provides a computer-readable storage medium having a computer program stored thereon, and the computer program, when executed by a processor, implements the method in the above embodiment.

[0174] Among them, the more specific forms that the readable storage medium may adopt may include, but are not limited to: portable disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories, optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0175] In a possible implementation manner, the present application may also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps in the above embodiment.

[0176] Among them, the program code for executing the present application can be written in any combination of one or more programming languages, and the program code can be executed entirely on the user device, partially on the user device, executed as an independent software package, partially on the user device and partially on a remote device, or entirely on a remote device.

[0177] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other, and the key point of each embodiment is to illustrate the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the relevant parts, reference can be made to the partial description of the method embodiment.

[0178] The above description is only for the embodiments of this specification and is not intended to limit this specification. For those skilled in the art, various changes and modifications can be made to this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included within the scope of the claims of this application.

Claims

1. A method for obtaining a dental model, characterized in that, Comprising: Obtaining first tooth model data of a patient's first tooth; The first tooth includes at least one attachment; Based on a neural network model, predicting the first tooth model data to obtain predicted point cloud data; the predicted point cloud data is tooth model data predicted by the neural network model without the first attachment, and the first attachment is the attachment on the first tooth; Determining a first region of the first tooth model data according to the predicted point cloud data; The first region includes the region corresponding to the first attachment; Generating second tooth model data according to the first region, the first tooth model data, and the predicted point cloud data; The second tooth model data is used to indicate the first tooth with the first attachment removed.

2. The method according to claim 1, wherein Determining the first region of the first tooth model data according to the predicted point cloud data specifically includes: Obtaining a first patch of the first tooth model data; the first patch is at least one patch in the first tooth model data; Determining whether the first patch belongs to the first region according to the distance information between the first patch and the corresponding patches of the predicted point cloud data, wherein each patch is composed of three adjacent data points in the point cloud data.

3. The method according to claim 2, characterized in that, Determining whether the first patch belongs to the first region according to the distance information between the first patch and the patches of the predicted point cloud data specifically includes: Determining the position information of the first patch; Determining the shortest distance information of the first patch according to the position information of the first patch and the position information of the corresponding patches of the predicted point cloud data; When the shortest distance information satisfies the first threshold condition, determining that the first patch does not belong to the first region.

4. The method according to claim 3, wherein When the shortest distance information satisfies the first threshold condition, determining that the first patch does not belong to the first region specifically includes: Determining a second patch in the predicted point cloud data according to the position information of the first patch and the position information of the corresponding patches of the predicted point cloud data, and the distance between the second patch and the first patch satisfies the first condition; When the distance between the first patch and the second patch satisfies the second threshold condition, determining that the first patch does not belong to the first region.

5. The method according to any one of claims 3, characterized in that, The position information includes the central point coordinate information of the patch.

6. The method according to claim 1, characterized in that, Generating second tooth model data according to the first region, the first tooth model data, and the predicted point cloud data specifically includes: Determining first point cloud data according to the first region, and the first point cloud data is other point cloud data in the first tooth model data except the first region; Fusing the first point cloud data and the predicted point cloud data to generate second tooth model data.

7. The method according to claim 1, characterized in that, Predicting the first tooth model data based on a neural network model includes: Obtaining second point cloud data and third point cloud data according to the first tooth model data; the data volumes of the second point cloud data and the third point cloud data are different; Performing feature extraction on the second point cloud data to obtain second point cloud feature data; Performing feature extraction on the third point cloud data to obtain third point cloud feature data; Input the second point cloud feature data and the third point cloud feature data into the prediction subnet of the neural network model to obtain predicted point cloud data.

8. The method according to claim 7, wherein Obtain second point cloud data based on the first tooth model data, including: Determine first candidate point cloud data corresponding to candidate vertices of the first tooth model data; Determine second candidate point cloud data in the first tooth model data according to the distance information between the first candidate point cloud data and the first tooth model data; Determine second point cloud data according to the first candidate point cloud data and the second candidate point cloud data.

9. The method according to claim 1, characterized in that, The method further includes: In response to receiving a display instruction, display the first tooth model data and / or the second tooth model data according to the content of the display instruction.

10. The method according to claim 9, characterized in that, Before displaying the second tooth model data, the method further includes: In response to receiving a removal instruction, remove the first attachment in the first tooth model data.

11. The method according to claim 1, characterized in that, The method further includes: In response to receiving an update instruction, update the first region of the first tooth model data; Update the second tooth model data according to the updated first region, the first tooth model data, and the predicted point cloud data; the second tooth model data is used to indicate the first tooth with the first attachment removed.

12. A display method for a dental model, characterized in that, including: Display the second tooth model data, which is obtained by the method for obtaining a tooth model according to any one of claims 1-11.

13. The method according to claim 12, characterized in that, The method further includes: Display the first tooth model data.

14. The method according to claim 12 or 13, characterized in that The method further includes: In response to a first display instruction input by the user, display the first tooth model data; Alternatively, in response to a second display instruction input by the user, display the second tooth model data.

15. An apparatus for obtaining a dental model, characterized in that, including: An acquisition module for acquiring first tooth model data of a patient's first tooth; The first tooth includes at least one attachment; A prediction module for predicting the first tooth model data based on a neural network model to obtain predicted point cloud data; the predicted point cloud data is tooth model data predicted by the neural network model without the first attachment, and the first attachment is an attachment on the first tooth; A determination module for determining a first region of the first tooth model data according to the predicted point cloud data; The first region includes the region corresponding to the first attachment; A generation module for generating second tooth model data according to the first region, the first tooth model data, and the predicted point cloud data; The second tooth model data is used to indicate the first tooth with the first attachment removed.

16. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1-14 above is implemented.

17. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, the method according to any one of claims 1-14 above is implemented.

Images