Three-dimensional model-based data processing method and apparatus, device and medium

Abstract

The embodiment of the invention relates to a three-dimensional model-based data processing method and apparatus, a device and a medium. The method comprises: obtaining a three-dimensional teeth model of a target oral cavity; and according to the three-dimensional teeth model, determining the tooth number corresponding to each tooth category in the target oral cavity. Therefore, the robustness and efficiency of tooth number determination are improved.

Description

Data processing method, device, equipment and medium based on three-dimensional model This disclosure claims the priority of the Chinese patent application filed with the China Patent Office on June 21, 2023, with application number 2023107389382 and invention name “Data processing method, device, equipment and medium based on three-dimensional model”, all contents of which are incorporated by reference in this disclosure. Technical Field The present disclosure relates to the field of three-dimensional digitization, and in particular to a data processing method, device, equipment and medium based on a three-dimensional model. Background Art Digital intraoral impressions are widely used in orthodontics due to their efficiency and safety. Orthodontists need to identify and mark the tooth positions of digital dental impression data in order to develop a treatment plan. This task poses great challenges due to the large differences in the shapes of different teeth, disordered and / or missing teeth in some patients, and the incompleteness of digital tooth models. In the related art, the tooth position number is manually marked directly on the digital impression. However, this marking method not only requires a lot of time and effort, but also has high professional and technical requirements for the operator, and is prone to omissions and misjudgments, resulting in a low accuracy rate in determining the tooth position number. Summary of the invention In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a data processing method, device, equipment and medium based on a three-dimensional model. The disclosed embodiment provides a data processing method based on a three-dimensional model, the method comprising the following steps: obtaining a three-dimensional model of teeth in a target oral cavity; and determining a tooth position number corresponding to each tooth category in the target oral cavity according to the three-dimensional model of teeth. The disclosed embodiment also provides a data processing device based on a three-dimensional model, including: an acquisition module for acquiring a three-dimensional model of teeth in a target oral cavity; and a determination module for determining a tooth position number corresponding to each tooth category in the target oral cavity according to the three-dimensional model of teeth. An embodiment of the present disclosure also provides an electronic device, which includes: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the three-dimensional model-based data processing method provided in the embodiment of the present disclosure. The present disclosure also provides a computer-readable storage medium having a computer program stored therein, and the computer program is used to execute the three-dimensional based The data processing method of the model. Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages: The tooth position number determination scheme provided in the embodiment of the present disclosure obtains a three-dimensional tooth model of the target oral cavity, and determines the tooth position number corresponding to each tooth category in the target oral cavity based on the three-dimensional tooth model. Thus, the tooth position number is accurately determined without relying on a priori detection results such as instance segmentation or target detection. The tooth position number determination is more efficient and more robust. It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. FIG1 is a schematic flow chart of a data processing method based on a three-dimensional model provided in an embodiment of the present disclosure; FIG2 is a schematic diagram of a three-dimensional tooth model provided in an embodiment of the present disclosure; FIG3 is a schematic diagram of a two-dimensional tooth image corresponding to multiple viewing angles provided by an embodiment of the present disclosure; FIG4 is a schematic diagram of a tooth category area provided by an embodiment of the present disclosure; FIG5 is a schematic diagram of a connection between a grid fixed point and a facet provided by an embodiment of the present disclosure; FIG6 is a schematic diagram of a scene of a tooth position number marking point provided by an embodiment of the present disclosure; FIG7 is a schematic diagram of sample data labeling provided by an embodiment of the present disclosure; FIG8 is a schematic diagram of the structure of a data processing device based on a three-dimensional model provided by an embodiment of the present disclosure; FIG. 9 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure. It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect. The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units. It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more". The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information. In order to solve the above problems, the present disclosure provides a method for determining tooth color. The method is introduced below in conjunction with a specific embodiment. FIG1 is a flow chart of a data processing method based on a three-dimensional model provided by an embodiment of the present disclosure. The method can be executed by a data processing device based on a three-dimensional model, wherein the device can be implemented by software and / or hardware and can generally be integrated in an electronic device. As shown in FIG1 , the method includes: Step 101, obtaining a three-dimensional model of teeth in a target oral cavity. Among them, the target oral cavity can be scanned by a preset three-dimensional data scanning device to obtain three-dimensional data of teeth. The preset three-dimensional data scanning device can be an intraoral scanner with high precision, etc. Among them, the intraoral scanner uses an invasive optical scanning head to directly scan the patient's oral cavity, and uses the structured light triangulation imaging principle to project active light patterns to the teeth, gums, mucosa, etc. in the oral cavity using a digital projection system. After the camera acquisition system obtains the pattern, it performs three-dimensional reconstruction and splicing through algorithm processing to obtain three-dimensional data of teeth. In addition, the above-mentioned acquired three-dimensional tooth model includes multiple mesh facets composed of multiple mesh vertices, wherein the mesh facets can generally be triangular facets composed of three mesh vertices, that is, from the partial three-dimensional tooth model shown in Figure 2, it can be seen that the triangular mesh facets constitute the corresponding three-dimensional tooth model. Step 102: determine the tooth position number corresponding to each tooth type in the target oral cavity according to the three-dimensional tooth model. In one embodiment of the present disclosure, after the three-dimensional tooth model is acquired, the tooth position number corresponding to each tooth type in the target oral cavity is determined according to the three-dimensional tooth model. It should be noted that in different application scenarios, the target There are different ways to mark the tooth position number corresponding to each tooth type in the mouth, as shown below: In some possible embodiments, the tooth three-dimensional model can be input into a pre-trained tooth position number determination model, which is a pre-trained model that can determine the tooth position number corresponding to each tooth category based on the tooth three-dimensional model. Furthermore, the tooth position number can be determined based on the output of the tooth position number determination model. In some possible embodiments, the tooth position number corresponding to each tooth category can be determined according to the tooth three-dimensional model according to an automatic recognition algorithm. In actual implementation, the automatic recognition algorithm in this embodiment is different in different scenarios. As a possible implementation method, the algorithm recognition process of the automatic recognition algorithm includes: obtaining multiple two-dimensional tooth images corresponding to multiple perspectives of the tooth three-dimensional model, wherein the tooth three-dimensional model contains multiple mesh patches composed of multiple mesh vertices. In this example, after obtaining the three-dimensional tooth model, the target detection network or instance segmentation network in the traditional technology is not used to identify a single tooth, and then the tooth position number information is inferred based on the number of single teeth identified. Instead, considering that the three-dimensional tooth model is three-dimensional data, both the target detection grid and the instance segmentation network will take a long time and be easily affected by the three-dimensional space noise data. The accuracy of the algorithm will also directly affect the accuracy of the subsequent tooth position number recognition. In this example, there is no need to obtain prior data such as the number of teeth based on algorithm processing such as instance segmentation or target detection. Instead, multiple two-dimensional tooth images corresponding to multiple perspectives of the three-dimensional tooth model are obtained. Each two-dimensional tooth image corresponds to a different perspective. Therefore, the subsequent confirmation of information such as the tooth position number is based on the two-dimensional tooth image. For example, as shown in FIG3 , four two-dimensional tooth images corresponding to the three-dimensional tooth model under four viewing angles may be determined, and the viewing angles of the four two-dimensional tooth images are different. Furthermore, multiple tooth category areas contained in each two-dimensional tooth image and a first tooth position number probability set for each pixel point in the multiple tooth category areas are identified, wherein the first tooth position number probability set includes multiple first tooth position number probabilities corresponding to each pixel point and multiple candidate tooth position numbers. In one embodiment of the present disclosure, multiple tooth category areas contained in each two-dimensional tooth image are identified, wherein each tooth category area corresponds to an area where the same tooth is located, each tooth category area belongs to an area where the same tooth position number is located, and different tooth category areas correspond to areas where different tooth position numbers are located. Continuing with the scene shown in FIG3 as an example, when different tooth category areas are marked with different colors, as shown in FIG4, the pixels belonging to the same tooth in the four two-dimensional tooth images obviously belong to the same tooth category area. For example, FIG4 above contains 12 single teeth, and 12 tooth category areas can be obtained after identification. It should be emphasized that in this embodiment, only tooth category areas are identified, and the tooth position number corresponding to each tooth category area is not clear at this time. In this embodiment, a first tooth position number probability set is determined for each pixel point in a plurality of tooth category areas, wherein the first tooth position number probability set includes a plurality of first tooth position number probabilities corresponding to each pixel point in the tooth category area and a plurality of candidate tooth position numbers. For example, when the target oral cavity corresponds to a single jaw including 16 single tooth categories, the plurality of candidate tooth position numbers correspond to 16 tooth position numbers. For a pixel point A belonging to the tooth category, its corresponding first tooth position number probability set includes the probabilities that A belongs to each of the 16 tooth position numbers. In this example, after the first tooth position number probability set is determined, the tooth position number corresponding to each tooth category in the target oral cavity is determined according to the first tooth position number probability set. In some possible implementations, the point with the largest probability value of the first tooth position number in each tooth category can be used as the tooth position number marking point of the corresponding tooth category, and the tooth number corresponding to the tooth position number marking point is determined to be the tooth position number of the corresponding tooth category. For example, the tooth category area of ​​tooth category z contains 10,000 pixels, among which the pixel point corresponding to the highest probability of the first tooth position number is W, then the tooth number 1 corresponding to W is determined to be the tooth number corresponding to tooth category z. In some possible implementations, a second tooth number probability set is determined for each mesh surface based on the first tooth number probability set, wherein the second tooth number probability set includes multiple second tooth number probabilities corresponding to each mesh surface and multiple candidate tooth numbers. The tooth position number recognition result in the two-dimensional tooth image is reflected in the tooth three-dimensional model. Therefore, it not only ensures the recognition accuracy of the tooth position number, but also avoids the related target detection and instance segmentation directly on the tooth three-dimensional model. In this embodiment, a second tooth number probability set is determined for each mesh patch based on the first tooth number probability set, wherein the second tooth number probability set includes multiple second tooth number probabilities corresponding to each mesh patch and multiple candidate tooth numbers. For example, when the target oral cavity corresponds to a single jaw including 16 single tooth categories, then the multiple candidate tooth numbers correspond to 16 tooth numbers. For mesh patch B, its corresponding second tooth number probability includes the probabilities that B belongs to the 16 tooth numbers respectively. It should be noted that in different application scenarios, the manner of determining the second tooth position number probability set of each mesh face according to the first tooth position number probability set is different, and an example is described as follows: In some possible embodiments, at least one associated pixel point corresponding to each mesh patch is determined among the pixel points corresponding to a plurality of tooth category regions. For example, the coordinate transformation relationship between each two-dimensional tooth image and the three-dimensional tooth model can be determined in advance, and then the corresponding two-dimensional coordinate point of any grid point in each grid patch in each two-dimensional tooth image can be determined according to the predetermined coordinate transformation relationship. If the corresponding two-dimensional coordinate point is found, the pixel point of the corresponding two-dimensional coordinate point is determined as the associated pixel point. For example, at least one viewpoint of the corresponding mesh surface can be determined from multiple viewpoints. The method comprises the steps of: measuring the viewing angle, determining at least one two-dimensional tooth image corresponding to at least one observation viewing angle as at least one associated two-dimensional tooth picture, and then determining the associated pixel points corresponding to each grid surface in each associated two-dimensional tooth picture to obtain at least one associated pixel point corresponding to each grid surface. Among them, in some possible implementation methods, the coordinate transformation relationship between each two-dimensional tooth image and the tooth three-dimensional model is predetermined, and then, each two-dimensional tooth image is mapped to the tooth three-dimensional model according to the predetermined coordinate transformation relationship, and the pixel points belonging to each grid surface in each associated two-dimensional tooth image are determined as associated pixel points according to the mapped three-dimensional tooth model. Among them, when there are multiple pixel points belonging to each grid surface in each associated two-dimensional tooth image, one of the pixel points can be randomly determined as the associated pixel point, and when a grid surface only corresponds to a single pixel point in the associated two-dimensional tooth image, the single pixel point can be determined as the associated pixel point of the corresponding grid surface. The coordinate transformation relationship is used to achieve the mapping between the original tooth image and the tooth three-dimensional data. It should be noted that in different application scenarios, the method of determining the coordinate transformation relationship is different, and the examples are as follows: In some possible embodiments, multiple reference pixel points are obtained in a two-dimensional tooth image, and multiple reference three-dimensional points that match the multiple reference pixel points are determined in a three-dimensional tooth model, wherein the reference pixel point and the corresponding reference three-dimensional point belong to the same point on the tooth, and the matching method of the multiple reference pixel points and the reference three-dimensional point can be implemented according to a feature matching algorithm, etc. Furthermore, the two-dimensional coordinates of each reference pixel point and the three-dimensional coordinates of the corresponding reference three-dimensional point are obtained, and the coordinate transformation relationship is determined according to the two-dimensional coordinates and the three-dimensional coordinates. Based on the coordinate transformation relationship, the three-dimensional coordinate position of the two-dimensional pixel point in the two-dimensional tooth image in the three-dimensional tooth model can be determined. In some possible embodiments, the intrinsic parameters and extrinsic parameters of the camera used to take the two-dimensional tooth image may also be obtained, and the coordinate transformation relationship may be determined based on the intrinsic parameters and extrinsic parameters of the camera and the triangle theorem. Of course, in the actual implementation process, the scale invariant feature transformation matching algorithm (Scale Invariant Feature Transform, SIFT), the local feature point detection algorithm (SpeededUpRobustFeature, SURF), the image feature description algorithm (Binary Robust Invariant Scalable Keypoints, BRISK), etc. can also be used to determine the coordinate transformation relationship, etc. Furthermore, after determining at least one associated pixel point corresponding to each mesh patch among the pixel points corresponding to the multiple tooth category areas, a first probability mean of at least one first tooth number probability corresponding to each candidate tooth number is calculated according to at least one first tooth number probability set corresponding to the at least one associated pixel point, and the first probability mean is used as the second tooth number probability of each mesh patch belonging to the corresponding candidate tooth number, so as to obtain the second tooth number of each mesh patch. Tooth position number probability set. For example, as shown in Table 1 below, for mesh patch C, if its corresponding observation perspectives are perspective 1, perspective 2, perspective 3 and perspective 4, then the associated pixel point in the associated two-dimensional tooth image corresponding to perspective 1 is pixel c1, the associated pixel point in the associated two-dimensional tooth image corresponding to perspective 2 is pixel c2, the associated pixel point in the associated two-dimensional tooth image corresponding to perspective 3 is pixel c3, and the associated pixel point in the associated two-dimensional tooth image corresponding to perspective 4 is pixel c4, wherein the first tooth position number probability set of c1 is the first column in Table 1, the first tooth position number probability set of c2 is the second column in Table 1, the first tooth position number probability set of c3 is the third column in Table 1, and the first tooth position number probability set of c4 is the fourth column in Table 1, then the first probability mean corresponding to candidate tooth position number 1 is calculated as: c1 belongs to the first tooth position number probability corresponding to candidate tooth position number 0, c2 The probability of the first tooth position number corresponding to candidate tooth position number 0, c3 belongs to the probability of the first tooth position number corresponding to candidate tooth position number 0, and c4 belongs to the mean value face[0] of the probability of the first tooth position number corresponding to candidate tooth position number 0. The first probability means corresponding to candidate tooth position number 1 are: c1 belongs to the probability of the first tooth position number corresponding to candidate tooth position number 1, c2 belongs to the probability of the first tooth position number corresponding to candidate tooth position number 1, c3 belongs to the probability of the first tooth position number corresponding to candidate tooth position number 1, and c4 belongs to the mean value face[1] of the probability of the first tooth position number corresponding to candidate tooth position number 15… The first probability means corresponding to candidate tooth position number 15 are: c1 belongs to the probability of the first tooth position number corresponding to candidate tooth position number 15, c2 belongs to the probability of the first tooth position number corresponding to candidate tooth position number 15, c3 belongs to the probability of the first tooth position number corresponding to candidate tooth position number 15, and c4 belongs to the mean value face of the probability of the first tooth position number corresponding to candidate tooth position number 15

[0015] Thus, the 16 second tooth number probabilities contained in the second tooth number probability value set corresponding to C are: face[0]=0…face[4]=(0.06+0.11) / 4=0.043; face[5]=(0.93+0.83+0.75) / 4=0.628; face[6]=(0.01+0.06+0.25) / 4=0.08…face

[0015] =0. Table 1 After determining the second tooth position number probability set, a third tooth position number probability set of each mesh vertex is determined according to the second tooth position number probability set, wherein the third tooth position number probability set includes multiple third tooth position number probabilities corresponding to each mesh vertex and multiple candidate tooth positions. In one embodiment of the present disclosure, after obtaining the second tooth position number probability set, the second tooth position number probability corresponding to the second tooth position number set is further migrated to each mesh vertex to obtain a third tooth position number probability set for each mesh vertex, wherein the third tooth position number probability set includes multiple third tooth position number probabilities corresponding to each mesh vertex and multiple candidate tooth positions. In this embodiment, multiple associated mesh patches connected to each mesh vertex can be determined, and then, based on the multiple second tooth position number probability sets corresponding to the multiple associated mesh patches, the second probability mean of the multiple second tooth position number probabilities corresponding to each candidate tooth position number is calculated, and the second probability mean is used as the third tooth position number probability that each mesh vertex belongs to the corresponding candidate tooth position number to obtain the third tooth position number probability set of the mesh vertex. For example, as shown in FIG5 , for mesh vertex V595, its five associated facets F1000, F1001, F1002, F1003, and F1004 are connected. Referring to Table 2 below, the first to fifth columns in Table 2 are the second tooth position number probability sets corresponding to F1000, F1001, F1002, F1003, and F1004, respectively. The second probability mean corresponding to candidate tooth position number 0 is calculated as follows: F1000 belongs to the second tooth position number probability corresponding to candidate tooth position number 0, F1001 belongs to the second tooth position number probability corresponding to candidate tooth position number 0, and F1002 belongs to the second tooth position number probability corresponding to candidate tooth position number 0. , the probability that F1003 belongs to the second tooth number corresponding to candidate tooth number 0, and the mean value vertex[0] of the probability that F1004 belongs to the second tooth number corresponding to candidate tooth number 0...the mean value of the second probability corresponding to candidate tooth number 15 is: the probability that F1000 belongs to the second tooth number corresponding to candidate tooth number 15, the probability that F1001 belongs to the second tooth number corresponding to candidate tooth number 15, the probability that F1002 belongs to the second tooth number corresponding to candidate tooth number 15, the probability that F1003 belongs to the second tooth number corresponding to candidate tooth number 15, and the mean value vertex

[0015] of the probability that F1004 belongs to the second tooth number corresponding to candidate tooth number 15. Thus, the 16 third tooth position number probabilities contained in the third tooth position number probability value set corresponding to V595 are: vertex[0]=0…vertex[3]=0.713 / 5=0.143; vertex[4]=(0.043+0.113+0.023+0.123) / 5=0.06; vertex[5]=(0.628+0.723+0.623+0.523) / 5=0.499; vertex[6]=(0.008+0.001+0.021+0.011) / 5=0.008; vertex[7]=0.123 / 5=0.025; vertex[8]=(0.175+0.06+0.05+0.01+0.02) / 5=0.063... vertex

[0015] =(0.008+0.001+0.021+0.011) / 5=0. Table 2 After obtaining the third tooth position number probability set, the tooth position number corresponding to each tooth category in the three-dimensional tooth model is determined according to the third tooth position number probability set. In one embodiment of the present disclosure, after obtaining the third tooth position number probability set, the probability value of the corresponding mesh vertex belonging to multiple candidate tooth position numbers is obtained, so that the tooth position number marking point corresponding to each tooth category in the three-dimensional tooth model is determined according to the third tooth position number probability set, and the tooth position number corresponding to the tooth position number marking point is used as the tooth position number of the corresponding tooth category. Thus, the tooth position number can be obtained by fusing the tooth position number probability of the pixel point corresponding to the two-dimensional tooth image with the tooth position number probability of the mesh patch, which has higher calculation efficiency and does not need to rely on the prior results of target detection and instance segmentation of the three-dimensional tooth model, thereby improving the robustness of tooth position number determination. Among them, in some possible embodiments, the maximum reference third tooth position number probability in the third tooth position number probability set in each grid vertex corresponding to each tooth category area can be determined. For example, in the N grid vertices (where N is a natural number) contained in the tooth category area 1, in the N third tooth position number probability sets corresponding to the N grid vertices, the maximum value in each third tooth position number probability set is determined as the maximum reference third tooth position number probability to obtain N maximum reference third tooth position number probabilities, and then, the tooth position number corresponding to the tooth category area 1 is determined according to the N maximum reference third tooth position number probabilities. For example, among the N maximum reference third tooth position number probabilities, m maximum reference third tooth position number probabilities are all tooth number 5, where m is less than or equal to N, and m The ratio to N is greater than the preset ratio threshold, which ensures that the maximum reference third tooth position number probability of the grid fixed points contained in most grid tooth category areas 1 corresponds to tooth number 5. Therefore, among all the maximum reference third tooth position number probabilities corresponding to the tooth number in each tooth category area, the maximum target third tooth position number probability is determined, that is, the maximum value is determined as the maximum target third tooth position number probability among the m maximum reference third tooth position number probabilities of tooth number 5, and then, the grid vertex corresponding to the maximum target third tooth position number probability is determined to be the tooth number corresponding to each tooth category area, that is, the grid fixed point corresponding to the maximum value determined among the m maximum reference third tooth position number probabilities is determined as the tooth number marking point, and the tooth number corresponding to the tooth number marking point is tooth No. 5. Furthermore, after determining the tooth position number marking point, relevant scenarios can be applied based on the tooth position number marking point. For example, in some possible application scenarios, the interference caused by some anisotropic shapes or buccal miscellaneous data can be denoised based on the tooth position number marking point determined above. In this embodiment, it is determined whether multiple tooth position number marking points to be screened belonging to the same tooth position number are included. If multiple tooth position number marking points to be screened belonging to the same tooth position number are included, the marking distance between each tooth position number marking point to be screened and the tooth position number marking points of other adjacent tooth positions is determined, and the tooth position number marking points to be screened whose marking distance is less than the preset distance threshold are deleted. That is, considering that the distance between different tooth position numbers should not be too close, the misjudged tooth position number marking point should be compared with the actual tooth position number marking point. The marking distance between the actual tooth position number and the tooth position number marking points of other adjacent tooth position numbers should be larger than the marking distance between the misjudged tooth position number marking points. Therefore, in the embodiment of the present disclosure, some tooth position marking points that may be misdetermined due to the above-mentioned interference can be deleted according to the distance between the tooth position number marking points of different tooth position numbers. For example, as shown in Figure 6, if the tooth position number marking points of the determined target oral cavity include two tooth position marking points P51 and P52 to be screened for tooth position number No. 5, where P52 may be buccal flesh, etc., and the other tooth positions adjacent to tooth position number No. 5 are tooth position numbers No. 4 and No. 6, respectively, where the tooth position number marking point of tooth position number No. 4 is P4, and the tooth position number marking point of tooth position number No. 6 is P6, the marking distances between P51 and P4 and P5 are L1 and L2, respectively, and the marking distances between P52 and P4 and P5 are L3 and L4, respectively (the specific marking distances are not marked in the figure), where, based on the above marking distances, it is found that the marking distance L3 between P52 and P4 is less than the preset distance threshold, and thus P52 is deleted. In other possible application scenarios, after the tooth position number is determined, other scenario requirements can also be met based on the tooth position number. For example, it can also be determined based on the missing tooth position number to identify missing teeth. Therefore, in the above embodiment, a three-dimensional tooth model of the target oral cavity is obtained, and multiple two-dimensional tooth images corresponding to multiple perspectives of the three-dimensional tooth model are obtained, wherein the three-dimensional tooth model contains multiple mesh patches composed of multiple mesh vertices, and multiple tooth category areas contained in each two-dimensional tooth image and a first tooth position number probability set for each pixel point in the multiple tooth category areas are identified, wherein the first tooth position number probability set contains multiple first tooth position number probabilities corresponding to each pixel point and multiple candidate tooth position numbers, and then, according to the first tooth position number probability set, a second tooth position number probability set is determined for each mesh patch, wherein the second tooth position number probability set contains multiple second tooth position number probabilities corresponding to each mesh patch and multiple candidate tooth position numbers, and a third tooth position number probability set of each mesh vertex is determined according to the second tooth position number probability set, wherein the third tooth position number probability set contains multiple third tooth position number probabilities corresponding to each mesh vertex and multiple candidate tooth position numbers, and the tooth position number corresponding to each tooth category in the three-dimensional tooth model is determined according to the third tooth position number probability set. Therefore, based on the fusion of the tooth position number probabilities of mesh patches and mesh vertices, the tooth position number can be accurately determined without relying on prior detection results such as instance segmentation or target detection. The tooth position number determination is more efficient and more robust. In summary, the data processing method based on the three-dimensional model of the embodiment of the present disclosure obtains the three-dimensional model of the teeth of the target oral cavity, and then determines the tooth position number corresponding to each tooth type in the target oral cavity according to the three-dimensional model of the teeth, thereby improving the robustness and efficiency of the determination of the tooth position number. Based on the above embodiment, in the embodiment of determining the tooth position number corresponding to each tooth category in the target oral cavity based on the first tooth position number probability set, multiple tooth category areas contained in each two-dimensional tooth image and the first tooth position number probability set of each pixel point in the tooth category area can be identified based on a pre-trained deep learning model, that is, each two-dimensional tooth image is input into a pre-trained deep learning model, and multiple tooth category areas output by the deep learning model and the first tooth position number probability set of each pixel point in the multiple tooth category areas are obtained. The training process of the deep learning model is described below in conjunction with the embodiment: In one embodiment of the present disclosure, a sample tooth three-dimensional model is obtained, and multiple initial sample two-dimensional tooth images corresponding to multiple viewing angles of the tooth three-dimensional model are obtained, each tooth category area in each initial sample two-dimensional tooth image is determined, and a sample tooth position number marking point is determined in the central area of ​​each tooth category area, that is, the sample tooth position number marking point is usually located in the central area of ​​the tooth category, so that the sample tooth position number area determined subsequently belongs to the inside of the tooth category area. Alternatively, as shown in FIG. 7, the central point of the tooth category area can be directly determined as the sample tooth position number marking point, etc. It should be emphasized that in this training process, the single point of the sample tooth position number mark is not used as the training sample data. Considering that the loss value of a single sample point in training is difficult to back-propagate, in the actual training process, after determining the sample tooth position number mark point, the sample tooth position number mark point is used as the center point to determine the sample tooth position number area of ​​each tooth category in each initial sample two-dimensional tooth image, where the sample tooth position number area belongs to the corresponding tooth category area, that is, continue to refer to Figure 7, diffuse the sample tooth position number mark point to the tooth category area, and train the deep learning model with the area as the granularity. Since the loss value of the regional granularity is easy to back-propagate, the difficulty of training the deep learning model is reduced, and the accuracy of the deep learning model obtained by training is improved, so that in actual application, the tooth position number mark point and some pixels nearby have the largest response, and the corresponding tooth number can be located more easily. In addition, in the present embodiment, it is also necessary to emphasize that when determining the sample tooth number area of ​​each tooth category in each initial sample two-dimensional tooth image with the sample tooth number marking point as the center point, it is not necessary to identify the contour of each tooth and determine the entire tooth area as the sample tooth number area. Instead, it is only necessary to ensure that the sample tooth number area belongs to the corresponding tooth category area. That is, continuing to refer to Figure 7, the sample tooth number area can be a partial area belonging to the tooth category area, thereby avoiding learning the noise data at the edge position of the tooth area, improving the model training efficiency, and helping to further ensure the accuracy of the trained deep learning model. Among them, in different application scenarios, the method of determining the sample tooth position number area of ​​each tooth category in each initial sample two-dimensional tooth image with the sample tooth position number mark point as the center point is Different, in some possible embodiments, the sample tooth position number marking point can be used as the center point, and the cluster scattered outward based on the Gaussian distribution can be used as the sample tooth position number area, wherein the method of determining the sample tooth position number area based on the Gaussian distribution and the sample tooth position number marking point can be implemented by the existing technology and will not be elaborated here. In some possible embodiments, the sample tooth position number marking point can be used as the center point, and a circular area with a preset radius distance from the center point can be determined as the sample tooth position number area, wherein the preset radius distance can be calibrated according to the actual scene, etc. Furthermore, a marked sample two-dimensional tooth image is obtained in the sample tooth number area corresponding to the tooth category area, and model parameters of a pre-constructed deep learning model are trained according to the initial sample two-dimensional tooth image and the marked sample two-dimensional tooth image to obtain a trained deep learning model, that is, the initial sample two-dimensional tooth image is input into the initial deep learning model, and the tooth number area corresponding to the tooth number output by the model is obtained, and the loss value between the output tooth number area and the sample tooth number area in the corresponding marked sample is calculated. If the loss value is greater than a preset loss threshold, the relevant loss value in the initial deep learning model is modified until the loss value is less than the preset loss threshold, and it is determined that the deep learning model training is completed. In summary, the data processing method based on the three-dimensional model of the embodiment of the present invention uses partial tooth areas belonging to the corresponding teeth as training data when the deep learning model is trained, thereby improving the accuracy of the deep learning model and ensuring the accuracy of identifying multiple tooth category areas contained in each two-dimensional tooth image based on the pre-trained deep learning model, as well as the first tooth position number probability set of each pixel point in the multiple tooth category areas. In order to implement the above embodiments, the present disclosure also proposes a data processing device based on a three-dimensional model. FIG8 is a schematic diagram of a data processing device based on a three-dimensional model provided by an embodiment of the present disclosure. The device can be implemented by software and / or hardware and can generally be integrated into an electronic device. As shown in FIG8 , the device includes: an acquisition module 810 and a determination module 820, wherein: An acquisition module 810 is used to acquire a three-dimensional model of teeth in a target oral cavity; The determination module 820 is used to determine the tooth position number corresponding to each tooth type in the target oral cavity according to the three-dimensional tooth model. The three-dimensional model-based data processing device provided in the embodiment of the present disclosure can execute the three-dimensional model-based data processing method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method, which will not be repeated here. In order to implement the above embodiments, the present disclosure further proposes a computer program product, including a computer program / instruction, which implements the method for determining tooth color in the above embodiments when executed by a processor. FIG. 9 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. 9, which shows a schematic diagram of the structure of an electronic device 900 suitable for implementing the embodiment of the present disclosure. The electronic device 900 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG9 is only an example and should not bring any limitation to the functions and scope of use of the embodiment of the present disclosure. As shown in Figure 9, the electronic device 900 may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a memory 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processor 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904. Typically, the following devices may be connected to the I / O interface 905: input devices 906 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 908 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 909. The communication device 909 may allow the electronic device 900 to communicate wirelessly or wired with other devices to exchange data. Although FIG. 9 shows an electronic device 900 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively. In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 909, or installed from the memory 908, or installed from the ROM 902. When the computer program is executed by the processor 901, the above functions defined in the method for obtaining the prepared tooth model of the embodiment of the present disclosure are executed. It should be noted that the computer-readable medium of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a A computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, device, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above. In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network. The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the relevant method steps in the above embodiments. The electronic device may be written in one or more programming languages ​​or a combination thereof to write computer program code for performing the operations of the present disclosure, including but not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). The flowcharts and block diagrams in the accompanying drawings illustrate systems, The possible architecture, function and operation of method and computer program product.In this regard, each square frame in flow chart or block diagram can represent a part of a module, program segment or code, and this module, program segment or code part comprises one or more executable instructions for realizing the logical function of regulation.It should also be noted that in some implementations as replacement, the function marked in the square frame can also occur in a sequence different from that marked in the accompanying drawings.For example, two square frames represented in succession can actually be executed substantially in parallel, and they can also be executed in the opposite order sometimes, depending on the function involved.It should also be noted that each square frame in the block diagram and / or flow chart, and the combination of the square frames in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or operation of regulation, or can be realized by a combination of special hardware and computer instructions. The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself. The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like. In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution. Furthermore, although the operations are depicted in a particular order, this should not be construed as requiring that These operations may be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of a separate embodiment may also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination. Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. The three-dimensional model-based data processing method provided in the embodiment of the present disclosure can accurately determine the tooth position number without relying on a priori detection results such as instance segmentation or target detection. The tooth position number is more efficient and robust in determination and has strong industrial applicability.

Claims

1. A data processing method based on a three-dimensional model, characterized in that: The following steps are involved: Obtaining a three-dimensional model of teeth in a target oral cavity; The tooth position number corresponding to each tooth type in the target oral cavity is determined according to the tooth three-dimensional model.

2. The method according to claim 1, characterized in that Determining the tooth position number corresponding to each tooth type in the target oral cavity according to the tooth three-dimensional model includes: Acquire a plurality of two-dimensional tooth images corresponding to a plurality of viewing angles of the three-dimensional tooth model, wherein the three-dimensional tooth model comprises a plurality of mesh facets composed of a plurality of mesh vertices; Identify multiple tooth category areas contained in each of the two-dimensional tooth images, and a first tooth position number probability set for each pixel point in the multiple tooth category areas, wherein the first tooth position number probability set includes multiple first tooth position number probabilities corresponding to each pixel point and multiple candidate tooth position numbers; The tooth position number corresponding to each tooth type in the target oral cavity is determined according to the first tooth position number probability set.

3. The method according to claim 1 or 2, characterized in that The step of determining the tooth position number corresponding to each tooth type in the target oral cavity according to the first tooth position number probability set includes: Determine a second tooth position number probability set for each mesh surface according to the first tooth position number probability set, wherein the second tooth position number probability set includes a plurality of second tooth position number probabilities corresponding to each mesh surface and the plurality of candidate tooth position numbers; Determine a third tooth position number probability set for each mesh vertex according to the second tooth position number probability set, wherein the third tooth position number probability set includes a plurality of third tooth position number probabilities corresponding to each mesh vertex and the plurality of candidate tooth position numbers; The tooth position number corresponding to each tooth category in the three-dimensional tooth model is determined according to the third tooth position number probability set.

4. The method according to any one of claims 1 to 3, characterized in that: The identifying of the multiple tooth category regions contained in each of the two-dimensional tooth images and the first tooth position number probability set of each pixel point in the tooth category region includes: Each of the two-dimensional tooth images is input into a pre-trained deep learning model to obtain a plurality of tooth category areas output by the deep learning model, and a first tooth position number probability set for each pixel point in the plurality of tooth category areas.

5. The method according to any one of claims 1 to 4, characterized in that: Before inputting each of the two-dimensional tooth images into the pre-trained deep learning model, the method further includes: Acquire a sample tooth three-dimensional model, and acquire a plurality of initial sample two-dimensional tooth images corresponding to a plurality of viewing angles of the tooth three-dimensional model; Determine each tooth category area in each of the initial sample two-dimensional tooth images, and determine a sample tooth position number marking point in the central area of ​​each tooth category area; Determine a sample tooth position number region of each tooth category in each of the initial sample two-dimensional tooth images with the sample tooth position number mark point as the center point, wherein the sample tooth position number region belongs to a corresponding tooth category region; Acquire a marked sample two-dimensional tooth image according to the sample tooth position number area corresponding to each tooth category area; The model parameters of the pre-constructed deep learning model are trained according to the initial sample two-dimensional tooth image and the labeled sample two-dimensional tooth image to obtain a trained deep learning model.

6. The method according to any one of claims 1 to 5, characterized in that: Determining a second tooth position number probability set for each mesh face according to the first tooth position number probability set includes: Determine at least one associated pixel point corresponding to each mesh surface patch among the pixel points corresponding to the plurality of tooth category regions; Calculate a first probability mean of at least one first tooth position number probability corresponding to each candidate tooth position number according to at least one first tooth position number probability set corresponding to at least one associated pixel point; The first probability mean is used as the second tooth position number probability that each mesh surface belongs to the corresponding candidate tooth position number, so as to obtain a second tooth position number probability set for each mesh surface.

7. The method according to any one of claims 1 to 6, characterized in that: The step of determining at least one associated pixel point corresponding to each mesh patch among the pixel points corresponding to the plurality of tooth category regions comprises: Determine at least one observation viewing angle from among the multiple viewing angles at which each mesh surface is observed, and determine at least one two-dimensional tooth image corresponding to the at least one observation viewing angle as at least one associated two-dimensional tooth picture; In each of the associated two-dimensional tooth images, associated pixel points corresponding to each of the mesh surfaces are determined to obtain at least one associated pixel point corresponding to each of the mesh surfaces.

8. The method according to any one of claims 1 to 7, characterized in that: The step of determining at least one associated pixel point corresponding to each mesh patch among the pixel points corresponding to the plurality of tooth category regions comprises: Mapping the plurality of two-dimensional tooth images into the three-dimensional tooth model according to a predetermined coordinate transformation relationship; According to the mapped three-dimensional tooth model, the pixel points belonging to each grid surface in each of the associated two-dimensional tooth images are determined as the associated pixel points.

9. The method according to any one of claims 1 to 8, characterized in that: The determining of a third tooth position number probability set of each mesh vertex according to the second tooth position number probability set includes: Determining a plurality of associated mesh patches connected to each mesh vertex; Calculate a second probability mean of the plurality of second tooth position number probabilities corresponding to each of the candidate tooth positions according to the plurality of second tooth position number probability sets corresponding to the plurality of associated mesh facets; The second probability mean is used as the third tooth position number probability that each mesh vertex belongs to the corresponding candidate tooth position number, so as to obtain a third tooth position number probability set of the mesh vertices.

10. The method according to any one of claims 1 to 9, characterized in that: The step of determining the tooth position number corresponding to each tooth category in the three-dimensional tooth model according to the third tooth position number probability set includes: Determine the maximum reference third tooth position number probability in the third tooth position number probability set in each mesh vertex corresponding to each of the tooth category areas; Determine the tooth position number corresponding to each of the tooth category areas according to all maximum reference third tooth position number probabilities corresponding to all of the mesh vertices corresponding to each of the tooth category areas; Determine a target third tooth position number probability among all maximum reference third tooth position number probabilities of the tooth position numbers corresponding to each of the tooth category areas; The grid vertex corresponding to the maximum target third tooth position number probability is determined as a tooth position number marking point, and the tooth position number corresponding to the tooth position number marking point is determined to be the tooth position number corresponding to each of the tooth category areas.

11. The method according to any one of claims 1 to 10, characterized in that: Before determining that the tooth position number corresponding to the tooth position number mark point is the tooth position number corresponding to each tooth category area, the method further includes: Determine whether multiple tooth position number marking points to be screened that belong to the same tooth position number are included; If there are multiple tooth position number marking points to be screened that belong to the same tooth position number, determine the marking distance between each tooth position number marking point to be screened and the tooth position number marking points of other adjacent tooth positions; Delete the tooth position number marking points to be screened whose marking distance is less than the preset distance threshold.

12. A data processing device based on a three-dimensional model, characterized in that: The device comprises: An acquisition module, used for acquiring a three-dimensional model of teeth in a target oral cavity; A determination module is used to determine each tooth in the target oral cavity according to the three-dimensional tooth model. The tooth number corresponding to the tooth type.

13. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the three-dimensional model-based data processing method described in any one of claims 1-11 above.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to execute the three-dimensional model-based data processing method described in any one of claims 1 to 11.

Images