Three-dimensional digital model acquisition method and apparatus, and device and storage medium

Abstract

A three-dimensional digital model acquisition method and apparatus, and a device and a storage medium. The method comprises: acquiring two-dimensional view images of a three-dimensional mesh model (S110); using a region classification model to perform classification processing on the two-dimensional view images, so as to obtain region categories of the two-dimensional view images (S120); and performing back-projecting, onto the three-dimensional mesh model, the two-dimensional view images that include the region categories, and determining a plurality of region categories of the three-dimensional mesh model, so as to obtain a three-dimensional digital model partitioned according to the region categories (S130). In this way, a region classification model is used to determine a plurality of region categories from two-dimensional views of a three-dimensional mesh model, and region categories of the three-dimensional mesh model are determined in a back-projection manner, and such a manner is not interfered with by factors such as the diversity of local regions in the three-dimensional mesh model and the quality of a mesh curved surface, thereby implementing fully automatic and high-precision classification of the three-dimensional mesh model, and providing a good foundation for subsequent dental restoration.

Description

Method, device, equipment and storage medium for obtaining three-dimensional digital model

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on December 15, 2023, with application number 202311737403.X, and invention name “Method, device, equipment and storage medium for obtaining three-dimensional digital models”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present disclosure relates to the field of oral digital technology, and in particular to a method, device, equipment and storage medium for obtaining a three-dimensional digital model. Background Art

[0003] In real-time intraoral scanning application scenarios, it is necessary to identify different area categories (such as normal tooth type, prepared tooth type, and gum type) from the scanned three-dimensional mesh model, so that users can obtain detailed information (such as cervical margin line, resting jaw position) from mesh areas of different area categories, so that users can perform tooth restoration based on the detailed information of the mesh area.

[0004] Therefore, proposing a high-precision and automated method for obtaining three-dimensional digital models is a technical problem that needs to be solved urgently in order to provide a good foundation for the subsequent tooth restoration process. Summary of the Invention

[0005] In order to solve the above technical problems, the present disclosure provides a method, device, equipment and storage medium for obtaining a three-dimensional digital model.

[0006] In a first aspect, the present disclosure provides a method for obtaining a three-dimensional digital model, the method comprising:

[0007] Acquire multiple two-dimensional view images of a three-dimensional mesh model;

[0008] Using a region classification model, classifying the two-dimensional view image to obtain a region category of the two-dimensional view image;

[0009] The two-dimensional view image containing the region category is back-projected onto the three-dimensional grid model, the grid face category of the three-dimensional grid model is determined, and a three-dimensional digital model having partitions according to the grid face category is obtained.

[0010] In a second aspect, the present disclosure provides a device for acquiring a three-dimensional digital model, the device comprising:

[0011] an acquisition module configured to acquire a two-dimensional view image of the three-dimensional mesh model;

[0012] a classification module configured to classify the two-dimensional view image using a region classification model to obtain a region category of the two-dimensional view image;

[0013] The determination module is configured to back-project the two-dimensional view image containing the region category onto the three-dimensional mesh model, determine the mesh face category of the three-dimensional mesh model, and obtain a three-dimensional digital model partitioned by mesh face category.

[0014] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0015] processor;

[0016] a memory configured to store executable instructions,

[0017] The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the method provided in the first aspect.

[0018] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor implements the method provided in the first aspect.

[0019] The technical solution provided by the embodiments of the present disclosure has the following advantages over the prior art:

[0020] The disclosed embodiments provide a method, apparatus, device, and storage medium for acquiring a three-dimensional digital model. The method acquires multiple two-dimensional view images of a three-dimensional mesh model; utilizes a regional classification model to classify the two-dimensional view images to obtain regional categories of the two-dimensional view images; and back-projects the two-dimensional view images containing the regional categories onto the three-dimensional mesh model to determine the regional categories of the three-dimensional mesh model, thereby obtaining a three-dimensional digital model partitioned by regional categories. Thus, the regional classification model is utilized to determine the regional categories from the two-dimensional views of the three-dimensional mesh model, and the regional categories of the three-dimensional mesh model are determined by back-projection. This method is not affected by factors such as the diversity of local regions in the three-dimensional mesh model and the quality of the mesh surface, thereby achieving fully automatic and high-precision classification of the three-dimensional mesh model, providing a good foundation for the subsequent tooth restoration process. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0022] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] FIG1 is a flow chart of a method for acquiring a three-dimensional digital model provided in an embodiment of the present disclosure.

[0024] FIG2 a is a schematic diagram of a two-dimensional view image corresponding to a first viewing angle provided by an embodiment of the present disclosure.

[0025] FIG2 b is a schematic diagram of a two-dimensional view image corresponding to a second viewing angle provided by an embodiment of the present disclosure.

[0026] FIG3 a is a schematic diagram of a classification result of a two-dimensional view image corresponding to a first perspective provided by an embodiment of the present disclosure.

[0027] FIG3 b is a schematic diagram of a classification result of a two-dimensional view image corresponding to a second viewing angle provided by an embodiment of the present disclosure.

[0028] FIG4 is a schematic diagram of a back-projection result of a three-dimensional mesh model provided by an embodiment of the present disclosure.

[0029] FIG5 is a schematic diagram of the process of S130 provided in an embodiment of the present disclosure.

[0030] FIG6 is a flow chart of a method for training a region classification model provided in an embodiment of the present disclosure.

[0031] FIG7 is a schematic structural diagram of a device for acquiring a three-dimensional digital model provided in an embodiment of the present disclosure.

[0032] FIG8 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0034] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0035] In the related art, manual, semi-automatic and fully automatic methods are used to directly classify the three-dimensional grid model to obtain a three-dimensional digital model with regional categories.

[0036] Manual classification of 3D mesh models requires manually identifying different area categories directly on the 3D mesh model using digital mapping software. However, this classification method requires a significant investment of time and effort, along with the support of relevant marking tools and software. This requires a high learning curve and is a significant challenge for the marking technician.

[0037] When using a semi-automatic method to classify 3D mesh models, the labeler selects several key points on the 3D mesh model and then uses a specific graph cut algorithm to determine the multiple regions of the 3D mesh model. However, this classification method requires the graph cut algorithm to be robust and places high demands on the labeler's labeling skills. It also consumes a certain amount of manual interaction and operation events.

[0038] When fully automated, 3D mesh model classification is performed, multiple regions of the model are automatically classified based on the model's differential geometric features (such as curvature and normal vectors). However, this classification method places high demands on the 3D mesh model's morphology and surface quality, and defects such as noise and surface defects can affect the accuracy of region classification.

[0039] It can be seen that the existing method for obtaining three-dimensional digital models has high requirements for labeling personnel or three-dimensional mesh models, and when classifying complex three-dimensional mesh models, the classification accuracy cannot meet the needs of subsequent tooth restoration.

[0040] In order to solve the above problems, the embodiments of the present disclosure provide a method, apparatus, device and storage medium for obtaining a three-dimensional digital model.

[0041] The following describes the method for obtaining a three-dimensional digital model provided by an embodiment of the present disclosure, with reference to Figures 1 to 6. In the embodiment of the present disclosure, the method for obtaining a three-dimensional digital model can be performed by an electronic device. The electronic device can include a device with communication capabilities, such as a tablet computer, desktop computer, or laptop computer, or a device simulated by a virtual machine or simulator.

[0042] FIG1 shows a schematic flow chart of a method for acquiring a three-dimensional digital model provided in an embodiment of the present disclosure.

[0043] As shown in FIG1 , the method for obtaining the three-dimensional digital model may include the following steps.

[0044] S110 , obtaining a two-dimensional view image of the three-dimensional mesh model.

[0045] In this embodiment, when designing a tooth restoration, the designer needs to provide a tooth restoration plan that is suitable for the user based on different area categories of the three-dimensional mesh model, so that the acquisition accuracy of the three-dimensional digital model has a significant impact on the tooth restoration design.

[0046] Among them, the three-dimensional mesh model can be understood as the digital impression data of the teeth. Compared with traditional physical models and plaster impressions, the process of obtaining digital impression data through an intraoral scanner is faster and more efficient, and there is no need to wait for the drying and curing process of the three-dimensional mesh model.

[0047] The two-dimensional view image is a two-dimensional image obtained by projecting the three-dimensional mesh model at different viewing angles.

[0048] The specific implementation of S110 includes but is not limited to the following methods: obtaining a three-dimensional mesh model of the scanned tooth; and projecting the three-dimensional mesh model at multiple viewing angles to obtain a two-dimensional view image.

[0049] Specifically, during the process of a 3D scanner scanning teeth, the electronic device reconstructs the mesh of the 3D scanning data collected by the 3D scanner in real time to generate a 3D mesh model of the scanned teeth. Then, based on the 3D morphological features of each area on the 3D mesh model, the 3D mesh model can be projected from multiple perspectives to obtain a 2D view image.

[0050] Among them, the three-dimensional morphological features can be understood as differential geometric features, which include but are not limited to features such as curvature and normal vector.

[0051] For ease of understanding, Figure 2a shows a two-dimensional view image corresponding to a first perspective, and Figure 2b shows a two-dimensional view image corresponding to a second perspective. These two two-dimensional view images may contain different three-dimensional morphological features, and the first perspective and the second perspective are not equal.

[0052] It should be noted that when projecting a 3D mesh model, only regions containing highly complex 3D features can be projected and their corresponding viewing angles determined. Regions containing less complex 3D features are not projected, thereby obtaining local region images of the 3D mesh model and enabling classification of the 3D mesh model based on these local region images. This ensures classification accuracy while reducing computational complexity.

[0053] S120 : Using the region classification model, classify the two-dimensional view image to obtain a region category of the two-dimensional view image.

[0054] In this embodiment, the two-dimensional view image is input into the region classification model, and the region classification model is used to determine the category of the pixel points of the two-dimensional view image, so that each two-dimensional view image is divided into different categories, thereby determining the region category of each two-dimensional view image.

[0055] The region classification model may include but is not limited to a U-Net model and other models, and a region classification model suitable for the scene is selected based on the three-dimensional grid model in different scenes.

[0056] The region category of the two-dimensional view image can be understood as a pixel region category. The region category of the two-dimensional view image can be one or more pre-labeled region categories.

[0057] In this embodiment, the step of determining the region category of the two-dimensional view image includes, but is not limited to: using a region classification model to calculate category probability data for each pixel region in the two-dimensional view image, wherein each pixel region corresponds to a piece of category probability data, and the category probability data includes probability values ​​of multiple region categories; for each pixel region in the two-dimensional view image, taking the region category with the largest probability value in the category probability data as the region category of the corresponding pixel region, to obtain the region category of the two-dimensional view image.

[0058] The pixel area refers to an area consisting of one pixel or multiple pixels in a two-dimensional view image.

[0059] Specifically, the region classification model performs feature processing on each pixel region in the 2D view image and, based on the feature processing results, determines the probability value of each pixel region in the 2D view image belonging to one or more region categories, thereby obtaining category probability data containing probability values ​​for multiple region categories. Next, for each pixel region, one or more probability values ​​corresponding to each pixel region are obtained from the category probability data, and the region category with the largest probability value is selected as the region category of the corresponding pixel region, thereby obtaining the region category of the 2D view image. It should be noted that the region classification model generally has one or more pre-set region categories. Each pixel region in the 2D view image is processed by the region classification model to determine the probability value of each pixel region corresponding to each pre-set region category.

[0060] Exemplarily, the preset regional categories of the regional classification model include preparation category, tooth category and gum category. There are 5 two-dimensional view images, and each two-dimensional view image has three pixel areas. The regional classification model predicts that the category probability data of pixel area a includes: preparation category probability 95%, tooth category probability 5%, and gum category probability 0%. The preparation category with a probability of 95% is used as the regional category of pixel area a; similarly, the regional classification model predicts that the category probability data of pixel area b includes: preparation category probability 0%, tooth category 85%, and gum category probability 15%. The tooth category with a probability of 85% is used as the regional category of pixel area b; similarly, the regional classification model predicts that the category probability data of pixel area c includes: preparation category probability 10%, tooth category probability 0%, and gum category probability 90%. The gum category with a probability of 90% is used as the regional category of pixel area c.

[0061] For ease of understanding, Figure 3a shows a schematic diagram of the classification results of the two-dimensional view image corresponding to the first perspective, and Figure 3b shows a schematic diagram of the classification results of the two-dimensional view image corresponding to the second perspective. As shown in Figures 3a and 3b, the area categories of the two-dimensional view image may include a preparation category 310, a tooth category 320, and a gum category 330.

[0062] S130 , back-projecting the plurality of two-dimensional view images containing region categories onto the three-dimensional grid model, determining the region categories of the three-dimensional grid model, and obtaining a three-dimensional digital model partitioned by region categories.

[0063] It can be understood that since the regional category of the two-dimensional view image is obtained after the category of the pixel points of the two-dimensional view image is determined, the regional category on the three-dimensional grid model can be determined by back-projecting the two-dimensional view image onto the three-dimensional grid model, and further a three-dimensional digital model with partitioning by regional category can be obtained.

[0064] The region classification of the 3D mesh model can be understood as the classification of mesh facets. Because noise and occlusion may occur during the process of determining the region classification of the 3D mesh model from multiple 2D view images, this can lead to incorrect segmentation. Therefore, after back-projecting the 2D view images, geometric algorithms such as graph cuts can be used to repair edge details to improve the accuracy of the 3D digital model.

[0065] Furthermore, after determining the three-dimensional digital models with regional categories, the three-dimensional digital models of the respective regional categories may be rendered and displayed.

[0066] Specifically, the electronic device can render and display the three-dimensional digital model separately according to the region type, so that users can design and make crowns, braces and other restorations based on the three-dimensional digital models of each region category.

[0067] Optionally, if the 3D digital model is an intraoral 3D digital model, the region categories on the intraoral 3D digital model include one or more of a tooth category, a gingival category, a preparation category, an implant category, an abutment category, and an inlay category. In other scenarios, the 3D digital model region categories may also include region categories corresponding to the scenario, etc.

[0068] For ease of understanding, FIG4 shows a schematic diagram of the rendering effect of a three-dimensional digital model. The three-dimensional digital model in FIG4 includes a three-dimensional mesh model corresponding to a preparation category 410 , a tooth category 420 , and a three-dimensional mesh model corresponding to a gum category 430 .

[0069] In this embodiment, after obtaining a three-dimensional digital model divided into regional categories, the user can scan the oral areas corresponding to the three-dimensional grid models of different regional categories using different point spacings, and then construct a three-dimensional grid model based on the three-dimensional data obtained by the rescan. Then, use Computer Aided Design (CAD) and Computer Aided Manufacturing (CAM) software to design and produce crowns, braces and other restorations to achieve tooth restoration for the user.

[0070] A method for acquiring a three-dimensional digital model according to an embodiment of the present disclosure obtains a two-dimensional view image of a three-dimensional mesh model; utilizes a regional classification model to classify the two-dimensional view image to obtain the regional category of the two-dimensional view image; and back-projects the two-dimensional view image containing the regional category onto the three-dimensional mesh model to determine the regional category of the three-dimensional mesh model, thereby obtaining a three-dimensional digital model partitioned by regional category. Thus, the regional category is determined from the two-dimensional view of the three-dimensional mesh model using the regional classification model, and the regional category of the three-dimensional mesh model is determined by back-projection. This method is not affected by factors such as the diversity of local regions in the three-dimensional mesh model and the quality of the mesh surface, thereby achieving fully automatic and high-precision classification of the three-dimensional mesh model, providing a good foundation for the subsequent tooth restoration process.

[0071] In another embodiment of the present disclosure, a specific method for determining the region category of the three-dimensional grid model is explained in detail.

[0072] FIG5 shows a schematic diagram of the process of S130 provided in an embodiment of the present disclosure.

[0073] As shown in FIG5 , the method for obtaining the three-dimensional digital model may include the following steps.

[0074] S510. Back-project the two-dimensional view image containing the region category onto a three-dimensional grid model, and determine grid areas corresponding to the pixel areas on the three-dimensional grid model, wherein each grid area corresponds to multiple pixel areas, and the multiple pixel areas corresponding to each grid area correspond to different two-dimensional view images.

[0075] The pixel region refers to a region composed of one pixel or multiple pixels in a two-dimensional view image, and each pixel region corresponds to a region category.

[0076] The grid area may be composed of one or more grid patches, and one grid patch corresponds to one pixel or multiple pixels.

[0077] Specifically, based on the mapping relationship between the two-dimensional view image and the three-dimensional grid model, after the two-dimensional view image containing the region category is back-projected onto the three-dimensional grid model, a grid area corresponding to multiple pixel areas is determined on the three-dimensional grid model.

[0078] It can be understood that since the two-dimensional view image is obtained by mapping the three-dimensional mesh model at multiple viewing angles, multiple pixel areas in different two-dimensional view images correspond to a mesh area on the three-dimensional mesh model.

[0079] S520 . For each grid area on the three-dimensional grid model, vote on the area categories of the multiple pixel areas corresponding to each grid area, determine the area category with the most votes as the area category of the corresponding grid area, and obtain the area category of the three-dimensional grid model.

[0080] It can be understood that since each pixel area in the two-dimensional view image corresponds to a region category, and one or more pixel areas in the two-dimensional view image correspond to a grid area in the three-dimensional grid model, after back-projecting the two-dimensional view image, one or more region categories corresponding to each grid area in the three-dimensional grid model are preliminarily determined, and then, by voting, the region category with the most votes is selected from the one or more region categories corresponding to each preliminarily determined grid area as the region category of the corresponding grid area.

[0081] For ease of understanding, five two-dimensional view images are back-projected onto a three-dimensional grid model. Each two-dimensional view image includes a d pixel area. After the five two-dimensional view images are back-projected onto the three-dimensional grid model, the D grid area on the three-dimensional grid model corresponds to the d pixel area. Among them, the region category of the d pixel area in three two-dimensional view images is the preparation category, and the region category of the d pixel area in two two-dimensional view images is the tooth category. Then, through back-projection, it can be preliminarily determined that the region category corresponding to the D grid area is one of the preparation category and the tooth category. Then, since the D grid area is determined to belong to the preparation category by three two-dimensional view images and the D grid area is determined to belong to the tooth category by two two-dimensional view images, the number of votes for the D grid area belonging to the preparation category is the largest, and the region category of the D grid area is determined to be the preparation category.

[0082] Therefore, since each pixel area in the two-dimensional view image corresponds to a region category, and one or more pixel areas in the two-dimensional view image correspond to a grid area in the three-dimensional grid model, after back-projecting the two-dimensional view image, the region category of each grid area on the three-dimensional grid model is specifically determined by voting, thereby accurately obtaining the region category of the three-dimensional grid model.

[0083] S530: Obtain a three-dimensional digital model partitioned by regional categories.

[0084] In this embodiment, specific implementations of S530 include but are not limited to the following: segmenting the three-dimensional grid model according to the region category of the three-dimensional grid model to obtain a three-dimensional digital model partitioned by region category.

[0085] Specifically, each grid area of ​​the three-dimensional digital model is partitioned according to the area category, for example, by clustering, so as to obtain a three-dimensional digital model partitioned by area category, that is, the boundaries between each partition are obtained. In other words, the boundaries between each partition of the three-dimensional digital model partitioned by area category are clear, and subsequent cutting can be performed based on the boundaries.

[0086] Optionally, the three-dimensional mesh model is partitioned according to one or more regional categories, such as tooth category, gum category, preparation category, implant category, abutment category, and inlay category.

[0087] In yet another embodiment of the present disclosure, the training process of the region classification model is explained in detail.

[0088] FIG6 shows a flow chart of a method for training a region classification model provided in an embodiment of the present disclosure.

[0089] As shown in FIG6 , the training method of the region classification model may include the following steps.

[0090] S610: Acquire multiple two-dimensional reference views of a reference three-dimensional model, and acquire a reference category of each two-dimensional reference view.

[0091] The reference three-dimensional model can be understood as a grid model having similar three-dimensional morphological features to the three-dimensional grid model.

[0092] The multiple 2D reference views are obtained by projecting the reference 3D model from multiple perspectives. As described above, when projecting the reference 3D model, only regions containing highly complex 3D features can be projected and their corresponding perspectives determined. Regions containing less complex 3D features are not projected, thereby obtaining local region images of the reference 3D model and enabling classification of the reference 3D model based on these local region images. This reduces reliance on the global information of the reference 3D model and reduces model training resources, while ensuring the classification accuracy of the regional classification model.

[0093] The reference category is the category labeling information of different areas on each two-dimensional reference view. Optionally, the reference category can be labeled using numbers, letters, binary, etc.

[0094] S620 : Based on the multiple two-dimensional reference views and the reference category of each two-dimensional reference view, fine-tune the pre-trained model to obtain a region classification model.

[0095] In order to configure the model to classify the three-dimensional mesh model, one or more channels in the pre-trained model may be pruned during fine-tuning of the pre-trained model. Specific implementations of S620 include, but are not limited to, the following:

[0096] Adjust the number of channels of each convolution kernel in the pre-trained model according to the current pruning rate to determine the currently adjusted pre-trained model; use the currently adjusted pre-trained model to classify multiple two-dimensional reference views to obtain a predicted category for each two-dimensional reference view; calculate the current accuracy of the pre-trained model based on the predicted category and the reference category; if the current accuracy does not reach the preset accuracy threshold, continue to adjust the number of channels of each convolution kernel in the pre-trained model according to the next pruning rate, and return to execute the above steps to calculate the next accuracy of the pre-trained model until the next accuracy reaches the preset accuracy threshold to obtain a regional classification model.

[0097] Therefore, when training the region classification model, multiple two-dimensional reference views and the reference category of each two-dimensional reference view are used to iteratively fine-tune the pre-trained network according to different pruning rates, so that the region classification model can be well configured to classify the three-dimensional mesh model, thereby obtaining a high-precision region segmentation model, which can be configured to automatically and accurately determine the region category of the three-dimensional mesh model.

[0098] The present disclosure also provides a device for acquiring a 3D digitized model configured to implement the above-described method for acquiring a 3D digitized model, which is described below in conjunction with FIG7 . In the present disclosure, the device for acquiring a 3D digitized model can be an electronic device. The electronic device can include a tablet computer, desktop computer, laptop computer, or other device with communication capabilities, as well as a device simulated by a virtual machine or simulator.

[0099] FIG7 shows a schematic structural diagram of a device for acquiring a three-dimensional digital model provided by an embodiment of the present disclosure.

[0100] As shown in FIG7 , the three-dimensional digital model acquisition device 700 may include:

[0101] An acquisition module 710 is configured to acquire a two-dimensional view image of a three-dimensional mesh model;

[0102] a classification module 720 configured to classify the two-dimensional view image using a region classification model to obtain a region category of the two-dimensional view image;

[0103] The determination module 730 is configured to back-project the two-dimensional view image containing the region category onto the three-dimensional mesh model, determine the region category of the three-dimensional mesh model, and obtain a three-dimensional digital model partitioned by region category.

[0104] The disclosed embodiment of a three-dimensional digital model acquisition device acquires a two-dimensional view image of a three-dimensional mesh model; utilizes a region classification model to classify the two-dimensional view image to obtain the region category of the two-dimensional view image; and back-projects the two-dimensional view image containing the region category onto the three-dimensional mesh model to determine the region category of the three-dimensional mesh model, thereby obtaining a three-dimensional digital model partitioned by region category. Thus, the region category is determined from the two-dimensional view of the three-dimensional mesh model using the region classification model, and the region category of the three-dimensional mesh model is determined by back-projection. This method is not affected by factors such as the diversity of local regions in the three-dimensional mesh model and the quality of the mesh surface, thereby achieving fully automatic and high-precision classification of the three-dimensional mesh model, providing a good foundation for the subsequent tooth restoration process.

[0105] In some embodiments of the present disclosure, the acquisition module 710 includes:

[0106] a three-dimensional mesh model acquisition unit configured to acquire a three-dimensional mesh model of the scanned tooth;

[0107] The projection unit is configured to project the three-dimensional grid model at multiple viewing angles to obtain multiple two-dimensional view images.

[0108] In some embodiments of the present disclosure, the classification module 720 includes:

[0109] a calculation unit configured to calculate, using the region classification model, category probability data for each pixel region in the two-dimensional view image, wherein each pixel region corresponds to a piece of category probability data, and the category probability data includes probability values ​​of a plurality of region categories;

[0110] The region category determination unit is configured to, for each pixel region in the two-dimensional view image, use the region category with the largest probability value in the category probability data as the region category of the corresponding pixel region to obtain the region category of the two-dimensional view image.

[0111] In some embodiments of the present disclosure, the determination module 730 includes:

[0112] a grid area determining unit configured to back-project the two-dimensional view image containing the area category onto the three-dimensional grid model, and determine grid areas corresponding to the pixel areas on the three-dimensional grid model, wherein each grid area corresponds to a plurality of pixel areas, and the plurality of pixel areas corresponding to each grid area correspond to different two-dimensional view images;

[0113] The voting unit is configured to vote for the area categories of multiple pixel areas corresponding to each grid area on the three-dimensional grid model, determine the area category with the most votes as the area category of the corresponding grid area, and obtain the area category of the three-dimensional grid model.

[0114] In some embodiments of the present disclosure, the determination module 730 further includes:

[0115] The segmentation unit is configured to segment the three-dimensional grid model according to the region category of the three-dimensional grid model to obtain the three-dimensional digital model partitioned by region category.

[0116] In some embodiments of the present disclosure, the device further includes:

[0117] The rendering and display module is configured to render and display the three-dimensional digital model.

[0118] In some embodiments of the present disclosure, when the three-dimensional digital model is an intraoral three-dimensional digital model, the area categories on the intraoral three-dimensional digital model include one or more of tooth category, gum category, preparation category, implant rod category, abutment category, and inlay category.

[0119] In some embodiments of the present disclosure, the device further includes:

[0120] a training data acquisition module configured to acquire a plurality of two-dimensional reference views of a reference three-dimensional model and acquire a reference category of each two-dimensional reference view;

[0121] The fine-tuning module is configured to fine-tune the pre-trained model based on the multiple two-dimensional reference views and the reference category of each two-dimensional reference view to obtain the region classification model.

[0122] In some embodiments of the present disclosure, the fine-tuning module includes:

[0123] An adjustment unit is configured to adjust the number of channels of each convolution kernel in the pre-trained model according to the current pruning rate, and determine a current adjusted pre-trained model;

[0124] a prediction unit configured to classify the plurality of two-dimensional reference views using the currently adjusted pre-trained model to obtain a predicted category for each two-dimensional reference view;

[0125] a calculation unit configured to calculate a current accuracy of the pre-trained model according to the predicted category and the reference category;

[0126] The iterative unit is configured to continue adjusting the number of channels of each convolution kernel in the pre-trained model according to the next pruning rate if the current accuracy does not reach the preset accuracy threshold, and return to execute the above steps to calculate the next accuracy of the pre-trained model until the next accuracy reaches the preset accuracy threshold to obtain the regional classification model.

[0127] It should be noted that the three-dimensional digital model acquisition device 700 shown in Figure 7 can execute the various steps in the method embodiments shown in Figures 1 to 6, and realize the various processes and effects in the method embodiments shown in Figures 1 to 6, which will not be repeated here.

[0128] FIG8 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.

[0129] As shown in FIG8 , the electronic device may include a processor 801 and a memory 802 storing computer program instructions.

[0130] Specifically, the processor 801 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0131] The memory 802 may include a large capacity memory configured to store information or instructions. By way of example and not limitation, the memory 802 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 802 may include removable or non-removable (or fixed) media. Where appropriate, the memory 802 may be inside or outside the integrated gateway device. In a specific embodiment, the memory 802 is a non-volatile solid-state memory. In a specific embodiment, the memory 802 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or flash memory, or a combination of two or more of these.

[0132] The processor 801 reads and executes the computer program instructions stored in the memory 802 to perform the steps of the method for obtaining a three-dimensional digital model provided in the embodiment of the present disclosure.

[0133] In one example, the electronic device may further include a transceiver 803 and a bus 804. As shown in FIG8, the processor 801, the memory 802, and the transceiver 803 are connected via the bus 804 and communicate with each other.

[0134] Bus 804 includes hardware, software or both. For example, and not limitation, bus may include Accelerated Graphics Port (AGP) or other graphics bus, Extended Industry Standard Architecture (EISA) bus, Front Side Bus (FSB), Hyper Transport (HT) interconnection, Industrial Standard Architecture (ISA) bus, InfiniBand interconnection, Low Pin Count (LPC) bus, memory bus, Micro Channel Architecture (MCA) bus, Peripheral Component Interconnect (PCI) bus, PCI-Express (PCI-X) bus, Serial Advanced Technology Attachment (SATA) bus, Video Electronics Standards Association Local Bus (VLB) bus or other suitable bus or two or more of these combinations. Where appropriate, bus 804 may include one or more buses. Although the present application embodiment describes and illustrates a specific bus, the application contemplates any suitable bus or interconnection.

[0135] The following is an embodiment of a computer-readable storage medium provided in an embodiment of the present disclosure. The computer-readable storage medium and the three-dimensional digital model acquisition method of the above-mentioned embodiments belong to the same inventive concept. For details not fully described in the embodiment of the computer-readable storage medium, please refer to the embodiment of the three-dimensional digital model acquisition method described above.

[0136] This embodiment provides a storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are configured to perform a method for obtaining a three-dimensional digital model. The method includes:

[0137] Obtaining a two-dimensional view image of a three-dimensional mesh model;

[0138] Using a region classification model, classifying the two-dimensional view image to obtain a plurality of region categories of the two-dimensional view image;

[0139] The two-dimensional view image containing the region category is back-projected onto the three-dimensional grid model, the region category of the three-dimensional grid model is determined, and a three-dimensional digital model having partitions according to the region category is obtained.

[0140] Of course, the storage medium containing computer-executable instructions provided in the embodiment of the present disclosure is not limited to the above method operations, and its computer-executable instructions can also execute related operations in the three-dimensional digital model acquisition method provided in any embodiment of the present disclosure.

[0141] Through the above description of the implementation methods, those skilled in the art can clearly understand that the present disclosure can be implemented with the help of software and necessary general-purpose hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for enabling a computer cloud platform (which can be a personal computer, server, or network cloud platform, etc.) to execute the three-dimensional digital model acquisition method provided in each embodiment of the present disclosure.

[0142] Note that the above are only preferred embodiments of the present disclosure and the technical principles employed. Those skilled in the art will understand that the present disclosure is not limited to the specific embodiments herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present disclosure. Therefore, although the present disclosure has been described in more detail through the above embodiments, the present disclosure is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present disclosure, and the scope of the present disclosure is determined by the scope of the appended claims. Industrial Applicability

[0143] In the method for acquiring a three-dimensional digital model provided in the present invention, the region category is determined from the two-dimensional view of the three-dimensional mesh model by utilizing a region classification model, and the region category of the three-dimensional mesh model is determined by back projection. This method will not be affected by factors such as the diversity of local regions in the three-dimensional mesh model and the quality of the mesh surface, thereby achieving fully automatic and high-precision classification of the three-dimensional mesh model, providing a good foundation for the subsequent tooth restoration process, and has strong industrial applicability.

Claims

1. A method for obtaining a three-dimensional digital model, wherein: include: Obtain a two-dimensional view image of a three-dimensional mesh model; Using a region classification model, classifying the two-dimensional view image to obtain a region category of the two-dimensional view image; The two-dimensional view image containing the region category is back-projected onto the three-dimensional grid model, the region category of the three-dimensional grid model is determined, and a three-dimensional digital model having partitions according to the region category is obtained.

2. The method according to claim 1, wherein: The step of obtaining a two-dimensional view image of a three-dimensional mesh model includes: Obtaining a three-dimensional mesh model of the scanned tooth; The three-dimensional grid model is projected at multiple viewing angles to obtain multiple two-dimensional view images.

3. The method according to claim 1, wherein: The utilizing the region classification model to classify the two-dimensional view image to obtain the region category of the two-dimensional view image includes: Calculating the category probability data of each pixel region in the two-dimensional view image by using the region classification model, wherein each pixel region corresponds to a category probability data, and the category probability data includes probability values ​​of multiple region categories; For each pixel region in the two-dimensional view image, the region category with the largest probability value in the category probability data is used as the region category of the corresponding pixel region to obtain the region category of the two-dimensional view image.

4. The method according to claim 1, wherein: The back-projecting of the two-dimensional view image containing the region category onto the three-dimensional mesh model to determine the region category of the three-dimensional mesh model comprises: Back-projecting the two-dimensional view image containing the region category onto the three-dimensional grid model, and determining a grid area corresponding to the pixel area on the three-dimensional grid model, wherein each grid area corresponds to a plurality of pixel areas, and the plurality of pixel areas corresponding to each grid area correspond to different two-dimensional view images; For each grid area on the three-dimensional grid model, voting is performed on the area categories of multiple pixel areas corresponding to each grid area, and the area category with the most votes is determined as the area category of the corresponding grid area, so as to obtain the area category of the three-dimensional grid model.

5. The method according to claim 1, wherein: The method of obtaining a three-dimensional digital model having partitions according to regional categories comprises: The three-dimensional grid model is segmented according to the region category of the three-dimensional grid model to obtain the three-dimensional digital model partitioned by region category.

6. The method according to claim 1, wherein: Also includes: The three-dimensional digital model is rendered and displayed.

7. The method according to claim 1, wherein: In the case where the three-dimensional digital model is an intraoral three-dimensional digital model, the region categories on the intraoral three-dimensional digital model include one or more of tooth categories, gum categories, preparation categories, implant stem categories, abutment categories, and inlay categories.

8. The method according to claim 1, wherein: Also includes: Acquire multiple two-dimensional reference views of the reference three-dimensional model, and acquire a reference category of each two-dimensional reference view; Based on the multiple two-dimensional reference views and the reference category of each two-dimensional reference view, the pre-trained model is fine-tuned to obtain the region classification model.

9. The method according to claim 8, wherein: The method of fine-tuning the pre-trained model based on the multiple two-dimensional reference views and the reference category of each two-dimensional reference view to obtain the region classification model includes: Adjust the number of channels of each convolution kernel in the pre-trained model according to the current pruning rate, and determine the current adjusted pre-trained model; Using the currently adjusted pre-trained model, classify the multiple two-dimensional reference views to obtain a predicted category of each two-dimensional reference view; Calculating the current accuracy of the pre-trained model according to the predicted category and the reference category; If the current accuracy does not reach the preset accuracy threshold, the number of channels of each convolution kernel in the pre-trained model continues to be adjusted according to the next pruning rate, and the above steps are returned to calculate the next accuracy of the pre-trained model until the next accuracy reaches the preset accuracy threshold to obtain the regional classification model.

10. A device for acquiring a three-dimensional digital model, wherein: include: An acquisition module is configured to acquire a two-dimensional view image of a three-dimensional mesh model; A classification module is configured to classify the two-dimensional view image using a regional classification model to obtain a regional category of the two-dimensional view image; The determination module is configured to back-project the two-dimensional view image containing the region category onto the three-dimensional grid model, determine the region category of the three-dimensional grid model, and obtain a three-dimensional digital model partitioned by region category.

11. An electronic device, wherein: include: processor; a memory configured to store executable instructions; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the following steps: Obtain a two-dimensional view image of a three-dimensional mesh model; Using a region classification model, classifying the two-dimensional view image to obtain a region category of the two-dimensional view image; The two-dimensional view image containing the region category is back-projected onto the three-dimensional grid model, the region category of the three-dimensional grid model is determined, and a three-dimensional digital model having partitions according to the region category is obtained.

12. The electronic device according to claim 11, wherein: When acquiring a two-dimensional view image of a three-dimensional mesh model, the processor executes the executable instructions to implement the following steps: Obtaining a three-dimensional mesh model of the scanned tooth; The three-dimensional grid model is projected at multiple viewing angles to obtain multiple two-dimensional view images.

13. The electronic device according to claim 11, wherein: When the two-dimensional view image is classified by using the region classification model to obtain the region category of the two-dimensional view image, the processor executes the executable instructions to implement the following steps: Calculating the category probability data of each pixel region in the two-dimensional view image by using the region classification model, wherein each pixel region corresponds to a category probability data, and the category probability data includes probability values ​​of multiple region categories; For each pixel region in the two-dimensional view image, the region category with the largest probability value in the category probability data is used as the region category of the corresponding pixel region to obtain the region category of the two-dimensional view image.

14. The electronic device according to claim 11, wherein: When back-projecting the two-dimensional view image containing the region category onto the three-dimensional mesh model to determine the region category of the three-dimensional mesh model, the processor executes the executable instructions to implement the following steps: Back-projecting the two-dimensional view image containing the region category onto the three-dimensional grid model, and determining a grid area corresponding to the pixel area on the three-dimensional grid model, wherein each grid area corresponds to a plurality of pixel areas, and the plurality of pixel areas corresponding to each grid area correspond to different two-dimensional view images; For each grid area on the three-dimensional grid model, voting is performed on the area categories of multiple pixel areas corresponding to each grid area, and the area category with the most votes is determined as the area category of the corresponding grid area, so as to obtain the area category of the three-dimensional grid model.

15. The electronic device according to claim 11, wherein: When obtaining the three-dimensional digital model with partitions according to regional categories, the processor executes the executable instructions to implement the following steps: The three-dimensional grid model is segmented according to the region category of the three-dimensional grid model to obtain the three-dimensional digital model partitioned by region category.

16. The electronic device according to claim 11, wherein: The processor executes the executable instructions to implement the following steps: The three-dimensional digital model is rendered and displayed.

17. The electronic device according to claim 11, wherein: The processor executes the executable instructions to implement the following steps: Acquire multiple two-dimensional reference views of the reference three-dimensional model, and acquire a reference category of each two-dimensional reference view; Based on the multiple two-dimensional reference views and the reference category of each two-dimensional reference view, the pre-trained model is fine-tuned to obtain the region classification model.

18. The electronic device according to claim 17, wherein: When the pre-trained model is fine-tuned based on the multiple two-dimensional reference views and the reference category of each two-dimensional reference view to obtain the region classification model, the processor executes the executable instructions to implement the following steps: Adjust the number of channels of each convolution kernel in the pre-trained model according to the current pruning rate, and determine the current adjusted pre-trained model; Using the currently adjusted pre-trained model, classify the multiple two-dimensional reference views to obtain a predicted category of each two-dimensional reference view; Calculating the current accuracy of the pre-trained model according to the predicted category and the reference category; If the current accuracy does not reach the preset accuracy threshold, the number of channels of each convolution kernel in the pre-trained model continues to be adjusted according to the next pruning rate, and the above steps are returned to calculate the next accuracy of the pre-trained model until the next accuracy reaches the preset accuracy threshold to obtain the regional classification model.

19. A computer-readable storage medium, wherein: The computer readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the following steps: Obtain a two-dimensional view image of a three-dimensional mesh model; Using a region classification model, classifying the two-dimensional view image to obtain a region category of the two-dimensional view image; The two-dimensional view image containing the region category is back-projected onto the three-dimensional grid model, the region category of the three-dimensional grid model is determined, and a three-dimensional digital model having partitions according to the region category is obtained.

20. The computer-readable storage medium of claim 19, wherein: When acquiring a two-dimensional view image of a three-dimensional mesh model, the processor executes the computer program to implement the following steps: Obtaining a three-dimensional mesh model of the scanned tooth; The three-dimensional grid model is projected at multiple viewing angles to obtain multiple two-dimensional view images.

Images