Automatic tooth boundary extraction method, image correction method and tooth treatment system
The automatic tooth boundary extraction method addresses CBCT image artifacts by using image processing models to register and adjust parameters, enhancing accuracy and efficiency in tooth boundary identification.
Patent Information
- Application Number
- CN202311671789.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-07
- Publication Date
- 2025-01-03
- Estimated Expiration
- 2043-12-07
AI Technical Summary
CBCT images suffer from artifacts such as motion and metal artifacts, leading to inaccurate representation of tooth boundaries, and current correction methods rely on manual operator annotation, resulting in poor accuracy and efficiency.
An automatic tooth boundary extraction method using image processing models that register and extract tooth boundaries from feature images, adjusting model parameters based on supervisory signals to achieve automated and accurate boundary extraction.
Improves the accuracy and efficiency of CBCT image processing by eliminating the need for manual annotation and effectively correcting artifacts, enabling precise tooth boundary identification.
Smart Images
Figure CN117670956B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of medical robot technology, and in particular to an automatic tooth boundary extraction method, an image correction method and a tooth treatment system. Background Art
[0002] With the continuous development of dental diagnostic technology, Cone Beam Computed Tomography (CBCT) has become the main three-dimensional image examination method in dental clinic. It can obtain high-resolution oral anatomical structure information non-invasively, which is of great value for diagnosis and surgical treatment planning.
[0003] However, artifacts occur during the CBCT image generation process, such as motion artifacts and severe metal artifacts from dental restorations. These artifacts have a negative impact on image quality and cause CBCT images to be unable to accurately reproduce the actual oral anatomical and physiological conditions, such as tooth boundaries.
[0004] Currently, most of the various CBCT image correction solutions require operators to manually perform image annotation, which results in poor correction accuracy and efficiency. Summary of the invention
[0005] The purpose of this application is to propose an automatic tooth boundary extraction method, an image correction method and a tooth treatment system to solve or alleviate the technical problems existing in the prior art.
[0006] A method for automatically extracting tooth boundaries, comprising:
[0007] Acquire tooth surface feature images and tooth anatomical feature images;
[0008] Registering the tooth surface feature image to the tooth anatomical feature image to obtain a registered image;
[0009] extracting the real tooth boundary from the registered image;
[0010] Extracting tooth boundaries from the tooth anatomical feature image based on a first image processing model;
[0011] Based on the supervisory signal and the tooth boundary actually extracted by the first image processing model, adjusting the model parameters of the first image processing model until the training is completed;
[0012] Automatically extract tooth boundaries from the target tooth anatomical feature image based on the trained first image processing model.
[0013] An image correction method, comprising:
[0014] Acquire a first modality image and a second modality image, wherein the first modality image is used to characterize tooth surface features, and the second modality image is used to characterize tooth anatomical features;
[0015] Mapping the first modality image onto the second modality image to obtain a registered image;
[0016] extracting the real tooth boundary from the registered image;
[0017] extracting tooth boundaries from the second modality image based on a set image correction model;
[0018] Based on the real tooth boundary and the tooth boundary actually extracted by the image correction model, adjusting the model parameters of the image correction model until the training is completed;
[0019] Based on the trained image correction model, tooth boundaries are automatically extracted from the second modality image to be corrected and artifacts on the second modality image to be corrected are removed.
[0020] A training method for an image correction model, comprising:
[0021] Acquire a first modality image and a second modality image, wherein the first modality image is used to characterize tooth surface features, and the second modality image is used to characterize tooth anatomical features;
[0022] Mapping the first modality image onto the second modality image to obtain a registered image;
[0023] extracting the real tooth boundary from the registered image;
[0024] extracting tooth boundaries from the second modality image based on a set image correction model;
[0025] Based on the supervisory signal and the tooth boundaries actually extracted by the image correction model, the model parameters of the image correction model are adjusted until the training is completed.
[0026] An image correction method, comprising:
[0027] Calling the image correction model trained according to any one of the methods described in the embodiments of the present application;
[0028] The second modality image to be corrected is input into the image correction model for correction to start extracting tooth boundaries and removing artifacts on the second modality image to be corrected.
[0029] A dental treatment system comprises: a robotic arm and a host, wherein the host is used to execute the method described in any one of the embodiments of the present application to automatically extract tooth boundaries from an anatomical feature image of a target tooth of a patient, and based on the automatically extracted tooth boundaries, control the robotic arm to perform a set target action on the target tooth of the patient.
[0030] In the solution provided in the embodiment of the present application, a tooth surface feature image and a tooth anatomical feature image are acquired; the tooth surface feature image is registered to the tooth anatomical feature image to obtain a registered image; the real tooth boundary is extracted from the registered image; in a first image processing model, the tooth boundary is extracted from the tooth anatomical feature image; based on the supervisory signal and the tooth boundary actually extracted by the first image processing model, the model parameters of the first image processing model are adjusted until the training is completed; based on the trained first image processing model, the tooth boundary of the target tooth anatomical feature image is automatically extracted, so that there is no need for the operator to manually perform image annotation, etc., thereby realizing automated processing of CBCT images, and there is no need for the operator to manually perform image annotation, etc., thereby improving the accuracy and efficiency of image processing. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Hereinafter, some specific embodiments of the present application will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:
[0032] Figure 1 This is a flow chart of an automatic tooth boundary extraction method according to an embodiment of the present application.
[0033] Figure 2 The figure is a flowchart of an image correction method according to an embodiment of the present application.
[0034] Figure 3 This is a schematic diagram of the structure of a dental treatment system according to an embodiment of the present application. DETAILED DESCRIPTION
[0035] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.
[0036] Figure 1 This is a flow chart of an automatic tooth boundary extraction method according to an embodiment of the present application. Figure 1 As shown, it includes:
[0037] S101, acquiring a tooth surface feature image and a tooth anatomical feature image;
[0038] S102, registering the tooth surface feature image with the tooth anatomical feature image to obtain a registered image;
[0039] S103, extracting the real tooth boundary from the registered image as a supervision signal;
[0040] S104, extracting tooth boundaries from the tooth anatomical feature image based on a first image processing model;
[0041] S105, adjusting the model parameters of the first image processing model based on the supervisory signal and the tooth boundary actually extracted by the first image processing model until the training is completed;
[0042] S106: Automatically extract tooth boundaries from the target tooth anatomical feature image based on the trained first image processing model.
[0043] Optionally, in this embodiment, a three-dimensional oral scanning device may be used to scan the user's oral cavity to obtain an oral scanning image, and the oral scanning image is used as the tooth surface feature image. For the tooth anatomical feature image, for example, a CBCT scanning device may be used to photograph the user's oral cavity to obtain a CBCT image, and the CBCT image is used as the tooth anatomical feature image.
[0044] Here, it should be noted that the above-mentioned oral scan images as the tooth surface feature images and the CBCT images as the tooth anatomical feature images are merely examples and are not exclusive limitations.
[0045] In this embodiment, since the model training is required, the tooth surface feature images and tooth anatomical feature images can come from multiple users. Therefore, a one-to-one correspondence between the tooth surface feature images, tooth anatomical feature images and users can be established by means of user IDs.
[0046] In this embodiment, before executing step S102, the tooth surface feature image and the tooth anatomical feature image may be geometrically aligned so that the registration processing of step S102 can be performed after the alignment. For example, the specific alignment processing may be implemented based on the following point cloud processing method.
[0047] In this embodiment, performing step S102 to register the tooth surface feature image with the tooth anatomical feature image to obtain a registered image includes:
[0048] Extracting feature descriptors from the tooth surface feature image to obtain at least one first feature descriptor;
[0049] Extracting feature descriptors from the tooth anatomical feature image to obtain at least one second feature descriptor;
[0050] Based on the first feature descriptor and the second feature descriptor, the tooth surface feature image is registered to the tooth anatomical feature image to obtain a registered image.
[0051] To this end, by extracting feature descriptors from the tooth surface feature image and the tooth anatomical feature image, and performing registration based on these descriptors, the accuracy and stability of registration can be improved. Feature descriptors can usually describe the local features of the image more accurately, which helps to improve the accuracy of registration. Furthermore, using feature descriptors for registration can improve the robustness to image changes and interference. Even if the tooth surface feature image and the tooth anatomical feature image have a certain degree of deformation, rotation or noise, the registration method based on feature descriptors can better cope with these situations. Finally, by extracting feature descriptors and performing registration based on them, automatic registration can be guaranteed, registration efficiency can be improved, and rapid registration can be performed on large-scale data; moreover, feature descriptors can usually better preserve the local feature information of the image, so registration based on feature descriptors can better preserve the details and features of the image, which is conducive to subsequent analysis and processing.
[0052] Optionally, extracting feature descriptors from the tooth surface feature image to obtain at least one first feature descriptor may include: performing a convolution operation on the tooth surface feature image to extract feature descriptors to obtain at least one first feature descriptor.
[0053] Optionally, performing a convolution operation on the tooth surface feature image to extract a feature descriptor to obtain at least one first feature descriptor includes:
[0054] Performing a convolution operation on the tooth surface feature image to generate a first three-dimensional feature map of the tooth, wherein the first three-dimensional feature includes a spatial feature and a structural feature of the tooth;
[0055] Performing residual convolution on the first stereo feature map of the tooth to generate a detail feature map of the tooth;
[0056] The tooth detail feature map is upsampled to obtain at least one first feature descriptor.
[0057] Optionally, the extracting feature descriptors from the tooth surface feature image to obtain the first feature descriptor includes: extracting point cloud data from the tooth surface feature image to obtain a tooth surface feature point cloud, and extracting feature descriptors from the tooth surface feature image based on the tooth surface feature point cloud to obtain the first feature descriptor.
[0058] Optionally, performing a convolution operation on the tooth anatomical feature image to extract a feature descriptor to obtain at least one second feature descriptor includes:
[0059] Performing a convolution operation on the tooth anatomical feature image to generate a second three-dimensional feature map of the tooth, wherein the second three-dimensional feature includes a spatial feature and a structural feature of the tooth;
[0060] Performing residual convolution on the second stereo feature map of the tooth to generate a detail feature map of the tooth;
[0061] The tooth detail feature map is upsampled to obtain at least one second feature descriptor.
[0062] Optionally, the extracting feature descriptors from the tooth surface feature image to obtain the first feature descriptor may further include: removing noise from the tooth surface feature point cloud.
[0063] Exemplarily, the removal of noise in the tooth surface feature point cloud may include: for any point cloud, calculating the actual distance and average distance from each point in the point cloud to the nearest K points; based on the actual distance and the average distance, calculating the mean and standard deviation of all average distances, and calculating the distance threshold according to the mean and standard deviation; if the average distance is outside the distance threshold, treating the point cloud as noise for removal, K is a positive integer, and the size of K is set according to the scene.
[0064] To this end, by calculating the actual distance and average distance from each point in each point cloud to its neighboring points, and judging based on the distance threshold, noise points can be more accurately identified and removed. Furthermore, this method makes judgments based on the distance threshold obtained by the mean and standard deviation of the average distance, and can adaptively adjust the standard for noise removal according to the specific point cloud data set, so it has certain applicability and robustness for different data sets. Finally, points whose average distance is outside the distance threshold are considered as noise and removed.
[0065] Optionally, extracting a feature descriptor from the dental anatomical feature image to obtain a second feature descriptor includes: extracting point cloud data from the dental anatomical feature image to obtain a dental anatomical feature point cloud, and extracting a feature descriptor from the dental anatomical feature image based on the dental anatomical feature point cloud to obtain a second feature descriptor.
[0066] In this embodiment, the tooth anatomical feature point cloud may be obtained by performing point cloud data extraction on the tooth anatomical feature image through voxelization or three-dimensional reconstruction.
[0067] Optionally, the registering the tooth surface feature image to the tooth anatomical feature image based on the first feature descriptor and the second feature descriptor to obtain a registered image includes:
[0068] Based on the first feature descriptor and the second feature descriptor, determining a transformation matrix for registering the tooth surface feature image to the tooth anatomical feature image;
[0069] Based on the transformation matrix, the tooth surface feature image is registered to the tooth anatomical feature image to obtain a registered image.
[0070] To this end, by determining the registration transformation matrix based on the first feature descriptor and the second feature descriptor, high-precision image registration can be achieved, so that the tooth surface feature image can be accurately aligned with the tooth anatomical feature image, thereby obtaining a high-quality registered image. Furthermore, using feature descriptors for registration can realize an automated registration process, thereby improving the efficiency and consistency of registration. In addition, since feature descriptors are usually highly robust and have a certain invariance to image rotation, translation and other transformations, registration based on feature descriptors can better adapt to changes between different images and improve the robustness of registration.
[0071] Optionally, determining a transformation matrix for registering the tooth surface feature image to the tooth anatomical feature image based on the first feature descriptor and the second feature descriptor comprises:
[0072] The following steps are performed for any first feature descriptor to determine the matching point of any first feature descriptor on the tooth anatomical feature image, so as to determine the transformation matrix based on the matching points of all first feature descriptors:
[0073] Calculating the similarity between any one of the first feature descriptors and all of the second feature descriptors;
[0074] The second feature descriptor with the greatest similarity is used as the matching point of any first feature descriptor on the tooth anatomical feature image.
[0075] To this end, by calculating the similarity between any first feature descriptor and all second feature descriptors, the matching point of the first feature descriptor on the dental anatomical feature image can be efficiently determined, thereby providing accurate matching information for the subsequent registration process. Furthermore, by determining the transformation matrix based on the matching points of all first feature descriptors, the information of multiple matching points can be comprehensively considered to obtain a more accurate and stable transformation matrix, which is conducive to achieving high-quality image registration. In the dental anatomical feature image, there may be complex structures and textures. The method based on the second feature descriptor with the greatest similarity as the matching point can better cope with such complex situations and improve the robustness and accuracy of the matching. Finally, the method can adaptively determine the matching point based on the similarity between the first feature descriptor and the second feature descriptor, is not affected by specific scenes or image changes, and has strong applicability and versatility.
[0076] Optionally, the method further comprises:
[0077] The tooth surface feature point cloud is used as the source point cloud, and the tooth anatomical feature point cloud is used as the target point cloud.
[0078] Furthermore, when calculating the similarity between any first feature descriptor and all the second feature descriptors, for any first feature descriptor in the source point cloud, the similarity (such as Euclidean distance) between it and all the second feature descriptors in the target point cloud is calculated.
[0079] Optionally, the method further comprises:
[0080] Form a matching pair with each first feature descriptor and its matching point on the tooth anatomical feature image;
[0081] The credibility of each matching pair is verified to select a correctly matched matching pair from them, and the transformation matrix is determined based on the first feature descriptor corresponding to the correctly matched matching pair and its matching points.
[0082] Specifically, the credibility verification for each matching pair includes the following steps: for any first feature descriptor, based on the above transformation matrix, determining other second feature descriptors that satisfy the transformation matrix;
[0083] Counting the number of second feature descriptors whose similarity with any first feature descriptor is greater than a set similarity threshold among other second feature descriptors;
[0084] If the number of the second feature descriptors is greater than a set number threshold, the corresponding matching pair is credible.
[0085] Optionally, in this embodiment, the transformation matrix is used to describe parameters such as rotation and translation to be performed when transforming the first feature descriptor into the tooth anatomical feature image.
[0086] Optionally, extracting the real tooth boundary from the registered image comprises:
[0087] The image converted from the tooth surface feature image to the registration image is segmented to extract the projection contour of the tooth on the tooth surface feature image and use it as the real tooth boundary.
[0088] Optionally, if the registration image is obtained based on the above-mentioned point cloud processing method, when segmenting the image converted from the tooth surface feature image to the registration image through registration, image segmentation can be performed based on the point cloud processing method.
[0089] Optionally, the image segmentation based on point cloud processing includes:
[0090] Encoding the registered image to calculate the self-attention of the point cloud data thereon and obtain a self-attention vector, wherein the self-attention vector includes key features of the tooth;
[0091] Adding the position of the point cloud data corresponding to the registered image to the self-attention vector to obtain an embedding vector;
[0092] The embedding vector is encoded and decoded to obtain a projection contour of the tooth on the tooth surface feature image on the tooth anatomical feature image.
[0093] Optionally, the method further includes: correcting the tooth anatomical feature image based on the tooth boundary extracted from the tooth anatomical feature image. To this end, by correcting the tooth anatomical feature image, the corrected tooth anatomical feature image can more accurately display the tooth anatomical structure, and the unreliable tooth boundary on the tooth anatomical feature image is corrected based on the reliable tooth boundary on the tooth surface feature image.
[0094] Optionally, the method further includes: performing auxiliary supervision on the training process of the first image processing model to improve the accuracy of model training.
[0095] Optionally, the auxiliary supervision of the training process of the first image processing model includes:
[0096] Based on the trained first discriminator, auxiliary supervision is performed on the training process of the first image processing model according to the corrected tooth anatomical feature image and the tooth surface feature image.
[0097] Optionally, the first discriminator based on the training is used to assist in supervising the training process of the first image processing model according to the corrected tooth anatomical feature image and the tooth surface feature image, including:
[0098] Based on the trained first discriminator, feature extraction is performed on the corrected tooth anatomical feature image and the tooth surface feature image, and based on the extracted features, it is determined whether the corrected tooth anatomical feature image and the tooth surface feature image belong to the same target user, so as to assist in supervising the training process of the first image processing model.
[0099] In this embodiment, the first discriminator is trained using positive samples and negative samples of the corrected dental anatomical feature image and the dental surface feature image, so that the first discriminator has correct model parameters to be able to distinguish whether the corrected dental anatomical feature image and the dental surface feature image belong to the same target user.
[0100] Optionally, the auxiliary supervision of the training process of the first image processing model includes:
[0101] synthesizing the tooth surface feature image and the corrected tooth anatomical feature image based on the second image processing model to obtain a synthesized surface feature image;
[0102] Based on the trained second discriminator, auxiliary supervision is performed on the training process of the first image processing model according to the synthesized surface feature image and the tooth anatomical feature image.
[0103] Here, it should be noted that only one of the first discriminator and the second discriminator may be selected to perform the above-mentioned supervision.
[0104] Optionally, the second discriminator based on the training is used to assist in supervising the training process of the first image processing model according to the synthesized surface feature image and the tooth anatomical feature image, including:
[0105] Based on the trained second discriminator, feature extraction is performed on the synthetic surface feature image and the dental anatomical feature image; based on the extracted features, it is determined whether the synthetic surface feature image and the dental anatomical structure image belong to the same target user, so as to assist in supervising the training process of the first image processing model.
[0106] In this embodiment, the second discriminator is trained using positive samples and negative samples of the synthetic surface feature image and the dental anatomical feature image so that the second discriminator has correct model parameters to be able to distinguish whether the corrected dental anatomical feature image and the dental surface feature image belong to the same target user.
[0107] Here, the first image processing model and the second image processing model may be a generator model structure (Generator Network), wherein the first image processing model and the second image processing model are both codec structures.
[0108] To this end, the method described may also include: based on the second image processing model, encoding and decoding the corrected dental anatomical feature image to obtain a restored dental anatomical feature image (such as a Cycle-CBCT image) for comparison with the dental anatomical feature image before correction to assist in determining the effect of the training of the first image processing model.
[0109] Furthermore, to this end, the method described may also include: encoding and decoding the synthesized surface feature image based on the first image processing model to obtain a restored tooth surface feature image (such as a cycle-mouth scan image) for comparison with the tooth surface feature image before synthesis, to assist in determining the effect of the training of the first image processing model.
[0110] In the above embodiment, the first discriminator and the second discriminator are trained discriminator networks (Discriminator Network).
[0111] To this end, the first image processing model, the second image processing model, the first discriminator, and the second discriminator mentioned above can be implemented based on a generative adversarial network model, for example.
[0112] Here, the above functions implemented by the first image processing model, the second image processing model, the first discriminator, and the second discriminator are not exclusive limitations, but only part of the functions implemented, not all of the functions.
[0113] Here, it should be noted that during the training process, the first image processing model, the second image processing model, the first discriminator, and the second discriminator need to participate together to realize the training of the first image processing model. After the training is completed, the automatic extraction of tooth boundaries is to be performed, which requires the participation of the trained first image processing model.
[0114] To this end, by extracting features from the corrected tooth anatomical feature image and the tooth surface feature image based on the trained first discriminator and the second discriminator and performing target user judgment, the training process can be supervised to ensure that the training process of the first image processing model is supervised, which helps to improve the accuracy and robustness of the model. Furthermore, by using the first and second discriminators to extract features and judge the target user, the training process can be automatically supervised to improve the training efficiency and consistency. In addition, by judging whether the corrected tooth anatomical feature image and the tooth surface feature image belong to the same target user, the model's ability to recognize and distinguish features of different users can be enhanced, and the robustness and generalization ability of the model can be improved. Finally, by extracting features from the synthetic surface feature image and the tooth anatomical feature image and performing target user judgment, the relevance of the data can be improved, ensuring that the training data is more in line with the actual application scenario, thereby improving the practicality and reliability of the model. Moreover, by introducing a discriminator for auxiliary supervision, the model's adversarial nature can be increased, and the model's ability to distinguish features of different users can be improved, thereby improving the model's security and privacy protection capabilities.
[0115] Figure 2 FIG. 1 is a flow chart of an image correction method according to an embodiment of the present application. Figure 2 As shown, it includes:
[0116] S201, acquiring a first modality image and a second modality image, wherein the first modality image is used to characterize tooth surface features, and the second modality image is used to characterize tooth anatomical features;
[0117] S202, mapping the first modality image onto the second modality image to obtain a registered image;
[0118] S203, extracting the real tooth boundary from the registered image;
[0119] S204, extracting tooth boundaries from the second modality image based on a set image correction model;
[0120] S205, adjusting the model parameters of the image correction model based on the real tooth boundary and the tooth boundary actually extracted by the image correction model until the training is completed;
[0121] S206: Automatically extract tooth boundaries from the second modality image to be corrected based on the trained image correction model and remove artifacts from the second modality image to be corrected.
[0122] In this embodiment, the artifacts include but are not limited to motion artifacts and severe metal artifacts from dental restorations.
[0123] In this embodiment, the first modality image may be the tooth surface feature image, and the second modality image may be the tooth anatomical feature image.
[0124] In this embodiment, the image correction model may be, for example, the first image processing model mentioned above.
[0125] Here, the exemplary description of the relevant steps in this embodiment can be found in the above Figure 1 Records of.
[0126] The present application embodiment provides a method for training an image correction model, which includes:
[0127] Acquire a first modality image and a second modality image, wherein the first modality image is used to characterize tooth surface features, and the second modality image is used to characterize tooth anatomical features;
[0128] Mapping the first modality image onto the second modality image to obtain a registered image;
[0129] extracting the real tooth boundary from the registered image;
[0130] extracting tooth boundaries from the second modality image based on a set image correction model;
[0131] Based on the supervisory signal and the tooth boundaries actually extracted by the image correction model, the model parameters of the image correction model are adjusted until the training is completed.
[0132] In this embodiment, in this embodiment, the first modality image may be the tooth surface feature image, and the second modality image may be the tooth anatomical feature image.
[0133] In this embodiment, the image correction model may be, for example, the first image processing model mentioned above.
[0134] The present application provides an image correction method, which includes:
[0135] Calling the image correction model trained based on the above embodiment of the present application;
[0136] The second modality image to be corrected is input into the image correction model for correction to start extracting tooth boundaries and removing artifacts on the second modality image to be corrected.
[0137] In the above model process, if the model parameters of the model need to be adjusted, for example, the loss value can be calculated based on the set loss function to estimate the matching degree between the real tooth boundary and the actually extracted tooth boundary, and the model parameters can be further adjusted using the gradient descent method based on the matching pairs.
[0138] Figure 3 This is a schematic diagram of the structure of a tooth treatment system according to an embodiment of the present application. Figure 3 As shown, it includes: a robotic arm 301 and a host 302, wherein the host is used to execute the method described in any one of the present application to automatically extract the tooth boundaries of the patient's target tooth anatomical feature image, and based on the automatically extracted tooth boundaries, control the robotic arm to perform the set target action for the patient's target teeth.
[0139] For detailed technical implementation of this embodiment, please refer to the above Figure 1 Records of.
[0140] The expressions "first", "second", "the first" or "the second" used in various embodiments of the present disclosure may modify various components regardless of order and / or importance, but these expressions do not limit the corresponding components. The above expressions are only configured for the purpose of distinguishing an element from other elements. For example, a first user device and a second user device represent different user devices, although both are user devices. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the present disclosure.
[0141] When one element (e.g., a first element) is referred to as being "(operably or communicatively) coupled" or "(operably or communicatively) coupled to" or "connected to" another element (e.g., a second element), it is understood that the one element is directly connected to the other element or the one element is indirectly connected to the other element via yet another element (e.g., a third element). Conversely, it is understood that when an element (e.g., a first element) is referred to as being "directly connected" or "directly coupled" to another element (the second element), no element (e.g., a third element) is interposed between the two.
[0142] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.
Claims
1. A method for automatically extracting tooth boundaries, characterized in that: include: Acquire tooth surface feature images and tooth anatomical feature images; Registering the tooth surface feature image to the tooth anatomical feature image to obtain a registered image; Extracting the real tooth boundary from the registered image as a supervision signal; Extracting tooth boundaries from the tooth anatomical feature image based on a first image processing model; Based on the supervisory signal and the tooth boundary actually extracted by the first image processing model, adjusting the model parameters of the first image processing model until the training is completed; Automatically extracting tooth boundaries from the target tooth anatomical feature image based on the trained first image processing model; Among them, based on the supervisory signal and the tooth boundary actually extracted by the first image processing model, the model parameters of the first image processing model are adjusted until the training is completed, including: calculating the loss value based on the set loss function to estimate the matching degree between the real tooth boundary and the actually extracted tooth boundary, and further adjusting the model parameters according to the matching degree using the gradient descent method.
2. The method according to claim 1, characterized in that The step of registering the tooth surface feature image with the tooth anatomical feature image to obtain a registered image comprises: Extracting feature descriptors from the tooth surface feature image to obtain at least one first feature descriptor; Extracting feature descriptors from the tooth anatomical feature image to obtain at least one second feature descriptor; Based on the first feature descriptor and the second feature descriptor, the tooth surface feature image is registered to the tooth anatomical feature image to obtain a registered image.
3. The method according to claim 2, characterized in that The extracting a feature descriptor from the tooth surface feature image to obtain a first feature descriptor comprises: extracting point cloud data from the tooth surface feature image to obtain a tooth surface feature point cloud, and extracting a feature descriptor from the tooth surface feature image based on the tooth surface feature point cloud to obtain a first feature descriptor; And / or, extracting a feature descriptor from the dental anatomical feature image to obtain a second feature descriptor, including: extracting point cloud data from the dental anatomical feature image to obtain a dental anatomical feature point cloud, and extracting a feature descriptor from the dental anatomical feature image based on the dental anatomical feature point cloud to obtain a second feature descriptor.
4. The method according to claim 2, characterized in that: The registering the tooth surface feature image to the tooth anatomical feature image based on the first feature descriptor and the second feature descriptor to obtain a registered image includes: Based on the first feature descriptor and the second feature descriptor, determining a transformation matrix for registering the tooth surface feature image to the tooth anatomical feature image; Based on the transformation matrix, the tooth surface feature image is registered to the tooth anatomical feature image to obtain a registered image.
5. The method according to claim 4, characterized in that The step of determining a transformation matrix for registering the tooth surface feature image to the tooth anatomical feature image based on the first feature descriptor and the second feature descriptor comprises: The following steps are performed for any first feature descriptor to determine the matching point of any first feature descriptor on the tooth anatomical feature image, so as to determine the transformation matrix based on the matching points of all first feature descriptors: Calculating the similarity between any one of the first feature descriptors and all of the second feature descriptors; The second feature descriptor with the greatest similarity is used as the matching point of any first feature descriptor on the tooth anatomical feature image.
6. The method according to claim 1, characterized in that The step of extracting the real tooth boundary from the registered image comprises: The image converted from the tooth surface feature image to the registration image is segmented to extract the projection contour of the tooth on the tooth surface feature image and use it as the real tooth boundary.
7. The method according to claim 6, characterized in that The method further includes: correcting the tooth anatomical feature image based on tooth boundaries extracted from the tooth anatomical feature image.
8. An image correction method, characterized in that: include: Acquire a first modality image and a second modality image, wherein the first modality image is used to characterize tooth surface features, and the second modality image is used to characterize tooth anatomical features; Mapping the first modality image onto the second modality image to obtain a registered image; extracting the real tooth boundary from the registered image; extracting tooth boundaries from the second modality image based on a set image correction model; Based on the real tooth boundary and the tooth boundary actually extracted by the image correction model, adjusting the model parameters of the image correction model until the training is completed; Automatically extracting tooth boundaries from the second modality image to be corrected based on the trained image correction model and removing artifacts from the second modality image to be corrected; Among them, based on the supervisory signal and the tooth boundary actually extracted by the first image processing model, the model parameters of the first image processing model are adjusted until the training is completed, including: calculating the loss value based on the set loss function to estimate the matching degree between the real tooth boundary and the actually extracted tooth boundary, and further adjusting the model parameters according to the matching degree using the gradient descent method.
9. A method for training an image correction model, characterized in that: include: Acquire a first modality image and a second modality image, wherein the first modality image is used to characterize tooth surface features, and the second modality image is used to characterize tooth anatomical features; Mapping the first modality image onto the second modality image to obtain a registered image; extracting the real tooth boundary from the registered image; extracting tooth boundaries from the second modality image based on a set image correction model; Based on the supervisory signal and the tooth boundaries actually extracted by the image correction model, adjusting the model parameters of the image correction model until the training is completed; Among them, based on the supervisory signal and the tooth boundary actually extracted by the first image processing model, the model parameters of the first image processing model are adjusted until the training is completed, including: calculating the loss value based on the set loss function to estimate the matching degree between the real tooth boundary and the actually extracted tooth boundary, and further adjusting the model parameters according to the matching degree using the gradient descent method.
10. An image correction method, characterized in that: include: Calling the image correction model trained by the method according to claim 9; The second modality image to be corrected is input into the image correction model for correction to start extracting tooth boundaries and removing artifacts on the second modality image to be corrected.
11. A dental treatment system, characterized in that: include: A robotic arm and a host, wherein the host is used to execute the method described in any one of claims 1 to 7 to automatically extract tooth boundaries from an anatomical feature image of a target tooth of a patient, and based on the automatically extracted tooth boundaries, control the robotic arm to perform a set target action on the target tooth of the patient.