Oral scanning data processing method and apparatus, and oral scanning system

Abstract

Disclosed in the present application are an oral scanning data processing method and apparatus, and an oral scanning system. The method comprises: obtaining scanning data collected by an intraoral scanner in real time, and tracking data collected by an optical tracker at the same time; transforming the scanning data and / or the tracking data into a unified coordinate system; on the basis of the tracking data in the unified coordinate system, determining the current pose of the intraoral scanner; and on the basis of the current pose, performing real-time reconstruction and stitching on the scanning data, so as to obtain a three-dimensional model of an oral region in the unified coordinate system. In the method, tracking data is used to determine the current pose of an intraoral scanner, and the current pose is used to assist with performing real-time reconstruction and stitching on currently collected scanning data, so as to obtain a three-dimensional model of an oral region, thereby improving the scanning experience and reducing the learning costs of a user of the intraoral scanner in terms of stitching position control and strategies, achieving the goal that the user can arbitrarily perform collection in an oral cavity at multiple angles to complete the acquisition of oral data, and improving the popularity of the intraoral scanner.

Description

Oral scanning data processing method and device and oral scanning system Cross-reference to related applications This application claims priority to the Chinese patent application No. 2024113368208, filed on September 24, 2024, with the Chinese Patent Office, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0001] The present application relates to the field of oral scanning, in particular to an oral scanning data processing method, device and oral scanning system. BACKGROUND

[0002] The acquisition device of the three-dimensional data of the teeth and gums inside the oral cavity is usually an intraoral scanner, also known as an oral digital impression device. The intraoral scanner can directly obtain the three-dimensional topographic data and color texture information of the teeth or gums, and provide data for dental restoration, implantation and orthodontic cases. Since the color texture information obtained by the intraoral scanner can intuitively and clearly reflect the internal conditions of the oral cavity, such as tooth morphology, tooth loss, gum disease and the like.

[0003] Due to the limitation of the narrow environment inside the oral cavity, the intraoral scanner for obtaining the three-dimensional data of the teeth and gums inside the oral cavity is usually designed to obtain only about 1 or 2 teeth in a single window range at a time. The single three-dimensional data is spliced by moving the intraoral scanner to collect the intraoral topographic features, and the entire three-dimensional topography and texture information inside the oral cavity is formed. However, the splicing fluency in the scanning process often depends on the common size of the frame data and the efficiency of the splicing algorithm controlled by the moving speed of the intraoral scanner. Scanning too fast can easily cause splicing loss, which can usually be spliced successfully through the common overlapping area by back splicing. The lack of intraoral three-dimensional features leads to splicing errors and the inability to obtain the three-dimensional topography of the global framework. The steps of manually distinguishing the maxilla, mandible and occlusion in the scanning process increase the learning cost of the user, among which the first two steps of scanning the maxilla and the mandible are more prone to errors. Therefore, there is currently a lack of a method that only requires the user to collect at random angles and does not need to follow many software rules or processes to complete the acquisition of the entire three-dimensional data, so as to promote the popularization and use of the intraoral scanner. SUMMARY

[0004] The present application provides an oral scanning data processing method, device and oral scanning system, which aims to reduce the operation complexity of the intraoral scanner.

[0005] A method for processing oral scanning data, comprising: obtaining scanning data collected by an intraoral scanner in real time and tracking data collected by an optical tracker at the same time; the scanning data comprising data of an oral region in a first coordinate system; the tracking data comprising data of a tracking feature in a second coordinate system; converting the scanning data and / or the tracking data to a unified coordinate system; determining a current pose of the intraoral scanner according to the tracking data in the unified coordinate system; and performing real-time reconstruction and splicing on the scanning data according to the current pose of the intraoral scanner to obtain a three-dimensional model of the oral region in the unified coordinate system.

[0006] Optionally, the method further comprises: when the scanning is completed, performing global optimization on the three-dimensional model, and displaying the three-dimensional model after the global optimization.

[0007] Optionally, the method further comprises: determining a current scanning region of the intraoral scanner; obtaining a target scanning region shown in a current scanning procedure; the current scanning procedure being a scanning procedure preset by a user before the scanning is started; and if the current scanning region is different from the target scanning region, generating a prompt notification; the prompt notification being used to remind the user to switch the current scanning procedure to a target scanning procedure matched with the current scanning region.

[0008] Optionally, the method further comprises: determining a current scanning region of the intraoral scanner; obtaining a target scanning region shown in a current scanning procedure; the current scanning procedure being a scanning procedure preset by a user before the scanning is started; and if the current scanning region is different from the target scanning region, switching the current scanning procedure to a target scanning procedure matched with the current scanning region.

[0009] Optionally, the process of determining the current scanning region of the intraoral scanner comprises one of the following: determining a current scanning pose of the intraoral scanner based on the current pose of the intraoral scanner, and determining the current scanning region of the intraoral scanner based on the current scanning pose; determining a current scanning pose of the intraoral scanner based on the current pose of the intraoral scanner, and determining the current scanning region of the intraoral scanner based on the current scanning pose and the three-dimensional model or based on the current scanning pose and the scanning data; and identifying the three-dimensional model to determine the current scanning region of the intraoral scanner.

[0010] Optionally, the tracking data further comprises pose data of a user's face in the second coordinate system; and the process of performing real-time reconstruction and splicing on the scanning data to obtain the three-dimensional model of the oral region in the unified coordinate system comprises: determining a current pose of the user's face according to the pose data of the user's face; correcting the current pose of the intraoral scanner by using the current pose of the user's face to obtain a current corrected pose of the intraoral scanner; and performing real-time reconstruction and splicing on the scanning data according to the current corrected pose of the intraoral scanner to obtain the three-dimensional model of the oral region in the unified coordinate system.

[0011] Optionally, the current pose of the intraoral scanner is corrected using the current pose of the user's face to obtain a current corrected pose of the intraoral scanner, including: determining a historical pose of the user's face based on pose data of the user's face collected by the optical tracker at a previous scanning time; calculating a pose deviation between the current pose of the user's face and the historical pose; and correcting the current pose of the intraoral scanner using the pose deviation to obtain the current corrected pose of the intraoral scanner.

[0012] An oral scanning data processing apparatus includes: a data collection unit configured to obtain scanning data collected by an intraoral scanner in real time and tracking data collected by an optical tracker at the same time; the scanning data includes data of an oral region in a first coordinate system; the tracking data includes data of a tracking feature in a second coordinate system; a data conversion unit configured to convert the scanning data and / or the tracking data to a unified coordinate system; the unified coordinate system includes the first coordinate system or the second coordinate system; a pose recognition unit configured to determine a current pose of the intraoral scanner according to the tracking data in the unified coordinate system; and a real-time reconstruction unit configured to perform real-time reconstruction and splicing on the scanning data according to the current pose of the intraoral scanner to obtain a three-dimensional model of the oral region in the unified coordinate system.

[0013] Optionally, the oral scanning data processing apparatus further includes a global optimization unit configured to perform global optimization on the three-dimensional model after scanning is completed and display the three-dimensional model after global optimization.

[0014] Optionally, the oral scanning data processing apparatus further includes a prompt notification unit configured to: determine a current scanning region of the intraoral scanner; obtain a target scanning region shown in a current scanning procedure; the current scanning procedure is a scanning procedure preset by the user before scanning starts; and generate a prompt notification if the current scanning region is different from the target scanning region; the prompt notification is used to remind the user to switch the current scanning procedure to a target scanning procedure matching the current scanning region.

[0015] Optionally, the oral scanning data processing apparatus further includes a procedure switching unit configured to: determine a current scanning region of the intraoral scanner; obtain a target scanning region shown in a current scanning procedure; the current scanning procedure is a scanning procedure preset by the user before scanning starts; and switch the current scanning procedure to a target scanning procedure matching the current scanning region if the current scanning region is different from the target scanning region.

[0016] Optionally, the process of determining the current scanning area of the intraoral scanner by the prompt notification unit or the process switching unit comprises one of the following: determining a current scanning posture of the intraoral scanner based on the current pose of the intraoral scanner, and determining the current scanning area of the intraoral scanner based on the current scanning posture; determining a current scanning posture of the intraoral scanner based on the current pose of the intraoral scanner, and determining the current scanning area of the intraoral scanner based on the current scanning posture and the three-dimensional model or based on the current scanning posture and the scanning data; and identifying the three-dimensional model to determine the current scanning area of the intraoral scanner.

[0017] Optionally, the tracking data further comprises pose data of the user's face in the second coordinate system, and the real-time reconstruction unit is specifically configured to: determine a current pose of the user's face according to the pose data of the user's face; correct the current pose of the intraoral scanner by using the current pose of the user's face to obtain a current corrected pose of the intraoral scanner; and perform real-time reconstruction and splicing on the scanning data according to the current corrected pose of the intraoral scanner to obtain the three-dimensional model of the oral cavity region in the unified coordinate system.

[0018] Optionally, the real-time reconstruction unit is specifically configured to: determine a historical pose of the user's face based on the pose data of the user's face collected by the optical tracker at the last scanning time; calculate a pose deviation between the current pose of the user's face and the historical pose; and correct the current pose of the intraoral scanner by using the pose deviation to obtain a current corrected pose of the intraoral scanner.

[0019] An oral cavity scanning system comprises an intraoral scanner, an optical tracker, and an oral cavity scanning data processing device; the intraoral scanner is configured to collect scanning data in real time; the scanning data comprises data of an oral cavity region in a first coordinate system, and the intraoral scanner is provided with a target; the optical tracker is configured to collect tracking data at the same time; the tracking data comprises data of a tracking feature in a second coordinate system; and the oral cavity scanning data processing device is configured to execute an oral cavity scanning data processing method.

[0020] Optionally, the tracking data collected by the optical tracker further comprises pose data of the user's face in the second coordinate system.

[0021] The technical scheme provided in the application obtains scanning data collected by an intraoral scanner in real time and tracking data collected by an optical tracker at the same time. The scanning data and / or the tracking data are converted to a unified coordinate system. The current pose of the intraoral scanner is determined according to the tracking data in the unified coordinate system. The scanning data is reconstructed and spliced in real time according to the current pose of the intraoral scanner, so as to obtain a three-dimensional model of the oral cavity region in the unified coordinate system. The application determines the current pose of the intraoral scanner at the same time by using the tracking data collected by the optical tracker, and uses the current pose to assist the scanning data collected at the current time to be reconstructed and spliced in real time, so as to obtain a three-dimensional model of the oral cavity region in the unified coordinate system. The scanning experience can be improved, the learning cost of a user of the intraoral scanner for splicing position control and strategy can be reduced, the goal that the user can randomly collect at multiple angles in the mouth to complete the acquisition of oral cavity data can be achieved, and the popularization rate of the intraoral scanner can be greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical schemes in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0023] Fig. 1 is a flowchart of a method for processing oral cavity scanning data according to an embodiment of the present application.

[0024] Fig. 2 is a flowchart of another method for processing oral cavity scanning data according to an embodiment of the present application.

[0025] Fig. 3 is a flowchart of another method for processing oral cavity scanning data according to an embodiment of the present application.

[0026] Fig. 4 is a flowchart of another method for processing oral cavity scanning data according to an embodiment of the present application.

[0027] Fig. 5 is a flowchart of another method for processing oral cavity scanning data according to an embodiment of the present application.

[0028] Fig. 6 is an architectural diagram of a device for processing oral cavity scanning data according to an embodiment of the present application.

[0029] Fig. 7 is an architectural diagram of an oral cavity scanning system according to an embodiment of the present application.

[0030] Fig. 8 is a display effect diagram of a three-dimensional model according to an embodiment of the present application.

[0031] Fig. 9 is a schematic diagram of the working principle of an optical tracker according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0033] In the present application, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The term “comprises”, “comprising” or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement “comprises a” does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0034] As shown in FIG. 1, a flowchart of a method for processing oral scanning data is provided in the embodiments of the present application, including the following steps.

[0035] S101: Obtain scanning data collected by an intraoral scanner in real time, and tracking data collected by an optical tracker at the same time.

[0036] The scanning data includes data of the oral cavity region in a first coordinate system, and the tracking data includes data of the tracking feature in a second coordinate system.

[0037] In some examples, the data of the oral cavity region in the first coordinate system can be understood as an image frame captured by the intraoral scanner.

[0038] Based on the working principle of the intraoral scanner, it can be known that the scanning process of the intraoral scanner is to continuously capture the teeth and gums in the oral cavity region following the movement of the lens of the intraoral scanner. Therefore, a frame of image obtained by the intraoral scanner each time can be regarded as single-frame data. Correspondingly, the tracking data collected by the optical tracker at the same time can be regarded as data of the tracking feature collected synchronously when the intraoral scanner scans each time.

[0039] The so-called tracking feature refers to the pose of the target preset on the intraoral scanner recorded by the optical tracker, which can be understood as: the optical tracker realizes tracking and positioning of the intraoral scanner through the target preset on the intraoral scanner. The data of the tracking feature in the second coordinate system can be understood as the pose data of the intraoral scanner in the second coordinate system.

[0040] In the possible implementation process, the working principle of the optical tracker tracking the intraoral scanner can be referred to FIG. 9. In addition, the number of targets preset on the body of the intraoral scanner can be set by the technician according to the actual situation, which can be set to 1, 2, 3, etc. The types of targets include but are not limited to: marker points, feature balls, and feature patterns, etc.

[0041] In some examples, the type of target preset on the intraoral scanner can be fixed, specifically, the target can be fixed on the body of the intraoral scanner, for example, the target can be a fixed plate provided with a marker point or a pattern, the fixed plate is directly fixed on the body of the intraoral scanner, or the target can be a marker point sticker, the sticker is directly pasted on the body of the intraoral scanner, or the target can be a pattern, the pattern is directly sprayed on the body of the intraoral scanner. In this way, the relative position between the target and the optical center of the intraoral scanner can be ensured to be unchanged, thereby improving the accuracy of determining the current pose of the intraoral scanner and improving the accuracy of the finally generated three-dimensional model.

[0042] In some examples, the target can also be movably fixed on the body of the intraoral scanner, or can also be rotatably fixed on the body of the intraoral scanner. In this way, the target can be adjusted in time, thereby adapting to the oral cavities of different types of users (such as adults and children), and ensuring that the target is continuously detected by the optical tracker during the scanning process.

[0043] In some examples, the pose can be understood as: position and attitude, the pose can be expressed by using a control rotation displacement relationship, for example, the pose can be represented by using an RT matrix. The RT matrix includes a rotation matrix (R) and a translation vector (T), which are combined to describe a rigid body transformation. The rotation matrix (R) is a 3x3 matrix, which is used to describe the rotation of the target. The translation vector (T) is a 3x1 vector, which is used to describe the translation of the target.

[0044] In some examples, the first coordinate system can be understood as: a coordinate system established by the camera of the intraoral scanner.

[0045] In some examples, the second coordinate system can be understood as: a coordinate system established by the binocular camera of the optical tracker.

[0046] S102: Convert the scanning data and / or tracking data to a unified coordinate system.

[0047] The unified coordinate system comprises the first coordinate system, the second coordinate system or a specified coordinate system.

[0048] It should be noted that the scanning data and the tracking data belong to different coordinate systems respectively, and therefore, the scanning data and / or the tracking data need to be converted to the unified coordinate system, so as to assist the real-time reconstruction and splicing of the scanning data by using the tracking data in the unified coordinate system.

[0049] In some examples, if the unified coordinate system is the second coordinate system, the scanning data can be converted from the first coordinate system to the second coordinate system by using a preset conversion matrix (i.e., a rotation and translation matrix) between the first coordinate system and the second coordinate system.

[0050] In some examples, if the unified coordinate system is the first coordinate system, the tracking data can be converted from the second coordinate system to the first coordinate system by using a preset conversion matrix (i.e., a rotation and translation matrix) between the first coordinate system and the second coordinate system.

[0051] In some examples, if the unified coordinate system is the specified coordinate system, the specified coordinate system can be another coordinate system different from the above-mentioned coordinate systems and predefined by a user, and the tracking data can be converted from the second coordinate system to the specified coordinate system and the scanning data can be converted from the first coordinate system to the specified coordinate system by using preset conversion matrices (i.e., rotation and translation matrices) between the specified coordinate system and the first coordinate system and the second coordinate system respectively.

[0052] S103: determining a current pose of the intraoral scanner according to the tracking data in the unified coordinate system.

[0053] Since the tracking data is the data of the tracking features corresponding to the target mark on the intraoral scanner, the current pose of the intraoral scanner can be determined according to the tracking data in the unified coordinate system.

[0054] S104: performing real-time reconstruction and splicing on the scanning data according to the current pose of the intraoral scanner to obtain a three-dimensional model of the oral cavity region in the unified coordinate system.

[0055] The current pose of the intraoral scanner can be used to assist the splicing of the scanning data, and the oral cavity region of other teeth and gums can be repositioned when the intraoral scanner scans the other teeth and gums, so that the user does not need to perform any operation to complete the reconstruction and splicing of the scanning data.

[0056] It can be understood that the method provided by the present application can assist the intraoral scanner in scanning, and the user only needs to focus on the data acquisition of the intraoral scanner, without worrying about the loss of splicing caused by too fast scanning, thereby effectively reducing the operation complexity of the intraoral scanner.

[0057] In some examples, the three-dimensional model of the oral cavity region in the unified coordinate system can be seen from FIG. 8. In the scanning process shown in FIG. 8, the intraoral scanner can only collect the position of one tooth each time. When the intraoral scanner dynamically scans the full dentition, the initial position of the tooth in the full dentition can be determined in real time by using the tracking data collected by the optical tracker, that is, by using the current pose of the intraoral scanner. The frame-by-frame splicing algorithm can be avoided in the real-time reconstruction of the scanning data, thereby reducing the operation complexity of the intraoral scanner.

[0058] It can be understood that the three-dimensional morphology acquisition (i.e., three-dimensional model scanning) of the oral cavity region by using the optical tracker in cooperation with the intraoral scanner can improve the scanning experience (the user only needs to focus on the scanning action and does not need to manually participate in the reconstruction of the scanning data), effectively reduce the cost of learning and using the intraoral scanner for the user, and the user can randomly scan at multiple angles in the oral cavity, thereby improving the popularity of the intraoral scanner.

[0059] Optionally, after the scanning is completed, the three-dimensional model is globally optimized, and the three-dimensional model after the global optimization is displayed.

[0060] It can be understood that in the real-time reconstruction of the scanning data, as the cumulative splicing of each single-frame data (i.e., image frame) in the scanning data, the error of the three-dimensional model obtained by real-time reconstruction becomes larger and larger. Therefore, the data of the oral cavity region and the data of the tracking features in the unified coordinate system are only used for initial matching. After the scanning is completed, the three-dimensional model needs to be globally and highly accurately optimized to obtain a more accurate three-dimensional model.

[0061] Compared with the existing oral scanning imaging using the intraoral scanner, the use of the optical tracker to assist the intraoral scanner in the scanning work has the following advantages: first, the user only needs to focus on the data collection of the intraoral scanner and does not need to worry about the loss of splicing caused by too fast scanning, thereby effectively reducing the operation complexity of the intraoral scanner; second, reducing the learning cost of the operator of the intraoral scanner and lowering the operation threshold of the intraoral scanner; third, reducing the rework behavior caused by the operation failure of the intraoral scanner (generally, taking the operation failure of the sudden movement of the hand by a certain distance when the intraoral scanner is scanned as an example, the operation failure will cause the scanning data to fail to obtain the overlapping area of the front and rear frames of the scanning object, resulting in the failure of the splicing of the front and rear frame data in the scanning data. Therefore, the scanning needs to be reworked or the scanning data needs to be manually reconstructed and spliced), thereby improving the efficiency of the oral scanning imaging.

[0062] In addition, when the user uses the intraoral scanner to scan the oral cavity of the patient, the patient's head can have a motion behavior, and the motion behavior of the head can cause the current scanning area of the intraoral scanner to deviate. In the embodiments of the present application, the deviation of the current scanning area can be compensated by correcting the current pose of the intraoral scanner.

[0063] It should be noted that, since the window range of the optical tracker is wide, the pose data of the user's face (i.e. the face of the patient, the patient being the object scanned by the intraoral scanner) can be collected by the optical tracker, that is, the tracking data collected by the optical tracker also includes the pose data of the user's face in the second coordinate system. The pose data of the user's face is used to correct the current pose of the intraoral scanner in real time.

[0064] Optionally, according to the current pose of the intraoral scanner, the scanning data is reconstructed and spliced in real time to obtain a three-dimensional model of the oral cavity region in a unified coordinate system. For details, please refer to the steps shown in FIG. 4 and the explanations of the steps.

[0065] The above-mentioned processes S101-S104 use the tracking data collected by the optical tracker to determine the current pose of the intraoral scanner at the same time, and use the current pose to assist the real-time reconstruction and splicing of the scanning data collected at the current time to obtain a three-dimensional model of the oral cavity region in a unified coordinate system. This can improve the scanning experience and reduce the learning cost of the user of the intraoral scanner for the splicing position control and strategy, achieve the goal that the user can randomly collect oral cavity data at multiple angles in the mouth, and greatly improve the popularity of the intraoral scanner.

[0066] It should be noted that, when the user uses the intraoral scanner to scan the oral cavity of the patient, the user usually specifies a scanning process in advance, which is used to indicate the current tooth region to be scanned in the oral cavity, such as the upper jaw region, the lower jaw region, the implant tooth region, the tooth preparation region, etc.

[0067] As shown in FIG. 2, the flowchart of another oral cavity scanning data processing method provided by the embodiments of the present application includes the following steps.

[0068] S201: Determine the current scanning area of the intraoral scanner.

[0069] The current scanning area can be understood as the oral cavity part currently scanned by the intraoral scanner. Correspondingly, the data currently scanned by the intraoral scanner can be an image frame of the current scanning oral cavity part.

[0070] In some examples, the oral cavity part is pre-divided into an upper jaw region and a lower jaw region.

[0071] In some examples, the oral cavity part is pre-divided into multiple tooth region categories, such as an upper anterior tooth region, a lower anterior tooth region, a left upper posterior molar region, a left lower posterior molar region, a right upper posterior molar region, a right lower posterior molar region, and the like.

[0072] In some examples, the oral cavity part is pre-divided into an implant tooth region or a tooth preparation region.

[0073] Optionally, the process of determining the current scanning region of the intraoral scanner includes one of the following: determining a current scanning posture of the intraoral scanner based on the current pose of the intraoral scanner, and determining the current scanning region of the intraoral scanner based on the current scanning posture; or determining a current scanning posture of the intraoral scanner based on the current pose of the intraoral scanner, and determining the current scanning region of the intraoral scanner based on the current scanning posture and the current scanning data; or determining a current scanning posture of the intraoral scanner based on the current pose of the intraoral scanner, and determining the current scanning region of the intraoral scanner based on the current scanning posture and the current three-dimensional model; or determining the current scanning region of the intraoral scanner by recognizing the three-dimensional model.

[0074] In some examples, after determining the current scanning region of the intraoral scanner, the data collected by the intraoral scanner can be marked, and according to the marking information, a specified region can be deleted in response to a user instruction or a scanning process.

[0075] It should be noted that the current scanning posture of the intraoral scanner has a corresponding relationship with the current scanning region of the intraoral scanner, and the current scanning posture of the intraoral scanner can be determined by recognizing the current pose of the intraoral scanner. Different poses of the intraoral scanner correspond to different scanning postures, and different scanning postures correspond to different current scanning regions.

[0076] In some examples, if the current scanning posture of the intraoral scanner is a reverse posture (i.e., the scanning window faces upward and the target faces downward), it can be determined that the current scanning region is the maxillary region, and if the current scanning posture of the intraoral scanner is a normal posture (i.e., the scanning window faces downward and the target faces upward), it can be determined that the current scanning region is the mandibular region. Based on this, the current scanning region can be quickly distinguished as the maxillary or mandibular region in the subsequent process, which is easy to solve the problem that the operator is prone to make mistakes when scanning the maxillary or mandibular region.

[0077] In some examples, based on the current pose of the intraoral scanner, it is determined that the current scanning posture of the intraoral scanner is toward the lower left, and then based on the current scanning posture, it is determined that the current scanning region of the intraoral scanner is the left lower posterior molar region.

[0078] In some examples, the scan data obtained by each acquisition of the intraoral scanner is reconstructed and spliced in real time, and a three-dimensional model obtained in real time can be used to represent the scanned oral cavity part in the oral cavity. By performing semantic recognition on the three-dimensional model, such as tooth position number recognition or deep learning, the current scanning area of the intraoral scanner can be obtained.

[0079] In some examples, to improve accuracy, the current scanning area of the intraoral scanner can be determined based on the current scanning posture and the current scan data or based on the current scanning posture and the current three-dimensional model. For example, semantic recognition can be performed on the real-time reconstructed three-dimensional model or a single frame of picture acquired in real time by the intraoral scanner, and the current scanning area of the intraoral scanner can be determined according to the recognized features.

[0080] For example, when the single frame of picture acquired in real time by the intraoral scanner or the real-time reconstructed three-dimensional model only contains a single arch dentition, it is determined that the intraoral scanner is performing single arch area scanning, and the current scanning area is the maxillary region or the mandibular region according to the current scanning posture of the intraoral scanner being a reverse posture or a normal posture. Further optionally, the current scanning area of the intraoral scanner can be determined to be the left lower molar region according to the current scanning posture of the intraoral scanner being toward the lower left, to be the right lower molar region according to the current scanning posture of the intraoral scanner being toward the lower right, to be the left upper molar region according to the current scanning posture of the intraoral scanner being toward the upper left, or to be the right upper molar region according to the current scanning posture of the intraoral scanner being toward the upper right.

[0081] When the single frame of picture acquired in real time by the intraoral scanner or the real-time reconstructed three-dimensional model contains a double arch dentition, it is determined that the intraoral scanner is performing full arch area scanning, and the current scanning area is the occlusal region (i.e., the tooth contact region of the maxilla and the mandible) according to the current scanning posture of the intraoral scanner being a side posture (i.e., the scanning window is toward the side and the target is toward the side). Further optionally, the current scanning area can be determined to be the left occlusal region according to the intraoral scanner being a left side posture (i.e., the scanning window is toward the right side and the target is toward the left side), or to be the right occlusal region according to the intraoral scanner being a right side posture (i.e., the scanning window is toward the left side and the target is toward the right side).

[0082] S202: Obtain a target scanning area shown in a current scanning procedure.

[0083] The current scanning procedure is a scanning procedure preset by a user before scanning starts.

[0084] It can be understood that when the user uses the intraoral scanner to perform the oral scanning, the user usually needs to preset a current scanning procedure, which is used to indicate the oral part (i.e., the target scanning area) that the user currently expects to scan, for example, to set a scanning procedure matched with the maxillary region, and the target scanning area is the maxillary region.

[0085] S203: If the current scanning area is different from the target scanning area, a prompt notification is generated.

[0086] The prompt notification is used to remind the user to switch the current scanning procedure to the target scanning procedure matched with the current scanning area.

[0087] It can be understood that if the current scanning area is different from the target scanning area, the prompt notification is generated, which can assist the user to take timely measures. For the oral scanning process, if the current scanning area is not matched with the target scanning area specified by the user due to user operation error, the user needs to be informed of the operation error details in a timely manner, so as to assist the user to correctly use the intraoral scanner to perform the oral scanning work, and effectively improve the user experience of the intraoral scanner.

[0088] It should be noted that the mismatch between the current scanning area and the target scanning area specified by the user may cause the storage location or storage file name of the three-dimensional model to be incorrect, and may also cause the model reconstruction parameters to be inappropriate, which are mostly caused by user operation error.

[0089] In some examples, if the current scanning area is the maxillary region and the target scanning area is the mandibular region, it can be determined that the user has an operation error, and a corresponding prompt notification is generated, which is used to remind the user to switch the current scanning procedure to the target scanning procedure matched with the maxillary region.

[0090] In some examples, the oral scanning procedure usually first scans the maxillary region, and the intraoral scanner is in a reverse posture (i.e., the scanning window faces upward and the target faces downward) when the maxillary region is scanned. Then, the mandibular region is scanned, and the intraoral scanner is in a normal posture (i.e., the scanning window faces downward and the target faces upward) when the mandibular region is scanned. Then, the left occlusal region in the full arch is scanned, and the intraoral scanner is in a left posture (i.e., the scanning window faces right and the target faces left) when the left occlusal region is scanned. Then, the right occlusal region in the full arch is scanned, and the intraoral scanner is in a right posture (i.e., the scanning window faces left and the target faces right) when the right occlusal region is scanned.

[0091] Therefore, generally, the dental arch data is usually divided into upper jaw, lower jaw and full jaw, and the user needs to manually define the process in the preset software interface during scanning. Once an error occurs, the upper and lower jaw data will be reversed and confused. The optical tracker can actively identify the scanning posture of the intraoral scanner, that is, the scanning data can be defined as upper jaw, lower jaw or full jaw. If the user selects the wrong process, the optical tracker can give an optimized prompt.

[0092] The above-mentioned processes S201-S203 use the current scanning area of the intraoral scanner to identify whether the current scanning operation deviates from the current scanning process. If the current scanning area is different from the target scanning area, it is considered that the current scanning operation deviates from the current scanning process, and a corresponding prompt notification is generated to remind the user to take corresponding processing measures, thereby improving the user experience of the intraoral scanner.

[0093] As shown in FIG. 3, it is a flowchart of another oral scanning data processing method provided by the embodiment of the application, which includes the following steps.

[0094] S301: Determine the current scanning area of the intraoral scanner.

[0095] The implementation process and execution principle of determining the current scanning area of the intraoral scanner can be referred to the explanation and description of the above-mentioned step S201, which will not be repeated here.

[0096] S302: Obtain the target scanning area shown in the current scanning process.

[0097] The current scanning process is a scanning process preset by the user before starting the scanning. The implementation process and execution principle of obtaining the target scanning area shown in the current scanning process can be referred to the explanation and description of the above-mentioned step S202, which will not be repeated here.

[0098] S303: If the current scanning area is different from the target scanning area, switch the current scanning process to the target scanning process matched with the current scanning area.

[0099] If the user pre-selects the operation option of automatic scanning, the user only needs to focus on the scanning operation. If the current scanning area is different from the target scanning area, the current scanning process is switched to the target scanning process matched with the current scanning area, and the user does not need to manually execute the corresponding process switching operation, which can effectively reduce the operation complexity of the intraoral scanner.

[0100] The above-mentioned processes of S301-S303 utilize the current scanning area of the intraoral scanner to identify whether the current scanning operation of the user deviates from the current scanning process. If the current scanning area is different from the target scanning area, it can be considered that the current scanning operation deviates from the current scanning process, and the current scanning process can be directly switched to the target scanning process matched with the current scanning area, to realize the full automation of the intraoral scanner, avoid the user's participation in the process switching operation, and thus effectively reduce the operation complexity of the intraoral scanner.

[0101] In some embodiments, the intraoral scanner can be handheld by the user or controlled by a robot gripper. When the intraoral scanner is controlled by the robot gripper, the robot can adjust the control instruction of the intraoral scanner according to the tracking data and the scanning data.

[0102] As shown in FIG. 4, it is a flowchart of another method for processing oral scanning data provided by an embodiment of the present application, which includes the following steps.

[0103] S401: Determine the current pose of the user's face according to the pose data of the user's face.

[0104] After the tracking data and the scanning data are converted to the unified coordinate system, the pose data of the user's face is also converted to the unified coordinate system, and accordingly, the current pose of the user's face is also converted to the unified coordinate system.

[0105] S402: Correct the current pose of the intraoral scanner by using the current pose of the user's face to obtain the current corrected pose of the intraoral scanner.

[0106] Since the current pose of the user's face and the current pose of the intraoral scanner are in the unified coordinate system, the current pose of the intraoral scanner can be corrected by using the current pose of the user's face to obtain the current corrected pose of the intraoral scanner, which can eliminate the scanning deviation caused by the movement of the patient's face.

[0107] It should be noted that the optical tracker usually has a window size that can cover all or part of the face, so the pose data of the user's face in the second coordinate system can be collected at the same time when the intraoral scanner is tracked. By using the facial feature information (which can include parts of the face with obvious features such as the tip of the nose, the corner of the eye, and the tip of the eyebrow, and can also include the marker points or targets of the face accessories), the spatial misalignment between the teeth in single frames caused by the small movement of the face can be identified. After the optical tracker obtains the current pose of the user's face, the deviation between the current pose of the user's face and the pose of the user's face collected at the last scanning time can be calculated, and the current pose of the intraoral scanner can be compensated by using the deviation to assist in the real-time reconstruction and splicing of the scanning data currently collected by the intraoral scanner, so as to obtain a more accurate three-dimensional model.

[0108] In some examples, if the current pose of the user's face deviates from the historical pose by a partial RT, the current pose of the intraoral scanner in the unified coordinate system can be corrected in synchronization according to the partial RT, and the current corrected pose of the intraoral scanner also deviates by the partial RT.

[0109] Optionally, the current pose of the intraoral scanner can be corrected using the current pose of the user's face to obtain the current corrected pose of the intraoral scanner. The implementation process can be referred to the steps shown in FIG. 5 and the explanations of the steps.

[0110] S403: Real-time reconstruction and splicing of the scanning data according to the current corrected pose of the intraoral scanner to obtain a three-dimensional model of the oral cavity region in the unified coordinate system.

[0111] The real-time reconstruction and splicing of the scanning data according to the current corrected pose of the intraoral scanner to obtain a three-dimensional model of the oral cavity region in the unified coordinate system can eliminate the modeling errors caused by scanning deviations in the three-dimensional model and improve the accuracy of the three-dimensional model.

[0112] The above-mentioned S401-S403 process uses the current pose of the user's face collected by the optical tracker to correct the current pose of the intraoral scanner to obtain the current corrected pose of the intraoral scanner, so as to eliminate the scanning deviations caused by the movement of the patient's face and effectively improve the accuracy of the three-dimensional model.

[0113] As shown in FIG. 5, another flowchart of a method for processing oral scanning data provided by an embodiment of the present application is shown, which includes the following steps.

[0114] S501: Determine a historical pose of the user's face based on the pose data of the user's face collected by the optical tracker at the last scanning time.

[0115] The pose data of the user's face collected by the optical tracker in real time can be pre-stored in the local to obtain the historical pose of the user's face in real time.

[0116] S502: Calculate the pose deviation between the current pose of the user's face and the historical pose.

[0117] The current pose of the user's face and the historical pose can both be expressed based on the RT matrix, and therefore the pose deviation between the current pose of the user's face and the historical pose can be regarded as the subtraction operation between two RT matrices.

[0118] S503: Correct the current pose of the intraoral scanner using the pose deviation to obtain the current corrected pose of the intraoral scanner.

[0119] The current pose of the intraoral scanner is corrected by using the pose deviation, so as to obtain a current corrected pose of the intraoral scanner, which can effectively eliminate the scanning deviation caused by the facial movement of the patient, avoid performing a rework behavior (such as re-splicing image frames) in the reconstruction process of the scanning data, effectively improve the reconstruction efficiency of the scanning data, and obtain a more accurate three-dimensional model.

[0120] As shown in FIG. 6, an architecture schematic diagram of an oral scanning data processing apparatus provided by an embodiment of the present application is shown, which includes the following units.

[0121] The data collection unit 100 is configured to obtain scanning data collected by the intraoral scanner in real time and tracking data collected by the optical tracker at the same time, wherein the scanning data includes data of the oral cavity region in the first coordinate system, and the tracking data includes data of the tracking feature in the second coordinate system.

[0122] The data conversion unit 200 is configured to convert the scanning data and / or the tracking data to a unified coordinate system, wherein the unified coordinate system includes the first coordinate system, the second coordinate system or a designated coordinate system.

[0123] The pose identification unit 300 is configured to determine the current pose of the intraoral scanner according to the tracking data in the unified coordinate system.

[0124] The real-time reconstruction unit 400 is configured to perform real-time reconstruction and splicing on the scanning data according to the current pose of the intraoral scanner, so as to obtain a three-dimensional model of the oral cavity region in the unified coordinate system.

[0125] The global optimization unit 500 is configured to perform global optimization on the three-dimensional model after the scanning is completed, and display the three-dimensional model after the global optimization.

[0126] The prompt notification unit 600 is configured to: determine a current scanning region of the intraoral scanner; obtain a target scanning region shown in a current scanning process; the current scanning process is a scanning process preset by a user before the scanning is started; and generate a prompt notification if the current scanning region is different from the target scanning region; the prompt notification is used to remind the user to switch the current scanning process to a target scanning process matched with the current scanning region.

[0127] The process switching unit 700 is configured to: determine a current scanning region of the intraoral scanner; obtain a target scanning region shown in a current scanning process; the current scanning process is a scanning process preset by a user before the scanning is started; and switch the current scanning process to a target scanning process matched with the current scanning region if the current scanning region is different from the target scanning region.

[0128] Optionally, the process in which the prompt notification unit 600 or the flow switching unit 700 determines the current scanning area of the intraoral scanner includes one of the following: determining a current scanning posture of the intraoral scanner based on the current pose of the intraoral scanner, and determining the current scanning area of the intraoral scanner based on the current scanning posture; determining a current scanning posture of the intraoral scanner based on the current pose of the intraoral scanner, and determining the current scanning area of the intraoral scanner based on the current scanning posture and the three-dimensional model or based on the current scanning posture and the scanning data; identifying the three-dimensional model to determine the current scanning area of the intraoral scanner.

[0129] Optionally, the tracking data further includes pose data of the user's face in the second coordinate system, and the real-time reconstruction unit 400 is specifically configured to: determine the current pose of the user's face according to the pose data of the user's face; correct the current pose of the intraoral scanner by using the current pose of the user's face to obtain a current corrected pose of the intraoral scanner; and perform real-time reconstruction and splicing on the scanning data according to the current corrected pose of the intraoral scanner to obtain the three-dimensional model of the oral cavity region in the unified coordinate system.

[0130] Optionally, the real-time reconstruction unit 400 is specifically configured to: determine a historical pose of the user's face based on the pose data of the user's face collected by the optical tracker at the last scanning time; calculate a pose deviation between the current pose of the user's face and the historical pose; and correct the current pose of the intraoral scanner by using the pose deviation to obtain a current corrected pose of the intraoral scanner.

[0131] The oral cavity scanning data processing apparatus shown above determines the current pose of the intraoral scanner at the same time by using the tracking data collected by the optical tracker, and performs real-time reconstruction and splicing on the scanning data collected at the current time by using the current pose to obtain the three-dimensional model of the oral cavity region in the unified coordinate system, which can improve the scanning experience, reduce the learning cost of the user of the intraoral scanner for the splicing position control and strategy, achieve the goal that the user can randomly collect at multiple angles in the mouth to complete the acquisition of the oral cavity data, and greatly improve the popularity of the intraoral scanner.

[0132] As shown in FIG. 7, it is an architecture schematic diagram of an oral cavity scanning system provided by an embodiment of the present application, which includes the following components.

[0133] The intraoral scanner 701, the optical tracker 702, and the oral cavity scanning data processing apparatus 703.

[0134] The intraoral scanner 701 is configured to collect scanning data in real time, and the scanning data includes data of the oral cavity region in a first coordinate system. The intraoral scanner 701 is provided with a target.

[0135] The optical tracker 702 is configured to collect tracking data including data of the tracking feature in the second coordinate system at the same time.

[0136] In addition, the tracking data collected by the optical tracker 702 further includes pose data of the face of the user in the second coordinate system.

[0137] It should be noted that the intraoral scanner 701 and the optical tracker 702 also need to perform target control operations, which are configured to synchronously control the trigger signals between the intraoral scanner 701 and the optical tracker 702, so that the data processing module 703 obtains the scanning data and the tracking data occurring at the same time.

[0138] The oral scanning data processing apparatus 703 is configured to perform the oral scanning data processing method shown in the above embodiments.

[0139] In some examples, the oral scanning data processing apparatus 703 includes, but is not limited to, a notebook computer, a tablet computer, a desktop computer, and the like.

[0140] In possible implementations, the oral scanning data processing apparatus 703 is respectively communicatively connected with the intraoral scanner 701 and the optical tracker 702.

[0141] The oral scanning system shown above, based on the combined use of the intraoral scanner and the optical tracker, in combination with the oral scanning data processing method shown in the embodiments of the present application, reduces the operation complexity of the intraoral scanner, so that the operation threshold of the intraoral scanner is effectively reduced.

[0142] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

[0143] While several inventive embodiments have been described and illustrated, it is to be understood that the embodiments herein described are merely by way of example and are not intended to be limiting. Many variations and modifications of the embodiments described herein can occur, and it is therefore the intention that encompassed variations and modifications, and well as the general principles and features described herein, be included within an embodiment of the application. Accordingly, the application is to be construed as not limited only to the embodiments described above but can encompass any and all modifications, and equivalents thereof. In addition, many modifications can be made to adapt a particular situation or material to the teachings of the application without departing from its central scope. Therefore, it is intended that this application not be limited to the particular embodiment disclosed, but that the application include all embodiments falling within the scope of the appended claims.

[0144] The above description is merely the preferred embodiments of the present application and the principle of the applied technology. It should be understood by those skilled in the art that the disclosed range of the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combinations of the above technical features or equivalent features without departing from the disclosed concept. For example, the above features can be replaced with the technical features disclosed in the present application (but not limited to) having similar functions to form technical solutions.

[0145] Industrial applicability: the oral scanning data processing method provided by the embodiments of the present application can obtain the scanning data collected by the intraoral scanner in real time and the tracking data collected by the optical tracker at the same time, which can improve the scanning experience, reduce the learning cost of the user of the intraoral scanner for the splicing position control and strategy, achieve the goal that the user can randomly collect at multiple angles in the mouth to complete the acquisition of the oral data, and greatly improve the popularity of the intraoral scanner.

Claims

1. A method for processing intraoral scanning data, comprising: obtaining scanning data collected by an intraoral scanner in real time, and tracking data collected by an optical tracker at the same time; the scanning data comprising data of a dental region in a first coordinate system; the tracking data comprising data of a tracking feature in a second coordinate system; converting the scanning data and the tracking data to a unified coordinate system; determining a current pose of the intraoral scanner according to the tracking data in the unified coordinate system; reconstructing and stitching the scanning data in real time according to the current pose of the intraoral scanner, to obtain a three-dimensional model of the dental region in the unified coordinate system.

2. The method of claim 1, further comprising: performing global optimization on the three-dimensional model after scanning is completed, and displaying the three-dimensional model after global optimization.

3. The method of claim 1 or 2, further comprising: determining a current scanning region of the intraoral scanner; obtaining a target scanning region shown in a current scanning procedure; the current scanning procedure being a scanning procedure preset by a user before scanning starts; generating a prompt notification if the current scanning region is different from the target scanning region; the prompt notification being used to remind the user to switch the current scanning procedure to a target scanning procedure matching the current scanning region.

4. The method of claim 1 or 2, further comprising: determining a current scanning region of the intraoral scanner; obtaining a target scanning region shown in a current scanning procedure; the current scanning procedure being a scanning procedure preset by a user before scanning starts; switching the current scanning procedure to a target scanning procedure matching the current scanning region if the current scanning region is different from the target scanning region.

5. The method of claim 3 or 4, wherein, The process of determining the current scanning region of the intraoral scanner comprises any one of the following: determining a current scanning pose of the intraoral scanner based on the current pose of the intraoral scanner, and determining the current scanning region of the intraoral scanner based on the current scanning pose; determining a current scanning pose of the intraoral scanner based on the current pose of the intraoral scanner, and determining the current scanning region of the intraoral scanner based on the current scanning pose and the three-dimensional model; determining a current scanning pose of the intraoral scanner based on the current pose of the intraoral scanner, and determining the current scanning region of the intraoral scanner based on the current scanning pose and the scanning data; or identifying the three-dimensional model to determine the current scanning region of the intraoral scanner. The tracking data further comprises pose data of a user's face in the second coordinate system; the method further comprises:

6. The method of any one of claims 1 to 5, wherein, determining a current pose of the user's face according to the pose data of the user's face; correcting the current pose of the intraoral scanner by using the current pose of the user's face, to obtain a current corrected pose of the intraoral scanner; ​ According to the current corrected pose of the intraoral scanner, the scan data is real-time reconstructed and spliced to obtain a three-dimensional model of the oral cavity region in the unified coordinate system.

7. The method of claim 6, wherein, The current pose of the intraoral scanner is corrected according to the current pose of the user's face to obtain a current corrected pose of the intraoral scanner, including: Based on the pose data of the user's face collected by the optical tracker at the last scanning time, a historical pose of the user's face is determined; A pose deviation between the current pose and the historical pose of the user's face is calculated; The current pose of the intraoral scanner is corrected according to the pose deviation to obtain a current corrected pose of the intraoral scanner.

8. The method of claim 1, wherein, The unified coordinate system includes the first coordinate system or the second coordinate system or a designated coordinate system.

9. An oral cavity scan data processing apparatus, comprising: a data collection unit configured to obtain scan data collected by an intraoral scanner in real time and tracking data collected by an optical tracker at the same time; The scan data includes data of an oral cavity region in a first coordinate system; the tracking data includes data of a tracking feature in a second coordinate system; a data conversion unit configured to convert the scan data and / or the tracking data to a unified coordinate system; a pose recognition unit configured to determine a current pose of the intraoral scanner according to the tracking data in the unified coordinate system; a real-time reconstruction unit configured to real-time reconstruct and splice the scan data according to the current pose of the intraoral scanner to obtain a three-dimensional model of the oral cavity region in the unified coordinate system.

10. An oral cavity scanning system, comprising: an intraoral scanner configured to collect scan data in real time; the scan data includes data of an oral cavity region in a first coordinate system, and the intraoral scanner is provided with a target; an optical tracker configured to collect tracking data at the same time; the tracking data includes data of a tracking feature in a second coordinate system; and an oral cavity scan data processing apparatus configured to perform the oral cavity scan data processing method of any one of claims 1-5 and 8.

11. The oral scanning system of claim 10, wherein, The tracking data further includes pose data of a user's face in the second coordinate system; The oral cavity scan data processing apparatus is further configured to perform the oral cavity scan data processing method of any one of claims 6-7.

Images