Smart scanning for intraoral scanners

Abstract

A method for intraoral scanning includes receiving one or more first intraoral scans of a patient's oral cavity; automatically determining a first scan role associated with the one or more first intraoral scans based on processing of the one or more first intraoral scans, wherein the first scan role is a first of an upper dental arch role, a lower dental arch role, or an occlusal role; and determining a first three-dimensional surface associated with the first scan role.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the dental field, and particularly to the use of machine learning and / or other techniques to automate the process of performing intraoral scans. Background Technology

[0002] For restorative dental work, an intraoral scanner can be used to generate one or more intraoral scans of the preparatory teeth and / or surrounding teeth on the patient's dental arch. These intraoral scans are then used to generate a virtual three-dimensional (3D) model of the tooth locations, including the preparatory teeth and surrounding teeth. For example, a virtual 3D model of the patient's dental arch can be generated. This virtual 3D model can then be sent to the laboratory. Similarly, for orthodontic dental work, intraoral scans of one or more dental arches are generated, and these scans are used to generate a virtual 3D model of said one or more dental arches and to generate a treatment plan.

[0003] Intraoral scanning involves significant user input, including manually entering patient information, selecting the patient to be scanned, selecting a segment of the dental arch to be scanned, indicating whether the scan was successful, manually entering commands to transition between stages or modes of the intraoral scan, manually selecting prescription details, and manually selecting the laboratory to send data to. Before, during, and after the intraoral scan, the user needs to operate various selections and buttons on the screen and the intraoral scanner. For example, before scanning, the user needs to fill in various prescription (Rx) selections. During scanning, the user needs to press buttons to start and stop the scan, mark areas requiring rescanning in restorative treatments, and press a button to begin post-processing when the scan is finished. Summary of the Invention

[0004] In a first aspect of this disclosure, a method includes: receiving one or more first intraoral scans of a patient's oral cavity; automatically determining a first scan role associated with the one or more first intraoral scans based on processing of the one or more first intraoral scans, wherein the first scan role is a first of an upper dental arch role, a lower dental arch role, or an occlusal role; and determining a first three-dimensional surface associated with the first scan role.

[0005] The second aspect of this disclosure may further extend the first aspect of this disclosure. In the second aspect of this disclosure, the method further includes: receiving one or more second intraoral scans of a patient's oral cavity without receiving instructions that the one or more second intraoral scans are associated with a second scan role; automatically determining a second scan role associated with the one or more second intraoral scans based on processing of the one or more second intraoral scans, wherein the second scan role is a second of an upper dental arch role, a lower dental arch role, or an occlusal role; and determining a second three-dimensional surface associated with the second scan role.

[0006] The third aspect of this disclosure may further extend the first or second aspect of this disclosure. In the third aspect of this disclosure, processing a first scan role includes inputting one or more first intraoral scans into a machine learning model that has been trained to classify intraoral scans as associated with a maxillary arch role, a mandibular arch role, or an occlusal role, wherein the machine learning model outputs a first scan role.

[0007] In a fourth aspect of this disclosure, a method includes: receiving one or more first intraoral scans of a patient's oral cavity; determining that the one or more first intraoral scans depict a first dental arch of the patient; determining a first identity of the first dental arch of the patient; and using the one or more first intraoral scans to determine a first three-dimensional surface of the first dental arch.

[0008] The fifth aspect of this disclosure may further extend the fourth aspect of this disclosure. In the fifth aspect of this disclosure, the method further includes: receiving user input instructing one or more first intraoral scans to depict a patient's second dental arch, the second dental arch having a second identity; determining that the user input is incorrect; and outputting a notification that one or more first intraoral scans depict a first dental arch with a first identity instead of a second dental arch with a second identity.

[0009] The sixth aspect of this disclosure may further extend the fourth or fifth aspect of this disclosure. In the sixth aspect of this disclosure, the method further includes: determining a first three-dimensional surface completion of the first dental arch; and, in response to determining the completion of the first dental arch, automatically generating a first three-dimensional model of the first dental arch.

[0010] The seventh aspect of this disclosure may further extend the fourth through sixth aspects of this disclosure. In the seventh aspect of this disclosure, one or more first intraoral scans of a patient's oral cavity are received without first receiving an indication of the identity of a first dental arch or an indication that a new dental arch is being scanned.

[0011] The eighth aspect of this disclosure may further extend the fourth through seventh aspects of this disclosure. In the eighth aspect of this disclosure, determining that one or more first intraoral scans depict a patient's first dental arch and determining the first identity of the patient's first dental arch includes: inputting one or more first intraoral scans into a machine learning model that has been trained to classify the intraoral scans as depicting a maxillary arch, a mandibular arch, or an occlusion, wherein the machine learning model outputs a first classification indicating the first identity of the first dental arch.

[0012] The ninth aspect of this disclosure may further extend the eighth aspect of this disclosure. In the ninth aspect of this disclosure, one or more first intraoral scans include a plurality of intraoral scans, and determining that the one or more first intraoral scans depict a patient's first dental arch and determining the first identity of the patient's first dental arch includes: inputting each of the plurality of intraoral scans into a machine learning model, wherein the machine learning model outputs a plurality of classifications, each of the plurality of classifications being associated with one of the plurality of intraoral scans; and determining that a majority of the classifications output by the machine learning model indicate the first identity of the first dental arch.

[0013] The tenth aspect of this disclosure may further extend the ninth aspect of this disclosure. In the tenth aspect of this disclosure, for at least one of the first three-dimensional surface or one or more intraoral scans in a first intraoral scan: if the tongue is depicted by at least a first threshold number of points in the first three-dimensional surface or intraoral scan, the lower dental arch is detected; if the upper palate is depicted by at least a second threshold number of points in the first three-dimensional surface or intraoral scan, the upper dental arch is detected; and if teeth from the lower dental arch are depicted by at least a third threshold number of points in the first three-dimensional surface or intraoral scan and the upper dental arch is depicted by at least a third threshold number of points in the first three-dimensional surface or intraoral scan, occlusion is detected.

[0014] The eleventh aspect of this disclosure may further extend the ninth or tenth aspect of this disclosure. In the eleventh aspect of this disclosure, one or more first intraoral scans include a plurality of intraoral scans, and wherein determining that the one or more first intraoral scans depict a patient's first dental arch and determining a first identity of the patient's first dental arch includes: inputting each of the plurality of intraoral scans into a machine learning model, wherein the machine learning model outputs a plurality of classifications, each of the plurality of classifications being associated with one of the plurality of intraoral scans; and determining a moving average of the plurality of classifications output by the machine learning model, wherein the moving average indicates a first identity of the first dental arch.

[0015] The 12th aspect of this disclosure may further extend aspects 9 through 11 of this disclosure. In the 12th aspect of this disclosure, one or more first intraoral scans include a plurality of intraoral scans received in a sequential order, wherein the one or more first intraoral scans are input into a machine learning model in that sequential order, and wherein the machine learning model is a recurrent neural network.

[0016] The 13th aspect of this disclosure may further extend aspects 9 through 12 of this disclosure. In the 13th aspect of this disclosure, for each of one or more first intraoral scans, a machine learning model outputs a confidence value, the method further comprising: for each of the one or more first intraoral scans, determining whether the confidence value associated with the output of the machine learning model for that intraoral scan is below a confidence threshold; and discarding those outputs of the machine learning model that have confidence values ​​below the confidence threshold.

[0017] The 14th aspect of this disclosure may further extend aspects 9 through 13 of this disclosure. In the 14th aspect of this disclosure, upon receiving one or more first intraoral scans and before the intraoral scan of the first dental arch is completed, the method further includes: generating a height map of the first dental arch by projecting at least a portion of a first three-dimensional surface of the first dental arch onto a plane; and processing the data from the height map using a machine learning model or an alternative machine learning model trained to classify the height map as depicting the maxillary arch, mandibular arch, or occlusion, wherein the machine learning model or the alternative machine learning model outputs a second classification indicating the first identity of the first dental arch with a higher level of accuracy compared to the first classification.

[0018] The 15th aspect of this disclosure may further extend the fourth through 14 aspects of this disclosure. In the 15th aspect of this disclosure, the method further includes: receiving a second intraoral scan depicting a first occlusal relationship between the upper and lower dental arches, the second intraoral scan having been generated at a first time; receiving a third intraoral scan depicting a second occlusal relationship between the upper and lower dental arches, the third intraoral scan having been generated at a second time; determining a first difference between the first and second occlusal relationships; determining a second difference between the first and second times; and determining, at least in part, whether the second and third intraoral scans depict the same occlusion of the patient or a different occlusion of the patient based on the first and second differences.

[0019] The 16th aspect of this disclosure may further extend aspects four through 15 of this disclosure. In the 16th aspect of this disclosure, determining that one or more first intraoral scans depict a patient's first dental arch and determining the first identity of the patient's first dental arch includes: determining whether a first three-dimensional surface or at least one of the one or more first intraoral scans includes a representation of the tongue or palate; and in response to determining that the first three-dimensional surface or at least one of the one or more first intraoral scans includes a representation of the tongue, determining that the first identity of the patient's first dental arch is mandibular; or in response to determining that the first three-dimensional surface or at least one of the one or more first intraoral scans includes a representation of the palate, determining that the first identity of the patient's first dental arch is maxillary.

[0020] The 17th aspect of this disclosure may further extend the fourth through 16 aspects of this disclosure. In the 17th aspect of this disclosure, determining that one or more first intraoral scans depict a patient's first dental arch and determining the first identity of the patient's first dental arch includes: determining whether at least one of the one or more first intraoral scans generated before the intraoral scanner is inserted into the patient's oral cavity depicts a nose or a chin; and in response to determining that at least one of the one or more first intraoral scans includes a representation of a chin, determining that the first identity of the patient's first dental arch is for the lower dental arch; or in response to determining that at least one of the one or more first intraoral scans includes a representation of a nose, determining that the first identity of the patient's first dental arch is for the upper dental arch.

[0021] The 18th aspect of this disclosure may further extend the 17th aspect of this disclosure. In the 18th aspect of this disclosure, the method further includes: detecting rotation of the intraoral scanner about its longitudinal axis after generating one or more first intraoral scans, based on data from an inertial measurement unit of an intraoral scanner that generates one or more first intraoral scans; receiving one or more second intraoral scans of a patient's oral cavity after the intraoral scanner has rotated about its longitudinal axis; determining that the one or more second intraoral scans depict the inferior dental arch if the first dental arch is a superior dental arch; and determining that the one or more second intraoral scans depict the superior dental arch if the first dental arch is a inferior dental arch.

[0022] The 19th aspect of this disclosure may further extend aspects four through 18 of this disclosure. In the 19th aspect of this disclosure, determining one or more first intraoral scans depicting a patient's first dental arch and determining the first identity of the patient's first dental arch includes: generating an image of the first dental arch, the image including a height map; and processing data from the image using a machine learning model that has been trained to classify the image of the dental arch as depicting a maxillary arch, mandibular arch, or occlusion, wherein the machine learning model outputs a classification indicating the first identity of the first dental arch.

[0023] The 20th aspect of this disclosure may further extend the 19th aspect of this disclosure. In the 20th aspect of this disclosure, a first three-dimensional surface is generated before determining the first identity of the first dental arch, and wherein an image of the first dental arch is generated by projecting at least a portion of the first three-dimensional surface of the first dental arch onto a two-dimensional surface.

[0024] The 21st aspect of this disclosure may further extend the fourth through 20th aspects of this disclosure. In the 21st aspect of this disclosure, the method further includes: marking one or more first intraoral scans as belonging to a first segment of a first dental arch; receiving one or more second intraoral scans of a patient's oral cavity; determining that the one or more second intraoral scans depict the first dental arch of a patient with a first identity; and marking one or more second intraoral scans as belonging to a second segment of the first dental arch.

[0025] The 22nd aspect of this disclosure may further extend the fourth through 21 aspects of this disclosure. In the 22nd aspect of this disclosure, the method further includes: determining whether one or more first intraoral scans depict a lingual view, a buccal view, or an occlusal view of a first dental arch.

[0026] The 23rd aspect of this disclosure may further extend the first through 22 aspects of this disclosure. In the 23rd aspect of this disclosure, a computer-readable medium includes instructions that, when executed by a processing apparatus, cause the processing apparatus to perform the method of any one of the first through 22 aspects of this disclosure.

[0027] The 24th aspect of this disclosure may further extend the first through 22 aspects of this disclosure. In the 24th aspect of this disclosure, a system includes: an intraoral scanner for generating the one or more intraoral scans; and a computing device connected to the intraoral scanner via a wired or wireless connection, the computing device being used to perform the methods of any one of the first through 22 aspects of this disclosure.

[0028] In a 25th aspect of this disclosure, a method includes: receiving one or more first intraoral scans of a patient's oral cavity; processing the one or more first intraoral scans; determining, based on the processing of the one or more first intraoral scans, a first of an upper or lower dental arch depicted in the one or more first intraoral scans; automatically generating a first three-dimensional surface of the first of the upper or lower dental arches using the one or more first intraoral scans; receiving one or more second intraoral scans of the patient's oral cavity; processing the one or more second intraoral scans; determining, based on the processing of the one or more second intraoral scans, a second of an upper or lower dental arch depicted in the one or more second intraoral scans; and automatically generating a second three-dimensional surface of the second of the upper or lower dental arches using the one or more second intraoral scans.

[0029] The 26th aspect of this disclosure may further extend the 25th aspect of this disclosure. In the 26th aspect of this disclosure, the method further includes: receiving one or more third intraoral scans of a patient's oral cavity; processing the one or more third intraoral scans; and determining, based on the processing of the one or more third intraoral scans, that a patient's occlusion is depicted in the one or more third intraoral scans.

[0030] The 27th aspect of this disclosure may further extend the 25th or 26th aspects of this disclosure. In the 27th aspect of this disclosure, when one or more first intraoral scans are received, a first three-dimensional surface is generated, the method further comprising: automatically determining that the user has transitioned from scanning the first of the upper or lower dental arches to scanning the second of the upper or lower dental arches; and switching from generating the first three-dimensional surface to generating the second three-dimensional surface in response to determining that one or more second intraoral scans depict the second of the upper or lower dental arches.

[0031] The 28th aspect of this disclosure may further extend aspects 25 to 27 of this disclosure. In the 28th aspect of this disclosure, a computer-readable medium includes instructions that, when executed by a processing apparatus, cause the processing apparatus to perform the method of any one of aspects 25 to 27 of this disclosure.

[0032] The 29th aspect of this disclosure may further extend aspects 25 through 27 of this disclosure. In aspect 29 of this disclosure, a system includes: an intraoral scanner for generating one or more first intraoral scans and one or more second intraoral scans; and a computing device connected to the intraoral scanner via a wired or wireless connection, the computing device performing the methods of any one of aspects 25 through 27 of this disclosure.

[0033] In a 30th aspect of this disclosure, a method includes: receiving a plurality of first intraoral scans of a dental arch; automatically determining, based on processing of the plurality of first intraoral scans, that the plurality of first intraoral scans depict a restorative dental object; determining a first resolution for a first portion of a three-dimensional model of the dental arch generated from the plurality of first intraoral scans; receiving a plurality of second intraoral scans of the dental arch; automatically determining, based on processing of the plurality of second intraoral scans, that the plurality of second intraoral scans failed to depict a restorative dental object; determining a second resolution for a second portion of a three-dimensional model of the dental arch generated from the plurality of second intraoral scans; and generating a three-dimensional model of the dental arch having the first portion and the second portion, wherein the first portion includes a depiction of the restorative dental object and is generated from the plurality of first intraoral scans and has the first resolution, and wherein the second portion is generated from the plurality of second intraoral scans and has the second resolution, wherein the first resolution is greater than the second resolution.

[0034] The 31st aspect of this disclosure may further extend the 30th aspect of this disclosure. In the 31st aspect of this disclosure, the three-dimensional model includes a single variable-resolution three-dimensional surface, wherein a first portion of the single variable-resolution three-dimensional surface has a first resolution, and a second portion of the variable-resolution three-dimensional surface has a second resolution.

[0035] The 32nd aspect of this disclosure may further extend the 30th or 31st aspects of this disclosure. In the 32nd aspect of this disclosure, the method further includes: receiving a plurality of first intraoral scans and a plurality of second intraoral scans based on uninterrupted continuous scanning of the dental arch, wherein no user input instructing a transition from a scan of the dental arch to a scan of the restorative dental object or instructing a transition from a scan of the restorative dental object to a scan of the dental arch is received.

[0036] The 33rd aspect of this disclosure may further extend aspects 30 through 32 of this disclosure. In the 33rd aspect of this disclosure, automatically determining that a plurality of first intraoral scans depict a restorative dental object based on processing of a plurality of first intraoral scans includes: processing data from the plurality of first intraoral scans using a trained machine learning model trained to identify restorative dental objects, wherein, for each intraoral scan, the trained machine learning model generates an output classifying the intraoral scan as containing or not containing a restorative dental object.

[0037] The 34th aspect of this disclosure may further extend aspects 30 through 33 of this disclosure. In the 34th aspect of this disclosure, for each intraoral scan, the trained machine learning model outputs a schematic diagram that includes an indication for each pixel in the intraoral scan regarding whether the pixel depicts a restorative dental object.

[0038] The 35th aspect of this disclosure may further extend aspects 30 to 34 of this disclosure. In the 35th aspect of this disclosure, the method further includes: determining a first region of the second portion that depicts a tooth-to-gingival boundary or a tooth-to-tooth boundary; determining a second region of the second portion that fails to depict a tooth-to-gingival boundary or a tooth-to-tooth boundary; and updating the three-dimensional model so that the second portion of the second region has a third resolution lower than the second resolution.

[0039] The 36th aspect of this disclosure may further extend aspects 30 to 35 of this disclosure. In aspect 35 of this disclosure, the method further includes: automatically determining, based on processing of a plurality of first intraoral scans, whether the plurality of first intraoral scans depict the maxillary arch, mandibular arch, or occlusion; and automatically determining, based on processing of a plurality of second intraoral scans, whether the plurality of second intraoral scans depict the maxillary arch, mandibular arch, or occlusion.

[0040] The 37th aspect of this disclosure may further extend aspects 30 to 36 of this disclosure. In the 37th aspect of this disclosure, a computer-readable medium includes instructions that, when executed by a processing apparatus, cause the processing apparatus to perform the method of any one of aspects 30 to 36 of this disclosure.

[0041] The 38th aspect of this disclosure may further extend aspects 30 to 36 of this disclosure. In aspect 38 of this disclosure, a system includes: an intraoral scanner for generating a plurality of first intraoral scans and a plurality of second intraoral scans; and a computing device connected to the intraoral scanner via a wired or wireless connection, the computing device performing the method of any one of aspects 30 to 36 of this disclosure.

[0042] In a 39th aspect of this disclosure, a method includes: receiving one or more intraoral scans of a patient's oral cavity; processing an input including data from the one or more intraoral scans using a trained machine learning model trained to classify tooth locations represented in the intraoral scans, wherein the trained machine learning model generates an output including one or more dental classifications, the one or more dental classifications including indications as to whether the one or more intraoral scans include a depiction of one or more types of restorative dental objects; determining, based on the dental classifications output by the trained machine learning model, that the one or more intraoral scans depict a restorative dental object; and using at least a portion of the one or more intraoral scans to determine a three-dimensional surface of the restorative dental object.

[0043] The 40th aspect of this disclosure may further extend the 39th aspect of this disclosure. In the 40th aspect of this disclosure, the trained machine learning model outputs a schematic diagram that includes an indication for each pixel in an intraoral scan regarding whether that pixel depicts a restorative dental object.

[0044] The 41st aspect of this disclosure may further extend the 39th or 40th aspects of this disclosure. In the 41st aspect of this disclosure, the one or more types of restorative dental objects include preparations, scanners, and dental implants, and wherein the one or more dental classifications include preparation classification, scanner classification, and dental implant classification.

[0045] The 42nd aspect of this disclosure may further extend the 41st aspect of this disclosure. In the 42nd aspect of this disclosure, the trained machine learning model outputs a probability map that, for each pixel in the intraoral scan, includes at least one of a first probability that the pixel depicts a preparation object, a second probability that the pixel depicts a scan body, or a third probability that the pixel depicts a dental implant.

[0046] The 43rd aspect of this disclosure may further extend the 42nd aspect of this disclosure. In the 43rd aspect of this disclosure, the probability map for each pixel in the one or more intraoral scans further includes at least one of the following: a probability that the pixel belongs to a dental category representing gingiva; a probability that the pixel belongs to a dental category representing dental attachments; a probability that the pixel belongs to a dental category representing brackets on teeth; or a probability that the pixel belongs to a dental category representing excess material, which includes material other than teeth, gingiva, the scanning body, or dental implants.

[0047] The 44th aspect of this disclosure may further extend aspects 41 through 43 of this disclosure. In the 44th aspect of this disclosure, the trained machine learning model is capable of distinguishing between multiple different types of scan bodies, and wherein, for each of the multiple different types of scan bodies, the output of the trained machine learning model includes the probability that the one or more intraoral scans include a depiction of that type of scan body.

[0048] The 45th aspect of this disclosure may further extend aspects 39 to 44 of this disclosure. In the 45th aspect of this disclosure, upon receiving the one or more intraoral scans, the one or more intraoral scans are processed during an intraoral scan session, and additional intraoral scans are generated simultaneously.

[0049] The 46th aspect of this disclosure may further extend aspects 39 through 45 of this disclosure. In the 46th aspect of this disclosure, the trained machine learning model divides the one or more intraoral scans into multiple regions, and wherein, for each region, the output includes an indication of whether the region contains a depiction of a restorative dental object.

[0050] The 47th aspect of this disclosure can further extend aspects 39 to 46 of this disclosure. In the 47th aspect of this disclosure, the method further includes: determining a central region of the one or more intraoral scans, wherein data from the one or more intraoral scans includes data from the central region and excludes data outside the central region.

[0051] The 48th aspect of this disclosure may further extend aspects 39 through 47 of this disclosure. In the 48th aspect of this disclosure, the method further includes: receiving one or more color images of a patient's oral cavity, wherein the one or more color images are associated with an intraoral scan in one or more intraoral scans and are captured by an intraoral scanner at substantially the same position and orientation as the intraoral scan; and generating input for the trained machine learning model, the input including data from the intraoral scans and data from the one or more color images, wherein the trained machine learning model uses the data from the one or more color images and the data from the intraoral scans to generate the output.

[0052] The 49th aspect of this disclosure can be further extended to aspects 39 through 48 of this disclosure. In the 49th aspect of this disclosure, the method further includes: receiving an additional image generated under illumination conditions in which at least one of infrared or ultraviolet light is used to illuminate the patient's oral cavity, wherein the additional image is associated with an intraoral scan in one or more intraoral scans and is captured by an intraoral scanner at substantially the same position and orientation as the intraoral scan; and generating input for a trained machine learning model, the input including data from the intraoral scan and data from the additional image, wherein the trained machine learning model uses the data from the additional image and the data from the intraoral scan to generate an output.

[0053] The 50th aspect of this disclosure may further extend the 49th aspect of this disclosure. In the 50th aspect of this disclosure, the one or more dental classifications also include at least one of an indication of whether the one or more intraoral scans include a depiction of real teeth or an indication of whether the one or more intraoral scans include a depiction of artificial teeth.

[0054] The 51st aspect of this disclosure may further extend aspects 39 to 50 of this disclosure. In the 51st aspect of this disclosure, the method further includes: receiving a plurality of additional intraoral scans depicting a restorative dental object, wherein the one or more intraoral scans and the plurality of additional intraoral scans are used to determine a three-dimensional surface of the restorative dental object; generating a height map by projecting the three-dimensional surface onto a plane; and processing the height map using the trained machine learning model or an additional trained machine learning model trained to identify the restorative dental object in the height map, wherein the trained machine learning model generates an output including an indication of whether the height map includes a depiction of the restorative dental object.

[0055] The 52nd aspect of this disclosure may further extend the 51st aspect of this disclosure. In the 52nd aspect of this disclosure, a height diagram depicts an occlusal view of the dental arch including a restorative dental object.

[0056] The 53rd aspect of this disclosure may further extend the 52nd aspect of this disclosure. In the 53rd aspect of this disclosure, the method further includes: determining the position of a tooth on the dental arch in which the restorative dental object is located; and marking the restorative object using the determined tooth position in a three-dimensional surface.

[0057] The 54th aspect of this disclosure may further extend aspects 39 through 53 of this disclosure. In the 54th aspect of this disclosure, the one or more intraoral scans are received without first receiving user input indicating that a restorative object will be scanned.

[0058] The 55th aspect of this disclosure may further extend aspects 39 to 54 of this disclosure. In the 55th aspect of this disclosure, the method further includes: prompting a user to identify the position of a tooth on the dental arch in which the restorative dental object is located.

[0059] The 56th aspect of this disclosure may further extend aspects 39 to 55 of this disclosure. In the 56th aspect of this disclosure, a computer-readable medium includes instructions that, when executed by a processing apparatus, cause the processing apparatus to perform the method of any one of aspects 39 to 55 of this disclosure.

[0060] The 57th aspect of this disclosure can be further extended to aspects 39 through 55 of this disclosure. In the 57th aspect of this disclosure, a system includes: an intraoral scanner for generating the one or more intraoral scans; and a computing device connected to the intraoral scanner via a wired or wireless connection, the computing device performing the method of any one of aspects 39 through 55 of this disclosure.

[0061] In aspect 58 of this disclosure, a method includes: determining a first three-dimensional surface of at least a portion of a dental arch using a plurality of first intraoral scans generated by an intraoral scanner at a first time; determining that the first three-dimensional surface depicts at least a portion of a prepared tooth or at least a portion of a region surrounding the prepared tooth; receiving one or more additional intraoral scans of the dental arch generated by the intraoral scanner at a second time; determining that the one or more additional intraoral scans depict at least a portion of the prepared tooth or a portion of a region surrounding the prepared tooth; determining a time difference between the first time and the second time; determining a time difference between the first three-dimensional surface and the one or more additional intraoral scans for at least one of the prepared tooth or the region surrounding the prepared tooth. Variation; determining, at least in part, based on the time difference and variation in at least one of the prepared tooth or the area surrounding the prepared tooth, whether to use a) a first three-dimensional surface, b) data from the one or more additional intraoral scans, or c) a combination of the first three-dimensional surface and data from the one or more additional intraoral scans to depict a portion of the prepared tooth or the area surrounding the prepared tooth; and generating a three-dimensional model of the dental arch, wherein a) the first three-dimensional surface, b) data from the one or more additional intraoral scans, or c) a combination of the first three-dimensional surface and data from the one or more additional intraoral scans are used to depict a portion of the prepared tooth or a portion of the area surrounding the prepared tooth in the three-dimensional model.

[0062] The 59th aspect of this disclosure may further extend the 58th aspect of this disclosure. In the 59th aspect of this disclosure, the one or more additional intraoral scans include a plurality of second intraoral scans, and the method further includes: using the plurality of second intraoral scans to determine a second three-dimensional surface of at least a portion of a dental arch; determining that the second three-dimensional surface includes a representation of at least a portion of a prepared tooth or a portion of a region surrounding the prepared tooth; and determining a variation on at least one of the prepared tooth or the region surrounding the prepared tooth based on a comparison of the first three-dimensional surface and the second three-dimensional surface; wherein determining whether to depict a portion of the prepared tooth or a portion of the region surrounding the prepared tooth using a) the first three-dimensional surface, b) data from the one or more additional intraoral scans, or c) a combination of the first three-dimensional surface and data from the one or more additional intraoral scans includes determining whether to depict a portion of the prepared tooth or a portion of the region surrounding the prepared tooth using a) the first three-dimensional surface, b) the second three-dimensional surface, or c) a combination of the first three-dimensional surface and the second three-dimensional surface; and wherein a) the first three-dimensional surface, b) the second three-dimensional surface, or c) a combination of the first three-dimensional surface and the second three-dimensional surface is used to depict a portion of the prepared tooth or a portion of the region surrounding the prepared tooth in a three-dimensional model.

[0063] The 60th aspect of this disclosure may further extend the 59th aspect of this disclosure. In the 60th aspect of this disclosure, the method further includes: determining that the time difference exceeds a time difference threshold; determining that the change in the prepared tooth exceeds a change threshold; and determining whether to use a) a first three-dimensional surface or b) a second three-dimensional surface based on the time difference exceeding the time difference threshold and the change in at least one of the prepared tooth or the surrounding area of ​​the prepared tooth exceeding the change threshold.

[0064] The 61st aspect of this disclosure may further extend the 59th or 60th aspects of this disclosure. In the 61st aspect of this disclosure, the method further includes: determining that the second time is at least a threshold time amount following the first time; identifying a retraction cord depicted in a first three-dimensional surface; determining that the second three-dimensional surface does not include a representation of the retraction cord; and determining an area where the second three-dimensional surface, rather than the first three-dimensional surface, is used to expose the marginal line of the prepared tooth for insertion and subsequent removal of the retraction cord.

[0065] The 62nd aspect of this disclosure may further extend aspects 59 through 61 of this disclosure. In the 62nd aspect of this disclosure, the method further includes: determining a second time after the first time; determining that the prepared tooth comprises less material in a portion of the prepared tooth from a second three-dimensional surface than in a portion of the prepared tooth from a first three-dimensional surface; and determining that the second three-dimensional surface is used for a portion of the prepared tooth.

[0066] The 63rd aspect of this disclosure can further extend aspects 59 through 62 of this disclosure. In the 63rd aspect of this disclosure, the second time is later than the first time, and the method further includes: receiving audio data that has been generated at a third time between the first and second times; determining that the audio data includes a sound associated with a dental drill; and determining the use of a second three-dimensional surface.

[0067] The 64th aspect of this disclosure may further extend aspects 59 through 63 of this disclosure. In the 64th aspect of this disclosure, the method further includes: determining an inertial state of the intraoral scanner between the generation of a plurality of first intraoral scans and the generation of a plurality of second intraoral scans, based on inertial measurement data for the intraoral scanner; wherein the inertial state of the intraoral scanner is used to determine whether to use a) a first three-dimensional surface, b) a second three-dimensional surface, or c) a combination of the first and second three-dimensional surfaces to depict a portion of the prepared tooth or a portion of the area surrounding the prepared tooth.

[0068] The 65th aspect of this disclosure may further extend aspects 59 through 64 of this disclosure. In the 65th aspect of this disclosure, the method further includes: identifying fluid obscuring a portion of the prepared tooth in a second three-dimensional surface based on processing the one or more additional intraoral scans using at least one of color image processing or a trained machine learning model; and determining to use a first three-dimensional surface instead of a second three-dimensional surface to depict the portion of the prepared tooth.

[0069] The 66th aspect of this disclosure may further extend the 65th aspect of this disclosure. In the 66th aspect of this disclosure, the liquid includes at least one of blood or saliva.

[0070] The 67th aspect of this disclosure may further extend aspects 59 to 66 of this disclosure. In the 67th aspect of this disclosure, the method further includes: outputting to a display an indication of whether a) a first three-dimensional surface, b) a second three-dimensional surface, or c) a combination of the first and second three-dimensional surfaces is determined to be used to depict a portion of the prepared tooth or a portion of the surrounding region of the prepared tooth; receiving user input indicating whether it is incorrect to use a) the first three-dimensional surface, b) the second three-dimensional surface, or c) a combination of the first and second three-dimensional surfaces to depict a portion of the prepared tooth or a portion of the surrounding region of the prepared tooth, wherein the user input indicates that one of a) the first three-dimensional surface, b) the second three-dimensional surface, or c) a combination of the first and second three-dimensional surfaces is correct for depicting a portion of the prepared tooth or a portion of the surrounding region of the prepared tooth; and updating a three-dimensional model of the dental arch using the correct one of a) the first three-dimensional surface, b) the second three-dimensional surface, or c) a combination of the first and second three-dimensional surfaces to depict a portion of the prepared tooth or a portion of the surrounding region of the prepared tooth.

[0071] The 68th aspect of this disclosure may further extend aspects 59 to 67 of this disclosure. In the 68th aspect of this disclosure, the method further includes: dividing a first three-dimensional surface into gingiva and one or more teeth, wherein one of the one or more teeth is a prepared tooth; and dividing a second three-dimensional surface into gingiva and one or more additional teeth, wherein one of the one or more additional teeth is a prepared tooth.

[0072] The 69th aspect of this disclosure may further extend aspects 58 to 68 of this disclosure. In the 69th aspect of this disclosure, the method further includes: outputting to a display an indication as to whether a) a first version of a three-dimensional surface, b) data from the one or more additional intraoral scans, or c) a combination of the first version of the three-dimensional surface and data from the one or more additional intraoral scans is determined to be used to depict a portion of the prepared tooth or a portion of the area surrounding the prepared tooth.

[0073] The 70th aspect of this disclosure may further extend aspects 58 to 69 of this disclosure. In the 70th aspect of this disclosure, the method further includes: determining that an intraoral scan of the prepared tooth is complete; automatically determining the contour of the edge line of the prepared tooth; and highlighting the contour of the edge line on a three-dimensional model.

[0074] The 71st aspect of this disclosure may further extend aspects 58 through 70 of this disclosure. In the 71st aspect of this disclosure, the method further includes: automatically processing data from a three-dimensional model to identify regions recommended for additional intraoral scans; and notifying a user to generate one or more additional intraoral scans depicting the region.

[0075] The 72nd aspect of this disclosure may further extend the 71st aspect of this disclosure. In the 72nd aspect of this disclosure, automatically processing data from a three-dimensional model to identify regions recommended for additional intraoral scanning includes: for a tooth represented in the three-dimensional model, determining the amount of imaged gingival tissue surrounding the tooth; and determining that the amount of imaged gingival tissue surrounding the tooth is less than a threshold in that region.

[0076] The 73rd aspect of the invention can further extend the 71st and 72nd aspects of the invention. In the 73rd aspect of this disclosure, the regions recommended for additional intraoral scanning include at least one of the following: missing palatal regions, unscanned teeth, incomplete scans of teeth, gaps in tooth scans, unclear edge lines, or regions with insufficient color information.

[0077] The 74th aspect of this disclosure may further extend aspects 58 through 73 of this disclosure. In the 74th aspect of this disclosure, the method further includes: after generating a three-dimensional model of the dental arch, generating a trajectory of a virtual camera showing the three-dimensional model of the dental arch from multiple view settings and multiple zoom settings, wherein one or more magnified views of the prepared teeth are included in the trajectory; and automatically executing the trajectory to display the three-dimensional model from the multiple view settings and the multiple zoom settings.

[0078] The 75th aspect of this disclosure may further extend the 74th aspect of this disclosure. In the 75th aspect of this disclosure, the method further includes: determining the trajectory of a virtual camera based on one or more scaling operations and one or more rotation operations manually performed by a user on one or more prior 3D models of the dental arch.

[0079] The 76th aspect of this disclosure may further extend aspects 58 to 75 of this disclosure. In the 76th aspect of this disclosure, a computer-readable medium includes instructions that, when executed by a processing apparatus, cause the processing apparatus to perform the method of any one of aspects 58 to 75 of this disclosure.

[0080] The 77th aspect of this disclosure may further extend aspects 58 through 75 of this disclosure. In the 77th aspect of this disclosure, a system includes: an intraoral scanner for generating a plurality of first intraoral scans and the one or more additional intraoral scans; and a computing device connected to the intraoral scanner via a wired or wireless connection, the computing device performing the methods of any one of aspects 58 through 75 of this disclosure.

[0081] In a 78th aspect of this disclosure, a method for automatically generating a prescription for treating one or more teeth in a patient's dental arch includes: receiving a plurality of intraoral scans of the patient generated by an intraoral scanner; using the plurality of intraoral scans to determine a three-dimensional surface of at least a portion of one or more of the patient's dental arches; automatically determining whether a restorative dental object is represented on the three-dimensional surface or in at least one of the plurality of intraoral scans; and automatically generating a prescription for treating the one or more teeth based at least in part on at least one of a) the presence or absence of a restorative dental object on the three-dimensional surface or in at least one of the one or more intraoral scans or b) the location of the restorative dental object in the patient's one or more dental arches.

[0082] The 79th aspect of this disclosure may further extend the 78th aspect of this disclosure. In the 79th aspect of this disclosure, the method further includes: comparing a three-dimensional surface of at least a portion of the patient's one or more dental arches with a previously generated three-dimensional surface of the patient's one or more dental arches; determining one or more differences between the three-dimensional surface and the previously generated three-dimensional surface based on the result of the comparison; and determining the one or more teeth to be treated based on the one or more differences.

[0083] The 80th aspect of this disclosure may further extend the 79th aspect of this disclosure. In the 80th aspect of this disclosure, the restorative dental object is a prepared tooth, and the patient's multiple intraoral scans depict the prepared tooth, the method further comprising: generating a first portion of a three-dimensional model of the one or more dental arches including the prepared tooth using a three-dimensional surface; and generating the remainder of the three-dimensional model using the previously generated three-dimensional surface.

[0084] The 81st aspect of this disclosure may further extend the 79th or 80th aspects of this disclosure. In the 81st aspect of this disclosure, the restorative dental object is a prepared tooth, and wherein the one or more teeth to be treated include a prepared tooth, the method further comprising: automatically determining, at least in part, an outer surface of a crown to be placed on the prepared tooth based on a portion of a previously generated three-dimensional surface depicting the tooth before it is polished to become a prepared tooth; and automatically determining, at least in part, an inner surface of the crown based on a representation of the prepared tooth in the three-dimensional surface.

[0085] The 82nd aspect of this disclosure may further extend aspects 79 through 81 of this disclosure. In the 82nd aspect of this disclosure, the restorative dental object is a prepared tooth, and the one or more teeth to be treated include the prepared tooth, the method further comprising: determining, based on the result of the comparison, one or more altered regions of the prepared tooth between a three-dimensional surface and a previously generated three-dimensional surface, wherein the one or more differences are included in the one or more altered regions; determining, based on the result of the comparison, one or more unaltered regions of the prepared tooth between the three-dimensional surface and the previously generated three-dimensional surface; determining the boundary between the one or more unaltered regions and the one or more altered regions; and automatically determining an edge line at least in part based on the boundary.

[0086] This 83rd aspect of the disclosure may further extend aspects 78 through 82 of the disclosure. In aspect 83 of the disclosure, the restorative dental object includes a scanning body, a dental implant, or a prepared tooth.

[0087] The 84th aspect of this disclosure may further extend aspects 78 through 83 of this disclosure. In the 84th aspect of this disclosure, the method further includes: dividing a three-dimensional surface of at least said portion of the one or more dental arches into a plurality of individual teeth; determining a tooth number for each of the one or more teeth to be treated; and automatically adding the tooth number for each of the one or more teeth to be treated to a prescription.

[0088] The 85th aspect of this disclosure may further extend aspects 78 through 84 of this disclosure. In the 85th aspect of this disclosure, the method further includes: determining the type of dental prosthesis for treating the one or more teeth; and adding an identifier of the type of dental prosthesis to a prescription.

[0089] The 86th aspect of this disclosure may further extend the 85th aspect of this disclosure. In the 86th aspect of this disclosure, determining the type of dental prosthesis includes: determining, at least in part, whether an inlay, onlay-only, crown, denture, veneer, or bridge is suitable for treating one or more teeth based on the geometry of the prepared material.

[0090] The 87th aspect of this disclosure may further extend aspects 85 and 86 of this disclosure. In the 87th aspect of this disclosure, the method further includes: identifying the dentist treating the patient; determining a recommended dental laboratory to which the prescription will be sent, based on at least one of the preparation material, the type of dental prosthesis, or historical statistics of dental laboratories used by the dentist; and adding the recommended dental laboratory to the prescription.

[0091] The 88th aspect of this disclosure may further extend the 87th aspect of this disclosure. In the 88th aspect of this disclosure, the method further includes: determining at least one of a) historical statistics of materials used by a dentist for dental prostheses, b) historical statistics of materials used by a recommended dental laboratory for dental prostheses, or c) materials available at a recommended dental laboratory; selecting a material for the dental prosthesis based on at least one of a) historical statistics of materials used by a dentist for dental prostheses, b) historical statistics of materials used by a recommended dental laboratory for dental prostheses, or c) materials available at a recommended dental laboratory; and adding the selected material to a prescription.

[0092] The 89th aspect of this disclosure may further extend aspects 87 and 88 of this disclosure. In the 89th aspect of this disclosure, the method further includes: receiving a color image of a dental arch, the color image having been generated by an intraoral scanner; determining the color of teeth adjacent to the preparation based on the color image; determining the color of a dental prosthesis based at least in part on the color of the teeth adjacent to the preparation; and automatically adding the color of the dental prosthesis to a prescription.

[0093] The 90th aspect of this disclosure may further extend aspects 78 to 89 of this disclosure. In the 90th aspect of this disclosure, the method further includes: receiving one or more two-dimensional images generated by an intraoral scanner; determining whether the one or more two-dimensional images depict the interior of the oral cavity; and, without user input, causing the intraoral scanner to begin generating the plurality of intraoral scans or to stop generating intraoral scans based on whether the one or more two-dimensional images depict the interior of the oral cavity.

[0094] The 91st aspect of this disclosure may further extend the 90th aspect of this disclosure. In the 91st aspect of this disclosure, determining that the one or more two-dimensional images depict the interior of a mouth includes: inputting the one or more two-dimensional images into a machine learning model trained to classify images as intraoral images, wherein the machine learning model outputs a classification for the one or more two-dimensional images indicating that the one or more two-dimensional images depict the interior of a mouth.

[0095] The 92nd aspect of this disclosure may further extend aspects 78 through 91 of this disclosure. In the 92nd aspect of this disclosure, the method further includes: inputting at least one of an intraoral scan or a height map generated from a three-dimensional surface into a machine learning model trained to classify the height map as a maxillary arch view, a mandibular arch view, or an occlusal view, wherein the machine learning model outputs a classification of one of the maxillary arch view, mandibular arch view, or occlusal view; and indicating one of the determined maxillary arch view, mandibular arch view, or occlusal view in a graphical user interface.

[0096] The 93rd aspect of this disclosure may further extend aspects 78 to 92 of this disclosure. In the 93rd aspect of this disclosure, the method further includes: determining a first three-dimensional surface of the upper dental arch, a second three-dimensional surface of the lower dental arch, and a third three-dimensional surface of occlusion, which have been generated to depict the relationship between the upper and lower dental arches; in response to determining the first three-dimensional surface of the upper dental arch, the second three-dimensional surface of the lower dental arch, and the third three-dimensional surface of occlusion, automatically determining occlusal contact areas on the upper and lower dental arches based on the first three-dimensional surface of the upper dental arch, the second three-dimensional surface of the lower dental arch, and the third three-dimensional surface of occlusion; and automatically generating an occlusal map depicting the occlusal contact areas on the upper and lower dental arches without first receiving a user request to generate the occlusal map.

[0097] The 94th aspect of this disclosure may further extend aspects 78 through 93 of this disclosure. In the 94th aspect of this disclosure, the method further includes: receiving a first intraoral scan depicting a first occlusal relationship between the upper and lower dental arches, the first intraoral scan having been generated at a first time; receiving a second intraoral scan depicting a second occlusal relationship between the upper and lower dental arches, the second intraoral scan having been generated at a second time; determining a first difference between the first and second occlusal relationships; determining a second difference between the first and second times; and determining, at least in part, based on the first and second differences, whether the first and second intraoral scans depict the same occlusion of the patient or different occlusions of the patient.

[0098] The 95th aspect of this disclosure may further extend the 94th aspect of this disclosure. In the 95th aspect of this disclosure, the method further includes: in response to determining that the first intraoral scan and the second intraoral scan depict the same occlusion, merging data from the first intraoral scan and the second intraoral scan to generate a three-dimensional surface depicting the occlusion.

[0099] The 96th aspect of this disclosure may further extend aspects 94 and 95 of this disclosure. The 96th aspect of this disclosure includes performing the following operations in response to determining that a first intraoral scan and a second intraoral scan depict different occlusions: generating a first three-dimensional surface depicting the first occlusion from the first intraoral scan; and generating a second three-dimensional surface depicting the second occlusion from the second intraoral scan.

[0100] The 97th aspect of this disclosure may further extend aspects 78 to 96 of this disclosure. In the 97th aspect of this disclosure, the method further includes: receiving biometric data of a user of an intraoral scanner; and automatically identifying the user of the intraoral scanner using the biometric data.

[0101] The 98th aspect of this disclosure may further extend the 97th aspect of this disclosure. In the 98th aspect of this disclosure: receiving biometric data includes receiving an image of a user's face generated by an intraoral scanner; and automatically determining the identity of the user of the intraoral scanner using the biometric data includes processing the image of the face using a trained machine learning model trained to perform facial recognition, wherein the identity of the user is determined from a list of possible users associated with a particular dental clinic.

[0102] The 99th aspect of this disclosure may further extend aspects 97 or 98 of this disclosure. In aspect 99 of this disclosure, the method further includes: determining, based on historical data about the user, whether the user a) performs only orthodontic dental procedures or b) performs only restorative dental procedures; automatically initiating a restorative dental procedure workflow in response to determining that the user performs only restorative dental procedures, wherein a prescription is for a dental prosthesis; and automatically initiating an orthodontic dental procedure workflow in response to determining that the user performs only orthodontic dental procedures, wherein a prescription is for orthodontics.

[0103] The 100th aspect of this disclosure may further extend aspects 78 to 99 of this disclosure. In the 100th aspect of this disclosure, where the patient is an unknown patient, the method further includes: determining a current date and time; determining a known patient scheduled for the current date and time; comparing a three-dimensional surface of the dental arch of the unknown patient with a three-dimensional surface of the dental arch of the known patient scheduled for the current date and time; determining a match between the three-dimensional surface of the dental arch of the unknown patient and the three-dimensional surface of the dental arch of the known patient; and verifying the unknown patient as a known patient.

[0104] The 101st aspect of this disclosure may further extend aspects 78 through 100 of this disclosure. In the 101st aspect of this disclosure, the method further includes, in response to determining that a restorative dental object is included in a patient's dental arch: determining that a restorative dental procedure is to be performed; and initiating a restorative dental procedure workflow.

[0105] The 102nd aspect of this disclosure may further extend aspects 78 through 101 of this disclosure. In the 102nd aspect of this disclosure, the method further includes, after determining that no restorative dental object is included in the patient's dental arch: determining that an orthodontic dental procedure is to be performed; and initiating the orthodontic dental procedure workflow.

[0106] The 103rd aspect of this disclosure may further extend aspects 78 to 102 of this disclosure. In the 103rd aspect of this disclosure, the patient is an unknown patient, and the method further includes: comparing a three-dimensional surface of the dental arch of the unknown patient with a plurality of three-dimensional surfaces of the dental arch of a known patient; determining a match between the three-dimensional surfaces of the dental arch of the unknown patient and the three-dimensional surfaces of the dental arch of the known patient; and identifying the unknown patient as a known patient.

[0107] The 104th aspect of this disclosure may further extend aspects 78 to 103 of this disclosure. In the 104th aspect of this disclosure, a computer-readable medium includes instructions that, when executed by a processing apparatus, cause the processing apparatus to perform the method of any one of aspects 78 to 103 of this disclosure.

[0108] The 105th aspect of this disclosure may further extend aspects 78 through 103 of this disclosure. In the 105th aspect of this disclosure, a system includes: an intraoral scanner for generating the plurality of intraoral scans; and a computing device connected to the intraoral scans via a wired or wireless connection, the computing device performing the method of any one of aspects 78 through 103 of this disclosure.

[0109] In a 106th aspect of this disclosure, a method includes: receiving an intraoral scan of a mouth, the intraoral scan having been generated by an intraoral scanner including a probe inserted into the mouth; determining from the intraoral scan a region in the intraoral scan representing a dirty region of an optical surface associated with the intraoral scanner; determining whether the region representing the dirty region of the optical surface satisfies one or more criteria; and in response to determining that the region representing the dirty region of the optical surface satisfies the one or more criteria, performing the following operations: determining that the optical surface is occluded; and generating a notification indicating that the optical surface is occluded.

[0110] The 107th aspect of this disclosure may further extend the 106th aspect of this disclosure. In the 107th aspect of this disclosure, the intraoral scan is generated based on incoherent light output by the intraoral scanner and reflected back into the intraoral scanner from an object in the oral cavity.

[0111] The 108th aspect of this disclosure may further extend aspects 106 and 107 of this disclosure. In the 108th aspect of this disclosure, a probe is inserted into a disposable sleeve, wherein the optical surface is a window of the disposable sleeve, and wherein the notification includes at least one of a first notification to replace the disposable sleeve or a second notification indicating the percentage of a dirty window of the disposable sleeve.

[0112] This 109th aspect of the disclosure may further extend aspects 106 to 108 of the disclosure. In aspect 109 of the disclosure, the optical surface is a window or mirror of the probe, and the notification includes notification of cleaning the probe of the intraoral scanner.

[0113] The 110th aspect of this disclosure may further extend aspects 106 to 109 of this disclosure. In the 110th aspect of this disclosure, the method further includes: rejecting intraoral scans, wherein rejected intraoral scans are not used during the generation of a three-dimensional model of the dental arch in the oral cavity.

[0114] The 111th aspect of this disclosure may further extend aspects 106 to 110 of this disclosure. In the 111th aspect of this disclosure, each point corresponds to a pixel in an intraoral scan, and the method further includes: in response to determining that a region representing a dirty area of ​​an optical surface satisfies one or more of the criteria, using an intraoral scan to determine a three-dimensional surface of tooth points in the oral cavity, wherein those pixels associated with points in the region representing a dirty area of ​​the optical surface are not used to determine the three-dimensional surface.

[0115] The 112th aspect of this disclosure may further extend aspects 106 to 111 of this disclosure. In the 112th aspect of this disclosure, the method further includes: in response to determining that a region representing a dirty region of an optical surface satisfies one or more of the criteria, discarding data of those points associated with the region representing the dirty region of the optical surface.

[0116] The 113th aspect of this disclosure may further extend aspects 106 to 112 of this disclosure. In the 113th aspect of this disclosure, the method further includes: identifying a cluster of points representing a dirty region of an optical surface; determining at least one of the size or shape of the cluster of points; determining whether the size or shape indicates a tooth location; and, in response to determining that the size or shape indicates a tooth location, using an intraoral scan to determine a three-dimensional surface of the tooth location, wherein those pixels associated with the points associated with the dirty region of the optical surface are used to determine the three-dimensional surface.

[0117] The 114th aspect of this disclosure may further extend aspects 106 to 113 of this disclosure. In the 114th aspect of this disclosure, the method further includes: processing at least one of intraoral scans or data from a three-dimensional surface using a trained machine learning model, the three-dimensional surface being generated from an intraoral scan and one or more additional intraoral scans, the trained machine learning model having been trained to identify areas in the intraoral scan obscured by a dirty probe or a dirty sleeve on the probe, wherein the trained machine learning model outputs a schematic diagram including, for each pixel in the intraoral scan or data from the three-dimensional surface, an indication of whether the pixel represents a dirty area of ​​the optical surface, each pixel being associated with one of the points.

[0118] The 115th aspect of this disclosure can further extend aspects 106 to 114 of this disclosure. In the 115th aspect of this disclosure, the method further includes: receiving a plurality of intraoral scans generated by an intraoral scanner, wherein the intraoral scan is one of the plurality of intraoral scans; for each of the plurality of intraoral scans, determining points representing dirty regions of an optical surface; and determining points representing dirty regions of an optical surface for at least a threshold amount for the plurality of intraoral scans; wherein the one or more criteria include a criterion that the number of points representing dirty regions of an optical surface for at least the threshold amount in the plurality of intraoral scans satisfies or exceeds a dirty region size threshold.

[0119] The 116th aspect of this disclosure may further extend aspects 106 to 115 of this disclosure. In the 116th aspect of this disclosure, the one or more criteria include a threshold, wherein the one or more criteria are satisfied when the number of points representing dirty areas of the optical surface exceeds the threshold.

[0120] The 117th aspect of this disclosure may further extend aspects 106 to 116 of this disclosure. In the 117th aspect of this disclosure, determining the region representing a dirty area of ​​the optical surface of the probe in the intraoral scan includes: determining the distance between a point depicted in the intraoral scan and the probe of the intraoral scanner; and determining points having a distance less than or equal to a distance threshold, wherein those points having a distance less than or equal to the distance threshold are points representing dirty areas of the optical surface.

[0121] The 118th aspect of this disclosure may further extend the 117th aspect of this disclosure. In the 118th aspect of this disclosure, the probe of the intraoral scanner includes a window and is inserted into a sleeve, wherein the sleeve includes a second window aligned with the window of the probe, and wherein a distance threshold is approximately the measured distance to the second window of the sleeve.

[0122] The 119th aspect of this disclosure may further extend aspects 117 and 118 of this disclosure. In the 119th aspect of this disclosure, the method further includes: identifying a cluster of points having a distance less than or equal to a distance threshold; determining at least one of the size or shape of the cluster of points; determining whether the size or shape indicates a tooth location; and, in response to determining that the size or shape indicates a tooth location, using an intraoral scan to determine a three-dimensional surface of the tooth location, wherein those pixels associated with points having a distance less than or equal to the distance threshold are used to determine the three-dimensional surface.

[0123] The 120th aspect of this disclosure may further extend aspects 117 and 118 of this disclosure. The 120th aspect of this disclosure includes, for each point having a distance less than or equal to a distance threshold, performing the following operations: determining a plurality of surrounding points within a threshold proximity range of the point on the plane; determining the distances of those points within the threshold proximity range of the point on the plane; determining the number of those points within the threshold proximity range of the point on the plane having a distance exceeding the distance threshold; determining whether the number of points within the threshold proximity range of the point on the plane having a distance exceeding the distance threshold exceeds the additional threshold; and classifying the point as an occluded point in response to determining that the number of points within the threshold proximity range of the point on the plane having a distance exceeding the distance threshold exceeds the additional threshold.

[0124] The 121st aspect of this disclosure may further extend aspects 117 to 120 of this disclosure. In the 121st aspect of this disclosure, the method further includes: receiving a second intraoral scan generated by an intraoral scanner; determining the distance between points depicted in the second intraoral scan and a probe of the intraoral scanner; determining those points in the second intraoral scan that have a distance less than or equal to a distance threshold; comparing those points in the second intraoral scan that have a distance less than or equal to the distance threshold with those points in the intraoral scan that have a distance less than or equal to the distance threshold; and based on the comparison, determining points in both the intraoral scan and the second intraoral scan that have a distance less than or equal to the distance threshold; wherein the one or more criteria include a criterion shared by multiple intraoral scans for the number of points having a distance less than or equal to the distance threshold.

[0125] The 122nd aspect of this disclosure may further extend aspects 106 to 121 of this disclosure. In the 122nd aspect of this disclosure, the method further includes: receiving an additional intraoral scan of the oral cavity, wherein determining from the intraoral scan a region representing a dirty area of ​​an optical surface associated with an intraoral scanner comprises: determining one or more unchanging points between the intraoral scan and the additional intraoral scan; and determining that the one or more unchanging points are regions representing dirty areas of an optical surface in the intraoral scan.

[0126] The 123rd aspect of this disclosure may further extend aspects 106 to 122 of this disclosure. In the 123rd aspect of this disclosure, a computer-readable medium includes instructions that, when executed by a processing apparatus, cause the processing apparatus to perform the method of any one of aspects 106 to 121 of this disclosure.

[0127] The 124th aspect of this disclosure may further extend aspects 106 to 122 of this disclosure. In the 124th aspect of this disclosure, a system includes: an intraoral scanner for generating an intraoral scan; and a computing device connected to the intraoral scanner via a wired or wireless connection, the computing device performing the method of any one of aspects 106 to 121 of this disclosure.

[0128] In a 125th aspect of this disclosure, a method includes: receiving an intraoral scan of a mouth, the intraoral scan having been generated by an intraoral scanner including a probe inserted into the mouth; receiving one or more two-dimensional (2D) images generated by the intraoral scanner, wherein the one or more 2D images are associated with the intraoral scan; determining from at least one of the intraoral scan or the one or more 2D images a number of points in the intraoral scan representing dirty areas of an optical surface associated with the intraoral scanner; determining whether the number of points representing dirty areas of the optical surface satisfies one or more criteria; and in response to determining that the number of points representing dirty areas of the optical surface satisfies the one or more criteria, performing the following operations: determining that the optical surface is occluded; and generating a notification indicating that the optical surface is occluded.

[0129] The 126th aspect of this disclosure may further extend the 125th aspect of this disclosure. In the 126th aspect of this disclosure, the one or more 2D images are color 2D images.

[0130] The 127th aspect of this disclosure may further extend the 125th or 126th aspects of this disclosure. In the 127th aspect of this disclosure, a system may include: an intraoral scanner for generating intraoral scans and 2D images; and a computing device connected to the intraoral scanner via a wired or wireless connection, the computing device performing the methods of any one of the 125th or 126th aspects of this disclosure. Alternatively, in the 127th aspect of this disclosure, a computer-readable medium may include instructions that, when executed by a processing device, cause the processing device to perform the methods of any one of the 125th or 126th aspects of this disclosure.

[0131] In a 128th aspect of this disclosure, a method includes: receiving a plurality of first scans of a prepared tooth; using the plurality of first scans to determine a first three-dimensional surface of the prepared tooth; receiving a plurality of second scans of a temporary dental prosthesis designed for the prepared tooth or an impression of the prepared tooth; using the plurality of second scans to determine a second three-dimensional surface representing a concave surface of the impression of the temporary dental prosthesis or the prepared tooth; automatically determining, for one or more segments of an edge line of the prepared tooth, whether to use the first three-dimensional surface, the second three-dimensional surface, or a combination of the first three-dimensional surface and the second three-dimensional surface for a three-dimensional model of the prepared tooth; and generating a three-dimensional model of the prepared tooth based at least in part on determining, for one or more segments of the edge line, whether to use the first three-dimensional surface, the second three-dimensional surface, or a combination of the first three-dimensional surface and the second three-dimensional surface for a three-dimensional model.

[0132] The 129th aspect of this disclosure may further extend the 128th aspect of this disclosure. In the 129th aspect of this disclosure, the method further includes: determining one or more segments of the edge line; for each of the one or more segments, determining a first quality rating for the segment when using a first three-dimensional surface, determining a second quality rating for the segment when using a second three-dimensional surface, and determining a third quality rating for the segment when using a combination of the first and second three-dimensional surfaces; wherein, for each segment, the use of the first three-dimensional surface, the second three-dimensional surface, or a combination of the first and second three-dimensional surfaces is determined by selecting an option associated with the highest quality rating from the first, second, and third quality ratings.

[0133] The 130th aspect of this disclosure may further extend the 129th aspect of this disclosure. In the 130th aspect of this disclosure, the method further includes: inverting a second three-dimensional surface; registering the inverted second three-dimensional surface with a first three-dimensional surface; and for each of the one or more segments, performing the following operations: determining a first depth of the segment from the first three-dimensional surface; determining a second depth of the segment from the second three-dimensional surface; and determining whether to use the first three-dimensional surface or the second three-dimensional surface based at least in part on a comparison of the first depth and the second depth.

[0134] The 131st aspect of this disclosure may further extend the 130th aspect of this disclosure. In the 131st aspect of this disclosure, the method further includes: determining which of a first depth or a second depth is a greater depth; and selecting the one of a first three-dimensional surface or a second three-dimensional surface having a greater depth.

[0135] The 132nd aspect of this disclosure may further extend any of the 128th to 131st aspects of this disclosure. In the 132nd aspect of this disclosure, the method further includes: inverting a second three-dimensional surface; registering the inverted second three-dimensional surface with a first three-dimensional surface; and for each of the one or more segments, performing the following operations: determining a first curvature of the segment from the first three-dimensional surface; determining a second curvature of the segment from the second three-dimensional surface; and determining whether to use the first three-dimensional surface or the second three-dimensional surface, at least in part, based on a comparison of the first curvature and the second curvature.

[0136] The 133rd aspect of this disclosure may further extend aspects 128 to 132 of this disclosure. In the 133rd aspect of this disclosure, the method further includes: automatically identifying the edge line of the prepared tooth in a three-dimensional model; and marking the edge line on the three-dimensional model.

[0137] The 134th aspect of this disclosure may further extend aspects 128 to 133 of this disclosure. In the 134th aspect of this disclosure, the method further includes: comparing a second three-dimensional surface with a first three-dimensional surface; determining, based on the result of the comparison, that the second three-dimensional surface matches the first three-dimensional surface; and determining that the second three-dimensional surface belongs to a concave surface of a temporary dental prosthesis or an impression of a prepared tooth.

[0138] The 135th aspect of this disclosure may further extend aspects 128 to 134 of this disclosure. In the 135th aspect of this disclosure, no user input is received to link a second three-dimensional surface to a first three-dimensional surface, wherein the second three-dimensional surface is compared with a plurality of three-dimensional surfaces, each of which is for different prepared teeth of the same patient or different patients.

[0139] The 136th aspect of this disclosure may further extend aspects 128 to 135 of this disclosure. In the 136th aspect of this disclosure, the method further includes: determining, based on the shape of the second three-dimensional surface, whether the second three-dimensional surface is a concave surface for preparing a temporary dental prosthesis for preparing a tooth or for preparing an impression of a tooth.

[0140] The 137th aspect of this disclosure can further extend aspects 128 to 136 of this disclosure. In aspect 136 of this disclosure, the method further includes: automatically identifying edge lines in a first three-dimensional surface; and automatically identifying edge lines in a second three-dimensional surface.

[0141] The 138th aspect of this disclosure may further extend the 137th aspect of this disclosure. In the 138th aspect of this disclosure: by inputting at least one of a first three-dimensional surface or a projection of the first three-dimensional surface onto one or more planes into a trained machine learning model, edge lines are automatically identified in the first three-dimensional surface, and the trained machine learning model outputs an indication of the edge lines of the first three-dimensional surface; and by inputting at least one of a second three-dimensional surface or a projection of the second three-dimensional surface onto one or more planes into a trained machine learning model, edge lines are automatically identified in the second three-dimensional surface, and the trained machine learning model outputs an indication of the edge lines of the second three-dimensional surface.

[0142] The 139th aspect of this disclosure may further extend the 128th to 138th aspects of this disclosure. In the 139th aspect of this disclosure, a computer-readable medium includes instructions that, when executed by a processing apparatus, cause the processing apparatus to perform the method of any one of the 128th to 138th aspects of this disclosure.

[0143] The 140th aspect of this disclosure may further extend the 128th to 138th aspects of this disclosure. In the 140th aspect of this disclosure, a system includes: an intraoral scanner for generating a plurality of first scans and a plurality of second scans; and a computing device connected to the intraoral scanner via a wired or wireless connection, the computing device performing the methods of any one of the 128th to 138th aspects of this disclosure.

[0144] In a 141st aspect of this disclosure, a method includes: receiving a plurality of first scans of an edentulous dental arch; receiving a plurality of second scans of an impression of a first denture designed for the edentulous dental arch or of at least a portion thereof; determining a three-dimensional surface of the edentulous dental arch using the plurality of first scans and the plurality of second scans; and generating a virtual three-dimensional model using the three-dimensional surface, wherein the virtual three-dimensional model can be used to manufacture a second denture for the edentulous dental arch.

[0145] The 142nd aspect of this disclosure may further extend the 141st aspect of this disclosure. In the 142nd aspect of this disclosure, the method further includes: determining a first three-dimensional surface using a plurality of first scans; determining a second three-dimensional surface using a plurality of second scans; comparing the second three-dimensional surface with the first three-dimensional surface; and determining a match between the second three-dimensional surface and the first three-dimensional surface based on the result of the comparison.

[0146] This 143rd aspect of the disclosure may further extend aspects 141 and 142 of the disclosure. In aspect 143 of the disclosure, the three-dimensional surface includes one or more mucco dynamic boundaries.

[0147] The 144th aspect of this disclosure may further extend aspects 141 to 143 of this disclosure. In the 144th aspect of this disclosure, the method further includes: receiving user input that associates a plurality of second scans with a plurality of first scans.

[0148] This 145th aspect of the disclosure may further extend aspects 141 to 144 of the disclosure. In aspect 145 of the disclosure, the impression is only a part of the edentulous dental arch.

[0149] The 146th aspect of this disclosure may further extend the 141 to 145 aspects of this disclosure. In the 146th aspect of this disclosure, a computer-readable medium includes instructions that, when executed by a processing apparatus, cause the processing apparatus to perform the method of any one of the 141 to 145 aspects of this disclosure.

[0150] The 147th aspect of this disclosure may further extend the 141st to 145th aspects of this disclosure. In the 147th aspect of this disclosure, a system includes: an intraoral scanner for generating a plurality of first scans and a plurality of second scans; and a computing device connected to the intraoral scanner via a wired or wireless connection, the computing device performing the method of any one of the 141st to 145th aspects of this disclosure. Attached Figure Description

[0151] Embodiments of this disclosure are illustrated in the accompanying drawings by way of example rather than limitation.

[0152] Figure 1 An embodiment of a system for performing intraoral scanning and / or generating virtual 3D models of intraoral points is shown.

[0153] Figure 2A The model training workflow and model application workflow of an intraoral scanning application according to an embodiment of the present disclosure are illustrated.

[0154] Figure 2B An example intraoral scanning workflow according to one embodiment of the present disclosure is shown.

[0155] Figure 3 This is a flowchart illustrating one embodiment of a method for training a machine learning model to identify scanned characters.

[0156] Figure 4 An example of a separate heightmap is shown, which is used to train a machine learning model to determine the scanning role.

[0157] Figure 5 An example bite view of the jaw is shown, which was used to train a machine learning model to determine the scan role.

[0158] Figure 6Several example jaw views are shown, which are used to train a machine learning model to determine the scan role.

[0159] Figure 7 This is a flowchart illustrating an embodiment of a method for automatically determining the scanning role of an intraoral scan.

[0160] Figures 8A-8B A flowchart is shown as an embodiment of a method for performing intraoral scanning without receiving user input for a specified scanning role.

[0161] Figure 9 This is a flowchart illustrating an embodiment of a method for training a machine learning model to classify dental objects, including restorative objects.

[0162] Figures 10A-10B A flowchart illustrating an embodiment of a method for automatically identifying restorative objects and generating a 3D model of a dental arch including such restorative objects is shown.

[0163] Figure 11 A flowchart illustrating an embodiment of a method for automatically identifying restorative objects and generating a 3D model of a dental arch including such restorative objects is shown.

[0164] Figure 12A-12B This shows intraoral scans of restorative and dental objects that occur naturally in the patient's oral cavity.

[0165] Figure 13A A variable resolution 3D model of the dental arch is shown.

[0166] Figure 13B A 3D model of the dental arch is shown.

[0167] Figure 13C It shows Figure 13B A variable resolution version of the 3D model of the dental arch.

[0168] Figures 14A-14B A flowchart illustrating an embodiment of a method for automatically generating and updating a 3D model of a prepared tooth when the prepared tooth is modified.

[0169] Figure 15A-15F A flowchart illustrating an embodiment of a method for determining which 3D surfaces to use to represent a restorative object of a 3D model is shown.

[0170] Figure 16 This is a flowchart illustrating one embodiment of a method for determining whether additional scanning of the teeth is recommended.

[0171] Figure 17 This is a flowchart illustrating one embodiment of a method for determining the contour of a prepared tooth edge line.

[0172] Figure 18 This is a flowchart illustrating one embodiment of a method for determining a trajectory to display a 3D model of the dental arch.

[0173] Figures 19A-19C A side view of the 3D surface of a prepared tooth is shown at various stages of tooth preparation.

[0174] Figure 20 This is a flowchart illustrating one embodiment of a method for automatically generating prescriptions for dental prostheses or orthodontics.

[0175] Figure 21 This is a flowchart illustrating one embodiment of a method for automatically determining a 3D model of a dental prosthesis.

[0176] Figures 22A-22C A flowchart illustrating an embodiment of a method for automatically generating one or more prescriptions for a patient's dental prostheses and / or orthodontics is shown.

[0177] Figure 23 This is a flowchart illustrating one embodiment of a method for automatically generating prescriptions for dental prostheses.

[0178] Figure 24 This is a flowchart illustrating one embodiment of a method for automatically determining when to start and / or stop generating intraoral scans.

[0179] Figure 25 This is a flowchart illustrating one embodiment of a method for automatically detecting multi-occlusal scanning scenarios.

[0180] Figure 26 This is a flowchart illustrating one embodiment of a method for automatically identifying patients.

[0181] Figure 27 This is a flowchart illustrating one embodiment of a method for automatically identifying a user of an intraoral scanner.

[0182] Figure 28 This is a flowchart illustrating one embodiment of a method for automatically detecting dirty optical surfaces of an intraoral scanner or a protective sleeve on an intraoral scanner.

[0183] Figure 29 This is a flowchart illustrating one embodiment of a method for determining how to use pixels of an intraoral scan associated with a dirty region of an optical surface.

[0184] Figure 30 This is a flowchart illustrating one embodiment of a method for using a trained machine learning model to identify dirty areas on the optical surface of an intraoral scanner or a protective sleeve on an intraoral scanner.

[0185] Figure 31This is a flowchart illustrating an embodiment of a method for determining which pixels in an intraoral scan or image represent dirty areas on the optical surface of an intraoral scanner or protective sleeves on an intraoral scanner.

[0186] Figures 32A-32B A probe of an intraoral scanner with a dirty optical surface is shown according to an embodiment of the present disclosure.

[0187] Figure 33 An intraoral scanner with a dirty surface is shown according to one embodiment of the present disclosure.

[0188] Figures 34A-34B The image shown is taken by an intraoral scanner with a dirty surface.

[0189] Figure 35 This is a flowchart illustrating an embodiment of a method for determining a 3D model of a dental prosthesis using scanning of the tooth locations to be received and scanning of the intaglio surface of an impression of the tooth locations, or a pre-existing dental prosthesis.

[0190] Figure 36 This is a flowchart illustrating an embodiment of a method for preparing a 3D surface for teeth by automatically determining 3D surface registration with a concave surface of a tooth location.

[0191] Figure 37 This is a flowchart illustrating an embodiment of a method for determining which 3D surfaces to use to generate segments of edge lines in a 3D model of a prepared tooth.

[0192] Figure 38 This is a flowchart illustrating an embodiment of a method for determining which 3D surfaces to use to generate segments of edge lines in a 3D model of a prepared tooth.

[0193] Figure 39 This demonstrates how selecting points from two different 3D surfaces generates a 3D model of a prepared tooth with sharp edge lines.

[0194] Figure 40A The crown set on the prepared tooth is shown.

[0195] Figure 40B It shows Figure 40A A side view of the prepared tooth surface.

[0196] Figure 40C It shows Figure 40A A 3D side view of the concave surface of the crown.

[0197] Figure 41 This is a flowchart illustrating an embodiment of a method for generating a 3D model of an edentulous dental arch for manufacturing dentures.

[0198] Figure 42 A block diagram of an example computing device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0199] This document describes methods and systems for simplifying and automating the process of performing intraoral scans. In some embodiments, user input is minimized. For example, in embodiments, user input for patient information, selecting the patient to be scanned, selecting the segment of the dental arch to be scanned, indicating whether the scan was successful, manually entering instructions to transition between stages or modes of intraoral scanning, selecting prescription details, selecting the laboratory to send data, etc., may be reduced or eliminated.

[0200] The embodiments disclosed herein provide automated systems and methods for automatically identifying segments and / or roles associated with received intraoral scans, for automatically identifying, classifying, and / or determining the location of restorative objects in intraoral scans, for automatically generating prescriptions for treating a patient's dental arch, and for automatically determining whether to apply restorative or orthodontic workflows to a patient. Additionally, the embodiments disclosed herein provide systems and methods for automatically detecting whether the surface of an intraoral scanner is dirty. Furthermore, the embodiments disclosed herein provide systems and methods for determining margin lines or other information using data from the concave surface of a denture, an elastomeric impression, or a temporary crown.

[0201] By utilizing one or more of the embodiments described herein, physicians are able to perform orthodontic and / or restorative dental procedures with significantly less training, in contrast to traditional workflows. Additionally, the embodiments enable faster full scans (e.g., less patient sitting time due to the potential elimination of lengthy prescription (Rx) generation processes and the potential elimination of erasing or marking areas). Furthermore, the embodiments reduce or eliminate the need for physicians to switch back and forth between scanning the patient and interfacing with a computer to view the scan results, minimizing cross-contamination (due to less or no need to touch the screen or keyboard).

[0202] In one example, a common sequence for generating a prescription for orthodontic treatment is (1) selecting a case definition dropdown menu, (2) selecting the orthodontic case type, (3) the patient name type, (4) selecting whether brackets will be used, and (5) selecting the number of occlusal scans (e.g., two occlusal scans). Additionally, the doctor performs a scan of the patient, which includes (1) pressing a button to start the scan mode, (2) manually selecting the upper arch for scanning, (3) scanning the upper arch, (4) manually selecting the lower arch for scanning, (5) scanning the lower arch, (6) selecting the occlusion for scanning, (7) scanning the occlusion, (8) performing an occlusal gap review, (9) deleting and rescanning if necessary, and (10) indicating that the scan is complete and a 3D model will be generated. Conversely, in this embodiment, a simplified sequence for generating a prescription for orthodontic treatment and performing a scan includes (1) starting the scan. All other operations can be automated. For example, the system can automatically identify the current role being scanned (e.g., mandibular arch, maxillary arch, occlusion) and generate an appropriate model accordingly. It can also automatically determine when the scan is complete (e.g., for a specific role or the whole) and begin post-processing (e.g., this can begin when sufficient data has accumulated and a new scanning area is started, or when the dentist has removed the intraoral scanner from the patient's mouth). The system can automatically perform occlusal gap review and notify the dentist if any issues are found. The system can also automatically determine if the current case type is orthodontic. Additionally, the system can automatically identify the scanned patient (e.g., based on a calendar system indicating patient appointments corresponding to the current time and / or based on a comparison of the intraoral scan with stored patient scan records).

[0203] In another example, a common sequence for generating a prescription for restorative dental treatment is (1) selecting the case definition dropdown menu, (2) selecting the restoration case type, and (3) the type in the patient's name.

[0204] (4) Select which teeth are to be prepared and / or implanted, (5) Define the type of implant, (6) Manually select the material to be used for the implant (oral restoration), (7) Manually set the type of preparation (e.g., bridge, crown, etc.), (8) Manually set the color definition for the restoration, and (9) Manually select the laboratory to be used. Additionally, the dentist performs a scan on the patient, which includes (1) pressing a button to start the scan mode, (2) manually selecting the upper dental arch for scanning, (3) scanning the upper dental arch, (4) manually selecting the lower dental arch for scanning, (5) scanning the lower dental arch, (6) manually selecting the occlusion for scanning, (7) scanning the occlusion, (8) manually selecting the teeth to be prepared for scanning, (9) scanning the prepared teeth, (10) performing an occlusal gap review, and (11) manually deleting and rescanning if necessary.

[0205] (12) Indicate that the scan is complete and a 3D model will be generated, (13) Review the 3D model, (14) Manually mark the edge lines in the 3D model, and send the 3D model to the selected laboratory. Conversely, in this embodiment, the simplified sequence for generating a prescription for restorative treatment and performing a scan includes (1) Starting the scan. All other operations can be automated. For example, the system can automatically identify the current role being scanned (e.g., mandibular arch, maxillary arch, occlusion) and whether the preparing teeth are being scanned, and generate appropriate models accordingly, and can automatically determine when the scan is complete (e.g., for a specific role or the whole) and begin post-processing (e.g., this can begin when sufficient data has accumulated and a new scanning area has begun or when the dentist has removed the intraoral scanner from the patient's mouth). The system can automatically perform an occlusal gap review and notify the dentist if any problems exist. The system can also automatically determine that the current case type is a restorative case type. Additionally, the system can automatically determine the identity of the patient being scanned (e.g., based on a calendar system indicating a patient appointment corresponding to the current time and / or based on a comparison of the intraoral scan with stored records of the patient scan). Utilizing each automated recognition and / or operation, the processing logic can provide visual and / or audio feedback to the user of the intraoral scanner, letting the user know that the system understands the current status of the scanning process. For example, the system can notify the user when the upper dental arch is detected, when preparing teeth are detected, when the lower dental arch is detected, when the patient's occlusion is detected, and when the scan is determined to be complete. The processing logic can also notify the physician when automatically generating a 3D model, automatically performing occlusal space review, or automatically determining one or more details of the prescription. Based on this information, the user knows where they are in the scanning workflow without needing to manually enter information.

[0206] Additionally, for restorative treatments, dentists often need to modify the prepared teeth (e.g., by performing additional drilling / polishing or by adding / removing retraction cord), followed by a rescan of the prepared teeth. In this embodiment, the system identifies what changes have occurred after each rescan and displays the areas where changes have been made, identifying changes to the teeth (e.g., to the margins and / or the shape of the preparation) and / or changes to the gingiva (e.g., between images taken before and after exposing the margins with retraction cord). The system includes logic for determining which portions of the data from each scan should be used to generate the highest quality 3D model. Additionally, after each scan / rescan, the system can calculate and display the occlusal surfaces, margins, insertion paths (including any insertion path issues), etc. Additionally, the system can automatically determine the color to be used for the restoration based on the color / hue of adjacent teeth, and can also automatically determine the material type, laboratory selection, etc., based on the dentist's usage patterns.

[0207] This document describes various embodiments. It should be understood that these various embodiments can be implemented as independent solutions and / or can be combined. Therefore, reference to an embodiment or an embodiment can refer to the same embodiment and / or different embodiments. Additionally, some embodiments are discussed with reference to restorative dentistry, and particularly with reference to tooth preparation. However, it should be understood that embodiments discussed with reference to restorative dentistry (e.g., restoration) can also be applied to orthodontic dentistry (e.g., orthodontics). Additionally, embodiments discussed with reference to tooth preparation can also be applied generally to teeth, and not just tooth preparation. Furthermore, embodiments discussed with reference to marginal lines can also be applied to other tooth features, such as cracks, debris, gingival lines, caries, etc.

[0208] This document discusses some embodiments with reference to intraoral scans and intraoral images. However, it should be understood that the embodiments described with reference to intraoral scans are also applicable to laboratory scans or model / impression scans. Laboratory scans or model / impression scans may include one or more images of tooth locations or models or impressions of tooth locations, which may or may not include height maps, and may or may not include color images.

[0209] Figure 1 An embodiment of a system 100 for performing intraoral scanning and / or generating virtual 3D models of intraoral points is illustrated. System 100 includes a dental clinic 108 and optionally one or more dental laboratories 110. Dental clinic 108 and dental laboratories 110 each include computing devices 105 and 106, which can be interconnected via a network 180. Network 180 can be a local area network (LAN), a public wide area network (WAN) (e.g., the Internet), a private WAN (e.g., an intranet), or a combination thereof.

[0210] Computing device 105 may be coupled to one or more intraoral scanners 150 (also referred to as scanners) and / or data storage 125 via wired or wireless connections. In one embodiment, multiple scanners 150 in a dental clinic 108 are wirelessly connected to computing device 105. In one embodiment, scanners 150 are wirelessly connected to computing device 105 via a direct wireless connection. In one embodiment, scanners 150 are wirelessly connected to computing device 105 via a wireless network. In one embodiment, the wireless network is a Wi-Fi network. In one embodiment, the wireless network is a Bluetooth network, a Zigbee network, or some other wireless network. In one embodiment, the wireless network is a wireless mesh network, examples of which include Wi-Fi mesh networks, Zigbee mesh networks, etc. In one example, computing device 105 may be physically connected to one or more wireless access points and / or wireless routers (e.g., Wi-Fi access points / routers). Intraoral scanners 150 may include wireless modules, such as Wi-Fi modules, and can join wireless networks via wireless access points / routers. Computing device 106 may also be connected to data storage (not shown). Data storage may be local data storage and / or remote data storage. Computing device 105 and computing device 106 may each include one or more processing devices, memory, secondary memory, one or more input devices (e.g., such as keyboard, mouse, tablet, touch screen, microphone, camera, etc.), one or more output devices (e.g., display, printer, touch screen, speaker, etc.), and / or other hardware components.

[0211] In this embodiment, the scanner 150 includes an inertial measurement unit (IMU). The IMU may include an accelerometer, gyroscope, magnetometer, pressure sensor, and / or other sensors. For example, the scanner 150 may include one or more microelectromechanical systems (MEMS) IMUs. The IMU can generate inertial measurement data, including acceleration data, rotation data, etc.

[0212] Computing device 105 and / or data storage 125 may be located in dental clinic 108 (as shown), dental laboratory 110, or one or more other locations such as a server farm providing cloud computing services. Computing device 105 and / or data storage 125 may be connected to components located in the same or different locations as computing device 105 (e.g., components in a second location away from dental clinic 108, such as a server farm providing cloud computing services). For example, computing device 105 may be connected to a remote server, where some operations of intraoral scanning application 115 are performed on computing device 105 and some operations of intraoral scanning application 115 are performed on the remote server.

[0213] Additional computing devices may be physically connected to computing device 105 via wired connections. Additional computing devices may also be wirelessly connected to computing device 105 via wireless connections, which may be direct wireless connections or wireless connections via a wireless network. In one embodiment, one or more additional computing devices may be mobile computing devices, such as laptops, notebooks, tablets, mobile phones, portable game consoles, etc. In another embodiment, one or more additional computing devices may be conventionally stationary computing devices, such as desktop computers, set-top boxes, game consoles, etc. Additional computing devices may act as thin clients of computing device 105. In one embodiment, the additional computing devices use Remote Desktop Protocol (RDP) to access computing device 105. In another embodiment, the additional computing devices use Virtual Network Control (VNC) to access computing device 105. Some additional computing devices may be passive clients, which do not have control over computing device 105 and receive visualizations of the user interface of intraorbital scanning application 115. In one embodiment, one or more additional computing devices may operate in master mode, and computing device 105 may operate in slave mode.

[0214] The intraoral scanner 150 may include a probe (e.g., a handheld probe) for optically capturing three-dimensional structures. The intraoral scanner 150 can be used to perform intraoral scans of a patient's oral cavity. An intraoral scanning application 115 running on computing device 105 can communicate with the scanner 150 to perform the intraoral scan. The result of the intraoral scan may be intraoral scan data 135A, 135B to 135N, which may include one or more sets of intraoral scans that may include intraoral images. Each intraoral scan may include a two-dimensional (2D) or 3D image that may include depth information (e.g., a height map) of a portion of a tooth location. In an embodiment, the intraoral scan includes x, y, and z information. In one embodiment, the intraoral scanner 150 generates a plurality of discrete (i.e., individual) intraoral scans.

[0215] In some embodiments, a set of discrete intraoral scans is merged into a smaller set of hybrid intraoral scans, where each hybrid scan is a combination of multiple discrete scans. Intraoral scan data 135A-135N may include raw scans and / or hybrid scans, each of which may be referred to as an intraoral scan (and in some cases, as an intraoral image). During scanning, the intraoral scanner may generate multiple (e.g., dozens) scans (e.g., height maps) per second (referred to as raw scans). To improve the quality of the captured data, a blending process may be used to combine sequences of raw scans into hybrid scans through some averaging process. Additionally, the intraoral scanner 150 may generate many scans per second. This may be too much data to process in real time using a machine learning model. Therefore, groups of similar scans may be combined into hybrid scans, and the hybrid scans may be fed into one or more trained machine learning models. This can significantly reduce the computational resources used to process intraoral scans without compromising quality. In one embodiment, each hybrid scan includes data from up to 20 raw scans and also includes scans that differ from each other by less than a threshold angular difference and / or from each other by less than a threshold positional difference. Therefore, some hybrid scans may include data from 20 scans, while other hybrid scans may include data from fewer than 20 scans. In one embodiment, an intraoral scan (which may be a hybrid scan) includes the height and intensity values ​​of each pixel in the image.

[0216] In embodiments, intraoral scan data 135A-135N may also include color 2D images of tooth locations and / or images at specific wavelengths (e.g., near-infrared (NIRI) images, infrared images, ultraviolet images, etc.). In embodiments, the intraoral scanner 150 alternates between generating 3D intraoral scans and one or more types of 2D intraoral images (e.g., color images, NIRI images, etc.) during scanning. For example, one or more 2D color images may be generated between the generation of a fourth and fifth intraoral scan. For example, some scanners may include multiple image sensors that simultaneously generate different 2D color images of different regions of the patient's dental arch. These 2D color images may be stitched together to form a single color representation of a larger field of view that combines the fields of view of multiple image sensors.

[0217] The scanner 150 can transmit intraoral scan data 135A, 135B to 135N to the computing device 105. The computing device 105 can store the intraoral scan data 135A-135N in the data storage 125.

[0218] According to one example, a user (e.g., a physician) can subject a patient to an intraoral scan. In doing so, the user can apply scanner 150 to one or more intraoral locations of the patient. The scan can be divided into one or more segments (also referred to as roles). For example, a segment may include the patient's lower dental arch, the patient's upper dental arch, one or more preparatory teeth of the patient (e.g., teeth of a patient to which a dental appliance such as a crown or other dental prosthesis will be applied), one or more teeth in contact with the preparatory teeth (e.g., teeth that are not affected by the dental appliance themselves, but which are located next to or engage with one or more such teeth when the mouth is closed), and / or the patient's occlusion (e.g., a scan performed with the patient's mouth closed, where the scan is directed towards the interface area between the patient's upper and lower teeth). Through this scanner application, scanner 150 can provide intraoral scan data 135A-135N to computing device 105. Intraoral scan data 135A-135N may be provided in the form of intraoral scan datasets, each of which may include 2D intraoral images (e.g., color 2D images) and / or 3D intraoral scans of regions of specific teeth and / or intraoral locations. In one embodiment, separate intraoral scan datasets are created for the maxillary arch, mandibular arch, patient occlusion, and each prepared tooth. Alternatively, a single large intraoral scan dataset is generated (e.g., for the mandibular arch and / or maxillary arch). Intraoral scans can be provided from scanner 150 to computing device 105 in the form of one or more points (e.g., one or more pixels and / or groups of pixels). For example, scanner 150 can provide intraoral scans as one or more point clouds. Intraoral scans can each include height information (e.g., a height map indicating the depth of each pixel).

[0219] The way a patient's mouth is scanned can depend on the procedure being applied. For example, if a maxillary or mandibular denture is being created, the entire edentulous arch of the mandible or maxilla may be scanned. Conversely, if a dental bridge is being created, only a portion of the entire arch may be scanned, including the edentulous area, adjacent prepared teeth (e.g., mating teeth), and the opposing arch and dentition. Alternatively, if a dental bridge is being created, a full scan of the upper and / or lower dental arches may be performed.

[0220] As a non-limiting example, dental procedures can be broadly categorized into prosthodontic (restorative) and orthodontic procedures, which are then further subdivided into specific forms of these procedures. Additionally, dental procedures may include the identification and treatment of gingival diseases, sleep apnea, and intraoral conditions. The term prosthodontic procedure refers to any procedure involving the oral cavity and relating to the design, manufacture, or installation of a dental prosthesis, or a real or virtual model thereof, at a tooth location within the oral cavity (intraoral location), or the design and preparation of an intraoral location for receiving such a prosthesis. For example, prostheses may include any restoration such as crowns, veneers, inlays, onlays, implants, and bridges, as well as any other artificial partial or complete dentures. The term orthodontic procedure specifically refers to any procedure involving the oral cavity and relating to the design, manufacture, or installation of orthodontic elements, or real or virtual models thereof, at an intraoral location within the oral cavity, or the design and preparation of an intraoral location for receiving such orthodontic elements. These elements may be appliances, including but not limited to brackets and wires, retainers, clear aligners, or functional appliances.

[0221] In this embodiment, an intraoral scan can be performed on the patient's oral cavity during a visit to dental clinic 108. The intraoral scan can be performed, for example, as part of a semi-annual or annual dental health check. The intraoral scan can also be performed before, during, and / or after one or more dental treatments, such as orthodontic treatment and / or prosthodontic treatment. The intraoral scan can be a complete or partial scan of the upper and / or lower dental arches and can be performed to collect information for performing dental diagnoses, generating treatment plans, determining the progress of treatment plans, and / or for other purposes. The dental information generated from the intraoral scan (intraoral scan data 135A-135N) can include 3D scan data, 2D color images, NIRI and / or infrared images, and / or ultraviolet images of all or part of the maxilla and / or mandible. The intraoral scan data 135A-135N may also include one or more intraoral scans illustrating the relationship between the upper and lower dental arches. These intraoral scans can be used to determine the patient's occlusion and / or to determine the patient's occlusal contact information. Patient occlusion may include the defined relationship between the teeth in the upper dental arch and the teeth in the lower dental arch.

[0222] For many dental prosthetic procedures (e.g., creating crowns, bridges, veneers, etc.), a patient's existing teeth are ground down into stumps. The tooth that is ground down is referred to in this text as a prepared tooth, or simply a preparation. A prepared tooth has a marginal line (also called an termination line), which is the boundary between the natural (unground) portion of the prepared tooth and the prepared (ground) portion. Prepared teeth are typically created so that crowns or other prostheses can be mounted or placed on them. In many cases, the marginal line of a prepared tooth is the sub-gingival margin (below the gingival line).

[0223] After creating the preparatory tooth, the dentist typically performs procedures to prepare it for scanning. Preparing the preparatory tooth for scanning may include wiping away blood, saliva, etc., from the preparatory tooth and / or separating the patient's gum from the preparatory tooth to expose the termination line. In some cases, the dentist will insert a thin suture (also called dental suture) around the preparatory tooth between the preparatory tooth and the patient's gum. The suture is then removed before generating a set of intraoral scans of the preparatory tooth. Subsequently, the soft tissue of the gum will return to its natural position and, in many cases, collapse backward across the termination line after a short period of time. Therefore, some intraoral scan data 135A-135N may include intraoral scans performed before the gum has collapsed across the termination line, while others may include intraoral scans performed after the gum has collapsed across the termination line. As a result, some intraoral scan data are superior to others when depicting preparatory teeth, particularly when depicting the termination line.

[0224] An intraoral scanner operates by moving scanner 150 inside a patient's oral cavity to capture all viewpoints of one or more teeth. In some embodiments, during a scan, scanner 150 calculates distances to a solid surface. These distances can be recorded as an image called a "height map." Each scan (e.g., optionally, a height map) is algorithmically overlaid or "stitched" with a previous set of scans to generate a long 3D surface. In this way, each scan is associated with a rotation or projection in space to determine how it fits into the 3D surface.

[0225] During intraoral scanning, intraoral scanning application 115 can register and stitch together two or more intraoral scans generated remotely from an intraoral scanning session. In one embodiment, performing registration includes capturing 3D data of individual points of surfaces in multiple scans and registering the scans by calculating transformations between scans. During intraoral scanning, one or more 3D surfaces can be generated based on the registered and stitched intraoral scans. The one or more 3D surfaces can be output to a display, allowing a physician or technician to view their scan progress to date. As each new intraoral scan is captured and registered to previous intraoral scans and / or 3D surfaces, the one or more 3D surfaces can be updated, and the updated 3D surfaces(s) can be output to a display. In one embodiment, separate 3D surfaces are generated for the maxilla and mandible. This process can be performed in real-time or near real-time to provide an updated view of the captured 3D surfaces during the intraoral scanning process.

[0226] When a scanning session or part of a scanning session associated with a specific scanning role (e.g., maxillary role, mandibular role, occlusal role, etc.) is completed (e.g., all scans for intraoral or dental points have been captured), the intraoral scanning application 115 can automatically generate a virtual 3D model of one or more dental points (e.g., maxillary and mandibular). The final 3D model can be a set of 3D points and their connections to each other (i.e., a mesh). To generate the virtual 3D model, the intraoral scanning application 115 can register and stitch together the intraoral scans associated with the specific scanning role generated from the intraoral scanning session. Registration performed at this stage can be more precise than registration performed during the capture of the intraoral scan and can take more time to complete. In one embodiment, performing scan registration includes capturing 3D data of individual points of a surface in multiple scans and registering the scans by calculating transformations between scans. The 3D data can be projected into the 3D space of the 3D model to form part of the 3D model. By applying appropriate transformations to the points of each registered scan and projecting each scan into 3D space, intraoral scans can be integrated into a common reference frame.

[0227] In one embodiment, registration is performed for adjacent or overlapping intraoral scans (e.g., each consecutive frame of an intraoral video). In one embodiment, hybrid scans are used to perform registration. A registration algorithm is performed to register two adjacent or overlapping intraoral scans (e.g., two adjacent hybrid intraoral scans) and / or to register an intraoral scan with a 3D model. This essentially involves determining a transformation to align one scan with the other scan and / or with the 3D model. Registration may involve identifying multiple points (e.g., point clouds) in each scan of a scan pair (or scan and 3D model), performing surface fitting on these points, and using a local search around the points to match the points of the two scans (or scan and 3D model). For example, an intraoral scan application 115 may match points from one scan with the nearest interpolated point on the surface of the other scan, iteratively minimizing the distance between the matched points. Other registration techniques may also be used.

[0228] The intraoral scan application 115 can repeatedly register for all intraoral scans in the intraoral scan sequence to obtain a transformation for each intraoral scan, thereby registering each intraoral scan with (multiple) previous intraoral scans and / or with a common reference frame (e.g., with a 3D model). The intraoral scan application 115 can integrate intraoral scans into a single virtual 3D model by applying appropriately determined transformations to each of the intraoral scans. Each transformation may include rotation about one to three axes and translation in one to three planes.

[0229] In many cases, data from one or more intraoral scans do not perfectly correspond to data from one or more other intraoral scans. Therefore, in an embodiment, the intraoral scan application 115 can process intraoral scans (e.g., it may be a mixed intraoral scan) to determine which intraoral scans (or which portions of intraoral scans) are used for portions of the 3D model (e.g., portions representing specific tooth locations). The intraoral scan application 115 can use data such as geometric data represented in the scans and / or timestamps associated with the intraoral scans to select the optimal intraoral scan for depicting tooth locations or portions of tooth locations (e.g., for depicting the edge lines of prepared teeth). In one embodiment, images are fed into a machine learning model that has been trained to select and / or rate tooth locations. In one embodiment, one or more scores are assigned to each scan, where each score may be associated with a specific tooth location and indicate the quality of the representation of that tooth location in the intraoral scan.

[0230] Additionally or alternatively, intraoral scans can be weighted based on scores assigned to those scans. The assigned weights can be associated with different tooth locations. In one embodiment, weights can be assigned to each scan for tooth locations (or multiple tooth locations) (e.g., assigned to each merged scan). During model generation, conflicting data from multiple intraoral scans can be combined using a weighted average to depict tooth locations. The applied weights can be those assigned based on the quality scores of the tooth locations. For example, processing logic can determine that data from a first set of intraoral scans for a specific overlapping region is of higher quality than data from a second set of intraoral scans for a specific overlapping region. Subsequently, when averaging the differences between the intraoral scan datasets, the first intraoral scan dataset can be weighted more heavily than the second intraoral scan dataset. For example, the first intraoral scan, assigned a higher rating, can be assigned 70% weight, while the second intraoral scan can be assigned 30% weight. Therefore, when the data is averaged, the merged result will look more like a depiction from the first intraoral scan dataset and less like a depiction from the second intraoral scan dataset.

[0231] Intraoral scanning application 115 can generate one or more 3D models from an intraoral scan and display these 3D models to a user (e.g., a physician) via a user interface. The physician can then visually examine the 3D models. The physician can virtually manipulate the 3D models via the user interface using appropriate user controls (hardware and / or virtual) about up to six degrees of freedom (i.e., translation and / or rotation about one or more of three mutually orthogonal axes) to allow viewing the 3D models from any desired direction. The physician can review (e.g., visually examine) the generated 3D models of intraoral points and determine whether the 3D models are acceptable (e.g., whether the edges of prepared teeth are accurately represented in the 3D model). In some embodiments, intraoral scanning application 115 automatically generates a sequence of views of the 3D model and scrolls through the views in the sequence. This may include zooming in, zooming out, rocking, rotating, etc.

[0232] The intraoral scanning application 115 may include logic for automating one or more operations that are traditionally performed manually by the user; these operations are referred to herein as smart scanning. The user can enter smart scanning mode by selecting to perform a smart scan from the user interface of the intraoral scanning application 115. Alternatively, the intraoral scanning application 115 may default to smart scanning mode. At any time, the user can choose to exit smart scanning mode. Multiple stages and workflows for intraoral scanning are provided, along with a description of how to remove each stage / workflow where the user needs to add input (in addition to the scan).

[0233] Automatic user identification (scanner login)

[0234] In some embodiments, one or more forms of automatic user identification are performed to determine the identity of the doctor (or other user) of scanner 150 and / or to log the doctor into intraoral scanning application 115 before or at another time. Examples of automatic user identification that may be used include facial recognition, fingerprint recognition, voice recognition, other biometric information, and / or scanner motion recognition. For facial recognition, the user may use scanner 150 to generate one or more images of their face. Alternatively, computing device 105 may include a separate camera (not shown) that can capture one or more images of the user's face. Computing device 105 may have images of the user's face from scanner 150 and / or a trained machine learning model trained to recognize the user's face from scanner 150, and may subsequently perform facial recognition to identify the user (e.g., by feeding the captured images into the trained machine learning model).

[0235] For speech recognition, scanner 150 and / or computing device 105 may include microphones. The one or more microphones may capture audio of the user speaking (e.g., uttering a specific login phrase). Computing device 105 may have recorded audio of the user of scanner 150 and / or a trained machine learning model trained to recognize the user's speech, and may subsequently perform speech recognition to identify the user (e.g., by inputting the captured audio into the trained machine learning model and / or comparing the captured audio with stored audio).

[0236] Scanner 150 may include one or more motion sensors (e.g., gyroscopes and / or accelerometers). Users can set a scanner motion password by selecting the option to record a scanner motion password in the intraoral scanning application 115, and then moving scanner 150 as they see fit (e.g., moving the scanner up, down, left, right, forward, backward, rotating the scanner, etc.). Once the scanner motion password is recorded, an unknown user can log in to their user ID or account on the intraoral scanning application 115 by moving scanner 150 using their recorded scanner motion password. The intraoral scanning application 115 can compare the received scanner motion (e.g., which may include a series of accelerations and / or rotations) with a set of recorded scanner motions. If a match is identified between the received scanner motion and the stored scanner motions associated with the user account, scanner motion identification can be successful, and the user can be identified and logged into their account.

[0237] The intraoral scanner 150 may include a touchpad or other touch-sensitive input device, which in this embodiment may function as a fingerprint reader. For fingerprint recognition, a user may press a finger (e.g., thumb) onto the touchpad or other touch-sensitive input device. The user's fingerprint may be determined based on the user's press on the touchpad or other touch-sensitive input device. The fingerprint may be compared with one or more stored fingerprints. If a match is identified between the detected fingerprint and a fingerprint associated with a user account, the doctor associated with that user account can be identified, and the doctor can be automatically logged into their user account.

[0238] Automatic patient identification

[0239] As described above, the intraoral scanning application 115 can automatically identify the user of the scanner 150 and log the user into their account on the intraoral scanning application 115 using one or more identification technologies. Additionally or alternatively, the intraoral scanning application 115 can automatically identify the patient. This patient identification can be performed before, during, or after the patient's intraoral scan.

[0240] For first-time patients, the username and details are added to the patient record manually or automatically. For manual entry, the physician or technician can input patient details into the intraoral scanning application 115. For automatic entry, the intraoral scanning application 115 can access practice management software, calendar software, and / or other software containing patient information, and can retrieve that patient information and automatically populate the patient input in the intraoral scanning application 115 using the retrieved patient information. For example, information about patients assigned to the current dental chair at the current time can be automatically retrieved from practice management software and / or calendar software.

[0241] Once user information has been added once, the doctor can simply begin a scan, and the intraoral scanning application can automatically identify the patient based on the patient's tooth shape. For example, the patient may have previously undergone an intraoral scan, and one or more 3D models of the patient's dental arch can be associated with input specific to that patient. The doctor can begin scanning the patient's mouth during a later visit without first entering information to identify the patient. The intraoral scanning application 115 can register and stitch the intraoral scans together to generate a 3D surface and / or 3D model of the dental arch as described above, and can compare the 3D surface or 3D model with stored 3D models of the dental arches of one or more patients. If a match or approximate match is found between the archived 3D surface or 3D model of the patient's dental arch and the stored 3D model, the patient can be automatically identified based on that match or approximate match. Subsequently, the 3D model generated based on the current visit can be automatically associated with the patient and stored in the patient file, and / or a prescription can be started for the patient, and patient information can be automatically added to the prescription.

[0242] Automatic start and stop of scanning

[0243] In an embodiment, scanner 150 may capture images (e.g., color images) at a frequency (e.g., at approximately 10 Hz) with or without illumination (e.g., with minimal illumination). The system may detect when scanner 150 begins to enter the oral cavity based on the generated images. In some embodiments, intraoral scanning application 115 or scanner 150 processes the received images to determine whether objects typically found in or around the oral cavity (e.g., teeth, lips, tongue, etc.) are identified. For example, intraoral scanning application 115 or scanner 150 may include a trained machine learning model trained to perform object classification of images and detect the presence of certain objects associated with the face, mouth, and / or oral cavity, such as teeth, lips, tongue, etc. When transitioning from an image where no intraoral objects are detected to an image where intraoral objects are detected, intraoral scanning application 115 or scanner 150 may determine that scanner 150 has been inserted into the patient's mouth. When the system detects that scanner 150 has been inserted into the oral cavity, intraoral scanner 150 may automatically begin generating an intraoral scan. This may include intraoral scanning application 115 sending a command to scanner 150 to begin scanning.

[0244] When transitioning from an image that detected an intraoral object to an image that did not, the intraoral scanning application 115 or scanner 150 can determine that the scanner 150 has been removed from the patient's oral cavity. When the system detects that the scanner 150 has been removed from the oral cavity, the intraoral scanner 150 can automatically stop generating intraoral scans. This may include the intraoral scanning application 115 sending a stop scan command to the scanner 150. In some cases, the intraoral scanning application 115 may automatically begin generating one or more 3D models of dental arches and / or perform post-processing and / or diagnostics on the generated 3D dental arch models in response to detecting the removal of the intraoral scanner 150 from the patient's oral cavity.

[0245] The intraoral object will initially appear in the field of view (FOV) of scanner 150 (e.g., in the FOV of the front camera of scanner 150 or in front of the FOV of scanner 150) and will subsequently be shown moving behind the camera as scanner 150 enters the oral cavity. For example, the image of the intraoral object will initially appear in the FOV of the front camera of scanner 150 (if the scanner includes multiple cameras) or in front of the FOV of scanner 150 (e.g., if the scanner includes only a single camera and / or a single FOV), and subsequently in the FOV of the rear camera of scanner 150 or behind the FOV of scanner 150 as the scanner enters the oral cavity. Additionally, as scanner 150 is removed from the oral cavity, the intraoral object will be shown moving in the opposite direction to that shown when scanner 150 was inserted into the oral cavity. For example, an image of an intraoral object will initially appear in the FOV of the rear camera of scanner 150 (if the scanner includes multiple cameras) or behind the scanner's FOV (e.g., if the scanner includes only a single camera and / or a single FOV), and subsequently in the FOV of the front camera of scanner 150 or in front of the scanner's FOV when the scanner is removed from the oral cavity. Intraoral scanning application 115 can detect movement of intraoral objects and / or detect transitions between detected and undetected intraoral objects, and can use this information to improve the accuracy of scan start and stop decisions.

[0246] In addition to automatically starting and stopping scans, the system can automatically start and stop one or more light sources based on a determination of whether the scanner 150 is in the oral cavity. For example, for a scanner 150 using a structured light (SL) projector, the intraoral scanning application 115 and / or the intraoral scanner 150 can automatically turn the structured light (SL) projector on and off. This can reduce or eliminate hazy light projection (known as the disco effect) onto objects in the dental clinic room when the scanner 150 is removed from the patient's mouth.

[0247] In one embodiment, the intraoral scanning application 115 provides feedback in a graphical user interface (GUI) to show that the system understands its location. This may include providing indications (e.g., visual indications) as to whether the intraoral scanning application 115 has detected the scanner 150 inside or outside the patient's mouth. Additionally or alternatively, the intraoral scanning application 115 may provide indications (e.g., visual indications) in the GUI as to whether the maxillary arch is currently being scanned, the mandibular arch is currently being scanned, the patient's occlusion is currently being scanned, and whether a restorative object has been detected.

[0248] Automatic role recognition

[0249] The scanning process typically involves several stages—so-called roles (also referred to as scan roles). Three main roles are the maxillary role (also referred to as the maxillary arch role), the mandibular role (also referred to as the mandibular arch role), and the occlusal role. The occlusal role refers to the role of the relative position of the maxilla and mandible when the jaw is closed. Traditionally, the user of scanner 150 selects a target role through the user interface of intraoral scanning application 115, and scanning only continues after such user input. In this embodiment, the intraoral scanning application is configured to eliminate such user input and automatically identify roles during scanning. In this embodiment, intraoral scanning application 115 automatically determines whether the user is currently scanning the teeth in the maxilla (maxillary role), scanning the teeth in the mandible (mandibular role), or scanning both the teeth in the maxilla and mandible when the patient's jaw is closed (occlusal role). Intraoral scanning application 115 can then assign the detected role to intraoral scan data, 3D surfaces, and / or the 3D model of the detected role. Therefore, intraoral scanning application 115 can automatically determine whether the user is scanning the maxilla, mandible, or occlusion, and appropriately label the intraoral scan based on such determination. Additionally or alternatively, the 3D surfaces and / or 3D models generated from such intraoral scans can also be labeled using defined roles.

[0250] In some embodiments, individual roles are assigned to each prepared tooth and / or other restorative object on the dental arch. Thus, roles may include maxillary roles, mandibular roles, occlusal roles, and one or more preparation roles, wherein a preparation role may be associated with a prepared tooth or another type of preparation or restorative object. In addition to automatically identifying maxillary, mandibular, and occlusal roles, the intraoral scanning application 115 may also automatically identify preparation roles from intraoral scan data, 3D surfaces, and / or 3D models. In some embodiments, a preparation may be associated with both a jaw role (e.g., a maxillary or mandibular role) and a preparation role.

[0251] In some embodiments, the intraoral scanning application 115 uses machine learning to detect whether the intraoral scan depicts the maxillary arch (maxillary role), the mandibular arch (mandibular role), or occlusion (occlusal role). In some embodiments, the intraoral scanning application 115 uses machine learning to detect whether the intraoral scan depicts the maxillary arch (maxillary role), the mandibular arch (mandibular role), occlusion (occlusal role), and / or preparation (preparation role). When generating intraoral scan data, intraoral scans from the intraoral scan data and / or 2D images from the intraoral scan data can be fed into a trained machine learning model that has been trained to identify roles. The trained machine learning model can then output a classification of roles(s) for the intraoral scan data. In some embodiments, the intraoral scanning application 115 generates a 3D surface by stitching together multiple intraoral scans and feeds data from the 3D surface into a trained machine learning model that outputs a classification of roles(s) for the 3D surface. The same ML model can be used to process both the intraoral scan data and the data from the generated 3D surface. Alternatively, different ML models can be used to process intraoral scan data and data from 3D surfaces. The one or more ML models can process 3D data (e.g., 3D surfaces) or 2D data (e.g., height maps or projections of 3D surfaces onto a 2D plane). The intraoral scan application 115 can provide feedback via a graphical user interface (GUI) to show the system its location (e.g., whether the current role is upper arch, lower arch, occlusion, or preparing teeth).

[0252] Examples of this implementation use machine learning to classify 2D images, intraoral scans, 3D surfaces, and / or height maps into their associated scan roles. One implementation uses a deep neural network to learn how to associate input images, intraoral scans, 3D surfaces, and / or height maps with human-labeled scan roles. The result of this training is a function that can directly predict the label of a scan role from the input images, intraoral scans, 3D surfaces, and / or height maps. Possible inputs may be individual height maps or intraoral scans, 3D surfaces, occlusal views of the jaw generated by stitching together multiple height maps or intraoral scans, and / or multiple jaw views (e.g., generated by stitching together multiple height maps and / or intraoral scans).

[0253] Using data from individual intraoral scans and / or images (e.g., individual height maps) and / or 3D surfaces from multiple intraoral scans and / or images (referred to as multiple jaw views) to determine roles can enable the selection or identification of the moment when one role changes to another. For example, intraoral scanning application 115 can automatically determine when a physician transitions from scanning the lower dental arch to scanning the upper dental arch, or vice versa. Processing logic can additionally determine when the scanning of the dental arch is complete and can automatically continue generating a 3D model of the dental arch and / or perform one or more other operations in response to such determination. For example, when all upper, lower, and occlusal roles are complete, intraoral scanning application 115 can calculate and display occlusal gap information (e.g., such as via an occlusal map) and / or automatically perform occlusal gap calculations (e.g., in a different part of the screen than where the 3D surface or 3D model is displayed).

[0254] In some embodiments, when an intraoral scan is received and a 3D surface is generated, the intraoral scanning application 115 continuously or periodically determines roles associated with the intraoral scan and / or the 3D surface. Role classification based on a single scan may not be as accurate as that based on data from multiple intraoral scans (e.g., from 3D surfaces or multiple jaw views). Since errors may be unavoidable for individual predictions (e.g., based on a single intraoral scan), an aggregated solution may be used in some embodiments. For example, the intraoral scanning application 115 may perform an initial classification of the intraoral scan and subsequently perform further classification on additional intraoral scans stitched to the initial intraoral scan and / or on 3D surfaces generated from the stitching of the initial intraoral scan to the additional intraoral scans. Further classification can be more accurate than the initial classification. Therefore, the accuracy of role classification can continuously improve as further intraoral scans are generated. Thus, roles associated with segments of the dental arch can be correctly classified in real time and can be further combined with corresponding segments of the same role / dental arch.

[0255] In one example, stitching can be incorporated, allowing classification to be performed across the entire 3D surface (e.g., segments of stitched scans). In another example, for better accuracy, intraoral scanning application 115 can use a statistical method based on classification of multiple intraoral scans. For example, intraoral scanning application 115 can classify a sequence of intraoral scans and then assign classifications to the entire sequence based on the majority classification of individual intraoral scans within the sequence. In one embodiment, intraoral scanning application 115 uses predicted moving averages where the dominant classification in some number (e.g., 5, 10, 20, 50, 100, etc.) of the most recent intraoral scans is determined to be the role of those most recent intraoral scans.

[0256] Each prediction or classification for a role obtained through intraoral scanning may be accompanied by an uncertainty value. The higher the uncertainty value, the lower the certainty of a correct prediction. In one embodiment, intraoral scanning applies 115 to discard the most uncertain predictions. This may include discarding predictions with uncertainty that fails to meet a certain criterion (e.g., failing to meet a threshold, such as a 50% certainty threshold) and / or discarding a set number of predictions with the highest uncertainty values. Thus, a moving average of the predictions can be used to determine the role, where one or more predictions in the moving average have been discarded.

[0257] Typically, it's best to remove moving tissue (lips, cheeks, tongue, etc.) from the input to make the final model cleaner. However, in the context of scanned character recognition, it's preferable to use the raw data with moving tissue, as it can provide additional information helpful in character identification. For example, the tongue is often associated with jaw characters, and the shape of the lips differs for the maxilla and mandible. All these specific features can give much better accuracy in character recognition compared to clean input (where moving tissue has already been removed from the scan). Therefore, in this embodiment, the scanned data input into the machine learning model to determine the character has not yet been processed by a moving tissue removal algorithm and / or has not yet had moving tissue removed.

[0258] The scanned role is just one of many possible features that can be identified given the inputs described above. Other features may also be identified. For example, the intraoral scanning application 115 can determine one or more additional classifications of intraoral scans, 3D surfaces, height maps, 2D images, etc. In embodiments, one or more trained machine learning models can be used to make such determinations. In one embodiment, a single machine learning model can be trained to assign multiple types of classifications to the input intraoral scan data (e.g., including role classification and one or more additional types of classification). In one embodiment, different trained machine learning models are used to determine different types of classifications. In one embodiment, the intraoral scanning application 115 determines, based on the input intraoral scan data and / or intraoral 3D surface data, whether the intraoral scan data depicts the lingual or buccal side of the jaw. In one embodiment, the intraoral scanning application 115 determines, based on the input intraoral scan data and / or intraoral 3D surface data, whether orthodontic treatment and / or restorative treatment should be performed. In one embodiment, the intraoral scanning application 115 determines, based on input intraoral scan data and / or intraoral 3D surface data, whether any brackets and / or attachments are detected on the patient's teeth, and optionally determines the location (e.g., segmentation) of such brackets and / or attachments. In this embodiment, all these features can be identified simultaneously from a single model and even support each other in terms of predictive accuracy.

[0259] By segmenting intraoral scan data and / or intraoral 3D surface data into dental categories such as teeth, gums, excess material, etc., the accuracy of role and / or other features (such as those described above) recognition can be improved. One or more trained machine learning models can be trained to perform this segmentation of intraoral scan data and / or intraoral 3D surface data. The same machine learning model can perform this segmentation as well as one or more of the categories described above. Alternatively, one or more separate machine learning models can perform this segmentation.

[0260] In one embodiment, soft tissue classification (e.g., tongue, cheek, lips, palate, etc.) is performed by inputting intraoral scans, height maps, images, 3D surfaces, projections of 3D models, etc., into one or more trained machine learning models.

[0261] In some embodiments, the input intraoral scan data / 3D surface data is limited to intraoral scans (e.g., height maps) and / or 3D surfaces (or projections of 3D surfaces onto one or more planes) generated by stitching together such intraoral scans. In other embodiments, the input layers / data used (e.g., input to one or more trained machine learning models) include color images (e.g., color 2D images) and / or images generated under specific lighting conditions (e.g., NIRI images). Scanner 150 can generate intraoral scans (which include height information) and color 2D images and / or other 2D images separately. Intraoral scans and 2D images can be generated in sufficiently close timeframes to depict the same or nearly identical surfaces. 2D images (e.g., color 2D images) can provide additional data that improves the differentiation between teeth and gums, tongue, etc., due to color differences between these objects.

[0262] To better account for multiple inputs, in some embodiments, a recurrent neural network (RNN) is used to classify roles and / or one or more additional features, as described above. Using an RNN allows the system to identify features based on a sequence of scans and can improve accuracy. In some embodiments, one or more trained machine learning models (which may or may not be RNNs) include multiple input layers, each of which can receive a separate intraoral scan. The trained ML model can then make a prediction or classification (e.g., for a scanned role) based on the multiple scans. This can be combined with input layers for additional information such as color 2D images and / or NIRI images.

[0263] In some embodiments, as described above, the processing logic automatically identifies scan roles and automatically assigns these automatically identified scan roles to the 3D surface of the dental arch, intraoral scan, and / or 3D model. Alternatively, in embodiments, the user can manually select scan roles, 3D surfaces of the dental arch, and / or 3D models of the dental arch for one or more intraoral scans. In this embodiment, the processing logic may automatically perform role classification as described herein and may output a warning if a role different from the one entered by the physician is detected. In one embodiment, the processing logic outputs a notification stating that an alternative role has been detected and asks the physician whether to assign the alternative role to the intraoral scan 3D surface and / or 3D model.

[0264] Multiple bite detection

[0265] Related to the detection of occlusal roles, the intraoral scanning application 115 in this embodiment can also detect multiple occlusal scenarios. In multiple occlusal scenarios, different occlusions can be recorded, showing different relationships between the maxilla and mandible. In addition to classifying occlusal roles, the intraoral scanning application 115 can compare and / or analyze scan data for different intraoral scans classified as occlusions. Alternatively or additionally, the intraoral scanning application 115 can apply machine learning to classify multiple occlusal scenarios. The intraoral scanning application 115 can detect differences between occlusions and determine whether such differences represent only a variation in a single occlusion or whether such differences represent multiple different occlusions (referred to as multiple occlusal detection). In some cases, multiple occlusions occur under the guidance of a physician, who may have instructed the patient to bite in different ways. The system in this embodiment can automatically detect such multiple occlusal scenarios using the application of machine learning or based on comparisons of multiple occlusal scans.

[0266] In one example, the physician may need to use scanner 150 to enter and exit the oral cavity and wait a few seconds. The scan of the occlusal portion can be recorded separately. Sometimes, unintentional movements can create discrepancies in the occlusal scan. To understand whether this is anticipated polyocclusion or an error, the system can output an indication of potential polyocclusion and request confirmation from the physician. Alternatively, the system can automatically determine the presence of polyocclusion. When making this determination, the intraoral scanning application 115 can consider occlusal position (e.g., differences between different occlusions) and / or the time taken to perform the occlusion-related scan. This feature can be useful in cases involving bite elevation.

[0267] Regarding the detection of multiple occlusions, some possible scenarios include: (a) both sides of the mouth show the same occlusal relationship, indicating that everything is normal and there is no multiple occlusion; and (b) both sides of the mouth show different occlusal relationships, indicating that it is necessary to determine whether it is a distortion or a multiple occlusion. In an embodiment, to determine whether the detected difference is due to distortion or a multiple occlusion, the system may consider both the time interval between two occlusal scans and the size of the change or difference in occlusion between the two scans. In one embodiment, the time interval between two occlusal scans is determined (e.g., it may be measured in seconds), and the time interval is compared with a time interval threshold. The time interval threshold may be, for example, 1 second, 2 seconds, 4 seconds, 10 seconds, etc. In one embodiment, the two occlusal scans are compared, and the difference in occlusion between the two scans is calculated (e.g., it may be measured in micrometers). The occlusal difference can then be compared with an occlusal difference threshold. The occlusal difference threshold may be, for example, 50 micrometers, 75 micrometers, 100 micrometers, 150 micrometers, 200 micrometers, etc. In one embodiment, if the time interval exceeds a time interval threshold and the occlusal difference exceeds an occlusal difference threshold, then the intraoral scanning application 115 determines that the two occlusal scans represent a multi-occlusal scenario.

[0268] Automatic identification of restorative objects

[0269] In some embodiments, the intraoral scanning application 115 is capable of automatically identifying intraoral scans and / or 3D surfaces depicting a restorative object. A restorative object may be, for example, a preparation or a scan body. Restorative objects may also include, for example, dental prostheses, such as implants, crowns, inlays, onlays, caps, veneers, etc. While the term preparation generally refers to the preparation of a tooth stump, including the margin and shoulder of the remaining tooth portion, the term preparation herein also includes artificial stumps, pivots, cores and posts, or other devices implantable in the oral cavity to receive crowns or other prostheses. The embodiments described herein with reference to tooth preparation are also applicable to other types of preparations, such as the aforementioned artificial stumps, pivots, etc. In some embodiments, processing logic automatically identifies restorative objects in intraoral scans, images, 3D surfaces, and / or 3D models. For example, the processing logic may perform pixel-level classification of intraoral scans, images, 3D surfaces, and / or 3D models, wherein at least one classification is for a restorative object.

[0270] In some embodiments, the intraoral scanning application 115 includes one or more trained machine learning models (e.g., neural networks) trained to perform classification of tooth locations, wherein at least one classification is for restorative objects. The trained machine learning models(s) may perform image-level / scan-level classification, pixel-level classification, or classification of groups of pixels. Traditionally, physicians manually identify restorative objects. This embodiment provides an improved user experience by eliminating the need for physicians to manually identify restorative objects. In this embodiment, multiple features (e.g., the type of restorative object) may be identified simultaneously from a single trained machine learning model.

[0271] Restorative treatment cases involve very specific objects, such as implants, scan bodies, and so-called preparations. Identification of these objects is useful for further processing and treatment. Manual identification is very time-consuming and error-prone. This embodiment eliminates the need for manual identification / input of restorative objects.

[0272] In one embodiment, intraoral scanning application 115 uses machine learning to classify intraoral scan data / intraoral 3D surface data into relevant dental classifications, which may include preparations, scan bodies, regular teeth, etc. One implementation uses a deep neural network to learn how to associate an input image with human-labeled dental classifications, where the dental classifications include regular teeth and one or more restorative objects. The result of this training is a trained machine learning model that can predict labels directly from the input scan data and / or 3D surface data. Input data may be individual intraoral scans (e.g., height maps), 3D surface data (e.g., 3D surfaces from multiple scans or projections of such 3D surfaces onto a plane), and / or other images (e.g., color images and / or NIRI images). Such data may be available in real-time during scanning. Additionally, the intraoral scan data associated with a single scan may be large enough (e.g., the scanner may have a sufficiently large FOV) to include at least one tooth and its surrounding environment. Given an input based on a single intraoral scan, the trained neural network is able to predict whether the scan (e.g., height) contains any of the aforementioned dental classifications. This predictive nature can be probabilistic: for each category, there is a probability that it will appear on the intraoral scan. This approach allows the system to identify regions on the 3D surface and / or 3D model generated from the intraoral scan that are associated with the restorative object and therefore should be treated differently from natural teeth. For example, such regions can be scanned at a higher resolution than other regions that do not include the restorative object, and / or a higher resolution can be used for the 3D surface generated from a scan that includes the restorative object. Thus, the system can automatically determine whether a scan depicts a restorative object, and when a 3D model is generated using that scan, further processing can be performed to generate a higher resolution for that region of the model. As a result, the restorative object in the 3D model can have a higher resolution than other objects (e.g., other teeth) in the 3D model. Therefore, in embodiments, 3D models with variable resolution can be generated.

[0273] In some embodiments, in addition to using a higher resolution to depict restorative objects, or instead of using a higher resolution to depict restorative objects, the intraoral scanning application 115 uses a higher resolution to depict other types of dental objects. For example, the intraoral scanning application 115 can identify the boundaries of gingiva and teeth (or multiple teeth and surrounding gingival margins) and can use a higher resolution for such boundaries in the 3D model. In another example, the intraoral scanning application can identify interdental regions between teeth (e.g., tooth-to-tooth boundaries) and can use a higher resolution to depict interdental regions in the 3D model. In yet another example, the intraoral scanning application is able to detect edge lines and depict edge lines in the 3D model using a higher resolution.

[0274] In one embodiment, a 3D model (or multiple portions of a 3D model initially generated at a first resolution) is generated, and smoothing and / or simplification operations are performed to reduce the first resolution to a lower second resolution for one or more regions of the 3D model or one or more regions of one or more portions of the 3D model (e.g., by reducing the number of points, vertices, and / or polygons per unit area). Two techniques that can be used to simplify 3D surfaces are point removal (where a certain percentage of points in a region are deleted) and edge shrinkage (where the two endpoints of a triangle's edge are replaced with a single point, and the triangle is subsequently redrawn). For example, an intraoral scanning application 115 can identify restorative objects, tooth-to-gingival boundaries, edge lines, and / or tooth-to-tooth boundaries in the 3D model. Processing logic can then reduce the resolution of those regions not included in the 3D model that are identified as restorative objects, tooth-to-gingival boundaries, edge lines, and / or tooth-to-tooth boundaries. In some embodiments, a multi-resolution 3D model may include more than two different resolutions. For example, gingiva can be represented using a first resolution, which is the lowest resolution. Natural teeth, excluding the areas adjacent to other teeth or gingiva, can be represented using a second resolution, which is a higher resolution than the first. Furthermore, tooth-to-gingiva boundaries, restorative objects, and tooth-to-tooth boundaries can be represented using a third resolution, which is higher than the second resolution. In some cases, the preparation of tooth margins can be represented using a fourth resolution, which is still higher than the third resolution.

[0275] In some embodiments, the intraoral scanning application 115 generates a multi-resolution 3D model of the dental arch by determining whether the intraoral scan / 3D surface depicts a restorative object and / or whether a higher resolution representation of another type of object is used. The intraoral scanning application 115 may use a first resolution for portions of the 3D model depicting restorative objects, margins, interdental regions, tooth-to-gingival boundaries, and / or other regions identified at a higher resolution, and may use a second resolution for portions of the 3D model not depicting restorative objects or other regions identified at a higher resolution. Restorative objects in the 3D model of the dental arch can benefit from increased resolution because these regions can be used, for example, to determine the size and / or shape of the internal surfaces of the oral restoration to be placed on the restorative object. Similarly, margins, interdental regions, and / or tooth-to-gingival boundaries can benefit from increased resolution. However, increasing the resolution of other regions of the 3D model can be disadvantageous and may unnecessarily increase processor utilization, memory utilization, etc. Multi-resolution 3D models of the dental arch offer the advantages of high resolution for restorative objects and optional other areas such as margins, interdental regions, and / or tooth-to-gingival boundaries, as well as lower resolution for the remainder of the 3D model of the dental arch.

[0276] After the intraoral scanning process is completed, the resulting 3D model may have a rough-looking surface and / or the file size of the 3D model may be too large, creating problems with saving to memory and / or transferring files containing the 3D model. As described above, in some embodiments, a higher resolution is initially used to depict one or more types of dental objects in the 3D model, and a lower resolution is initially used to depict one or more other types of dental objects in the 3D model. Alternatively, a 3D model with a single resolution may be generated initially. In either case, after the 3D model is generated, smoothing and / or simplification operations may be performed on one or more regions of the 3D model to reduce the resolution of the 3D model at one or more regions. This allows the intraoral scanning application 115 to avoid over-smoothing or over-simplification of regions of interest (thus hiding tooth features such as margins, tooth-to-gingival boundaries, tooth-to-tooth boundaries, tooth features, etc.) and / or under-smoothing or under-simplification of other regions. Thus, high resolution can be maintained in regions of interest while smoothing and / or simplification can be performed on regions of non-interest to reduce the resolution at those regions. Regions of interest can be identified by inputting a 3D model, multiple parts of a 3D model, a projection of the 3D model onto one or more planes, and / or an intraoral scan used to generate the 3D model into a trained ML model trained to perform dental object classification (e.g., pixel-level classification of dental objects) and / or by applying a cost function to the output of the trained ML model. This can include identifying restorative objects, natural teeth, gingival margins, tooth-to-tooth boundaries, tooth-to-gingival boundaries, edge lines, and / or other dental objects or classifications.

[0277] In some embodiments, different regions may have varying degrees of reduced resolution. For example, a region representing the gingiva (which may not be a region of interest) may have the largest reduction in resolution, while a region representing natural teeth (which may also not be a region of interest) may have a smaller reduction in resolution. For example, a region of interest (AOI) may include restorative objects, tooth-to-tooth boundaries, tooth-to-gingiva boundaries, and / or edge lines. In embodiments, the AOI is user-selectable. For example, a user can select from a drop-down menu which types of dental objects or classifications correspond to the AOI.

[0278] To better locate the target object, the system can divide the heightmap into regions (grid-like) and detect restorative objects for each such region. For example, a region may comprise a block of pixels. As a variation of this approach, a central region can be used, such that an object is identified only if it is fully presented on the intraoral scan rather than only partially viewed. For example, if a restorative object is identified in the central region of an intraoral scan, the intraoral scan can be classified as a restorative object scan. However, if the central region of the intraoral scan does not contain a restorative object (even if some other regions of the intraoral scan do contain restorative objects), the intraoral scan may not be classified as a restorative object scan.

[0279] In some embodiments, the system can not only identify the presence of restorative objects on intraoral scans and / or 3D surfaces (e.g., height maps), but also segment the intraoral scans and / or 3D surfaces according to any dental classification discussed herein. Thus, each pixel of the intraoral scan and / or 3D surface can be classified as belonging to a specific dental classification. This approach allows for better localization of restorative objects and ultimately enables complete segmentation of the 3D model by combining the segmentation from the intraoral scans.

[0280] For better accuracy, additional inputs can be used, such as color layers, NIRI layers, layers for multiple scans, etc., as discussed elsewhere in this paper. Restorative object recognition can be combined with other recognition problems, such as tooth / gingival / excess material segmentation, bracket / attachment detection, role detection, etc.

[0281] Automated prescription (Rx)

[0282] In this embodiment, as further detailed below, the intraoral scanning application 115 can automatically generate prescriptions for treating a patient. The intraoral scanning application 115 can automatically generate prescriptions for orthodontic treatment and / or restorative treatment. Prescriptions for orthodontic treatment may include a treatment plan for applying a series of aligners to the patient's teeth to correct malocclusion; prescriptions for restorative treatment may include information for dental caps, bridges, dentures, crowns, etc.

[0283] Different clinics may focus solely on orthodontic treatment, solely on restorative treatment, or both. Clinic information can be used to automatically determine whether to generate an orthodontic or restorative treatment prescription. For example, for an orthodontic clinic, this can be automatically determined to generate an orthodontic treatment prescription. For clinics that perform only restorative treatment or both, additional information can be used to automatically generate prescriptions.

[0284] In some cases, a dentist may perform a pre-scan of a patient's dental arch (e.g., before performing any treatment). A pre-scan 3D model of the patient's dental arch can be generated based on the pre-scan. Intraoral scanning applications 115 can save the pre-scan and / or pre-scan 3D model to the patient's record and can recognize the saved scan / 3D model as a pre-scan / pre-scan 3D model. In one example, a pre-scan 3D model can be generated before the teeth are ground down to form the prepared tooth. The pre-scan 3D model can provide information about the shape, color, position, etc. of the teeth before they are ground down to form the prepared tooth. The pre-scan 3D model can then be used for various purposes, such as determining how much of the tooth has been ground down to generate the preparation, thereby determining the shape of the restoration, etc.

[0285] In some cases, intraoral scanning applications 115 (or doctors or clinics) can use older patient scans and / or 3D models of the patient's dental arch. When the system has access to older patient scans / 3D models, it can use these older scans and / or 3D models for a variety of purposes. A non-exhaustive list of uses for older scans and / or 3D models includes: (b) detecting the patient's name (as described above), (c) detecting which tooth is being treated, (d) eliminating the need for pre-scans, and / or (d) calculating the target crown structure and refining margins (where the new and old teeth have the same shape). For (b) and (c), fine local registration of specific teeth may be desired to improve discrepancy accuracy (e.g., if the older scan is from a significant past time).

[0286] In one example, intraoral scanning application 115 can compare a patient's current intraoral scan (or a 3D surface or 3D model generated from the current intraoral scan) with a previous intraoral scan (or a 3D surface or 3D model generated from a previous intraoral scan). Based on this comparison, intraoral scanning application 115 can determine the differences between the patient's current teeth and the patient's previous teeth. These differences can indicate which teeth(s) are being treated. Information about the teeth being treated can be added to the prescription.

[0287] In another example, the intraoral scanning application 115 can provide the physician with an indication that a previous scan (or 3D model) of the patient's dental arch exists. Based on this indication, the physician can choose not to perform a pre-scan. In this scenario, the previously generated scan and / or the 3D model of the patient's dental arch can be used for the same purpose as the pre-scan 3D model would typically be used for. For example, the physician can skip performing a pre-scan before forming the preparatory teeth.

[0288] In another example, the intraoral scanning application 115 can use one or more older patient scans (or 3D models generated from older patient scans) to calculate the target crown structure and / or margins. For example, the one or more older 3D models can provide the shape of the crown.

[0289] Once the intraoral scan is received, basic analysis of the scan can be performed. The intraoral scan application 115 can search for one or more different types of problems. The types of problems that can be searched include:

[0290] (a) Locating teeth and / or other preparation objects, and (b) Searching for scanned bodies. Searching for teeth and / or preparation objects can be performed via 3D image processing and / or via application to machine learning (ML) classification, as described in more detail below. Searching for scanned bodies can be performed via ML classification, as described further below.

[0291] A prescription (Rx) can be automatically generated and / or filled using suggestions based on examinations of the prepared tooth (e.g., including determination of the position and / or shape of the prepared tooth) and / or examinations of the scan body (e.g., including determination of the position, spacing, angle, shape, type, etc. of the scan body). The prescription may include suggestions for appropriate dental appliances ordered from a laboratory based on the prepared tooth. Examples of dental appliances that can be automatically added to a prescription include crowns, bridges, inlays, veneers, and dentures. In embodiments, appropriate dental appliances (e.g., oral restorations) can be determined based on the geometry, position, and / or number of restorative objects (e.g., prepared teeth). For example, clinical decisions regarding a determined prescription (such as a single crown or bridge) can be rule-based or learned (e.g., using machine learning). For example, a machine learning model can be trained using a training dataset that includes inputs to 3D surfaces and / or projections of 3D surfaces, as well as the type of dental prosthesis placed on the prepared tooth and / or other restorative objects in the 3D surface. Machine learning models can be trained to take one or more projections of a 3D surface / 3D model or a 3D surface / 3D model as input and output a prediction of the dental prosthesis to be used.

[0292] This system can recommend materials for dental prostheses based on prior history (general statistics, past physician statistics, laboratory material availability, etc.) and can recommend colors based on contralateral and adjacent teeth. Color can be determined based on the best estimate of adjacent teeth and / or on pre-treatment scans and / or previously generated scans. The determined color can be automatically added to the prescription.

[0293] The system can recommend laboratories (e.g., dental laboratory 110) for use based on the dentist's previous experience. In one embodiment, information such as the laboratory to be used, the materials to be used, and the colors to be used can be automatically added to the prescription. Once the prescription is generated, it can be automatically sent to the dental laboratory 110 indicated in the prescription. Before the prescription is sent to the dental laboratory 110, the dentist can review and approve it.

[0294] This system can automatically generate prescriptions for restorative and / or orthodontic treatments, such as for crowns, canopies, bridges, alignments, etc. For example, the prescription may include the automatic selection of specific materials and / or dental laboratories. Decisions regarding materials and / or dental laboratories can be based on historical statistics and / or machine learning applications. For instance, a machine learning model can be trained using training data including 3D surfaces / models of dental arches and / or preparations labeled with the materials and / or laboratories used. The machine learning model can be trained to receive one or more projections of the 3D surface of the dental arch or preparation and output predictions of the materials and / or laboratories to be used. Preferences can be associated with case type, username, type of treatment, treatment area, etc. The system can learn physician preferences associated with these attributes and, most of the time, obtain default decision-making power based on such learning. When the system sees no single correct option, it can display multiple (e.g., two) options for the physician to choose from.

[0295] The physician can review other properties of the generated 3D model and / or automatically generated prescription. The physician can make any changes they deem appropriate to the 3D model and / or prescription. This may include generating one or more additional intraoral scans and using these scans to update the 3D model, changing the materials used for the dental restoration, changing the laboratory to which the prescription is sent, changing the color of the restoration, etc. Each decision made by the physician can be entered into a learning database specific to that physician and used to update one or more machine learning models that can be specifically trained for that physician's preferences. Once the physician (e.g., a dentist) has determined that the 3D model and / or prescription is acceptable, the physician can instruct computing device 105 to send the prescription to computing device 106 of the dental laboratory 110. Alternatively, such instructions can be generated and sent automatically.

[0296] Intraoral scanning application 115 and / or a separate dental modeling application can analyze a 3D model to determine if it is suitable for fabrication of a dental prosthesis. The intraoral scanning application or dental modeling application may include logic to identify edge lines and / or modify the surface of one or more tooth points and / or modify edge lines. If the 3D model is deemed suitable (or can be modified to place it in a condition deemed suitable), a dental prosthesis can be fabricated from the 3D model.

[0297] In an embodiment, the intraoral scanning application 115 analyzes the generated 3D surface or 3D model and determines one or more quality ratings for the 3D model or surface. Different quality ratings can be assigned to different parts of the 3D model, such as multiple parts of the edge line, multiple areas of the prepared tooth, multiple areas around the prepared tooth, etc. The intraoral scanning application 115 can provide feedback on areas that fail to meet specific quality criteria and can benefit from rescanning to generate a better quality 3D model of the tooth locations. For example, the intraoral scanning application 115 can determine the amount of scanned gingiva or gum tissue around the tooth, particularly around the prepared tooth. The scanned amount of gingiva can be compared to a scanned gingival threshold, and if the detected scanned amount of gingiva around the tooth is less than the threshold, the intraoral scanning application 115 can mark the tooth for further gingival scanning. In one example, it is beneficial to scan at least 3 mm of gingival tissue around each tooth. Therefore, if the outer boundary of any scanned area of ​​gingival tissue around the tooth is less than 3 mm from the tooth, that area of ​​gingival tissue and / or the tooth can be marked for further scanning by the dentist.

[0298] In another example, the intraoral scanning application 115 can detect missing regions where no intraoral scan data was generated. This may include missing jaw scan data, unscanned teeth, incomplete scans of teeth, holes or gaps in the scan information (e.g., gaps above a threshold size), etc. Such missing regions can be marked for further scanning by the physician. In another example, the intraoral scanning application 115 can detect whether a scanning protocol has been followed and can mark one or more deviations from the scanning protocol. For example, the intraoral scanning application can determine whether one or more occlusal scans are missing. In another example, the intraoral scanning application 115 determines whether any portion of the edge line is unclear, poorly formed, or occluded, as described above. In another example, the intraoral scanning application determines the color quality of the region based on a 2D color image of the region. If an inadequate 2D color image of the region has been generated, the color quality of the region may be low. Therefore, the intraoral scanning application 115 can mark the region for further scanning to receive additional color information for that region. In another example, surface quality (e.g., the number of known points on the surface) may depend on the number of scans already received for that surface. A region can be generated by performing a small number of scans on its surface, but this region has low determinism or low quality. Intraoral scanning application 115 can mark regions with too few data points for further scanning.

[0299] Typically, after a physician completes an intraoral scan and a virtual 3D model has been generated, the physician manually moves the model in 3D to determine its suitability. In this embodiment, the system automatically determines and generates rotation paths and / or scaling sequences, as the physician will do after the scan and after preparation. This can then be automatically replayed for the physician. The system can learn anticipated movements, scaling, rotations, etc., and create trajectories. The system can additionally or alternatively display multiple views on the screen simultaneously, with or without motion. If problem areas have been identified, the system can generate arrows or other identifiers pointing to and / or emphasizing those problem areas (e.g., unclear borders or small distances from the opposite jaw). The automatically generated trajectories can additionally or alternatively magnify the identified problem areas.

[0300] Restorative workflow

[0301] In some embodiments, the intraoral scanning application 115 automatically executes or follows a restorative workflow. Restorative workflows are among the most complex (manual or automated) procedures in dentistry, and dentists typically spend considerable time learning how to perform them. In this embodiment, many aspects of the restorative workflow can be automated, saving significant time and reducing complexity for the dentist. Additionally, automated restorative workflows can reduce the training time required to train dentists to use the intraoral scanning application 115.

[0302] In embodiments, restorative workflows can be performed with or without pre-scanning. In some embodiments, a physician may generate a pre-scanned 3D model of the patient's mouth before performing one or more restorative procedures, such as grinding teeth to form a preparation, tooth extraction, implant insertion, etc. In some embodiments, a prior 3D model of the patient's dental arch may already exist and can be used for the same purpose as the pre-scanned 3D model.

[0303] Intraoral scanning application 115 may have full segmentation and recognition capabilities for teeth and gums, the scanned volume, and other common elements in the oral cavity, as described above. From an intraoral scan as described above and / or from a 3D surface, intraoral scanning application 115 may perform these recognitions and / or segmentations. In one embodiment, to perform an automated restorative workflow, segmentation of teeth, gums, the scanned volume, and prepared teeth is performed automatically.

[0304] A complex aspect of restorative workflows is that dentists may scan teeth or other dental points multiple times during tooth preparation, and may modify these points between scans. A 3D surface and / or 3D model can be generated based on an initial set of intraoral scans. Subsequently, after the dentist makes some changes to the tooth position (e.g., by grinding the teeth), a second set of intraoral scans can be generated. Traditionally, dentists need to mark the 3D model or 3D surface, which includes the dental points, in some way to identify which parts of the 3D model / 3D surface to retain and which parts should be covered using data from the new set of intraoral scans. This system can assume that changes between sets of intraoral scans (e.g., rework of preparation materials, removal of dental floss, addition of dental floss, etc.) all occur between scans.

[0305] In one embodiment, one or more criteria may be used to determine possible changes made to the 3D model or 3D surface of the tooth location. In one embodiment, whenever the dentist removes the scanner 150 from the patient's mouth and stops scanning, the intraoral scanning application 115 measures the amount of time elapsed between scans. The elapsed time between scans may be compared to a time threshold. The time threshold may be, for example, 10 seconds, 30 seconds, 1 minute, or other time thresholds. If the elapsed time exceeds the time threshold, the intraoral scanning application 115 may determine that the 3D surface / 3D model has likely been modified.

[0306] In some embodiments, the scanner 150 and / or computing device 150 includes a microphone. The microphone can receive audio between scans and evaluate the audio to determine whether a specific sound, such as the distinctive sound of a drill bit, is detected from the audio. In one embodiment, the audio features (e.g., an audio fingerprint) of the received audio are compared with stored audio features (e.g., audio fingerprints) associated with a drill bit or other tool used to modify a tooth location. This may include generating an audio fingerprint from the audio using an audio fingerprint recognition algorithm. If the audio features of the received audio match stored audio features associated with a modification of a tooth location (e.g., a dental drill), the intraoral scanning application 115 can determine that the 3D surface / 3D model may have been modified.

[0307] The intraoral scanner 150 may include one or more motion sensors (e.g., accelerometers and / or gyroscopes) that can be used to detect movement of the scanner 150. Data from the motion sensors can be evaluated to determine whether the scanner 150 has left the doctor's hand between scans (e.g., placed on a surface or in a basket). In one embodiment, a specific motion profile may indicate that the scanner has been removed from the patient's mouth and / or placed on a surface. Additionally, no movement within a threshold duration can indicate that the scanner 150 has been placed by the doctor (e.g., on a surface or in a basket). Therefore, the received motion data can be compared with one or more motion criteria to determine whether the scanner 150 has been placed between scans. Data from the motion sensors may additionally or alternatively be used to determine whether the scanner 150 remains stationary in the doctor's hand between scans (e.g., whether the doctor takes a short break). The received motion data may be compared with one or more additional motion criteria to determine whether the scanner 150 remains stationary in the doctor's hand. Such criteria may include, for example, a second threshold duration that is shorter than the threshold duration that can be used to determine that the scanner 150 has been placed by the doctor. Such criteria may also include a negation rule, wherein the rule of holding the scanner in the hand is not met if a movement profile indicating removal of the scanner from the patient's mouth and / or placement of the scanner on a surface is identified.

[0308] Assuming the scanning has stopped and a period of time has elapsed between scans (e.g., one or more of the above criteria have been met), the system can assume possible variations in the 3D surface / model. These variations may be related to, for example, the gingival line, margin line, and / or the shape of the preparation. For each prepared tooth, the system determines whether there are variations in the 3D surface associated with the prepared tooth. This determination can be made easier by finding the precise location of the prepared tooth represented in the new scan relative to the already generated 3D surface / model from the surrounding unchanged features (e.g., unchanged tissue). Based on previously performed classification and segmentation, the system knows which pixels in the 3D surface / model depict the preparation and which pixels depict the surrounding features, as well as which pixels in the new intraoral scan depict the preparation and which pixels depict the surrounding features.

[0309] This system can review new areas of prepared teeth and compare specific portions of the prepared tooth from a new scan with specific portions of the prepared tooth in a 3D surface / model. For example, removing dental sutures will alter the gingival shape of the area surrounding the prepared tooth. In one embodiment, the system accomplishes this through a 3D comparison between the 3D surface and a new intraoral scan. The intraoral scanning application 115 can then update the 3D surface or 3D model by replacing data associated with some portions of the prepared tooth or the area surrounding the prepared tooth with data from one or more new intraoral scans. Subsequently, the intraoral scanning application 115 can display the modified 3D model or the 3D surface of the dental arch. In some embodiments, the GUI of the intraoral scanning application 115 shows the current surface after the change and the previous surface when it was intended to be changed. For example, the differences between the changed surface and the original surface can be shown using dashed lines, perspective meshes, or other visualizations different from the current visualization of the changed surface.

[0310] In one embodiment, the system will adopt the shape of the tooth revealed by a new portion of the scan, below which the gingival line was previously located. However, the system may also maintain a representation of where the gingival line was previously located. This could be a better representation of the surfaces (note that it may not be a single surface, but multiple surfaces), and the system can examine whether using this would provide better clinical outcomes. The system can also detect changes between intraoral scans of the region (assuming the perspective between scans is nearly identical) (as opposed to changes between scans and 3D surfaces or 3D models). In some cases, the system can observe that dental filament is already between the gum and the tooth. In this case, the system can anticipate that one or more subsequent scans will be without dental filament. The system can also detect changes based on gingival bleeding and / or excess saliva.

[0311] In some embodiments, the intraoral scanning application 115 detects margin lines and / or dental retraction lines on the scanned and / or 3D surface / model. The intraoral scanning application 115 can determine that areas within the margin lines and / or dental retraction lines will be modified, and areas outside the margin lines and / or dental retraction lines will not be modified. Another change a dentist may make to a prepared tooth is to rework it, such as by further grinding the prepared tooth. Sometimes this is performed after occlusal gaps show that the distance between the prepared tooth and the opposing teeth on the opposing dental arch is too small. The prepared tooth may also be reworked after shape analysis of the prepared tooth indicates that the shape should be modified, after a problematic insertion path for an oral prosthesis on the prepared tooth is reported (e.g., if the insertion path is blocked), or when the system reports low-quality margin lines. For each of these cases, the dentist will typically rework the prepared tooth.

[0312] With each change, the system will need to understand which part of the tooth position has changed, remove that changed part from the previous 3D representation, and replace it with information from the new scan data. In some embodiments, this can be achieved with the help of the scanner 150 knowing its precise location, for example, by determining the movement of the scanner 150 between scans based on motion data and / or an evaluation of intraoral scans.

[0313] In one example, the intraoral scanning application 115 may include logic for automatically identifying (e.g., highlighting) edge lines in images and / or 3D models of prepared teeth. This can make it easier for a physician to check the accuracy of the edge lines. The intraoral scanning application 115 may additionally mark and / or highlight specific segments of unclear, uncertain, and / or undefined edge lines. Additionally or alternatively, the intraoral scanning application 115 may mark and / or highlight specific areas (e.g., surfaces) that are unclear, uncertain, and / or undefined. For example, segments of acceptable edge lines may be shown in a first color (e.g., green), while segments of unacceptable edge lines may be shown in a second color (e.g., red). In one embodiment, a trained machine learning model is used to identify edge lines in prepared teeth.

[0314] Intraoral scanning application 115 may additionally or alternatively include logic for automatically correcting tooth surfaces in images and / or 3D models of teeth and / or for modifying unacceptable edge lines of prepared teeth. This may be referred to as “virtual cleanup” or “sculpting” of the edge lines. In one embodiment, intraoral scanning application 115 includes logic for performing such virtual cleanup or sculpting, as set forth in U.S. Publication No. 2021 / 0059796 entitled “Automatic Detection, Generation, and / or Correction of Tooth Features in Digital Models,” which is incorporated herein by reference.

[0315] In one embodiment, a trained machine learning model is used to modify images and / or 3D models of the prepared teeth, such as to correct the edge lines of the prepared teeth (e.g., to sculpt or perform virtual cleaning of the edge lines). Updated edge lines (e.g., virtual cleaning or sculpted edge lines) can be indicated in the modified image and / or modified 3D model. A physician can examine the modified edge lines to determine their accuracy.

[0316] In one example, a portion of the actual edge line of a scanned, prepared tooth may not be clearly defined in the 3D model. For instance, during the initial 3D data collection step (e.g., via scanning) that leads to the generation of the first 3D virtual model, a portion of the physical tooth surface may have been covered by foreign matter (e.g., saliva, blood, or debris). This portion of the physical tooth surface may also be obscured by another element (e.g., a portion of the gum, cheek, tongue, dental instrument, artificial, etc.). Alternatively, for example, during the initial 3D data collection step (e.g., via scanning) that leads to the generation of the first virtual 3D model, the area may have been deformed or otherwise defective and may not properly correspond to the physical tooth surface (e.g., due to some defects in the actual scanning process). Automatic correction can be performed to remove the representation of foreign matter and reveal the underlying tooth surface and / or edge line. If automatic correction of the tooth surface and / or edge line is performed, the obscured area can be created, and the obscuring object can be removed from the 3D model.

[0317] When changes in tooth position are small (e.g., tens to hundreds of micrometers), it can be difficult to detect the change and determine how to update the 3D surface and / or 3D model. On the other hand, when changes in tooth position are large (e.g., on the order of millimeters), there is no obfuscation and replacement of the 3D surface and / or 3D model is easy. Such small changes can be when 3D variations are on the scale of tens to hundreds of micrometers. Such dimensional changes may sometimes be within the noise range and may differ from previous scans.

[0318] The averaging (as is common during scanning) will be mixed with the previously depicted changes in tooth locations. Therefore, in some embodiments, each scan will not be averaged with previous scans until a decision can be made regarding whether a change has been made to the tooth locations. Additionally, even after a decision is made, records of different scans can be retained so that the decision can be reversed if new information or user guidance is provided indicating that a change occurred without being detected, or that no change occurred without being detected.

[0319] Small differences at tooth locations between an earlier scan generated before tooth location modification and a later scan generated after tooth location modification can be at the error level of each surface point on the scan. However, differences are typically detected for regions comprising multiple points rather than at a single point. This regional difference in tooth locations can be represented by creating a difference map between the earlier scan (generated before modification) and the later scan (generated after modification). A low-pass filter can be applied to the difference map to determine whether the difference is a point difference or a regional difference. Point differences are often noise, while regional differences are highly probable to be actual differences in tooth locations.

[0320] Additionally, the intraoral scanning application 115 is capable of detecting specific types of common differences between the 3D model or 3D surface and an intraoral scan generated after changes to the tooth locations depicted in the 3D model or 3D surface. For example, differences in certain areas (such as when dental retraction cords are removed) will have specific locations and be around the teeth. The intraoral scanning application may include one or more rules for detecting signs of such common differences and / or may include one or more machine learning models that have been trained to receive data from two intraoral scans (or data from a set of two intraoral scans) and identify specific types of differences between the data from the two intraoral scans or the set of the two intraoral scans.

[0321] In one embodiment, if the intraoral scanning application 115 automatically decides whether the detected differences are related to modifications of tooth locations or to noise and / or errors, the GUI of the intraoral scanning application 115 will display these results (e.g., in a highlighted manner). This may include showing a first 3D surface resulting from the decision made by the intraoral scanning application 115, and alternatively showing changes from a previous 3D surface or 3D model. The intraoral scanning application 115 may also display a second 3D surface that will be generated by a different decision simultaneously with the first 3D surface. The intraoral scanning application 115 may output a request for automatic approval or rejection. This approval or rejection may take the form of voice approval or rejection, pressing one or more buttons on an input device of the scanner 150 and / or computing device 105 (e.g., a touchscreen, touchpad, mouse, etc.), a gesture detected based on motion data from the scanner 150, and / or some other type of input.

[0322] In an embodiment, the intraoral scanning application 115 can detect transitions between different patterns or stages of a restorative workflow. For example, the intraoral scanning application 115 can detect when dental floss (also referred to as dental filament or gingival retraction filament) is inserted between the tooth and gingiva around the prepared tooth and / or when dental floss is removed from between the tooth and gingiva. Additionally, the intraoral scanning application 115 can detect the grinding stages or rounds of the prepared tooth during the formation of the prepared tooth. When a stage of the restorative workflow / treatment is identified, and when a transition between a “pattern” or treatment stage is identified, the intraoral scanning application can output a notification to the physician that the intraoral scanning application is accurately tracking the restorative workflow or treatment. If the intraoral scanning application 115 has incorrectly identified a stage of the workflow or treatment, the physician can provide input indicating that the intraoral scanning application 115 is incorrect and / or indicating the correct stage or more. This input can be used to perform further training of the intraoral scanning application 115 (e.g., one or more machine learning models of the intraoral scanning application 115) to improve accuracy.

[0323] In this embodiment, each intraoral scan is recorded and processed individually, and these intraoral scans can be reprocessed after receiving one or more additional scans and / or after scanning of the dental arch or tooth points is completed. Reprocessing can be performed using intraoral scans and additional intraoral scans (and / or 3D surfaces or 3D models), which provides improved accuracy. Therefore, even if the system does not accurately identify the stage of the workflow or treatment in real time and / or does not detect the correct scan role in real time, the system is able to update and correct errors in the classification of earlier treatment stages or patterns and / or errors in the classification of subsequent scan patterns. This may eliminate any need for rescanning.

[0324] During rescanning, some scans may be discarded because the physician's intentions are not always known in advance, and some decisions made by the intraoral scanning application may be incorrect. In an embodiment, the intraoral scanning application 115 maintains (e.g., stores) all scan data from the scanning session. If the intraoral scanning application 115 makes an incorrect decision and receives correction from the physician, it can always perform a recalculation using the stored but previously unused data without requiring a rescan of the patient. Recalculation may include redetermining the 3D surface and / or 3D model using scans different from those previously used and / or using different weights than those previously used. Such recalculation may be performed to determine updates for any of the following: segmentation, role recognition, restorative workflow recognition, orthodontic workflow recognition, determination of changes in tooth position, and / or other predictions, classifications, or determinations discussed herein.

[0325] In some cases, a physician may provide ambiguous or unclear input that can be interpreted in multiple ways, each potentially leading to different results. Additionally, in some cases, the intraoral scanning application determines an equal (or nearly equal) probability that two different results are correct (e.g., a modification of tooth placement and no modification of tooth placement). In such cases, where the intraoral scanning application receives ambiguous or unclear input or cannot automatically determine the correct output, the intraoral scanning application 115 may suggest two or three options for the physician to make a decision. In an embodiment, a separate 3D surface or 3D model may be displayed for each option. The physician can then select the correct option. Once the correct option is selected, knowledge of the correct option can be used to perform retraining. This may include retraining one or more machine learning models.

[0326] Use of gravure surfaces in restorative cases

[0327] In some embodiments, the 3D model generated based on scans of tooth locations in a patient's mouth is inaccurate or of suboptimal quality because there are insufficient features at the tooth locations to perform scan registration and stitching and / or generate an accurate depiction of the tooth locations. In this case, additional scans of the intaglio surfaces of the dental prosthesis manufactured for the tooth locations and / or the impressions taken from the tooth locations can be generated. Subsequently, scan data from the intaglio surfaces of the impressions or dental prostheses can be used in conjunction with the scan data of the tooth locations to generate a more accurate 3D model of the tooth locations. Additionally, intraoral scans of prepared teeth may include unclear, occluded, or poorly defined margins. In this case, scan data from the intaglio surfaces of the impressions of prepared teeth used for dental prostheses can improve the quality, definition, and / or sharpness of the margins in the 3D model of the prepared teeth. Thus, scanning of the intaglio surfaces can improve, for example, the depiction of margins around the prepared teeth, the depiction of edentulous areas of the dental arch, etc.

[0328] In some cases, physicians do not need to provide any input to the intraoral scanning application 115 indicating that they are performing a scan of a concave surface or a scan of a concave surface for a specific patient. In some embodiments, when generating one or more scans of the concave surface of a dental prosthesis (e.g., a temporary dental prosthesis) or impression, those scans are automatically analyzed. Based on this analysis, the intraoral scanning application 115 can determine that the scan is of a concave surface rather than a tooth location in the oral cavity. For example, a scan of a tooth location has an overall mound-like shape with predominantly convex surfaces. On the other hand, a scan of a concave surface typically has a valley-like shape with predominantly concave surfaces. Based on this information, the intraoral scanning application can automatically determine whether the received scan is for an object in the oral cavity or for a concave surface of a dental prosthesis or impression. In one embodiment, a trained machine learning model outputs a classification of whether the scan is for an object in the oral cavity or a concave surface.

[0329] In some embodiments, the intraoral scanning application 115 generates a 3D surface and / or 3D model of the intaglio surface and compares the 3D surface and / or 3D model with stored 3D models of one or more patients. The intraoral scanning application 115 may additionally compare the intraoral scan with stored 3D models of one or more patients. For the intraoral scan and / or 3D surface of the intaglio surface, the intraoral scanning application may invert the data from the intraoral scan and / or 3D surface before comparing it with stored 3D models. Subsequently, comparisons can be made with various 3D models until a match is identified. For example, a match can be made between the intaglio surface and the surface of a specific prepared tooth on the dental arch of a specific 3D model of a specific patient. Once a match is identified, the intraoral scanning application 115 can automatically identify the patient and / or the specific prepared tooth associated with the intaglio scan data. Intraoral scanning application 115 can additionally and automatically identify 3D models and / or specific regions of 3D models (e.g., regions associated with identified prepared teeth) to combine with 3D surface and / or intaglio scan data to generate updated 3D models (e.g., with improved edge lines).

[0330] For example, a temporary crown can be fabricated for placement on a prepared tooth. The concave surface of the temporary crown can be scanned using scanner 150, and the scan of the concave surface of the temporary crown can be used in conjunction with a scan of the prepared tooth to determine the margin line around the prepared tooth. In an embodiment, intraoral scanning application 115 can automatically acquire concave scan data and combine it with scan data from the prepared tooth to determine where the margin line is not using dental floss. This may include portions of the concave scan data used to determine some areas for the margin line (e.g., areas obscured by the gum in the intraoral scan data from the prepared tooth), and portions of the intraoral scan data from the prepared tooth used to determine other areas for the margin line. In other cases, intraoral scanning application 115 may determine to use only data from the scan of the prepared tooth for the margin line, or only data from the scan of the concave surface of the temporary crown for the margin line. Intraoral scanning application 115 can automatically determine which data from the intraoral scan and which data from the scan of the concave surface are used to determine the margin line. For example, 90% of the margin line can be determined based on intraoral scan data of the prepared tooth (e.g., because these portions of the margin line are exposed in the intraoral scan data), and 10% of the margin line can be determined based on scan data of the concave surface of the temporary crown (e.g., because these portions of the margin line are obscured in the intraoral scan data).

[0331] Scanning of tooth locations becomes complicated by the areas where patients have missing teeth (referred to as edentulous regions). For example, in cases where two or more adjacent teeth are missing, there may be a large span of soft tissue that needs to be scanned. Scanning edentulous regions of the dental arch is particularly challenging because there may not be enough geometric reference points to perform scan registration and stitching. Additionally, soft gingival tissue may shift or deform between scans, reducing the accuracy of the resulting 3D model. Furthermore, for soft gingival tissue, it may be advantageous to capture the full range of possible locations and / or shapes of the soft gingival tissue, which is often not possible with intraoral scans of the edentulous region alone. Therefore, this system often cannot accurately capture the complete envelope of the edentulous region.

[0332] Therefore, in this embodiment, an impression of the edentulous region can be obtained (e.g., using an elastomeric impression material), wherein the impression captures the envelope of movement of the soft tissue in the edentulous region. The concave surface of this impression can then be scanned using scanner 150. Alternatively or additionally, the concave surface of a previously manufactured denture can be scanned using scanner 150. Scanning the concave surface 150 can capture the complete motion envelope of the soft tissue. An additional scan of the edentulous region can be generated. The scan of the edentulous region can be automatically combined with scans of the impression or the concave surface of the denture to generate a 3D model that can be used to manufacture a new denture for the patient. For example, the combined scans can be used to determine the concave surface of the new denture to be manufactured.

[0333] Automatic detection of dirty optical surfaces

[0334] The intraoral scanner 150 operates in a non-sterile environment. Saliva, blood, and other substances can accumulate on the optical surfaces of the scanner head, obstructing the light path into and out of the scanner 150. The optical surfaces can be, for example, windows or mirrors within the scanner head. Most intraoral scanners 150 have an exit window above which the scanner is constructed, and below which the system intends to scan teeth and other dental objects. Additionally, most intraoral scanners 150 include a folding mirror within the scanner head that reflects light so that it exits the scanner at an angle (e.g., a right angle, an acute angle, or an obtuse angle) to the longitudinal axis of the scanner 150. For scanners that include an exit window, the exit window may become contaminated. For scanners without an exit window, the folding mirror may become contaminated. Obstruction or contamination of the optical surfaces (e.g., exit windows and / or mirrors) can negatively impact the accuracy of intraoral scan data, such as intraoral scans, intraoral color images, and NIRI images generated by the scanner 150. For scanners with closed tips (e.g., ...), Scanner or Scanners (e.g., those with open tips), dust and grime accumulate on the exit window of the scanner head and / or on the exit window of the protective sleeve covering at least a portion of the scanner head. Scanner, Scanner or The scanner may have dust and dirt accumulating on the folding mirror and / or lens in the scanner head and / or on the folding mirror within the sleeve or the scanner head attachment. Interference or obstruction on the optical surface may include, but is not limited to, dust, blood, or dirt on the exit window, the folding mirror, the glass, the lens, or any other object or surface (referred to as the optical surface) in the optical path of the scanner 150.

[0335] In embodiments, the intraoral scanning application 115 and / or the intraoral scanner 150 automatically detect obstructions (e.g., dust, dirt, blood, saliva, etc.) on the optical surface of the scanner 150. Obstructions can be detected using image processing and / or machine learning applications. In some embodiments, dirty optical surfaces are detected by generating and / or analyzing depth maps / height maps. In other embodiments, dirty optical surfaces are identified without using depth maps or determining depth.

[0336] Intraoral scanning application 115 can determine the obstruction level of the optical surface. If the obstruction level exceeds an obstruction threshold (e.g., a dirtiness threshold), intraoral scanning application 115 can generate a warning message to clean the optical surface and / or replace the protective sleeve or accessory on scanner 150. In an embodiment, a threshold amount of obstruction that meets the criteria for "dirty" on the optical surface can be set. A default threshold can be set automatically, and the dentist can adjust the threshold via user input. When the system detects that the optical surface (e.g., sleeve, lens, window, and / or mirror) has reached the threshold, a message can be generated and / or the scanning can be stopped. In one embodiment, the system can decide to output a pop-up warning to the dentist with "Please replace the sleeve." If the dentist ignores the notification, the system can automatically pause the scanning and / or prevent the virtual 3D model from being sent to the laboratory (e.g., in the case of an extremely dirty sleeve / mirror / window). This also prevents the dentist from reusing the sleeve, which reduces the risk of cross-infection between patients.

[0337] In one embodiment, depth information is used to detect dirty optical surfaces (i.e., obstructions on the optical surface). This may include generating a depth map or height map and subsequently comparing the height / depth in the depth / height map to a depth threshold. The depth of the optical surface (e.g., the distance between the optical surface and the focusing optics) may be known, and the depth threshold (also referred to as a distance threshold) may be set based on the known depth of the optical surface. Intraoral scanning application 115 may determine which depth from the depth map, if any, is equal to or less than the depth threshold. Each point or pixel associated with a depth value less than or equal to the depth threshold may be identified as occluded. The depth detection capability is not limited to the exit window and extends into the scanner. In one embodiment, each depth detected inside scanner 150 (e.g., at a certain depth of a mirror, lens, etc.) is considered an interference or obstruction on the optical surface because the optical path inside scanner 150 should be clear.

[0338] In one example, the intraoral scanner 150 includes known distances to the exit window of the sleeve (or the exit window of the internal mirror or probe). These values ​​can be determined by calibration or by design and can be stored by the intraoral scanning application 115 and / or the scanner 150. The intraoral scanning application 115 can receive a depth map with one or more distance candidates. For each candidate in the depth map, the intraoral scanner 115 can use a threshold to check if it is close to the exit window (or mirror). Different thresholds can be used to determine whether a candidate is below or above the exit window. The intraoral scanning application 115 can count the number of candidates close to the exit window. If the number of candidates is greater than the threshold, the system can output a notification that the sleeve may be dirty. Alternatively or additionally, the processing logic can output a notification indicating the percentage of the optical surface that is dirty. This notification can be updated when the optical surface becomes dirtier and / or is cleaned. Additionally or alternatively, the intraoral scanning application 115 can determine the size of the dirty area (the number of adjacent points that meet the criteria for dirt) and determine whether the size of the dirty area exceeds a size threshold. If so, the intraoral scanner 115 can determine that the scanner 150 has a dirty optical surface. Additionally or alternatively, the intraoral scanning application 115 can determine whether a threshold number of scans (e.g., continuous scans) have dirty areas exceeding a threshold in size. If so, the intraoral scanner 115 can determine that the scanner 150 has a dirty optical surface.

[0339] In one embodiment, the intraoral scanning application 115 compares two or more intraoral scans to determine unchanged pixels / points between the two or more intraoral scans. If most or some of the points / pixels differ between the scans, but some remain unchanged, the intraoral scanning application 115 can determine that those unchanged points / pixels are occluded by a dirty optical surface. In one embodiment, the intraoral scanning application generates a color image as an average of multiple color images. This average can then be analyzed (e.g., using a trained ML model) to identify moving and non-moving objects. Moving objects may appear as smeared objects in the combined image and can be associated with clean areas of the optical surface, while non-moving objects may appear as sharp or clear objects in the combined image and can be associated with dirty areas of the optical surface.

[0340] In this embodiment, the intraoral scanning application 150 can distinguish between dirty lenses, dirty mirrors, a dirty exit window of the scanner 150, and a dirty sleeve on the scanner 150. In this embodiment, the intraoral scanning application 150 can output an indication of which(s) of optical components are detected as dirty. If a protective sleeve (e.g., the window of the protective sleeve) or a protective accessory (e.g., the mirror of the protective accessory) is detected as dirty, the user can correct the problem by replacing the dirty sleeve or accessory with a clean one. If the exit window of the scanner 150 is detected as dirty, the dentist may need to clean the exit window before continuing the scanning procedure. Typically, nothing prevents dentists from using dirty sleeves, dirty accessories, dirty lenses, etc. In this embodiment, the system automatically detects dirty sleeves or accessories and prevents further scanning until the dirty sleeve or accessory is replaced or cleaned.

[0341] Dirty sleeves and other dirty optical surfaces result in significant time wastage during the modeling phase of dental appliance creation. For example, computer-aided drafting (CAD) designers spend an average of 8 minutes processing a virtual 3D model generated using a clean intraoral scanner, compared to approximately 12 minutes processing a virtual 3D model generated using a dirty scanner. Therefore, applying a dirty sleeve / dirty scanner detection method can reduce CAD designer workload. Additionally, it can reduce the number of rejected cases and clinical escalations.

[0342] In some embodiments, one or more optical surfaces of the intraoral scanner 150 may fog up when the scanner head of the intraoral scanner 150 is inserted into the oral cavity. This may occur, for example, if a protective sleeve is replaced and / or a cold sleeve is inserted into the patient's oral cavity before the sleeve has a chance to be heated to near the patient's body temperature. In some cases, fogging can be interpreted as a dirty optical surface. In some cases, color images generated by the intraoral scanner 150 can be used to detect the color, opacity, and / or transparency of areas that have been identified as dirty. For example, multiple color images and / or intraoral scans may be input into a trained ML model that has been trained to identify dirty optical surfaces and fogged optical surfaces. The ML model may output an indication of whether an optical surface is dirty or fogged. In one embodiment, the intraoral scanner 150 determines the temperature of one or more areas of the scanner 150 and outputs a notification indicating that the detected dirty surface may be due to fogging, and waits to determine whether the occlusion of the optical surface is automatically cleared (e.g., waits a few seconds).

[0343] For any automated decisions made by the intraoral scanning application 115, such as automated scan role determination, automated prescription generation, automated selection of multiple parts of the intraoral scan for 3D surfaces, automated classification of dental objects, etc., the physician can override the automated decision. In each example of an automated decision made by the intraoral scanning application 115, the intraoral scanning application 115 can provide instructions for the automated decision made and options for the physician to change the automated decision to a different decision. When such a manual override occurs, the original decision, the details leading to the original decision, and the physician's manual decision are recorded. This data can then be used to retrain one or more components of the intraoral scanning application 115 (e.g., one or more trained ML models) to improve the accuracy of the system.

[0344] Figure 2A A model training workflow 205 and a model application workflow 217 for an intraoral scanning application according to one embodiment of the present disclosure are illustrated. In the embodiment, the model training workflow 205 may be executed on a server including or excluding the intraoral scanning application, and the trained model is provided to the intraoral scanning application (e.g., in...). Figure 1 On a computing device 105, a model application workflow 217 can be executed. The model training workflow 205 and the model application workflow 217 can be executed by processing logic executed by the processor of the computing device. One or more of these workflows 205, 217 can be, for example, executed by one or more machine learning modules implemented in the intraoral scanning application 115 or... Figure 42 This is achieved through other software and / or firmware executed on the processing unit of the computing device 4200 shown in the figure.

[0345] Model training workflow 205 is used to train one or more machine learning models (e.g., deep learning models) to perform one or more classification, segmentation, detection, and recognition tasks on intraoral scan data (e.g., 3D scans, height maps, 2D color images, NIRI images, etc.) and / or 3D surfaces generated based on intraoral scan data. Model application workflow 217 is to apply the one or more trained machine learning models to perform classification, segmentation, detection, and recognition tasks on intraoral scan data (e.g., 3D scans, height maps, 2D color images, NIRI images, etc.) and / or 3D surfaces generated based on intraoral scan data. One or more of the machine learning models can receive and process 3D data (e.g., 3D point clouds, 3D surfaces, multiple parts of a 3D model, etc.). One or more of the machine learning models can receive and process 2D data (e.g., 2D images, height maps, projections of 3D surfaces onto a plane, etc.).

[0346] This paper describes many different machine learning outputs. Specific numbers and arrangements of machine learning models are described and illustrated. However, it should be understood that the number and type of machine learning models used, as well as the arrangement of such models, can be modified to achieve the same or similar final results. Therefore, the machine learning model arrangements described and illustrated are merely examples and should not be construed as limiting.

[0347] In an embodiment, one or more machine learning models are trained to perform one or more of the following tasks. Each task can be performed by a separate machine learning model. Alternatively, a single machine learning model can perform each or a subset of the tasks. Additionally or alternatively, different machine learning models can be trained to perform different combinations of tasks. In one example, one or more machine learning models can be trained, wherein the trained ML model is a single shared neural network with multiple shared layers and multiple higher-level, different output layers, wherein each of the output layers outputs a different prediction, classification, label, etc. The one or more trained machine learning models can be trained to perform the following tasks:

[0348] I) Scan Role Classification—This may include classifying intraoral scans, collections of intraoral scans, 3D surfaces generated from multiple intraoral scans, 3D models generated from multiple intraoral scans, etc., as associated with maxillary, mandibular, or occlusal roles. This may also include classifying intraoral scans as associated with preparation roles.

[0349] II) Classification of scan views—This may include classifying intraoral scans or sets of intraoral scans into views depicting the lingual, buccal, or occlusal sides of the jaw. Other views may also be identified, such as the right side of the jaw, the left side of the jaw, etc.

[0350] III) Dental Object Segmentation—This may include performing point-level classification (e.g., pixel-level or voxel-level classification) on different types of dental objects from intraoral scans, sets of intraoral scans, 3D surfaces generated from multiple intraoral scans, 3D models generated from multiple intraoral scans, etc. Different types of dental objects may include, for example, teeth, gingival margins, palate, prepared teeth, restorative objects other than prepared teeth, implants, brackets, dental attachments, soft tissue, retraction cord (dental floss), blood, saliva, etc. In some embodiments, different types of restorative objects, different types of implants, different types of brackets, different types of attachments, and different types of soft tissue (e.g., tongue, lips, cheeks, etc.) may be identified.

[0351] IV) Scan Success Determination and / or Scan Quality Ranking—This can include assigning quality values ​​to individual scans, 3D surfaces, 3D models, etc. Quality values ​​above a threshold can be determined as scan success. This can also include assigning quality values ​​to multiple parts or regions of a 3D surface or 3D model. Multiple parts or regions with quality values ​​below a threshold can be flagged for rescanning. V) Prescription Generation—This can include predicting prescription parameters based on intraoral scans, a collection of intraoral scans, 3D surfaces generated from multiple intraoral scans, 3D models generated from multiple intraoral scans, etc. Examples of prescription parameters that can be predicted include whether the prescription is for orthodontic or restorative treatment, one or more teeth to be treated, the type of dental restoration to be used, the color to be used for the dental restoration, the material to be used for the dental restoration, the laboratory to be used, etc. Each of the different types of predictions / classifications associated with prescription generation can be determined by a separate ML model or by an ML model trained to generate multiple different outputs. For example, a separate ML model can be trained to determine the dental laboratory, the type of dental prosthesis, the material of the dental prosthesis, the color of the dental prosthesis, etc.

[0352] VI) Case type classification—This may include determining whether orthodontic treatment and / or restorative treatment will be performed on a patient based on intraoral scans, a collection of intraoral scans, 3D surfaces generated from multiple intraoral scans, 3D models generated from multiple intraoral scans, etc.

[0353] VII) Detection of changes in tooth surface – This can include determining whether the dentist made any changes to one or more tooth locations between intraoral scans, such as by grinding teeth, adding dental floss, removing dental floss, etc., and whether changes have occurred between scans, such as blood accumulation, blood removal, saliva accumulation, saliva removal, etc. Such determination can be based on input from one or more first scans or a 3D surface / 3D model generated from said one or more first scans and one or more second scans, or from a 3D surface generated from said one or more second scans. A machine learning model can identify areas of change and any changes made, and can determine which parts of the earlier 3D surface / 3D model should be replaced using data from said one or more second scans.

[0354] VIII) Dirty Optical Surface Detection—This may include classifying an intraoral scanner or protective sleeve / accessory as dirty based on one or more intraoral scans. Additionally, this may include pixel-level classification of scanned areas as dirty, and / or may include determining which parts of the scanner are dirty (e.g., the window of the protective sleeve, the window of the scanner head, lenses, folding mirrors, etc.). IX) Scan Completion Recognition—This may include determining when a scan of the maxilla, mandible, and / or occlusion is complete based on intraoral scans, a collection of intraoral scans, and / or 3D surfaces generated from multiple intraoral scans. This may also include determining when the entire scan is complete.

[0355] Once the scanning of a segment is complete, the processing logic can automatically generate a 3D model of that segment (e.g., dental arch). Once the scanning of all segments is complete (e.g., upper dental arch, lower dental arch, and occlusion), the processing logic can automatically perform post-processing, perform occlusal contact analysis, perform diagnostics, etc. X) Detection of insertion / withdrawal from the oral cavity—this may include determining whether the scanner is in the oral cavity, whether the scanner is being inserted into the oral cavity, and / or whether the scanner is being withdrawn from the oral cavity based on one or more 2D images.

[0356] XI) Edge line recognition / labeling—This can include performing pixel-level recognition / classification of prepared edge lines around teeth based on intraoral scans, a collection of intraoral scans, 3D surfaces generated from multiple intraoral scans, 3D models generated from multiple intraoral scans, etc. This may also include labeling and identifying…

[0357] Edge lines. Edge line identification and marking are described in U.S. Patent No. 2021 / 0059796. XII) Tooth numbering classification—This may include performing pixel-level identification / classification and / or group / patch-level identification / classification of each tooth from 3D surface data. Teeth may be classified using one or more standard tooth numbering schemes, such as the American Dental Association (ADA) tooth numbering system.

[0358] XIII) Mobile Tissue (Redundant Tissue) Identification / Removal — This may include performing pixel-level identification / classification of mobile tissues (e.g., tongue, fingers, lips, etc.) from intraoral scans, and optionally removing such mobile tissues from the intraoral scans. Mobile tissue identification and removal are described in U.S. Publication No. 2020 / 0349698 entitled “Redundant Material Removal Using Machine Learning,” which is incorporated herein by reference.

[0359] XIV) Insertion path prediction—This can include predicting the insertion path of dental prostheses based on 3D surfaces, 3D models, etc.

[0360] XV) Multiple occlusion detection — This can include identifying the presence or absence of multiple occlusions based on multiple intraoral scans (e.g., each depicting a slightly different occlusion) and / or 3D surfaces generated from intraoral scans.

[0361] XVI) Gravure Surface Inspection / Use—This may include classifying intraoral scans, collections of intraoral scans, 3D surfaces generated from multiple intraoral scans, 3D models generated from multiple intraoral scans, etc., as gravure surfaces that depict or do not depict impressions or dental restorations. This may also include determining the match between gravure surfaces and preparations associated with gravure surfaces.

[0362] XVII) Doctor speech recognition — This may include identifying speech as belonging to a specific doctor (e.g., one of a group of possible doctors) based on the doctor's audio.

[0363] XVIII) Doctor facial recognition — This can include identifying a face as belonging to a specific doctor (e.g., one of a group of possible doctors) based on a doctor's facial image.

[0364] XIX) Motion pattern recognition — This can include identifying the scanner user as a specific doctor (e.g., one of a group of possible doctors) based on motion data generated by the scanner.

[0365] XX) 3D model observation trajectory generation — This may include determining the observation trajectory of a 3D model from a 3D model of the dental arch (or one or more projections of the 3D model).

[0366] XXI) Tooth-to-gingival boundary recognition / labeling — This can include performing pixel-level recognition / classification of tooth-to-gingival boundaries around one or more teeth based on intraoral scans, a collection of intraoral scans, 3D surfaces generated from multiple intraoral scans, 3D models generated from multiple intraoral scans, etc.

[0367] XXII) Tooth-to-tooth (interdental region) boundary recognition / labeling — This may include pixel-level recognition / classification of tooth-to-tooth boundaries for one or more interdental regions between teeth, based on intraoral scans, a collection of intraoral scans, 3D surfaces generated from multiple intraoral scans, 3D models generated from multiple intraoral scans, etc.

[0368] Note that for any of the tasks described above related to intraoral scans / 3D surfaces / 3D models, although they are described as being performed based on inputs of intraoral scans, 3D surfaces, and / or 3D models, it should be understood that these tasks can also be performed based on 2D images such as color images, NIRI images, etc. Any of these tasks can be performed using an ML model with multiple input layers or channels, where the first layer may include an intraoral scan / 3D surface (or a projection of a 3D surface) / 3D model (or a projection of a 3D model), the second layer may include a 2D color image, the third layer may include a 2D NIRI image, and so on. In another example, the first layer or channel may include a first 3D scan, the second layer or channel may include a second 3D scan, and so on.

[0369] One type of machine learning model that can be used to perform some or all of the above requirements is an artificial neural network, such as a deep neural network. Artificial neural networks typically include feature representation components with classifier or regression layers that map features to a desired output space. For example, a convolutional neural network (CNN) hosts multiple layers of convolutional filters. Pooling is performed, and nonlinearities can be addressed at lower layers, typically with additional layers of perceptrons on top of these lower layers, thus mapping the top-level features extracted by the convolutional layers to a decision (e.g., a classification output). Deep learning is a class of machine learning algorithms that use cascaded, multi-layered nonlinear processing units for feature extraction and transformation. Each subsequent layer uses the output from the previous layer as input. Deep neural networks can learn in a supervised (e.g., classification) and / or unsupervised (e.g., pattern analysis) manner. Deep neural networks consist of a hierarchical structure of layers, where different layers learn different levels of representation corresponding to different levels of abstraction. In deep learning, each level learns to transform its input data into a slightly more abstract and comprehensive representation. For example, in image recognition applications, the raw input can be a pixel matrix; the first representation layer can extract pixels and encode edges; the second layer can assemble the edge arrangement and encode it; the third layer can encode higher-level shapes (e.g., teeth, lips, gums, etc.); and the fourth layer can identify the scanned character. It's worth noting that the deep learning process can learn on its own which features will be optimally placed at which level. The "depth" in "deep learning" refers to the number of layers through which the data is transformed. More precisely, deep learning systems have a large credit allocation path (CAP) depth. CAP is a transformation chain from input to output. CAP describes the underlying causal relationship between the input and output. For feedforward neural networks, the CAP depth can be the network depth and can be the number of hidden layers plus one. For recurrent neural networks where the signal may propagate through layers more than once, the CAP depth may be unlimited.

[0370] In one embodiment, the U-net architecture is used for one or more machine learning models. U-net is a type of deep neural network that combines an encoder and a decoder, with appropriate connections between them to capture both local and global features. The encoder is a series of convolutional layers that increase the number of channels while decreasing the height and width as they process from input to output, while the decoder increases the height and width and decreases the number of channels. Layers from the encoder with the same image height and width can be connected to the output from the decoder. Any or all convolutional layers from the encoder and decoder can use conventional or depthwise separable convolutions.

[0371] In one embodiment, one or more machine learning models are recurrent neural networks (RNNs). An RNN is a type of neural network that includes memory that enables the neural network to capture time dependencies. An RNN is able to learn an input-output mapping that depends on both the current input and past inputs. The RNN will process past and future scans and make predictions based on this continuous scan information. An RNN can be trained using a training dataset to generate a fixed number of outputs (e.g., to classify time-varying data such as video data into a fixed number of categories). One type of RNN that can be used is a long short-term memory (LSTM) neural network.

[0372] A common architecture for this task is LSTM (Long Short-Term Memory). Unfortunately, LSTM is not well-suited for images because it cannot capture spatial information like convolutional networks. To address this, a variant of LSTM, ConvLSTM, can be used, which incorporates convolutional operations within LSTM units. ConvLSTM is a variant of LSTM that includes convolutional operations within LSTM units. ConvLSTM replaces matrix multiplication with convolutional operations at each gate in the LSTM unit. In this way, it captures underlying spatial features through convolutional operations on multidimensional data. The main difference between ConvLSTM and LSTM lies in the number of input dimensions. Since LSTM input data is one-dimensional, it is not suitable for spatial sequence data such as video, satellite, and radar image datasets. ConvLSTM is designed to take 3D data as its input. In one embodiment, a CNN-LSTM machine learning model is used. CNN-LSTM is an integration of CNN (convolutional layers) and LSTM. First, the CNN part of the model processes the data, and the one-dimensional result is fed into the LSTM model.

[0373] In one embodiment, a class of machine learning models called MobileNet is used. MobileNet is an efficient machine learning model based on a streamlined architecture that uses depthwise separable convolutions to build lightweight deep neural networks. MobileNet can be a convolutional neural network (CNN) that can perform convolutions in both the spatial and channel domains. MobileNet can comprise a stack of separable convolutional modules, which consist of depthwise convolutions and pointwise convolutions (conv 1x1). Separable convolutions perform convolutions independently in the spatial and channel domains. This decomposition of convolutions can significantly reduce computational costs from HWNK. 2 M decreased to HWNK 2 (Depth) plus HWNM(conv 1x1) (total HWN)

[0374] (K 2 +M)), where N represents the number of input channels, K 2Here, M represents the size of the convolution kernel, H represents the number of output channels, and HxW represents the spatial size of the output feature map. This can reduce the bottleneck of computational cost for 1x1 convolutions.

[0375] In one embodiment, a Generative Adversarial Network (GAN) is used for one or more machine learning models. A GAN is a class of artificial intelligence systems that uses two artificial neural networks competing against each other in a zero-sum game framework. A GAN includes a first artificial neural network that generates candidate images and a second artificial neural network that evaluates the generated candidate images. The GAN learns to map from a latent space to a specific data distribution of interest (a data distribution that ranges from photographs to input images that are indistinguishable to the human eye), while the discriminator network distinguishes instances from the training dataset from candidates generated by the generator. The training objective of the generative network is to improve the error rate of the discriminator network (e.g., by generating new synthetic instances that appear to come from the training dataset to deceive the discriminator network). The generative and discriminator networks are trained together; the generative network learns to generate images that are increasingly difficult for the discriminator network to distinguish from real images (from the training dataset), while the discriminator network simultaneously learns to better distinguish synthetic images from images from the training dataset. The two networks of the GAN are trained once they reach a balance. A GAN may include a generator network that generates images of artificial mouths and a discriminator network that segments images of artificial mouths. In an embodiment, the discriminator network may be MobileNet.

[0376] In one embodiment, one or more machine learning models are conditional generative adversarial (cGAN) networks, such as pix2pix. These networks not only learn a mapping from an input image to an output image, but also learn a loss function to train that mapping. A GAN is a generative model that learns a mapping from a random noise vector z to an output image y, G: z→y. Conversely, a conditional GAN ​​learns a mapping from an observed image x and a random noise vector z to y: G: {x, z}→y. The generator G is trained to produce an output that cannot be distinguished from a “real” image by an adversarially trained discriminator D, which is trained to detect the generator’s “impersonation” as well as possible. In embodiments, the generator may include a U-net or encoder-decoder architecture. In embodiments, the discriminator may include a MobileNet architecture. An example of a cGAN machine learning architecture that can be used is the pix2pix architecture described in “Image-to-Image Translation Using Conditional Adversarial Networks” ArXiv Preprint (2017) by Isola, Phillip, et al.

[0377] Training a neural network can be achieved through supervised learning, which involves feeding the network a training dataset consisting of labeled inputs, observing its output, constraining the error (by measuring the difference between the output and the labeled value), and using techniques such as deep gradient descent and backpropagation to adjust the network's weights across all its layers and nodes to minimize the error. In many applications, this process is repeated among many labeled inputs in the training dataset to produce a network that can produce the correct output when provided with inputs different from those present in the training dataset. This induction is achieved in high-dimensional settings such as large images, when sufficiently large and diverse training datasets are available.

[0378] For model training workflow 205, a training dataset containing hundreds, thousands, tens of thousands, hundreds of thousands, or more intraoral scans, images, and / or 3D models should be used to form the training dataset. In an embodiment, up to millions of patient dentition cases that have undergone restorative and / or orthodontic procedures can be used to form the training dataset, where each case can include various labels for one or more types of useful information. Each case can include, for example, data showing one or more tooth locations, such as 3D models, intraoral scans, height maps, color images, NIRI images, etc.; data showing pixel-level segmentation of the data (e.g., 3D models, intraoral scans, height maps, color images, NIRI images, etc.) into various dental categories (e.g., teeth, restorative objects, gingival margins, moving tissues, maxilla, etc.); data showing one or more specified categories of the data (e.g., scan role, in mouth, not in mouth, lingual view, buccal view, occlusal view, anterior view, left view, right view, etc.); etc. This data can be processed to generate one or more training datasets 236 for training one or more machine learning models. Machine learning models can be trained, for example, to automate one or more processes that traditionally require physician input during intraoral scanning, such as inputting scanning roles, inputting commands to start or stop scanning, identifying which areas of the 3D surface or 3D model need to be updated after modifying prepared teeth, and generating orthodontic or restorative prescriptions. Such trained machine learning models can be added to intraoral scanning applications and can be applied to significantly reduce the level of user input associated with intraoral scanning and / or simplify the scanning process.

[0379] In one embodiment, generating one or more training datasets 236 includes collecting one or more intraoral scans with labels 210 and / or one or more 3D models with labels 212. The labels used may depend on what a particular machine learning model will be trained to do. For example, to train a machine learning model to perform the classification of tooth locations (e.g., tooth location classifier 268), training dataset 236 may include pixel-level labels for various types of tooth locations. Training datasets may also be generated that include physician voice data, physician facial images, and / or other information.

[0380] The processing logic may collect a training dataset 236, which includes 2D or 3D images, intraoral scans, 3D surfaces, 3D models, height maps, etc., of tooth locations (e.g., dental arches) with one or more associated labels (e.g., pixel-level labeled dental classifications in the form of mappings (e.g., probabilistic maps), image-level labels of scan roles, etc.). In embodiments, the size of one or more images, scans, surfaces, and / or models, and optionally associated probabilistic maps in the training dataset 236 may be adjusted. For example, a machine learning model may be used for images with certain pixel size ranges, and their size may be adjusted if one or more images fall outside those pixel size ranges. For example, methods such as nearest neighbor interpolation or box sampling may be used to adjust the image size. The training dataset may be additionally or alternatively augmented. Training large-scale neural networks typically uses tens of thousands of images that are not readily available in many real-world applications. Data augmentation can be used to artificially increase the effective sample size. Common techniques include randomly rotating, shifting, cropping, flipping, etc., existing images to increase the sample size.

[0381] To perform training, the processing logic inputs (multiple) training datasets 236 into one or more untrained machine learning models. The machine learning models can be initialized before the first input is fed into them. The processing logic trains the untrained machine learning models(s) based on the training datasets(s) to generate one or more trained machine learning models(s) that perform the various operations described above.

[0382] Training can be performed by simultaneously inputting one or more images, scans, or 3D surfaces (or data from images, scans, or 3D surfaces) into a machine learning model. Each input may include data from an image, an intraoral scan, or a 3D surface from a training data item in the training dataset. For example, a training data item may include a height map and an associated probability map, which can be input into the machine learning model. As mentioned above, training data items may also include color images, images generated under specific lighting conditions (e.g., UV or IR radiation), etc. Additionally, the pixels of an image may include height values ​​or may include both height and intensity values. The data input into the machine learning model may include a single layer (e.g., height values ​​from only a single image) or multiple layers. If multiple layers are used, one layer may include height values ​​from the image / scan / surface, and a second layer may include intensity values ​​from the image / scan / surface. Additionally or alternatively, additional layers may include three layers for color values ​​(e.g., separate layers for each color channel, such as R, G, and B layers), a layer for pixel information from an image generated under specific lighting conditions, etc. In some embodiments, data from multiple images / scans / surfaces are fed together into a machine learning model, where the multiple images / scans / surfaces may all be the same tooth location. For example, a first layer may include height values ​​from a first scan of the tooth location, a second layer may include height values ​​from a second scan of the tooth location, a third layer may include height values ​​from scans of the tooth location, and so on. In some embodiments, an RNN is used. In this embodiment, the second layer may include the previous output of the machine learning model (which is generated by processing the previous input).

[0383] Machine learning models process inputs to generate outputs. Artificial neural networks consist of an input layer composed of values ​​from data points (e.g., intensity and / or height values ​​of pixels in a height map). The next layer is called a hidden layer, and nodes in the hidden layer each receive one or more input values. Each node contains parameters (e.g., weights) applied to the input values. Thus, each node essentially feeds the input values ​​into a multivariate function (e.g., a nonlinear mathematical transformation) to produce an output value. The next layer can be another hidden layer or an output layer. In either case, nodes in the next layer receive output values ​​from nodes in the previous layer, and each node applies weights to those values ​​and then generates its own output value. This can be performed at each layer. The final layer is the output layer, where there is a node for each class, prediction, and / or output that the machine learning model can produce. For example, for an artificial neural network being trained to perform tooth location classification, there might be a first classification (redundant material), a second classification (teeth), a third classification (gingiva), a fourth classification (restorative objects), and / or one or more additional dental classifications. Furthermore, this classification can be determined for each pixel in the image. Furthermore, classification and prediction can be determined for each pixel in the image / scan / surface, for the entire image / scan / surface, or for each pixel region or group of pixels in the image / scan / surface. For pixel-level segmentation, for each pixel in the image / scan / surface, the final layer applies the probability that the pixel belongs to the first category, the probability that the pixel belongs to the second category, the probability that the pixel belongs to the third category, and / or one or more additional probabilities that the pixel belongs to other categories.

[0384] Therefore, the output may include one or more predictions and / or one or more probability maps. For example, for each pixel in the input image / scan / surface, the output probability map may include a first probability that the pixel belongs to a first dental category, a second probability that the pixel belongs to a second dental category, and so on. For example, the probability map may include the probability of a pixel belonging to a dental category representing a tooth, gingival margin, or restorative object. In other embodiments, different dental categories may represent different types of restorative objects.

[0385] The processing logic can then compare the generated probability map and / or other outputs with known probability maps and / or labels included in the training data items. The processing logic determines the error (i.e., classification error) based on the difference between the output probability map and / or(multiple) labels and the provided probability map and / or(multiple) labels. The processing logic adjusts the weights of one or more nodes in the machine learning model based on this error. An error term or increment can be determined for each node in the artificial neural network. Based on this error, the artificial neural network adjusts one or more parameters (weights for one or more inputs to the node) for one or more of its nodes. Parameters can be updated in a backpropagation manner, such that the nodes in the highest layer are updated first, followed by the nodes in the next layer, and so on. An artificial neural network contains multiple layers of "neurons," where each layer receives values ​​from neurons in previous layers as input. The parameters of each neuron include weights associated with the values ​​received from each neuron in previous layers. Therefore, adjusting the parameters can include adjusting the weights assigned to each of the inputs of one or more neurons in one or more layers of the artificial neural network.

[0386] Once the model parameters have been optimized, model validation can be performed to determine if the model has improved and to determine the current accuracy of the deep learning model. After one or more rounds of training, the processing logic can determine whether a stopping criterion has been met. The stopping criterion can be a target accuracy level, the target number of processed images from the training dataset, the target amount of change in parameters on one or more previous data points, a combination thereof, and / or other criteria. In one embodiment, the stopping criterion is met when at least a minimum number of data points have been processed and at least a threshold accuracy has been reached. The threshold accuracy can be, for example, 70%, 80%, or 90% accuracy. In one embodiment, the stopping criterion is met if the accuracy of the machine learning model has stopped improving. If the stopping criterion has not yet been met, further training is performed. If the stopping criterion has been met, training can be completed. Once the machine learning model has been trained, a reserved portion of the training dataset can be used to test the model.

[0387] For example, in one embodiment, a machine learning model (e.g., a tooth location classifier 268) is trained to segment intraoral images by classifying regions of those intraoral images into one or more dental classifications. Similar processes can be performed to train machine learning models to perform other tasks such as those described above. For example, a set of many (e.g., thousands to millions) intraoral scans of 3D models and / or dental arches with labeled dental classifications can be collected. In one example, each point in the 3D model may include a label having a first value representing a natural tooth, a second value representing a restorative object, and a third value representing a gingiva / gingival margin. For example, one of the three values ​​may be 1, and the other two values ​​may be 0.

[0388] The tooth location classifier 268 may include one or more machine learning models that operate on 3D data, or it may include one or more machine learning models that operate on 2D data. If the tooth location classifier 268 includes machine learning models that operate on 2D data, a set of images (e.g., height maps) may be generated for each 3D model with labeled dental classifications. Each image may be generated by projecting the 3D model (or a portion of the 3D model) onto a 2D surface or plane. In some embodiments, different images of the 3D model may be generated by projecting the 3D model onto different 2D surfaces or planes. For example, a first image of the 3D model may be generated by projecting the 3D model onto a 2D surface at a top-down viewpoint, a second image may be generated by projecting the 3D model onto a 2D surface at a first lateral viewpoint (e.g., buccal viewpoint), a third image may be generated by projecting the 3D model onto a 2D surface at a second lateral viewpoint (e.g., lingual viewpoint), and so on. Each image may include a height map that includes depth values ​​associated with each pixel of the image. For each image, a probabilistic map or mask can be generated based on the labeled dental classifications in the 3D model and a 2D surface on which the 3D model is projected. The probabilistic map or mask can have a size equal to the pixel size of the generated image. Each point or pixel in the probabilistic map or mask can include a probability value indicating the probability that the point represents one or more dental classifications. For example, there can be four dental classifications, including a first dental classification representing excess material, a second dental classification representing teeth, a third dental classification representing gums, and a fourth dental classification representing restorative objects. For example, a point with the first dental classification can have the value (1, 0, 0, 0).

[0389] (100% probability for the first dental classification and 0% probability for the second, third, and fourth dental classifications), points with the second dental classification can have values ​​(0, 1, 0, 0), points with the third dental classification can have values ​​(0, 0, 1, 0), and points with the fourth dental classification can have values ​​(0, 0, 0, 1). If the machine learning model is trained to perform image-level classification / prediction as opposed to pixel-level classification / segmentation, then, unlike the schematic diagram with pixel-level values, a single value or label can be associated with the generated image.

[0390] Training datasets can be collected, where each data item in the training dataset can include an image (e.g., an image including a heightmap) or a 3D surface and an associated probability map (which could be a 2D schematic if associated with an image, or a 3D schematic if associated with a 3D surface) and / or other labels. Additional data can also be included in the training data items. Segmentation accuracy can be improved through additional classes, inputs, and multiple view support. Multiple information sources can be incorporated into the model input and used jointly for prediction. Multiple dental classifications can be predicted simultaneously from a single model or using multiple models. Multiple problems can be solved simultaneously: role classification, tooth / gingival / restorative object segmentation, view determination, etc. Accuracy is higher than traditional image and signal processing schemes.

[0391] Additional data may include color image data. For example, a corresponding color image may also exist for each intraoral scan or image (which may be monochrome). Each data item may include a scan (e.g., a height map) and a color image. Two different types of color images are available. One type of color image is a viewfinder image, and the other type is a scan texture. A scan texture may be a combination or blend of multiple different viewfinder images. Each intraoral scan may be associated with a corresponding viewfinder image generated approximately simultaneously with the generation of the intraoral image. If blended scanning is used, each scan texture may be based on a combination of viewfinder images associated with the original scan used to generate a particular blended scan.

[0392] The default method can be based solely on depth information and still allow differentiation of several dental categories, such as teeth, gums, excess material (e.g., moved tissue), restorative objects, etc. However, sometimes depth information is insufficient for good accuracy. For example, a partially scanned tooth might appear as gum or even monochromatic excess material. In such cases, color information can be helpful. In one embodiment, color information is used as an additional 3 layers (e.g., RGB), resulting in a 4-layer input to the network. Two types of color information can be used, which may include viewfinder images and scan textures. Viewfinder images offer better quality but require alignment relative to the height map. Scan textures are aligned with the height map but may have color artifacts.

[0393] Another type of additional data may include images generated under specific lighting conditions (e.g., images generated under ultraviolet or infrared illumination). The additional data may be 2D or 3D images and may or may not include height maps.

[0394] In some embodiments, the set of data points is associated with the same tooth location and is labeled sequentially. In some embodiments, a recurrent neural network is used, and the data points are fed into the machine learning model in ascending order during training.

[0395] In some embodiments, each image or scan includes two values ​​for each pixel in the image, where the first value represents the height (e.g., a height map is provided), and the second value represents the intensity. Both the height and intensity values ​​can be used to train a machine learning model.

[0396] In one example, a confocal intraoral scanner can determine the height of a point on a surface (captured by pixels of the intraoral image) based on the scanner's focus setting, which produces the maximum intensity of that point on the surface. The focus setting provides either a height or depth value for that point. Intensity values ​​(called rank) are typically discarded. However, the intensity values ​​(rank) associated with the height or depth values ​​can be retained and included in the input data provided to a machine learning model.

[0397] Once one or more trained ML models 238 are generated, they can be stored in model memory 245 and added to an intraoral scanning application (e.g., intraoral scanning application 115). The intraoral scanning application 115 can then use the one or more trained ML models 238 along with additional processing logic to implement a “smart scanning” mode, in which user-initiated information is minimized or even eliminated in some cases.

[0398] In one embodiment, the model application workflow 217 includes one or more trained machine learning models that serve as a tooth position classifier 268, a scan completion recognizer 267, and a role recognizer 264. In embodiments, this logic can be implemented as a single machine learning model or a single combination of machine learning models. For example, the role recognizer 264, scan completion recognizer 267, and tooth position classifier 268 may share one or more layers of a deep neural network. However, each of these logics may include different higher-level layers of a deep neural network trained to generate different types of output. For simplicity, the examples shown only illustrate some of the functions described in the task list above. However, it should be understood that any other tasks can also be added to the model application workflow 217.

[0399] For model application workflow 217, according to one embodiment, an intraoral scanner generates a sequence of intraoral scans 248. A 3D surface generator 255 can perform registration between these intraoral scans, stitching the intraoral scans together, and generating a 3D surface 260 from the intraoral scans. When generating further intraoral scans, these can be registered and stitched together to the 3D surface 260, thereby increasing the size and data volume of the 3D surface 260. Input data 262 may include one or more of the intraoral scans 248 and / or the generated 3D surfaces 260.

[0400] Input data 262 may be fed into a tooth location classifier 268, which may include a trained neural network. Based on the input data 262, the tooth location classifier 268 outputs information about tooth location classification 270, which may be a point-level (e.g., pixel-level) classification of the input data. This may include outputting a set of classification probabilities for each pixel and / or a single classification for each pixel. The output tooth location classification 270 may be, for example, a mask or mapping of classifications and / or classification probabilities. In one embodiment, the tooth location classifier 268 identifies for each pixel whether it represents a tooth, gingival margin, or restorative object. If a pixel is below a probability threshold for a tooth, gingival margin, or restorative object, the tooth location classifier 268 may additionally classify the pixel as "other". In one embodiment, the tooth location classifier 268 additionally classifies pixels representing movable tissue (extra tissue), palate, prepared teeth, restorative objects other than prepared teeth, implants, brackets, dental attachments, tongue, soft tissue, etc. The tooth location classifier 268 can be trained to classify any one or more of the dental classifications described. In some embodiments, different types of restorative objects, different types of implants, different types of brackets, different types of attachments, and different types of soft tissue (e.g., tongue, lips, cheeks, etc.) can be identified.

[0401] When a single intraoral scan 248 has been generated, the input data 262 for the tooth location classifier 268 can include that single scan. Once multiple scans 248 have been generated, the input data 262 for the tooth location classifier 268 can include multiple scans. Classification based on multiple scans may be more accurate than classification based on a single scan. Once a 3D surface 260 has been generated, the input data 262 for the tooth location classifier 268 can include the 3D surface (e.g., one or more projections of the 3D surface onto one or more planes), which can produce more accurate segmentation.

[0402] Input data 262 can be fed into a role recognizer 264, which may include a trained neural network. Based on the input data 262, the role recognizer 264 outputs a classification of the scanned role 266 associated with the input data 262. For example, the role recognizer 264 may classify the input data 262 as associated with a maxillary role, a mandibular role, or an occlusal role. When a single intraoral scan 248 has been generated, the input data 262 of the role recognizer 264 may include that single scan. Once multiple scans 248 have been generated, the input data 262 of the role recognizer 264 may include the multiple scans. Classification based on multiple scans may be more accurate than classification based on a single scan. Once a 3D surface 260 has been generated, the input data 262 of the role recognizer 264 may include the 3D surface or multiple projections of the 3D surface onto a plane, which can produce a more accurate role classification.

[0403] Optionally, segmentation information (e.g., pixel-level tooth location classification 270) can be input as an additional layer into the character recognizer 264. This can improve the accuracy of the character recognizer 264. For example, the pixel-level tooth location classification 270 may include information about pixels or points classified as the maxilla and pixels or points classified as the tongue. Typically, a scan with at least a threshold number of pixels / points classified as maxilla is a scan of the maxillary arch. Similarly, a scan with at least a threshold number of pixels / points classified as tongue is typically a scan of the mandibular arch. Therefore, tooth location classification information can help improve the accuracy of the character recognizer.

[0404] In one embodiment, the lower dental arch is detected if at least a first threshold number of points in a first three-dimensional surface or intraoral scan depicts the tongue. The first threshold number may be, for example, 2%, 5%, 10%, 15%, 20%, 25%, 30%, 35%, or some other percentage of the total number of pixels / points in the scan, scan set, or 3D surface. In one embodiment, the upper dental arch is detected if at least a second threshold number of points in a first three-dimensional surface or intraoral scan depicts the maxilla. The second threshold number may be, for example, 2%, 5%, 10%, 15%, 20%, 25%, 30%, 35%, or some other percentage of the total number of pixels / points in the scan, scan set, or 3D surface. In one embodiment, occlusion is detected if at least a third threshold number of points in a first three-dimensional surface or intraoral scan depicts teeth from the lower dental arch and at least a third threshold number of points in a first three-dimensional surface or intraoral scan depicts the upper dental arch. The third threshold number can be, for example, 2%, 5%, 10%, 15%, 20%, 25%, 30%, 35%, or some other percentage of the total number of pixels / points in a scan, scan set, or 3D surface.

[0405] The role recognizer 264 may also include logic that performs one or more operations based on the output of a trained ML model. For example, the trained ML model may process each intraoral scan 248 to determine the role classification for that scan. The additional logic of the role recognizer 264 may then determine a moving median or moving average of the ML model's output for a window of intraoral scans, and determine the role classification based on the moving median or moving average.

[0406] Input data 262 can be fed into a scan completion recognizer 267, which may include a trained neural network. Based on the input data 262, the scan completion recognizer 267 can output a prediction, referred to as completion data 269, regarding whether the scan of a specific region segment (e.g., the upper or lower dental arch) is complete. The scan completion recognizer 267 can additionally determine whether the scan of all segments is complete. Optionally, the scan completion recognizer 267 can receive role classification data 266 output by the role recognizer 264 as an additional input layer. The role classification information 266 can improve the accuracy of determining the completion of a segment and / or all segments. For example, this can increase the probability of maxillary segment completion when the role changes from maxillary to mandibular. Similarly, this can increase the likelihood of mandibular segment completion when the role changes from mandibular to maxillary. Additionally, this information can indicate the possibility of completion of both the upper and lower segments when the role changes from maxillary or mandibular to occlusal.

[0407] Upon completion of a segment scan (e.g., the upper or lower dental arch), the 3D model generator 276 performs more precise registration and stitching of the intraoral scan 248 based on the input data 262 to generate a 3D model 278 of the completed segment. In an embodiment, the 3D model generator 276 automatically generates a 3D model of the segment in response to receiving completion data 269 indicating the completion of the segment scan. The 3D model generator 276 may additionally receive role classification information 266 from a role recognizer 264 and may automatically and appropriately label the generated 3D model (e.g., as the upper or lower dental arch) based on the role classification information 266. The 3D model generator 276 may additionally receive tooth location classification information 270 from a tooth location classifier 268 and may optionally apply tooth location classification to the generated 3D model. For example, the 3D model generator 276 may label teeth, gingival margins, prepared teeth and other restorative objects, brackets, dental attachments, etc., in the 3D model.

[0408] Once the 3D models of the upper and lower dental arches are completed and the occlusal roles are scanned, the post-processor 283 can automatically perform one or more post-processing operations. This may include generating occlusal maps, analyzing the occlusal contact between the upper and lower dental arches, determining the edge lines of prepared teeth, and determining the quality of the edge lines. Many other post-processing operations can also be performed.

[0409] Prescription generator 272 can automatically initiate a prescription 274 for a patient and fill in some or all of the information for prescription 274. Prescription generator 272 can receive input data 262 and / or a 3D model, and can determine the patient's identity based on inputting the input data 262 and / or the 3D model into a machine learning model trained to identify past patients based on their dental arches, and / or can compare the input data 262 and / or the 3D model with stored 3D models of known patients. If the trained ML model outputs an identification of a specific patient, or if there is a match between the input data and a portion of a stored 3D model of the patient's dental arch, prescription generator 272 can determine the identity of the patient associated with the input data 262 and / or the 3D model, and can fill in one or more patient details in prescription 274 based on stored information about the identified patient. Such information may include the patient's name, gender, age, allergies, etc.

[0410] The prescription generator 272 may additionally receive information about dental classification 270 from input data 262 and / or may receive one or more 3D models of the patient's dental arch. The 3D models may include point-level labels for the dental classification. The dental classification may include information such as the identification of natural teeth, the identification of one or more restorative objects (e.g., tooth preparation), the identification of the gingival margin, etc.

[0411] Prescription generator 272 can automatically predict whether a patient needs orthodontic and / or restorative treatment based on input data 262, dental classification information 270, and / or (multiple) 3D models 278. Prescription generator 272 may include information associated with a specific dental clinic where a model application workflow 217 is executed. This information may include historical information about multiple restorative treatments and / or multiple orthodontic treatments already performed at the dental clinic. If only restorative treatment has been performed, prescription generator 272 can automatically determine that restorative treatment will be performed and can initiate a restorative workflow. If only orthodontic treatment has been performed, prescription generator 272 can automatically determine that orthodontic treatment will be performed and initiate an orthodontic workflow. If the dental clinic performs both restorative and orthodontic treatment, prescription generator 272 can use input data 262, dental classification 270, and / or (multiple) 3D models 278 to determine whether orthodontic or restorative treatment should be performed. In one embodiment, if no restorative object is identified and one or more other cues indicate that orthodontic treatment should be performed (e.g., malocclusion is detected), the prescription generator 272 may determine that orthodontic treatment should be performed and initiate an orthodontic workflow. In some embodiments, if no restorative object is detected, the processing logic may determine that the input data (e.g., from a scanning session) comes from a patient scheduled visit (e.g., an examination). For example, this determination may be made based on comparing the current date associated with the input data with the dates of one or more previously generated 3D models. If the date indicates a periodic or regular timing of the scanning session, the processing logic may determine that the current input data is associated with a scheduled patient visit and not necessarily with orthodontic or restorative treatment. Additionally, if a first 3D model is generated without a restorative object, and a second 3D model or 3D surface with one or more restorative objects is subsequently generated on the same day, the processing logic may determine that the first 3D model is a pre-treatment 3D model. In some embodiments, if no restorative object is detected, the processing logic may determine that some medical treatment workflow other than an orthodontic or restorative workflow should be performed. In one embodiment, if one or more restorative objects are identified, the prescription generator 272 can determine that restorative treatment should be performed and can begin the restorative workflow.

[0412] Prescription generator 272 can identify the tooth number associated with the restorative object (e.g., optionally according to the American Dental Association (ADA) tooth numbering system) based on input data 262, dental classification data 270, and / or 3D model 278. In one embodiment, prescription generator 272 includes a trained machine learning model that has been trained to determine the tooth number associated with the restorative object. Prescription 274 can then be filled with information about which teeth will receive oral restorations.

[0413] Prescription generator 272 may include a trained machine learning model that has been trained to determine the type of dental prosthesis to be applied to each restorative object. The machine learning model may have been trained using training data including scans, images, 3D models, 3D surfaces, etc., of the dental arches of the restorative objects, as well as labels indicating what type of dental prosthesis is applied to those restorative objects. Input data 262, dental classification 270, and / or (multiple) 3D models 278 may be input into the trained ML model, which can output a prediction of the type of dental prosthesis to be applied to each restorative object. Prescription 274 can be automatically filled using an indication of which dental prosthesis to use for each tooth number of the restorative object.

[0414] Prescription generator 272 may include a trained machine learning model that has been trained to determine the type of material to be used for each dental prosthesis included in prescription 274. The machine learning model may have been trained using training data including scans, images, 3D models, 3D surfaces, etc., of the dental arches of the restorative objects, and labels indicating what type of material is used for the dental prostheses applied to those restorative objects. Input data 262, dental classification 270, and / or (multiple) 3D models 278 may be input into the trained ML model, which can output a prediction of the type of material to be used for the dental prosthesis to be applied to each restorative object. Prescription 274 can automatically fill in an indication of what material is used for each dental prosthesis included in prescription 274.

[0415] Prescription generator 272 may include a trained machine learning model that has been trained to determine the laboratory included in prescription 274 for manufacturing each dental prosthesis. The machine learning model may have been trained using training data including scans, images, 3D models, 3D surfaces, etc., of the dental arches of the restorative objects, as well as labels indicating which laboratory should be used to manufacture the dental prostheses applied to those restorative objects. Input data 262, dental classification 270, and / or (multiple) 3D models 278 may be input into the trained ML model, which can output a prediction of the dental laboratory for each restorative object to be used for the dental prosthesis to be applied to that object. Prescription 274 may automatically fill in instructions on which laboratory should be used for each dental prosthesis included in prescription 274.

[0416] In an embodiment, one or more of the ML models of the prescription generator 272 identified above may be combined into a single trained ML model (e.g., a single deep neural network).

[0417] In some implementations of model application workflow 217, the dirty scanner determiner 280 automatically detects one or more dirty optical surfaces of the scanner. The dirty scanner determiner 280 may or may not use a trained ML model to detect dirty optical surfaces. In one embodiment, the dirty scanner determiner 280 includes a trained ML model that has been trained to receive input data 262.

[0418] (e.g., intraoral scans and / or color images) and outputs a classification of dirty or clean optical surfaces. In one embodiment, the trained ML model outputs pixel-level classifications of clean and dirty areas within the scanner's field of view (FOV).

[0419] Instead of using a machine learning (ML) model to identify dirty areas on an optical surface, or in addition to using an ML model, the dirt scanner determiner 280 can use image processing techniques to identify dirty areas on the optical surface. In one embodiment, the dirt scanner determiner 280 determines dirty areas on the optical surface based on depth information from an intraoral scan. If an area of ​​the optical surface is contaminated with dirt, dust, blood, etc., the detected depth of pixels associated with that area will typically be much smaller than the depth of pixels not associated with dirty areas. The detected depth (or height) can be compared to one or more depth thresholds (or one or more height thresholds), and the depth of the dirty area at or below said one or more depth thresholds (or at or above said one or more height thresholds) can be detected.

[0420] The dirty scanner determiner 280 can determine the size of a dirty area and / or the percentage of dirty optical surfaces. If a dirty area is detected to have a size exceeding a size threshold and / or the percentage of dirty optical surfaces exceeds a threshold, the dirty scanner determiner 280 can determine that the scanner (or a sleeve or accessory on the scanner) is dirty. For example, the dirty scanner determiner 280 can output scanner cleanliness information 282. If the scanner cleanliness information 282 indicates a dirty scanner, the dirty scanner determiner 280 can output a notification to replace the sleeve or accessory on the scanner or to clean the scanner. Alternatively or additionally, the processing logic can output an indication of the amount or percentage of dirty optical surfaces (e.g., the window of the sleeve). This indication can appear once a threshold amount of optical surface dirt is detected and can be updated as the optical surface becomes dirtier and / or cleaner. In some embodiments, different dirt thresholds are used. If the amount of occluded pixels exceeds a first dirt threshold, a notification can be output. If the amount of occluded pixels exceeds a second, larger dirtiness threshold, the scan can be automatically paused and / or prescription 274 generated by the intraoral scan using the dirtiness scanner can be prevented from being sent to the dental laboratory.

[0421] In this embodiment, the dirty scanner determiner 280 can determine which optical surfaces are dirty. The dirty scanner determiner 280 can output different notifications based on which surfaces are dirty. For example, if the sleeve window is dirty, the dirty scanner determiner 280 can output a notification to replace the sleeve. However, if the scanner window or mirror is dirty, the dirty scanner determiner 280 can output a notification to clean the window or mirror.

[0422] Figure 2B An example intraoral scanning workflow 284 according to one embodiment of the present disclosure is shown. The intraoral scanning workflow 284 can be, for example, by… Figure 1 System 100 executes the procedure. An example intraoral scanning workflow 284 begins with a physician opening the intraoral scanner and inserting it into the patient's oral cavity (box 285). Processing logic (e.g., executed on scanner 150 and / or computing device 105) then automatically detects when the scanner is inserted into the patient's oral cavity (box 286). In one embodiment, the scanner begins generating periodic images (e.g., color 2D images) when it is opened, and these images are fed into a trained ML model that has been trained to detect insertions into the oral cavity. The processing logic may detect when the scanner is input into the patient's oral cavity in response to an instruction from the ML model to output one or more images depicting objects in the oral cavity. At box 287, the processing logic automatically begins generating the intraoral scan. This may also include automatically outputting structured light if the scanner uses structured light to determine depth information for the scan.

[0423] At box 288, the processing logic automatically determines a first dental arch (e.g., upper or lower dental arch) associated with one or more generated intraoral scans. In one embodiment, the intraoral scan is input into a trained ML model trained to determine the scan role associated with the scan. The processing logic may detect that the scanner is scanning the first dental arch in response to an indication output by the ML model that one or more scans depict the first dental arch. At box 289, the processing logic automatically generates a first 3D surface of the first dental arch by registering and stitching together the scans of the dental arch. When additional scans are received and classified as scans of the first dental arch, these scans may be stitched onto the first 3D surface. The 3D surface, or one or more views or portions of the 3D surface, may be input into the trained ML model or another trained ML model trained to determine the scan role associated with the 3D surface. The processing logic may confirm that the scanner is scanning the first dental arch in response to an indication output by the ML model that the 3D surface is the first dental arch. Alternatively, if the ML model determines that the 3D surface is of the second dental arch, the classification of the first dental arch can be changed to the classification of the second dental arch for the 3D surface and the classification of the associated intraoral scan for generating the 3D surface.

[0424] As the dentist continues scanning the patient's mouth, the dentist will eventually complete scanning the first dental arch and begin scanning the second dental arch (e.g., switching from scanning the upper dental arch to scanning the lower dental arch, or vice versa). At box 290, the processing logic automatically detects the switch from scanning the first dental arch to scanning the second dental arch and determines that one or more recent intraoral scans are of the second dental arch. In one embodiment, the intraoral scan is input to a trained ML model, which is trained to determine the scan role associated with that scan. The processing logic may detect that the scanner is scanning the second dental arch (and has switched from scanning the first dental arch) in response to an instruction from the ML model that one or more scans depict the second dental arch. At box 290, the processing logic automatically generates a second 3D surface of the second dental arch by registering and stitching the scans together. When additional scans are received and classified as scans of the second dental arch, these scans can be stitched onto the second 3D surface. A second 3D surface, or one or more views or portions of a second 3D surface, may be input into the trained ML model or another trained ML model trained to determine the scanning role associated with the 3D surface. The processing logic may confirm that the scanner is scanning the second dental arch in response to an ML model output indicating that the 3D surface is the second dental arch. Alternatively, if the ML model determines that the 3D surface is the first dental arch, the classification of the second dental arch may become the classification of the first dental arch for the second 3D surface and the classification of the associated intraoral scan for generating the second 3D surface.

[0425] Once the dentist has completed scanning the first and second dental arches (e.g., the patient's upper and lower dental arches), the dentist can transition to scanning the patient's occlusion. The dentist can instruct the patient to close their mouth, and one or more scans of the closed mouth can be generated to show the relationship between the upper and lower dental arches. At box 292, the processing logic automatically detects the switch from scanning the second dental arch to scanning the occlusion and determines one or more recent intraoral scans that represent the patient's occlusion (occlusal role). In one embodiment, the intraoral scans(s) are input into a trained ML model that is trained to determine the scan role associated with the scan. The processing logic can detect that the scanner is scanning the patient's occlusion (and has switched from scanning the second dental arch) in response to an instruction from the ML model outputting one or more scans depicting the patient's occlusion.

[0426] At box 292, the processing logic automatically determines the relative position and orientation of the first 3D surface with respect to the second 3D surface based on one or more scans of the patient's occlusion. At box 294, the processing logic can automatically generate 3D models of the first dental arch (e.g., the maxillary arch) and the second dental arch (e.g., the mandibular arch). Alternatively, if it can be determined at this stage that the entire first dental arch is completed on the first 3D surface, a 3D model of the first dental arch can be automatically generated after box 290. Additionally, if it can be determined at this stage that the entire second dental arch is completed on the second 3D surface, a 3D model of the second dental arch can be automatically generated after box 292.

[0427] At box 295, the processing logic automatically performs one or more post-processing operations on the first and / or second 3D models. This may include, for example, automatically determining occlusal clearance information of teeth in the first and second dental arches, and performing occlusal clearance analysis to determine if any problematic contact points exist between teeth in the upper and lower dental arches.

[0428] At box 296, the processing logic automatically generates review trajectories for the first and / or second dental arches. This may include determining a sequence of views of the first and / or second dental arches and transitions between views. The trajectory may include rotating the 3D model, translating the 3D model, zooming in or out on certain areas of the 3D model (e.g., for potential problem areas, such as areas identified from occlusal gap analysis), etc. The trajectory may be automatically determined based on the physician's historical review of the 3D model of the dental arch. In one embodiment, the one or more 3D models are input into a trained ML model that has been trained to generate review trajectories for the 3D surface / 3D model based on the input. At box 297, the processing logic automatically displays the first and / or second 3D models based on the automatically determined review trajectories. At any time during the display of the 3D models, the physician may instruct the processing logic to pause, rewind, accelerate, slow down, etc., the trajectory. The physician may also cancel the automatically determined review trajectories and manually manipulate the views of the first and / or second 3D models at any time.

[0429] Figure 3 , Figures 7-11 , Figures 14A-18 , Figures 20-31 , Figures 35-38 and Figure 41 This is a flowchart illustrating various methods that can be executed to implement "smart scanning," which reduces the amount of user input and streamlines the scanning process. The methods can be executed by processing logic, which may include hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions running on a processing device to execute hardware emulation), firmware, or a combination thereof. In one embodiment, at least some operations of the methods are performed by the computing device of the scanning system and / or by a server computing device (e.g., by...). Figure 1 Computing device 105 or Figure 42 The computing device 4200 executes the commands.

[0430] Figure 3This is a flowchart illustrating an embodiment of a method 300 for training a machine learning model to identify scan roles. At block 302 of method 300, processing logic collects a training dataset that may include intraoral scans of tooth locations (e.g., height maps), 3D surfaces of tooth locations, 2D images of tooth locations, and / or projections of the 3D surfaces of tooth locations. Each data item in the training dataset (e.g., intraoral scan, image, 3D surface, etc.) may include one or more labels. Data items in the training dataset may include image-level labels indicating scan roles. For example, some intraoral scans may include labels for the lower dental arch, some intraoral scans may include labels for the upper dental arch, and some intraoral scans may include labels for occlusion. Data items in the training dataset may also include other labels, such as pixel-level classifications of dental categories, such as teeth, gingival margins, restorative objects, etc. (See reference...) Figure 9 The training of the ML model to perform pixel-level classification for dental classification is described in more detail. Data items may also include other labels, such as labels for lingual view, buccal view, left side of the dental arch, right side of the dental arch, occlusal view, etc. As mentioned above, several other types of labels can also be associated with data items in the training dataset.

[0431] At box 304, data items from the training dataset are fed into the untrained machine learning model. At box 306, the machine learning model is trained ba...

Claims

1. A method for operating a computing device connected to an intraoral scanner via a wired or wireless connection, wherein, The method includes: Receive one or more first intraoral scans of the patient's oral cavity from the intraoral scanner and during an intraoral scanning session in the patient's oral cavity; During the intraoral scanning session and based on the processing of the one or more first intraoral scans, a first identity of the patient's first dental arch associated with the one or more first intraoral scans is automatically determined, wherein the first identity is the first of a classification of the upper dental arch, a classification of the lower dental arch, or a classification of occlusion, wherein, for an intraoral scan in the one or more first intraoral scans, the lower dental arch is detected if at least a threshold number of points in the intraoral scan depict the tongue; and Determine the first three-dimensional surface of the first dental arch.

2. The method according to claim 1, further comprising: Receive one or more second intraoral scans of the patient's oral cavity without receiving instructions that the one or more second intraoral scans are associated with a second identity of the patient's second dental arch; Based on the processing of the one or more second intraoral scans, a second identity associated with the one or more second intraoral scans is automatically determined, wherein the second identity is the second of the classification of the maxillary arch, the classification of the mandibular arch, or the classification of the occlusion; as well as Determine the second three-dimensional surface of the second dental arch.

3. The method according to claim 1, wherein, Processing the first identity includes: The one or more first intraoral scans are input into a machine learning model, which has been trained to classify the intraoral scans as being associated with the upper dental arch, the lower dental arch, or the occlusion, wherein the machine learning model outputs the first identity.

4. The method according to claim 1, wherein, Automatically determining the first identity of the patient's first dental arch includes: The one or more first intraoral scans are used to depict the patient's first dental arch; Determine the first identity of the patient's first dental arch.

5. The method according to claim 4, further comprising: Receive user input, the user input instructing the one or more first intraoral scans to depict the patient’s second dental arch, the second dental arch having a second identity; The user input is determined to be incorrect; as well as The output provides a notification that the one or more first intraoral scans depict the first dental arch with the first identity, rather than the second dental arch with the second identity.

6. The method according to claim 4, further comprising: The first three-dimensional surface of the first dental arch is now determined; as well as In response to determining that the first dental arch is complete, a first three-dimensional model of the first dental arch is automatically generated.

7. The method according to claim 4, wherein, Receive one or more first intraoral scans of the patient's oral cavity without first receiving an indication of the identity of the first dental arch or an indication that a new dental arch is being scanned.

8. The method according to claim 4, wherein, Determining the first dental arch of a patient as depicted by the one or more first intraoral scans and determining the first identity of the patient's first dental arch includes: The one or more first intraoral scans are input into a machine learning model, which has been trained to classify the intraoral scans as depicting the upper dental arch, the lower dental arch, or the occlusion, wherein the machine learning model outputs a first classification indicating the first identity of the first dental arch.

9. The method according to claim 8, wherein, The one or more first intraoral scans include multiple intraoral scans, and wherein determining that the one or more first intraoral scans depict the patient's first dental arch and determining the first identity of the patient's first dental arch includes: Each of the plurality of intraoral scans is input into the machine learning model, wherein the machine learning model outputs multiple classifications, each of which is associated with one of the plurality of intraoral scans; and The majority of the classifications output by the machine learning model indicate the first identity of the first dental arch.

10. The method according to claim 9, wherein, For the first three-dimensional surface or at least one of the intraoral scans in one or more first intraoral scans: The lower dental arch is detected when the tongue is depicted at least a first threshold number of points in the first three-dimensional surface or the intraoral scan; The maxillary arch is detected when the maxilla is depicted at least a second threshold number of points in the first three-dimensional surface or the intraoral scan. as well as The occlusion is detected when at least a third threshold number of points in the first three-dimensional surface or the intraoral scan depict teeth from the lower dental arch and at least a third threshold number of points in the first three-dimensional surface or the intraoral scan depict the upper dental arch.

11. The method according to claim 9, wherein, The one or more first intraoral scans include multiple intraoral scans, and wherein determining that the one or more first intraoral scans depict the patient's first dental arch and determining the first identity of the patient's first dental arch includes: Each of the plurality of intraoral scans is input into the machine learning model, wherein the machine learning model outputs multiple classifications, each of which is associated with one of the plurality of intraoral scans; and Determine the moving average of the plurality of categories output by the machine learning model, wherein the moving average indicates the first identity of the first dental arch.

12. The method according to claim 9, wherein, The one or more first intraoral scans include multiple intraoral scans received in a sequential order, wherein the one or more first intraoral scans are input into the machine learning model in the said sequential order, and wherein the machine learning model is a recurrent neural network.

13. The method according to claim 9, wherein, For each of the one or more first intraoral scans, the machine learning model outputs a confidence value, and the method further includes: For each of the one or more first intraoral scans, determine whether the confidence value associated with the output of the machine learning model for that intraoral scan is below a confidence threshold; and The output of the machine learning model with a confidence value below the stated confidence threshold is discarded.

14. The method according to claim 9, wherein, When the one or more first intraoral scans are received and before the intraoral scan of the first dental arch is completed, the one or more first intraoral scans are input into the machine learning model, the method further includes: A height map of the first dental arch is generated by projecting at least a portion of the first three-dimensional surface of the first dental arch onto a plane; and The data from the height map is processed using the machine learning model or an alternative machine learning model that has been trained to classify the height map as depicting the upper dental arch, the lower dental arch, or the occlusion, wherein the machine learning model or the alternative machine learning model outputs a second classification that, compared to the first classification, indicates the first identity of the first dental arch with a higher level of accuracy.

15. The method according to claim 4, further comprising: Receive a second intraoral scan depicting the first occlusal relationship between the upper and lower dental arches, the second intraoral scan having been generated in the first moment; Receive a third intraoral scan depicting a second occlusal relationship between the upper and lower dental arches, the third intraoral scan having been generated at a second time; Determine the first difference between the first occlusal relationship and the second occlusal relationship; Determine a second difference between the first time and the second time; as well as The second intraoral scan and the third intraoral scan are determined, at least in part, based on the first difference and the second difference, whether they depict the same occlusion or a different occlusion of the patient.

16. The method according to claim 4, wherein, Determining the first dental arch of a patient as depicted by the one or more first intraoral scans and determining the first identity of the patient's first dental arch includes: Determine whether the first three-dimensional surface or at least one of the intraoral scans of one or more first intraoral scans includes a representation of the tongue or palate; and In response to determining a representation of the tongue on the first three-dimensional surface or at least one of the one or more first intraoral scans, the first identity of the patient's first dental arch is determined to be mandibular; or In response to determining that the first three-dimensional surface or at least one of the first intraoral scans includes a representation of the maxilla, the first identity of the patient's first dental arch is determined to be maxillary.

17. The method according to claim 4, wherein, Determining the first dental arch of a patient as depicted by the one or more first intraoral scans and determining the first identity of the patient's first dental arch includes: Determine whether at least one of the one or more first intraoral scans generated before the intraoral scanner is inserted into the patient's mouth depicts the nose or the chin; and In response to determining that at least one of the one or more first intraoral scans includes a representation of the mandible, the first identity of the patient's first dental arch is determined to be for the mandibular arch; or In response to determining that at least one of the one or more first intraoral scans includes a representation of the nose, the first identity of the patient's first dental arch is determined to be for the upper dental arch.

18. The method of claim 17, further comprising: Based on data from the inertial measurement unit of the intraoral scanner that generates the one or more first intraoral scans, after the one or more first intraoral scans are generated, the rotation of the intraoral scanner about the longitudinal axis of the intraoral scanner is detected. After the intraoral scanner rotates around the longitudinal axis, it receives one or more second intraoral scans of the patient's oral cavity; If the first identity of the first dental arch is the upper dental arch, determine that the one or more second intraoral scans depict the lower dental arch; as well as If the first identity of the first dental arch is the lower dental arch, then the one or more second intraoral scans are determined to depict the upper dental arch.

19. The method according to claim 4, wherein, Determining the first dental arch of a patient as depicted by the one or more first intraoral scans and determining the first identity of the patient's first dental arch includes: Generate an image of the first dental arch, the image including a height map; and Data from the images is processed using a machine learning model that has been trained to classify images of dental arches as depicting the upper dental arch, the lower dental arch, or the occlusion, wherein the machine learning model outputs a classification indicating the first identity of the first dental arch.

20. The method according to claim 19, wherein, The first three-dimensional surface is generated before determining the first identity of the first dental arch, and wherein the image of the first dental arch is generated by projecting at least a portion of the first three-dimensional surface of the first dental arch onto a two-dimensional surface.

21. The method according to claim 4, further comprising: The one or more first intraoral scans are marked as belonging to the first segment of the first dental arch; Receive one or more second intraoral scans of the patient's oral cavity; The one or more second intraoral scans are used to determine the first dental arch of the patient with the first identity; as well as The one or more second intraoral scans are marked as belonging to the second segment of the first dental arch.

22. The method according to claim 4, further comprising: Determine whether the one or more first intraoral scans depict a lingual view, a buccal view, or an occlusal view of the first dental arch.

23. A computer-readable medium includes instructions that, when executed by a processing means, cause the processing means to perform the method according to any one of claims 1 to 22.

24. A system for performing intraoral scanning and / or generating a virtual three-dimensional model of intraoral locations, comprising: An intraoral scanner, used to generate one or more intraoral scans; and A computing device connected to an intraoral scanner via a wired or wireless connection, the computing device being used to perform the method according to any one of claims 1 to 22.

Images