Methods and systems for patient dental representation

Abstract

Described herein are systems and methods for representing dentation of a subject from intraoral images or videos.

Description

Attorney Docket No. 57364-725601METHODS AND SYSTEMS FOR PATIENT DENTAL REPRESENTATIONCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims benefit to U.S. Provisional Application No. 63 / 680,101, filed August 7, 2024, which is incorporated herein by reference in its entirety.BACKGROUND

[0002] Dental professionals and orthodontists may treat and monitor a patient’s dental condition based on in-person visits. Treatment and monitoring of a patient’s dental condition may require a patient to schedule multiple in-person visits to a dentist or orthodontist. The quality of treatment and the accuracy of monitoring may vary depending on how often and how consistently a patient sees a dentist or orthodontist. In some cases, suboptimal treatment outcomes may result if a patient is unable or unwilling to schedule regular visits to a dentist or orthodontist.SUMMARY

[0003] Recognized herein is a need for dental monitoring solutions to allow dental assessment, optionally remote dental assessment, without requiring a dental professional to be physically present with the patient. However, such conventional teledentistry solutions may provide inadequate levels of supervision. Further, such conventional teledentistry solutions may be limited by an inaccurate or insufficient monitoring of a patient’s dental condition based on one or more photos taken by the patient, if the photos do not adequately capture various intraoral features.

[0004] In an aspect, provided herein is a method for generating an aligned dental representation of a subject, comprising: providing an oral image that includes at least part of one tooth; providing perspective of the oral image; generating distance data of at least two marks on the oral image; generating a reference plane; and projecting the oral image based on (i) the perspective of the oral image and (ii) the distance data on the reference plane to generate the aligned dental representation of the subject.

[0005] In some cases, the oral image was acquired using a mobile digital camera and a hollow tubular adapter, wherein a first opening of the hollow tubular adapter is configured to attach to the mobile digital camera, and wherein a second opening of the hollow tubular adapter is configured to contact lips of the subject. In some cases, an outer surface of theAttorney Docket No. 57364-725601 second opening of the hollow tubular adapter is configured to contact an inner portion of the lips of the subject.

[0006] In some cases, the reference plane is parallel to a sagittal plane of a head of the subject. In some cases, the reference plane intersects a midline of a dentition of the subject. In some cases, the reference plane is parallel to a transverse plane of a head of the subject. In some cases, the reference plane intersects a virtual line crossing two opposite molars of the subject. In some cases, the reference plane is parallel to a coronal plane of a head of the subject. In some cases, the reference plane intersects a virtual line crossing two opposite molars of the subject.

[0007] In some cases, the generating the distance data uses structure from motion, depth estimation neural network, or multi-view stereo algorithm. In some cases, the aligned dental representation is used to generate a cylindrical projection. In some cases, the aligned dental representation is used to generate a three-dimensional representation.

[0008] In another aspect, provided herein is a system, comprising: one or more memories; and at least one processor each coupled to at least one of the one or more memories and configured to perform operations, comprising: receiving an oral image that includes at least part of one tooth; receiving perspective of the oral image; generating distance data of at least two marks on the oral image; generating a reference plane; and projecting the oral image based on (i) the perspective of the oral image and (ii) the distance data on the reference plane to generate an aligned dental representation of a subject.

[0009] In some cases, the oral image was acquired using a mobile digital camera and a hollow tubular adapter, wherein a first opening of the hollow tubular adapter is configured to attach to the mobile digital camera, and wherein a second opening of the hollow tubular adapter is configured to lips of the subject. In some cases, an outer surface of the second opening of the hollow tubular adapter is configured to contact an inner portion of the lips of the subject.

[0010] In some cases, the reference plane is parallel to a sagittal plane of a head of the subject. In some cases, the reference plane intersects a midline of a dentition of the subject. In some cases, the reference plane is parallel to a transverse plane of a head of the subject. In some cases, the reference plane intersects a virtual line crossing two opposite molars of the subject. In some cases, the reference plane is parallel to a coronal plane of a head of the subject. In some cases, the reference plane intersects a virtual line crossing two opposite molars of the subject.Attorney Docket No. 57364-725601

[0011] In some cases, the generating the distance data uses structure from motion, depth estimation neural network, or multi-view stereo algorithm. In some cases, the aligned dental representation is used to generate a cylinder projection. In some cases, the aligned dental representation is used to generate a three-dimensional representation.

[0012] In another aspect, provided herein is a non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations, comprising: receiving an oral image that includes at least part of one tooth; receiving perspective of the oral image; generating distance data of at least two marks on the oral image; generating a reference plane; and projecting the oral image based on (i) the perspective of the oral image and (ii) the distance data on the reference plane to generate an aligned dental representation of a subject.

[0013] In some cases, the oral image was acquired using a mobile digital camera and a hollow tubular adapter, wherein a first opening of the hollow tubular adapter is configured to attach to the mobile digital camera, and wherein a second opening of the hollow tubular adapter is configured to contact lips of the subject. In some cases, an outer surface of the second opening of the hollow tubular adapter is configured to contact an inner portion of the lips of the subject.

[0014] In some cases, the reference plane is parallel to a sagittal plane of a head of the subject. In some cases, the reference plane intersects a midline of a dentition of the subject. In some cases, the reference plane is parallel to a transverse plane of a head of the subject. In some cases, the reference plane intersects a virtual line crossing two opposite molars of the subject. In some cases, the reference plane is parallel to a coronal plane of a head of the subject. In some cases, the reference plane intersects a virtual line crossing two opposite molars of the subject.

[0015] In some cases, the generating the distance data uses structure from motion, depth estimation neural network, or multi-view stereo algorithm. In some cases, the aligned dental representation is used to generate a cylinder projection. In some cases, the aligned dental representation is used to generate a three-dimensional representation.

[0016] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although suitable methods and materials are described below, methods and materials similar or equivalent to those described herein can be used in the practice of the present invention. In case of conflict, the patent specification, including definitions, willAttorney Docket No. 57364-725601 control. All materials, methods, and examples are illustrative only and are not intended to be limiting.

[0017] As used herein, the terms “comprising” and “including” or grammatical variants thereof are to be taken as specifying inclusion of the stated features, integers, actions or components without precluding the addition of one or more additional features, integers, actions, components or groups thereof. This term is broader than, and includes the terms "consisting of and "consisting essentially of' as defined by the Manual of Patent Examination Procedure of the United States Patent and Trademark Office.

[0018] The term "method" refers to manners, means, techniques and procedures for accomplishing a given task including, but not limited to, those manners, means, techniques and procedures either known to, or readily developed from known manners, means, techniques and procedures by practitioners of architecture and / or computer science.

[0019] Implementation of the method and system of the present invention involves performing or completing selected tasks or steps manually, automatically, or a combination thereof. Moreover, according to actual instrumentation and equipment of preferred embodiments of methods, apparatus and systems of the present invention, several selected steps could be implemented by hardware or by software on any operating system of any firmware or a combination thereof. For example, as hardware, selected steps of the invention could be implemented as a chip or a circuit. As software, selected steps of the invention could be implemented as a plurality of software instructions being executed by a computer using any suitable operating system. In any case, selected steps of the method and system of the invention could be described as being performed by a data processor, such as a computing platform for executing a plurality of instructions.INCORPORATION BY REFERENCE

[0020] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. To the extent publications and patents or patent applications incorporated by reference contradict the disclosure contained in the specification, the specification is intended to supersede and / or take precedence over any such contradictory material.Attorney Docket No. 57364-725601BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are incorporated herein and form a part of the specification.

[0022] FIG. l is a flowchart illustrating a method for generating a dental representation, according to some embodiments.

[0023] FIG. 2 is an illustrative example of up and down arches and alignment lines, according to some embodiments.

[0024] FIG. 3 A and 3B is an illustrative example of a dental adapter, according to some embodiments.

[0025] FIG. 4 shows a block diagram of a system configured to generate a dental representation, according to some embodiments.

[0026] FIG. 5 is an illustrative example of head reference alignment lines, according to some embodiments.

[0027] FIG. 6 shows an example image and representation associated with a potential patient, according to some embodiments.

[0028] FIG. 7 schematically illustrates a computer system that is programmed or otherwise configured to implement at least some of the methods disclosed herein, in accordance with some embodiments.

[0029] In the drawings, like reference numbers generally indicate identical or similar elements. Additionally, generally, the left-most digit(s) of a reference number identifies the drawing in which the reference number first appears.DETAILED DESCRIPTION

[0030] Provided herein are systems, apparatuses, devices, methods, and / or computer program product embodiments, and / or combinations and sub-combinations thereof, for generating dentition representation. As used herein, oral imaging may refer to the acquisition of one or more intraoral videos and / or intraoral images that can produce oral video and / or oral images or an oral video frame. The embodiments disclosed herein may provide a convenient solution and user experience for dental professionals for dentition representation. The embodiments disclosed herein may provide dentists and orthodontists with a detailed analysis of a patient’s dental condition based on one or more full arch scans captured remotely by the patient.Attorney Docket No. 57364-725601

[0031] In one or more embodiments, the dentition representation comprises: an oral image that includes at least part of one tooth; a perspective of the oral image; optionally, generating distance data of at least two marks on the oral image; generating a reference plane; and projecting the oral image based on the perspective and the distance data to the reference plane to generate an aligned dental representation.

[0032] In one or more embodiments, the oral image is acquired using a mobile digital camera and a hollow tubular adapter, wherein one opening of the hollow tubular adapter is attached to the mobile digital camera and a second opening of the hollow tubular adapter contacts the user inner lips. The lips can lay on the outer of the second opening of the hollow tubular.

[0033] In one or more embodiments, the reference plane is parallel to the head sagittal plane. In one or more embodiments, the reference plane intersects dentition midline. In one or more embodiments, the reference plane is parallel to the head transverse plane.

[0034] In one or more embodiments, the reference plane intersects a virtual line crossing two opposite molars. In one or more embodiments, the reference plane is parallel to the head coronal plane. In one or more embodiments, the reference plane intersects a virtual line crossing two opposite molars.

[0035] In one or more embodiments, the generating distance data uses structure from motion, depth estimation neural network, or multi-view stereo algorithm. In one or more embodiments, the aligned dental representation is used to generate a cylinder projection. In one or more embodiments, the aligned dental representation is used to generate a Three Dimension representation.

[0036] As used herein, the term “dental scan” refers to a video or an image capture of the intraoral perspective imaging of the teeth arch or of a tooth and / or a video or an image capture of extraoral perspective imaging a user’s facial features, including, but not limited to, the teeth arch, one or more teeth, the upper lip, the lower lip, the nose, one or more nostrils, one or more ears, one or more eyes, etc. of a patient.

[0037] The term “real-time,” as used herein, generally refers to a simultaneous or substantially simultaneous occurrence of a first event or action with respect to an occurrence of a second event or action. A real-time action or event may be performed within a response time of less than one or more of the following: ten seconds, five seconds, one second, a tenth of a second, a hundredth of a second, a millisecond, or less relative to at least another event or action. A real-time action may be performed by one or more computer processors.Attomey Docket No. 57364-725601

[0038] Whenever the term “at least,” “greater than,” or “greater than or equal to” precedes the first numerical value in a series of two or more numerical values, the term “at least,” “greater than” or “greater than or equal to” applies to each of the numerical values in that series of numerical values. For example, greater than or equal to 1, 2, or 3 is equivalent to greater than or equal to 1, greater than or equal to 2, or greater than or equal to 3.

[0039] Whenever the term “no more than,” “less than,” or “less than or equal to” precedes the first numerical value in a series of two or more numerical values, the term “no more than,” “less than,” or “less than or equal to” applies to each of the numerical values in that series of numerical values. For example, less than or equal to 3, 2, or 1 is equivalent to less than or equal to 3, less than or equal to 2, or less than or equal to 1.

[0040] The terms “a,” “an,” and “the,” as used herein, generally refer to singular and plural references unless the context clearly dictates otherwise.

[0041] Implementation of the methods and systems of the present embodiments can involve performing or completing selected tasks or steps manually, automatically, or a combination thereof. Moreover, according to actual instrumentation and equipment of particular embodiments of methods, apparatus and systems of the present disclosure, several selected steps may be implemented by hardware or by software on any operating system of any firmware or a combination thereof. For example, as hardware, selected steps of the embodiments described herein may be implemented as a chip or a circuit. As software, selected steps of the embodiments described herein may be implemented as a plurality of software instructions being executed by a computer using any suitable operating system. In any case, selected steps of the method and system of the embodiments described herein may be described as being performed by a data processor, such as a computing platform for executing a plurality of instructions.Overview

[0042] Embodiments described herein include methods and systems for patient onboarding and intraoral and / or extraoral assessment. As used herein, “intraoral assessment” may refer to assessment of an appearance or condition of an intraoral region of a subject (e.g., a region within the subject’s mouth), and “extraoral assessment” may refer to assessment of an appearance or condition of an extraoral region of a subject (e.g., a region outside the subject’s mouth). The embodiments disclosed herein can transfer one or more intraoral and / or extraoral videos and / or images taken by users using a mobile device such as a smartphone into a dental scan clip. The embodiments disclosed herein may provide dentists and orthodontists with a detailed analysis of the patient’s dental condition based on a video or on one or moreAttorney Docket No. 57364-725601 images of at least one tooth, such as one or more teeth, one or more dental arches or both dental arches of a subject, captured remotely by a user that has been converted by the present disclosure into a dental scan clip. In some embodiments, the user is the subject. In some embodiments the user is a person other than the subject, present at the same location as the subject and assessment of the captured video or image(s) is performed at a remote location.

[0043] As used herein the term “dental scope” or “dental adapter” refer to a device that can be attached or coupled to a camera or mobile phone and enables capturing at least one intraoral image or videos of a subject. An example and possible uses are described in US Patent application 17 / 336,997, which is incorporated by reference herein. The embodiments described herein may be implemented using a software application that is configured to enable a dental patient to capture videos and / or images of intraoral and / or extraoral regions. The software application may be used by a user or a subject (e.g., a dental patient) in conjunction with a mobile device to remotely monitor a dental condition of the subject. The dental condition may comprise a development, a growth, a movement, an appearance, a condition, a physical arrangement, a position, and / or an orientation of the subject’s teeth or / and the subject’s soft tissues in the oral cavity.

[0044] The intraoral adapter may be used for dental scanning for capturing a dental scan clip to be used in remote monitoring. As used herein, “remote monitoring” may refer to monitoring a condition of an intraoral and / or extraoral region of a subject, wherein the monitoring is performed at one or more locations remote from the subject. For example, a dentist or a medical specialist may monitor the intraoral and / or extraoral anatomy or intraoral and / or extraoral condition at a first location that is different than a second location at which the subject is located. The first location and the second location may be separated by a distance spanning at least 1 meter, 1 kilometer, 10 kilometers, 100 kilometers, 1000 kilometers, or more.

[0045] The remote monitoring may be performed by assessing an intraoral and / or extraoral condition of the subject using one or more dental scan clips generated from a dental scan captured from the subject when the subject is located remotely from the dentist or a dental office. In some cases, the remote monitoring may be performed in real-time such that a dentist is able to assess the dental condition when a subject uses a mobile device to acquire one or more videos or images of one or more intraoral and / or extraoral regions in the patient’s mouth. The remote monitoring may be performed using equipment, hardware, and / or software that is not physically located at a dental office.Attorney Docket No. 57364-725601

[0046] As used herein, the term “dental scan” may refer to a movie, video or animation, representing an edited capture of intraoral imaging that includes the teeth of the user using the capturing device with a selected start point, the appearance of the user’s teeth, and a selected end point.

[0047] As used herein, the term “3D dental data” may refer to a file or computer-readable data that represents the appearance of a user or generic intraoral imaging.

[0048] As used herein, the term “3D model” may refer to a file or computer-readable data that represents the appearance of a user or generic intraoral imaging and can be presented as a copy of user or a generic intraoral copy.Method for Generating an Aligned Dental Representation

[0049] In one or more embodiments, a method to generate dentition representation, is provided. In some cases, the method comprises: providing an oral image that includes at least part of one tooth, at least part of two teeth, at least part of two teeth at the same arch; providing perspective of the oral image; generating distance data of at least two marks on the oral image; generating a reference plane; projecting the oral image based on the perspective and the distance data to the reference plane to generate aligned dental representation.

[0050] In one or more embodiments, the oral image is a two-dimension (2D) image, a frame from a video file, or a movie. In one or more embodiments, the oral image represents a capture of visual light wavelength by a camera.

[0051] In one or more embodiments, the oral image is acquired using a mobile digital camera and a hollow tubular adapter, wherein one opening of the hollow tubular adapter is attached to the mobile digital camera and a second opening of the hollow tubular adapter contacts the user inner lips. The lips can lay on the outer of the second opening of the hollow tubular.

[0052] In one or more embodiments, the reference plane is parallel to the head sagittal plane.

[0053] In one or more embodiments, the reference plane intersects dentition midline.

[0054] In one or more embodiments, the reference plane is parallel to the head transverse plane.

[0055] In one or more embodiments, the reference plane intersects a virtual line crossing two opposite molars. In one or more embodiments, the reference plane intersects a virtual line crossing two opposite molars of the upper or the lower arch

[0056] In one or more embodiments, the reference plane is parallel to the head coronal plane.Attorney Docket No. 57364-725601

[0057] In one or more embodiments, the generating the reference plane comprises selection of defined reference plane or user selection reference plane.

[0058] In one or more embodiments, the reference plane is parallel to a virtual line crossing two opposite molars.

[0059] In one or more embodiments, the generating distance data uses structure from motion, depth estimation neural network, or multi-view stereo algorithm.

[0060] In one or more embodiments, the aligned dental representation is used to generate a cylinder projection.

[0061] In one or more embodiments, the aligned dental representation is used to generate a three-dimensional representation.

[0062] FIG. l is a flowchart illustrating a method for generating a dentition representation, according to some embodiments described herein. Method 100 can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown in FIG. 1, as will be understood by a person of ordinary skill in the art. In some aspects, method 100 may be implemented by system 400 (e.g., computing device 404A, computing device 404B, or server(s) 402), as described below. However, method 100 is not limited to that example embodiment.

[0063] At 110, an oral image can be provided.

[0064] At 120, a distance data can be generated.

[0065] At 130, a reference plane can be generated.

[0066] At 140, the oral image can be projected on the generated reference plane.

[0067] At 150, an aligned dental representation can be generated.

[0068] FIG. 2 is an illustrative example 200 of up and down arches and alignment lines, according to some embodiments.

[0069] An upper arch 201 and lower arch 202 represent full dentition of an adult. Line 203 marks the space between the upper central incisors. According to one or more embodiments, line 203 can be used for generating a reference plane. Line 204 marks the space between the lower central incisors. According to one or more embodiments, line 204 can be used for generating a reference plane.

[0070] Line 205 marks the virtual line crossing two upper opposite molars. According to one or more embodiments, line 205 can be used for generating a reference plane. Line 206Attorney Docket No. 57364-725601 marks the virtual line crossing two lower opposite molars. According to one or more embodiments, line 206 can be used for generating a reference plane.

[0071] FIG. 5 is an illustrative example of head reference alignment lines 500, according to some embodiments.

[0072] According to some embodiments, head sagittal plane marked in S, can be used for generating a reference plane. According to some embodiments, head dentition midline plane marked in S, can be used for generating a reference plane.

[0073] According to some embodiments, head transverse plane marked in A, can be used for generating a reference plane.

[0074] FIG. 3 A and 3B is an illustrative example of a dental adapter, according to some embodiments.

[0075] FIG. 3 A schematically illustrate another possible example of a dental adapter.

[0076] Intraoral adaptor 300 can comprise: an elongated housing of a viewing channel 305 made of distal element 302 and a proximal element 301. In some cases, a mounting mechanism 304 is configured to couple the intraoral adapter 300 to a mobile device.

[0077] FIG. 3B schematically illustrate another possible example of a dental adapter.

[0078] Intraoral adaptor 350 can comprise: an elongated housing of a viewing channel made of distal element 355 and a proximal element 352. In some cases, a mounting mechanism 354 is configured to couple the intraoral adapter 310 to a mobile device 360 with securing element 354. At some embodiments a handle 356 can be attached to the intraoral adapter 310.Systems for Generating an Aligned Dental Representation

[0079] In one or more embodiments, a system to generate aligned dental representation is provided. In some cases, the system comprises: one or more memories; and at least one processor each coupled to at least one of the one or more memories and configured to perform operations comprising: receiving an oral image that includes at least part of one tooth, at least part of two teeth, at least part of two teeth at the same arch; receiving perspective of the oral image; generating distance data of at least two marks on the oral image; generating a reference plane; and projecting the oral image based on the perspective and the distance data to the reference plane to generate aligned dental representation.

[0080] In one or more embodiments, the oral image is a two-dimension (2D) image, a frame from a video file, or a movie. In one or more embodiments, the oral image represents a capture of visual light wavelength by a camera.Attorney Docket No. 57364-725601

[0081] In one or more embodiments, the oral image is acquired using a mobile digital camera and a hollow tubular adapter, wherein one opening of the hollow tubular adapter is attached to the mobile digital camera and a second opening of the hollow tubular adapter contacts the user inner lips. The lips can lay on the outer of the second opening of the hollow tubular.

[0082] In one or more embodiments, the reference plane is parallel to the head sagittal plane.

[0083] In one or more embodiments, the reference plane intersects dentition midline.

[0084] In one or more embodiments, the reference plane is parallel to the head transverse plane.

[0085] In one or more embodiments, the reference plane intersects a virtual line crossing two opposite molars.

[0086] In one or more embodiments, the reference plane is parallel to the head coronal plane.

[0087] In one or more embodiments, the reference plane intersects a virtual line crossing two opposite molars.

[0088] In one or more embodiments, the generating the reference plane comprises selection of defined reference plane or user selection reference plane.

[0089] In one or more embodiments, the generating the distance data uses structure from motion, depth estimation neural network, or multi-view stereo algorithm.

[0090] In one or more embodiments, the aligned dental representation is used to generate a cylinder projection.

[0091] In one or more embodiments, the aligned dental representation is used to generate a three-dimensional representation.

[0092] In one or more embodiments, a non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations to generate aligned dental representation is provided. In some cases, the operations comprise: receiving an oral image that includes at least part of one tooth, at least part of two teeth, at least part of two teeth at the same arch; receiving perspective of the oral image; generating distance data of at least two marks on the oral image; generating a reference plane; and projecting the oral image based on the perspective and the distance data to the reference plane to generate aligned dental representation.Attorney Docket No. 57364-725601

[0093] In one or more embodiments, the oral image is a two-dimension (2D) image, a frame from a video file, or a movie. In one or more embodiments, the oral image represents a capture of visual light wavelength by a camera.

[0094] In one or more embodiments, the oral image is acquired using a mobile digital camera and a hollow tubular adapter, wherein one opening of the hollow tubular adapter is attached to the mobile digital camera and a second opening of the hollow tubular adapter contacts the user inner lips. The lips can lay on the outer of the second opening of the hollow tubular.

[0095] In one or more embodiments, the reference plane is parallel to the head sagittal plane.

[0096] In one or more embodiments, the reference plane intersects dentition midline.

[0097] In one or more embodiments, the reference plane is parallel to the head transverse plane.

[0098] In one or more embodiments, the reference plane intersects a virtual line crossing two opposite molars.

[0099] In one or more embodiments, the reference plane is parallel to the head coronal plane.

[0100] In one or more embodiments, the reference plane intersects a virtual line crossing two opposite molars.

[0101] In one or more embodiments, the generating the reference plane comprises selection of defined reference plane or user selection reference plane.

[0102] In one or more embodiments, the generating distance data uses structure from motion, depth estimation neural network, or multi-view stereo algorithm.

[0103] FIG. 4 shows a block diagram of a system 400 configured to generate a dentition representation, according to some embodiments. As shown in FIG. 4, system 400 can include one or more servers 402, a computing device 404 A, and a computing device 404B. Server(s) 402 and computing devices 404A and 404B may be communicatively coupled to each other via a network 406. Network 406 may comprise one or more networks such as local area networks (LANs), wide area networks (WANs), enterprise networks, the Internet, etc., and may include one or more of wired and / or wireless portions.

[0104] In an embodiment, server(s) 402 may form a network-accessible server set (e.g., a cloud-based environment or platform). Server(s) 402 may be accessible via network 406 (e.g., in a “cloud-based” embodiment) to build, deploy, and manage applications and services.Server(s) 402 may be co-located (e.g., housed in one or more nearby buildings with associatedAttorney Docket No. 57364-725601 components such as backup power supplies, redundant data communications, environmental controls, etc.) to form a datacenter, or may be arranged in other manners. Accordingly, in an embodiment, server(s) 402 may be a datacenter in a distributed collection of datacenters.

[0105] Server(s) 402 may be configured to execute one or more software applications (or “applications”) and / or services. Examples of such applications and / or services include dentition representation or aligned dental representation. It is further noted that dentition representation or aligned dental representation generator may be implemented on and executed by computing devices 404A and / or 404B in lieu of server(s) 402. Additional details regarding dentition representation or aligned dental representation generator 408

[0106] Computing device 404A may be utilized by a user, such as a patient or potential patient or by dentist or orthodontist or by dental clinic worker. As shown in FIG. 4, computing device 404A may comprise a display screen 412A and a camera 414A. Computing device 404A may also be configured to execute a capture application 416. Dental adapter 418A may be attached to camera 414A and enables the capturing of intraoral images and / or videos of the user. An example and possible uses of dental adapter 418A are described in U.S. Patent Application Serial No. 17 / 336,997, which is incorporated by reference herein. Dental adapter 418A may be utilized during execution of capture application 416. Capture application 416, in conjunction with dental adapter 418A, may be configured to enable a dental patient to perform a self-scan of the intraoral region of the dental patient. For example, capture application 416, during execution thereof, may activate camera 414A and may instruct instruction the patient (for example, via a user interface 420 A of capture application 416 displayed via a display screen 422A of computing device 404A) to place dental adapter 418A inside the user’s mouth. Capture application 416 may provide a command to camera 414A to initiate video capture and / or capture one or more images while dental adapter 418A is inside the user’s mouth. The user may move dental adapter 418A and camera 414A in different directions to capture different areas of the patient’s intraoral region. The captured video and / or images may be referred to as a dental scan. In an embodiment, capture application 416 may instruct camera 414A to capture video and / or images for a predetermined amount of time, for example, 15 seconds, 30 seconds, 1 minute, 2 minutes, etc. The dental scan may be provided to dentition representation or aligned dental representation generator 408, for example, via network 406. Capture application 416 may also be configured to determine and / or generate 3D dental data, for example, based on the dental scan utilizing photogrammetry -based techniques known to persons of ordinary skill in the art.Attomey Docket No. 57364-725601

[0107] Dentition representation or an aligned dental representation 408 may be configured to generate a dentition representation or aligned dental representation. For example, dentition representation or aligned dental representation 48 may receive oral image that includes at least part of one tooth, at least part of two teeth, at least part of two teeth at the same arch; generating distance data of at least two marks on the oral image; generating a reference plane; and projecting the oral image based on the perspective and the distance data to the reference plane to generate aligned dental representation.

[0108] A dentist may utilize computing device 404B to access and view the dentition representation or aligned dental representation. For example, computing device 404B may comprise an application (e.g., records application 424) that is configured to access the dentition representation or aligned dental representation from server(s) 402. Records application 424 may further include a user interface 420B (which is displayed via a display screen 422B of computing device 404B) that enables the dentist to view the dentition representation or aligned dental representation.

[0109] Computing device 404A may be located in a first location, and computing device 404B may be located in a second location that is different thant the first location. This enables the dentist to remotely monitor the patient. The remote monitoring may be performed by assessing an intraoral condition of the subject using the dentition representation or aligned dental representation from the subject when the subject is located remotely from the dentist or a dental office. In some cases, the remote monitoring may be performed in real-time such that a dentist is able to assess the dental condition when a subject uses computing device 404A to acquire one or more videos or images and / or a 3D model of one or more intraoral regions in the patient’s mouth.

[0110] Examples of computing devices 402A and 404B include, but are not limited to any type of stationary device, such as a desktop computer or PC (personal computer), or mobile computing device (such as a laptop computer, a notebook computer, a tablet computer, etc.).

[0111] In one or more embodiments, dentition representation or aligned dental representation generator 408 may also be configured to select the most optimal images from a dental scan. For example, dentition representation or aligned dental representation generator 408 may be configured to analyze extraoral video and / or images and determine optimal frames from the video and / or optimal images from the images. Optimal frames and / or images may comprise particular features of the intraoral regions, dentition representation or aligned dental representation generator 408 may utilize various object or image recognition-based techniques known to persons of ordinary skill in the art. Such object or image recognition-Attorney Docket No. 57364-725601 based techniques may be configured to detect various intraoral features corresponding to the clip start point, the clip end point, and / or other features (e.g., some tooth or part of the arch,) to select particular frames and / or images.

[0112] Dentition representation or aligned dental representation generator 408 can, for example, implement a method of machine learning framework to select the most optimal images from a dental scan. In order to transfer a given capability into a machine, so-called supervised machine-learning techniques can be employed. Broadly speaking, a training dataset containing input images together with requested output (the "ground-truth") is assembled. This dataset can be assembled from input images which should resemble the expected input images to the machine after deployment as much as possible. The requested output can be any desired output type, e.g image classification, segmentation mask, image augmentation, text etc. The requested output can be gathered by manual human labeling, or by fusing data with other means of data gathering. This dataset is then fed into a machinelearning framework which uses an optimization process to control internal parameters in a model architecture (e.g SVM, neural network, random forest etc.). The optimization process goal is to reduce the error between the machine's output and the ground-truth output for each image in the training dataset. Once the optimization process has reached the required accuracy on the training dataset the machine's performance can be tested on a similar test dataset to validate the performance in real-world settings.

[0113] FIG. 6 shows example images and representation associated with a potential patient, according to some embodiments.

[0114] An oral image 601 can be provided. A reference plane 603 can be generated from at least two marks on the image 602. Then, the oral image 604 based on the perspective and the distance data 605 can be projected to the reference plane to generate aligned dental representation 606. In this case, the dental representation is parallel to the transverse plane.

[0115] Various embodiments may be implemented, for example, using one or more well- known computer systems, such as computer system 700 shown in FIG. 7. One or more computer systems 700 may be used, for example, to implement any of the embodiments discussed herein, as well as combinations and sub-combinations thereof.

[0116] Computer system 700 may include one or more processors (also called central processing units, or CPUs), such as a processor 704. Processor 704 may be connected to a communication infrastructure or bus 706.Attorney Docket No. 57364-725601

[0117] Computer system 700 may also include user input / output device(s) 703, such as monitors, keyboards, pointing devices, etc., which may communicate with communication infrastructure 706 through user input / output interface(s) 702.

[0118] One or more of processors 704 may be a graphics processing unit (GPU). In an embodiment, a GPU may be a processor that is a specialized electronic circuit designed to process mathematically intensive applications. The GPU may have a parallel structure that is efficient for parallel processing of large blocks of data, such as mathematically intensive data common to computer graphics applications, images, videos, etc.

[0119] Computer system 700 may also include a main or primary memory 708, such as random access memory (RAM). Main memory 708 may include one or more levels of cache. Main memory 708 may have stored therein control logic (i.e., computer software) and / or data.

[0120] Computer system 700 may also include one or more secondary storage devices or memory 710. Secondary memory 710 may include, for example, a hard disk drive 712 and / or a removable storage device or drive 714. Removable storage drive 714 may be a floppy disk drive, a magnetic tape drive, a compact disk drive, an optical storage device, tape backup device, and / or any other storage device / drive.

[0121] Removable storage drive 714 may interact with a removable storage unit 718. Removable storage unit 718 may include a computer usable or readable storage device having stored thereon computer software (control logic) and / or data. Removable storage unit 718 may be a floppy disk, magnetic tape, compact disk, DVD, optical storage disk, and / any other computer data storage device. Removable storage drive 714 may read from and / or write to removable storage unit 718.

[0122] Secondary memory 710 may include other means, devices, components, instrumentalities or other approaches for allowing computer programs and / or other instructions and / or data to be accessed by computer system 700. Such means, devices, components, instrumentalities or other approaches may include, for example, a removable storage unit 722 and an interface 720. Examples of the removable storage unit 722 and the interface 720 may include a program cartridge and cartridge interface (such as that found in video game devices), a removable memory chip (such as an EPROM or PROM) and associated socket, a memory stick and USB port, a memory card and associated memory card slot, and / or any other removable storage unit and associated interface.

[0123] Computer system 700 may further include a communication or network interface 724. Communication interface 724 may enable computer system 700 to communicate and interact with any combination of external devices, external networks, external entities, etc.Attorney Docket No. 57364-725601(individually and collectively referenced by reference number 728). For example, communication interface 724 may allow computer system 700 to communicate with external or remote devices 728 over communications path 726, which may be wired and / or wireless (or a combination thereof), and which may include any combination of LANs, WANs, the Internet, etc. Control logic and / or data may be transmitted to and from computer system 700 via communication path 726.

[0124] Computer system 700 may also be any of a personal digital assistant (PDA), desktop workstation, laptop or notebook computer, netbook, tablet, smart phone, smart watch or other wearable, appliance, part of the Intemet-of-Things, and / or embedded system, to name a few non-limiting examples, or any combination thereof.

[0125] Computer system 700 may be a client or server, accessing or hosting any applications and / or data through any delivery paradigm, including but not limited to remote or distributed cloud computing solutions; local or on-premises software (“on-premise” cloudbased solutions); “as a service” models (e.g., content as a service (CaaS), digital content as a service (DCaaS), software as a service (SaaS), managed software as a service (MSaaS), platform as a service (PaaS), desktop as a service (DaaS), framework as a service (FaaS), backend as a service (BaaS), mobile backend as a service (MBaaS), infrastructure as a service (laaS), etc.); and / or a hybrid model including any combination of the foregoing examples or other services or delivery paradigms.

[0126] Any applicable data structures, file formats, and schemas in computer system 700 may be derived from standards including but not limited to JavaScript Object Notation (JSON), Extensible Markup Language (XML), Yet Another Markup Language (YAML), Extensible Hypertext Markup Language (XHTML), Wireless Markup Language (WML), Message Pack, XML User Interface Language (XUL), or any other functionally similar representations alone or in combination. Alternatively, proprietary data structures, formats or schemas may be used, either exclusively or in combination with known or open standards.

[0127] In some embodiments, a tangible, non-transitory apparatus or article of manufacture comprising a tangible, non-transitory computer useable or readable medium having control logic (software) stored thereon may also be referred to herein as a computer program product or program storage device. This includes, but is not limited to, computer system 700, main memory 708, secondary memory 710, and removable storage units 718 and 722, as well as tangible articles of manufacture embodying any combination of the foregoing. Such control logic, when executed by one or more data processing devices (such as computer system 700), may cause such data processing devices to operate as described herein.Attorney Docket No. 57364-725601

[0128] Based on the teachings contained in this disclosure, it will be apparent to persons skilled in the relevant art(s) how to make and use embodiments of this disclosure using data processing devices, computer systems and / or computer architectures other than that shown in FIG. 7. In particular, embodiments can operate with software, hardware, and / or operating system implementations other than those described herein.

[0129] It is to be appreciated that the Detailed Description section, and not any other section, is intended to be used to interpret the claims. Other sections can set forth one or more but not all exemplary embodiments as contemplated by the inventor(s), and thus, are not intended to limit this disclosure or the appended claims in any way.

[0130] While this disclosure describes exemplary embodiments for exemplary fields and applications, it should be understood that the disclosure is not limited thereto. Other embodiments and modifications thereto are possible, and are within the scope and spirit of this disclosure. For example, and without limiting the generality of this paragraph, embodiments are not limited to the software, hardware, firmware, and / or entities illustrated in the figures and / or described herein. Further, embodiments (whether or not explicitly described herein) have significant utility to fields and applications beyond the examples described herein.

[0131] Embodiments have been described herein with the aid of functional building blocks illustrating the implementation of specified functions and relationships thereof. The boundaries of these functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternate boundaries can be defined as long as the specified functions and relationships (or equivalents thereof) are appropriately performed. Also, alternative embodiments can perform functional blocks, steps, operations, methods, etc. using orderings different than those described herein.

[0132] References herein to “one embodiment,” “an embodiment,” “an example embodiment,” or similar phrases, indicate that the embodiment described can include a particular feature, structure, or characteristic, but every embodiment can not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it would be within the knowledge of persons skilled in the relevant art(s) to incorporate such feature, structure, or characteristic into other embodiments whether or not explicitly mentioned or described herein. Additionally, some embodiments can be described using the expression “coupled” and “connected” along with their derivatives. These terms are not necessarily intended as synonyms for each other. For example, some embodiments can be described using the termsAttorney Docket No. 57364-725601“connected” and / or “coupled” to indicate that two or more elements are in direct physical or electrical contact with each other. The term “coupled,” however, can also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other.

[0133] The breadth and scope of this disclosure should not be limited by any of the abovedescribed exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

Claims

Attorney Docket No. 57364-725601CLAIMSWHAT IS CLAIMED IS:

1. A method for generating an aligned dental representation of a subject , comprising: providing an oral image that includes at least part of one tooth; providing perspective of the oral image; generating distance data of at least two marks on the oral image; generating a reference plane; and projecting the oral image based on (i) the perspective of the oral image and (ii) the distance data on the reference plane to generate the aligned dental representation of the subject. The method of claim 1, wherein the oral image was acquired using a mobile digital camera and a hollow tubular adapter, wherein a first opening of the hollow tubular adapter is configured to attach to the mobile digital camera, and wherein a second opening of the hollow tubular adapter is configured to contact lips of the subject.3 The method of claim 2, wherein an outer surface of the second opening of the hollow tubular adapter is configured to contact an inner portion of the lips of the subject. The method of claim 1, wherein the reference plane is parallel to a sagittal plane of a head of the subject.5 The method of claim 4, wherein the reference plane intersects a midline of a dentition of the subject.6 The method of claim 1, wherein the reference plane is parallel to a transverse plane of a head of the subject.7 The method of claim 6, wherein the reference plane intersects a virtual line crossing two opposite molars of the subject.8 The method of claim 1, wherein the reference plane is parallel to a coronal plane of a head of the subject.9 The method of claim 8, wherein the reference plane intersects a virtual line crossing two opposite molars of the subject.10 The method of claim 1, wherein the generating the distance data uses structure from motion, depth estimation neural network, or multi-view stereo algorithm.Attorney Docket No. 57364-72560111. The method of claim 1, wherein the aligned dental representation is used to generate a cylindrical projection.

12. The method of claim 1, wherein the aligned dental representation is used to generate a three-dimensional representation.

13. A system, comprising: one or more memories; and at least one processor each coupled to at least one of the one or more memories and configured to perform operations, comprising: receiving an oral image that includes at least part of one tooth; receiving perspective of the oral image; generating distance data of at least two marks on the oral image; generating a reference plane; and projecting the oral image based on (i) the perspective of the oral image and (ii) the distance data on the reference plane to generate an aligned dental representation of a subject.

14. The system of claim 13, wherein the oral image was acquired using a mobile digital camera and a hollow tubular adapter, wherein a first opening of the hollow tubular adapter is configured to attach to the mobile digital camera, and wherein a second opening of the hollow tubular adapter is configured to lips of the subject.

15. The system of claim 14, wherein an outer surface of the second opening of the hollow tubular adapter is configured to contact an inner portion of the lips of the subject.

16. The system of claim 13, wherein the reference plane is parallel to a sagittal plane of a head of the subject.

17. The system of claim 16, wherein the reference plane intersects a midline of a dentition of the subject.

18. The system of claim 13, wherein the reference plane is parallel to a transverse plane of a head of the subject.

19. The system of claim 18, wherein the reference plane intersects a virtual line crossing two opposite molars of the subject.

20. The system of claim 13, wherein the reference plane is parallel to a coronal plane of a head of the subject.Attorney Docket No. 57364-72560121. The system of claim 20, wherein the reference plane intersects a virtual line crossing two opposite molars of the subject.

22. The system of claim 13, wherein the generating the distance data uses structure from motion, depth estimation neural network, or multi-view stereo algorithm.

23. The system of claim 13, wherein the aligned dental representation is used to generate a cylinder projection.

24. The system of claim 13, wherein the aligned dental representation is used to generate a three-dimensional representation.

25. A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations, comprising: receiving an oral image that includes at least part of one tooth; receiving perspective of the oral image; generating distance data of at least two marks on the oral image; generating a reference plane; and projecting the oral image based on (i) the perspective of the oral image and (ii) the distance data on the reference plane to generate an aligned dental representation of a subject.

26. The non-transitory computer-readable medium of claim 25, wherein the oral image was acquired using a mobile digital camera and a hollow tubular adapter, wherein a first opening of the hollow tubular adapter is configured to attach to the mobile digital camera, and wherein a second opening of the hollow tubular adapter is configured to contact lips of the subject.

27. The non-transitory computer-readable medium of claim 26, wherein an outer surface of the second opening of the hollow tubular adapter is configured to contact an inner portion of the lips of the subject.

28. The non-transitory computer-readable medium of claim 25, wherein the reference plane is parallel to a sagittal plane of a head of the subject.

29. The non-transitory computer-readable medium of claim 28, wherein the reference plane intersects a midline of a dentition of the subject.Attorney Docket No. 57364-72560130. The non-transitory computer-readable medium of claim 25, wherein the reference plane is parallel to a transverse plane of a head of the subject.

31. The non-transitory computer-readable medium of claim 30, wherein the reference plane intersects a virtual line crossing two opposite molars of the subject.

32. The non-transitory computer-readable medium of claim 25, wherein the reference plane is parallel to a coronal plane of a head of the subject.

33. The non-transitory computer-readable medium of claim 32, wherein the reference plane intersects a virtual line crossing two opposite molars of the subject.

34. The non-transitory computer-readable medium of claim 25, wherein the generating the distance data uses structure from motion, depth estimation neural network, or multi-view stereo algorithm.

35. The non-transitory computer-readable medium of claim 25, wherein the aligned dental representation is used to generate a cylinder projection.

36. The non-transitory computer-readable medium of claim 25, wherein the aligned dental representation is used to generate a three-dimensional representation.

Images