Systems and methods for intraoral scan registration

Abstract

An intraoral scanning system (102) is provided that includes a handheld intraoral scanning device (104) and one or more client devices (106). A handheld intraoral scanning device (104) captures a plurality of two-dimensional (2D) scan images (110) during a scanning session of a dental arch (506). A handheld intraoral scanning device (104) provides three-dimensional (3D) surface information (510) based on a plurality of two-dimensional scanned images (110). A handheld intraoral scanning device (104) establishes a connection to one or more wireless full duplex communication channels. The one or more client devices (106) establish a connection to one of the one or more wireless full duplex communication channels by forwarding the identification number to the handheld intraoral scanning device (104). One or more client devices (106) receive the 3D surface information (510) and present the 3D surface information (510) as an interactive 3D graphical representation (112).

Description

Technical Field

[0001] Exemplary embodiments of the present invention generally relate to intraoral scan registration, and more particularly, to an intraoral scanning system and method for intraoral scan registration based on edge computing. Background Art

[0002] Typically, an intraoral scanner is available to users (e.g., dentists in the dental industry). For example, an intraoral scan of a patient's dental arch captured by the intraoral scanner can be used to create a digital impression of the patient's oral cavity. The intraoral scanner includes a light source projected onto the dental arch to capture the intraoral scan. The intraoral scan is then processed to generate three-dimensional (3D) information (e.g., a 3D model) of the dental arch, and the 3D model can be displayed on a screen for examination by the dentist.

[0003] It is worth noting that real-time feedback of the captured intraoral scan is important for post-processing of the 3D model. Traditionally, real-time feedback is collected by using real-time reconstruction of the 3D model based on incoming sub-scans (e.g., intraoral scans) from the intraoral scanner on a computing device with powerful processing capabilities (e.g., Power PC). If the registration of the 3D information (e.g., point cloud) is lost, the dentist can be notified through a user interface (UI) (e.g., a UI of the computing device). The dentist may then need to capture the intraoral scan again before completing the scanning session of the dental arch to compensate for the lost 3D information.

[0004] A conventional pipeline for generating 3D models using intraoral scans can include capturing sub-scan data and transmitting the captured sub-scan data to a computing device using a dedicated network. The transmitted sub-scan data can be separated into texture and magnitude images on the computing device. Furthermore, scanner calibration parameters and filtering are applied to the texture and magnitude images to generate point cloud data on the computing device. Furthermore, point cloud registration between the sub-scans is performed on the computing device to find the spatial transformation between each sub-scan, and the individual point clouds are stitched together to reconstruct the cumulative 3D model. This 3D model can be rendered on the computing device.

[0005] Traditional 3D model generation methods have several drawbacks. For example, the process requires external infrastructure to transmit subscan data over a dedicated network. Furthermore, computing equipment is required to align multiple initial scans to generate 3D information. Given the large volume of subscan data that needs to be sent over a dedicated network and displayed to the dentist in real time, this can place significant demands on the dental clinic's dedicated network infrastructure in terms of throughput and latency. Furthermore, if the dedicated network infrastructure fails to meet these requirements, subscan data may be lost or delayed en route, complicating interaction between the dentist and the intraoral scanner. Furthermore, the number of intraoral scanners that can be simultaneously connected to the dedicated network in a dental clinic may be limited. This limitation is particularly significant if some 3D model processing needs to be performed on a cloud server. Typically, a dental clinic's treatment rooms may not have the hardware requirements for intraoral scanning. Therefore, the clinic may require additional computing equipment, or multiple treatment rooms may need to rely on shared computing equipment. Furthermore, traditional methods allow a single computing device to process a single intraoral scanner at a time, making parallel processing difficult. Therefore, improved intraoral scanning systems and methods are needed to overcome these drawbacks. Summary of the Invention

[0006] The present invention provides an intraoral scanning system, method and computer programmable product for performing intraoral scan registration using a handheld intraoral scanning device and presenting three-dimensional (3D) surface information on one or more client devices.

[0007] In one aspect, the present invention discloses an intraoral scanning system. The intraoral scanning system includes a handheld intraoral scanning device and one or more client devices. The handheld intraoral scanning device can be configured to capture multiple two-dimensional (2D) scan images during a dental arch scanning session. The handheld intraoral scanning device can also provide 3D surface information based on the multiple 2D scan images captured during the scanning session.

[0008] The 3D surface information may be provided in real time, such that the display unit is configured to display the 3D surface information in real time, and / or such that the 3D surface information is wirelessly transmitted in real time.

[0009] The handheld intraoral scanning device may further include a web server interface configured to communicate via a web network and establish a connection to the one or more wireless full-duplex communication channels. Furthermore, each of the one or more client devices is configured to establish a connection to one of the one or more wireless full-duplex communication channels by forwarding an identification number to the handheld intraoral scanning device via the web network. Each of the one or more client devices is further configured to receive 3D surface information via the one of the one or more wireless full-duplex communication channels and present the 3D surface information as an interactive 3D graphical representation compatible with a web browser.

[0010] Rendering the 3D surface information as an interactive 3D graphical representation may be performed in real time, such that the display unit may be configured to display the rendered 3D surface information in real time.

[0011] Thus, the intraoral scanning system may be capable of processing multiple two-dimensional scan images to provide or generate three-dimensional surface information in a handheld intraoral scanning device, thereby eliminating the use of additional infrastructure (e.g., computing devices). The intraoral scanning system may also enable one or more client devices to communicate directly with the handheld intraoral scanning device to receive the three-dimensional surface information and present an interactive three-dimensional graphical representation for user access.

[0012] In some embodiments, the web server interface and one or more client devices can be connected to a public web network. The intraoral scanning system enables the web server interface of the handheld intraoral scanning device and one or more client devices to connect to one of the one or more wireless full-duplex communication channels (or a common channel).

[0013] In some embodiments, multiple of the one or more client devices can be configured to receive the three-dimensional surface information via one of the one or more wireless full-duplex communication channels. Multiple of the one or more client devices can also present the 3D surface information as an interactive 3D graphical representation compatible with a web browser. Thus, the intraoral scanning system enables multiple users to access the interactive 3D graphical representation simultaneously.

[0014] In some embodiments, the one or more client devices may be at least one of a display unit, a tablet computer, or a smartphone.The one or more client devices include the capability to present an interactive 3D graphical representation to a user.

[0015] In some embodiments, one or more client devices may be computers. The computer may be, for example, a computing device with powerful processing capabilities.

[0016] In some embodiments, the bandwidth of one of the one or more wireless full-duplex communication channels is monitored by a monitoring unit of the intraoral scanning system (102). When the bandwidth is below a minimum bandwidth, the handheld intraoral scanning device can be configured to downsample the 3D surface information for transmission via one of the one or more wireless full-duplex communication channels. Thus, the intraoral scanning system can transmit the 3D surface information in real time or near real time without delay.

[0017] In some embodiments, the bandwidth of one of the one or more wireless full-duplex communication channels is monitored by a monitoring unit of the intraoral scanning system. The handheld intraoral scanning device may further include a temporary storage unit configured to store the 3D surface information when the bandwidth of one of the one or more wireless full-duplex communication channels is determined to be less than a minimum bandwidth. When the bandwidth is determined to be greater than or equal to the minimum bandwidth, the temporary storage unit may transmit the stored 3D surface information. If real-time transmission of the 3D surface information is not required, the intraoral scanning system may transmit the stored 3D surface information when the bandwidth is determined to be greater than or equal to the minimum bandwidth.

[0018] In some embodiments, when the bandwidth of one of the one or more wireless full-duplex communication channels is below a minimum bandwidth for a period longer than a maximum period, the handheld intraoral scanning device can be configured to compress the 3D surface information and store it in a memory unit of the handheld intraoral scanning device. The intraoral scanning system can compress the 3D surface information to conserve storage space in the memory unit. When the bandwidth is determined to be above or equal to the minimum bandwidth, the user can later access the stored 3D surface information.

[0019] In some embodiments, the monitoring unit can be configured to determine when connection to one of the one or more wireless full-duplex communication channels is lost. The handheld intraoral scanning device can be configured to compress the 3D surface information and store it in a memory unit of the handheld intraoral scanning device based on the determination of connection loss. The intraoral scanning system can store the 3D surface information when connection is lost to securely store the provided 3D surface information based on the plurality of captured 2D scan images.

[0020] In some embodiments, the handheld intraoral scanning device can be configured to transmit the 3D surface information stored in the memory unit via the wireless communication interface of the intraoral scanning system at the end of the scanning session. The intraoral scanning system can store the 3D surface information in the memory unit of the handheld intraoral scanning device, and the user can use the information at any time after the scanning session ends.

[0021] In some embodiments, the handheld intraoral scanning device further includes a monitoring unit configured to transmit a status input based on at least one of: a determination that the bandwidth is less than a minimum bandwidth, or a determination that the connection to one of the one or more wireless full-duplex communication channels has been lost. The handheld intraoral scanning device further includes a scan feedback unit configured to receive the status input from the monitoring unit and, concurrently with receiving the status input, provide a scan feedback signal to a user of the handheld intraoral scanning device. The scan feedback signal can be configured to guide the user to areas of the dental arch where scan quality during the scanning session is low and 3D surface information cannot be provided. The feedback signal provided by the intraoral scanning system can be used to guide the user to capture a larger number of multiple two-dimensional scan images, thereby accurately generating three-dimensional surface information.

[0022] In some embodiments, the scanning feedback signal may include an acoustic feedback signal configured to guide the user towards an area of ​​the dental arch. The acoustic feedback signal may be, for example, a sound that the user may utilize.

[0023] In some embodiments, the scanning feedback signal may include at least one of tactile feedback or light emitted by a plurality of light-emitting diodes of the handheld intraoral scanning device. The intraoral scanning system may provide tactile feedback and visual feedback in the form of light, which the user may utilize to capture a greater number of two-dimensional scan images.

[0024] In some embodiments, the handheld intraoral scanning device may include a vibrator configured to provide tactile feedback. An increase in vibration may indicate an increase in the distance between the area of ​​the dental arch and the handheld intraoral scanning device. A decrease in vibration may indicate a decrease in the distance between the area of ​​the dental arch and the handheld intraoral scanning device. Thus, the intraoral scanning system can determine the area of ​​the dental arch from which a greater number of multiple 2D scan images are required.

[0025] In some embodiments, the plurality of LEDs are divided into a left group of LEDs and a right group of LEDs. The left and right groups of LEDs are configured to emit flashes when the handheld intraoral scanning device is positioned to the right or left side of the dental arch area, respectively. Arranging the plurality of LEDs in this manner enables efficient use of the handheld intraoral scanning device without repeatedly rotating the handheld intraoral scanning device during scanning.

[0026] In some embodiments, the handheld intraoral scanning device can be configured to broadcast the 3D surface information to multiple client devices among the one or more client devices connected to the handheld intraoral scanning device via one of the one or more wireless full-duplex communication channels. Thus, the intraoral scanning system can simultaneously utilize multiple client devices among the one or more client devices.

[0027] In some embodiments, the handheld intraoral scanning device can be configured to generate a 3D model representation of the dental arch by combining multiple 3D surface information provided by the handheld intraoral scanning device. The handheld intraoral scanning device can be configured to transmit the 3D model via one of one or more wireless full-duplex communication channels. Thus, the intraoral scanning system can generate the 3D model within the handheld intraoral scanning device without requiring external infrastructure, such as a computing device.

[0028] On the other hand, the present disclosure provides a method for intraoral scan registration. The method may include capturing multiple two-dimensional (2D) scan images during a scanning session of a dental arch by a handheld intraoral scanning device. The method may also include providing three-dimensional (3D) surface information based on the multiple 2D scan images captured during the scanning session by the handheld intraoral scanning device. The method may also include: establishing a connection to one of the one or more wireless full-duplex communication channels by forwarding an identification number to the handheld intraoral scanning device via a web network by one or more client devices. The method may also include: receiving the 3D surface information via one of the one or more wireless full-duplex communication channels by one or more client devices. The method may also include presenting the 3D surface information as an interactive 3D graphical representation compatible with a web browser on the one or more client devices.

[0029] In yet another aspect, the present disclosure provides an intraoral scanning system for generating a three-dimensional (3D) model and presenting an interactive 3D graphical representation based on the 3D model. The intraoral scanning system may include a handheld intraoral scanning device configured to capture a first plurality of two-dimensional (2D) scanned images and a second plurality of two-dimensional scanned images containing surface information of a patient's dental arch during a first time frame and a second time window, respectively. The first time window is before the second time window. The handheld intraoral scanning device may also be configured to process the first plurality of two-dimensional scanned images and the second plurality of two-dimensional scanned images into first 3D surface information and second 3D surface information, respectively. The handheld intraoral scanning device may also be configured to generate a first 3D scan patch and a second 3D scan patch by using calibration data stored on a memory unit, by transforming the first 3D surface information into a first real-world 3D coordinate and a first texture information, and by transforming the second 3D surface information into a second real-world 3D coordinate and a second texture information. The handheld intraoral scanning device may also be configured to register the second 3D scan patch to at least the first 3D scan patch by locating corresponding data points between the first and second 3D scan patches. The handheld intraoral scanning device may also be configured to fuse the first and second 3D scan patches together to form a 3D model. The handheld intraoral scanning device may also be configured to store the first and second texture information along with the formed 3D model. The handheld intraoral scanning device includes a web server interface configured to communicate via a web network and establish a connection to one or more wireless full-duplex communication channels. The intraoral scanning system may also include one or more client devices. Each of the one or more client devices may be configured to establish a connection to one of the one or more wireless full-duplex communication channels by forwarding an identification number to the handheld intraoral scanning device via the web network. Each of the one or more client devices may also be configured to receive the 3D model via the one or more wireless full-duplex communication channels. Each of the one or more client devices may also be configured to present the 3D model as an interactive 3D graphical representation compatible with a web browser.

[0030] In another aspect, the present disclosure provides a method for generating a three-dimensional (3D) model and presenting an interactive 3D graphical representation based on the 3D model. The method may include capturing, using a handheld intraoral scanning device, a first plurality of two-dimensional (2D) scanned images and a second plurality of two-dimensional scanned images containing surface information of a patient's dental arch during a first time window and a second time window, respectively. The first time window is before the second time window. The method may also include processing, using the handheld intraoral scanning device, the first plurality of two-dimensional scanned images and the second plurality of two-dimensional scanned images into first three-dimensional (3D) surface information and second three-dimensional surface information, respectively. The method may also include generating, using calibration data, a first 3D scan patch and a second 3D scan patch by transforming the first 3D surface information into first real-world 3D coordinates and first texture information, and transforming the second 3D surface information into second real-world 3D coordinates and second texture information, using the handheld intraoral scanning device. The method may also include registering, using the handheld intraoral scanning device, the second 3D scan patch to at least the first 3D scan patch by locating corresponding data points between the first 3D scan patch and the second 3D scan patch. The method may also include: fusing the first 3D scanned surface and the second 3D scanned surface together to form a 3D model through a handheld intraoral scanning device. The method may also include: storing the first texture information and the second texture information together with the formed 3D model through the handheld intraoral scanning device. The method may also include: establishing a connection to one of the one or more wireless full-duplex communication channels by forwarding the identification number to the handheld intraoral scanning device via a web network through one or more client devices. The method may also include: receiving the 3D model through the one of the one or more wireless full-duplex communication channels through the one or more client devices. The method may also include presenting the 3D model as an interactive 3D graphical representation compatible with a web browser on the one or more client devices.

[0031] In yet another aspect, the present disclosure provides a computer programmable product comprising a non-transitory computer-readable medium having computer-executable instructions stored thereon, the computer-executable instructions causing the processing circuit to perform operations when executed by a processing circuit. The operations may include: capturing a plurality of two-dimensional (2D) scan images of a dental arch during a scanning session by a handheld intraoral scanning device. The operations may also include: providing three-dimensional (3D) surface information based on the plurality of two-dimensional scan images captured during the scanning session by the handheld intraoral scanning device. The operations may also include: establishing, by one or more client devices, a connection to one of the one or more wireless full-duplex communication channels by forwarding an identification number to the handheld intraoral scanning device via a web network. The operations may also include: receiving, by the one or more client devices, the 3D surface information via the one of the one or more wireless full-duplex communication channels. The operations may also include presenting the 3D surface information as an interactive 3D graphical representation compatible with a web browser on the one or more client devices.

[0032] The foregoing summary is illustrative only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.

[0033] Effects of the Invention

[0034] According to the present disclosure, an intraoral scanning system, method and computer programmable product are provided.One of the objects of the present disclosure is to provide enhanced processing capabilities in a handheld intraoral scanning device to provide three-dimensional (3D) surface information.

[0035] A conventional system may include an intraoral scanner for capturing a two-dimensional (2D) intraoral scan of a patient's dental arch. The conventional intraoral scanner may have limited processing power and may only be used to capture a two-dimensional intraoral scan of the dental arch. In order to generate three-dimensional information of the dental arch, the conventional intraoral scanner may need to rely on external infrastructure, such as a computing device (e.g., a Power PC). In order to utilize the computing device, the conventional intraoral scanner may need to be connected to the computing device via a network. In some cases, the computing device may be located in a different location from the intraoral scanner. For example, the computing device may be bulky and may need to be placed in a room different from the dental clinic treatment room. In this case, the user (e.g., a dentist) may need to access the computing device in another room after capturing the two-dimensional intraoral scan. The user may need to use a network to connect the conventional intraoral scanner to the computing device. The computing device can generate three-dimensional information of the dental arch based on the two-dimensional intraoral scan and present the three-dimensional information on a display screen. Therefore, using a conventional intraoral scan registration system may be time-consuming and difficult for the user.

[0036] On the other hand, the intraoral scanning system of the present disclosure includes a handheld intraoral scanning device. The intraoral scanning system can provide enhanced processing capabilities in the handheld intraoral scanning device. The handheld intraoral scanning device can include a web server interface configured to communicate via a web network and establish a connection to a communication network. The handheld intraoral scanning device of the present disclosure can capture multiple two-dimensional scan images of a patient's dental arch. Based on the multiple captured two-dimensional scan images, the handheld intraoral scanning device can provide three-dimensional surface information of the dental arch. Therefore, the intraoral scanning system of the present disclosure can eliminate the need for external infrastructure (e.g., a computing system) to generate three-dimensional surface information. In addition, the intraoral scanning system of the present disclosure includes one or more client devices that can be communicatively connected to the handheld intraoral scanning device via a communication network. For example, the one or more client devices can be smartphones, tablet computers, or laptop computers, which can be retained in the same room (e.g., a treatment room) as the handheld intraoral scanning device. The one or more client devices can receive the three-dimensional surface information and present an interactive three-dimensional graphical representation based on the three-dimensional surface information. Therefore, the intraoral scanning system further eliminates the need to switch between different rooms to access the three-dimensional surface information. Therefore, the intraoral scanning system can provide a user-friendly and time-efficient process for intraoral scan registration.

[0037] Furthermore, in conventional systems, when the network infrastructure fails to meet infrastructure requirements, scan data (e.g., two-dimensional scans) may be lost or delayed during transmission to the computing device. Furthermore, the number of intraoral scanners that can be connected to the network simultaneously in a dental clinic may be limited. Furthermore, conventional systems allow a single computing device to process a single intraoral scanner at a time, making parallel processing difficult. On the other hand, the intraoral scanning system of the present disclosure is capable of monitoring the bandwidth of the communication network. If the bandwidth is less than the required bandwidth, the intraoral scanning system of the present disclosure can downsample the three-dimensional surface information before transmitting it to one or more client devices. Furthermore, after the dental arch scanning session is completed, the intraoral scanning system is capable of storing the three-dimensional surface information in a memory unit of the handheld intraoral scanning device. Thus, the intraoral scanning system of the present disclosure eliminates the problem of scan data loss. The intraoral scanning system can also enable the handheld intraoral scanning device to simultaneously broadcast 3D surface information to multiple client devices among one or more client devices connected to the handheld intraoral scanning device over the same network. Thus, multiple users can access the 3D surface information simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The present disclosure is illustrated by way of example and not limitation in the figures of the accompanying drawings in which like references refer to like elements and in which:

[0039] Figure 1is a diagram of a network environment of an intraoral scanning system for oral scan registration according to an exemplary embodiment;

[0040] Figure 2 A block diagram illustrating a handheld intraoral scanning device according to an exemplary embodiment is shown;

[0041] Figure 3 A block diagram illustrating one or more client devices according to an exemplary embodiment is shown;

[0042] Figure 4 is a schematic diagram illustrating an environment in which a handheld intraoral scanner communicates with one or more client devices according to an exemplary embodiment;

[0043] Figure 5 A schematic diagram illustrating capturing a plurality of two-dimensional (2D) scan images to generate three-dimensional (3D) surface information according to an exemplary embodiment;

[0044] Figure 6 2 is a schematic diagram illustrating transmitting 3D surface information and presenting an interactive 3D graphic representation according to an exemplary embodiment;

[0045] Figure 7 A sequence diagram depicting rendering of an interactive 3D graphical representation based on transmitted 3D surface information according to an exemplary embodiment;

[0046] Figure 8 shows an example flow chart of different scenarios involving transmitting 3D surface information to one or more client devices according to an example embodiment;

[0047] Figure 9 An example flow chart of generating a 3D model based on 3D surface information according to another exemplary embodiment is shown;

[0048] Figure 10 A sequence diagram depicting rendering of interactive 3D graphics based on a transported 3D model according to an exemplary embodiment;

[0049] Figure 11 is a schematic diagram depicting an exemplary environment for capturing multiple 2D scan images and presenting interactive 3D graphical representations in real time according to an exemplary embodiment; and

[0050] Figure 12 Illustration of an intermediate processing unit in communication with a handheld intraoral scanner and one or more client devices. DETAILED DESCRIPTION

[0051] In the following description, for ease of explanation, numerous specific details are set forth to provide a comprehensive understanding of the present disclosure. However, it will be appreciated by those skilled in the art that the present disclosure may be practiced without these specific details. In other cases, systems and methods are shown in block diagram form only to avoid obscuring the present disclosure.

[0052] References in this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic associated with the embodiment is included in at least one embodiment of the present disclosure. The phrase "in one embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment, nor does it necessarily refer to separate or alternative embodiments that are mutually exclusive of other embodiments. Furthermore, the terms "one" and "an" herein do not imply a limitation on quantity, but rather mean that there is at least one of the referenced item. Furthermore, various features are described that some embodiments may exhibit that other embodiments may not exhibit. Similarly, various requirements are described herein that may be requirements for some embodiments but may not be requirements for other embodiments.

[0053] Some embodiments of the present disclosure will be described more fully below with reference to the accompanying drawings, which illustrate some, but not all, embodiments of the present disclosure. Indeed, the various embodiments of the present disclosure may be embodied in a variety of different forms and should not be construed as limited to the embodiments described herein; rather, these embodiments are provided so that the present disclosure can satisfy applicable legal requirements. The same reference numerals refer to the same elements throughout. As used herein, the terms "data," "content," "information," and similar terms may be used interchangeably to refer to data that can be transmitted, received, and / or stored in accordance with embodiments of the present disclosure. Additionally, the terms "processor," "controller," and "processing circuitry," and similar terms may be used interchangeably to refer to a processor that can process information in accordance with embodiments of the present disclosure. Additionally, terms such as "electronic device," "electronic device," and "device" may be used interchangeably to refer to an electronic device monitored by a system in accordance with embodiments of the present disclosure. Accordingly, the use of any such terms should not be construed as limiting the spirit and scope of the embodiments of the present disclosure.

[0054] The embodiments described herein are for illustrative purposes only and are subject to numerous variations. It should be understood that various omissions and equivalent substitutions may be made, depending on the circumstances, but are intended to encompass applications or implementations without departing from the spirit or scope of this disclosure. Furthermore, it should be understood that the wording and terminology used herein are for descriptive purposes only and should not be construed as limiting. Any headings used in this specification are for convenience only and have no legal or limiting effect.

[0055] The terms "for example," "for instance," and "such as," and the verbs "comprise," "have," "include," and their other verb forms, used in this specification and claims, when used in conjunction with a list of one or more components or other items, are to be understood as open-ended, meaning that the list should not be construed to exclude additional components or items. Other terms are to be interpreted using their broadest reasonable meaning unless they are used in a context that requires a different interpretation.

[0056] The present invention provides an intraoral scanning system, method, and computer programmable product for performing intraoral scan registration using a handheld intraoral scanning device and presenting three-dimensional (3D) surface information on one or more client devices.

[0057] For example, the following reference Figure 1 An exemplary network environment for an intraoral scanning system for oral scan registration is provided.

[0058] Figure 1 1 is a diagram of a network environment 100 of an intraoral scanning system 102 for oral scan registration according to an exemplary embodiment. The intraoral scanning system 102 may include a handheld intraoral scanning device 104 and one or more client devices 106. The network environment 100 may also include a communication channel 108 that may be configured to communicatively couple the handheld intraoral scanning device 104 and the one or more client devices 106. The network environment 100 may also include a plurality of two-dimensional (2D) scans 110 and an interactive three-dimensional (3D) graphical representation 112. Furthermore, one or more components may be rearranged, modified, added, and / or deleted without departing from the scope of the present disclosure.

[0059] The intraoral scanning system 102 can be used for registration of intraoral scans. The intraoral scanning system 102 can include multiple devices, such as a handheld intraoral scanning device 104 and one or more client devices 106, which can communicate with each other to register intraoral scans.

[0060] The handheld intraoral scanning device 104 may include enhanced processing capabilities that may be required to process multiple 2D scan images 110. The handheld intraoral scanning device 104 may be configured to capture multiple 2D scan images 110 during a dental arch scanning session. The multiple 2D scan images 110 may be images of a user's (e.g., a patient's) dental arch. The multiple 2D scan images 110 may include images of the patient's dental arch from various angles. A user (e.g., a dentist) may use the handheld intraoral scanning device 104 to capture multiple 2D scan images 110 during a dental arch scanning session.

[0061] Based on the captured plurality of 2D scan images 110, the handheld intraoral scanning device 104 can be configured to provide (or generate) 3D surface information. The 3D surface information can be a digital representation of the patient's dental arch depicted in 3D space. The 3D surface information can include, for example, 3D point cloud data corresponding to the plurality of 2D scan images 110. The 3D point cloud data can correspond to 3D real-world coordinates.

[0062] The handheld intraoral scanning device 104 may include a web server interface that may be configured to communicate over a web network and establish a connection to the communication channel 108. The handheld intraoral scanning device 104 may be configured to execute the web server interface to provide 3D surface information based on the 2D scan image 110. The handheld intraoral scanning device 104 may also include a processing unit, a memory unit, a communication interface, and additional components. The processing unit, the memory unit, the communication interface, and the additional components may be communicatively coupled to each other. Details of the components of the handheld intraoral scanning device 104 are further described in Figure 2 Given in.

[0063] The one or more client devices 106 may include processing capabilities required to process the 3D surface information. The one or more client devices 106 may be configured to establish a connection to one of the communication channels 108. The one or more client devices 106 may also receive the 3D surface information from the handheld intraoral scanning device 104. The one or more client devices 106 may also present the 3D surface information as an interactive 3D graphical representation 112.

[0064] The interactive 3D graphical representation 112 can be presented on the display units of one or more client devices 106. A user (e.g., a dentist) can view the interactive 3D graphical representation 112 on the display units of one or more client devices 106. The user can modify the view of the interactive 3D graphical representation 112 according to their preferences. For example, the viewing angle of the interactive 3D graphical representation 112 can be changed, or the interactive 3D graphical representation 112 can be zoomed in or out, according to the user's preferences. The interactive 3D graphical representation 112 can be presented independently on one or more client devices 106. Therefore, the interactive 3D graphical representation 112 can be independently accessed by multiple users of the corresponding one or more client devices 106.

[0065] The one or more client devices 106 can be any user-accessible device, such as a display unit, a mobile phone, a smartphone, a tablet computer, a computer, an artificial reality (XR) device, etc. In some examples, the display unit can be part of the one or more client devices 106. The display unit of the one or more client devices 106 can be a touch screen display. The one or more client devices 106 can include a processing unit, a memory unit, and a communication interface. The processor, memory, and communication interface can be communicatively coupled to each other. Additional, different, or fewer components can be provided. Moreover, one or more components can be rearranged, changed, added, and / or removed without departing from the scope of the present disclosure. For example, Figure 3 Details of the components of one or more client devices 106 are further provided in.

[0066] The communication channel 108 can be any combination of a wired, wireless, or wired and wireless communication network, such as cellular, wireless fidelity (Wi-Fi), the Internet, a local area network, etc. Depending on the embodiment, the communication channel 108 can be one or more wireless full-duplex communication channels. In one embodiment, the communication channel 108 can include one or more networks, such as a data network, a wireless network, a telephone network, or any combination thereof. It is contemplated that the data network can be any local area network (LAN), metropolitan area network (MAN), wide area network (WAN), a public data network (e.g., the Internet), a short-range wireless network, or any other suitable packet-switched network, such as a commercially owned proprietary packet-switched network (e.g., a proprietary cable or fiber optic network, etc.), or any combination thereof. In addition, the wireless network can be, for example, a cellular network and can employ various technologies, including Enhanced Data Rates for Global Evolution (EDGE), General Packet Radio Service (GPRS), Global System for Mobile Communications (GSM), Internet Protocol Multimedia Subsystem (IMS), Universal Mobile Telecommunications System (UMTS), and the like, as well as any other suitable wireless medium, such as Worldwide Interoperability for Microwave Access (WiMAX), Long Term Evolution (LTE) networks (e.g., LTE-Advanced Pro), 5G New Radio networks, ITU-IMT 2020 networks, Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Wireless Fidelity (Wi-Fi), Wireless LAN (WLAN), Bluetooth, Internet Protocol (IP) data delivery, satellite, Mobile Ad Hoc Networks (MANETs), and the like, or any combination thereof. The handheld intraoral scanning device 104 can be configured to communicate with one or more client devices 106 via a communication channel 108.

[0067] In operation, the user may require dental treatment. In this case, the dentist may use the intraoral scanning system 102 to provide the dental treatment to the user. In an embodiment, the user may be present at a dental clinic. In this case, the intraoral scanning system 102 may be used in the treatment room of the dental clinic. In another embodiment, the user may request a home visit for dental treatment. In this case, the intraoral scanning system 102 may be used at the user's home. To begin the dental treatment, the dentist may use a handheld intraoral scanning device 104 to capture a plurality of 2D scan images 110 of the user's dental arch. Based on the captured plurality of 2D scan images 110, the handheld intraoral scanning device 104 may provide 3D surface information. For example, in Figure 5 Details of capturing multiple 2D scan images 110 and providing 3D surface information are further provided in .

[0068] Furthermore, after capturing the plurality of 2D scan images 110, the dentist may need to view the human-readable 3D data of the plurality of 2D scan images 110 as part of the dental treatment. In order to view the human-readable 3D data of the plurality of 2D scan images 110, the handheld intraoral scanning device 104 and the one or more client devices 106 may need to be connected to a common communication channel in the communication channel 108. Thus, the web server interface of the handheld intraoral scanning device 104 may communicate over a web network to establish a connection with the one or more wireless full-duplex communication channels. The one or more client devices 106 may also establish a connection with one of the one or more wireless full-duplex communication channels. In order to establish the connection, the one or more client devices 106 may forward an identification number to the handheld intraoral scanning device 104 over the web network. The connection details of the one or more client devices 106 and one of the one or more wireless full-duplex communication channels are, for example, Figure 6 Further provided in.

[0069] After the handheld intraoral scanning device 104 and one or more client devices 106 are connected via a common communication channel, the one or more client devices 106 can receive the 3D surface information. For example, a dentist can use a tablet computer as one of the client devices to receive the 3D surface information. The tablet computer can be configured to present the 3D surface information as an interactive 3D graphical representation 112 that is compatible with the tablet computer's web browser. The interactive 3D graphical representation 112 can be used by the dentist as human-readable 3D data of the multiple 2D scan images 110. For example, the interactive 3D graphical representation 112 can be modified by using gestures as input to the tablet computer by the dentist. For example Figure 6 Details of presenting the 3D surface information as an interactive 3D graphical representation 112 are further provided.

[0070] Figure 2A block diagram 200 of a handheld intraoral scanning device 104 is shown according to an exemplary embodiment. Figure 1 The element pair Figure 2 The handheld intraoral scanning device 104 may include at least one processing unit (hereinafter also referred to as “processing unit 202 ”), a memory unit 204 , a web server interface 206 , a monitoring unit 208 , a temporary storage unit 210 , a scan feedback unit 212 , an input / output (I / O) unit 214 , and a communication interface 216 .

[0071] Processing unit 202 can be implemented in a variety of different ways. For example, processing unit 202 can be implemented as one or more of a variety of hardware processing devices, such as a coprocessor, a microprocessor, a controller, a digital signal processor (DSP), a processing element with or without an attached DSP, or various other processing circuits, including integrated circuits, such as, for example, an ASIC (application-specific integrated circuit), an FPGA (field-programmable gate array), a microcontroller unit (MCU), a hardware accelerator, a dedicated computer chip, etc. In an embodiment, processing unit 202 can be implemented as a high-performance microprocessor having a series of system-on-chips (SoCs) including relatively powerful and power-efficient graphics processing units (GPUs) and central processing units (CPUs) in a small form factor. For example, the form factor of processing unit 202 can be 70 millimeters (mm) x 45 mm. Therefore, in some embodiments, processing unit 202 can include one or more processing cores configured for independent execution. A multi-core processor can enable multi-processing within a single physical package. In addition, or alternatively, processing unit 202 can include one or more processors configured in coordination via a bus to enable independent execution of instructions, pipelining, and / or multi-threaded processing.

[0072] In some embodiments, processing unit 202 can be configured to capture multiple 2D scan images 110 of a dental arch of a user (e.g., a patient requiring dental treatment) during a scanning session. Multiple 2D scan images 110 can include images of the patient's dental arch from various angles. Based on multiple 2D scan images 110, processing unit 202 can provide 3D surface information. For example, the 3D surface information may include 3D point cloud data corresponding to the multiple 2D scan images 110.

[0073] In addition, or alternatively, the processing unit 202 may include one or more processors capable of handling a large workload and operations to provide support for big data analysis. In an exemplary embodiment, the processing unit 202 may communicate with the memory unit 204 via a bus to transfer information between components of the handheld intraoral scanning device 104.

[0074] The memory unit 204 can be non-transitory and can, for example, include one or more volatile and / or non-volatile memories. In other words, for example, the memory unit 204 can be an electronic storage device (e.g., a computer-readable storage medium) that includes gates configured to store data (e.g., bits) that can be retrieved by a machine (e.g., a computing device such as the processing unit 202). The memory unit 204 can be configured to store information, data, content, applications, instructions, etc. to enable the apparatus to perform various functions in accordance with exemplary embodiments of the present disclosure. For example, the memory unit 204 can be configured to store 3D surface information after a dental arch scanning session is completed. In some cases, when the bandwidth of one of the communication channels 108 is below a minimum bandwidth for a duration longer than a maximum period, or when the connection to one of the communication channels 108 is lost, the memory unit 204 can be configured to store compressed 3D surface information. In some embodiments, the memory unit 204 can be configured to store calibration data required to generate a 3D model corresponding to a plurality of 2D scanned images. As Figure 2 As illustrated, the memory unit 204 can be configured to store instructions for execution by the processing unit 202. Therefore, whether configured by hardware or software methods or a combination thereof, the processing unit 202 can represent an entity (e.g., physically implemented in a circuit) that is capable of performing operations according to embodiments of the present disclosure when configured accordingly. Therefore, for example, when the processing unit 202 is implemented as a microprocessor, the processing unit 202 can be specially configured hardware for performing the operations described herein. Alternatively, as another example, when the processing unit 202 is implemented as an executor of software instructions, the instructions can specifically configure the processing unit 202 so that it performs the algorithms and / or operations described herein when the instructions are executed. The processing unit 202 may include a clock, an arithmetic logic unit (ALU), and logic gates configured to support the operations of the processing unit 202, etc.

[0075] The web server interface 206 can be software, hardware, or a combination thereof, which can be configured to store data and provide the data to a web browser on one or more client devices 106. For example, 3D surface information can be provided to a web browser of one or more client devices 106 via the web server interface 206. Since any web browser can access the web server interface 206, one or more client devices 106 do not need to install additional software to connect to the web server interface 206. The web server interface 206 can communicate with one of the communication channels 108 via a web network. In an example, the web server interface 206 and the one or more client devices 106 can communicate with a public wireless full-duplex communication channel via a web network to transmit and receive 3D surface information. The web server interface 206 and the web browser can communicate via Hypertext Transfer Protocol (HTTP), Simple Mail Transfer Protocol (SMTP), or File Transfer Protocol (FTP). Once the web server interface 206 and the web browser are connected, the web server interface 206 can provide a web application on the web browser. The details of the connection between the web server interface 206 and the web browser will be described in, for example Figure 6 Further provided in.

[0076] The monitoring unit 208 can be software, hardware, or a combination thereof, and can be configured to monitor the bandwidth of one of the communication channels 108 (e.g., a wireless full-duplex communication channel) through which the handheld intraoral scanning device 104 can connect to one or more client devices 106. Furthermore, the monitoring unit 208 can be configured to monitor the connection between the handheld intraoral scanning device 104 and one of the communication channels 108 through which the one or more client devices 106 can connect. Furthermore, the monitoring unit 208 can be configured to transmit status input to the one or more client devices 106 based on the monitored bandwidth and connection.

[0077] In an embodiment, based on monitoring, the monitoring unit 208 may determine that the bandwidth is below a minimum bandwidth. In this case, the monitoring unit 208 may provide information that the bandwidth is below the minimum bandwidth to the processing unit 202. The processing unit 202 may downsample the 3D surface information based on the received information. In another embodiment, based on monitoring, the monitoring unit 208 may determine that the bandwidth is below the minimum bandwidth for a duration longer than a maximum period. In this case, the processing unit 202 may compress the 3D surface information and store it in the memory unit 204. In some embodiments, based on monitoring, the monitoring unit 208 may determine that the connection between the handheld intraoral scanning device 104 and one or more client devices 106 has been lost. In this case, the processing unit 202 may compress the 3D surface information and store it in the memory unit 204. Details of the monitoring of bandwidth and connection and the transmission of status input will be described, for example, in Figure 8 Further provided in.

[0078] The temporary storage unit 210 can be software, hardware, or a combination thereof, and can be configured to store the 3D surface information when it is determined that the bandwidth of one of the communication channels 108 (e.g., a wireless full-duplex communication channel) is less than a minimum bandwidth. The temporary storage unit 210 can also transmit the stored 3D surface information to one or more client devices 106 when it is determined that the bandwidth is greater than or equal to the minimum bandwidth. Examples of the temporary storage unit 210 can include, but are not limited to, random access memory (RAM) or cache memory.

[0079] The scan feedback unit 212 can be software, hardware, or a combination thereof, which can be configured to receive status input from the monitoring unit. Based on the received status input, the scan feedback unit 212 can provide a scan feedback signal to the user of the handheld intraoral scanning device 104. In an embodiment, the scan feedback signal is used to provide guidance to the user to the area of ​​the dental arch where the scan quality of the scan session is low and 3D surface information cannot be provided. For example, the scan feedback signal can be used to provide acoustic feedback signals, tactile feedback, or visual feedback. The details of providing the scan feedback signal will be described, for example, in Figure 8 Further provided in.

[0080] The I / O unit 214 may include circuitry and / or software that may be configured to provide output to a user of the handheld intraoral scanning device 104. The I / O unit 214 may include a speaker 214A, a vibrator 214B, and a plurality of light emitting diodes (LEDs) 214C. In an embodiment, the speaker 214A may be configured to output an acoustic feedback signal to guide the user. The vibrator 214B may be, for example, a transducer configured to convert a scanning feedback signal, which may be an electrical signal, into a mechanical output, such as tactile feedback in the form of vibration to guide the user. The plurality of LEDs 214C may be configured to output the scanning feedback signal in the form of light to guide the user. For example, the plurality of LEDs 214C may be divided into a left group of LEDs and a right group of LEDs to emit flashes. Detailed information on the acoustic feedback signal, tactile feedback, and visual feedback such as light will be provided, for example, in Figure 8 Further provided in.

[0081] The communication interface 216 may include input and output interfaces for supporting communications to and from the handheld intraoral scanning device 104. The communication interface 216 may be a device or circuitry implemented in hardware or a combination of hardware and software that is configured to receive and / or transmit data to and from the handheld intraoral scanning device 104. In this regard, the communication interface 216 may include, for example, an antenna (or multiple antennas) and supporting hardware and / or software for enabling communication with a wireless communication network. Additionally or alternatively, the communication interface 216 may include circuitry for interacting with the antenna(s) to enable transmission of signals via the antenna(s) or to process reception of signals received via the antenna(s). In some environments, the communication interface 216 may alternatively or additionally support wired communications. Thus, for example, the communication interface 216 may include a communication modem and / or other hardware and / or software for supporting communication via cable, digital subscriber line (DSL), universal serial bus (USB), or other mechanisms.

[0082] Figure 3 A block diagram 300 of one or more client devices 106 is shown according to an example embodiment. Figure 3 Combine Figure 1 and Figure 2 The one or more client devices 106 may include a processing unit 302 , a memory unit 304 , a display unit 306 , and a communication interface 308 .

[0083] The processing unit 302 can be implemented in a number of different fashions. For example, the processing unit 302 can be implemented as one or more of various hardware processing means such as a coprocessor, a microprocessor, a controller, a digital signal processor (DSP), a processing element with or without an accompanying DSP, or various other processing circuitry including integrated circuits such as, for example, an ASIC, an MCU, a programmable logic device (PLD), a microcontroller, a hardware accelerator, a special- purpose computer chip, or the like. In some embodiments, the processing unit 302 can comprise one or more processing cores configured to perform independently. A multi-core processing can enable parallel processing of data. In addition or alternatively, the processing unit 302 can include one or more processors configured in tandem via the bus to enable independent execution of instructions, pipelining and / or multithreading.

[0084] In some embodiments, the processing unit 302 can include processing power necessary to process 3D surface information. The processing unit 302 can be configured to establish a connection to one of the communication channels 108. The processing unit 302 can also receive 3D surface information from the handheld intraoral scanning device 104. The processing unit 302 can also render the 3D surface information as an interactive 3D graphical representation 112. Additionally, or alternatively, the processing unit 302 can include one or more processors capable of handling a large workload and operations to provide support for big data analytics. In an example embodiment, the processing unit 302 can communicate with the memory unit 304 via a bus to pass information between components of the one or more client devices 106.

[0085] The memory unit 304 can be non-transitory and can include, for example, one or more volatile memory devices and / or non-volatile memory devices. In other words, for example, the memory unit 304 can be an electronic storage device (e.g., computer readable storage medium) that includes gates configured to store data (e.g., bits) that can be retrievable by a machine (e.g., a computing device like the processing unit 302). The memory unit 304 can be configured to store information, data, content, applications, instructions, etc. for enabling the apparatus to perform various functions in accordance with example embodiments of the present disclosure. For example, the memory unit 304 can be configured to store received 3D surface information. In some embodiments, the memory unit 304 can be configured to store the interactive 3D graphical representation 112. In embodiments, the memory unit 304 can be configured to store a 3D model received from the handheld intraoral scanning device 104.

[0086] As Figure 3As shown in the example, memory unit 304 can be configured to store instructions for execution by processing unit 302. Therefore, whether configured by hardware or software methods or a combination thereof, processing unit 302 can represent an entity (e.g., physically implemented in a circuit) that is capable of performing operations according to embodiments of the present disclosure when configured accordingly. Therefore, for example, when processing unit 302 is implemented as a microprocessor, processing unit 302 can be specially configured hardware for performing the operations described herein. Alternatively, as another example, when processing unit 302 is implemented as an executor of software instructions, the instructions can specifically configure processing unit 302 so that it performs the algorithms and / or operations described herein when the instructions are executed. Processing unit 302 may include a clock, an ALU, and logic gates configured to support the operations of processing unit 202, etc.

[0087] The display unit 306 may be configured to display a web browser, web applications, and interactive 3D graphical representations 112. In some embodiments, the display unit 306 may be externally connected to one or more client devices 106. Examples of the display unit 306 may include, but are not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, Electroluminescent (ELD) displays , plasma display, or cathode ray tube (CRT) display. In an embodiment, the display unit 306 may be a touch screen display. The display unit 306 may receive input in the form of gestures from the user to control the interactive 3D graphical representation 112.

[0088] The communication interface 308 may include input interfaces and output interfaces for supporting communications to and from one or more client devices 106. The communication interface 308 may be a device or circuit implemented in hardware or a combination of hardware and software that is configured to receive and / or transmit data to / from one or more client devices 106. In this regard, the communication interface 308 may include, for example, an antenna (or multiple antennas) and supporting hardware and / or software for enabling communications with a wireless communication network. Additionally or alternatively, the communication interface 308 may include circuitry for interacting with the antenna(s) to enable transmission of signals via the antenna(s) or to process reception of signals received via the antenna(s). In some environments, the communication interface 308 may alternatively or additionally support wired communications. Thus, for example, the communication interface 308 may include a communication modem and / or other hardware and / or software for supporting communications via cable, DSL, USB, or other mechanisms.

[0089] Figure 4 FIG4 is a diagram 400 illustrating an environment in which a handheld intraoral scanning device 104 and one or more client devices 106 communicate, according to an exemplary embodiment. Figure 1 、 Figure 2 and Figure 3 The element pairs in Figure 4 The schematic diagram 400 may include the handheld intraoral scanning device 104 and the communication channel 108. The schematic diagram 400 may also include one or more client devices 106.

[0090] The one or more client devices 106 may include a client device 402, a client device 404, and a client device 406. According to one embodiment, the one or more client devices 106 may be at least one of a display unit, a tablet computer, or a smartphone. For example, the client device 402 may be a smartphone of a user (e.g., a dentist). In another example, the client device 404 may be a tablet computer. In some embodiments, the one or more client devices 106 may be computers. For example, the client device 406 may be a computer with enhanced processing power (e.g., a PowerPC). The client device 402, the client device 404, and the client device 406 may be used by one or more users to view the interactive 3D graphical representation 112.

[0091] In some embodiments, the handheld intraoral scanning device 104 and the web server interfaces of one or more client devices 106 can be connected to a public web network. The web network can provide access to various web pages and web applications. The public web network may be required to transmit data, such as 3D surface information, which can be transmitted via a web application that is accessible via a web browser of one or more client devices 106. In one example, the web network can be hosted directly on the handheld intraoral scanning device 104, and one or more client devices 106 can connect to the web network hosted directly on the handheld intraoral scanning device 104.

[0092] Once the handheld intraoral scanning device 104 and the one or more client devices 106 are connected to the public web network, the one or more client devices 106 can be configured to connect to one of the one or more wireless full-duplex communication channels by forwarding an identification number to the handheld intraoral scanning device 104. For example, the identification number can be a unique serial number of the handheld intraoral scanning device 104, which can be entered by a user of a client device (e.g., client device 402) (e.g., a dentist) through a web browser on the client device 402. A web browser can be presented on the one or more client devices 106 based on receiving the input from the user. For example, the user can select a web browser on the one or more client devices 106 to present the web browser.

[0093] In one example, a multicast domain name system (mDNS) protocol can be used to resolve a host name (e.g., an identification number) into an Internet Protocol (IP) address to access a web server interface on the handheld intra-oral scanning device 104. In some embodiments, an alias can be provided for the identification number of the handheld intra-oral scanning device 104, which can be entered by a user in a web browser to connect to the web server interface. Similarly, multiple client devices (e.g., client device 404 and client device 406) can also connect to the web server interface of the handheld intra-oral scanning device 104 based on the receipt of the identification number by the respective users of the multiple client devices.

[0094] Once one or more client devices 106 are connected to the web server interface of the handheld intraoral scanning device 104, the handheld intraoral scanning device 104 can provide a web application to the user of the one or more client devices 106, thereby enabling interaction with the handheld intraoral scanning device 104 and providing a view for presenting the interactive 3D graphical representation 112. In one example, the web server interface of the handheld intraoral scanning device 104 and the web application of the one or more client devices 106 can communicate using different communication protocols (e.g., the WebSocket protocol). In some embodiments, the web application can be based on different web communication languages, such as Hypertext Markup Language (HTML), JavaScript, and WebAssembly. The WebAssembly module of the web application can implement compiled code, allowing access to the web server interface to run on the one or more client devices 106 at a speed close to that of the native machine, and thereby reducing the requirements for resources required on the one or more client devices 106.

[0095] Once the handheld intraoral scanning device 104 is connected to one or more client devices 106, a user (e.g., a dentist) can initiate a scanning session and present a web application through a web browser on one or more client devices 106. The handheld intraoral scanning device 104 can generate 3D surface information using the captured multiple 2D scan images 110. For example, Figure 5 Details of capturing multiple 2D scan images 110 and generating 3D surface information are further provided in .

[0096] Figure 5 is a schematic diagram 500 illustrating capturing a plurality of 2D scan images 110 to generate 3D surface information, according to an exemplary embodiment. Figure 5 Combine Figure 1 、 Figure 2 、 Figure 3 and Figure 4Schematic diagram 500 may include a user (eg, dentist 502) and a patient 504. Dentist 502 may use handheld intraoral scanning device 104 to capture a plurality of two-dimensional scan images 110 of a dental arch 506 of patient 504.

[0097] In an exemplary scenario, a dentist 502 and a patient 504 may be present at a dental clinic. The dentist 502 may initiate a scanning session of the dental arch 506 of the patient 504 to begin dental treatment for the patient 504. The handheld intraoral scanning device 104 may include a built-in camera. The built-in camera of the handheld intraoral scanning device 104 may be placed inside the mouth of the patient 504 and moved around the teeth and gums of the patient 504 to record the oral morphology of the patient 504. In one example, the handheld intraoral scanning device 104 may record the size and shape of each tooth, interdental spaces, the appearance of the palatal surface, gums, implants, prostheses, and other elements that make up the interior of the patient's 504 mouth. In one embodiment, the built-in camera of the handheld intraoral scanning device 104 may be moved multiple times over the teeth and gums of the patient 504 to capture multiple 2D scan images 110. For example, the multiple 2D scan images 110 may include scan image 508A, scan image 508B, and scan image 508N. Scanned image 508A, scanned image 508B, and scanned image 508N may be used to generate 3D surface information 510 .

[0098] Once the scanning session begins and a plurality of 2D scan images 110 may be captured, the handheld intraoral scanning device 104 may be configured to process the plurality of 2D scan images 110 by applying calibration parameters and filtering noise from the plurality of 2D scan images 110. For example, calibration parameters of a built-in camera may be applied to process the plurality of 2D scan images 110. Additionally, noise may be filtered from the plurality of 2D scan images 110. Based on the processing of the plurality of 2D scan images 110, the handheld intraoral scanning device 104 may be configured to provide 3D surface information 510. In some embodiments, the 3D surface information 510 may be 3D point cloud data. The 3D surface information 510 may be transmitted to one or more client devices 106 connected to the handheld intraoral scanning device 104. For example, Figure 6 Details of transmitting the 3D surface information 510 to one or more client devices 106 are further provided.

[0099] Figure 6 is a schematic diagram 600 illustrating the transmission of 3D surface information 510 and the rendering of an interactive 3D graphical representation, according to an exemplary embodiment. Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 and Figure 5 The element pairs in Figure 6Schematic diagram 600 may include a handheld intraoral scanning device 104 and a client device 404, such as a tablet computer. The handheld intraoral scanning device 104 may include generated 3D surface information 510. The client device 404 may present an interactive 3D graphical representation 112.

[0100] In some embodiments, the handheld intraoral scanning device 104 can be configured to broadcast the generated 3D surface information 510 to multiple client devices among the one or more client devices 106 connected to the handheld intraoral scanning device 104 via one of the one or more wireless full-duplex communication channels. In one example, the client device 404 can receive an identification number from a user (e.g., a dentist) to connect to a web server interface. For example, the user can enter the identification number in a web browser 602 of the client device 404. The client device 404 can forward the identification number to the handheld intraoral scanning device 104 to connect to the web server interface. Based on this connection, the handheld intraoral scanning device 104 can broadcast the 3D surface information 510 to the client devices 404.

[0101] Similarly, client device 402 and client device 406 can be connected to a web server interface. In this case, handheld intraoral scanning device 104 can broadcast 3D surface information to client device 402 and client device 406 (e.g., Figure 4 In one embodiment, the handheld intraoral scanning device 104 may broadcast the 3D surface information to multiple client devices in the one or more client devices 106 using different communication protocols (e.g., WebSocket).

[0102] In some embodiments, the plurality of client devices 106 can be configured to independently receive 3D surface information 510 via the one of the one or more wireless full-duplex communication channels. Broadcasting 3D surface information 510 via WebSocket enables the one or more client devices 106 to independently receive 3D surface information 510. The 3D surface information 510 can be independently received in real time, i.e., the 3D surface information 510 can be received nearly instantaneously, or in near real time, i.e., the 3D surface information 510 is received with minimal latency. The plurality of client devices 106 can also independently render the 3D surface information 510 into an interactive 3D graphics representation 112 compatible with a web browser. For example, client device 404 can independently render the 3D surface information 510 into an interactive 3D graphics representation 112 compatible with a web browser 602. In one example, the interactive 3D graphics representation 112 can be rendered on a web application running on the web browser 602.

[0103] In embodiments, the WebSocket may enable independent rendering of the 3D surface information 510 as an interactive 3D graphics representation 112 on the plurality of client devices in the one or more client devices 106. In some embodiments, the independent rendering of the 3D surface information 510 as an interactive 3D graphics representation 112 may be performed using a WebGL JavaScript application programming interface (API). The WebGL JavaScript API may be used to render high-performance interactive 3D graphics representations 112 on a web application running on a web browser 602.

[0104] In this example, the interactive 3D graphical representation 112 can be independently presented on the client device 402 for the user of the client device 402. The interactive 3D graphical representation 112 can also be independently presented on the client device 404 for the user of the client device 404. Each of the one or more client devices 106 can have a separate session with the web server interface of the handheld intraoral scanning device 104. Each of the one or more client devices 106 can receive a copy of the 3D surface information 510 from the web server interface. Thus, different users can independently access the interactive 3D graphical representation 112. In some cases, a single user can use multiple of the one or more client devices 106 to view the interactive 3D graphical representation 112 from different angles.

[0105] The user can interact with the interactive 3D graphical representation 112. For example, the interactive 3D graphical representation 112 can be rotated, enlarged, or reduced, as desired by the user. In some embodiments, the interactive 3D graphical representation 112 may be 3D point cloud data. In some embodiments, the interactive 3D graphical representation 112 may be a 3D model of the dental arch 506 of the patient 504. Figure 7 The end-to-end process of rendering interactive 3D graphics representations based on the transferred 3D surface information is further explained.

[0106] Figure 7 is a sequence diagram 700 illustrating rendering an interactive 3D graphical representation based on transmitted 3D surface information according to an exemplary embodiment. Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 and Figure 6 Elements to explain Figure 7 The sequence diagram 700 may include the handheld intra-oral scanning device 104 and the one or more client devices 106. The sequence diagram 700 may depict operations performed by at least one of the handheld intra-oral scanning device 104 and the one or more client devices 106.

[0107] At step 702, the web server interface of the handheld intraoral scanning device 104 may establish a connection with one of the communication channels 108. For example, the web server interface of the handheld intraoral scanning device 104 may communicate via a web network to establish a connection with one or more wireless full-duplex communication channels. Figure 2 and Figure 4 Details of the connection of the web server interface to one or more wireless full-duplex communication channels are further provided in.

[0108] At step 704, the web browser 602 may be presented on one or more client devices 106. For example, the web browser 602 may be presented on a client device 404 (e.g., a tablet computer). In the exemplary scenario, the dentist 502 may be available to the client device 404. The dentist 502 may operate the client device 404 to access the web browser 602 on the client device 404. The presentation details of the web browser 602 may be, for example, Figure 4 Further provided in.

[0109] At step 706, an identification number may be received. The identification number may be received via a web browser presented on one or more client devices 106 (e.g., client device 404). For example, dentist 502 may provide the identification number as input on a web browser of client device 404. For example, Figure 4 Details of the receipt of the identification number are further provided in.

[0110] At 708, the identification number may be forwarded to the handheld intraoral scanning device 104. One or more client devices 106 may forward the identification number entered by the user (e.g., dentist 502) to the handheld intraoral scanning device 104. The identification number may be used to establish a connection with the web server interface of the handheld intraoral scanning device 104. For example, Figure 4 Further details on identification number forwarding are provided in .

[0111] At 710, a connection to one of the one or more wireless full-duplex communication channels may be established. Based on the identification number forwarded by the web network, a connection may be established between the one or more client devices 106 and the one of the one or more wireless full-duplex communication channels. Details of the connection to the one of the one or more wireless full-duplex communication channels are further described, for example, in Figure 4 Available in.

[0112] At 712, a plurality of 2D scan images 110 may be captured. The handheld intraoral scanning device 104 may be configured to capture a plurality of 2D scan images 110 of the dental arch 506 of the patient 504 during the scanning session. Details of the captured plurality of 2D scan images 110 are further described, for example, in Figure 5 Available in.

[0113] At 714, 3D surface information 510 may be provided. The handheld intraoral scanning device 104 may be configured to provide the 3D surface information 510 based on the captured plurality of 2D scan images 110. For example, the 3D surface information 510 may be 3D point cloud data. Details of providing the 3D surface information 510 are described in, for example, Figure 5 Further provided in.

[0114] At 716, 3D surface information may be received. The one or more client devices 106 may be configured to receive the 3D surface information 510 from the intraoral scanning device 104. The one or more client devices 106 may receive the 3D surface information 510 by using one of the communication channels 108. Details of the one or more client devices 106 receiving the 3D surface information 510 are described in, e.g., Figure 5 Further provided in.

[0115] At 718, the interactive 3D graphics representation 112 may be rendered. The one or more client devices 106 may be configured to render the interactive 3D graphics representation 112 based on the 3D surface information 510. In an embodiment, the one or more client devices 106 may independently render the interactive 3D graphics representation 112. Details of the rendering of the interactive 3D graphics representation 112 may be provided, for example, in FIG. Figure 6 Further provided in.

[0116] It should be understood that each step in sequence diagram 700 can be implemented in various ways, such as hardware, firmware, a processor, circuitry, and / or other communication devices associated with executing software containing one or more computer program instructions. For example, one or more of the above steps can be implemented by computer program instructions. In this regard, the computer program instructions implementing the above steps can be stored in the memory 204 of the handheld intraoral scanning device 104 according to the disclosed embodiments. It should be understood that any such computer program instructions can be loaded onto a computer or other programmable device (e.g., hardware) to produce a machine, causing the resulting computer or other programmable device to implement the functions specified in sequence diagram 700. These computer program instructions can also be stored in a computer-readable memory, which can instruct the computer or other programmable device to operate in a specific manner, causing the instructions stored in the computer-readable memory to produce an article of manufacture, which, when executed, implements the functions specified in sequence diagram 700. The computer program instructions can also be loaded onto a computer or other programmable device to cause a series of operations to be performed on the computer or other programmable device, thereby producing a computer-implemented process, such that the instructions executed on the computer or other programmable device provide the operations for implementing the functions specified in sequence diagram 700.

[0117] Therefore, the steps of sequence diagram 700 support a combination of means for performing the specified functions and a combination of operations for performing the specified functions. It should also be understood that one or more steps in sequence diagram 700 and a combination of steps in sequence diagram 700 can be implemented by a computer system based on dedicated hardware that performs the specified functions or a combination of dedicated hardware and computer instructions. Figure 7 700 is a sequence diagram for intraoral scan registration. Fewer, more, or different steps may be provided.

[0118] The handheld intraoral scanning device 104 may also be configured to monitor the bandwidth and connectivity of the communication channel 108. Based on this monitoring, the handheld intraoral scanning device 104 may perform steps on the generated 3D surface information before transmitting the 3D surface information to one or more client devices 106, such as Figure 8 shown.

[0119] Figure 8 An example flow chart 800 is shown that includes different scenarios for transmitting 3D surface information to one or more client devices 106 in accordance with an example embodiment. Figure 8 Combine Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 、 Figure 6 and Figure 7The flowchart 800 may depict operations performed by at least one of the handheld intraoral scanning device 104 and the one or more client devices 106 .

[0120] At step 802, a plurality of 2D scan images 110 may be captured. The handheld intraoral scanning device 104 may be configured to capture a plurality of 2D scan images 110 of the dental arch 506 of the patient 504 during a scanning session. Figure 5 Details of capturing the plurality of 2D scan images 110 are further provided in .

[0121] At step 804, 3D surface information may be provided. The intraoral scanning device 104 may be configured to provide 3D surface information based on the captured plurality of 2D scan images 110. For example, the 3D surface information may be 3D point cloud data. Details of the 3D surface information may be provided, for example, in Figure 5 Further provided in.

[0122] At step 806, the bandwidth of one of the one or more wireless full-duplex communication channels may be monitored. In some embodiments, the bandwidth of the one of the one or more wireless full-duplex communication channels may be monitored by the monitoring unit 208 of the handheld intraoral scanning device 104. For example, the bandwidth of the one of the one or more wireless full-duplex communication channels may be monitored to determine the throughput and latency of the one of the one or more wireless full-duplex communication channels.

[0123] At step 808, the handheld intraoral scanning device 104 may check whether the bandwidth of the one of the one or more wireless full-duplex communication channels is less than a minimum bandwidth. In some embodiments, the monitoring unit 208 of the handheld intraoral scanning device 104 may be configured to check whether the bandwidth of the one of the one or more wireless full-duplex communication channels is less than the minimum bandwidth.

[0124] At step 810, based on determining that the bandwidth of the one of the one or more wireless full-duplex communication channels is greater than or equal to the minimum bandwidth, the handheld intra-oral scanning device 104 may transmit the 3D surface information 510 to the one or more client devices 106. For example, the handheld intra-oral scanning device 104 may transmit the 3D surface information 510 to the one or more client devices 106 via the one of the one or more wireless full-duplex communication channels. The transmission details of the 3D surface information 510 may be, for example, Figure 6 Further provided in.

[0125] At step 812, based on determining that the bandwidth of the one of the one or more wireless full-duplex communication channels is below the minimum bandwidth, the 3D surface information 510 can be downsampled. In some embodiments, the handheld intraoral scanning device 104 can be configured to downsample the 3D surface information 510 for transmission via the one of the one or more wireless full-duplex communication channels. After downsampling, the 3D surface information 510 can be transmitted to the one or more client devices 106.

[0126] In one embodiment, the broadcasted 3D surface information 510 may be downsampled or heavily compressed using lossy compression to ensure that the one or more client devices 106 receive the 3D surface information 510 in real time. Furthermore, the remaining data of the 3D surface information 510 may be buffered on the handheld intraoral scanning device 104 for later transmission during or after the scanning session when the bandwidth of the one or more wireless full-duplex communication channels is greater than or equal to a minimum bandwidth. In some cases, the 3D surface information 510 that needs to be transmitted for further post-processing on a remote server may be buffered by the connected one or more client devices 106. In some embodiments, the plurality of 2D scan images 110 and the 3D surface information 510 may be stored on the handheld intraoral scanning device 104 or on one or more client devices 106 and later transmitted for presentation or post-processing at a remote server in the dental office or in the cloud.

[0127] In an embodiment, when one or more client devices 106 are computers with high computing power, a WebAssembly module can be used to perform certain processing on the one or more client devices 106. In this way, the battery of the handheld intraoral scanning device 104 can be conserved. Thus, a web application for scanning can be loaded directly from the handheld intraoral scanning device 104 to any of the one or more client devices 106 without the need for installation.

[0128] It should be noted that the 3D surface information 510 required to render the interactive 3D graphical representation 112 on one or more client devices 106 during a scanning session is intended to provide visual feedback to the dentist 502 and, therefore, can be highly compressed even using lossy compression, as long as the interactive 3D graphical representation 112 appears intact. This compression can reduce the amount of data (e.g., the 3D surface information 510) that needs to be sent over the communication channel 108 during a scanning session. The amount of data to be transmitted can even be adjusted based on the available bandwidth of the communication channel 108. If the bandwidth drops below a minimum bandwidth, a less dense data set of the 3D surface information 510 can be sent; and when the bandwidth rises above the minimum bandwidth, the remaining data can be transmitted.

[0129] At step 814, the 3D surface information 510 may be stored in the temporary storage unit 210. In some embodiments, the temporary storage unit 210 may be configured to store the 3D surface information 510 when it is determined that the bandwidth of the one of the one or more wireless full-duplex communication channels is below a minimum bandwidth. Storing the 3D surface information 510 in the temporary storage unit 210 may enable quick retrieval of the 3D surface information 510 when needed. The monitoring unit 208 may be configured to continuously monitor the bandwidth of the one of the one or more wireless full-duplex communication channels. Based on determining that the bandwidth of the one of the one or more wireless full-duplex communication channels is greater than or equal to the minimum bandwidth, the temporary storage unit 210 may transmit the 3D surface information 510 to the one or more client devices 106.

[0130] At step 816, the handheld intraoral scanning device 104 may be configured to check whether the bandwidth of the one of the one or more wireless full-duplex communication channels is below a minimum bandwidth for a duration longer than a maximum period. In some embodiments, the monitoring unit 208 may be configured to check whether the bandwidth of the one of the one or more wireless full-duplex communication channels is below the minimum bandwidth for a duration longer than a maximum period.

[0131] Based on determining that the bandwidth of the one of the one or more wireless full-duplex communication channels is not less than the minimum bandwidth for a duration longer than the maximum time period, the handheld intraoral scanning device 104 can be configured to transmit the 3D surface information 510 to the one or more client devices 106 via the one of the one or more wireless full-duplex communication channels, as described in step 810.

[0132] At step 818, based on determining that the bandwidth of the one or more wireless full-duplex communication channels has been below the minimum bandwidth for a duration longer than the maximum period, the handheld intraoral scanning device 104 may be configured to compress the 3D surface information 510 and store it in the memory unit 204 of the handheld intraoral scanning device 104. The 3D surface information 510 may be stored on the memory unit 204 or on a disk associated with the handheld intraoral scanning device 104, after which one or more client devices 106 may download the information or upload it to a powerful scanning server for post-processing and diagnosis. Furthermore, the 3D surface information 510 may be compressed and stored for subsequent use. Furthermore, since most of the processing of the 3D surface information 510 may have already been performed on the handheld intraoral scanning device 104, the intraoral scanning system 102 may be able to remove noise and irrelevant data from the 3D surface information 510, thereby helping to even further reduce the size of the 3D surface information 510.

[0133] At step 820, the connection to the one of the communication channels 108 may be monitored. In some embodiments, the monitoring unit 208 may be configured to monitor the connection to the one of the one or more wireless full-duplex communication channels.

[0134] At step 822 , the monitoring unit 208 may be configured to check whether the connection to the one of the one or more wireless full-duplex communication channels is lost.

[0135] Based on determining that the connection to the one of the one or more wireless full-duplex communication channels has not been lost, the handheld intraoral scanning device 104 can be configured to transmit the 3D surface information 510 to the one or more client devices 106 via the one of the one or more wireless full-duplex communication channels, as described in step 810.

[0136] In some embodiments, the handheld intraoral scanning device 104 can be configured to transmit the 3D surface information 510 stored in the memory unit 204 via the wireless communication interface of the intraoral scanning system 102 upon completion of the scanning session. After the dentist 502 scans the dental arch 506, the handheld intraoral scanning device 104 can transmit the 3D surface information 510 to the memory unit 204 via the wireless communication interface of the communication channel 108. For example, the wireless communication interface can be a web network or a full-duplex communication channel.

[0137] Furthermore, based on determining that the connection to the one of the one or more wireless full-duplex communication channels is lost, the handheld intraoral scanning device 104 may be configured to compress the 3D surface information 510 and store it in the memory unit 204 of the handheld intraoral scanning device 104 , as depicted in step 818 .

[0138] This connection monitoring and storage of the 3D surface information 510 ensures the safety of the 3D surface information 510 in the event of a connection loss. Furthermore, the intraoral scanning system 102 allows the scanning session to continue even if the battery of one or more client devices 106 runs out, providing flexibility in where and when the 3D surface information 510 is sent. Often, the real-time feedback required by the dentist 502 while scanning the dental arch 506 without any wait time may be more important than the resolution of the 3D surface information 510. On the other hand, when the dentist 502 is examining the 3D surface information 510 or 3D model to perform diagnostic work, the time spent optimizing the 3D model becomes less important. Therefore, the 3D surface information 510 can be easily sent to a server for post-processing, and then the processed 3D surface information 510 can be sent back to one or more client devices 106 for presentation in the dental office or on a cloud server.

[0139] At step 824, based on determining that the bandwidth of the one of the one or more wireless full-duplex communication channels is below at least one of a minimum bandwidth or a connection is lost, the monitoring unit 208 can be configured to transmit a status input to the scan feedback unit 212. In some embodiments, the scan feedback unit 212 can receive the status input from the monitoring unit 208. The status input can be, for example, a control signal transmitted to the scan feedback unit 212.

[0140] At step 826, while receiving the status input from the monitoring unit 208, the scan feedback unit 212 can be configured to provide a scan feedback signal to the user of the handheld intraoral scanning device 104 (e.g., the dentist 506). The scan feedback signal can be configured to guide the user to areas of the dental arch 506 where the scan quality of the scanning session is low and the 3D surface information 510 cannot be provided. The scan feedback signal can help the dentist 504 capture a larger number of the plurality of two-dimensional scan images 110.

[0141] In some embodiments, the scan feedback signal may include an acoustic feedback signal configured to guide the user toward the area of ​​the dental arch 506. In some embodiments, the acoustic feedback signal may be output by the speaker 214A of the handheld intraoral scanning device 104. For example, the speaker 214A may output a beeping sound as the acoustic feedback signal. In another embodiment, the speaker 214A may output a continuous beeping sound as the acoustic feedback signal. In an exemplary scenario, the intensity of the acoustic feedback signal increases when the handheld intraoral scanning device 104 moves toward an area of ​​low scan quality during a scanning session. Furthermore, the intensity of the acoustic feedback signal decreases when the handheld intraoral scanning device 104 moves away from the area of ​​low scan quality during a scanning session.

[0142] In some embodiments, the scan feedback signal may include at least one of tactile feedback or light emitted by the plurality of LEDs 241C of the handheld intraoral scanning device 104. The dentist 502 may utilize the tactile feedback and / or the emitted light to guide the handheld intraoral scanning device 104 to areas of low scan quality during the scanning session.

[0143] In some embodiments, the handheld intraoral scanning device 104 may include a vibrator 214B that can be configured to provide tactile feedback. In some cases, an increase in vibration of the vibrator 214B may indicate that the distance between the handheld intraoral scanning device 104 and the area of ​​low scan quality during the scanning session of the dental arch 506 has increased. Additionally, a decrease in vibration of the vibrator 214B may indicate that the distance between the handheld intraoral scanning device 104 and the area of ​​low scan quality during the scanning session of the dental arch 506 has decreased. This feedback provides guidance to the dentist 502 to accurately place the handheld intraoral scanning device 104 in the area of ​​low scan quality during the scanning session.

[0144] In some embodiments, the plurality of LEDs 241C of the handheld intraoral scanning device 104 can be divided into a left group of LEDs and a right group of LEDs. For example, the left group of LEDs can be arranged on the left side of the handheld intraoral scanning device 104, and the right group of LEDs can be arranged on the right side of the handheld intraoral scanning device 104. The left group of LEDs and the right group of LEDs can be configured to emit a flash when the handheld intraoral scanning device 104 is positioned to the right or left side of the dental arch 506, respectively. The dentist 502 can use the flash to accurately position the handheld intraoral scanning device 104 in areas of poor scan quality during a scanning session. Based on the guidance received by the dentist 502, the dentist 502 can further capture a larger number of the plurality of two-dimensional scan images 110.

[0145] It should be understood that each step in flowchart 800 can be implemented in various ways, such as hardware, firmware, a processor, circuitry, and / or other communication devices associated with executing software containing one or more computer program instructions. For example, one or more of the above steps can be implemented by computer program instructions. In this regard, the computer program instructions implementing the above steps can be stored in the memory 204 of the handheld intraoral scanning device 104 according to the disclosed embodiments. It should be understood that any such computer program instructions can be loaded onto a computer or other programmable device (e.g., hardware) to produce a machine, causing the resulting computer or other programmable device to implement the functions specified in flowchart 800. These computer program instructions can also be stored in a computer-readable memory, which can instruct the computer or other programmable device to operate in a specific manner, causing the instructions stored in the computer-readable memory to produce an article of manufacture, which, when executed, implements the functions specified in flowchart 800. The computer program instructions can also be loaded onto a computer or other programmable device to cause a series of operations to be performed on the computer or other programmable device, thereby producing a computer-implemented process, such that the instructions executed on the computer or other programmable device provide the operations for implementing the functions specified in flowchart 800.

[0146] Therefore, the steps of flowchart 800 support combinations of means for performing the specified functions and combinations of operations for performing the specified functions. It should also be understood that one or more steps in flowchart 800 and combinations of steps in flowchart 800 can be implemented by a computer system based on dedicated hardware that performs the specified functions or a combination of dedicated hardware and computer instructions. Figure 8 Flowchart 800 is for intraoral scan registration. Fewer, more, or different steps may be provided.

[0147] In some embodiments, the handheld intraoral scanning device 104 may be further configured to generate a 3D model representation of the dental arch 506 by combining the plurality of 3D surface information 510 provided by the handheld intraoral scanning device 104 during a scanning session of the dental arch 506. The handheld intraoral scanning device 104 may also transmit the 3D model to one or more client devices 106 via the communication channel 108. Figure 9 The generation of 3D models is further explained.

[0148] Figure 9 An example flowchart 900 for generating a 3D model based on 3D surface information 510 according to another example embodiment is shown. Figure 9 Combine Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 、 Figure 6 、 Figure 7 and Figure 8 The flowchart 900 may depict operations performed by at least one of the handheld intraoral scanning device 104 and the one or more client devices 106 .

[0149] At step 902, the handheld intraoral scanning device 104 may be configured to capture a first plurality of two-dimensional scan images and a second plurality of two-dimensional scan images containing surface information of the dental arch 506 of the patient 504 during a first time window and a second time window, respectively. The first time window precedes the second time window. For example, the handheld intraoral scanning device 104 captures multiple scan images (e.g., multiple two-dimensional scan images) per sub-scan. The multiple scan images of each sub-scan may be combined into raw scan data. The built-in camera of the handheld intraoral scanning device 104 may be moved around the teeth and gums of the patient 504 to capture the first plurality of 2D scan images and the second plurality of 2D scan images.

[0150] At step 904, the handheld intraoral scanning device 104 may be configured to process the first plurality of two-dimensional scan images and the second plurality of two-dimensional scan images into first three-dimensional surface information and second three-dimensional surface information, respectively. The first three-dimensional surface information and the second three-dimensional surface information may be generated by combining raw scan data from the first plurality of two-dimensional scan images and the second plurality of two-dimensional scan images. In one embodiment, the processing of the first plurality of two-dimensional scan images and the second plurality of two-dimensional scan images may be based on focused scanning.

[0151] The scanning device preferably also includes an optical component for directing light from the light source to the surface of the dental object. The specific arrangement of the optical component depends on whether the scanning device is a focused scanning device, a scanning device utilizing the principle of triangulation, confocal scanning, defocused depth, light field scanning, stereo, deep learning-based three-dimensional measurement, or any other type of scanning device. Light reflected from the dental object in response to illumination of the dental object is directed toward (one or more) image sensors using the optical components of the scanning device. The (one or more) image sensors are configured to generate multiple images based on the incoming light received from the illuminated dental object. The image sensor can be a high-speed image sensor, for example, an image sensor configured to capture images with an exposure time of less than 1 / 1000 second or a frame rate exceeding 250 frames per second (fps). For example, the image sensor can be a rolling shutter (CCD) or a global shutter sensor (CMOS). For example, in-focus measurements can be performed on the first plurality of 2D scan images and the second plurality of 2D scan images to generate first 3D surface information and second 3D surface information, respectively. In another embodiment, the handheld intraoral scanning device 104 can utilize triangulation to generate the first 3D surface information and the second 3D surface information. For example, the handheld intraoral scanning device 104 can project a time-varying illumination pattern, where different patterns are projected onto the scanned object, while simultaneously recording two-dimensional images of different reflected pattern configurations. The time-varying structured light pattern can take the form of a Gray code sequence and a phase-shifted sequence of stripes, bars, or a checkerboard. Projected features can then be tracked between the first and second plurality of two-dimensional scanned images, and the correspondence between the projected features can be determined to triangulate the depth information, thereby generating the first and second three-dimensional surface information.

[0152] At 906, the handheld intraoral scanning device 104 can be configured to generate a first 3D scan patch by transforming the first 3D surface information into first real-world 3D coordinates and first texture information using the calibration data stored in the memory unit 204. The handheld intraoral scanning device 104 can also generate a second 3D scan patch by transforming the second 3D surface information into second real-world 3D coordinates and second texture information using the calibration data stored in the memory unit 204. For example, the first 3D surface information and the second 3D surface information can be transformed using calibration data of a built-in camera. The first real-world 3D coordinates and the first texture information can be extracted from the first plurality of 2D scan images to generate the first 3D scan patch. This can be achieved by adding a Bayer filter to the image sensor of the scanning device. In this way, if the scanner's probe light is a polychromatic white light source (e.g., a wide wavelength spectrum range of 400-750 nm), RGB values ​​can be directly extracted from the two-dimensional image. Alternatively, a monochrome image sensor (without a color filter) can be used to quickly switch between one or more monochromatic light sources (e.g., within 200 milliseconds) (e.g., switching between red, green, and blue light sources) to obtain color texture from a set of two-dimensional images. The intensities recorded by the sensor can then be combined and converted into an RGB image. Similarly, a second real-world three-dimensional coordinate and second texture information can be extracted from a second plurality of two-dimensional scan images to generate a second 3D scan patch.

[0153] At step 908, the handheld intraoral scanning device 104 may be configured to register the second 3D scan patch to at least the first 3D scan patch by locating corresponding data points between the first 3D scan patch and the second 3D scan patch. For example, the data points between the first 3D scan patch and the second 3D scan patch may be mapped to register the second 3D scan patch to at least the first 3D scan patch.

[0154] At step 910, the handheld intraoral scanning device 104 may be configured to fuse the first 3D scan patch and the second 3D scan patch together to form a 3D model. Based on the registration of the second 3D scan patch to at least the first 3D scan patch, the handheld intraoral scanning device 104 may fuse the first 3D scan patch and the second 3D scan patch together to form a 3D model. For example, a spatial transformation may be performed between each sub-scan of the 3D point cloud data (e.g., the first 3D surface information and the second 3D surface information), and the individual 3D point clouds may be stitched together to reconstruct the 3D model. A registration algorithm may be used to first align and / or register the incoming scan data / scan patch with the current 3D representation, and then fuse the new scan patch with the 3D representation before the scanner generates the next scan patch. Registering / registering the new scan data / scan patch should be understood as determining the position of the scan patch within the (current) digital 3D representation, while fusing / fusing the scan data / scan patch should be understood as making the scan data / scan patch part of the digital 3D representation. For registration of scan patches, a variant of the Iterative Closest Point (ICP) algorithm can be used.

[0155] At step 912, the handheld intraoral scanning device 104 may be configured to store the first texture information and the second texture information together with the formed 3D model. For example, the handheld intraoral scanning device 104 may store the first texture information and the second texture information together with the formed 3D model in the memory unit 204 of the handheld intraoral scanning device 104.

[0156] It should be understood that each step in flowchart 900 can be implemented in various ways, such as hardware, firmware, a processor, circuitry, and / or other communication devices associated with executing software containing one or more computer program instructions. For example, one or more of the above steps can be implemented by computer program instructions. In this regard, the computer program instructions implementing the above steps can be stored in the memory 204 of the handheld intraoral scanning device 104 according to the disclosed embodiments. It should be understood that any such computer program instructions can be loaded onto a computer or other programmable device (e.g., hardware) to produce a machine, causing the resulting computer or other programmable device to perform the functions specified in flowchart 900. These computer program instructions can also be stored in a computer-readable memory, which can instruct the computer or other programmable device to operate in a specific manner, causing the instructions stored in the computer-readable memory to produce an article of manufacture, which, when executed, performs the functions specified in flowchart 900. The computer program instructions can also be loaded onto a computer or other programmable device to cause a series of operations to be performed on the computer or other programmable device, thereby producing a computer-implemented process, such that the instructions executed on the computer or other programmable device provide the operations for performing the functions specified in flowchart 900.

[0157] Therefore, the steps of flowchart 900 support combinations of means for performing the specified functions and combinations of operations for performing the specified functions. It should also be understood that one or more steps in flowchart 900 and combinations of steps in flowchart 900 can be implemented by a computer system based on dedicated hardware that performs the specified functions or a combination of dedicated hardware and computer instructions. Figure 9 Flowchart 900 is used to generate a 3D model. Fewer, more, or different steps may be provided.

[0158] In some embodiments, the handheld intraoral scanning device 104 may be configured to transmit the 3D model to one or more client devices 106, such as Figure 10 shown.

[0159] Figure 10 is a sequence diagram 1000 depicting rendering of an interactive 3D graphical representation 112 based on a transferred 3D model, according to an exemplary embodiment. Figure 10 Combine Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 、 Figure 6 、 Figure 7 、 Figure 8 and Figure 9 The sequence diagram 1000 may depict operations performed by the handheld intraoral scanning device 104 and at least one of the one or more client devices 106 to transmit a 3D model.

[0160] At step 1002, the web server interface of the handheld intra-oral scanning device 104 may establish a connection with one of the communication channels 108. For example, the web server interface of the handheld intra-oral scanning device 104 may communicate via a web network to establish a connection with one or more wireless full-duplex communication channels, such as Figure 7 As described in step 702.

[0161] At step 1004, the web browser 602 may be presented on one or more client devices 106. For example, the web browser 602 may be presented on a client device 404 (e.g., a tablet computer) such as Figure 7 as described in step 704.

[0162] At step 1006, an identification number may be received. The identification number may be received via a web browser presented on one or more client devices 106 (e.g., client device 404), such as Figure 7 as described in step 706.

[0163] At step 1008, the identification number may be forwarded to the handheld intraoral scanning device 104. One or more client devices 106 may forward the identification number entered by the user (eg, dentist 502) to the handheld intraoral scanning device 104, such as Figure 7 as described in step 708.

[0164] At step 1010, a connection to the one of the one or more wireless full-duplex communication channels may be established. Based on the identification number forwarded by the web network, a connection may be established between one or more client devices 106 and the one of the one or more wireless full-duplex communication channels, such as Figure 7 as described in step 710.

[0165] At step 1012, a plurality of two-dimensional scan images 110 may be captured. The handheld intraoral scanning device 104 may be configured to capture a plurality of two-dimensional scan images 110 of the dental arch 506, such as Figure 7 as described in step 712.

[0166] At step 1014, three-dimensional surface information 510 may be provided. The handheld intraoral scanning device 104 may be configured to provide 3D surface information based on the captured plurality of 2D scan images 110, such as Figure 7 as described in step 714.

[0167] At step 1016, a 3D model may be generated. The handheld intraoral scanning device 104 may be configured to generate a 3D model based on the 3D surface information 510. For example, Figure 9 Details of 3D model generation are further provided.

[0168] At step 1018, the one or more client devices 106 may be configured to receive the 3D model from the intraoral scanning device 104. The one or more client devices 106 may receive the 3D model by using the one of the communication channels 108.

[0169] At step 1020, the interactive 3D graphics representation 112 may be presented. The one or more client devices 106 may be configured to present the interactive 3D graphics representation 112 based on the 3D model. In one embodiment, the one or more client devices 106 may independently present the interactive 3D graphics representation 112 compatible with the web browser 602. For example, in Figure 6 Details of the presentation of the interactive 3D graphical representation 112 are further provided in .

[0170] It should be understood that each step in sequence diagram 1000 can be implemented in various ways, such as hardware, firmware, a processor, a circuit, and / or other communication devices associated with executing software containing one or more computer program instructions. For example, one or more of the above steps can be implemented by computer program instructions. In this regard, the computer program instructions implementing the above steps can be stored in the memory 204 of the handheld intraoral scanning device 104 according to the disclosed embodiments. It should be understood that any such computer program instructions can be loaded onto a computer or other programmable device (e.g., hardware) to produce a machine, causing the resulting computer or other programmable device to implement the functions specified in sequence diagram 1000. These computer program instructions can also be stored in a computer-readable memory, which can instruct the computer or other programmable device to operate in a specific manner, causing the instructions stored in the computer-readable memory to produce an article of manufacture, which, when executed, implements the functions specified in sequence diagram 1000. The computer program instructions can also be loaded onto a computer or other programmable device to cause a series of operations to be performed on the computer or other programmable device, thereby producing a computer-implemented process, such that the instructions executed on the computer or other programmable device provide the operations for implementing the functions specified in sequence diagram 1000.

[0171] Therefore, the steps of sequence diagram 1000 support a combination of means for performing the specified functions and a combination of operations for performing the specified functions. It should also be understood that one or more steps in sequence diagram 1000 and a combination of steps in sequence diagram 1000 can be implemented by a computer system based on dedicated hardware that performs the specified functions or a combination of dedicated hardware and computer instructions. Figure 10 The sequence diagram 1000 is used to generate and transmit a 3D model. Fewer, more, or different steps may be provided.

[0172] Figure 11 is a schematic diagram 1100 depicting an exemplary environment for capturing multiple 2D scanned images 110 and presenting interactive 3D graphical representations 1110 in real time, according to an exemplary embodiment. Figure 11 Combine Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 、 Figure 6 、 Figure 7 、 Figure 8 、 Figure 9 and Figure 10 Schematic diagram 1100 may include a dentist 1102 and a patient 1104.

[0173] The dentist 1102 may use the handheld intraoral scanning device 104 to capture a plurality of 2D scan images 110 of the dental arch 1106 of the patient 1104. The handheld intraoral scanning device 104 may process the captured plurality of 2D scan images 110 and generate 3D surface information 510 of the dental arch 1106 of the patient 1104.

[0174] The dentist 1102 can enter the identification number of the handheld intraoral scanning device 104 on a web browser of a computer 1108 (e.g., a client device). The handheld intraoral scanning device 104 and the computer 1108 can be connected on a public web network and through one of the communication channels 108 based on the identification number forwarded to the handheld intraoral scanning device 104.

[0175] Once the handheld intraoral scanning device 104 and the computer 1108 are connected, the 3D surface information 510 of the dental arch 1106 can be transmitted to the computer 1108. A web application can be displayed on the web browser of the computer 1108. The multiple 2D scan images 110 captured by the dentist 1102 can be displayed on the web application. Furthermore, an interactive 3D graphical representation 1110 can be generated in real time based on the 3D surface information 510 and displayed on the web application. The interactive 3D graphical representation 1110 can be manipulated by the dentist 1102 as desired, such as being rotated or viewed from multiple perspectives.

[0176] Thus, the intraoral scanning system 102 can process multiple 2D scan images 110 independently of any external device during scan registration. A user (e.g., a dentist) can obtain visual feedback during the scanning session and access a web server interface on the handheld intraoral scanning device 104 through one or more client devices 106. Furthermore, 3D surface information 510 can be broadcast to present an interactive 3D graphical representation 1110 on one or more client devices 106. Because the 3D surface information 510 can be stored on the handheld intraoral scanning device 104, the user can switch to a different client device while scanning. If the connection between the handheld intraoral scanning device 104 and one or more client devices 106 is lost, the scanning session can be resumed without losing any scan data.

[0177] Figure 12 4 is a schematic diagram 400 illustrating an environment for a handheld intraoral scanning device 104 to communicate with one or more client devices 106 , according to an exemplary embodiment. Figure 12 Combine Figure 1 、 Figure 2 、 Figure 3 and Figure 4. In this example, the schematic 400 may include an intermediate processing unit 1201 configured to communicate with the handheld intraoral scanning device 104 via a wireless link 1202 and to communicate with one or more client devices (402, 404, 406) via a communication channel 108. The wireless link 1202 may be based on a Bluetooth protocol or a WIFI protocol. The bandwidth of the wireless link 1202 is greater than that of the communication channel 108. The intermediate processing unit is configured to partially process the 3D surface information received from the handheld intraoral scanning device to reduce the bandwidth required for the communication channel 108. The partial processing of the 3D surface information may involve compressing the information according to a compression protocol (e.g., H.265, which is an efficient video encoding operation). The partial processing of the 3D surface information may also include deleting 3D surface information that is not relevant to the 3D model representation of the dental arch.

[0178] Since scan registration and rendering no longer rely on a high-performance computer (such as a Power PC), the PowerPC or server can be used solely for heavy computing. In addition, the intraoral scanning system 102 is capable of processing multiple scans in parallel. Therefore, the number of scans that can be post-processed simultaneously can depend only on the processing power of a single server, so multiple high-performance computers are not necessarily required. In addition, post-processing can be delegated to servers in the cloud. Therefore, the intraoral scanning system 102 provides intraoral scan registration by using edge computing technology, thereby transferring processing to the handheld intraoral scanning device 104.

[0179] Many modifications and other embodiments of the invention described herein will be apparent to those skilled in the art after having the benefit of the teachings presented in the foregoing description and the associated drawings. Therefore, it should be understood that the invention is not limited to the specific embodiments disclosed, and modifications and other embodiments are intended to be included within the scope of the appended claims. In addition, although the foregoing description and the associated drawings describe exemplary embodiments in the context of specific elements and / or functional combinations, it should be understood that other embodiments may provide different elements and / or functional combinations without departing from the scope of the appended claims. In this regard, for example, in addition to the combinations explicitly described above, some elements and / or functional combinations that may be proposed in the appended claims may also be considered. Although specific terms are used herein, they are used in a general and descriptive sense only and not for limiting purposes.

[0180] Project List:

[0181] 1. An intraoral scanning system (102), comprising:

[0182] A handheld intraoral scanning device (104) is configured as follows:

[0183] capturing a plurality of two-dimensional (2D) scan images (110) during a scanning session of a dental arch (506);

[0184] providing three-dimensional (3D) surface information (510) based on a plurality of two-dimensional scan images (110) captured during a scanning session, and

[0185] wherein the handheld intraoral scanning device (104) includes a web server interface (206) configured to communicate via a web network and establish a connection to one or more wireless full-duplex communication channels; and

[0186] One or more client devices (106), wherein each of the one or more client devices (106) is configured to:

[0187] establishing a connection to one of the one or more wireless full-duplex communication channels by forwarding the identification number to the handheld intraoral scanning device (104) via the web network;

[0188] receiving 3D surface information via the one of the one or more wireless full-duplex communication channels (510); and

[0189] The 3D surface information (510) is rendered as an interactive 3D graphical representation (112) compatible with a web browser (602).

[0190] 2. The intraoral scanning system (102) of item 1, wherein the web server interface (206) and the one or more client devices (106) are connected to a public web network.

[0191] 3. The intraoral scanning system (102) of any preceding claim, wherein a plurality of the one or more client devices (106) are configured to:

[0192] receiving 3D surface information via a wireless full-duplex communication channel of one or more wireless full-duplex communication channels (510); and

[0193] The 3D surface information (510) is rendered as an interactive 3D graphical representation (112) compatible with a web browser (602).

[0194] 4. The intraoral scanning system (102) according to any of the preceding claims, wherein the one or more client devices (106) are at least one of the following: a display unit, a tablet computer, or a smartphone.

[0195] 5. The intraoral scanning system (102) according to any one of items 1 to 3, wherein the one or more client devices (106) are computers.

[0196] 6. The intraoral scanning system (102) according to any of the preceding claims, wherein the bandwidth of the one of the one or more wireless full-duplex communication channels is monitored by a monitoring unit (208) of the intraoral scanning system (102), and

[0197] When the bandwidth is below a minimum bandwidth, the handheld intraoral scanning device (104) is configured to downsample the 3D surface information (510) for transmission over the one of the one or more wireless full-duplex communication channels.

[0198] 7. The intraoral scanning system (102) of item 6, wherein the bandwidth of the one of the one or more wireless full-duplex communication channels is monitored by a monitoring unit (208) of the intraoral scanning system (102), and

[0199] The handheld intraoral scanning device (104) includes a temporary storage unit (210) configured to:

[0200] When it is determined that the bandwidth of the one of the one or more wireless full-duplex communication channels is below a minimum bandwidth, storing the 3D surface information (510); and

[0201] When it is determined that the bandwidth is higher than or equal to the minimum bandwidth, the stored 3D surface information is transmitted (510).

[0202] 8. The intraoral scanning system (102) of item 7, wherein when the bandwidth of one of the one or more wireless full-duplex communication channels is lower than a minimum bandwidth for a duration longer than a maximum period, the handheld intraoral scanning device (104) is configured to compress the 3D surface information (510) and store it in a memory storage unit (204) of the handheld intraoral scanning device (104).

[0203] 9. The intraoral scanning system (102) according to item 7, wherein:

[0204] The monitoring unit (208) is configured to determine when connection to the one of the one or more wireless full-duplex communication channels is lost; and

[0205] The handheld intraoral scanning device (104) is configured to compress and store the 3D surface information (510) in a memory unit (204) of the handheld intraoral scanning device (104) based on the determination of the connection loss.

[0206] 10. The intraoral scanning system (102) of item 9, wherein the handheld intraoral scanning device (104) is configured to transmit the 3D surface information (510) stored in the memory unit (204) via the wireless communication interface of the intraoral scanning system (102) upon completion of the scanning session.

[0207] 11. The intraoral scanning system (102) according to any one of items 7 to 10, wherein the handheld intraoral scanning device (104) further comprises:

[0208] A monitoring unit (208) configured to input the transmission status based on at least one of: determining that the bandwidth is less than a minimum bandwidth, or determining that a connection to the one of the one or more wireless full-duplex communication channels is lost; and

[0209] A scanning feedback unit (212) is configured to:

[0210] receiving status input from a monitoring unit (208), and

[0211] While receiving the status input, a scan feedback signal is provided to a user of the handheld intraoral scanning device (104), wherein the scan feedback signal is configured to provide guidance to the user to areas of the scanning session of the dental arch (506) where the scan quality is low and no 3D surface information (510) can be provided.

[0212] 12. The intraoral scanning system (102) of item 11, wherein the scanning feedback signal comprises an acoustic feedback signal configured to guide the user toward the region of the dental arch (506).

[0213] 13. The intraoral scanning system (102) according to item 11, wherein the scanning feedback signal includes at least one of the following: tactile feedback, or light emitted by multiple light emitting diodes of the handheld intraoral scanning device (104).

[0214] 14. The intraoral scanning system (102) of item 13, wherein the handheld intraoral scanning device (104) includes a vibrator (214B) configured to provide tactile feedback, and wherein an increase in vibration indicates an increase in the distance between the area of ​​the dental arch (506) and the handheld intraoral scanning device (104), and wherein a decrease in vibration indicates a decrease in the distance between the area of ​​the dental arch (506) and the handheld intraoral scanning device (104).

[0215] 15. The intraoral scanning system (102) according to item 13, wherein the plurality of light emitting diodes (214C) are divided into a left group of light emitting diodes and a right group of light emitting diodes, and

[0216] Wherein, when the handheld intraoral scanning device (104) is respectively arranged on the right side or the left side of the area of ​​the dental arch (506), the left group and the right group are configured to emit flashes.

[0217] 16. An intraoral scanning system (102) according to any of the preceding items, wherein the handheld intraoral scanning device (104) is configured to broadcast the 3D surface information (510) to a plurality of client devices among the one or more client devices (106) connected to the handheld intraoral scanning device (104) via the one of the one or more wireless full-duplex communication channels.

[0218] 17. The intraoral scanning system (102) according to any of the preceding claims, wherein the handheld intraoral scanning device (104) is configured to generate a 3D model representation of the dental arch (506) by combining a plurality of 3D surface information (510) provided by the handheld intraoral scanning device (104), and wherein,

[0219] The handheld intraoral scanning device (104) is configured to transmit the 3D model via the one of the one or more wireless full-duplex communication channels.

[0220] 18. A method (700) for intraoral scan registration, comprising:

[0221] capturing (712) a plurality of two-dimensional (2D) scan images (110) during a scanning session of a dental arch (506) via a handheld intraoral scanning device (104);

[0222] providing (714) three-dimensional (3D) surface information (510) based on a plurality of two-dimensional scan images (110) captured during a scanning session via a handheld intraoral scanning device (104);

[0223] establishing (710) a connection to one of the one or more wireless full-duplex communication channels by forwarding (708) the identification number to the handheld intraoral scanning device (104) via a web network, by one or more client devices (106);

[0224] receiving (716), by one or more client devices (106), 3D surface information (510) via the one of the one or more wireless full-duplex communication channels; and

[0225] The 3D surface information (510) is rendered (718) on one or more client devices (106) as an interactive 3D graphical representation (112) compatible with a web browser (602).

[0226] 19. An intraoral scanning system (102), comprising:

[0227] A handheld intraoral scanning device (104) configured to:

[0228] capturing a first plurality of two-dimensional (2D) scan images and a second plurality of two-dimensional scan images containing surface information of a dental arch (506) of a patient during a first time window and a second time window, respectively, and wherein the first time window precedes the second time window;

[0229] processing the first plurality of two-dimensional scan images and the second plurality of two-dimensional scan images into first three-dimensional (3D) surface information and second three-dimensional surface information, respectively;

[0230] Generate a first three-dimensional scan patch and a second three-dimensional scan patch by transforming the first three-dimensional surface information into first real-world three-dimensional coordinates and first texture information, and transforming the second three-dimensional surface information into second real-world three-dimensional coordinates and second texture information, using calibration data stored on the memory unit (204);

[0231] registering the second 3D scan patch to at least the first 3D scan patch by locating corresponding data points between the first 3D scan patch and the second 3D scan patch;

[0232] fusing the first 3D scan patch and the second 3D scan patch to form a 3D model;

[0233] storing the first texture information and the second texture information together with the formed 3D model, and

[0234] wherein the handheld intraoral scanning device (104) includes a web server interface (206) configured to communicate via a web network and establish a connection to one or more wireless full-duplex communication channels; and

[0235] One or more client devices (106), wherein each of the one or more client devices (106) is configured to:

[0236] establishing a connection to one of the one or more wireless full-duplex communication channels by forwarding the identification number to the handheld intraoral scanning device (104) via the web network;

[0237] receiving the 3D model via the one of the one or more wireless full-duplex communication channels; and

[0238] The 3D model is rendered into an interactive 3D graphical representation (112) compatible with a web browser (602).

[0239] 20. A method for intraoral scan registration, comprising:

[0240] capturing, by a handheld intraoral scanning device (104), a first plurality of two-dimensional (2D) scanned images and a second plurality of two-dimensional scanned images containing surface information of a dental arch (506) of a patient during a first time window and a second time window, respectively, wherein the first time window is before the second time window;

[0241] Processing the first plurality of two-dimensional scan images and the second plurality of two-dimensional scan images into first three-dimensional (3D) surface information and second three-dimensional surface information respectively by a handheld intraoral scanning device (104);

[0242] By using the handheld intraoral scanning device (104), the first 3D surface information is transformed into first real-world 3D coordinates and first texture information, and the second 3D surface information is transformed into second real-world 3D coordinates and second texture information, thereby generating a first 3D scan patch and a second 3D scan patch;

[0243] registering the second 3D scan patch to at least the first 3D scan patch by locating corresponding data points between the first 3D scan patch and the second 3D scan patch using a handheld intraoral scanning device (104);

[0244] fusing the first 3D scanned surface piece and the second 3D scanned surface piece together to form a 3D model using a handheld intraoral scanning device (104);

[0245] storing the first texture information and the second texture information together with the formed 3D model by a handheld intraoral scanning device (104);

[0246] establishing, by the one or more client devices, a connection to one of the one or more wireless full-duplex communication channels by forwarding the identification number to the handheld intraoral scanning device via the web network;

[0247] receiving, by the one or more client devices, the 3D model via the one of the one or more wireless full-duplex communication channels; and

[0248] The 3D model is rendered as an interactive 3D graphical representation (112) compatible with a web browser (602) on one or more client devices.

[0249] 21. A computer programmable product comprising a non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed by a processing circuit, cause the processing circuit to perform operations comprising:

[0250] capturing (902) a first plurality of two-dimensional (2D) scan images and a second plurality of two-dimensional scan images containing surface information of a dental arch (506) of a patient during a first time window and a second time window, respectively, wherein the first time window precedes the second time window;

[0251] processing the first plurality of two-dimensional scan images and the second plurality of two-dimensional scan images into first three-dimensional (3D) surface information and second three-dimensional surface information, respectively;

[0252] generating (906) a first three-dimensional scan patch and a second three-dimensional scan patch by transforming the first three-dimensional surface information into first real-world three-dimensional coordinates and first texture information, and transforming the second three-dimensional surface information into second real-world three-dimensional coordinates and second texture information, using the calibration data;

[0253] registering (908) the second 3D scan patch to at least the first 3D scan patch by locating corresponding data points between the first 3D scan patch and the second 3D scan patch;

[0254] fusing (910) the first 3D scan patch and the second 3D scan patch together to form a 3D model; and

[0255] The first texture information and the second texture information are stored together with the formed 3D model (912).

Claims

1. An intraoral scanning system (102), comprising: A handheld intraoral scanning device (104), the handheld intraoral scanning device being configured to: capturing a plurality of two-dimensional (2D) scan images (110) during a scanning session of a dental arch (506); providing three-dimensional (3D) surface information (510) based on the plurality of two-dimensional scan images (110) captured during the scanning session, and wherein the handheld intraoral scanning device (104) includes a web server interface (206) configured to communicate via a web network and establish a connection to one or more wireless full-duplex communication channels; and One or more client devices (106), wherein each of the one or more client devices (106) is configured to: establishing a connection to one of the one or more wireless full-duplex communication channels by forwarding the identification number to the handheld intraoral scanning device (104) via the web network; receiving the 3D surface information via the one of the one or more wireless full-duplex communication channels (510); and The 3D surface information (510) is rendered as an interactive 3D graphical representation (112) compatible with a web browser (602).

2. The intraoral scanning system (102) according to claim 1, wherein: The web server interface (206) and the one or more client devices (106) are connected to a public web network.

3. The intraoral scanning system (102) according to any one of the preceding claims, wherein: A plurality of the one or more client devices (106) are configured to: receiving the 3D surface information via a wireless full-duplex communication channel of the one or more wireless full-duplex communication channels (510); and The 3D surface information (510) is presented as an interactive 3D graphical representation (112) compatible with the web browser (602).

4. The intraoral scanning system (102) according to any one of the preceding claims, wherein: The one or more client devices (106) are at least one of: a display unit, a tablet computer, or a smartphone.

5. The intraoral scanning system (102) according to any one of claims 1 to 3, wherein: The one or more client devices (106) are computers.

6. The intraoral scanning system (102) according to any one of the preceding claims, wherein: The bandwidth of the one of the one or more wireless full-duplex communication channels is monitored by a monitoring unit (208) of the intraoral scanning system (102), and When the bandwidth is below a minimum bandwidth, the handheld intraoral scanning device (104) is configured to downsample the 3D surface information (510) for transmission over the one of the one or more wireless full-duplex communication channels.

7. The intraoral scanning system (102) according to claim 6, wherein: The bandwidth of the one of the one or more wireless full-duplex communication channels is monitored by the monitoring unit (208) of the intraoral scanning system (102), and The handheld intraoral scanning device (104) includes a temporary storage unit (210), wherein the temporary storage unit is configured to: When it is determined that the bandwidth of the one of the one or more wireless full-duplex communication channels is below a minimum bandwidth, storing the 3D surface information (510); and When it is determined that the bandwidth is higher than or equal to the minimum bandwidth, the stored 3D surface information is transmitted (510).

8. The intraoral scanning system (102) according to claim 7, wherein: When the bandwidth of the one of the one or more wireless full-duplex communication channels is below a minimum bandwidth for a duration longer than a maximum period, the handheld intraoral scanning device (104) is configured to compress the 3D surface information (510) and store it in a memory unit (204) of the handheld intraoral scanning device (104).

9. The intraoral scanning system (102) according to claim 7, wherein: The monitoring unit (208) is configured to determine when connection to the one of the one or more wireless full-duplex communication channels is lost; and The handheld intraoral scanning device (104) is configured to compress the 3D surface information (510) and store it in a memory unit (204) of the handheld intraoral scanning device (104) based on the determination of connection loss.

10. The intraoral scanning system (102) according to claim 9, wherein: The handheld intraoral scanning device (104) is configured to transmit the 3D surface information (510) stored in the memory unit (204) via the wireless communication interface of the intraoral scanning system (102) upon completion of the scanning session.

11. The intraoral scanning system (102) according to any one of claims 7 to 10, wherein: The handheld intraoral scanning device (104) further includes: The monitoring unit (208) is configured to input based on at least one of the following transmission status: determining that the bandwidth is less than a minimum bandwidth, or determining that a connection to the one of the one or more wireless full-duplex communication channels is lost; and A scanning feedback unit (212), wherein the scanning feedback unit is configured to: receiving said status input from said monitoring unit (208), and While receiving the status input, a scanning feedback signal is provided to a user of the handheld intraoral scanning device (104), wherein the scanning feedback signal is configured to provide guidance to the user to areas of the dental arch (506) where the scan quality of the scanning session is low and the 3D surface information (510) cannot be provided.

12. The intraoral scanning system (102) according to any one of the preceding claims, wherein: The handheld intraoral scanning device (104) is configured to broadcast the 3D surface information (510) to a plurality of client devices among the one or more client devices (106) connected to the handheld intraoral scanning device (104) via the one of the one or more wireless full-duplex communication channels.

13. The intraoral scanning system (102) according to any one of the preceding claims, wherein: The handheld intraoral scanning device (104) is configured to generate a 3D model representation of a dental arch (506) by combining a plurality of 3D surface information (510) provided by the handheld intraoral scanning device (104), and wherein, The handheld intraoral scanning device (104) is configured to transmit the 3D model via the one of the one or more wireless full-duplex communication channels.

14. A method (700) for intraoral scan registration, comprising: capturing (712) a plurality of two-dimensional (2D) scan images (110) during a scanning session of a dental arch (506) via a handheld intraoral scanning device (104); providing (714) three-dimensional (3D) surface information (510) based on the plurality of 2D scan images (110) captured during the scanning session via a handheld intraoral scanning device (104); establishing (710) a connection to one of the one or more wireless full-duplex communication channels by forwarding (708) the identification number to the handheld intraoral scanning device (104) via a web network via one or more client devices (106); receiving (716), by the one or more client devices (106), the 3D surface information (510) via the one of the one or more wireless full-duplex communication channels; as well as The 3D surface information (510) is rendered (718) on the one or more client devices (106) as an interactive 3D graphical representation (112) compatible with a web browser (602).

15. The method of claim 14, comprising connecting the web server interface (206) and the one or more client devices (106) to a public web network.

Images