Generating high resolution scans of oral lumens

Abstract

The invention relates to generating a high resolution scan of an oral lumen. An aspect includes generating a high resolution surface scan of an oral lumen, including sampling a surface of the oral lumen by capturing a plurality of individual images of the surface with a scanner, detecting a bandwidth of transmission, upon detecting that the bandwidth is below a threshold, reducing the resolution of the one or more individual images by converting the individual images into corresponding low-resolution frames and residual frames, transmitting the corresponding low-resolution frames, sequentially stitching the low-resolution frames over a first time period to continue the surface scan to generate a stitched model, and selectively replacing the low resolution frames with the corresponding residual frames within the stitched model for a second time period. A high resolution surface scan of the oral lumen is then generated.

Description

Technical Field

[0001] The present disclosure generally relates to intraoral scanning, and more particularly to generating high-resolution scans of an oral cavity under variable bandwidth conditions. Background Art

[0002] In the field of intraoral imaging in a wireless scenario, digital impression using an intraoral scanner faces challenges when the transmission bandwidth intermittently drops below the rate of acquiring data, thereby impeding seamless data transmission. Traditional methods adapt to the amount of data in the transmission bandwidth by discarding the captured frames. However, this may introduce interruptions in the scan stream, forcing the operator to intervene by repositioning the scanner and, in some cases, re-acquiring a specific section of the oral cavity. These interruptions not only impede the efficiency of the scanning process but also result in a suboptimal user experience for users and patients, etc. Summary of the Invention

[0003] According to an embodiment of the present disclosure, a method is disclosed that includes sampling a surface of an oral cavity by capturing a plurality of single images of the surface using a scanner, detecting the transmission bandwidth, reducing the resolution of one or more single images associated with at least one region of the surface by converting the one or more single images into corresponding low-resolution frames and residual frames associated with the at least one region when the detected bandwidth is below a predetermined threshold, and sequentially transmitting the corresponding low-resolution frames to a receiver at a bandwidth below the predetermined threshold. Additionally, the method includes the receiver sequentially stitching the corresponding low-resolution frames during a first time period to continue the surface scan to generate a stitched model, during a second time period after the first time period, selectively updating the corresponding low-resolution frames within the stitched model based on the corresponding residual frames, and generating a high-resolution surface scan of the oral cavity. The update may include combining the residual frame with the low-resolution frame to retrieve the original high-resolution frame. Alternatively, if the original high-resolution frame is retained, the low-resolution frame may be replaced.

[0004] In one embodiment, the method may further include registering and stitching the corresponding low-resolution frames associated with the at least one region with a plurality of single images of other regions in sequence to generate a stitched model of the surface.

[0005] In one embodiment, the method may further include sequentially sending each low-resolution frame with an associated ID, storing the corresponding residual frames of the one or more single images in an image buffer of the scanner, identifying at least one region having a sampling rate lower than a predetermined sampling rate, and using the associated ID of each low-resolution frame to request reception of the corresponding residual frames associated with the identified at least one region.

[0006] In one embodiment, the method may further include maintaining a queue of the captured individual images for transmission to a receiver, detecting a queue length that exceeds a predetermined maximum queue length, and determining that the bandwidth is below a predetermined threshold.

[0007] In one embodiment, the method may further include maintaining a queue of the captured individual images for sending to a receiver, detecting a queue length that is below a predetermined maximum queue length, and determining that the bandwidth is above a predetermined threshold.

[0008] Aspects described below include a non-transitory computer-readable storage medium that includes computer-executable instructions that, in response to execution by a processor, cause a system to perform any of the methods described.

[0009] Aspects described below also include a system that has a device for generating a high-resolution surface scan of an oral cavity. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The apparatus and techniques for generating a high-resolution surface scan of an oral cavity are described with reference to the following drawings. The same numbers are used throughout the drawings to refer to like features and components. The drawings are illustrative embodiments. They do not show all embodiments. Additionally or alternatively, other embodiments may be used. Details that may be apparent or unnecessary may be omitted to save space or for more effective illustration. Some embodiments may be implemented with additional components or steps and / or without all of the components or steps shown. When the same number appears in different drawings, it refers to the same or similar components or steps.

[0011] Figure 1 A block diagram depicting a data processing environment in which illustrative embodiments may be implemented.

[0012] Figure 2 A block diagram depicting a data processing system in which illustrative embodiments may be implemented.

[0013] Figure 3 A schematic diagram depicting a system in which illustrative embodiments may be implemented.

[0014] Figure 4 A flowchart depicting a method for generating a high-resolution surface scan in accordance with an illustrative embodiment of the present disclosure.

[0015] Figure 5 A flowchart depicting a process for generating a high-resolution surface scan in accordance with an illustrative embodiment of the present disclosure.

[0016] Figure 6 A flowchart depicting a process for replacing or stitching together low-resolution frames with residual frames in accordance with an illustrative embodiment of the present disclosure. Detailed Implementation Modes

[0017] Overview

[0018] In the following detailed description, numerous specific details are set forth by way of example in order to provide a thorough understanding of the relevant teachings. It should be understood, however, that the teachings of the present invention may be practiced without these details. In other instances, well-known methods, procedures, and / or components have been described at a relatively high level without detailed description in order to avoid unnecessarily obscuring aspects of the teachings.

[0019] Illustrative embodiments recognize that in the field of intraoral imaging, the utilization of intraoral scanners can be integral for obtaining digital impressions of the oral cavity. These scanners can play a key role in transmitting the recorded data, typically utilizing various connection options such as wireless networks or the public Internet. However, significant challenges can arise when the transmission bandwidth intermittently drops below the rate at which data is acquired, resulting in an interruption of smooth data transmission.

[0020] Illustrative embodiments recognize that an attempt can be made to send a continuous data stream, and when experiencing an intermittent drop in transmission bandwidth, frames can be discarded to adjust to the available bandwidth. However, discarding frames can lead to an interruption of the scan stream, thereby generating a degraded or incomplete surface scan and losing sampled data for certain regions of the oral cavity.

[0021] Illustrative embodiments implement a method and system for generating a high-resolution surface scan of the oral cavity. As used herein, "high resolution" generally can refer to a resolution that exceeds a predetermined threshold resolution, and "low resolution" generally can refer to a resolution that is below a predetermined threshold resolution.

[0022] In an exemplary aspect, a method of generating a high-resolution surface scan of an oral cavity is disclosed. The method includes: sampling the surface of the oral cavity by capturing a plurality of individual images of the surface using an intraoral scanner; detecting a transmission bandwidth between the intraoral scanner and a remote receiver; and reducing the resolution of one or more of the individual images in response to detecting that the bandwidth is below a predetermined threshold. The individual images can be individual "2D images" or individual "3D depth images". The resolution of one or more of the individual images can be reduced by converting the one or more individual images into corresponding one or more low-resolution frames and corresponding one or more residual frames. During a first time period when the bandwidth is low, the one or more low-resolution frames can be sent one by one to the remote receiver and sequentially stitched into a model generated by the receiver. When the bandwidth is low (below the predetermined threshold), transmitting and stitching the low-resolution frames can allow the intraoral surface scan to continue to generate a stitched model without experiencing interruptions caused by low-bandwidth transmission. During a second time period that begins after the first time period has started, the low-resolution frames of the stitched model can be selectively updated (such as replaced or combined) within the stitched model with the corresponding residual frames to obtain the original high-resolution data of the intraoral scanner and generate a high-resolution overall 3D image of the oral cavity. In one aspect, the high-resolution overall 3D image can achieve the same quality and accuracy as if all high-resolution frames from the camera had been transmitted at full bandwidth. Additionally, the selective update can be based on a predetermined threshold of the desired sampling density such that a predetermined quality can be achieved.

[0023] Exemplary embodiments are described for certain types of machines. Illustrative embodiments are also described only by way of example with respect to other scenarios, objects, measurements, devices, data processing systems, environments, components, and applications. Any particular manifestation of these and other similar artifacts is not intended to limit the present disclosure. Any suitable manifestation of these and other similar artifacts can be selected within the scope of the exemplary embodiments.

[0024] Furthermore, the illustrative embodiments can be implemented for any type of data, data source, or access to a data source over a data network. Within the scope of the present disclosure, any type of data storage device can provide data to embodiments of the present disclosure locally at a data processing system or over a data network. In cases where embodiments are described using a mobile device, within the scope of the illustrative embodiments, any type of data storage device suitable for use with a mobile device can provide data to such embodiments locally at the mobile device or over a data network.

[0025] The use of specific codes, hardware, algorithms, designs, architectures, protocols, layouts, diagrams, and tools is only used as an example to describe illustrative embodiments, rather than a limitation on the illustrative embodiments. In addition, for the sake of clarity in description, in some instances, specific software, tools, and data processing environments are only used as examples to describe illustrative embodiments. Illustrative embodiments can be used in combination with other structures, systems, applications, or architectures for equivalent or similar purposes. For example, within the scope of the present disclosure, other equivalent devices, structures, systems, applications, or their architectures can be used in combination with such embodiments of the present disclosure. Illustrative embodiments can be implemented in hardware, software, or a combination thereof.

[0026] The examples in the present disclosure are only for clear description and are not limited to the illustrative embodiments. Additional data, operations, actions, tasks, activities, and manipulations can be conceived from the present disclosure, and these additional data, operations, actions, tasks, activities, and manipulations can be contemplated within the scope of the illustrative embodiments.

[0027] Any advantages listed herein are only examples and are not intended to limit the illustrative embodiments. Additional or different advantages can be achieved through specific illustrative embodiments. In addition, a specific illustrative embodiment may have some, all, or none of the advantages listed above.

[0028] Referring to the accompanying drawings, and in particular, referring to Figure 1 and Figure 2 , these drawings are example diagrams of a data processing environment in which the illustrative embodiments can be implemented. Figure 1 and Figure 2 are only examples and are not intended to assert or imply any limitations on the environment in which different embodiments can be implemented. Many modifications can be made to the depicted environment based on the following description for a particular implementation.

[0029] Figure 1 FIG. depicts a block diagram of a network of a data processing system in which the illustrative embodiments can be implemented. The data processing environment 100 is a computer network in which the illustrative embodiments can be implemented. The data processing environment 100 includes a network 102. The network 102 is a medium for providing a communication link between various devices and computers connected together within the data processing environment 100. The network 102 can include connections such as wired, wireless communication links, or fiber optic cables.

[0030] The client or server is merely an example role of certain data processing systems connected to network 102 and is not intended to exclude other configurations or roles of these data processing systems. Server 104 and server 106, along with storage unit 108, are coupled to network 102. Software applications can be executed on any computer in data processing environment 100. Clients 110, clients 112, and clients 114 are also coupled to network 102. A data processing system such as server 104 or server 106 or a client (clients 110, clients 112, clients 114) can include data and can have software applications or software tools executed thereon. Server 104 can include one or more GPUs (graphics processing units) for training one or more models.

[0031] Merely by way of example and not implying any limitation to such an architecture, Figure 1 depicts certain components that can be used in an example implementation of an embodiment. For example, servers and clients are merely examples and do not imply a limitation to the client-server architecture. As another example, one embodiment can be distributed across several data processing systems and data networks as shown, while another embodiment can be implemented on a single data processing system within the scope of the illustrative embodiment. The data processing systems (server 104, server 106, clients 110, clients 112, clients 114) also represent example nodes in clusters, partitions, and other configurations suitable for implementing embodiments.

[0032] Device 120 is an example of the devices described herein. Any software application described as being executed in Figure 1 another data processing system can be configured to be executed in any device here in a similar manner. Any data or information stored or generated in Figure 1 another data processing system can be configured to be stored or generated in any device here in a similar manner.

[0033] The surface scan generation component 124 can execute as part of the client application 122, the server application 116, or on any data processing system herein. The surface scan generation component 124 can also execute as a cloud service communicatively coupled to the system services, hardware resources, or software elements described herein. The database 118 of the storage unit 108 stores one or more data in a repository for the computations herein. The surface scan generation component 124 can execute a method that includes: sampling a surface by capturing a plurality of individual images of the surface of the oral cavity with a scanner, detecting a transmission bandwidth, and when the detected bandwidth is below a predetermined threshold, reducing the resolution of one or more of the individual images by converting one or more of the individual images associated with at least one region of the surface into corresponding low-resolution frames and residual frames associated with the at least one region, and transmitting the corresponding low-resolution frames to a receiver at the bandwidth. The surface scan generation component 124 can execute stitching of the corresponding low-resolution frames to an in-progress model in a first time period to continue the surface scan to generate a stitched model, and selectively replace the corresponding low-resolution frames with the corresponding residual frames within the stitched model and generate a high-resolution surface scan of the oral cavity in a second time period.

[0034] The server application 116 implements the embodiments described herein. The server application 116 can use data from the storage unit 108 to generate a high-resolution surface scan of the oral cavity. The server application 116 can also obtain data from any client for computations. The server application 116 can also execute in any data processing system (server 104 or server 106, client 110, client 112, client 114), such as the client application 122 in client 112, and does not need to execute in the same system as server 104.

[0035] The server 104, server 106, storage unit 108, client 110, client 112, client 114, and device 120 can be coupled to the network 102 using a wired connection, wireless communication protocol, or other suitable data connection. The clients 110, 112, and 114 can be, for example, personal computers or network computers.

[0036] In the depicted example, server 104 can provide data, such as boot files, operating system images, and applications, to clients 110, 112, and 114. In this example, clients 110, 112, and 114 can be clients of server 104. Clients 110, 112, and 114 or some combination thereof can include their own data, boot files, operating system images, and applications. Data processing environment 100 can include additional servers, clients, and other devices not shown. Server 104 includes server application 116, which can be configured to implement one or more of the functions described herein in accordance with one or more embodiments.

[0037] Data processing environment 100 can also be the Internet. Network 102 can represent a collection of networks and gateways that communicate with each other using Transmission Control Protocol / Internet Protocol (TCP / IP) and other protocols. At the core of the Internet is a backbone of data communication links between major nodes or host computers, including thousands of commercial, government, educational, and other computer systems that route data and messages. Of course, data processing environment 100 can also be implemented as many different types of networks, such as an intranet, a local area network (LAN), or a wide area network (WAN). Figure 1 This is intended to be an example, and not an architectural limitation on different illustrative embodiments.

[0038] Among other uses, data processing environment 100 can be used to implement a client-server environment in which exemplary embodiments can be implemented. A client-server environment enables software applications and data to be distributed across a network so that the applications work by using the interactivity between client data processing systems and server data processing systems. Data processing environment 100 can also adopt a service-oriented architecture, in which interoperable software components distributed across a network can be packaged together as a coherent business application. Data processing environment 100 can also take the form of a cloud and adopt a cloud computing model for service delivery to enable convenient on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services), which can be rapidly provisioned and released with minimal management effort or interaction with the service provider.

[0039] Refer to Figure 2 , which depicts a block diagram of a data processing system in which illustrative embodiments can be implemented. Data processing system 200 is an example of a computer, such as Figure 1The server 104, server 106, or client (scanner unit / acquisition unit) 110, client 112, client 114, surface scan generation component 124 in [the above], or another type of device that can have computer-usable program code or instructions for implementing the processing of the exemplary embodiments

[0040] The data processing system 200 also represents a data processing system or a configuration therein, such as Figure 1 the device 120 in [the above], where computer-usable program code or instructions for implementing the processes of the exemplary embodiments can be established. The data processing system 200 is described only as an example as a computer and is not limited thereto. Without departing from the general description of the operation and function of the data processing system 200 described herein, embodiments in the form of other devices such as Figure 1 the device 120 in [the above] can modify the data processing system 200, for example, by adding a touch interface, and even removing some of the depicted components from the data processing system 200.

[0041] In the depicted example, the data processing system 200 employs a hub architecture, including a north bridge and a memory controller hub (NB / MCH) 202, and a south bridge and an input / output (I / O) controller hub (SB / ICH) 204. The processing unit 206, main memory 208, and graphics processor 210 are coupled to the north bridge and memory controller hub (NB / MCH) 202. The processing unit 206 can include one or more processors and can be implemented using one or more heterogeneous processor systems. The processing unit 206 can be a multi-core processor. In certain embodiments, the graphics processor 210 can be coupled to the north bridge and memory controller hub (NB / MCH) 202 through an Accelerated Graphics Port (AGP).

[0042] In the depicted example, a local area network (LAN) adapter 212 is coupled to a south bridge and an input / output (I / O) controller hub (SB / ICH) 204. An audio adapter 216, a keyboard and mouse adapter 220, a modem 222, a read-only memory (ROM) 224, a universal serial bus (USB) and other ports 232, and PCI / PCIe devices 234 are coupled to the south bridge and the input / output (I / O) controller hub (SB / ICH) 204 via a bus 218. A hard disk drive (HDD) or a solid state drive (SSD) 226a and a CD-ROM 230 are coupled to the south bridge and the input / output (I / O) controller hub (SB / ICH) 204 via a bus 228. The PCI / PCIe devices 234 can include, for example, an Ethernet adapter, an add-in card, and a PC card for a notebook computer. PCI uses a card bus controller, while PCIe does not. The read-only memory (ROM) 224 can be, for example, a flash binary input / output system (BIOS). The hard disk drive (HDD) or the solid state drive (SSD) 226a and the CD-ROM 230 can use, for example, an integrated drive electronics (IDE), a serial advanced technology attachment (SATA) interface, or variants such as external SATA (eSATA) and micro SATA (mSATA). A super I / O (SIO) device 236 can be coupled to the south bridge and the input / output (I / O) controller hub (SB / ICH) 204 via the bus 218.

[0043] Memories, such as a main memory 208, a read-only memory (ROM) 224, or a flash memory (not shown), are some examples of computer-usable storage devices. A hard disk drive (HDD) or a solid state drive (SSD) 226a, a CD-ROM 230, and other similar usable devices are some examples of computer-usable storage devices that include computer-usable storage media.

[0044] An operating system runs on the processing unit 206. The operating system coordinates and provides control over Figure 2 the various components within the data processing system 200. The operating system can be a commercially available operating system for any type of computing platform, including but not limited to server systems, personal computers, and mobile devices. An object-oriented or other type of programming system can operate with the operating system and provide calls to the operating system from programs or applications executing on the data processing system 200.

[0045] For the operating system, the object-oriented programming system, and such as Figure 1Instructions for applications or programs such as server application 116 and client application 122 in [the context] are located on a storage device, such as in the form of code 226b on a hard disk drive (HDD) or solid state drive (SSD) 226a, and can be loaded into at least one of one or more memories such as main memory 208 for execution by processing unit 206. The processes of the illustrative embodiments can be executed by processing unit 206 using computer-implemented instructions that can be located in a memory, such as main memory 208, read-only memory (ROM) 224, or in one or more peripheral devices.

[0046] In addition, in one case, code 226b can be downloaded from remote system 214b via network 214a, where similar code 214c is stored on storage device 214d. In another case, code 226b can be downloaded to remote system 214b via network 214a, where the downloaded code 214c is stored on storage device 214d.

[0047] Figure 1 and Figure 2 The hardware in [the context] can vary according to the implementation. In addition to Figure 1 and Figure 2 the hardware depicted in [the context], or as an alternative thereto, other internal hardware or peripheral devices such as flash memory, equivalent non-volatile memory, or optical disk drives can be used. Further, the processes of the illustrative embodiments can be applied to a multiprocessor data processing system.

[0048] In some illustrative examples, data processing system 200 can be a personal digital assistant (PDA), which is typically configured with flash memory to provide non-volatile memory for storing operating system files and / or user-generated data. The bus system can include one or more buses, such as a system bus, an I / O bus, and a PCI bus. Of course, the bus system can be implemented using any type of communication structure or architecture that provides data transfer between different components or devices attached to the structure or architecture.

[0049] The communication unit can include one or more devices for sending and receiving data, such as a modem or a network adapter. The memory can be, for example, main memory 208 or a cache, such as the cache found in the north bridge and memory controller hub (NB / MCH) 202. The processing unit can include one or more processors or CPUs.

[0050] Figure 1 and Figure 2The examples depicted herein and the above examples are not meant to imply architectural limitations. For example, data processing system 200 can be a tablet computer, a laptop computer, or a telephone device in addition to taking the form of a mobile or wearable device.

[0051] In cases where a computer or data processing system is described as a virtual machine, virtual device, or virtual component, the virtual machine, virtual device, or virtual component operates in the manner of data processing system 200 using virtualized representations of some or all of the components depicted in data processing system 200. For example, in a virtual machine, virtual device, or virtual component, processing unit 206 is represented as a virtualized instance of all or some number of hardware processing units 206 available in the host data processing system, main memory 208 is represented as a virtualized instance of all or some portion of main memory 208 available in the host data processing system, and hard disk drive (HDD) or solid state drive (SSD) 226a is represented as a virtualized instance of all or some portion of hard disk drive (HDD) or solid state drive (SSD) 226a available in the host data processing system. In such a case, the host data processing system is represented by data processing system 200.

[0052] Method and system for generating a high-resolution surface scan of an oral cavity

[0053] Figure 3 A schematic diagram depicting a system in which illustrative embodiments can be implemented. System 300 is an example of a combination of a client 110 (scanner unit or acquisition unit), a surface scan generation component 124, and computers (such as servers 104 and 106) for scanning a patient's oral cavity and generating a high-resolution surface scan of the oral cavity. System 300 includes a scanner unit 302. Scanner unit 302 can be a portable device for sampling the surface of the oral cavity by capturing a plurality of high-resolution individual images of the surface of the oral cavity. Scanner unit 302 can include or be formed by a dental camera 308 for taking individual images of the oral cavity. Scanner unit 302 can be connected to a receiver 304 via a network 306. Receiver 304 is a remote computing device or cloud computing device that receives a plurality of individual images from scanner unit 302 via network 306. When it is detected that network 306 has sufficient or high bandwidth, scanner unit uses network 306 to continuously send individual images of the oral cavity to receiver 304. Sufficient or high bandwidth can be that the bandwidth of network 306 exceeds a predetermined bandwidth threshold. Receiver 304 stitches together the individual images received from scanner unit 302 to generate a high-resolution merged surface scan of the oral cavity.

[0054] Refer to Figure 3, the scanner unit 302 further includes a subsampling module 310 and an image buffer 312. The subsampling module 310 can be a computer algorithm stored in the dental camera 308, but is not limited thereto. The subsampling module 310 is used to subsample high-resolution individual images when it detects that the transmission bandwidth of the network 306 is low. The low bandwidth can be the bandwidth of the network 306 that is lower than a predetermined threshold. Subsampling includes reducing a high-resolution individual image into a low-resolution frame and a residual frame. The residual frame can be the same as the high-resolution individual image, or can be the remaining part of the high-resolution image after subsampling. In addition, when detecting the low transmission bandwidth, the subsampling module 310 can send the low-resolution frame to continue the scanning of the surface, while storing the residual frame in the image buffer 312 of the scanner unit 302. The image buffer 312 can be a storage module of the dental camera 308, which stores the residual frames of the corresponding low-resolution frames for the receiver 304 to request and receive later, without any constraints and limitations. In some cases, the residual frame can be a higher-resolution frame generated after removing the low-resolution frame from the high-resolution individual image. Therefore, in this case, a single image can be regenerated by stitching together the respective low-resolution frames and the residual frames using an appropriate stitching algorithm.

[0055] When detecting a low transmission bandwidth, the scanner unit 302 using the subsampling module 310 reduces the resolution of an individual image to convert the individual image into a low-resolution frame, rather than discarding the high-resolution image and interrupting the scanning of the surface. In addition, at low bandwidths, the scanner transmits the corresponding low-resolution frame of the individual image to the receiver via the network 306. This reduction in resolution can enable the uninterrupted transmission of frames to the receiver even at low bandwidths.

[0056] The receiver 304 is configured to stitch a plurality of received high-resolution individual images and low-resolution frames in the order in which they are received from the scanner unit 302. The receiver 304 includes a stitching module 314 that is configured to stitch the sequentially received individual images and low-resolution frames together to generate a stitched model. A merging module 316 is included within the receiver 304 and is communicatively coupled to the stitching module 314. The merging module 316 is configured to continuously monitor the stitched model generated by the stitching module 314 and, in response to detecting that the bandwidth is sufficient or higher than a predetermined threshold, request from the image buffer 312 one or more residual frames of the corresponding one or more low-resolution frames stitched within the stitched model. The merging module 316 can be connected to the image buffer 312 via the network 306 to send a request and receive one or more residual frames of the corresponding low-resolution frames stitched within the stitched model over the high bandwidth of the network. The stitching module 314 stitches the received residual frames with their respective low-resolution frames (or replaces the low-resolution frames with the residual frames if the residual frames have sufficient quality or the same quality as the original individual images of the dental camera 308) within the stitching module to generate a high-resolution merged model of the scanned surface or oral cavity.

[0057] Referring Figure 3 , the receiver 304 can be a server, such as server 104, server 106, a remote computing device, or a cloud server including the surface scan generation component 124. The stitching module 314 and the merging module 316 can be computer program code stored in the memory of the receiver 304. In one embodiment, the receiver 304 is a remote computing device that includes a processing unit having the stitching module 314 and the merging module 316, which can sequentially receive individual images and low-resolution frames from the scanner based on the bandwidth of the network 306 and stitch the individual images and low-resolution frames together to generate a stitched model.

[0058] In one embodiment, the subsampling module 310 generates an ID and associates the ID with each low-resolution frame and the corresponding residual frame. The residual frames with their associated IDs can be further stored in the image buffer 312 to enable the merging module 316 to request the residual frames using the associated ID of the residual frame. The merging module 316 monitors the stitched model generated by the stitching module 314 and identifies at least one region of the surface within the stitched model having a low sampling rate. The low sampling rate is a sampling rate lower than a predetermined sampling rate. Additionally, the merging module 316 requests the image buffer to send the residual frames of the corresponding low-resolution frames present within the identified at least one region having a low sampling rate by using the ID associated with the residual frames.

[0059] In another embodiment, the merging module 316 determines the position of the dental camera 308 using the low-resolution frame and requests the image buffer 312 to send a high-resolution residual frame or a single image associated with the same position of the dental camera 308 to generate a high-resolution merged model of the oral cavity.

[0060] In addition, in one embodiment, the stitching module 314 stitches the received residual frames with their respective low-resolution frames to generate a high-resolution merged model. In another embodiment, the stitching module 314 replaces the low-resolution frames with the corresponding residual frames received from the image buffer 312 to generate a high-resolution merged model of the oral cavity.

[0061] The receiver 304 can present the high-resolution merged model or surface scan of the oral cavity within the user interface 318.

[0062] The scanner unit 302 sends the single images at high bandwidth and the low-resolution frames at low bandwidth in the order in which they are captured or generated. The scanner maintains a queue of the captured single images in the sequence to be sent to the receiver. In addition, the scanner unit 302 detects whether the queue length exceeds or is below a predetermined maximum queue length. If the queue length is below the predetermined maximum queue length, the scanner determines that the transmission bandwidth is high and transmits the high-resolution single images captured by the dental camera 308. Conversely, if the queue length exceeds the predetermined maximum queue length, the scanner determines that the transmission bandwidth is low and converts the single images into low-resolution frames and residual frames. To detect the bandwidth, the receiver sends an acknowledgment signal to the transmitter. This causes the corresponding image to be removed from the queue and enables the low bandwidth to be inferred from the queue length. In addition, the scanner unit 302 sends the low-resolution frames to the receiver at low transmission bandwidth while storing the residual frames in the image buffer 312 to be requested by the merging module 316 of the receiver 304 at a later time in response to detecting a high transmission bandwidth.

[0063] Figure 4Depicts a flowchart of a process 400 for generating a high-resolution surface scan according to an illustrative embodiment of the present disclosure. In block 402, a plurality of individual images are captured. A dental camera 308 is provided and a plurality of individual images associated with each region of the surface of the oral cavity are continuously captured for transmission to a receiver 304, thereby generating a high-resolution scan model of the oral cavity. In block 404, the transmission bandwidth of the network connecting the scanner unit 302 and the receiver 304 is determined. In block 406, when the calculated transmission bandwidth is high, the plurality of individual images are transmitted from the scanner unit 302 to the receiver 304. Conversely, in block 408, when the bandwidth is determined to be low, the individual images are converted into low-resolution frames and sent to the receiver 304. In block 410, the low-resolution frames are sequentially stitched with the individual images to generate a stitched model. In block 412, the corresponding residual frames of the low-resolution frames are stored in an image buffer for reception at a later time when a high transmission bandwidth is detected. In block 414, the receiver 304 receives the residual frames and replaces the stitched low-resolution frames with the corresponding residual frames. Then, in block 416, the receiver 304 generates a high-resolution merged model, which is a high-resolution surface scan model of the oral cavity.

[0064] Figure 5 Depicts a flowchart of a process 500 for generating a high-resolution surface scan according to an illustrative embodiment of the present disclosure. Process 500 is an example illustration of a similar process 400. In block 502, the oral cavity is scanned by capturing a plurality of individual images of the surface of the oral cavity / teeth using a dental camera 308. In block 504, the transmission bandwidth is detected by comparing a transmission queue with a predetermined maximum queue length. In block 506, when there is a low transmission bandwidth, the resolution of the individual images is reduced by converting the individual images into corresponding low-resolution frames and residual frames. In block 508, the scanner unit 302 transmits the low-resolution frames to the receiver 304 during the low transmission bandwidth period. In block 510, the receiver 304 sequentially stitches the low-resolution frames with the individual images at a first time period to continue the surface scan and generate a stitched model. In block 512, the low-resolution frames are selectively updated within the stitched model based on the corresponding residual frames by requesting and receiving the residual frames from the scanner unit 302. In block 514, a high-resolution surface scan or merged model is generated for the oral cavity.

[0065] Figure 6FIG. 600 is a flow chart depicting a process 600 of replacing or stitching together low-resolution frames with residual frames according to an exemplary embodiment of the present disclosure. Process 600 is an example implementation of blocks 508, 510, and 512 of process 500. At block 602, when the detected bandwidth is below a predetermined threshold, the scanner unit 302 transmits a low-resolution frame with an associated ID. At block 604, the corresponding residual frame with the associated ID is stored within the image buffer 312. At block 606, the received low-resolution frame is sequentially stitched with a plurality of existing individual images of other parts of the oral cavity or a model that has already been generated to generate an updated model. At block 608, the merging module 316 identifies at least one region in the model where the sampling rate is lower than a predetermined sampling rate within the stitched model. At block 610, when the detected bandwidth is above a predetermined threshold, the merging module 316 requests to receive the corresponding residual frame associated with the identified at least one region with the associated ID of the corresponding low-resolution frame. At block 612, the merging module 316 replaces the corresponding low-resolution frame with the corresponding residual frame or stitches them together to generate a high-resolution surface scan. SUMMARY OF THE INVENTION

[0067] No particular manifestation of these and other similar example processes is intended to limit the present disclosure. Any suitable manifestation of these and other similar example processes may be selected within the scope of the illustrative embodiments.

[0068] Accordingly, in an illustrative embodiment, a computer-implemented method, system or apparatus, and a computer program product are provided for generating a high-resolution surface scan and other related features, functions, or operations of the oral cavity. In the case where an embodiment is described with respect to a type of device, the computer-implemented method, system or apparatus, computer program product, or a part thereof is adapted or configured to be used with an appropriate and comparable manifestation of that type of device.

[0069] In the case where an embodiment is described as being implemented in an application, within the scope of the illustrative embodiments, a delivery of the application in a software as a service (SaaS) model may be envisioned. In the SaaS model, the ability to implement the application of the embodiment is provided to a user by executing the application in a cloud infrastructure. The user may access the application using various client devices through a thin client interface such as a web browser or other lightweight client applications. The user does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage of the cloud infrastructure. In some cases, the user may not even manage or control the capabilities of the SaaS application. In some other cases, the SaaS implementation of the application may allow for possible exceptions for limited user-specific application configuration settings.

[0070] The present disclosure can be a system, method, and / or computer program product at any possible technical detail integration level. The computer program product can include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to execute aspects of the present disclosure.

[0071] A computer-readable storage medium can be a tangible device that is capable of storing and retaining instructions for use by an instruction execution device. A computer-readable storage medium can be, by way of example and not limitation, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer-readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanical encoding device such as a punched card or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be construed to be a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through a wire.

[0072] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within the respective computing / processing device.

[0073] The computer-readable program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and procedural programming languages such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on a dedicated system or the user's computer, partly on the user's computer or a dedicated system as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server, etc. In the latter case, the remote computer may be connected to the user's computer through any type of network connection, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, an electronic circuit, including, for example, a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA), may execute the computer-readable program instructions by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit so as to perform aspects of the present disclosure.

[0074] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0075] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions executed via the processor of the computer or other programmable data processing apparatus create a device for implementing the functions / acts specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium, which may direct a computer, a programmable data processing apparatus, and / or other devices to operate in a particular manner, such that the computer-readable storage medium in which the instructions are stored comprises an article of manufacture including instructions for implementing aspects of the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0076] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other devices to produce a computer-implemented process, such that the instructions executed on the computer, other programmable apparatus, or other devices implement the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0077] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions, which includes one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, depending on the functionality involved, two blocks shown in succession may actually be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0078] All features disclosed in the specification, including the claims, abstract, and drawings, and all steps in any method or process disclosed, may be combined in any combination, except for combinations where at least some of such features and / or steps are mutually exclusive. Each feature disclosed in the specification, including the claims, abstract, and drawings, may be replaced by alternative features serving the same, equivalent, or similar purpose, unless expressly stated otherwise.

Claims

1. A method of generating a high resolution surface scan of an intraoral cavity, the method comprising: sampling the surface by capturing multiple individual images of the surface of the oral cavity with a scanner; Calculate the bandwidth of the transmission; In response to calculating that the bandwidth is below a predetermined threshold, reducing the resolution of the one or more single images of the plurality of single images associated with at least one region of the surface by converting the one or more single images associated with the at least one region of the surface into corresponding one or more low-resolution frames and corresponding one or more residual frames associated with the at least one region of the surface; as well as sequentially transmitting the corresponding one or more low-resolution frames to a receiver at the bandwidth; sequentially stitching, by the receiver, the corresponding one or more low-resolution frames over a first time period to continue scanning the surface, thereby generating a stitched model; selectively updating the corresponding one or more low-resolution frames based on the corresponding one or more residual frames within the stitched model at a second time period beginning after the first time period begins; as well as Generate a high-resolution surface scan of the oral cavity.

2. The method according to claim 1, wherein: The corresponding one or more residual frames have a resolution that is the same as or different from a resolution of the one or more single images, in response to the corresponding one or more residual frames having a resolution lower than a resolution of the one or more single images, updating the corresponding one or more low-resolution frames by adding the corresponding one or more residual frames to the corresponding one or more low-resolution frames, and In response to the corresponding one or more residual frames having a resolution that is the same as a resolution of the one or more single images, the corresponding one or more low-resolution frames are updated by replacing the corresponding one or more low-resolution frames with the corresponding one or more residual frames.

3. A method according to claim 1, wherein sequentially stitching the corresponding one or more low-resolution frames includes sequentially aligning and stitching the corresponding one or more low-resolution frames associated with the at least one area with multiple single images of the remaining area that have been scanned to generate a stitched model of the surface.

4. The method of claim 1, wherein a scanner stores the corresponding one or more residual frames of the one or more single images in an image buffer of the scanner. 5 . The method of claim 4 , wherein the receiver requests reception of the corresponding one or more residual frames from the scanner upon detecting that the bandwidth is above a predetermined threshold.

6. The method of claim 5, wherein the corresponding one or more low-resolution frames are sent together with an associated identification ID to enable the corresponding one or more residual frames to be requested by a receiver from a scanner.

7. The method according to claim 6, wherein: a receiver mapping a first surface of the stitched model to identify a first region having a sampling rate lower than a predetermined sampling rate; as well as The receiver requests, for the first region, the corresponding one or more residual frames of the corresponding one or more low-resolution frames associated with the first region using the associated ID of the corresponding one or more low-resolution frames.

8. The method of claim 7, wherein the received corresponding one or more residual frames replace the corresponding one or more low-resolution frames, or the received corresponding one or more residual frames are stitched together to generate a high-resolution surface scan.

9. The method of claim 1 , wherein detecting the bandwidth comprises: maintaining a queue of the plurality of individual images captured for transmission to a receiver; detecting, based on an acknowledgement signal from a receiver, a queue length exceeding a predetermined maximum queue length; as well as It is determined that the bandwidth is below a predetermined threshold.

10. The method according to claim 9, wherein: The detection bandwidth also includes: maintaining a queue of the plurality of individual images captured for transmission to a receiver; detecting a queue length falling below a predetermined maximum queue length; and It is determined that the bandwidth is above a predetermined threshold.

11. The method of claim 10, wherein the scanner transmits the plurality of individual images of the surface upon detecting a bandwidth above a predetermined threshold.

12. The method of claim 11, wherein the plurality of individual images are high resolution images of the surface.

13. The method according to claim 7, further comprising: identifying a second surface of the stitched model having a sampling rate higher than a predetermined sampling rate; as well as A resolution of one or more individual images associated with the second surface is reduced.

14. The method according to claim 7, further comprising: visualizing, on a user interface, at least one region within the stitched model having a sampling rate lower than a predetermined sampling rate; as well as The user is directed to capture one or more new single images from the at least one area using the scanner.

15. A system for generating a high resolution surface scan of an intraoral cavity, the system comprising: at least one processor configured to: sampling the surface by capturing multiple individual images of the surface of the oral cavity with a scanner; Calculate the bandwidth of the transmission; In response to calculating that the bandwidth is below a predetermined threshold, reducing the resolution of the one or more single images of the plurality of single images associated with at least one region of the surface by converting the one or more single images associated with the at least one region of the surface into corresponding one or more low-resolution frames and corresponding one or more residual frames associated with the at least one region of the surface; as well as sequentially transmitting the corresponding one or more low-resolution frames to a receiver at the bandwidth; sequentially stitching, by the receiver, the corresponding one or more low-resolution frames over a first time period to continue scanning the surface, thereby generating a stitched model; selectively updating the corresponding one or more low-resolution frames based on the corresponding one or more residual frames within the stitched model at a second time period beginning after the first time period begins; as well as Generate a high-resolution surface scan of the oral cavity.

16. The system of claim 15, wherein the receiver is configured to sequentially register and stitch the corresponding one or more low-resolution frames associated with the at least one region with a plurality of single images of other regions of the intraoral cavity to generate a stitched model.

17. The system of claim 16, wherein the receiver is configured to: receiving, in response to detecting that the bandwidth is above a predetermined threshold, the corresponding one or more residual frames of the corresponding low-resolution frame from a scanner; selectively replacing the corresponding one or more low-resolution frames with the corresponding one or more residual frames within the stitched model; as well as Generate a high-resolution surface scan of the oral cavity.

18. The system of claim 17, wherein the scanner comprises an image buffer configured to store the corresponding one or more residual frames of the one or more single images.

19. The system of claim 15, further comprising: The user interface is configured to visualizing at least one region within the stitched model having a sampling rate lower than a predetermined sampling rate; as well as The user is directed to capture one or more new single images from the at least one area using the scanner.

20. A non-transitory computer-readable storage medium storing one or more programs that, when executed by a computer, cause the computer to: sampling the surface by capturing multiple individual images of the surface of the oral cavity with a scanner; Calculate the bandwidth of the transmission; In response to calculating that the bandwidth is below a predetermined threshold, reducing the resolution of the one or more single images of the plurality of single images associated with at least one region of the surface by converting the one or more single images associated with the at least one region of the surface into corresponding one or more low-resolution frames and corresponding one or more residual frames associated with the at least one region of the surface; as well as sequentially transmitting the corresponding one or more low-resolution frames to a receiver at the bandwidth; sequentially stitching, by the receiver, the corresponding one or more low-resolution frames over a first time period to continue scanning the surface, thereby generating a stitched model; selectively updating the corresponding one or more low-resolution frames with the corresponding one or more residual frames within the stitched model at a second time period beginning after the first time period begins; as well as Generate a high-resolution surface scan of the oral cavity.

Images