Intraoral scanning system for providing a feedback signal comprising a quality level of a three-dimensional model

Abstract

The present disclosure relates to an intraoral scanner system configured to display a feedback signal on a three-dimensional (3D) model of a dental object, wherein the system comprises: at least one projection unit configured to emit a plurality of configurations of probe light having the form of an illumination pattern; at least one image sensor unit housing an array of sensor elements configured to provide a two-dimensional (2D) image with three-dimensional (3D) data based on captured reflections of the probe light from the dental object; one or more processors configured to determine a 3D model of the dental object by stitching the 2D images, update the 3D model by stitching the 2D images provided by the at least one image sensor unit onto the 3D model, determine a quality level of data in the 3D model at a location in the 3D model, and determine a feedback signal comprising the quality level of the data in the 3D model and the corresponding location of the data in the 3D model, when the one or more processors do not update the 3D model. The system further comprises a display unit configured to display the 3D model in real-time, and the display unit is further configured to display the quality level of the data at the location on the 3D model determined by the feedback signal when the feedback trigger signal is determined.

Description

Technical Field

[0001] This disclosure relates to an intraoral scanning system configured to determine feedback signals on a three-dimensional model of a dental object. More specifically, this disclosure relates to improved methods for determining and visualizing the quality level of various parts of a three-dimensional model. Background Technology

[0002] Three-dimensional (3D) representation is of great significance in many fields, such as medicine and civil engineering. In dentistry, the 3D digital representation of dental objects represents a clear advantage in helping dentists provide optimal treatment to patients. In fact, digital dentistry offers opportunities to expand dentists' professional skills by utilizing new treatment options and provides possibilities for more cost-effective ways of caring for their patients.

[0003] An intraoral scanner system is a device or system used in dentistry to capture digital impressions of a patient's dentition. In a typical scanning scenario using a handheld intraoral scanner, a digital 3D representation is reconstructed in real time and simultaneously rendered onto a monitor to provide visual guidance and user feedback. This means the user needs to look at the monitor to see if their scan is progressing, as the 3D representation is built on the monitor in real time, providing visual feedback on progress and data coverage.

[0004] During intraoral scanning, users can regularly switch their gaze between the patient's oral cavity and the monitor. However, for some users, their gaze is mostly focused on the patient's oral cavity. For these users, the intraoral scanning system generates a large amount of feedback data to be displayed, but the users do not pay attention to this feedback data. Summary of the Invention

[0005] One aspect of this disclosure is to overcome the aforementioned drawback of generating a large amount of feedback data that is not being displayed and is not being paid attention to by the user.

[0006] According to this aspect, an intraoral scanning system is disclosed. The intraoral scanning system is configured to display feedback signals on a three-dimensional (3D) model of a dental object. The system may include at least one projection unit configured to emit probe light of various configurations having an illumination pattern; and at least one image sensor unit housing an array of sensor elements configured to provide a two-dimensional (2D) image with three-dimensional (3D) data based on the reflection of the probe light captured from the dental object. The system also includes one or more processors configured to: determine a 3D model of a dental arch by stitching together 2D images, and update the 3D model by stitching together the 2D images provided by the at least one image sensor unit. The one or more processors are also configured to determine the quality level of the data in the 3D model at its location within the 3D model, and

[0007] Determine the quality level of the data in the 3D model and the feedback signal of the corresponding location of the data in the 3D model.

[0008] Data in the 3D model may include color data, structural data, and / or diagnostic data of the 3D model. The one or more processors are also configured to determine a feedback trigger signal when the one or more processors are not updating the 3D model. 15. Diagnostic data may include the difference between infrared data and color data captured by the image sensor unit. Color data may be red, green, blue, fluorescent red, and fluorescent green. Emitted infrared light is partially reflected from the surface of the tooth object, and most of it is reflected within the inner surface of the tooth object located below its surface. Emitted visible light is mostly reflected from the surface of the tooth object, and by determining the difference between the infrared data and the color data, diagnostic data will be generated that includes infrared data with a reduced amount of surface reflection. This means that caries visible in the infrared data will have improved visibility, i.e., contrast, in the diagnostic data.

[0009] The system includes a display unit configured to display a 3D model in real time, and the display unit is also configured to display the quality level of data at a location in the 3D model determined by a feedback trigger signal when a feedback trigger signal is determined.

[0010] When one or more processors fail to determine a feedback trigger signal while updating the 3D model, this can indicate that the handheld intraoral scanner, including at least a projection unit and at least one image sensor unit, is not generating a 2D image capable of updating the 3D model. This could be caused by erroneous movement of the handheld intraoral scanner inside the patient's mouth or by the entire handheld intraoral scanner being pulled out of the patient's mouth. When the user pulls the tip of the handheld intraoral scanner out of the mouth, it can indicate that the user is looking at the display unit to check the 3D model and its quality. The system is then configured to update the displayed 3D model along with the quality level of data at the location in the 3D model determined by the feedback signal. Thus, the quality level is not updated in real time, as this would result in unnecessary consumption of the processing power of the one or more processors.

[0011] The failure to update the feedback trigger signal includes not stitching the 2D image provided by at least one image sensor unit onto the 3D model.

[0012] A 3D model should be understood as a point cloud, a point cloud with additional attributes such as point normals and point colors, a triangulated point cloud (triangular mesh), a polygonal mesh, a volumetric representation / voxel model, or a parametric representation (e.g., spline representation). The 3D model can also include a virtual representation of an object in three dimensions, where internal regions (structures, etc.) are arranged proportionally and relative to other internal and surface features of the object being modeled within the three physical dimensions of the volume. For example, a volumetric representation of a tooth could include the outer surface and the internal structure of the tooth's interior (below the tooth surface), arranged proportionally relative to the tooth such that a cross-section through the volumetric model substantially corresponds to a cross-section through the tooth, thus revealing the location and size of the internal structure.

[0013] In this context, feedback signals should be understood as signals provided to the output device to offer visual feedback to the user. Alternatively, feedback signals may refer only to the actual feedback provided to the user when the scanning process is paused. This feedback can involve various information related to the scanned dental object. Examples include: whether scan data has been acquired, the rate at which new scan data is acquired, whether the scan patch has been correctly registered, whether the registration of the scan patch has been unsuccessful, the progress of the scan, whether the dental object has been completely scanned, and deviations from the recommended scan path.

[0014] The intraoral scanning system includes an output device for providing the feedback described herein. As an example, the output device can be configured to receive a feedback signal that is displayed instantaneously without modulation based on, for example, the rate at which scan data is acquired.

[0015] The system may include a handheld intraoral scanner comprising at least one projection unit, at least one image sensor unit, a gyroscope configured to determine the orientation of the handheld intraoral scanner, and / or an accelerometer configured to determine the motion of the handheld intraoral scanner. The one or more processors may be configured to determine a feedback trigger signal by comparing the orientation and / or motion of the handheld intraoral scanner with the scanning orientation range and / or scanning motion pattern of the handheld intraoral scanner. A user may stop moving the handheld intraoral scanner, which will be indicated to the one or more processors via an output from the accelerometer. In this example, the accelerometer will provide an output including acceleration at a level corresponding to a stationary handheld intraoral scanner. A user may move the handheld intraoral scanner in various locations, where the movement does not correspond to a scanning scene. The gyroscope providing the orientation of the handheld intraoral scanner to the one or more processors can be used to determine whether the movement of the handheld intraoral scanner corresponds to a scanning scene. For example, the orientation of the handheld intraoral scanner that does not correspond to a scanning scene can be used by the one or more processors to determine a trigger signal or to maintain the display of a 3D model with a quality level. In another scenario, when a user moves a handheld intraoral scanner and simultaneously orients the device to correspond to the scanning scene, the quality level display will be removed and replaced by only displaying / updating the 3D model in real time.

[0016] An intraoral scanning system can be referred to as a system.

[0017] An intraoral scanning system may include a memory unit that includes the scanning orientation range of a handheld intraoral scanner and can determine a feedback trigger signal when the orientation of the handheld intraoral scanner is outside the scanning orientation range. The orientation of the handheld intraoral scanner outside the scanning orientation range corresponds to a non-scanning scenario, thereby displaying the quality level in the 3D model. For example, if the tip of the handheld intraoral scanner is oriented upwards while being moved, it corresponds to a highly improbable posture of the patient being scanned. In this particular example, the orientation of the handheld intraoral scanner corresponds to the patient lying prone and looking down at the floor.

[0018] When the movement of the handheld intraoral scanner is dissimilar to a reference motion pattern, or when no movement is measured, a feedback trigger signal can be determined. The reference motion pattern is stored in the system's memory and corresponds to the motion pattern in the scanning state of the dental object. The motion pattern corresponding to the scanning state may have an acceleration below a predetermined acceleration threshold during the manufacture of the handheld intraoral scanner. For example, if the user's movement of the handheld intraoral scanner does not allow the system to stitch the acquired 2D images onto a 3D model, this corresponds to a non-scanning state. Such movement may occur when the user moves the handheld intraoral scanner within or between rooms.

[0019] Handheld intraoral scanning devices may include a user interface in which a feedback trigger signal is determined when the user interface receives input from a user of the intraoral scanning system. For example, regardless of the orientation and movement of the handheld intraoral scanning device, the user requests the display of a quality level in a 3D model by pressing a button on the handheld intraoral scanning device or an external device of the system. This button may be a virtual button displayed on a display unit. In another example, the user may want the quality level displayed while scanning the patient; the user can then press a button on the handheld intraoral scanning device to display the quality level in the 3D model. When the user releases the button, the quality level is no longer seen in the 3D model. During the patient scan and while the user presses the button, the displayed quality level is updated at a second frame rate lower than the first frame rate used to update the 3D model by stitching 2D images onto it.

[0020] The first frame rate can be between 20 and 40 2D images per second, where all pixels of the 2D images are captured simultaneously. The second frame rate can be between 1 and 10 2D images per second. The second frame rate can be at least half of the first frame rate.

[0021] Feedback signals may include the quality level of the three-dimensional geometric data, color data, and / or diagnostic data of the dental object.

[0022] The projection unit may include multiple light sources, including a first light source configured to emit visible light, a second light source configured to emit infrared light, and / or a third light source configured to emit ultraviolet (UV) light, wherein the visible light includes probe light. A real-time mode is determined by a first frame rate, and at least one image sensor unit is configured to capture fluorescence signals from the dental object at a second frame rate based on the emitted UV light, and / or at least one image sensor unit is configured to capture reflected infrared light at a third frame rate, wherein the first frame rate is greater than both the second and third frame rates, and wherein the feedback trigger signal is determined by either the second or third frame rate. The second and third frame rates may be 3 to 5 times lower than the first frame rate. The second and third frame rates do not correspond to real-time. The user can perceive the quality level in the 3D model in a non-real-time manner to reduce the processing power required for visualization to generate the feedback signal. This would not be the case if the feedback signal were generated according to a first frame rate that provides real-time updates of the 3D model.

[0023] The real-time mode can be determined by a first frame rate, and the one or more processors can be configured to determine a feedback trigger signal at a frame rate of at least 1 / 2 of the first frame rate.

[0024] The quality level can be determined based on the captured two-dimensional image, wherein one or more attributes (i.e., data) of the captured two-dimensional image are analyzed by one or more processors.

[0025] The quality level can be determined by the following items (i.e., one or more attributes), such as the amount of data at a location in the 3D model, the amount of data taken from different angles relative to the location in the 3D model, the quality of the data at a location in the 3D model, the quality of the data taken from different angles relative to the location in the 3D model, signal-to-noise ratio, sharpness, contrast, color, accuracy, and / or dynamic range.

[0026] Signal-to-noise ratio (SNR) can be measured in captured two-dimensional images. For example, poor lighting conditions, poor image sensor performance and / or poor light source performance, and high moisture levels inside the patient's oral cavity can degrade the SNR, thereby reducing the quality level.

[0027] The dynamic range of focus in a two-dimensional image can also reduce the quality level.

[0028] The quality level can be below a quality threshold level, and the 3D model will be completely transparent or have a specific color at the corresponding location of the determined quality level, as determined by the feedback signal. When the quality level is above the quality threshold level and continues to improve, the 3D model will become less transparent or its color will change at the corresponding location of the determined quality level.

[0029] The quality threshold level can vary between different parts of the 3D model. For example, treatment or preparation may not be performed in the first part of the 3D model, but may be performed in the second part. In most cases, more precise data will be needed in the 3D model where at least some treatment or preparation is required. For example, treatment may be for caries, cracks, or tissue diseases such as inflammation, periodontitis, and gingivitis. Preparation may be for restorations. The quality level and location in the feedback signal are determined based on the treatment type or preparation type, and the treatment type or preparation type can be one or more of the following: caries treatment, crack treatment, tissue disease, and restoration preparation.

[0030] Dental objects can include "restorations," which can generally be understood as components that restore the structure or function of an existing dentition, such as crowns, bridges, veneers, inlays, high-end inlays, amalgam, composite materials, and various underlying structures such as inner crowns, as well as temporary restorations used in the fabrication of permanent restorations. Precision and quality levels are particularly important in the vicinity of these restorations, thus altering this threshold.

[0031] For some users, coupling audio feedback to the feedback signal is advantageous, meaning the frequency of the audio feedback corresponds to the quality level at that location in the feedback signal. For example, in a scanning scenario, as a user moves a handheld intraoral scanner from one area to another on a patient's teeth, the audio frequency will change due to the difference in quality levels between the two areas. Furthermore, the frequency of the audio feedback signal determines the rate of the generated feedback trigger signal. The system may include a speaker for playing the audio feedback. The one or more processors may be configured to provide audio feedback via the system's speaker, wherein the timing of the audio feedback playback is synchronized with the determination of the feedback trigger signal.

[0032] The synchronization of audio feedback and the visualization of feedback signals will enhance the user's ability to grasp the quality of the 3D model during scanning.

[0033] The one or more processors can be configured to change the geometry of the illumination pattern, and wherein the one or more processors can be configured to determine a feedback trigger signal when the illumination pattern is a specific pattern (such as a line pattern, a checkerboard pattern, or any pattern different from the illumination pattern used in probe light to provide a 2D image with 3D data for determining a 3D model). Attached Figure Description

[0034] The various aspects of this disclosure can be best understood in conjunction with the accompanying drawings and the following detailed description. The drawings are schematic and simplified for clarity; they show only details to facilitate understanding of the claims, while other details are omitted. Throughout the text, the same reference numerals are used for the same or corresponding parts. Each individual feature of each aspect can be combined with any or all features of other aspects. These and other aspects, features, and / or technical effects will be apparent and elucidated from the illustrations described below, in which:

[0035] Figure 1 An example of an intraoral scanning system is shown;

[0036] Figure 2 An example of an intraoral scanning system is shown;

[0037] Figures 3A to 3E An example scenario including an intraoral scanning system is shown;

[0038] Figure 4A and Figure 4B An example of an intraoral scanning system and a scanning sequence timetable is shown;

[0039] Figures 5A to 5C Different examples of displaying feedback signals on a 3D model are shown; and

[0040] Figure 6A and Figure 6B This shows yet another example of displaying feedback signals on a 3D model. Detailed Implementation

[0041] The detailed description described below, in conjunction with the accompanying drawings, is intended as a description of various configurations. This detailed description includes specific details to provide a thorough understanding of the various concepts. However, it will be apparent to those skilled in the art that these concepts can be practiced without these specific details. Various aspects of devices, systems, media, programs, and methods are described by various blocks, functional units, modules, components, circuits, steps, processes, algorithms, etc. (collectively, “elements”). Depending on a particular application, design constraints, or other reasons, these elements may be implemented using electronic hardware, computer programs, or any combination thereof.

[0042] Electronic hardware may include microprocessors, microcontrollers, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functions described throughout this disclosure. The term "computer program" should be interpreted broadly as meaning instructions, instruction sets, code, code segments, program code, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description languages, or others.

[0043] Scanning to provide intraoral scanning data can be performed by a dental scanning system, which may include intraoral scanning devices such as the TRIOS series scanners from 3Shape A / S. The dental scanning system may include wireless capabilities, such as those provided by a wireless network unit. The scanning device may employ scanning principles such as triangulation-based scanning, confocal scanning, focused scanning, ultrasound scanning, X-ray scanning, stereoscopic vision, motion reconstruction, optical coherence tomography (OCT), or any other scanning principle. In an embodiment, the scanning device acquires surface information by projecting a pattern and translating a focal plane along the optical axis of the scanning device, capturing multiple 2D images at different focal plane positions, such that each series of captured 2D images corresponding to each focal plane forms a 2D image stack. The acquired 2D images are also referred to herein as raw 2D images, where "raw" in the context means an image that has not yet undergone image processing. The focal plane position is preferably moved along the optical axis of the scanning system, such that 2D images captured at several focal plane positions along the optical axis form a stack of 2D images (also referred to herein as subscans) for a given viewpoint of the object, i.e., a stack of 2D images for a given arrangement of the scanning system relative to the object. A new stack of 2D images for that viewpoint can be captured after moving the scanning device relative to the object or imaging the object from a different viewpoint. The focal plane position can be changed by means of at least one focusing element (e.g., a moving focusing lens). During the scanning process, the scanning device is typically moved and angled relative to the dental arch, such that at least some subscan groups at least partially overlap, enabling the reconstruction of a digital dental 3D model by stitching the overlapping 3D subscans together in real time, and displaying the progress of the virtual 3D model on a monitor as feedback to the user. The result of the stitching is a digital 3D representation of a surface larger than that that can be captured by a single subscan, i.e., a digital 3D representation of a surface larger than the field of view of the 3D scanning device. Stitching, also known as registration and fusion, works by identifying overlapping areas of the 3D surface in various sub-scans and transforming the sub-scans to a common coordinate system to match these overlapping areas, ultimately resulting in a digital 3D model. The Iterative Closest Point (ICP) algorithm can be used for this purpose. Another example of a scanning device is a triangulation scanner, where a time-varying pattern is projected onto a dental object, and image sequences of different pattern configurations are acquired by one or more cameras positioned at an angle relative to the projection unit.

[0044] The color texture of a tooth object can be captured by illuminating the object with different monochromatic lights such as red, green, and blue, or by illuminating the object with multicolor lights such as white light. A 2D image can be captured during white light flashing.

[0045] Typically, the process of acquiring real-time surface information of a dental object to be scanned requires the scanning device to illuminate the surface and acquire a large number of 2D images. High-speed cameras with frame rates of 300 to 2000 2D frames per second are typically used, depending on the technology and the resolution of the 2D images. The large amount of image data that needs to be processed by the scanning device is necessary to either directly forward the raw image data stream to an external processing device or to perform some image processing before transmitting the data to an external device or display. This process requires multiple electronic components within the scanner to operate at high workloads, thus necessitating high current requirements.

[0046] The scanning device includes one or more light projectors configured to generate an illumination pattern to be projected onto a three-dimensional dental object during the scanning process. The light projector preferably includes a light source, a mask with a spatial pattern, and one or more lenses, such as a collimating lens or a projection lens. The light source can be configured to generate light of a single wavelength or a combination of multiple wavelengths (monochromatic or polychromatic). The combination of wavelengths can be generated by using a light source configured to produce light including different wavelengths (such as white light). Alternatively, the light projector can include multiple light sources (such as LEDs) that each produce light of different wavelengths (such as red, green, and blue), which can be combined to form light including different wavelengths. Thus, the light generated by the light source can be defined by wavelengths defining a specific color, or by a range of different wavelengths defining combinations of colors such as white light. In an embodiment, the scanning device includes a light source configured to excite fluorescent material in the tooth to obtain fluorescence data from the dental object. Such a light source can be configured to produce a narrow wavelength range. In another embodiment, the light from the light source is infrared (IR) light, which is capable of penetrating dental tissue. The light projector can be a DLP projector that uses a micromirror array to generate a time-varying pattern, a diffractive optical element (DOF), or a backlit mask projector, wherein the light source is placed behind a mask with a spatial pattern, thereby patterning the light projected onto the surface of the dental object. The backlit mask projector may include a collimating lens for collimating the light from the light source, which is placed between the light source and the mask. The mask may have a checkerboard pattern, such that the generated illumination pattern is a checkerboard pattern. Alternatively, the mask may have other patterns such as lines or dots.

[0047] The scanning device preferably also includes optical components for guiding light from the light source onto the surface of the dental object. The specific arrangement of the optical components depends on whether the scanning device is a focusing scanning device, a triangulation-based scanning device, or any other type of scanning device. A focusing scanning device is further described in EP 2 442 720 B1 of the same applicant, which is incorporated herein by reference in its entirety.

[0048] Light reflected from a dental object in response to illumination is directed to an image sensor using the optics of a scanning device. The image sensor is configured to generate multiple images based on the incident light received from the illuminated dental object. The image sensor unit can be a high-speed image sensor, such as one configured to acquire images at exposure times less than 1 / 1000 of a second or frame rates exceeding 250 frames per second (fps). As an example, the image sensor can be a rolling shutter (CCD) or a global shutter sensor (CMOS). The image sensor can be a monochrome sensor comprising an array of color filters (such as Bayer filters) and / or additional filters that can be configured to substantially remove one or more color components from the reflected light and retain only the remaining unremoved components before converting the reflected light into an electrical signal. For example, such additional filters can be used to remove specific portions of the white light spectrum (such as the blue component) and retain only the red and green components from the signal generated in response to the excitation of fluorescent materials on the tooth.

[0049] A network unit can be configured to connect a dental scanning system to a network comprising multiple network elements, including at least one network element configured to receive processed data. The network unit can include a wireless network unit or a wired network unit. A wireless network unit is configured to wirelessly connect the dental scanning system to a network comprising multiple network elements, including at least one network element configured to receive processed data. A wired network unit is configured to establish a wired connection between the dental scanning system and the network comprising multiple network elements, including at least one network element configured to receive processed data.

[0050] The dental scanning system preferably also includes a processor configured to generate scan data (such as extraoral scan data and / or intraoral scan data) by processing two-dimensional (2D) images acquired by the scanning device. The processor may be part of the scanning device. As an example, the processor may include a field-programmable gate array (FPGA) and / or an advanced reduced instruction set machine (ARM) processor located on the scanning device. The scan data includes information related to a three-dimensional dental object. The scan data may include any of the following: 2D images, 3D point clouds, depth data, texture data, intensity data, color data, and / or combinations thereof. As an example, the scan data may include one or more point clouds, where each point cloud includes a set of 3D points describing the three-dimensional dental object. As another example, the scan data may include images, each image including image data described, for example, by image coordinates and a timestamp (x, y, t), where depth information can be inferred from the timestamp. The image sensors of the scanning device may acquire multiple raw 2D images of the dental object in response to illumination of the dental object using one or more light projectors. These multiple raw 2D images may also be referred to herein as a 2D image stack. The 2D images can then be provided as input to a processor, which processes the 2D images to generate scan data. Processing of the 2D images may include steps to determine which part of each 2D image is in focus in order to derive / generate depth information from the images. The internal depth information can be used to generate a 3D point cloud comprising a set of 3D points in space, described, for example, by Cartesian coordinates (x, y, z). The 3D point cloud can be generated by the processor or by another processing unit. Each 2D / 3D point may also include a timestamp indicating when the 2D / 3D point was recorded, i.e., which image in the stack of 2D images the point originated from. The timestamp is associated with the z-coordinate of the 3D point; that is, the z-coordinate can be inferred from the timestamp. Therefore, the processor's output is scan data, which may include image data and / or depth data, described, for example, by image coordinates and timestamps (x, y, t) or, alternatively, by (x, y, z). The scanning device can be configured to transmit other types of data besides the scan data. Examples of data include 3D information, texture information (such as infrared (IR) images, fluorescence images, reflective color images, X-ray images, and / or combinations thereof).

[0051] Figure 1An example of an intraoral scanning system 1 is shown. In this example, system 1 includes a handheld intraoral scanning device 100, which includes at least one projection unit 7, at least one image sensor unit 5, and one or more processors 11. In this example, system 1 includes one or more external computers 102 and a server 103 communicating with the external computers 102. The one or more processors 11 may be distributed among the external computers 102, the server 103, and the handheld intraoral scanning device 100. At least one projection unit 7 is configured to emit probe light of various configurations having an illumination pattern, and at least one image sensor unit 5 houses an array of sensor elements configured to provide a two-dimensional (2D) image with three-dimensional (3D) data based on the reflection of the probe light captured from the dental object. The one or more processors are configured to determine a 3D model of the dental object by stitching together 2D images, update the 3D model by stitching together 2D images provided by at least one image sensor unit onto the 3D model, determine the quality level of data at a location in the 3D model, and determine a feedback signal including the quality level of the data in the 3D model and the corresponding location of the data in the 3D model. A feedback trigger signal is determined when the one or more processors are not updating the 3D model. System 1 includes a display unit 105 configured to display the 3D model in real time, and the display unit 105 is further configured to display the quality level of the data at the location in the 3D model determined by the feedback signal when the feedback trigger signal is determined.

[0052] Figure 2One or more processors 11 are shown. These processors 11 receive 2D images acquired by image sensor unit 5 and determine 4 a 3D model of the dental object being scanned. Additional 2D images acquired by image sensor unit 5 are stitched 4 onto the 3D model. 6 A quality level of data (e.g., the quality of image data in the acquired 2D images) is determined, and a feedback signal 6 is determined, which includes the quality level of the data in the 3D model and the corresponding position of that data in the 3D model. In this example, a feedback trigger signal 10 is determined because the 3D model is not updated 3 by these processors 11. Not updating the feedback trigger signal includes not stitching the 2D image onto the 3D model. The one or more processors 11 are configured to prepare 8 a 3D model to be displayed on display unit 105. The one or more processors 11 are configured to prepare 8 a feedback signal to be displayed on display unit 105. The feedback signal includes a quality level and a corresponding position, and the one or more processors 11 are configured to align the position of the quality level relative to the 3D model and apply that quality level as an overlay on the 3D model. The overlay can be a colored overlay, where the color can be changed or toned down according to the quality level. The overlay can also be a layer with a transparency level, where the transparency level is adjusted according to the quality level. The overlay is applied to the 3D model at a location determined by a feedback signal.

[0053] Figures 3A to 3E Different scenarios involving a handheld intraoral scanning device 100 are illustrated. In these examples, the handheld intraoral scanning device includes a gyroscope configured to determine the orientation of the handheld intraoral scanning device and / or an accelerometer configured to determine the motion of the handheld intraoral scanning device. The one or more processors 11 are configured to determine a feedback trigger signal by comparing the orientation and / or motion of the handheld intraoral scanning device with the scanning orientation range and / or scanning motion pattern of the handheld intraoral scanning device 100. Figure 3A A scanning scenario is illustrated, in which the tip of a handheld intraoral scanner 100 is positioned inside the patient's oral cavity 31, and the scanner moves 32 according to typical movements reflecting the scanning scenario. In this example, a feedback trigger signal is not identified. Furthermore, Figure 3A It also shows that the user is primarily focused on the patient's oral cavity 31, therefore, there is no need to update the 3D model using feedback signals on the display unit 105. Figure 3B The illustration shows a scenario where scanning is paused by pulling the handheld intraoral scanning device 100 out of the oral cavity 31. In this example, since the image sensor unit 5 has not acquired a relevant 2D image, the one or more processors 11 will not update the 3D model. Figure 3BIt can also reflect scenarios where the scan is stationary or moving in a direction that does not reflect the scanning scene, and in both scenarios, a feedback trigger signal is generated. When the movement of the handheld intraoral scanning device 100 is not similar to the reference motion pattern, or when no movement is measured, the feedback trigger signal is determined, and the reference motion pattern is stored in the memory unit of the system 1, and the reference motion pattern corresponds to the motion pattern in the scanning state of the tooth object 32. Figure 3C The illustration shows a scenario where the tip of scanner 100 is placed inside the patient's mouth 32, still acquiring 2D images, but scanner 100 is not moving. When the quality level of the portion of the 3D model overlapping the acquired 2D images is obtained, the one or more processors 11 stop stitching the acquired 2D images onto the 3D model. In this example, a feedback trigger signal is generated because the 3D model is not being updated. Figure 3D An example is shown where the orientation 33 of the handheld intraoral scanner 100 does not reflect the scanning scenario, and in this example, a feedback trigger signal is generated. System 1 includes a memory unit that includes the scanning orientation range of the handheld intraoral scanner 100, and determines the feedback trigger signal when the orientation of the handheld intraoral scanner 100 is outside the scanning orientation range. In this specific example, the orientation being outside the scanning orientation range is reflected by the tip of the handheld intraoral scanner 100 pointing upwards, and is guided in a manner that would reflect a very abnormal posture if the patient is being scanned. Figure 3E An example is shown where a feedback trigger signal 10 is generated when a user presses button 35 on scanner 100. The feedback signal is no longer displayed when the user presses button 35 again or releases button 35 from its initial press.

[0054] Orientation can be defined by the XYZ coordinate system.

[0055] Figure 4A and Figure 4B An example is shown of generating a feedback trigger signal 10 based on a scan sequence timing table 40, including a time slot 45 for emission. Figure 4A In the scan sequence time table 40, time slots (45A, 45B, 45C) for emitting probe light and time slots (45D, 45D') for emitting ultraviolet light are included, and a feedback trigger signal 10 is generated each time a fluorescence signal is captured based on the emitted ultraviolet light (45D, 45D'). Figure 4B In this process, a feedback trigger signal 10 is generated each time a fluorescence signal is captured based on emitted ultraviolet light (45D, 45D') and / or each time reflected infrared light is captured based on emitted infrared light (45E, 45E'). Figure 4AIn this configuration, the real-time mode is determined by a first frame rate of emitted probe light (45A, 45B, 45C), and at least one image sensor unit 5 is configured to capture fluorescence signals from the tooth object at a second frame rate based on emitted UV light (45D, 45D'), wherein the first frame rate is greater than the second frame rate, and the feedback trigger signal 10 is determined at the second frame rate. Figure 4B In the real-time mode, the first frame rate of the emitted probe light (45A, 45B, 45C) is determined, and at least one image sensor unit 5 is configured to capture fluorescence signals from the dental object at a second frame rate based on the emitted UV light (45D, 45D'), and / or, at least one image sensor unit 5 is configured to capture reflected infrared light at a third frame rate based on the emitted infrared light (45E, 45E'), the first frame rate being greater than both the second and third frame rates, and the feedback trigger signal 10 being determined at either the second or third frame rate.

[0056] Figures 5A to 5C An example of displaying feedback signal 52 on 3D model 50 is shown. Figure 5A In the middle, the quality level of the feedback signal 52 is displayed as a colored overlay on the 3D model 50; in Figure 5B In this context, the quality level is displayed as a transparent area in the 3D model at the location determined by the feedback signal; and... Figure 5C In this model, quality levels (52A, 52B) are depicted as color gradients. For example, at position 52B, the color of the quality level is different from that at position 52A, and this color is determined based on the quality level.

[0057] Figure 6A and Figure 6B An example is shown where the quality threshold level can be varied between certain parts of a 3D model 50. Figure 6AAs can be seen, the 3D model 50 includes a marked area 62, which includes areas in the 3D model where the user has planned to apply treatment or preparation to the corresponding tooth object of the patient. In this example, if the quality level 52 is below a quality threshold level, the 3D model is completely transparent or has a specific color at the corresponding location of the determined quality level, as determined by feedback signals. As the quality level 52 is above the quality threshold level and continues to improve, the 3D model becomes less / more transparent or changes color at the corresponding location of the determined quality level. The quality threshold level varies between different parts of the 3D model. For example, no treatment or preparation may be performed at the first part 63 of the 3D model 50, while treatment or preparation may be performed at the second part 62 of the 3D model 50. In most cases, more precise data will be needed in the 3D model 50 where at least some treatment or preparation will be performed. For example, treatment may be for caries, cracks, or tissue diseases such as inflammation, periodontitis, and gingivitis. Preparation may be for restorations. The quality level and location in the feedback signal are determined based on the treatment type or preparation type, and the treatment type or preparation type can be one or more of the following: caries treatment, crack treatment, tissue disease, and restoration preparation.

[0058] Thanks to the teachings presented in the foregoing description and associated drawings, numerous modifications and other embodiments of the inventions set forth herein will come to the mind of those skilled in the art to which these inventions pertain. Therefore, it should be understood that these inventions are not limited to the specific embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the appended claims. Furthermore, although exemplary embodiments have been described in the context of certain exemplary combinations of elements and / or functions in the foregoing description and associated drawings, it should be recognized that alternative embodiments may provide different combinations of elements and / or functions without departing from the scope of the appended claims. In this regard, for example, combinations of elements and / or functions other than those explicitly described above are also contemplated as may be set forth in some of the appended claims. Although specific terminology is used herein, it is used only in a general and descriptive sense and not for purposes of limitation.

[0059] project

[0060] 1. An intraoral scanning system configured to display feedback signals on a three-dimensional (3D) model of a dental object, wherein the system includes:

[0061] At least one projection unit is configured to emit probe light in a variety of configurations having an illumination pattern.

[0062] At least one image sensor unit, housing an array of sensor elements configured to provide a two-dimensional (2D) image with three-dimensional (3D) data based on the reflection of probe light captured from the dental object.

[0063] One or more processors are configured as follows:

[0064] The 3D model of the tooth object is determined by stitching together 2D images.

[0065] The 3D model is updated by stitching a 2D image provided by at least one image sensor unit onto the 3D model.

[0066] Determine the quality level of the data at its location within the 3D model, and

[0067] Determine the quality level of the data in the 3D model and the feedback signal corresponding to the location of that data in the 3D model.

[0068] When one or more processors fail to determine a feedback trigger signal while updating the 3D model,

[0069] The display unit is configured to display a 3D model in real time, and the display unit is also configured to display the quality level of data at the location in the 3D model determined by the feedback signal when a feedback trigger signal is determined.

[0070] 2. The intraoral scanning system according to Project 1, wherein not updating the feedback trigger signal includes not stitching the 2D image provided by at least one image sensor unit onto the 3D model.

[0071] 3. The intraoral scanning system according to any one of the preceding items, comprising a handheld intraoral scanning device, the handheld intraoral scanning device comprising:

[0072] The at least one projection unit

[0073] The at least one image sensor unit

[0074] A gyroscope configured to determine the orientation of a handheld intraoral scanning device and / or an accelerometer configured to determine the motion of the handheld intraoral scanning device, and

[0075] The one or more processors are configured to determine a feedback trigger signal by comparing the orientation and / or movement of the handheld intraoral scanning device with the scanning orientation range and / or scanning motion pattern of the handheld intraoral scanning device.

[0076] 4. The intraoral scanning system according to Project 3 includes a memory unit that includes a scanning orientation range of a handheld intraoral scanning device and determines a feedback trigger signal when the orientation of the handheld intraoral scanning device is outside the scanning orientation range.

[0077] 5. The intraoral scanning system according to any one of items 3 and 4, wherein a feedback trigger signal is determined when the motion of the handheld intraoral scanning device is not similar to a reference motion pattern, or when no motion is measured, and the reference motion pattern is stored in a memory unit of the system, and the reference motion pattern corresponds to a motion pattern in the scanning state of the tooth object.

[0078] 5. The intraoral scanning system according to any one of the preceding items, comprising a user interface, wherein a feedback trigger signal is determined when the user interface receives input from a user of the intraoral scanning system.

[0079] 6. The intraoral scanning system according to any one of the preceding items, wherein the feedback signal includes the quality level of three-dimensional geometric data, color data and / or diagnostic data of the tooth object.

[0080] 7. The intraoral scanning system according to any one of the preceding items, wherein the projection unit includes a plurality of light sources, wherein the plurality of light sources include a first light source configured to emit visible light, a second light source configured to emit infrared light, and / or a third light source configured to emit UV light, and wherein the visible light includes probe light.

[0081] 8. The intraoral scanning system according to Item 7, wherein the real-time mode is determined by a first frame rate, and wherein at least one image sensor unit is configured to capture fluorescence signals from a tooth object based on emitted UV light at a second frame rate, and / or, at least one image sensor unit is configured to capture reflected infrared light at a third frame rate, the first frame rate being greater than the second and third frame rates, and wherein a feedback trigger signal is determined by the second or third frame rate.

[0082] 9. The intraoral scanning system according to Project 8, wherein the second frame rate and the third frame rate do not correspond to real time.

[0083] 10. An intraoral scanning system according to any one of items 1 to 7, wherein the real-time mode is determined by a first frame rate, and wherein the one or more processors are configured to determine a feedback trigger signal at a frame rate of at least 1 / 2 of the first frame rate.

[0084] 11. The intraoral scanning system according to any one of the preceding items, wherein the quality level is determined by one or more of the following:

[0085] The amount of data at a location in the 3D model.

[0086] Compared to the amount of data obtained from different angles in a 3D model,

[0087] The quality of the data at the location in the 3D model, and

[0088] The quality of data obtained from different angles relative to the position in the 3D model.

[0089] 12. The intraoral scanning system according to any one of the preceding items, wherein the quality level is displayed in one or more colors, or in one or more transparency levels in a 3D model.

[0090] 13. The intraoral scanning system according to any one of the preceding items, wherein the quality level is below a quality threshold level, and the 3D model is completely transparent or has a specific color at the corresponding location of the quality level determined by the feedback signal.

[0091] 14. The intraoral scanning system according to any one of the preceding items, wherein the quality level and location in the feedback signal are determined based on the treatment type or preparation type, and wherein the treatment type or preparation type is one of the following:

[0092] Caries treatment

[0093] Crack treatment

[0094] Periodontal treatment, and

[0095] Preparation of the prosthesis.

[0096] 15. The intraoral scanning system according to Item 6, wherein the diagnostic data includes the difference between infrared data and color data captured by the image sensor unit.

[0097] 16. The intraoral scanning system according to any one of the preceding items, wherein the one or more processors are configured to provide audio feedback via a speaker of the system, and wherein the timing of playing the audio feedback is synchronized with the determination of the feedback trigger signal.

Claims

1. An intraoral scanning system configured to display feedback signals on a three-dimensional (3D) model of a dental object, wherein, The system includes: At least one projection unit is configured to emit probe light in a variety of configurations having an illumination pattern. At least one image sensor unit, housing an array of sensor elements configured to provide a two-dimensional (2D) image with three-dimensional (3D) data based on the reflection of probe light captured from the dental object; One or more processors are configured as follows: The 3D model of the tooth object is determined by stitching together the 2D images. The 3D model is updated by stitching the 2D images provided by the at least one image sensor unit onto the 3D model. Determine the quality level of the data in the 3D model at its location within the 3D model, wherein the quality level can be determined based on one or more attributes of the 2D image. Determine the quality level of the data in the 3D model and the feedback signal of the corresponding position of the data in the 3D model. When one or more processors fail to determine a feedback trigger signal while updating the 3D model; and The display unit is configured to display the 3D model in real time, and the display unit is also configured to display the quality level of the data at the location on the 3D model determined by the feedback signal when the feedback trigger signal is determined.

2. The intraoral scanning system according to claim 1, wherein, The non-updating of the feedback trigger signal includes not stitching the 2D image onto the 3D model.

3. The intraoral scanning system according to any one of the preceding claims, comprising a handheld intraoral scanning device, the handheld intraoral scanning device comprising: The at least one projection unit The at least one image sensor unit A gyroscope, configured to determine the orientation of the handheld intraoral scanning device, and / or an accelerometer, configured to determine the motion of the handheld intraoral scanning device, and The one or more processors are configured to determine the feedback trigger signal by comparing the orientation and / or movement of the handheld intraoral scanning device with the scanning orientation range and / or scanning movement pattern of the handheld intraoral scanning device.

4. The intraoral scanning system of claim 3, comprising a memory unit including the scanning orientation range of the handheld intraoral scanning device, and determining the feedback trigger signal when the orientation of the handheld intraoral scanning device is outside the scanning orientation range.

5. The intraoral scanning system according to any one of claims 3 and 4, wherein, When the motion of the handheld intraoral scanning device is not similar to a reference motion pattern, or when no motion is measured, the feedback trigger signal is determined, and the reference motion pattern is stored in the memory unit of the system, and the reference motion pattern corresponds to the motion pattern in the scanning state of the tooth object.

6. The intraoral scanning system according to any one of the preceding claims, wherein, The feedback signal includes the quality level of the three-dimensional geometric data, color data, and / or diagnostic data of the dental object.

7. The intraoral scanning system according to any one of the preceding claims, wherein, The projection unit includes multiple light sources, including a first light source configured to emit visible light and a second light source configured to emit infrared light, wherein the visible light includes the detection light.

8. The intraoral scanning system according to claim 7, wherein, The real-time mode is determined by a first frame rate, and wherein the at least one image sensor unit is configured to capture fluorescence signals from the tooth object based on emitted UV light at a second frame rate, and / or, the at least one image sensor unit is configured to capture reflected infrared light based on the emitted infrared light at a third frame rate, wherein the first frame rate is greater than both the second frame rate and the third frame rate, and wherein the feedback trigger signal is determined by the second frame rate or the third frame rate.

9. The intraoral scanning system according to claim 8, wherein, The second frame rate and the third frame rate do not correspond to real-time.

10. The intraoral scanning system according to any one of claims 1 to 7, wherein, The real-time mode is determined by a first frame rate, and wherein the one or more processors are configured to determine the feedback trigger signal at a frame rate that is at least half of the first frame rate.

11. The intraoral scanning system according to any one of the preceding claims, wherein, The quality level is determined by one or more of the following properties of the two-dimensional image: The amount of data at the location in the 3D model. Compared to the amount of data obtained from different angles in the 3D model, The quality of the data at the location in the 3D model, and The quality of data obtained from different angles relative to the position in the 3D model.

12. The intraoral scanning system according to any one of the preceding claims, wherein, The quality level is displayed in one or more colors, or in one or more transparency levels in the 3D model.

13. The intraoral scanning system according to any one of the preceding claims, wherein, The quality level is below a quality threshold level, and the 3D model is either completely transparent or has a specific color at the corresponding location of the quality level determined by the feedback signal.

14. The intraoral scanning system according to any one of the preceding claims, wherein, The quality level and the location in the feedback signal are determined based on the treatment type or preparation type, and wherein the treatment type or the preparation type is one of the following: Caries treatment Crack treatment Tissue diseases, and Preparation of the prosthesis.

15. The intraoral scanning system according to claim 6, wherein, The diagnostic data includes the difference between infrared data and color data captured by the image sensor unit.

Images